Keep log file size fixed without logrotate

Question

Is there any way to keep file size of log file fixed without rotating it by a new empty file and deleting (or archiving) the old file. For example, if I set log file maximum size to 1MB, after the file size increase beyond that limit it will be automatically clamped, the text is added on 'tail' and the oldest part of text is poped out to keep the file size 1MB.

Answer 1

You could write a little bash script to do this. Just tail the file to a certain byte count using tail -c and overwrite the file.

from man tail:

-c, --bytes=N
              output the last N bytes; alternatively, use +N to  output  bytes
              starting with the Nth of each file

   If  the  first  character of N (the number of bytes or lines) is a `+',
   print beginning with the Nth item from the start of each  file,  other‐
   wise, print the last N items in the file.  N may have a multiplier suf‐
   fix:  b  512,  kB  1000,  K  1024,  MB  1000*1000,  M   1024*1024,   GB
   1000*1000*1000, G 1024*1024*1024, and so on for T, P, E, Z, Y.

Answer 2

I'm certain the original poster has found a solution after 8 years. Here's another one for others that may read this thread...

curtail limits the size of a program's output and preserves the last 200MB of output with the following command:

run_program | curtail -s 200M myprogram.log

https://github.com/Comcast/Infinite-File-Curtailer

Answer 3

 *      prune - clips off the tops of a list of files described in the
 *              file /etc/default/prune. Designed to be run periodically
 *              from cron, the idea is to automatically shorten log
 *              files that grow forever. This file contains a list of
 *              files (full pathname), and the number of blocks that
 *              should be retained.  To retain some sanity, prune will
 *              clip the file after the next newline. The file
 *              /etc/default/prune should look something like:
 *
 *                      /usr/adm/aculog         10
 *                      /usr/adm/leo.log        5
 *                      /usr/adm/messages       200
 *                      /usr/adm/sup_log        5
 *
 *              The crontab entry on infoswx for prune looks like:
 *
 *                      0 5 * * * /etc/prune >/usr/adm/prune.log 2>&1
 *
 *              Compile with:  cc -O -o prune prune.c
 *
 *              The following defines may be adjusted to suit your taste
 *              and your system block size.
 *
 *      Ray Davis  infoswx!bees  09/25/85

https://groups.google.com/group/mod.sources/browse_thread/thread/b2136b3ab7e924e/f785b37c218bb576?hl=en&ie=UTF-8&q=prune+log+file&pli=1

Many (most? all?) daemons will re-open their log files if sent a HUP signal (e.g. by same cron job that runs prune)

Answer 4

Your only solution may be to write your own userspace file system or contribute to an existing one. Look at the partial list at Filesystem in Userspace

If you don't have the skills to contribute, offer a project publicity or $$$ or both, to add it for you.

I wish I had the time to do it, I have always wanted something exactly like this.

Answer 5

You can do something similar using a FIFO, which is sort of like a zero-byte sized file.

However, note that if nothing is READING from this file, then the syslog process may become blocked, and will stop writing to ALL of your logfiles. I'm not sure if this behavior has been changed with newer versions of Ubuntu/CentOS.

One example here

For another example, try something like this.

Make your FIFO:

sudo mkfifo /var/log/everything.fifo

And add this to (r)syslog.conf, then restart syslog:

*.*     |/var/log/everything.fifo

Then view the FIFO from one window:

cat /var/log/everything.fifo

And in another window, send some stuff to syslog:

logger Test1
logger Test2
logger Test3

You should see the "Test*" lines in the output of cat above.

This feature can be great for debugging, especially if you don't care to keep the data around for longer. For example, if you only want to see everything except for thefirewall spam, you can do something like this:

grep -vi "kernel: .* on wan" /var/log/everything.fifo

Answer 6

Here's my second answer. This is a pretty hackish.

Use watch(1) to repeatedly execute tail --bytes=1024 (the last 1024 bytes of the logfile, thanks to @jjclarkson for that answer).

watch --no-title tail --bytes=1024 /var/log/messages >/tmp/messages.watch

And then view the file with:

less --raw-control-chars /tmp/messages.watch

The difference between watch and a while loop is that watch will only update /tmp/messages.watch if there were changes to /var/log/messages .

while true; do
    tail --bytes=1024 /var/log/messages > /tmp/messages.watch
    sleep 1
done

And well, I guess you could put a test in the while loop so that tail is only executed if /var/log/messages was updated, but I won't figure that out now.