20

when tailing multiple files at once as shown below, is any there any way to show the file name at the start of each line?

tail -f one.log two.log

current output

==> one.log <==
contents of one.log here...
contents of one.log here...

==> two.log <==
contents of one.log here...
contents of two.log here..

Looking for something like

one.log: contents of one.log here...
one.log: contents of one.log here...
two.log: contents of two.log here...
two.log: contents of two.log here...
mtk
  • 26,802
  • 35
  • 91
  • 130

6 Answers6

13

Short answer

GNU Parallel has a set of nice options which make it really easy to do such things:

parallel --tagstring "{}:" --line-buffer tail -f {} ::: one.log two.log

The output would be:

one.log:	contents of one.log here...
one.log:	contents of one.log here...
two.log:	contents of two.log here...
two.log:	contents of two.log here...

More explanation

  • The option --tagstring=str tags each output line with string str. From parallel man page:
--tagstring str
                Tag lines with a string. Each output line will be prepended with
                str and TAB (\t). str can contain replacement strings such as {}.

                --tagstring is ignored when using -u, --onall, and --nonall.

  • All occurrences of {} will be replace by parallel's arguments which, in this case, are log file names; i.e. one.log and two.log (all arguments after :::).

  • The option --line-buffer is required because the output of a command (e.g. tail -f one.log or tail -f two.log) would be printed if that command is finished. Since tail -f will wait for file growth, it is required to print the output on line basis which --line-buffer does so. Again from parallel man page:

--line-buffer (alpha testing)
                Buffer output on line basis. --group will keep the output
                together for a whole job. --ungroup allows output to mixup with
                half a line coming from one job and half a line coming from
                another job. --line-buffer fits between these two: GNU parallel
                will print a full line, but will allow for mixing lines of
                different jobs.
cartoonist
  • 1,356
  • 10
  • 11
9
tail  -f ...your-files | 
    awk '/^==> / {a=substr($0, 5, length-8); next}
                 {print a":"$0}'

\thanks{don_cristti}

JJoao
  • 11,887
  • 1
  • 22
  • 44
  • @don_crissti, thank you! (1) a vs file -- no more wine to my glass! (2) good idea. I start to do something like this but I lasily said to my self "no one will make tail -f of files with spaces" :) -- I will use your excellent suggestions. – JJoao Apr 14 '15 at 07:17
  • can you explain length-8? why it is 8 here, I know only 8 works. – Shicheng Guo May 24 '17 at 23:52
  • 1
    tail -n 1 *.txt | awk '/^==>/ {a=substr($0, 5, length-8); next} {print a,$1}' | awk '$2>0 {if ($2 !~/chrY/) print $1}' | xargs -I {} qsub {} – Shicheng Guo May 25 '17 at 00:04
  • @ShichengGuo, 8 = length("==> ")+length(" <==") – JJoao May 25 '17 at 07:27
  • @ShichengGuo, Are planning a massive cluster job submission? -- good :) ! – JJoao May 25 '17 at 07:28
  • 1
    surprisingly/amazingly your joojoo magic `awk` solution works! – Trevor Boyd Smith Sep 13 '19 at 15:28
  • Explanation of the magic numbers: `tail -f` includes filenames output in the format: `==> filename <==\n`. So, in the `substr` function, the `5` removes the `==> ` and the `length-8` removes the ` <==\n`. – rinogo Sep 25 '20 at 17:01
  • 1
    With a slight modification, this command pairs nicely with `logtop` to show the number of lines written per second to each log file: `tail -f * | awk '/^==> / {a=substr($0, 5, length-8); next} {print a}' | logtop` – rinogo Sep 25 '20 at 17:05
3

If tail is not mandatory, you can use grep to achieve this:

grep "" *.log

This will print the filename as the prefix of each output line.

Output breaks if *.log expands to only one file. In this regard:

grep '' /dev/null *.log
serenesat
  • 1,286
  • 1
  • 13
  • 29
  • 1
    I need to show file name in output of `tail -f` not `grep`. – mtk Apr 13 '15 at 10:15
  • @serenesat this would just print the entire contents of the files wouldn't it? The OP was asking for the filename to be printed when tail is specified – rahul Apr 13 '15 at 10:18
  • you can also just do `--with-filename` or `-H` to always force filename prepend. – Trevor Boyd Smith Sep 13 '19 at 15:21
  • i really love this answer! does exactly EXACTLY what is needed with a solution that is like 13 characters long. as opposed to tricky `awk` solution or `parallel` which isn't installed. – Trevor Boyd Smith Sep 13 '19 at 15:22
  • only problem is that if your logfiles are 1 million lines long. then your grep will spam 1 million lines to console (or over ssh)... – Trevor Boyd Smith Sep 13 '19 at 15:24
  • 1
    `tail -f` keeps printing the contents of a file as they are added. Your solution does not really answer the question because `grep` will exit as soon as it's done, i.e. it will only print the _current_ contents of a file. – A.P. Jun 20 '21 at 07:09
  • Rather than the `/dev/null` kludge, use `grep -H`. – Jim L. Dec 02 '22 at 18:20
2

Use multitail

multitail -N 1 --mergeall --label "one.log: " one.log --label "two.log: " two.log
JamesThomasMoon
  • 826
  • 9
  • 20
  • Thanks @Stéphane Chazelas but your suggested code snippet causes the shell instance to exit and adds global variable `file`. Please remove the `exec` and make `file` local. – JamesThomasMoon Dec 03 '22 at 04:28
  • No. The body of the function was a subshell which is how you get local variables in standard sh – Stéphane Chazelas Dec 03 '22 at 17:54
0

My idea would be to create a single file with merged logs from several files, like someone suggested here and prepend filenames:

$ tail -f /var/log/syslog | sed -u -E 's,(^.+$),/var/log/syslog: \1,g' >> /tmp/LOG &&
$ tail -f /var/log/Xorg.0.log | sed -u -E 's,(^.+$),/var/log/Xorg.0.log: \1,g' >> /tmp/LOG &&
$ tail -f /tmp/LOG
Community
  • 1
Arkadiusz Drabczyk
  • 25,049
  • 5
  • 53
  • 68
0

Something with xargs and sed could work:

$ xargs -I% -P0 sh -c "tail -f % | sed s/^/%:/g" <<EOT
one.log
two.log
EOT
FloHimself
  • 11,272
  • 3
  • 22
  • 24