1

I want to process all changed logfiles from apache and made a simple oneliner:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }'

This is exactly my start. But if I pipe that output further, I get no output:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }' | wc -l

Anyway, if I use 'ls', 'tail' (what I really want), there is never output (in another console, running the first statement, I know, there are changes). I tried with adding stdbuf, fflush(), nothing works. I dont understand the problem! I want to keep this onliner into a script, which I can use as a string point. Otherwise, I could (but dont wish to) use xargs. Any help or tips would be fine!

mabra
  • 161
  • 1
  • 7
  • So it still doesn't work with the correct quoting in place? – jesse_b Jul 18 '18 at 22:14
  • 1
    turn off buffering (or at least make it line-based) for *all* the things. https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe – thrig Jul 18 '18 at 22:25
  • Thanks for all hints! It just needs to much time to solve this problem - I am working for hours on that. After applying all recommendations to disable buffering (some of them I tried also bevore) and none worked, I am going back to use xargs. It is even an unsupported debian (squeeze - a new hardware is on my table sinde two years ...). Thanks anyone. – mabra Jul 18 '18 at 23:26
  • 1
    I answered the question, but it's really not clear what your desired result is. The `awk` command here does nothing at all. `wc` and `tail` don't print anything until all data is read. `ls` doesn't read stdin at all. – jordanm Jul 19 '18 at 01:35
  • you may have a different problem to me, however I found I can get non-buffered output from `awk` with `fflush()` -- As: `gawk -e '{ prnt ; fflush(); }'`. It must have the empty parentheses"()". Alternatively try ` ... | mawk` as: `mawk -Winteractive -e '{ print $0; }' | ...` – will Sep 12 '19 at 06:18

1 Answers1

2

The reason you do not get output is because wc -l is consuming all of the output. You can use tee in combination with process substitution to get awk to output to the console as well as read by wc:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }' | tee >(wc -l)

This is still slightly problematic as inotifywait is a command that never exits, but wc -l does not print output until it has read all the data, in this case when the pipe closes. You can do your counting directly in awk:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk 'BEGIN { total=0 } { print; total++; print total } END { print "final total: " total }'
jordanm
  • 41,988
  • 9
  • 116
  • 113