5

I need to select certain lines from the log file and also save them to a text file. I tried the following but none of them are working as expected. The file "todel.txt" shows 0 bytes.

tail -f general.log | grep Some_word > >(tee -a todel.txt)

tail -f general.log | grep Some_word ; tee  todel.txt

tail -f general.log | grep Some_word | tee -a todel.txt
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
shantanuo
  • 143
  • 1
  • 8
  • Perhaps "Some_word" is case sensitive? In which case you should use _grep -i_. – sandyp Nov 18 '13 at 21:25
  • Are you sure, the required lines really appear there? By default `tail -f` will give you last 10 lines of input and whatever is written there unless the file is removed and recreated, in which case it quits. GNU `tail` has the `--retry` option which will wait until the file is recreated (although I think this option is non-POSIX). – peterph Nov 25 '13 at 23:15

4 Answers4

8

You need to add stdbuf(1) into your pipeline:

tail -f general.log | stdbuf -oL grep Some_word | tee -a todel.txt

This will set grep's stdout stream buffering mode to be line-buffered, otherwise grep waits to get at least 4096 bytes from the stream (this is the default on Linux for buffered i/o).

Alternatively, you can also call grep with --line-buffered:

tail -f general.log | grep --line-buffered Some_word | tee -a todel.txt

See Turn off buffering in pipe and http://www.pixelbeat.org/programming/stdio_buffering/ for in-detail explanations.

Community
  • 1
tkrennwa
  • 3,445
  • 1
  • 14
  • 17
1

The third one should be working fine assuming that you have access to write files in the current directory.

For example: tail -f general.log | grep "Some_word" | tee todel.txt

That is properly-defined tee syntax. If it's not working, you're doing something else wrong.

Also, it's a good idea to put your search string in quotes. If you have a filename matching your search string in the current working directory, it may be considered a file argument instead of a search string.

Magellan
  • 576
  • 3
  • 10
  • I think you mean tee instead of tail in your command line example. – dmourati Nov 18 '13 at 06:23
  • Sure enough. Thanks. What I get for multi-tasking. – Magellan Nov 18 '13 at 06:42
  • Using `tail -f ... |grep` allows one to grep the new lines incoming (`-f` option to `tail`). AFAIK `grep` has no such option. – zhenech Nov 18 '13 at 07:34
  • You're right. Wasn't paying attention. – Magellan Nov 18 '13 at 16:33
  • 1
    Consider a text file with only a few characters in it. If you run `tail -f...` and pipe it into the `grep | tee` machinery, you would end up in exactly the scenario of the question: `grep` would wait for more input before processing it, while `tee` does not receive anything at all. Of course, if `general.log` is sufficiently large, the standard approach without altering the output buffer mode to line-buffered would work, but only if you it larger than the buffer size. And then you would need to wait for filling up `grep`'s buffer again. See http://unix.stackexchange.com/a/102505/50954 – tkrennwa Nov 25 '13 at 12:50
  • Good stuff. Thanks. had no idea that the default buffer was that large. – Magellan Nov 25 '13 at 16:51
0
grep "my words" > mytextfile.txt
tail -f access.log > last_log.txt
tail -f access.log | grep "IP address" > hacker.log
Ilja
  • 130
  • 1
  • 6
0

It's possible the lines you're looking for are being sent to stderr instead of stdout, in which case ending the line with 2>&1 should fix the problem:

tail -f general.log | grep Some_word | tee -a todel.txt 2>&1
Matthew Cline
  • 3,265
  • 4
  • 24
  • 38