5

I am aware that STDOUT is usually buffered by commands like mawk (but not gawk), grep, sed, and so on, unless used with the appropriate options (i.e. mawk --Winteractive, or grep --line-buffered, or sed --unbuffered). But the buffering doesn't happen when STDOUT is a terminal/tty, in which case it is line buffered.

Now, what I don't get is why STDOUT is buffered outside of a loop send to a pipe, even though the final destination is the terminal.

A basic example :

$ while sleep 3; do echo -n "Current Time is ";date +%T; done | mawk '{print $NF}'
^C

Nothing happens for a long time, because mawk seems to be buffering it's output.

I wasn't expecting that. mawk's output is the terminal, so why is its STDOUT buffered ?

Indeed, with the -Winteractive option the output is rendering every 3 seconds :

$ while sleep 3; do echo -n "Current Time is ";date +%T; done | mawk -Winteractive '{print $NF}'
10:57:05
10:57:08
10:57:11
^C

Now, this behavior is clearly mawk related, because it isn't reproduced if I use for example grep. Even without its --line-buffered option, grep doesn't buffer its STDOUT, which is the expected behavior given that grep's STDOUT is the terminal :

$ while sleep 3; do echo -n "Current Time is ";date +%T; done | grep Current
Current Time is 11:01:44
Current Time is 11:01:47
Current Time is 11:01:50
^C
ChennyStar
  • 1,319
  • 9
  • 22
  • `awk` (= gawk) will show time at "3": `apt install gawk` and do `$ while sleep 3; do echo -n "Current Time is ";date +%T; done | awk '{print $NF}'` .... Info https://superuser.com/questions/75875/awk-mawk-nawk-gawk-what – Knud Larsen Jun 25 '21 at 12:10
  • I know that, `gawk` never buffers (I said it in my first paragraph). But that's beside the point, I'm trying to understand why `mawk` is buffering even when `STDOUT` is tty (terminal). Or, in other words, why my `| mawk` example behaves differently from my `| grep` example in terms of buffering. – ChennyStar Jun 25 '21 at 12:13

1 Answers1

12

It's not that it's buffering its output.

mawk is the only utility that I know that buffers its input.

See also https://github.com/ThomasDickey/original-mawk/issues/41#issuecomment-241070898

In other words, mawk will not start processing its input (let alone print anything if that processing involves printing) until it has accumulated a buffer full of input.

You can verify it by running:

(echo 1; sleep 1; echo 2) | mawk '{system("echo "$1)}'

It can be disabled with the -Winteractive option. Note that with -Winteractive, records are lines regardless of the value of RS.

Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501
  • Thanks ! I scratched my head on that one... And yes, the answer is in the manpages : `-W interactive sets unbuffered writes to stdout and line buffered reads from stdin.` – ChennyStar Jun 25 '21 at 13:06
  • Additional info : `stdbuf -i0` doesn't seem to be working with `mawk` (not that it's important, given that mawk has the `-Winteractive` option). I discovered it after digging for some more infos on `STDIN` buffering (and incidentally finding another of your answers, https://unix.stackexchange.com/questions/411263/stdin-unbuffered#answer-411330 - it seems your are the stdio guru on this forum ;-) – ChennyStar Jun 25 '21 at 14:53