Multiple pv order

Question

I have a script on a Linux machine with a fancy pv piped to a second pv that count a subset of the outputted lines.

Here's the script:

max=1000
for (( i=0; i<max; i++ )); do
  [[ $(shuf -i 1-100 -n 1) -lt 20 ]] && echo REMOVE || echo LEAVE
done | pv -F "%N %b / $(numfmt --to=si $max) %t %p %e" -c -N 'Lookups' -l -s $max \
  | grep --line-buffered '^REMOVE' \
  | pv -F "%N %b / $(numfmt --to=si $max)" -c -N 'Deletes' -l -s $max \
  >/dev/null
stty sane

what I would expect is that the first pv always shows first, and the second always second.

Like this example output:

$ ./fancy_pv.sh
  Lookups: 1.00k / 1.0K 0:00:03 [===============================================================================================================================================================================================================================================================================================================================================================>] 100%            
  Deletes:  189  / 1.0K

But that's not the case, sometimes they swap positions and I see something like this:

$ ./fancy_pv.sh
  Deletes:  199  / 1.0K
  Lookups: 1.00k / 1.0K 0:00:03 [===============================================================================================================================================================================================================================================================================================================================================================>] 100%            

And sometimes I also see something like this:

$ ./fancy_pv.sh
  Lookups:  321  / 1.0K 0:00:01 [===============================================================================================================>                                                                                                                                                                                                                                                 ] 32% ETA 0:00:02
  Deletes:  198  / 1.0K
  Lookups: 1.00k / 1.0K 0:00:03 [===============================================================================================================================================================================================================================================================================================================================================================>] 100%            

I know it must be because of the way pv deletes the line and redraws it, but is there anything I can do to prevent it from messing with the order?

stty sane is there to sanitize the prompt because sometimes pv leaves the terminal unusable.

Thanks

Answer 1

The two pv processes in the pipe may start in any order. The output from the latest pv will be in the bottom line.

Delay pv you want in the bottom line. Instead of pv … (where … denotes all its arguments) use a subshell:

( </dev/null sleep 1; exec pv … )

In theory the other pv may still start after the delayed one, but in a not-totally-overloaded system it's almost certain the delayed pv will start last.

sleep should not read from its stdin anyway; </dev/null is in case your sleep is weird.

I'm not sure if some race condition can cause an extra (stale) line to appear. If so, delaying one pv should also (almost certainly) help. In my tests the output gets mangled when the terminal needs to be "additionally" updated. Therefore:

• Do not resize the terminal while pvs are running.
• Avoid scrolling:
  • Before you run the script, invoke clear (or hit Ctrl+L). This will clear the screen, place the prompt on top and provide room below without the need of scrolling later.
  • Do not type while pvs are running; especially multiple Enters (that could eventually scroll the text) should be avoided.
  • In general do not let anything else than pvs print to the terminal until pvs finish. This applies to other parts of the pipe (e.g. via /dev/tty), asynchronous processes in the script (e.g. simply via their stdout), processes outside of the script (e.g. via /dev/tty* or /dev/pts/*).