0

How might a node in a Bash pipeline only peek at, but not consume, its input stream?

For example, how might I modify the following script so that it outputs "print" instead of nothing? In particular, how might I modify or replace grep --quiet print so that it only peeks at, but does not consume, its input?

printf "%s\n" a b print c |
  if grep --quiet print
  then
    grep print | cat
  fi

Output:

Desired output:

print

Ideally, I seek a program peek that is similar to grep, but that doesn't consume its input. peek print would return error code 0 if it finds "print" in the input and if not, returns a non-zero error code.

Here's how peek would work in my example:

printf "%s\n" a b print c |
  if peek print
  then
    grep print | cat
  fi

Output:

print

The reason I want to do this instead of simply filtering the results with grep print is to avoid processing empty search results that might cause an error.

Derek Mahar
  • 505
  • 3
  • 13
  • 1
    In the case where your Q is "how can I ran a command (eg. `cvsq`) only if its input (eg. the result of `grep`) is [not empty](https://unix.stackexchange.com/questions/33049/), then you probably don't have to save the entire search result in a variable or tempfile, a single byte should suffice: `PRODUCER | grep PATTERN | { t=$(dd bs=1 count=1 2>/dev/null; printf x); t=${t%x}; [ "$t" ] && { printf '%s' "$t"; cat; } | CONSUMER; }` –  Feb 03 '20 at 18:40
  • That's an interesting solution. Can you break it down for me? – Derek Mahar Feb 03 '20 at 18:56
  • I think I see how your script works. It reads a single byte from the input stream and then pipes this byte and the rest of the input stream to the consumer only if the original input has at least one byte. – Derek Mahar Feb 03 '20 at 19:01
  • It's basically the same thing as the accepted answer from the linked Q, only simplified a bit (it won't handle NUL bytes in the input). In fact, you really can _peek_ at a pipe in linux with the `tee(2)` system call, but there's no generally available shell utility that can use it, so the only thing left are such hacks. –  Feb 03 '20 at 19:03
  • @mosvy, if you were to promote your [comment](https://unix.stackexchange.com/questions/565581/how-can-a-node-in-a-bash-pipeline-peek-at-but-not-consume-its-input-stream/565591?noredirect=1#comment1051757_565581) to an answer, I would accept it as the answer. – Derek Mahar Feb 05 '20 at 22:05
  • I don't think my solution is different enough from the accepted answer from the linked Q –  Feb 06 '20 at 10:58

4 Answers4

2

A straight forward approach I can think of is to use a temporary storage for output. It can be done with minimal modification of your code like this:

atmp=$(mktemp "/tmp/XXXXXX")

printf "%s\n" a b print c | tee $atmp | 
  if grep --quiet print
  then
    grep print $atmp 
  fi  
rm $atmp #cleanup

For small volumes of output that may be acceptable.

Tagwint
  • 2,410
  • 1
  • 13
  • 22
  • I'm not able to grasp the purpose of this. Why not just `print ... > "$atmp"; grep print "$atmp"` ? ;-) –  Feb 03 '20 at 17:35
  • 1
    No difference if all you need is to get the grepped line printed. The question was phrased more generally about reusing the output. – Tagwint Feb 03 '20 at 17:45
  • @mosvy I want to avoid processing empty search results. In my specific case, the search results may include CSV rows, including a header, which the script subsequently passes to `csvq` to filter the CSV rows. In the event that the search finds no CSV rows and no header, `csvq` reports an error which I want to avoid. – Derek Mahar Feb 03 '20 at 17:47
  • @DerekMahar you should give a better, more complete example –  Feb 03 '20 at 17:53
  • @mosvy I think my example captures the essence of what I want to achieve, though stating more clearly why I wanted to do this might have been beneficial. – Derek Mahar Feb 03 '20 at 17:55
  • @mosvy I made changes to my question that I hope clarify my goal. – Derek Mahar Feb 03 '20 at 18:51
  • I accepted this answer because it addressed the question more than the specific example. – Derek Mahar Feb 05 '20 at 21:52
  • Two of three users rejected my [suggested edit](https://unix.stackexchange.com/review/suggested-edits/325356), so I will post it here instead: `atmp=$(mktemp "/tmp/XXXXXX"); printf "%s\n" a b print c | if grep print > $atmp; then cat $atmp; fi; rm $atmp #cleanup;`. This stores only the search results of `grep print` instead of all of the original input. – Derek Mahar Feb 05 '20 at 22:04
0

Another approach similar to @Tagwint's answer is to store the result of grep print in a shell variable and then print the contents of that variable only if there are search results:

printf "%s\n" a print b print c |
 if result=$(grep print)
 then
   echo "$result";
 fi

Output:

print
print

This would be acceptable where the search results are not too large.

Derek Mahar
  • 505
  • 3
  • 13
  • how's that different from `printf '%s\n' .. | grep print`? –  Feb 03 '20 at 17:37
  • See @Tagwint's comment at https://unix.stackexchange.com/questions/565581/how-can-a-node-in-a-bash-pipeline-peek-at-but-not-consume-its-input-stream/565591?noredirect=1#comment1051730_565585 and my comment at https://unix.stackexchange.com/questions/565581/how-can-a-node-in-a-bash-pipeline-peek-at-but-not-consume-its-input-stream/565591?noredirect=1#comment1051732_565585. – Derek Mahar Feb 03 '20 at 17:50
0

While not a standard Unix utility or Bash script, I decided to solve the problem in this specific example using ifne from moreutils:

printf "%s\n" a b print c | grep print | ifne echo found

Output:

found

printf "%s\n" a b c | grep print | ifne echo found

Output:

@Nick Wirth referred to lfne in his answer to How to check if a pipe is empty and run a command on the data if it isn't?.

Thank you to @mosvy for referring me to this existing question.

Derek Mahar
  • 505
  • 3
  • 13
  • I can't also accept this answer, but it gets honorable mention because it solves the problem that the example in the question raises using handy tool `ifle` from package moreutils. – Derek Mahar Feb 05 '20 at 22:00
0

I also solved the problem in this specific example using fionread which @mosvy implemented in his answer to How to check if a pipe is empty and run a command on the data if it isn't?.

printf "%s\n" a b print c | grep print | ./fionread echo found

Output:

found

printf "%s\n" a b c | grep print | ./fionread echo found

Output:

Note that gcc easily compiles fionread.c on FreeBSD 11.3:

$ gcc -o fionread fionread.c
$ echo test | grep test | ./fionread echo found
found
$ echo | grep test | ./fionread echo found
$
Derek Mahar
  • 505
  • 3
  • 13
  • I can't also accept this answer, but it gets honorable mention because it solves the problem that the example in the question raised and because `fionread` is a useful tool and smaller than `ifle` from package moreutils. – Derek Mahar Feb 05 '20 at 21:59