4

If I have:

node foo.js | node bar.js

is there a way to determine if there is a queue in between them that's building up? in other words if the producer is outpacing the consumer w.r.t. stdio?

Alexander Mills
  • 9,330
  • 19
  • 95
  • 180
  • 1
    If it were Linux, one way I can think of is to run `strace` on the two processes and see if either were blocking on `read`/`write` calls. – muru Apr 27 '18 at 05:35
  • Similar (duplicate?): [Print current number of bytes in a pipe](//unix.stackexchange.com/q/330374) – Stéphane Chazelas Apr 27 '18 at 08:48
  • You could also have the producer use `poll()`/`select()` to check if the pipe is full. Or do non-blocking I/O and take action upon EAGAIN (meaning the pipe is full) – Stéphane Chazelas Apr 27 '18 at 08:50
  • 2
    Possible duplicate of [Print current number of bytes in a pipe](https://unix.stackexchange.com/questions/330374/print-current-number-of-bytes-in-a-pipe) – meuh Apr 27 '18 at 08:53
  • 1
    Your usage of the term _stdio_ indicates a possible confusion. `stdio` is the C API than deals with `FILE` objects which implement their own buffering on top of file descriptors. For instance if your producer uses that API and does a `puts("foo")`, with default buffering settings, that `foo` will *not* be written to the pipe, the data is only written upon `fflush()` or when the stdio buffer is full. – Stéphane Chazelas Apr 27 '18 at 09:05
  • Can't happen. The kernel manages buffer space for the pipe; the reader is started when there is something to read, the writer stopped when it is full. – vonbrand Apr 27 '18 at 13:41

1 Answers1

2

You can use ioctl(,FIONREAD,) on pipes in Linux to get the current number of bytes available to read. This might be found in man 7 pipe (FIONREAD is also in man tty_ioctl). You can either integrate this into your application or run something like this bit of perl in parallel with it, using the pipe as stdin:

#!/usr/bin/perl
# https://unix.stackexchange.com/a/440361/119298
use strict;
require 'sys/ioctl.ph';
my $v = pack("l",0);
$| = 1;
# while(1)
{
  die "ioctl: $!" if ioctl(STDIN, &FIONREAD, $v)!=0;
  my @r = unpack("l",$v);
  printf "%d ",$r[0];
#  sleep 1;
}

So the usage would be

node foo.js | (myperl & node bar.js)

Try it out with something like

while sleep 1;do date;done |
while sleep 1.1;do myperl 2>/dev/null; dd status=none count=1 bs=20;done

I get warnings from perl about _FORTIFY_SOURCE, hence the stderr redirection. The dd reads less than the length of date so you can see the result gradually increasing.

For completeness, you can get the capacity of a pipe with fcntl(,F_GETPIPE_SZ) and set it with fcntl(,F_SETPIPE_SZ,newvalue) up to /proc/sys/fs/pipe-max-size (unless root or have CAP_SYS_RESOURCE). See man fcntl.

meuh
  • 49,672
  • 2
  • 52
  • 114