Putting process substituted file descriptor into variable

Question

I have long command roughly this:

$ command-outer "long command which references a file here: filename1, and another file reference here: filename2"

The files are the output of another command. So I am doing:

$ command-outer "long ... "<(command-inner "again long params")"...  "\
  <(command-inner "again long params")" ..."

For readability I'd like to extract the inner unnamed/anonymous pipes (the ones with <()) from the long command invocation. However I can't seem to do so; doing the following:

RESULT_FILE_DESCRIPTOR1=<(command-inner ....)

leads to the file descriptor already being closed when I actually use RESULT_FILE_DESCRIPTOR1 in the params list of command-outer

And invocating command-outer on the same line, thus:

    RESULT_FILE_DESCRIPTOR1=<(command-inner ....) outer-command "long ... $RESULT_FILE_DESCRIPTOR1 ... "

gives back an empty result RESULT_FILE_DESCRIPTOR1, which is not a big surprise since a:

FOO=BAR echo $FOO

Also doesn't work.

They aren't anonymous; they just have names you don't normally see or care about. The `<(…)` construct is called "process substitution". Please update your question (title and body) to use the correct terminology. — Jonathan Leffler, Sep 28 '15 at 13:44
I know, but they are sometimes called "anonymous named" pipes https://en.wikipedia.org/wiki/Process_substitution#Anonymous_named_pipe if this is stlll incorrect should we be a good citizen and update wikipedia as well? /update seems wikipedia is the only one using that term, I'll update — hbogert, Sep 28 '15 at 19:36
@hbogert - no, you were totally right. they're anonymous pipes. *usually*, anyway. named pipes are sometimes used by some shells when the underlying os doesnt support the `/dev/fd/[num]` links. the `mkfifo` command can get you a named pipe, but most unix `/dev/` filesystems will provide links to pipe descriptors as well. do `echo <(:)` and see for yourself. — mikeserv, Sep 28 '15 at 23:11

mikeserv · Accepted Answer · 2015-09-28T12:02:58.117

Rather than naming the results you can just name the command.

param_set_1(){
    input_command \
        -lots     \
        -of       \
        -params
}
param_set_2(){
    input_command \
        -lots     \
        -of       \
        -other    \
        -params
}

command_outer -params <(param_set_1) <(param_set_2)

You can also often refer to file-descriptors by name.

param_set_1 | { param_set_2 | command_outer -params /dev/fd/3 -; } 3<&0

If you really want the results in a current-shell variable and in a file-descriptor, then you should steer clear of pipes. You run the risk of filling the pipe buffer with the output of your command while concurrently trying to assign those results to a parent-shell variable before the pipe-buffer can be drained by the reading process and... well, it's not pretty. So, you should take it all in one go, and use here-documents.

unset fd3 fd4
{   command_outer -params /dev/fd/[34]
}   3<<FD3 4<<FD4
${fd3=$(param_set_1)}
FD3
${fd4=$(param_set_2)}
FD4

You understood the problem better than I did myself. – hbogert Sep 28 '15 at 14:05 — hbogert, Sep 28 '15 at 14:05

score 3 · Answer 2 · answered Sep 28 '15 at 10:06

You can explicitly use named pipes (which, I think, process substitution does use in the details):

trap cleanup EXIT
cleanup() { rm -f "$inner1" "$inner2"; }

inner1=$(mktemp -u) && mkfifo "$inner1"
command-inner "again long params" > "$inner1" &

inner2=$(mktemp -u) && mkfifo "$inner2"
command-inner "again long params" > "$inner2" &

command-outer "long ... $inner1...  $inner2 ..."

Putting process substituted file descriptor into variable

2 Answers2

Linked