Preferred syntax for two lines long pipe

Question

When writing a long pipe it is usually clearer to separate it in two lines.

This long command line:

ruby -run -e httpd -- -p 5000 . 2>&1 | tee >(grep -Fq 'WEBrick::HTTPServer#start' && open localhost:5000) 

Could be divided as:

ruby -run -e httpd -- -p 5000 . 2>&1 \
   | tee >(grep -Fq 'WEBrick::HTTPServer#start' && open localhost:5000) 

Or:

ruby -run -e httpd -- -p 5000 . 2>&1 |
    tee >(grep -Fq 'WEBrick::HTTPServer#start' && open localhost:5000) 

In short:

command1 \
    | command2

Or:

command1 |
    command2

I realize that this might be a style (opinion) issue, but: Is there a preferred way, and, if so, why?

Answer 1

Ask your self what would this do?

command1 \ 
   | command2

Can't see the difference. Neither can I, but the shell can. Look closely, there is a space after the \. This stops the newline from being escaped.

Therefore use the other form, as it is safer. Shown here with the same error (a space after the | in this case). But it does not cause a bug.

command1 | 
    command2

Answer 2

I'm going to disagree with most folks here; I always prefer to wrap before a joining operator such as a pipe:

command1 \
| command 2

(You don't need to indent the second line; the pipe itself links it very obviously to the first.)

There are a few reasons for this:

• It's easier to see the joiner; it doesn't get lost amongst the details of the line.  (This is especially important if the line is long, and the joiner might have got scrolled out of sight, or lost amongst line wrapping.)  When you scan code quickly, you look down the left-hand side, because that's where the overall structure is: in the indentation, the braces, or whatever a particular languages uses.  Pipes and other joiners are important to the structure, so they too should be on the left.

• It lines up if you're spanning 3 or more lines.  Again, this makes the structure of the pipeline easy to take in at a glance.

• It's closer to the way we think.  (This is the most subtle and contentious point…)  If you're reading a list out slowly, so someone can write it down, you'd say “[Item 1]… (pause)… and [Item 2]… (pause)… and [Item 3].”; it would feel unnatural to say “[Item 1] and… (pause)… [Item 2] and… (pause)… [Item 3].”  That's because we think of the joiner as attaching to the following item more than the previous one.  (You can think of the minus sign in arithmetic in similar terms; it works like addition, but connects more closely to the following number by negating it.)  Code is easier to follow when it reflects our thinking.

I've tried both ways in many languages over the years, and have found that putting joiners on the following line really does help in most cases.

Answer 3

Well, just to avoid it looking like nobody would prefer:

command1 \
   | command2

I'm going to say that I do.

I see the trailing space problem raised by ctrl-alt-delor as a non-issue. Editors can warn about it; git warns about it. To top it off, the shell would raise a syntax error on | command2, providing the user with the file and line number of the error and cease interpreting the rest of the file:

$ cat f.sh
#!/bin/bash

echo foo \ 
| command2

echo bar
$ ./f.sh
foo  
./f.sh: line 4: syntax error near unexpected token `|'
./f.sh: line 4: `| command2'

There's also the fact that there are more uses for line-continuation escapes. For example, to break simple commands that have many arguments:

ffmpeg \
  -f x11grab \
  -video_size "$size" \
  -framerate "${framerate:-10}" \
  -i "${DISPLAY}${offset}" \
  -c:v ffvhuff \
  -f matroska \
  -

Should we avoid such usage too because we can't trust ourselves not to put a space after the escape?

My preference is purely a matter of readability and quite subjective. Here's a real-life example from my shell history (with details substituted with foobar):

org-table-to-csv foobar.org \
| cq +H -q "
  select foo
    from t
    where bar = 'baz'
      and foo != ''" \
| sed -r 's/^|$/'\''/g' \
| sed -r ':b;$!{N;bb};s/\n/, /g'

Compare to:

org-table-to-csv foobar.org |
  cq +H -q "
    select foo
      from t
      where bar = 'baz'
        and foo != ''" |
  sed -r 's/^|$/'\''/g' |
  sed -r ':b;$!{N;bb};s/\n/, /g'

Here's another:

sed 's/ .*//' <<< "$blame_out"
| sort \
| uniq \
| tee >(sed "s/^/from pipe before grep filtering: /" > /dev/tty) \
| grep -vF "$(git show -s --format=%h "$from_commit")" \
| tee >(sed "s/^/from pipe before git show: /" > /dev/tty) \
| xargs git show -s --format='%cI %h' \
| tee >(sed "s/^/from pipe after git show: /" > /dev/tty) \
| sort -k1 \
| tail -1 \
| cut -d' ' -f2

Compare to:

sed 's/ .*//' <<< "$blame_out"
  sort |
  uniq |
  tee >(sed "s/^/from pipe before grep filtering: /" > /dev/tty) |
  grep -vF "$(git show -s --format=%h "$from_commit")" |
  tee >(sed "s/^/from pipe before git show: /" > /dev/tty) |
  xargs git show -s --format='%cI %h' |
  tee >(sed "s/^/from pipe after git show: /" > /dev/tty) |
  sort -k1 |
  tail -1 |
  cut -d' ' -f2

Answer 4

I thought the answer to this was easy, but I can see @JoL and @gidds disagree with me.

My brain prefers reading a line and not having to scan the next line \
:

  foo bar baz ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... \

In the above I will have to see \
, what is on line 2 \
, before I can tell \
, what the command does \
. Maybe the command is complete \
? Or maybe the command continues \
  on the next line \
?

To me it is much easier to read,
if \ is only used,
when a command cannot fit on a line.

Reading through my code, I also see comments as an issue:

foo ... ... ... ... ... ... ... ... |
    # Now this does bar
    bar ... ... ... ... ... ... ... ... ||
    # And if that fails: fubar
    fubar

I am not sure how you would at all do comments in the middle of a pipeline if you use \ + newline before | or || or &&. If that is not possible, I think this is the most important problem. Code is not maintainable without comments, and comments should normally be as close to the code as possible to encourage updating the documentation when you change the code.

Emacs does the indentation for me automatically, so the indentation is not even an extra burden:

# This is indented automatically in emacs
ruby -run -e httpd -- -p 5000 . 2>&1 |
    # Send the output to the screen and to grep
    tee >(grep -Fq 'WEBrick::HTTPServer#start' &&
              # If grep matches, open localhost:5000
              open localhost:5000) 
# Here is where emacs indents the next command to