51

I read that it is bad to write things like for line in $(command), the correct way seem to be instead:

command | while IFS= read -r line; do echo $line; done

This works great. But what if what I want to iterate on is the contents of a variable, not the direct result of a command?

For example, imagine that you create the following file quickfox:

The quick brown
foxjumps\ over -
the
lazy ,
dog.

I would like to be able to do something like this:

# This is just for the example,
# I could of course stream the contents to `read`
variable=$(cat quickfox);
while IFS= read -r line < $variable; do echo $line; done; # this is incorrect
Jonathan H
  • 2,303
  • 3
  • 20
  • 27

2 Answers2

68

In modern shells like bash and zsh, you have a very useful `<<<' redirector that accepts a string as an input. So you would do 

while IFS= read -r line ; do echo $line; done <<< "$variable"

Otherwise, you can always do

echo "$variable" | while IFS= read -r line ; do echo $line; done
Jeff Schaller
  • 66,199
  • 35
  • 114
  • 250
lgeorget
  • 13,656
  • 2
  • 41
  • 63
  • Sorry, I should have thought about echoing the contents of course. But thanks for the quick answer anyway! – Jonathan H Apr 11 '16 at 19:17
  • 2
    you need to double-quote `$variable` when you use it, otherwise the `while` loop will get only one line of input. See, for example, the difference in output between `echo $variable` vs `echo "$variable"` or `cat <<< $variable` vs `cat <<< "$variable"`. – cas Apr 11 '16 at 22:54
  • @cas Actually it depends on what is inside $variable. In the case presented by the OP ("variable=$(cat quickfox)") it works without the additional quotes. But for the general case, you're right. I edit my answer. Thanks. – lgeorget Apr 12 '16 at 06:55
  • The `variable=$(cat quickfox)` in the OP's question itself provides an example of what I was talking about. Using that `$variable` inside double-quotes includes the newlines, using it without has the newlines translated to spaces by the shell. If you're reading and processing line-by-line, this makes a huge difference - with the former you have multiple input lines, with the latter you have just one input line. The input data is superficially similar but, in practice, completely different in those two cases. – cas Apr 12 '16 at 08:57
  • for example: with that input data, `cat <<< "$variable" | wc -l` returns 5. `cat <<< $variable | wc -l` returns 1. If you want/need to preserve whitespace (including newlines, tabs, single or multiple spaces) in a variable then you MUST double-quote the variable when you use it, otherwise they'll all be transformed into a single space between each "word". – cas Apr 12 '16 at 08:59
  • @cas My shell (zsh 5.2 (x86_64-pc-linux-gnu)) outputs 5 in both cases. – lgeorget Apr 14 '16 at 21:05
  • The OP uses `bash` (as do I and the majority of linux and osx users), and the question is tagged with `/bash`, not `/zsh`. – cas Apr 14 '16 at 23:05
4

Example how to iterate over env variables with multi-line values: 

for var in $(compgen -v | grep -Ev '^(BASH)'); do
    echo "$var=${!var}"
done

Explained:

  1. $(compgen -v | grep -Ev '^(BASH)') - grep a list of env keys, skip those that started with BASH
  2. echo "$var=${!var}" - print key=value pairs
FelikZ
  • 223
  • 2
  • 7