Did the shellshock bash fix break basic command line tools like echo and cat?

Question

I used to be able to do things like:

X=123 cat <<EOF
X is $X
EOF

or even simpler:

X=123 echo $X

The first one still seems to work on Mac OS X after install the bash fix, however neither seem to work on my Ubuntu 14.04 instance in AWS anymore. What makes it that echo or cat no longer have access to these environment variables? Stranger still, when I pass the env vars to a NodeJS app I don't seem to have any issues:

cat <<EOF > test.js
console.log('X is ' + process.env.X);
EOF
X=123 node test.js

This seems to work in bash scripts as well:

cat <<EOF > test.sh
echo X is \$X
EOF
chmod +x test.sh
X=123 ./test.sh

`X=123 echo $X` should never have worked. Have you looked at the `test.sh` file that you created? Does it say `echo X is $X` or does it say `echo X is `? — G-Man Says 'Reinstate Monica', Oct 07 '14 at 22:42
You're absolutely right, the shell was stripping out $X since it was evaluating it. I've updated the question to reflect that (basically it works in Mac OS X, not in Ubuntu). — Abdullah Jibaly, Oct 07 '14 at 23:05

vinc17 · Accepted Answer · 2014-10-08T07:23:02.440

8

In any POSIX shell, when you write

X=123 echo $X

the $X is expanded before the whole command is executed, i.e. if $X is initially unset, you get:

X=123 echo

which is then executed. You can see more or less what the shell is doing with set -x:

$ set -x
$ X=123 echo X=$X
+ X=123
+ echo X=
X=
$ set +x

You can see that echo (actually the shell itself, which does the expansion before executing echo) still has access to the environment:

$ X=123 eval 'echo $X'
123

The issue with cat <<EOF is similar. Note that concerning bash, there was a bug in old versions (before 4.1), described in the CHANGES file as:

Fixed a bug that caused variable expansion in here documents to look in any temporary environment.

This may be the cause of the behavior observed under Mac OS X. Do not rely on this bug.

edited Oct 08 '14 at 07:23

answered Oct 07 '14 at 23:17

vinc17

11,912
38
45

Thanks for the awesome explanation! I'm still stumped by why the `cat < – Abdullah Jibaly Oct 08 '14 at 00:49
Probably worth adding to that code a "set +x" to stop debugging. – rooby Oct 08 '14 at 02:20
@AbdullahJibaly The behavior under Mac OS X is probably due to a bug in old bash versions (now fixed). See my edited answer. – vinc17 Oct 08 '14 at 07:26

score 6 · Answer 2 · answered Oct 08 '14 at 06:13

Your questions regarding the here-documents are probably unrelated. The thing is, bash will perform the expansions and the assignment concurrently - so a leading X=123... should not affect the value expanded from within the here-document. This is because the here-document is an input file-descriptor that bash must build and pass to cat upon invocation in the same way it must assign 123 to $X before passing that plus the rest of its environment off to cat at execve time.

Consider:

X=321; X=123 bash <<HEREDOC
echo "$X is not yet \$X and $$ is not yet \$$."
HEREDOC

OUTPUT

321 is not yet 123 and 17134 is not yet 17225.

Did the shellshock bash fix break basic command line tools like echo and cat?

2 Answers2

OUTPUT

Linked