2

I am trying to understand shell expansions in bash (GNU bash, version 4.4.20(1)-release (i686-pc-linux-gnu)).

Typing in my interactive bash shell

x='$(id)'
$x
$(echo $x)

I was expecting from any of last 2 lines an error of the form

bash: uid=xxx(user): command not found

but got

bash: $(id): command not found.

I don't undestand why command substitution does not occur here. Shouldn't it be realised after variable expansion ? My guess is that it has to do with Shell operations as described here https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Operation

Can someone explains this behavior ?

I am just interested in understanding more precisely bash expansions. I am not interested in running actual script in my question.

ilkkachu
  • 133,243
  • 15
  • 236
  • 397
InfiniteLooper
  • 175
  • 1
  • 10
  • How often did you get the error message? – AdminBee Oct 21 '21 at 13:44
  • any time I type the command – InfiniteLooper Oct 21 '21 at 13:51
  • 1
    I believe the left-to-right expansion at that level expands `$x` and does not reconsider the `$(id)`, but would like to back it up with references to sources before answering. Notice the careful semicolon & comma punctuation at https://www.gnu.org/savannah-checkouts/gnu/bash/manual/html_node/Shell-Expansions.html#Shell-Expansions where it says "The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and filename expansion." – Jeff Schaller Oct 21 '21 at 14:10
  • 1
    If you are expecting the shell, when given the command `$x`, to expand it to `$(id)` and _expand it again_ to the output of `id`: the shell never re-parses the result of an expansion in search for further expansion-triggering characters. More on this [here](https://unix.stackexchange.com/a/599927/315749) - an answer of mine to a different question (I am unable to find a better reference at the moment). – fra-san Oct 21 '21 at 14:14
  • @JeffSchaller : Indeed, reading those pages, i didn't noticed such subtility. I am not familiar with those differences. You mean a colon in this context separates things that are done at the same time and not semicolons ? – InfiniteLooper Oct 21 '21 at 14:20
  • 2
    @InfiniteLooper, yes, it means exactly that. It could perhaps be written a bit more clearly, but it's supposed to parse like this: "(1) brace expansion; then (2) tilde expansion; then (3) parameter, variable and arithmetic expansions and command substitution, left-to-right; then (4) word-splitting; then (5) filename expansion". – ilkkachu Oct 21 '21 at 14:21
  • 1
    _Not_ having the results of e.g. variable expansions considered again for other expansions is important for correctness, it allows e.g. storing an arbitrary filename in a variable without having issues immediately when it's expanded if the filename e.g. contains a `$`. Consider that a filename like `A $2.00 thingy` might otherwise trigger expansion of the second positional parameter (or command line argument) via the `$2` there, probably referring to a different filename. And files called `$(rm -rf $HOME)` would be far worse... – ilkkachu Oct 21 '21 at 14:23

2 Answers2

5

$(…) is a command substitution (“process substitution” is <(…) and the like). Variable substitutions and command substitutions occur in the same pass, from left to right in the string. The only things that occur on the result of these substitutions are word splitting and globbing.

So x='$(id)' sets x to the 5-character string $(id). Then, to run $x, the shell replaces $x by the value $(id). This does not contain any whitespace or globbing character so it is treated as a command name.

Contrast with:

x='@(id)'
shopt -s extglob
echo /none/$x /usr/bin/$x

Assuming that the file /none/id doesn't exist but /usr/bin/id does, the echo command expands to three words: echo (obviously), /none/@(id) (the glob pattern /none/@(id) doesn't match anything so it's left unchanged), and /usr/bin/id (the glob pattern /usr/bin/@(id) matches one file, so it's replaced by the one-element list of matches).

In the bash manual, the relevant sentence is at the beginning of the Shell Expansions section.

The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and filename expansion.

Everything between two semicolons is one pass. Each pass works on the result of the previous pass.

Beware that a single sentence (even a complex one like the one I cited above) can't tell the whole story. Shell semantics is convoluted. I doubt that any shell's manual has the details of all the corner cases. The POSIX specification is more formal but doesn't cover bash-specific extensions and even it leaves some really odd cases undefined.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
-1

It's the quoting. Single quotes (') define a literal string and no interpolation or escaping can occur. Double quotes (") allows for interpolation as well as escaping.

Here's an example:

$ x='$(id)'
$ echo 'The variable x contains the value \"$x\"'
The variable x contains the value \"$x\"
$ echo "The variable x contains the value \"$x\""
The variable x contains the value "$(id)"

$ x="$(id)"
$ echo "The variable x contains the value \"$x\""
The variable x contains the value "uid=1000(myusername)..."

Here's another example of interpolation and escaping based on single quote vs double quote:

$ echo 'The current directory is $PWD according to the variable \"\$PWD\".'
The current directory is $PWD according to the variable \"\$PWD\".

$ echo "The current directory is $PWD according to the variable \"\$PWD\"."
The current directory is /tmp according to the variable "$PWD".
jetole
  • 119
  • 2