Why {1,2} printed by a command in $() is not interpolated?

Question

I am in a directory in which I have two text files:

$ touch test1.txt
$ touch test2.txt

When I try to list the files (with Bash) using some pattern it works:

$ ls test?.txt
test1.txt  test2.txt
$ ls test{1,2}.txt
test1.txt  test2.txt

However, when a pattern is produced by a command enclosed in $(), only one of patterns work:

$ ls $(echo 'test?.txt')
test1.txt  test2.txt
$ ls $(echo 'test{1,2}.txt')
ls: cannot access test{1,2}.txt: No such file or directory

What's going on here? Why the pattern {1,2} does not work?

Answer 1

It's a combination of two things. First, brace expansion is not a pattern that matches file names: it's a purely textual substitution — see What is the difference between `a[bc]d` (brackets) and `a{b,c}d` (braces)? . Second, when you use the result of a command substitution outside double quotes (ls $(…)), what happens is only pattern matching (and word splitting: the “split+glob” operator), not a complete re-parsing.

With ls $(echo 'test?.txt'), the command echo 'test?.txt' outputs the string test?.txt (with a final newline). The command substitution results in the string test?.txt (without a final newline, because command substitution strips trailing newlines). This unquoted substitution undergoes word splitting, yielding a list consisting of the single string test?.txt since there are no whitespace characters (more precisely, no characters in $IFS) in it. Each element of this one-element list then undergoes conditional wildcard expansion, and since there is a wildcard character ? in the string the wildcard expansion does happen. Since the pattern test?.txt matches at least one file name, the list element test?.txt is replace by the list of file names that match the patterns, yielding the two-element list containing test1.txt and test2.txt. Finally ls is called with two arguments test1 and test2.

With ls $(echo 'test{1,2}'), the command echo 'test{1,2}' outputs the string test{1,2} (with a final newline). The command substitution results in the string test{1,2}. This unquoted substitution undergoes word splitting, yielding a list consisting of the single string test{1,2}. Each element of this one-element list then undergoes conditional wildcard expansion, which does nothing (the element is left as is) since there is no wildcard character in the string. Thus ls is called with the single argument test{1,2}.

For comparison, here's what happens with ls $(echo test{1,2}). The command echo test{1,2} outputs the string test1 test2 (with a final newline). The command substitution results in the string test1 test2 (without a final newline). This unquoted substitution undergoes word splitting, yielding two strings test1 and test2. Then, since neither of the strings contains a wildcard character, they're left alone, so ls is called with two arguments test1 and test2.

Answer 2

The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and filename expansion.

Brace expansion won't happen after command substitution. You can use eval to force another round of expansion:

eval echo $(echo '{1,2}lala')

It's result is:

1lala 2lala

Answer 3

That problem is very specific to bash, and it's because they decided in bash to separate the brace expansion from filename expansion (globbing), and to perform it first, before all the other expansions.

From the bash manpage:

The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and pathname expansion.

In your example, bash will only see your braces after it had performed the command substitution (the $(echo ...)), when it's just too late.

This is different from all the other shells, which perform the brace expansion just before (and some even as a part of) pathname expansion (globbing). That includes but is not limited to csh where brace-expansions were first invented.

$ csh -c 'ls `echo "test{1,2}.txt"`'
test1.txt test2.txt
$ ksh -c 'ls $(echo "test{1,2}.txt")'
test1.txt  test2.txt

$ var=nope var1=one var2=two bash -c 'echo $var{1,2}'
one two
$ var=nope var1=one var2=two csh -c 'echo $var{1,2}'
nope1 nope2

The latter example is the same in csh, zsh, ksh93, mksh or fish.

Also, notice that brace expansion as part of globbing is also available via the glob(3) library function (at least on Linux and all the BSDs), and in other independent implementations (eg. in perl: perl -le 'print join " ", <test{1,2}.txt>').

Why that was done differently in bash has probably a story behind it, but FWIW I wasn't able to find any logical explanation, and I find all the post-hoc rationalizations unconvincing.

Answer 4

It works if you remove the quotes

$ ls $(echo test{1,2})
test1  test2

Answer 5

Please try:::

ls $(echo test{1,2}\.txt)

With a BackSlash. It works now. Also remove the like the earlier poster said, the quotes. The Dot is not for matching pattern, but to be taken literally as Period here.