2

Does there exist a sufficiently pathological filename or collection of filenames with the constraint that it must have a given prefix and suffix, say file_<your_stuff_here>.txt which would make it dangerous to eval or backtick the output of ls in a bash script? Where we say "dangerous" means that it allows anyone with file creation permissions to the relevant directory the ability to run any command.

For example, let the bash script be just

#!/usr/bin/env bash
hello=(`ls -t`)

It seems like the filesystem sanitizes special characters, spaces for instance, by wrapping the filenames in quotes. Will it catch all such things?

AdminBee
  • 21,637
  • 21
  • 47
  • 71
lanf
  • 145
  • 5
  • 4
    The filesystem does not sanitize anything. **Recent** versions of **GNU** ls quote file names with some problematic characters, but I don't know if this is completely compatible with shell quoting. – Gilles 'SO- stop being evil' Feb 03 '21 at 14:35
  • 1
    What do you get with `touch 'file_$(date).txt'; eval $(ls)` in an otherwise empty directory? Your script doesn't `eval` it though, it splits and globs the output of `ls` and puts it in an array. – muru Feb 03 '21 at 14:42
  • 1
    @Gilles'SO-stopbeingevil', depends. With `--quoting-style=shell` (or `shell-escape`) it seems to get stuff right. That looks to be the default on my Ubuntu 20.04. But plain `ls -Q` just adds double-quotes, without regard for e.g. `$` signs inside. – ilkkachu Feb 03 '21 at 16:00
  • Why doesn't the script eval it? Isn't that what backticks do? Please excuse my ignorance here – lanf Feb 03 '21 at 16:41
  • 1
    @bxw, no, backticks don't eval. what backticks, or the saner `$(...)` syntax, does is called command substitution, it's an expansion the same as variable expansion. Expansions happen at a certain point in the command line processing, and in particular, special operators and quotes that come up as the result of expansions, aren't processed as special (that step has already been done at that point). What `eval` does, is to for another round of _all_ the processing steps, including operators and quotes. – ilkkachu Feb 03 '21 at 17:39
  • 1
    So, `foo='"hi there"'; echo $foo` outputs `"hi there"` (with quotes, they're not special as the result of the expansion), but `eval echo $foo` outputs `hi there` (without quotes, since the eval removed them). Similarly, `foo='echo hi; echo hey'; $foo` prints `hi; echo hey` (the semicolon isn't special), but `eval $foo` would print `hi`, `hey` on two lines (because now the eval had the semicolon treated as a command terminator, and processed both commands. And so on. – ilkkachu Feb 03 '21 at 17:41
  • 1
    Which means that for the most part, doing just a command substitution doesn't really open command injections. Using `eval` with the results, or otherwise shoving the output as part of some shell command, can do that. You get other problems from using `$(ls)`, though. – ilkkachu Feb 03 '21 at 17:43
  • You say "for the most partt", what are the exceptions? What "other problems" would I run into from using substitution like this? – lanf Feb 03 '21 at 18:09
  • 1
    @bxw, I don't think I have any in mind, but I'm not saying "doesn't ever" since I can't preclude the possibility that someone will find a way to do it. For the problems with `$(ls)` in general, see https://unix.stackexchange.com/a/632465/170373 and the links therein. – ilkkachu Feb 03 '21 at 19:34
  • 1
    And in any case GNU ls only does this default quoting when its output is a terminal, not when its output is piped or captured by a command substitution. – Gilles 'SO- stop being evil' Feb 05 '21 at 15:09

2 Answers2

5

filename ... dangerous to eval

Sure.

$ touch 'file $(date >&2).txt'
$ bash -c 'eval ls *'
Wed 03 Feb 2021 06:07:08 PM EET
ls: cannot access 'file': No such file or directory
ls: cannot access '.txt': No such file or directory

Don't eval stuff.

or backtick the output of ls in a bash script?

I'm not exactly sure what this means.

If you mean what you had as an example,

hello=(`ls -t`)

or with a saner syntax,

files=( $(ls -t) )

then you just the get the output of ls wordsplit to the array.

$ declare -p files
declare -a files=([0]="file" [1]="\$(date" [2]=">&2).txt")

The biggest problem here, even before the possible command injection, is that the space in the filename broke it, we got two array entries instead of one. See the page about parsing ls on Greg's wiki. No, you can't work around it by adding quotes, word splitting doesn't work like that.

So, don't use ls there. Just let the shell generate the list of filenames:

files=(*)
declare -p files
declare -a files=([0]="file \$(date >&2).txt")

The only problem here, is that Bash doesn't give a good way of sorting files by their date, so ls -t is tempting. The good alternatives are to put the date in the filename itself so the default sorting gives you sorting by date, or to use Zsh, since it can sort by date. Or do ugly hacks to work around the issue (caveat, I haven't tested that solution).

The same problem with eval comes if you need to pass the filename to anything that takes just a shell script, like

ssh somehost "do something with $file"      # wrong
ssh somehost "do something with '$file'"    # still wrong, the name
                                            # could contain single ticks

It seems like the filesystem sanitizes special characters, spaces for instance, by wrapping the filenames in quotes. Will it catch all such things?

Oh dear, oh no it doesn't. If the filesystem did something to prevent storing special characters to the shell, half of the posts on unix.SE wouldn't be needed.

If you want the pain that comes from too much knowledge, here's an essay by David Wheeler about that: Fixing Unix/Linux/POSIX Filenames: Control Characters (such as Newline), Leading Dashes, and Other Problems

There's also the other one by him, Filenames and Pathnames in Shell: How to do it Correctly. Lots of the subject matter of that has also been discussed here on unix.SE.

Also, quotes don't even help.

$ touch '"quoted name"' 'othername'   # two files
$ printf "%s\n" $(ls)                 # oops
othername
"quoted
name"
$ printf "%s\n" *                     # this works better
othername
"quoted name"

Because when word splitting happens, quotes are just a regular character. (Unless you set IFS to include quotes, which probably just makes it worse.) Besides, even if the name is wrapped in quotes, it could still have quotes in the middle, breaking that. You'd need to take care the escape or quote those correctly, too.

It's GNU ls that does the quoting, depending on version and settings:

$ ls -l
total 0
-rw-rw-r-- 1 ilkkachu ilkkachu 0 Feb  3 18:30  othername
-rw-rw-r-- 1 ilkkachu ilkkachu 0 Feb  3 18:30 '"quoted name"'

That's the same as ls --quoting-style=shell. Incidentally, it seems to get it right, for all of newlines, dollar signs, and quotes. But do you trust it to get it right? If you do, and you know how to use it correctly, you may be able to use it to get the sorted listing.

ilkkachu
  • 133,243
  • 15
  • 236
  • 397
1
hello=(`ls -t`)

That form seems safe. Bash will perform just split + glob on the results of the command substitution, and not interpret it as syntax:

a=(`echo '[0]=1'`)
typeset -p a

declare -a a=([0]="[0]=1")  # aha!

However, the (obscure) declare "var=val" or local "var=val" forms are not, as they work similarly to eval "var=val":

cd "$(mktemp -d)"
touch '$(yes BOOBS >&2&)'
declare -a "a=($(ls))"

BOOBS
BOOBS
BOOBS
...

Same with declare -a a="($(ls))", declare -a a="(`ls`)", etc.

Or even with the non-quoted forms too, provided that the variable was already declared as an array:

cd "$(mktemp -d)"
a=(1 2 3)
touch '($(yes BOOBS >&2&))'
declare a=$(ls)  # I forgot that a was an array

BOOBS
BOOBS
BOOBS
...