Why are quotes necessary for files argument when calling this Bash script?

Question

I'm quite new to Bash scripting. I have a "testscript", which I used as the basis for a more advanced / useful script:

#!/bin/bash
files=$1
for a in $files
do
    echo "$a"
done

When I call this without any quotes it just picks up one file in a directory:

testscript *.txt

But when I call it with quotes it works correctly and picks out all the text files:

testscript '*.txt'

What is going on here?

To be very, **very** clear, the right way to fix this is to run `for a in "$@"; do` (or `for a; do`) in your script, thus leaving the globbing to the outer shell, not to leave out the quotes. — Charles Duffy, Aug 26 '16 at 23:10
Also see: [Why does my shell script choke on whitespace or other special characters?](http://unix.stackexchange.com/q/131766/135943) — Wildcard, Aug 27 '16 at 08:06
See also [When is double-quoting necessary?](https://unix.stackexchange.com/q/68694/80216) — G-Man Says 'Reinstate Monica', Aug 28 '22 at 03:26

score 29 · Accepted Answer · answered Aug 26 '16 at 14:10

29

When you call a program

testscript *.txt

then your shell does the expansion and works out all the values. So it might, effectively call your program as

testscript file1.txt file2.txt file3.txt file4.txt

Now your program only looks at $1 and so only works on file1.txt.

By quoting on the command line you are passing the literal string *.txt to the script, and that is what is stored in $1. Your for loop then expands it.

Normally you would use "$@" and not $1 in scripts like this.

This is a "gotcha" for people coming from CMD scripting, where the command shell doesn't do globbing (as it's known) and always passes the literal string.

answered Aug 26 '16 at 14:10

Stephen Harris

42,369
5
94
123

6

Just to clarify (for people other than the author of the above answer), using `"$@"` (as opposed to `$@` or `$1` `$2` `$3`) will cause each filename to be quoted `"file1.txt"` `"file2.txt"` etc. For `file1.txt` this is meaningless, but if you have `my file.txt`, the quoting is *critical* to prevent the shell parsing to turn it into two filenames, one named `my` and one named `file.txt`. Always quote user input and glob expansion lest you be very unhappy some day. – Seth Robertson Aug 26 '16 at 16:44
2

And this isn't just a theoretical - Mac OS X once shipped with an update script that didn't properly quote arguments and ended up deleting peoples' hard drives in some circumstances. – fluffy Aug 26 '16 at 21:48
2

@fluffy, do you have a link about that? – Wildcard Aug 27 '16 at 08:05
@Wildcard Unfortunately I'm unable to find any articles about it, but it was big news in the tech world when it happened. I want to say it was in 2003/2004 or thereabouts, back when Apple was still getting the hang of being a UNIX distributor. – fluffy Aug 27 '16 at 20:29
1

@wildcard Ah, found it! http://www.xlr8yourmac.com/OSX/itunes2_erased_drives.html - it was actually an iTunes upgrade script that was the culprit. – fluffy Aug 27 '16 at 20:34
@fluffy something similar happened to Steam a few months ago, although I'm not sure it was related to globbing in particular. – Darkhogg Aug 27 '16 at 20:40

score 7 · Answer 2 · answered Aug 26 '16 at 14:09

Without quotes the shell expands *.txt before invoking the script, so $1 is only the first file that gets expanded. All of the txt files are arguments to your script at that point (assuming there aren't too many).

With quotes that string is passed without being expanded into the script, which then lets the for do the expansion, as you're hoping for.

Why are quotes necessary for files argument when calling this Bash script?

2 Answers2