escaping filename results from grep

Question

How do I escape the output of grep so that bash can read the file names correctly?

I have a text file that is the output of a find command. For each line, I want to make a symlink. For now, I'm just testing my loop with ls. However, I am not quoting the string output of grep properly for special characters. This causes filesystem commands on each file to fail.

$ tree dir/
dir/
├── Another & file ' name.txt
└── file name.txt

0 directories, 2 files

$ cat files.txt
dir
dir/Another & file ' name.txt
dir/file name.txt

$ grep file files.txt | awk -v q='"' '{printf(q"%s"q"\n", $0);}'
"dir/Another & file ' name.txt"
"dir/file name.txt"

$ while read p ; do 
    echo $p; ls $(grep file files.txt | awk -v q='"' '{printf(q"%s"q"\n", $0);}') ; 
done < files.txt
dir
ls: cannot access '"dir/Another': No such file or directory
ls: cannot access '&': No such file or directory
ls: cannot access 'file': No such file or directory
...
dir/Another & file ' name.txt
ls: cannot access '"dir/Another': No such file or directory
ls: cannot access '&': No such file or directory
ls: cannot access 'file': No such file or directory
...

I've tried both single quotes and double quotes. How can I escape this to execute commands on the paths outputted from grep?

Try using a for loop instead of [parsing the output of ls](http://mywiki.wooledge.org/ParsingLs). — clk, Jul 28 '16 at 04:37
What is that loop supposed to do? It looks you're taking each line in `files.txt` one-by-one, and then grepping the lines out of the same file again? Do you want to find duplicates or partial matches from the file, ...? If you just want to do something with every file (line) matching the grep, would just something like this work: `grep file files.txt | while read p ; do echo "<$p>" ; ln -s "$p" make-links-here/ ; done` — ilkkachu, Jul 28 '16 at 08:48
@ilkkachu in this contrived example, the loop really does nothing. In my real-life project, the loop finds one match out of thousands of lines in the file of filesystem entries, and makes a symlink to it. — user394, Jul 28 '16 at 12:11

Stéphane Chazelas · Accepted Answer · 2016-07-28T05:03:37.963

In

ls $(grep file file.txt)

You're using the split+glob operator incorrectly and that's where your problem lies. You don't want to insert quotes in the output of grep as that output is not interpreted as shell code (thankfully!), but you need to tune your split+glob operator.

Here

You don't want the glob part (the expansion of * words for instance into the list of files in the current directory), so you need to disable it with:
```
set -o noglob
```
You want to split the output of grep on newline characters only (that still means your file names can't contain newline characters, but that's a limitation on the format of your file.txt file) with:
```
IFS='
'
```
or
```
IFS=$'\n'
```
in some shells.

And then you can invoke your split+glob operator (leaving $(...) unquoted), but more like:

ls -d -- $(grep file files.txt)

That split+glob operator is a misfeature inherited from the Bourne shell. With modern shells, there are other ways to split some text into the list of its lines.

With zsh:

ls -d -- ${(f)"$(grep file files.txt)"}

The, f parameter expansion flag is to split on newline (linefeed), short for ps:\n:' (s:string: used to split on arbitrary strings, p to understand escape sequences like \n in that string). We quote $(...) to disable the split+glob operator, though in the case of zsh, there's only the split part (no glob).

With bash:

readarray -t files < <(grep file files.txt)
ls -d -- "${files[@]}"

Where you could want to insert quotes is when using xargs as xargs's input format is a blank separated list where the separators can be quoted with single quote, double quote or backslash (but in a different way than shell quoting is done).

So you could do:

 sed '
   s/"/"\\""/; # escape the " characters themselves
   s/^/"/;s/$/"/; # insert one at the beginning and one at the end
 ' < files.txt | xargs ls -d --

What exactly is the split+glob operator? The `$(...)` part? I tried to google that phrase but got no results. — user394, Jul 28 '16 at 12:12
@user394 That's what happen when you leave an expansion (parameter expansion, command substitution or arithmetic expansion) unquoted. See [Security implications of forgetting to quote a variable in bash/POSIX shells](http://unix.stackexchange.com/q/171346) and/or https://unix.stackexchange.com/search?q=%22split%2Bglob%22 — Stéphane Chazelas, Jul 28 '16 at 12:52

escaping filename results from grep

1 Answers1