How do I work with "find" strings and special characters?

Question

I am attempting to write some some bash shell script (for the first time) to perform a few sequential actions (some copy, encrypt, upload, and simple logic checks) and I am struggling to work with the strings find is returning as they have special characters /\*)'(" and spaces.

I have seen suggestions to fix this using printf %q but I haven't been able to figure out the correct bash syntax.

Here is an excerpt of how I'm trying to change the strings:

#!/bin/sh

find "/upload" -type f |
    while read -r path; do
        fixedpath=printf %q "$path"
        fixedpath=printf '%q' "$path"
        fixedpath=printf '%q' $path
        fixedpath=printf "%q" "$path"
        fixedpath="printf %q $path"
        fixedpath="printf '%q' $path"
        fixedpath=$(printf '%q' $path)
        echo $path >> list.txt
        echo $fixedpath >> list.txt
    done
exit 0

I could think of another 6-12 ways to write the printf function as well but I think you get the point, which one do I need to have a valid path returned that I can use to pass as an argument to other commands?

I am working on Ubuntu 16.

Processing the output of `find` is generally risky for the reasons you're noting. Depending on what you're trying to do your best bet might be to use the `-exec` option to `find` so you don't have to pass it to anything else. Another option, if your `find` supports it and the output processing handles it would be to use nul terminated output with `-print0` and then something on the other side that knows how to process nul terminated strings as well (like `xargs -0` or many that have a `-z` flag or similar — Eric Renouf, Dec 25 '17 at 15:55
Only the last one of your assignments actually calls `printf`, the first four set `fixedpath` and call a command named `%q`, the two next ones just assign to `fixedpath`. — ilkkachu, Dec 25 '17 at 16:31
In any case, it seems you're just printing the output from `find` to a file here. You might be better off just doing what you need from the output of `find` directly, or with `find -exec`. So what it is you're actually trying to achieve in the end? — ilkkachu, Dec 25 '17 at 16:33
@ilkkachu the reason I am not using -exec is I need to perform more if statements checking for file size, modification date, as we all reading and comparing integers from another file, and then finally passing the path as an argument to multiple different programs / functions. — user267461, Dec 25 '17 at 17:18
Write a script (`the-script`) that takes a filename as input and does the things you want to do. Then `find … -exec the-script {} \;` — Fox, Dec 25 '17 at 18:52

Kusalananda · Answer 1 · 2018-07-18T19:15:39.600

Do not parse the output from find, especially not if you're expecting weird and wonderful filenames.

Instead, either execute the commands you need to execute through -exec, or embed a script in the find command itself:

find /upload -type f -exec sh -c '
    for pathname do
        printf "Would do something with \"%s\" here...\n" "$pathname"
    done' sh {} +

Obviously, you may put the embedded script in its own file too:

#!/bin/sh

for pathname do
    printf 'Would do something with "%s" here...\n' "$pathname"
done

And then...

find /upload -type f -exec /path/to/script.sh {} +

Further reading on this subject:

m0dular · Answer 2 · 2017-12-28T20:09:37.220

Note that using #!/bin/sh will (probably) limit you to POSIX specs, not bash. Since you mention bash you should change the shebang to #!/bin/bash.

See this page, particularly the "Other Examples" section. My preferred method if you need to use find is:

#!/bin/bash

while IFS= read -r -d '' f; do
   echo "$f"
   # Other actions on "$f" here
done < <(find /upload -type f -print0)

exit

This will safely handle special characters by using a null delimiter.

Alternatively, it's simpler to use a couple of bash shell options and a for loop:

shopt -s nullglob globstar

for f in /upload/**; do
   [[ -f $f ]] || continue
   echo "$f"
   # Other actions on "$f" here
done

That will recursively find regular files under /upload. Be sure to quote "$f" in both methods.

score 0 · Answer 3 · answered Jul 19 '18 at 23:37

The script fails (expands the asterisks *) when this lines get executed:

echo $path
echo $fixedpath

Example:

$ path='test *'
$ echo "$path"
test *

$ echo $path
test file 1 file2 A longer file in this directory Some_Other_file ......

Solution? : Quote your var expansions.

Having said that, the only assignment that may work is (also quote it):

fixedpath=$(printf '%q' "$path")
echo "$path" >> list.txt
echo "$fixedpath" >> list.txt

But reading from the output of find is a bad idea which might fail in similar ways as not quoting variables. Read the other answers for that part, they are good.

How do I work with "find" strings and special characters?

3 Answers3