I want to pipe file names to other programs, but they all choke when the names contain spaces.
Let's say I have a file called.
foo bar
How can I get find to return the correct name?
Obviously I want:
foo\ bar
or:
"foo bar"
EDIT: I don't want to go through xargs, I want to get a correctly formatted string out of find so that I can pipe the string of file names directly to another program.
POSIXLY:
find . -type f -exec sh -c '
for f do
: command "$f"
done
' sh {} +
With find supports -print0 and xargs supports -0:
find . -type f -print0 | xargs -0 <command>
-0 option tells xargs to use the ASCII NUL character instead of space to end (separate) the filenames.
Example:
find . -maxdepth 1 -type f -print0 | xargs -0 ls -l
Using -print0 is one option, but not all programs support using nullbyte-delimited data streams, so you'll have to use xargs with the -0 option for some things, as Gnouc's answer noted.
An alternative would be to use find's -exec or -execdir options. The first of the following will feed the filenames to somecommand one at a time, while the second will expand to a list of files:
find . -type f -exec somecommand '{}' \;
find . -type f -exec somecommand '{}' +
You may find that you are better off using globbing in many cases. If you have a modern shell (bash 4+, zsh, ksh), you can get recursive globbing with globstar (**). In bash, you have to set this:
shopt -s globstar
somecommand ./**/*.txt ## feeds all *.txt files to somecommand, recursively
I have a line saying shopt -s globstar extglob in my .bashrc, so this is always enabled for me (and so are extended globs, which are also useful).
If you don't want recursiveness, obviously just use ./*.txt instead, to use every *.txt in the working directory. find has some very useful fine-grained searching capabilities, and is mandatory for tens of thousands of files (at which point you'll run into the shell's maximum number of arguments), but for day-to-day usage it is often unnecessary.
So if you don't want to use xargs (thus likely nor e.g. parallel), a find output can be read and processed line by line like following way:
find . -type f | while read x; do
# do something with $x
done
find ./ | grep " "
will get you the files and directories contains spaces
find ./ -type f | grep " "
will get you the files contains spaces
find ./ -type d | grep " "
will get you the directories contains spaces
By setting the Internal Field Separator to newline, the shell will ignore the spaces:
IFS=$'\n' eval 'for i in `find . -type f -name "*"`;do echo $i;done'
Personally, I'd use the -exec find action to solve this sort of problem. Or, if necessary, xargs, which allows for parallel execution.
However, there is a way to get find to produce a bash-readable list of filenames. Unsurprisingly, it uses -exec and bash, in particular an extension to the printf command:
find ... -exec bash -c 'printf "%q " "$@"' printf {} ';'
However, while that will print out correctly shell-escaped words, it will not be usable with $(...), because $(...) does not interpret quotes or escapes. (The resut of $(...) is subject to word splitting and pathname expansion, unless surrounded by quotes.) So the following will not do what you want:
ls $(find ... -exec bash -c 'printf "%q " "$@"' printf {} +)
What you would have to do is:
eval "ls $(find ... -exec bash -c 'printf "%q " "$@"' printf {} +)"
(Note that I have made no real attempt to test the above monstrosity.)
But then you might as well do:
find ... -exec ls {} +
If all you want is to escape the spaces, you can do this:
find (...) | sed -e's/ /\\ /g' | whatever...
But as far as I know only xargs requires such treatment. For example:
find . -type f -name '* *' | sed -e's/ /\\ /g' | xargs ls -l
It might work for you. It doesn't escape anything but the spaces, but covers the most usual case. Quotes in file names might be a problem still.
If you are going to use the filenames in a script you might want to combine BASH arrays with find. Something like this:
readarray -d '' ALL_JSON_FILES < <(find "my_src_dir" -name "*.json" -print0)
for TEAM in "${ALL_JSON_FILES[@]}"
do
Inspired from SO questions:
https://stackoverflow.com/a/54561526/671282\
https://stackoverflow.com/a/8880633/671282
There are many different answers, depending on how exactly you want to use the output, as well as what assumptions you are making about what odd characters aren't in the filenames. The find command doesn't have an option to escape special characters, but if it did, it's choice of what to escape might not match the exact needs of your program. Considering that the only illegal characters in filenames are '/' and NULL, there are a lot of edge cases.
In my case, I wanted to process the file names as elements in an array in Bash, so I wanted something like:
FILES=( $(find . -type f) )
That doesn't work with spaces (or tabs, for that matter). That also kills the newlines from the find command, making them useless as separators. You can set the field separator in Bash to something different. Ideally, you would set it to null and use -print0 in find, but null is not allowed as a field separator in Bash. My solution is to pick a character that we assume is not in any filenames, like 0x01 (ctrl-a), and use that:
IFS=$'\x01'
FILES=( $(find . -type f | sed -e 's/$/\x01/') )
unset IFS
for F in "${FILES[@]}"; do
useful_command "$F"
done
Note the need to unset IFS to restore it to the default. That won't work with filenames with newlines in them, but should work with most other filenames.
If you're really paranoid, then you'll need to do a find piped to hexdump, split out the results to get all the hex values, and look for one that isn't in the results. Then use that value. I'm sure Johnny Drop Tables has files with every hex code in the file names. If you're paranoid, create file and directory names using all 253 legal characters and test. Probably the only solutions that would pass that test would be ones using 'find -print0' piped to xargs or a custom C program.
find . -type f -name \*\ | sed -e 's/ /<thisisspace>/g'
