Process a file list with space in file names

Question

I am trying develop a bash script which processes a list of files which may contain space names.

(I have already consulted Trouble in script with spaces in filename and Why I can't escape spaces on a bash script?, but can not seem to perfect my script.)

Here is my script. The actual process is more complicated which I reduce here to simple file command.

#!/bin/bash

if [ $# -eq 0 ]; then
   echo "Usage is $0 <files to be tested>"
   exit
fi

allfilestobetesed=$@
for filetobetested in "$allfilestobetested"
do
    file "$filetobetested"
done

How do I improve my script?

Are the names with spaces being quoted when invoking the script? — Eric Renouf, Oct 23 '15 at 15:03
@EricRenouf No. This is mostly `*` or `*.tex` or something similar. — Masroor, Oct 23 '15 at 15:06
The shell will expand them correctly there, so that should be OK I think. — Eric Renouf, Oct 23 '15 at 15:09
@Masroor you made a mistake in copying what was posted. Use `"@"` not `'"$@"'` Also it is generally more helpful to put the `set -x` inside the script. — rocky, Oct 23 '15 at 15:33
@rocky I thought that at first too, but that's just how `set -x` ends up displaying the `"$@"` part, I tried it locally and saw the same thing, it just turned out to be a typo further down — Eric Renouf, Oct 23 '15 at 15:41

score 4 · Accepted Answer · answered Oct 23 '15 at 15:06

4

You should probably get rid of your allfilestobetested variable if you can:

for filetobetested in "$@"; do
    file "$filetobetested"
done

will expand correctly

answered Oct 23 '15 at 15:06

Eric Renouf

18,141
4
49
65

Funny, this won't work if you run it like this: `bash a.sh $(ls -1)` (supposing that you save this script as `a.sh`). – Kira Oct 23 '15 at 17:23
1

@Kira Right, because then you're getting a string as output from `ls -1` and using that string as the arguments to `a.sh`, which will be subject to word splitting rules. Basically meaning all the spaces will be processed before you start running `a.sh`. That means you can't do much to fix it from within – Eric Renouf Oct 23 '15 at 17:43
But does is strip out the '\n' too? I've tried splitting the output in the loop by line breaks with `IFS=$'\n'`, but with no success =/ – Kira Oct 23 '15 at 18:07
1

@kira I fear you've set for yourself a very difficult task if you want to try to use `ls` there. You might want to read http://mywiki.wooledge.org/ParsingLs for a good discussion of why it's going to be very difficult to get it to do what you want. Also, as I noted, if you are setting `IFS` inside your script that's too late, since bash is doing the word splitting to assign the output of `ls` to the arguments to the script before the script is invoked. It's too late when you're inside a.sh, the damage is already done so to speak – Eric Renouf Oct 23 '15 at 18:10

rocky · Answer 2 · 2015-10-23T15:37:27.020

2

You were pretty close. First, you got right using $@ instead of $*.

You need to worry about shell expansion so don't use " unnecessarily as happened in the assignment. That will remove the boundaries between tokens. However you do need it around "@_".

So we have:

if [ $# -eq 0 ]; then
   echo "Usage is $0 <files to be tested>"
   exit
fi

for filetobetested in "$@"
do
    file "$filetobetested"
done

Lastly, my bash debugger http://bashdb.sf.net can help you follow stuff like this. Section 1.2 of the bashdb manual describes how to set PS4 to give more information from set -x tracing.

edited Oct 23 '15 at 15:37

answered Oct 23 '15 at 15:14

rocky

1,978
11
23

Thanks, but how does this answer differ from the other answer already posted? – Masroor Oct 23 '15 at 15:17
1

I started writing it it at the same time as the first answer. It took me longer to write though because I was explaining _why_ the changes. – rocky Oct 23 '15 at 15:28

score 0 · Answer 3 · answered Oct 24 '15 at 08:37

0

You don't need "$@":

    for filetobetested
    do
        file "$filetobetested"
    done

answered Oct 24 '15 at 08:37

RobertL

6,690
1
19
38

Yes, I do. The `file` command is not the actual command in the real script, where the files need to be processed individually. – Masroor Oct 24 '15 at 12:59
No you don't. My statement has nothing to do with whether you use the `file` command or not. `for xyz in "$@"` means exactly the same thing as `for xyz` with no `in` clause. From the dash (posix shell) manpage: _Omitting `in word` ... is equivalent to `in "$@"`_. The bash shell works the same way. – RobertL Oct 24 '15 at 23:34

Process a file list with space in file names

3 Answers3