0

I'm trying to do that most basic of things: perform one or more commands on all files within a particular folder.

In this instance, it's converting eps files to PDF using macOS's pstopdf command.

#!/bin/zsh

FILES=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)

for f in "$FILES"
do
   echo "$f"
   pstopdf "$f"
done

echo "$f" produces a correct list of all the files; but I then get a second list of files -- seemingly from pstopdf itself** -- starting with File name too long: /Users/Ben/Pictures/Stock Illustrations/Flock wallpaper.eps, but the rest of the files are listed correctly.

However, the pstopdf command doesn't create any PDF files.

** I've tried commenting out the echo and the pstopdf command, so I know that each produces a list of filenames.

If I run pstopdf <file.eps> in the Terminal, I get no output to the CLI (e.g. no filename listed), but the file is processed and a PDF file created.

I dare say I could use xargs in the find command, though I prefer the more structured approach of a loop with arguments, not least because it gives the option of multiple commands and other logic, and it's easier to read.

There is a similar question here: I get message "File name too long" when running for..in and touch

But I don't understand how the answer applies. It says "or touch them one by one" (which is what I want), but if I do something like:

FILES=/Users/Ben/Pictures/Stock\ Illustrations/*

I just get "No such file or directory".

benwiggy
  • 103
  • 4
  • that last assignment in itself shouldn't give any error, it just assigns the string (without glob expansion) to `FILES`. depending on where and how you use it, you might get that error, possibly due to word splitting messing the result up when expanded. But you'd have to show what you're actually doing there. – ilkkachu Aug 30 '22 at 11:01
  • 1
    Your problem with the `for f in "$FILES"` looks exactly the same as in the question you linked, `$FILES` will be a single string containing _all_ the filenames, joined together. – ilkkachu Aug 30 '22 at 11:02
  • @ilkkachu so `for f in $files` doesn't split it up, like say in python? Can you give the correct wording? – benwiggy Aug 30 '22 at 11:04
  • no, `for f in $files` does wordsplit on whitespace by default (and expand globs), but `for f in "$files"` doesn't. But the splitting is rife with issues, and ignores the fact that filenames can themselves contain whitespace, so it's usually best avoided. I'm not sure what Python behaviour you refer to, since something like `for i in list` in Python would loop over each element of the list (without textual splitting), while `for i in string` would loop over each _character_ of the string (not splitting on whitespace) – ilkkachu Aug 30 '22 at 11:07

2 Answers2

2
FILES=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)
for f in "$FILES"

As in the linked post, $FILES here is a single string containing all the filenames. With enough files, it's too long for a single filename. With just a few names, you'd be attempting to access a file with embedded newlines in the name.

Here, you'd be better off by having find call pstopf itself, e.g.

$ find "/Users/Ben/Pictures/Stock Illustrations" -type f -print -exec pstopdf {} \;

If you want to use a shell loop over find's output, you'd have to do something like this (in Bash, I didn't pick up the mention of zsh there) :

set -f      # disable globbing
IFS=$'\n'   # set IFS to just the newline (Bash/ksh/zsh, not POSIX)
files=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)
for f in $files; do
    echo "$f"
    pstopdf "$f"
done

Or use find ... -print0 | while IFS= read -r -d '' f; do ...; done in Bash.

Or use for f in "/Users/Ben/Pictures/Stock Illustrations"/**/*; do ... in Bash (with shopt -s globstar), ksh or zsh (with some variations in specifics between the shells.

Both shell solutions would have issues with filenames containing newlines, which I hope you don't have, but which sadly are allowed filenames.

ilkkachu
  • 133,243
  • 15
  • 236
  • 397
  • Whoops, [Jeff's answer in the other one](https://unix.stackexchange.com/a/405530/170373) did also have the `find -exec`. – ilkkachu Aug 30 '22 at 11:12
  • Thank you. I prefer the more structured multi-line approach with a for loop. – benwiggy Aug 30 '22 at 11:14
  • 1
    @benwiggy that is your call, of course, but note that the for loop approach above will be slower and less efficient than `find` and also cannot deal with file names with newline characters. The pure `find` approach is far better: it is more concise, more efficient, and more robust to strange file names. – terdon Aug 30 '22 at 11:15
  • 1
    @benwiggy, there's also `for f in /some/path/**/*; do ...` for a recursive glob, if you don't need the more specific filters `find` has. – ilkkachu Aug 30 '22 at 11:18
  • I'm still getting the File name too long error with your `IFS=$'\n'` method. – benwiggy Aug 30 '22 at 11:25
  • Yes, the wildcard filepath works now - though I'm sure I ruled that out previously for some other error! – benwiggy Aug 30 '22 at 11:27
  • 1
    @benwiggy, ah, note that `$'...'` isn't POSIX, it works in Bash/ksh/zsh and Busybox, but not all pure POSIX shells. You could use `IFS=''` with a literal newline there in a POSIX shell, but it's somewhat hard to read. – ilkkachu Aug 30 '22 at 11:35
  • Is it easy to add multiple commands to an exec option? – benwiggy Aug 30 '22 at 11:45
  • @benwiggy, you could just add multiple `-exec` options, like `find ... -exec somecmd {} \; -exec othercmd {} \;` and it should run them each in order. That's more like running `somecmd "$f" && othercmd "$f"` in the shell, in that if any of the commands fails, it would stop there and not run the next one. (That's what I think it should do, since the `-exec` stanzas also act as conditions.) Or you could run all that via a shell `find ... -exec sh -c 'somecmd "$1"; othercmd "$1"' sh {} \;` or `find ... -exec sh -c 'for f do somecmd "$f" && othercmd "$f"' sh {} +` – ilkkachu Aug 30 '22 at 12:14
  • OP is using zsh, not bash nor sh. – Stéphane Chazelas Aug 30 '22 at 12:17
  • `find "/Users/Ben/Pictures/Stock Illustrations" -type f -print -exec pstopdf {} \;` could end up calling `pstopdf` on files created by a previous invocation of `pstopdf`. – Stéphane Chazelas Aug 30 '22 at 12:18
  • I guess I'm looking for the shell equivalent of passing files to a function. – benwiggy Aug 30 '22 at 12:30
1
FILES=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)

Is the syntax for a scalar variable assignment (in zsh or any other Korn-like shell), so you get a $FILES with only one value which is the whole output of find.

Here, you'd rather want:

files=( ${(0)"$(find "/Users/Ben/Pictures/Stock Illustrations" -type f -print0)"} )

If you wanted $files to be an array with elements being each file found by find. You need find to supply that list 0-delimited and split it on 0s with the 0 parameter expansion flag, as that's the only byte value that cannot occur in a file path.

Here though, you don't need find since zsh globs can do the same with:

files=( "/Users/Ben/Pictures/Stock Illustrations"/**/*(ND.) )

Where **/* does recursive globbing and . is the equivalent of -type f.

To loop over them, you want:

for f in $files; do
  print -r -- $f
  pstopdf $f
done

Or:

for f ($files) {
  print -r -- $f
  pstopdf $f
}

"$files" would join the elements of the array with the first character of $IFS (same as "${files[*]}" in ksh) so would not be what you want. Using $files unquoted in zsh loops over the non-empty elements of the array. "${files[@]}" to loop over all the elements though here it makes no difference as the array is guaranteed not to contain empty elements.

Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501