Applying bash function to each file in subfolder recursively

Question

I am trying to write a script that will apply a bash function for each file in a directory recursively. For example if the directory tests had all my files and sub-directories in it, the script

find tests -type f -print0 | xargs -0 echo

Works perfectly. Now I want to be able to apply a bash function rather than just echo, so I came up with something like this:

function fun() {
    echo $1
}

export -f fun

find tests -type f -print0 | xargs -0 bash -c 'fun "$@"'

However this only outputs a single file in tests, where before it output all the files. I would expect these two to run the same no?

You can try the exec option of find, i think will do the job. http://stackoverflow.com/questions/5119946/find-exec-with-multiple-commands — George Vasiliou, Nov 22 '16 at 00:44
Is your function even defined in that bash process? If you say `bash -c`, you get a non-interactive shell, which doesn't look at `.bashrc`. Secondly, does your function look at multiple arguments? Thirdly, you should say `bash -c 'fun "$@"' sh`, because the first word after the command string after `bash -c` is assigned to `$0`, and therefore isn't included in `"$@"`. … … … … … … … … … … … … … … … … OK, ignore the first point; I hadn't noticed that you had exported the function. — G-Man Says 'Reinstate Monica', Nov 22 '16 at 00:50
And I hadn't noticed that you showed us the function! (D'oh! facepalm!) My second point is that, (as you might have noticed from your experiment with `echo`) `xargs` executes the command with all the files at once (or, at least, as many as can be passed on one command line — this is in the thousands) and not once per command. Since your function prints only `$1` (and not all the arguments), you get only one. — G-Man Says 'Reinstate Monica', Nov 22 '16 at 01:04

score 2 · Answer 1 · edited Apr 13 '17 at 12:36

2

Use a shell script.

To quote from this excellent answer:

Do other programs besides your shell need to be able to use it? If so, it has to be a script.

As already noted in other answers, it may be possible to work around this by exporting the function, but it's certainly a lot simpler to just use a script.

#!/bin/bash
# I'm a script named "fun"!  :)
printf '%s\n' "$@"

Then you just put the fun script somewhere in your PATH (perhaps in ~/bin), make sure it's executable, and you can run:

find tests -type f -exec fun {} +

edited Apr 13 '17 at 12:36

Community

1

answered Nov 22 '16 at 23:44

Wildcard

35,316
26
130
258

Nice, I guess this is how I check all ruby files syntax: `find . -name "*.rb" -exec ruby -c {} \;` – Nakilon Aug 29 '18 at 09:29

score 1 · Answer 2 · answered Nov 22 '16 at 03:07

1

There two ways to solve this issue: you could either tell xargs to run command with only one argument at a time using -n1 or loop for all arguments inside your function fun:

function fun() { echo $1 }
export -f fun
find tests -type f -print0 | xargs -n1 -0 bash -c 'fun "$@"' --

or

function fun() { while [ -n "$1" ]; do echo $1; shift; done }
export -f fun
find tests -type f -print0 | xargs -0 bash -c 'fun "$@"' --

answered Nov 22 '16 at 03:07

Fedor Dikarev

1,761
8
13

1

Maybe a clearer way to loop through arguments is with a `for`: `function fun() { for a; do echo "$a"; done }` – trosos Nov 22 '16 at 03:12
1

@trosos, or just `printf '%s\n' "$@"`, since [printf is better than echo.](http://unix.stackexchange.com/q/65803/135943) – Wildcard Nov 22 '16 at 23:44

score 1 · Answer 3 · answered Mar 04 '17 at 14:30

1

If you use GNU Parallel it looks like this:

function fun() {
    echo "$@"
}
export -f fun

find tests -type f -print0 | parallel -0 -Xj1 fun

answered Mar 04 '17 at 14:30

Ole Tange

33,591
31
102
198

Applying bash function to each file in subfolder recursively

3 Answers3