Passing second argument to GNU parallel while feeding ls

Question

I process files found by ls as

ls /folder/ | parallel -j20 ./command {}

but I need to pass the job number as well. I tried

ls /folder/ | parallel -j20 ./command {1} {2} ::: {1..20}
ls /folder/ | parallel -j20 ./command {} {} ::: {1..20}

but it does not work. I also tried {#} for passing the job number.

Note that you should never use the output of `ls` in a script. Using a `for` loop or `find` is always the better solution. — mashuptwice, Feb 20 '22 at 11:19
https://unix.stackexchange.com/questions/128985/why-not-parse-ls-and-what-to-do-instead https://mywiki.wooledge.org/ParsingLs — mashuptwice, Feb 20 '22 at 11:30
@mashuptwice oh thanks. I always work with well-formatted directories, and thus, never thought of that. — Googlebot, Feb 20 '22 at 11:32
Please [edit] your question and show us the commands you are trying to run. Give us a few file names and then what number should be passed an how. Are you trying to launch things like `./command file1.txt 1` and `./command file2.txt 2` etc.? — terdon, Feb 20 '22 at 11:57
@terdon `./command` script is irrelevant here. Consider it as a bash script with `echo "$1 $2"`. Yes, the aim is to run `/command file1.txt 1` or `/command 1 file1.txt`. The order is not important as I can change the arguments inside the script. — Googlebot, Feb 20 '22 at 12:55

steeldriver · Answer 1 · 2022-02-20T13:51:54.510

2

The {#} replacement string should be exactly what you need here. For example, given

$ ls
 file1  'file2 with spaces'  'file3'$'\n''with'$'\n''newlines'   file4   file5

then

parallel --null echo {#} {} ::: *
1 file1
2 file2 with spaces
3 file3
with
newlines
4 file4
5 file5

or, if you have enough files to exceed the ARG_MAX limit you could use

printf '%s\0' * | parallel --null echo {#} {}

edited Feb 20 '22 at 13:51

answered Feb 20 '22 at 13:04

steeldriver

78,509
12
109
152

You could also just do `parallel --null echo {#} {} ::: *`. – terdon Feb 20 '22 at 13:19
@terdon good point - I'm overcomplicating things as usual... – steeldriver Feb 20 '22 at 13:52

score 0 · Answer 2 · answered Feb 20 '22 at 19:38

It is unclear to me what you want to do. Maybe one of these?

ls /folder/ | parallel -j20 ./command {1} {2} :::: - ::: {1..20}
ls /folder/ | parallel -j20 ./command {1} {2} :::: - :::+ {1..20}
ls /folder/ | parallel -j20 ./command {1} {#}
ls /folder/ | parallel -j20 ./command {1} {%}

Passing second argument to GNU parallel while feeding ls

2 Answers2