2

I ran into a situation where I was piping the output of compgen -G to xargs basename and could not get it to work until I added the xargs -I parameter as seen below. Here is a script demonstrating what I was trying to do, with explanatory comments. I have two questions, and they appear after the script.

# create three files for testing:
touch /tmp/my.file.1.2.txt
touch /tmp/1.my.file.2.txt
touch /tmp/1.2.my.file.txt

# create a glob and verify that it works with compgen:
glob=/tmp/*.file*.txt
compgen -G "$glob"

#output:
/tmp/1.2.my.file.txt
/tmp/1.my.file.2.txt
/tmp/my.file.1.2.txt

# try to get the basename of each file using xargs.
# I thought this would work, but it does not.
compgen -G "$glob" | xargs basename

#output:
basename: extra operand ‘/tmp/my.file.1.2.txt’
Try 'basename --help' for more information.

# eventually I discovered that this would work.
# however, I don't understand why this would work
# but the previous attempt would not, since I
# think that this command is just a more
# explicitly-specified version of the previous 
# one.
compgen -G "$glob" | xargs -I{} basename {}

#output:
1.2.my.file.txt
1.my.file.2.txt
my.file.1.2.txt

Other commands work with xargs without the -I parameter. For example, compgen -G "$glob" | xargs ls -al works just fine.

Question 1: What is it about basename in this script that requires the -I parameter?

Question 2: Until observing this result, I would have thought that xargs basename and xargs -I{} basename {} were synonyms of each other, but obviously they are not. What is the difference?

I doubt that it matters, but just in case: this is occurring on bash 5.0.17(1)-release running on Ubuntu 20.04.4 (5.13.0-35-generic).

I know there are other ways to generate this list of files, but I am concerned because I am clearly not understanding something fundamental here that I need to understand in order to avoid errors in the future.

chris
  • 123
  • 5

1 Answers1

3

POSIX basename only handles one name at a time, optionally with a suffix to be removed. xargs basename tries to run basename with as many arguments as possible at once, which fails; xargs -I{} basename {} causes xargs to run basename once per name to process.

Compare the outputs of

printf "foo\nbar\nbaz" | xargs echo basename

and

printf "foo\nbar\nbaz" | xargs -I{} echo basename {}

GNU basename supports a couple of options to allow multiple names; if you don’t need to specify a suffix:

xargs basename -a

and if you do,

xargs basename -s .suffix
Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164
  • 1
    In any case, it should rather be `compgen -G "$glob" | xargs -rd '\n' basename -a --` here (with the GNUisms). Still doesn't work for filenames with newline characters, the whole `bash` completion framework is broken by design and can't cope with arbitrary file paths. – Stéphane Chazelas Mar 21 '22 at 15:27