How do I loop over the lines in STDIN and run a shell command?

Question

I'd like to run a shell command on each line taken from STDIN.

In this case, I'd like to run xargs mv. For example, given two lines:

mfoo foo
mbar bar

I'd like to run:

xargs mv mfoo foo
xargs mv mbar bar

I've tried the following strategies with ruby, awk, and xargs. However, I'm doing it wrong:

Just xargs:

$ echo "mbar bar\nmbaz baz" | xargs mv
usage: mv [-f | -i | -n] [-v] source target
       mv [-f | -i | -n] [-v] source ... directory

Through awk:

$ echo "mbar bar\nmbaz baz" | awk '{ system("xargs $0") }'

Through ruby:

$ echo "mbar bar\nmbaz baz" | ruby -ne '`xargs mv`'
$ ls
cat  foo  mbar mbaz

I have some questions:

• How do I do what I'm trying to do?
• What is wrong with each of my attempts?
• Is there a better way to "think about" what I'm trying to do?

I'm especially confused that my xargs attempt isn't working because the following works:

$ echo "foo\nbar" | xargs touch
$ ls
bar foo

Answer 1

echo "mbar bar\nmbaz baz" | xargs mv

With xargs you should use the -t option to see what's going on. So in your above case if we were to invoke xargs with -t, what do we see:

mv mbar bar mbaz baz

So obviously it's not correct. What happened was that xargs like a hungry crocodile, ate all the args fed to it via the pipe by echo. So you need a way to limit the release of arguments to the croc. And since you requested on a per-line basis, then what you need is the -l or the -L for POSIX option.

echo "mbar bar\nmbaz baz" | xargs -l -t mv

POSIX-ly way:

echo "mbar bar\nmbaz baz" | xargs -L 1 -t mv

mv mbar bar
mv mbaz baz

And this is what you wanted. H.T.H

Answer 2

You can go this way:

echo -e "foo bar\ndoo dar" | xargs -n 2 mv

To read from file:

cat lines.txt | xargs -n 2 mv

or

xargs -a lines.txt -n 2 mv

Answer 3

xargs might not behave how you expect. See @MichaelHomer's comment to this question:

xargs mv mfoo foo will wait for input and run mv mfoo foo $x_1 $x_2 $x_3... for every line $x_n of input it gets. @MichaelHomer

This can be confirmed by reading the xargs man page:

DESCRIPTION
The xargs utility reads space, tab, newline and end-of-file delimited strings from the standard input and executes utility with the strings as arguments.

In practice, it seems like xargs will ignore newlines when feeding args to the command line utility given as it's first arg. For example, notice what happens to the newline:

$ echo "foo bar\ncat dog"
foo bar
cat dog
$ echo "foo bar\ncat dog" | xargs echo
foo bar cat dog

This behavior can be controlled with the -n flag:

-n number

Set the maximum number of arguments taken from standard input for
each invocation of utility.

Going back to the previous example if we wanted xargs take only 2 args then quit then we can use @ddnomad's approach:

$ echo "foo bar\ncat dog" | xargs -n 2 echo
foo bar
cat dog

Also, as shown in the comments in Rakesh Sharma's answer, on BSD xargs you can specify the number of lines to be read with the -L flag. From the man page:

-L number

Call utility for every number lines read. If EOF is reached and
fewer lines have been read than number then utility will be called with the available lines.

Equally helpful for testing is the -t switch:

Revisiting our example, both of the followings will work; however, the second example is better because it's what was intending in this case:

$ echo "foo bar\ncat dog" | xargs -n 2 echo
foo bar
cat dog
$ echo "foo bar\ncat dog" | xargs -L 1 echo
foo bar
cat dog

Note from the man page:

The -L and -n options are mutually exclusive; the last one given will be used.

Answer 4

There is a very simple solution that does not use xargs.
Define this function:

$ mv2() { while (($# >1 )); do echo mv "$1" "$2"; shift 2; done; }

Then, just call it with the list of filenames needed (no newlines required):

$ mv2 mbar bar mbaz baz
mv mbar bar
mv mbaz baz

Remove the echo to make the function really work.