Piping commands after a piped xargs

Question

HP-UX ***** B.11.23 U ia64 **** unlimited-user license

find . -type d -name *log* | xargs ls -la

gives me the directory names (the ones which contain log in the directory name) followed by all files within that directory.

The directories /var/opt/SID/application_a/log/, /var/opt/SID/application_b/log/, /var/opt/SID/application_c/log/ and so on contain log files.

I want only the two latest logfiles to be listed by the ls command, which I usually find using ls -latr | tail -2.

The output has to be something like this..

/var/opt/SID/application_a/log/
-rw-rw-rw-   1 user1    user1      59698 Jun 11  2013 log1
-rw-rw-rw-   1 user1    user1      59698 Jun 10  2013 log2
/var/opt/SID/application_b/log/
-rw-rw-rw-   1 user1    user1      59698 Jun 11  2013 log1
-rw-rw-rw-   1 user1    user1      59698 Jun 10  2013 log2
/var/opt/SID/application_c/log/
-rw-rw-rw-   1 user1    user1      59698 Jun 11  2013 log1
-rw-rw-rw-   1 user1    user1      59698 Jun 10  2013 log2

find . -type d -name *log* | xargs ls -la | tail -2 does not give me the above result. What I get is a list of last two files of find . -type d -name *log* | xargs ls -la command.

So can I pipe commands after a piped xargs? How else do I query, to get the resultant list of files in the above format?

find . -type d -name *log* | xargs sh -c "ls -ltr | tail -10"

gives me a list of ten directory names inside the current directory which happens to be /var/opt/SID and that is also not what I want.

You should quote the `*log*` otherwise the shell will expand it. — Anthon, Jun 12 '15 at 13:28
Be aware that `sh -c` expects the command name (parameter 0) as its second argument, so you should always do `find . -type d -name *log* | xargs sh -c "ls -ltr | tail -10" lstail` (notice the `lstail` at the end, which will serve as `$0` for the created shell). Otherwise the first of your results will fill that role and go unused. — Jonas, Aug 16 '18 at 05:45

score 137 · Accepted Answer · edited Jun 16 '18 at 20:51

137

You are almost there. In your last command, you can use -I to do the ls correctly

-I replace-str
Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character. Implies -x and -L 1.

So, with

find . -type d -name "*log*" | xargs -I {} sh -c "echo {}; ls -la {} | tail -2"

you will echo the dir found by find, then do the ls | tail on it.

edited Jun 16 '18 at 20:51

Scott - Слава Україні

10,332
10
39
68

answered Jun 12 '15 at 13:17

fredtantini

4,153
1
14
21

find . -type d -name "*log*" | xargs -I {} sh -c "echo {};ls -ltr {} | grep -E \"file_in|file_out\"|tail -5" I used this one. – anotherperson1 Jun 15 '15 at 12:10
4

I do with `xargs -n 1 sh -c 'echo $0'` – Ginhing Apr 21 '16 at 03:16
1

on macOS the replacement didn't work within the quoted command, so `echo $0` was helpful (and more understandable) – mpuncel Nov 18 '16 at 17:31
4

To make it work with filenames with spaces, use `-print0`, `xargs -0` and escape `{}` into double quotes _inside_ the `sh` command: `find . -type d -name "*log*" -print0 | xargs -0 -I {} sh -c "echo \"{}\";ls -la \"{}\" | tail -2"` – Benedikt Köppel Mar 17 '18 at 17:11
I use "xargs -n 1 sh -c "echo \$0 | $func"` if I need to use the $func variable – god Feb 19 '19 at 09:06
why does xargs see the pipe. Shouldn't it be handled by bash? – Talespin_Kit Jan 13 '23 at 09:38

Borisu · Answer 2 · 2021-02-22T09:30:46.560

Just in addition to fredtantini and as a general clarification (since the docs are a bit confusing):

The xargs -I {} will take the '{}' characters from the standard input and replace them with whatever comes in from the pipe. This means you could actually replace {} with any character combination (maybe to better suite your preferred programming flavor). For example: xargs -I % sh -c "echo %". If you always use the xargs -I {} you can replace it with xargs -i as it is the shorthand. EDIT: The xargs -i option has been deprecated, so stick to the xargs -I{}.

The sh -c will tell your bash/shell to read the next command from a string and not from the standard input. So writing sh -c "echo something" is equivalent to echo something.

The xargs -I {} sh -c "echo {}" will read the input you created with sh -c which is echo {}. Since you told it to replace {} with the arguments you got from the pipe, that's what will happen.

You can easily test this even without piping, just type the above command in a terminal. Whatever you write next will get outputted to the terminal (Ctrl-D to exit).

In the ls -la {} command the same thing happens again. The {} is replaced with the contents of the pre-pipe command.

"If you always use the `xargs -I {}` you can replace it with `xargs -i` as it is the shorthand." Note that `-i` is not POSIX. — kelvin, Feb 20 '21 at 20:38

score 10 · Answer 3 · answered Aug 13 '17 at 19:57

GNU Parallel makes this kind of tasks easy:

find . -type d -name "*log*" | parallel --tag "ls -la {} | tail -2"

If you do not want to do a full install of GNU Parallel you can do a minimal installation: http://git.savannah.gnu.org/cgit/parallel.git/tree/README

Piping commands after a piped xargs

3 Answers3

Linked