0

Disclaimer: Yes, finding files in a script with ls is bad, but find can't sort by modification date.

Using ls and xargs with echo everything is fine:

$ ls -t1 ssl-access*.gz | xargs -n 1 echo
ssl-access.log.2.gz
ssl-access.log.3.gz
ssl-access.log.4.gz
[...]

Changing echo to zcat:

$ ls -t1 ssl-access*.gz | xargs -n 1 zcat
gzip: ssl-access.log.2.gz.gz: No such file or directory
gzip: ssl-access.log.3.gz.gz: No such file or directory
gzip: ssl-access.log.4.gz.gz: No such file or directory
[...]

Duplicate file suffixes?! What is going on here?

UPDATE:

OS is Debian 5.

zcat is a shell script at /bin/zcat:

#!/bin/sh
PATH=${GZIP_BINDIR-'/bin'}:$PATH
exec gzip -cd "$@"
Christoph Wurm
  • 5,678
  • 5
  • 25
  • 28
  • Unable to reproduce. Perhaps try taking a look at your `zcat` executable (probably a shell script) and see if it is auto-appending a `.gz` extension. – jw013 Oct 21 '11 at 20:51
  • 3
    You actually can sort by modified time if you use `-printf %t` (gnu-only) or `-exec stat -c '%X' {}\;` then pipe it to `sort`. – Shawn J. Goff Oct 21 '11 at 21:10
  • What happens on your system when you `zcat` a file that is not compressed? – Shawn J. Goff Oct 21 '11 at 23:14
  • @ShawnJ.Goff: I get the error message `gzip: test.txt: not in gzip format` with return code 1. – Christoph Wurm Oct 21 '11 at 23:19
  • 2
    See [Shell script for moving oldest files?](http://unix.stackexchange.com/q/22674) for better ways to sort files by date. The only advantage of `xargs` is running the command for a batch of files at once, and you're not using this capability, so don't even consider using it. About the strange behavior you observe: what does `type zcat` show? What is the ***full*** output from `ls -t1 ssl-access*.gz`? – Gilles 'SO- stop being evil' Oct 22 '11 at 02:11
  • the zcat that is part of the gzip package is a binary file, not a shell script. the binary zcat is functionally identical to `gunzip -c`. On systems that have a compress package installed, in there is already a zcat program that reads `.Z` files, so the gzip edition of `zcat` is installed as `gzcat` (which you noted below you don't have. can you modify your script to use `ls -t1 ssl-access*.gz | xargs -n 1 gunzip -c` ? – Tim Kennedy Oct 22 '11 at 04:52
  • Ugh, iPhone keyboard and I cannot click on edit. Could you post the output of `ls -alF ssl-access*.gz`? Maybe they are symlinks pointing to missing files. – janmoesen Oct 22 '11 at 07:12
  • Try adding `--verbose` to the `xargs` and `zcat` invocations. It should at least tell you which program is adding the extra `.gz`. – James Sneeringer Oct 23 '11 at 00:53
  • I was just about to post the output to all the commands you requested when it hit me. In my shell configuration I'm aliasing the `ls` command with `ls -C --color=always --indicator-style=slash --group-directories-first`. The `--color=always` seems to screw everything up when the output of `ls` is reused. I use this option to view colored output even when piping into `less` (got it from [this answer](http://unix.stackexchange.com/questions/7164/scrolling-through-ls-output-without-a-mouse/7178#7178)) What is the correct way of handling this question? – Christoph Wurm Oct 23 '11 at 08:07

2 Answers2

4

I found this problem on my system caused by color ls. In my .bash_profile, I had this:

alias ls="ls --color"

I found the result by sending it to stat, which printed something handy:

$ ls local4.notice-201207* | xargs -n1 -P4 -I{} stat {}
stat: cannot stat `\033[0mlocal4.notice-20120711.gz\033[0m': No such file or directory

Look at those null color codes! It was confusing zcat, which attempted to add a .gz suffix to find a file. The problem was easily solved by changing the ls to color=auto, which disables color output when STDOUT is glued to another process instead of a terminal

alias ls="ls --color=auto"

Good luck!

Aaron Kondziela
  • 56
  • 2
1

You might be using an OS where zcat is for compressed files while gzcat is for gzipped ones. In such case, this should work:

   ls -t ssl-access*.gz | xargs -n 1 gzcat

Edit:

I reproduced a similar but not identical behavior on Solaris:

$ ls -t1 *.gz | xargs -n 1 zcat          
c.gz.Z: No such file or directory
b.gz.Z: No such file or directory
a.gz.Z: No such file or directory

On a Debian based distribution (Ubuntu), I do not reproduce your problem.

This might work anyway:

cat $(ls -t ssl-access*.gz) | zcat

or the equivalent:

zcat <(cat $(ls -t ssl-access*.gz))

or this simpler one that avoids an unnecessary cat

zcat $(ls -t ssl-access*.gz)

None of these will handle odd filenames like the ones with embedded spaces.

jlliagre
  • 60,319
  • 10
  • 115
  • 157