1

I need to pipe the output of one command to two other commands.

cat filename.txt | tail -n 1
cat filename.txt | wc -l

Since the file is huge I want to avoid reading it twice. I checked tee command but it redirects output to file which I don't want. There are many related posts but did not find anything relevant

terdon
  • 234,489
  • 66
  • 447
  • 667
chhaya vishwakarma
  • 693
  • 6
  • 12
  • 18

3 Answers3

2
{   
    {   tee /dev/fd/3 | wc -l >&4
    }   3>&1 | tail -n1
}   <filename.txt 4>&1

It doesn't avoid reading it twice - but on multicore systems it will likely be concurrent. The output order is in no way guaranteed, however.

Probably the fastest way to simulate what you're asking for, though, is:

dd bs=64k <filename.txt | tail -n1

dd will report its read/writes to stderr and still copy all of the file to stdout.

mikeserv
  • 57,448
  • 9
  • 113
  • 229
2

If I understood correctly, you want to get the last line of the file along with the line count:

$ cat a
aaa
bb
ccc
$
$ awk 'END{print $0; print NR}' a
ccc
3

Since you need them to save in variables:

$ out=$(awk 'END{print $0"|"NR}' a)
$ last_line=$(echo $out | awk -F"|" '{print $1}' )
$ tot_cnt=$(echo $out | awk -F"|" '{print $2}' )
$ echo $last_line
ccc
$ echo $tot_cnt
3
Guru
  • 5,855
  • 1
  • 19
  • 20
1

Let the cat sleep, you don't need it.

Use :

tail -n 1 filename.txt
wc -l filename.txt
heemayl
  • 54,820
  • 8
  • 124
  • 141
  • 2
    Yes, but this still reads the input file twice which is what the OP is trying to avoid. – terdon Jul 23 '15 at 10:11
  • I can't avoid cat.. – chhaya vishwakarma Jul 23 '15 at 10:12
  • 1
    @heemayl: you are perfectly right, and it is not just syntactic sugar. Measuring the performance shows that getting rid of `cat` is much faster: `tail` knows how big the file is and can be optimized to check directly at the file end before returning the last line, but it does not know how _big_ `stdin` is going to be, so it has to wait until `cat` is done before returning its answer. A similar thing applies to `wc`, which performs better against a file than against `stdin`. – Emeric Jul 23 '15 at 10:13