How do I get a single total of lines with `wc -l`?

Question

I've added a git alias to give me the line counts of specific files in my history:

[alias]
lines = !lc() { git ls-files -z ${1} | xargs -0 wc -l; }; lc

However, wc -l is reporting multiple totals, such that if I have more than ~100k lines, it reports the total for them, then moves on. Here's an example:

<100k lines (desired output)

$ git lines \*.xslt
  46 packages/NUnit-2.5.10.11092/doc/files/Summary.xslt
 232 packages/NUnit-2.5.10.11092/samples/csharp/_UpgradeReport_Files/UpgradeReport.xslt
 278 total

>100k lines (had to pipe to grep "total")

$ git lines \*.cs | grep "total"
 123569 total
 107700 total
 134796 total
 111411 total
  44600 total

How do I get a true total from wc -l, not a series of subtotals?

Answer 1

Try this, and apologies for being obvious:

cat *.cs | wc -l

or, with git:

git ls-files -z ${1} | xargs -0 cat | wc -l

If you actually want the output to look like wc output, with both individual counts and a sum, you could use awk to add up the individual lines:

git ls-files -z ${1} | xargs -0 wc -l |
awk '/^[[:space:]]*[[:digit:]]+[[:space:]]+total$/{next}
     {total+=$1;print}
     END {print total,"total"}'

That won't be lined up as nicely as wc does it, in case that matters to you. To do that, you'd need to read the entire input and save it, computing the total, and then use the total to compute the field width before using that field width to print a formatted output of the remembered lines. Like home renovation projects, awk scripts are never ever really finished.

(Note to enthusiastic editors: the regular expression in the first awk condition is in case there is a file whose name starts with "total" and a space; otherwise, the condition could have been the much simpler $2 == "total".)

Answer 2

If you're running Linux, your wc probably comes from GNU Coreutils and has a --files0-from option to read a file (or stdin) containing an arbitrarily long list of NUL-terminated names of file to count. The GNU Coreutils wc documentation says "This is useful when the list of file names is so long that it may exceed a command line length limitation. In such cases, running wc via xargs is undesirable because it splits the list into pieces and makes wc print a total for each sublist rather than for the entire list."

So try this:

lc() { git ls-files -z ${1} | wc -l --files0-from=- ; } 

Edit: Since your wc is from the last millennium and doesn't have that option, here is a more portable solution, assuming you have awk and do not have any files named "total". It will filter the output of wc, omitting any total lines and instead summing them up and printing out the grand total at the end.

One thing I do not know is whether the git alias implementation will have problems with the $1 and $2 inside single quotes, which need to be passed unchanged to awk.

lc() {
  git ls-files -z ${1} |
  xargs -0 wc -l |
  awk 'BEGIN { total=0; } { if (NF==2 && $2 == "total") total += $1; else print; } END { print total, "total"; }' ;
}

Answer 3

The problem is xargs which is splitting the command into multiple runs, so wc is reporting the total for each time. You have a few options, you could keep things the way they are and parse the wc output:

git ls-files -z ${1} | xargs -0 wc -l | awk '/total/{k+=$1}END{print k,"total"}';

You could cat the files:

git ls-files -z ${1} | xargs -0 cat | wc -l

Or you could skip xargs altogether (adapted from here):

unset files i; while IFS= read -r -d $'\0' name; do 
 files[i++]="$name"; 
done < <(git ls-files -z ${1} ) && wc -l "${files[@]}"

That will break if your list of files is longer than ARG_MAX though.

Answer 4

j=0; for i in *.php *.js *.css; do let j+=`wc -l $i | awk {'print $1'}`; done; echo $j;