0

I am looping through a list of files, extracting the final line, and printing out columns 8, 9, and 10. I need to also print to the output the 'event number', which is essentially the total number of records being processes (NR). How do I print the event/record number in the first column, outputting to the output file, such as what I have below?

for i in `ls -d *mcp`; do
tail -1 "$i" | awk  '{ printf "%s %s %s\n", $8, $9, $10}' >> ${Pout}${output}
done
echo "Finished Looping through each file."

What I want as the output is:

1 45 60 5
2 30 67 3
3 40 12 4
.
.
.

where the '45 column represents $8, 60 represents $9, and 5 represents $10. the 1,2,3, etc. is what I need to output. I essentially need to print the line number.

geeb.24
  • 207
  • 1
  • 2
  • 7
  • When I use this for loop structure: `for i in `ls -d *mcp`; do tail -1 "$i" | awk '{ printf "%d %s %s %s\n",NR, $8, $9, $10}' >> ${Pout}${output} done` I get the following output: `1 -0.242 125.104 35.0 1 -6.308 151.717 28.1 1 13.764 144.429 130.0 1 -56.022 -27.779 109.3 1 -9.461 156.412 4.0` Instead of all ones in the first column I want 1,2,...n. Does that clear things up? – geeb.24 May 13 '15 at 19:30
  • I understand your suggestion. However, that line in my for loop was working for me, which is why I didn't change it. Yes, it may be longer than what you wrote, but I didn't change it because it did what I needed it to do. – geeb.24 May 13 '15 at 19:35
  • Using backtics and ls means you are using two unnecessary processes. See whether my proposals below work for you. – Janis May 13 '15 at 19:43
  • (1) [Don’t parse the output of `ls`](http://mywiki.wooledge.org/ParsingLs).  `for i in \`ls -d *mcp\`` isn’t just inefficient; it produces ***wrong results*** if filenames contain certain special characters.  (2) Don’t post multi-line commands or output in comments.  Clarifications to the question, to include things that you’ve tried, results that you’ve gotten, and results that you want, belong *in* the question — [edit] the question to put them there.  (3) When you do use command substitution, use `$(…)` instead of `\`…\``.  (4) If you must display a `\`` in code in a comment, type `\\``. – G-Man Says 'Reinstate Monica' May 14 '15 at 09:37

2 Answers2

1

With GNU awk (version 4.x) try this:

awk 'ENDFILE { printf "%d %s %s %s\n", ++c, $8, $9, $10}' *mcp > "${Pout}${output}"

echo "Finished Looping through each file."

With other awks and shells like bash try:

for f in *mcp
do
    awk -v c="$((++c))" 'END { printf "%d %s %s %s\n", c, $8, $9, $10}' "$f"
done > "${Pout}${output}"

echo "Finished Looping through each file."
Janis
  • 14,014
  • 3
  • 25
  • 42
1

Try this:

for i in ./*.mcp; do
    if [ -f "$i" ]; then
        tail -1 "$i"
    fi
done | awk '{ print NR, $8, $9, $10 }'
lcd047
  • 7,160
  • 1
  • 22
  • 33
  • This solution works! Is it correct that the if statement checks if the file exists, then it performs the 'tail' command? Thank you. – geeb.24 May 13 '15 at 19:46
  • @user78872: Yes, exactly. On a side note: you can also "add line numbers" to a file with `cat(1)`, like this: `cat -n file`. – lcd047 May 13 '15 at 19:49
  • Can you identify any situation in which `for i in *mcp` fails (i.e., any reason why one should use `./` in that context)? – G-Man Says 'Reinstate Monica' May 14 '15 at 09:44
  • @G-Man: _Any_ situation? Say if one of the files in named `-foo.mcp`. – lcd047 May 14 '15 at 09:49
  • Well, I believe that the "right" way to handle that concern is to say `tail -1 -- "$i"`, but I suppose that `for i in ./*mcp` is an effective way of handling it. – G-Man Says 'Reinstate Monica' May 14 '15 at 09:53
  • 1
    @G-Man: Well, back in the days when I worked on Apollo workstations (before Apollo was bought by HP), `tail`, like most other utilities, didn't accept `--`. But, other than old dogs vs. new tricks, there's an informative [answer](http://unix.stackexchange.com/a/110756/111878) about (among other things) the difference between `--` and `./*`. – lcd047 May 14 '15 at 10:12
  • @lcd047: Interesting post.  Thanks for pointing it out to me. – G-Man Says 'Reinstate Monica' May 14 '15 at 18:54