1

I am using awk below to print 8th column and remove duplicates in that very column.

awk -F "," '{print $8}' filecsv | awk '!NF || !seen[$0]++'

How to do it with just one awk instead running awk twice in the above pipline

irom
  • 453
  • 2
  • 10
  • 20

1 Answers1

3
awk -F , '!seen[$8]++ { print $8 }' filecsv

This checks whether the value of the eighth field has already been seen, and only if it hasn’t, prints it.

Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164
  • 1
    note that `seen` is not an `awk` command but a variable name that can be *anything* (e.g. `!_[$8]++`). This lead to some confusion on the strange syntax when I first saw this really cool solution. – pLumo Sep 04 '18 at 15:05
  • 1
    @RoVo indeed; it’s quite a common AWK “trick”, see [this answer](https://unix.stackexchange.com/a/30178/86440) for example. – Stephen Kitt Sep 04 '18 at 15:08