72

How can I skip the first 6 lines/rows in a text file (input.txt) and process the rest with awk? The format of my awk script (program.awk) is:

BEGIN {
} 

{ 
process here
} 

END {

}

My text file is like this:

0
3
5
0.1 4.3
2.0 1.5
1.5 3.0
0.3 3.3
1.5 2.1
.
.
.

I want to process the file starting from:

0.3 3.3
1.5 2.1
.
.
.
amatek
  • 833
  • 1
  • 6
  • 7

5 Answers5

101

Use either of the two patterns:

NR>6 { this_code_is_active }

or this:

NR<=6 { next }
{ this_code_is_active }

Use FNR instead of NR if you have many files as arguments to awk and want to skip 6 lines in every file.

Janis
  • 14,014
  • 3
  • 25
  • 42
51

Try:

awk 'FNR > 6 { #process here }' file
cuonglm
  • 150,973
  • 38
  • 327
  • 406
  • 15
    Good! But you didn't explain why this is better - for multiple files `FNR` is the row number in each file, while `NR` is the number in the whole input (not an issue when piping). – Tomasz Gandor Aug 08 '17 at 05:46
9

You may also skip an arbitrary number of lines at the beginning or the end of the file using head or tail programs.

For your concrete question,

tail input.txt -n+7 | program.awk

will do, provided your program.awk file is executable. Otherwise, you may use

tail input.txt -n+7 | awk -f program.awk

This way, you will spare a comparison for each line and you don't need to change the logic of your AWK code.

tail will start streaming text starting at the seventh line, skipping the six first lines.

This will not be a huge deal in performance, especially if text process is simple thanks to caching. However, for long files and repeated use in cloud environment may save some cost.

Dr. Windows
  • 91
  • 1
  • 1
  • 3
    Correct, but as a rule of thumb, one should avoid piping when you can easily do it with one tool alone. Think of a huge text file piping through both commands just to remove a few lines. – Philippos Sep 01 '17 at 11:21
  • 2
    I disagree about avoiding piping. Do one thing and do it well. Adage aside, I do whatever comes to me from memory first. Scripting vs CLI too though. Small sets on a CLI, I go for least effort. Then a script may work on large datasets, or may desire to reduce the scope of future maintenance, so maybe less pipes. – Kevin Jul 27 '20 at 20:56
1

Another one:

$ cat print_from.awk

#print 20 lines from line 6 (from your file: 0.3 3.3)
BEGIN{n=0}
/^0.3 3.3.*/ {n=NR; m=n+20}
{
  if (n > 1 && NR < m) {print $1} 
}

execute with:

awk -f print_from.awk your-text-file
G-Man Says 'Reinstate Monica'
  • 22,130
  • 27
  • 68
  • 117
user388201
  • 11
  • 1
  • This is a decent solution — to a ***different*** problem. The question says (twice!) that the requirement is to skip the first 6 lines. For example, if the first seven lines are `ant`, `bat`, `cat`, `dog`, `elk`, `fox` and `giraffe`, it should start processing with `giraffe`. But that’s just an example; you have written an answer that starts processing with ``giraffe``, regardless of the line number. Also, the question says “process the rest”, not “process the next 20.” – G-Man Says 'Reinstate Monica' Dec 30 '19 at 07:39
-1
for pkg in $(pip list | awk 'FNR > 2 { print $1 }'); do pip install --upgrade $pkg; done
AdminBee
  • 21,637
  • 21
  • 47
  • 71
Fabiano
  • 1
  • 1
  • 1
    Welcome to Unix & Linux! Your answer is a bit terse and could be improved by explaining the issue, i.e., the underlying cause of the problem and why your solution correctly would resolve it for the user in the question. – Kusalananda Feb 21 '23 at 18:22