Printing Specified Lines From a File

Last updated: March 18, 2024

Reviewed by: Eric Martin

1. Overview

In our day-to-day work with Linux systems, we often encounter situations where we need to extract specific lines from a file by line numbers, for example, analyzing data or filtering logs.

In this tutorial, we’ll explore how we can efficiently achieve this task using sed and awk.

2. Introduction to the Problem

First, let’s prepare a text file as the example input:

$ cat input.txt
The Line Number 1
The Line Number 2
The Line Number 3
The Line Number 4
The Line Number 5
The Line Number 6
The Line Number 7
The Line Number 8
The Line Number 9
...
The Line Number 26
The Line Number 27
The Line Number 28
The Line Number 29
The Line Number 30

As we can see, the file input.txt has 30 lines. Each line in the file contains its line number so that we can later easily verify whether the output is expected.

Now, let’s look into the core problem. In practical scenarios, we often encounter diverse requirements for extracting lines from the input file. These requirements may include extracting specific lines such as lines x, y, and z, retrieving a range of lines from line x to line y, selecting lines at specific intervals (every x-th line), and many more.

To ensure we discuss the most frequently encountered scenarios, let’s extract lines from the input.txt file based on a combination of the following three requirements:

  • Output lines 2, 4, and 7
  • Line numbers from 10 to 20: Print if the line number is even
  • Line number > 20: Skip

Therefore, the expected output looks like the following:

The Line Number 2
The Line Number 4
The Line Number 7
The Line Number 10
The Line Number 12
The Line Number 14
The Line Number 16
The Line Number 18
The Line Number 20

Next, let’s see how to get this output using sed and awk.

3. Using the sed Command

The sed command is a convenient text-processing utility. We often see compact and effective sed commands to address text-related challenges. So, first, let’s use sed to output the required lines.

We have a combination of three requirements. We’ll start from the first and then extend the sed command to cover the second and third requirements.

3.1. Printing Lines x, y, and z

By default, the sed command processes input line by line and prints each line:

$ sed '' input.txt 
The Line Number 1
The Line Number 2
The Line Number 3
The Line Number 4
The Line Number 5
The Line Number 6
The Line Number 7
The Line Number 8
The Line Number 9
...
The Line Number 30

If we want sed only to print the specific lines, we can pass the -n option to suppress output and use the ‘p‘ command to print the lines we want:

$ sed -n '2p; 4p; 7p;' input.txt 
The Line Number 2
The Line Number 4
The Line Number 7

As the output above shows, the first requirement has been successfully met.

3.2. Output Every x-th Line

Our second task is to output even-numbered lines from line 10 to line 20. sed allows us to use ‘n~x p‘ to print from line ‘n‘  every ‘x-th‘ line until the end of the file. The ‘x’ here is the interval. For example, let’s output from the second line, every fifth line from the input.txt file:

$ sed -n '2~5p' input.txt 
The Line Number 2
The Line Number 7
The Line Number 12
The Line Number 17
The Line Number 22
The Line Number 27

Therefore, ‘10~2p‘ prints all even lines from line 10 to the end of the file:

$ sed -n '2p; 4p; 7p; 10~2p;' input.txt 
The Line Number 2
The Line Number 4
The Line Number 7
The Line Number 10
The Line Number 12
The Line Number 14
The Line Number 16
The Line Number 18
The Line Number 20
The Line Number 22
The Line Number 24
The Line Number 26
The Line Number 28
The Line Number 30

Alternatively, we can use the ‘n‘ command to overwrite the current line with the next line:

$ sed -n '2p; 4p; 7p; 9,${n;p}' input.txt 
The Line Number 2
The Line Number 4
The Line Number 7
The Line Number 10
The Line Number 12
The Line Number 14
The Line Number 16
The Line Number 18
The Line Number 20
The Line Number 22
The Line Number 24
The Line Number 26
The Line Number 28
The Line Number 30

In the command above, ‘9, ${n;p}’ means from line 9 until the end of the file ($), we apply the action {n;p}. So the workflow looks like this:

  • Read line 9
  • (n) Read the next line (10) and overwrite the current line (9)
  • (p) Print the current line (10)
  • Read line 11
  • (n) Read the next line (12) and overwrite the current line (11)
  • (p) Print the current line (12)
  • Read line 13
  • ….

3.3. Applying the Range Limit

We’ve seen the ‘n~x’ address offers a fixed line interval (x). But it doesn’t support a range. sed has the ‘q‘ command to quit further processing. For example, ‘5q‘ tells sed to quit when it reaches line 5. Our third requirement is to suppress lines if the line number exceeds 20. Therefore, we can append ‘21q‘ to the command to solve the problem:

$ sed -n '2p; 4p; 7p; 10~2p; 21q' input.txt 
The Line Number 2
The Line Number 4
The Line Number 7
The Line Number 10
The Line Number 12
The Line Number 14
The Line Number 16
The Line Number 18
The Line Number 20

For the ‘9, ${n;p}‘ approach, we can simply replace ‘$‘ with ‘20‘. This asks sed to apply the {n;p} action only between line 9 and line 20. After line 20, no line will be printed:

$ sed -n '2p; 4p; 7p; 9,20{n;p}' input.txt 
The Line Number 2
The Line Number 4
The Line Number 7
The Line Number 10
The Line Number 12
The Line Number 14
The Line Number 16
The Line Number 18
The Line Number 20

So far, we’ve solved the problem using sed. As we can see, by using sed‘s address system, we can solve most line number-based output problems. However, as sed doesn’t support variables and mathematical calculations, some dynamic requirements can be challenges for sed, for example, print lines if their line numbers correspond to values in the Fibonacci sequence.

4. Using the awk Command

awk is another powerful text-processing utility. It’s a C-like scripting language that supports variables, loops, logical checks, functions, mathematical calculations, etc. Therefore, awk is more flexible to use than sed.

As we don’t need to use special address syntax and commands in awk, we can implement the line number check logic in one shot:

$ awk 'NR == 2 || NR == 4 || NR == 7 || (NR >= 10 && NR <= 20 && NR % 2 == 0)' input.txt
The Line Number 2
The Line Number 4
The Line Number 7
The Line Number 10
The Line Number 12
The Line Number 14
The Line Number 16
The Line Number 18
The Line Number 20

As the example above shows, we’ve got the expected output. And the awk code is pretty easy to understand. It’s worth noting that NR is the index number of the current record, which is the line number by default.

Finally, to see awk‘s flexibility, let’s address the previous challenge – “Print lines if their line numbers correspond to values in the Fibonacci sequence”:

$ awk 'BEGIN{a=0;b=1} {while(NR>=a){fib[a]; c=a+b; a=b; b=c} } NR in fib' input.txt 
The Line Number 1
The Line Number 2
The Line Number 3
The Line Number 5
The Line Number 8
The Line Number 13
The Line Number 21

The Fibonacci numbers’ implementation is straightforward. Additionally, since we don’t know how many lines the input file may have and don’t want to generate numbers we don’t need, we added a check in a while loop to only generate Fibonacci numbers if the last Fibonacci number is less than the current line number.

Through this example, we may realize that if the requirement is complex, awk is probably a better choice than sed.

5. Conclusion

In this article, we’ve explored using sed and awk to print specified lines from a file based on line numbers.

sed can solve most line-number-based problems using its address system. However, awk can be a better choice if the requirement is dynamic and complex.

Comments are open for 30 days after publishing a post. For any issues past this date, use the Contact form on the site.