Guide to the getline Function in AWK

Last updated: March 18, 2024

Reviewed by: Liam Williams

1. Overview

The getline function in AWK is a powerful and advanced mechanism to read input from sources other than the main input file. While the default behavior of awk is to read input line-by-line, getline gives us explicit control over how to read input.

In this tutorial, we’ll explore the getline function in detail by writing AWK scripts to solve a sample scenario.

2. Scenario Setup

First of all, let’s note that we’ll use the GNU AWK implementation of AWK to execute our scripts:

$ awk --version | head -1
GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.1.0, GNU MP 6.2.1)

Now, let’s look at the mean.awk script. It contains a function mean() that calculates the integer mean of an array of integers:

$ cat mean.awk
function mean(arr) {
    total = 0
    for (i = 1; i <= length(arr); i++) {
        total += arr[i]
    }
    return int(total/length(arr))
}

The mean() function accepts an array and returns the average value of the integers in the array.

Next, we’ve also got a test plan for the mean() function in the test_cases_mean.txt file:

$ cat test_cases_mean.txt
3
3
1 2 3
2
2
7 5
6
5
3 2 1 6 8
4

The first line denotes the total number of test cases. Then, each test case comprises three lines containing the count of numbers, space-separated numbers, and the expected mean value, respectively.

Lastly, let’s note that our goal is to write a test script to read and execute the test plan. So, we’ll reuse the mean.awk and test_cases_mean.txt files.

3. Reading the Next Record

We can use the getline function to read the next record from the main input file explicitly. Let’s learn more about this behavior in this section.

3.1. Writing the Test Script

Firstly, let’s write the @include statement in test_script_v1.awk to reuse the mean() function defined in the mean.awk script:

@include "mean.awk"

Now, let’s write the BEGIN block and call getline without any arguments:

BEGIN {
    getline
    printf("Count of test cases: %d\n", $0)
}

When we don’t specify any arguments, getline reads the next record from the main input file. Further, it also sets it as the current record ($0).

Next, let’s start writing the main block in our test_script_v1.awk script:

N=$0;
getline;

Since we’ve already read the first line in the BEGIN block, the main block only gets input records from the second record onwards. So, we’ve initialized N with the first line of the first test case that contains the count of numbers. Additionally, we call getline to read the space-separated list of numbers into the current record ($0).

Moving on, we can split the input record ($0) to populate the nums array and pass it to the mean() function:

split($0, nums, " ")
actual=mean(nums)

Next, let’s call getline to read the next line containing the expected mean value and capture the value in the expected variable:

getline
expected=$0

Lastly, let’s compare the actual and expected values to determine if the mean function works correctly for this test case:

test_status=(actual==expected ? "pass": "fail")
print(test_status)

That’s it! Our script is ready for use.

3.2. Test Script in Action

Before executing our test script script, let’s look at the test_script_v1.awk script in its entirety:

$ cat test_script_v1.awk
@include "mean.awk"
BEGIN {
    getline
    printf("Count of test cases: %d\n", $0)
}
{
    N=$0
    getline
    split($0,nums," ")
    actual=mean(nums)
    getline
    expected=$0
    test_status=(actual==expected ? "pass": "fail")
    print(test_status)
}

It’s important to note that the main block executes repeatedly until the end of the input file.

Now, we can run our test script to validate the test cases:

$ awk -f test_script_v1.awk test_cases_mean.txt
Count of test cases: 3
pass
pass
pass

Perfect! Our test script confirms that the mean function is working as expected.

4. Reading Into Variable

We can pass a variable as an argument to the getline function to read the next record from the main input file directly into the variable.

Let’s use this approach to write the test_script_v2.awk by modifying the test_script_v1.awk script:

$ cat test_script_v2.awk
@include "mean.awk"
BEGIN {
    getline count;
    printf("Count of test cases: %d\n", count)
}
{
    N=$0
    getline nums_str
    split(nums_str, nums, " ")
    actual=mean(nums)
    getline expected
    test_status=(actual==expected ? "pass": "fail")
    print(test_status)
}

We can notice that we’ve reduced multiple statements into a single one by combining read and assign operations for the expected variable. Further, we should remember that the main block executes repeatedly until the end of input.

Like earlier, let’s run the test script to validate the test plan:

$ awk -f test_script_v2.awk test_cases_mean.txt
Count of test cases: 3
pass
pass
pass

It works as expected.

5. Reading From a File

In this section, we’ll learn how to use getline to read from a file.

5.1. Usage

By default, getline reads the next record from the main input file. However, we can use the redirection operator to read from a different source:

getline [var] < file

When we provide a variable argument (var), the content goes into var. Otherwise, it’s read into the current record ($0).

5.2. Test Script

Using this approach, let’s write the test_script_v3.awk script that accepts the TEST_PLAN_FILE parameter:

$ cat test_script_v3.awk
@include "mean.awk"
BEGIN {
    getline count < TEST_PLAN_FILE
    printf("Count of test cases: %d\n", count)
    while (count-->0) {
        getline _ < TEST_PLAN_FILE # skip record
	getline num_str < TEST_PLAN_FILE
	split(num_str, nums, " ")
	actual=mean(nums)
	getline expected < TEST_PLAN_FILE
	test_status=(actual==expected ? "pass" : "fail")
	print(test_status)
    }
    close(TEST_PLAN_FILE)
}

Now, let’s break down the nitty-gritty of the logic.

Firstly, we transferred the logic into the BEGIN block and replaced the main block with a while loop. Further, we used getline function to read from the TEST_PLAN_FILE file.

As a best practice, we should always close the file after use. Additionally, let’s note that it’s a common practice to use _ variable when we want to read a record without the intention to use it.

Like always, let’s verify the test script by setting the TEST_PLAN_FILE parameter as test_cases_mean.txt:

$ awk -v TEST_PLAN_FILE="test_cases_mean.txt" -f test_script_v3.awk
Count of test cases: 3
pass
pass
pass

Great! We got this one right.

6. Reading From a Pipe

In this section, we’ll learn how to use getline to read from a pipe by solving a use case of creating a test plan generator script.

6.1. Usage

We can pipe the output of a shell command to getline and read its content:

cmd | getline [var]

On passing a variable (var) argument, getline puts the content into the variable. Otherwise, the content goes into the current record ($0).

6.2. Test Script With Remote Test Plan

Using curl, we can fetch a test plan from a remote location. For simplicity, let’s see how to use a local URL to retrieve the test plan:

$ curl --silent file://$(pwd)/test_cases_mean.txt
3
3
1 2 3
2
2
7 5
6
5
3 2 1 6 8
4

We can replace the URL with a remote server URL that hosts our test plan.

Now, let’s look at the test_script_v4.awk script that accepts the TEST_PLAN_URL parameter to get a remote test plan and validates it:

$ cat test_script_v4.sh
@include "mean.awk"
BEGIN {
    remote_test_plan="curl --silent "TEST_PLAN_URL
    remote_test_plan | getline count
    printf("Count of test cases: %d\n", count)
    while (count-- > 0) {
        remote_test_plan | getline _  # skip record
        remote_test_plan | getline num_str
        split(num_str, nums, " ")
        actual=mean(nums)
        remote_test_plan | getline expected
        test_status=(actual==expected ? "pass" : "fail")
        print(test_status)
    }
    close(remote_test_plan)
}

We defined the remote_test_plan variable with the command string to fetch the remote test plan. Further, we used getline to execute the remote_test_plan command and read its content from the pipe.

Lastly, let’s see the test_script_v4.sh script in action by passing the URL of the locally hosted test plan:

$ awk -v TEST_PLAN_URL="file://$(pwd)/test_cases_mean.txt" -f test_script_v4.sh
Count of test cases: 3
pass
pass
pass

Fantastic! It looks like we’ve nailed this one.

7. Conclusion

In this article, we learned about the getline function in AWK. Furthermore, we learned about the split() and close() functions in AWK while solving the use case of writing a test script for validating a test plan.

Lastly, we explored multiple usage patterns of getline, namely, reading from the main input file, a secondary file, and the output of an external curl command via pipe expressions.

Comments are open for 30 days after publishing a post. For any issues past this date, use the Contact form on the site.