How to Capture Stdout Line by Line in Bash

1. Introduction

In Bash, we often run commands with a long execution time and continuous output. At times, we’d like to capture the output of a long-running command line-by-line and then do something with each line. In this tutorial, we’ll see how to accomplish that.

2. The Long Running Command

For this tutorial, we’ll be using the ping command as the long-running command for the sake of demonstration. The ping command checks if a remote computer is reachable at regular intervals and prints the output continuously.

$ ping -c 10 baeldung.com
PING baeldung.com (172.66.43.8) 56(84) bytes of data.
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=1 ttl=57 time=23.4 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=2 ttl=57 time=29.8 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=3 ttl=57 time=42.6 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=4 ttl=57 time=24.3 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=5 ttl=57 time=26.2 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=6 ttl=57 time=56.8 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=7 ttl=57 time=27.3 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=8 ttl=57 time=36.2 ms
64 bytes from 172.66.43.8 (172.66.43.8): icmp_seq=9 ttl=57 time=33.0 ms

--- baeldung.com ping statistics ---
10 packets transmitted, 9 received, 10% packet loss, time 9012ms
rtt min/avg/max/mdev = 23.446/33.294/56.810/10.132 ms

We used the -c argument to ping baeldung.com 10 times, or the command will keep pinging indefinitely.

3. Basic Setup

The most basic way to capture the output of a command line-by-line is as follows:

$ ping -c 10 baeldung.com |
while IFS= read -r line
do
  echo $(date +%H:%m:%S) $line
done

In the above snippet, we piped the output into a while loop and used a read command on stdout, which essentially reads the output byte-by-byte and splits it into chunks based on the IFS variable we’ve set. Then, it stores the chunk in the line variable, which we process inside the loop. For the sake of demonstration, we are reprinting the line with a timestamp at the start of the line. We’ll see the following output when we run the above snippet:

11:09:34 PING baeldung.com (172.66.40.248) 56(84) bytes of data.
11:09:34 64 bytes from 172.66.40.248 (172.66.40.248): icmp_seq=1 ttl=57 time=20.2 ms
11:09:35 64 bytes from 172.66.40.248 (172.66.40.248): icmp_seq=2 ttl=57 time=21.3 ms
...
11:09:43 64 bytes from 172.66.40.248 (172.66.40.248): icmp_seq=10 ttl=57 time=17.7 ms
11:09:43
11:09:43 --- baeldung.com ping statistics ---
11:09:43 10 packets transmitted, 10 received, 0% packet loss, time 9003ms
11:09:43 rtt min/avg/max/mdev = 17.747/26.898/46.550/9.512 ms

We observe that our defined timestamp precedes every line in the output.

4. Accessing Variables Outside the Loop

One problem with the above approach is that any variables we set inside the loop are removed from memory when the loop is completed. This happens because the loop runs in a separate subshell, and that shell finishes when the loop is done. To define and access variables from outside the loop, say a counter that counts the number of lines, we need to use parentheses:

$ ping -c 10 baeldung.com | {
  i=0
  while IFS= read -r line
  do
    i=$((i+1))
    echo $i $line
  done
  echo "TOTAL LINES: $i"
}

This will give us the following output:

1 PING baeldung.com (172.66.40.248) 56(84) bytes of data.
2 64 bytes from 172.66.40.248 (172.66.40.248): icmp_seq=1 ttl=57 time=27.3 ms
3 64 bytes from 172.66.40.248 (172.66.40.248): icmp_seq=2 ttl=57 time=18.9 ms
...
9 64 bytes from 172.66.40.248 (172.66.40.248): icmp_seq=8 ttl=57 time=20.4 ms
10
11 --- baeldung.com ping statistics ---
12 10 packets transmitted, 8 received, 20% packet loss, time 9032ms
13 rtt min/avg/max/mdev = 18.872/25.420/49.099/9.629 ms
TOTAL LINES: 13

We see that we are able to define and access the variable i outside as well as inside the loop.

5. Conclusion

In this article, we looked at how we can capture and process the output of a long-running command line-by-line. We can use these methods in various scenarios where we have to log, transform, or send the output of commands to another service.