The Pipe Buffer Capacity in Linux

1. Overview

A pipe allows the transfer of standard output to some other destination. It’s used in Linux to send the output of one process to another process. If the command produces output that fills the pipe’s buffer capacity, the pipe function won’t work. In this tutorial, we’ll discuss this along with what is pipe buffer capacity and methods to find it in Linux.

2. What Is Pipe Buffer Capacity?

A pipe has a limited capacity. If it is full, then a write(2) will block or fail. Applications should not rely on a particular capacity as different implementations have different limits for the pipe capacity. An application in the design phase itself should take care that a reading process consumes data as soon as it’s available. This is just to make sure that the writing process does not remain blocked.

2.1. Pipe or FIFO: Major Characteristics

An attempt to write to a pipe or FIFO has two major characteristics: atomicity and blocking.

A write is atomic if the whole amount written in one operation is not interleaved with data from any other process. This happens when multiple writers send data to a single reader. To be performed atomically, applications must know how large the expected write request is. This maximum is called PIPE_BUF. The IEEE Std 1003.1-2001 requires that writes of PIPE_BUF or fewer bytes shall be atomic.

Blocking is only possible with O_NONBLOCK clear — no subsequent I/O operations can cause the calling process to wait. If there’s enough space available, the application should write all the data requested to be written immediately. Otherwise, the calling thread may block — it’ll pause until enough space is available for writing.

2.2. PIPE_BUF

We learned that as the output data is written to the pipe as a contiguous sequence, the writes of less than PIPE_BUF bytes are atomic. As the kernel may interleave the data with data written by other processes, these writes of more than PIPE_BUF bytes become nonatomic.

POSIX.1 requires PIPE_BUF to be at least 512 bytes. On Linux, PIPE_BUF is 4,096 bytes. The precise semantics depend on whether the file descriptor is nonblocking (O_NONBLOCK), whether there are multiple writers to the pipe, and on n, the number of bytes to be written:

3. Methods to Find Pipe Buffer Capacity

In Linux, pipe buffer capacity in most cases is the same as the system page size. For example, in Linux 2.6.11, the pipe capacity is 16 pages (65,536 bytes in a system with a page size of 4,096 bytes). Since Linux 2.6.35, the default pipe capacity is 16 pages, but the capacity can be queried and set using the fcntl(2) F_GETPIPE_SZ and F_SETPIPE_SZ operations.

3.1. Bash/Perl Combination

Let’s write a little bash/perl combination that we’ll use to test the pipe capacity on Linux:

#!/bin/bash 
test $# -ge 1 || { echo "usage: $0 write-size [wait-time]"; exit 1; } 
test $# -ge 2 || set -- "$@" 1 
bytes_written=$( 
{ 
  exec 3>&1 
   { 
perl -e ' 
$size = $ARGV[0]; 
$block = q(a) x $size; 
$num_written = 0; 
sub report { print STDERR $num_written * $size, qq(\n); } 
report; while (defined syswrite STDOUT, $block) { 
$num_written++; report; 
}
' "$1" 2>&3 
   } | (sleep "$2"; exec 0<&-); } | tail -1 ) 
printf "write size: %10d; bytes successfully written before error: %d\n" \ "$1" "$bytes_written"

$ /bin/bash -c 'for p in {0..18}; do /tmp/ts.sh $((2 ** $p)) 0.5; done'

write size:          1; bytes successfully written before error: 65536
write size:          2; bytes successfully written before error: 65536
write size:          4; bytes successfully written before error: 65536
write size:          8; bytes successfully written before error: 65536
write size:         16; bytes successfully written before error: 65536
write size:         32; bytes successfully written before error: 65536
write size:         64; bytes successfully written before error: 65536
write size:        128; bytes successfully written before error: 65536
write size:        256; bytes successfully written before error: 65536
write size:        512; bytes successfully written before error: 65536
write size:       1024; bytes successfully written before error: 65536
write size:       2048; bytes successfully written before error: 65536
write size:       4096; bytes successfully written before error: 65536
write size:       8192; bytes successfully written before error: 65536
write size:      16384; bytes successfully written before error: 65536
write size:      32768; bytes successfully written before error: 65536
write size:      65536; bytes successfully written before error: 65536
write size:     131072; bytes successfully written before error: 0
write size:     262144; bytes successfully written before error: 0

This example errors at the point when the write size becomes greater than 65536 (64 KiB), at which it blocks.

3.2. /proc Files

The value in /proc/sys/fs/pipe-max-pages sets an upper limit, in pages, on the capacity for a pipe. It can be set even by an unprivileged user — for example, one without the CAP_SYS_RESOURCE capability. The default value for this limit is 16 times the default pipe capacity, the lower limit of which is two pages.

The value in /proc/sys/fs/pipe-user-pages-hard sets the hard limit on the total size (in pages) of all pipes. It can be created or set by a single unprivileged user — for example, one with neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capability.

The value in /proc/sys/fs/pipe-user-pages-soft sets the soft limit on the total size (in pages) of all pipes. It can be created or set by a single unprivileged user — for example, one with neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capability.

4. The Pipe Buffer Capacity on Various Linux Systems

Let’s see some alternatives to explore the actual pipe buffer capacity using shell commands on Linux Systems. We’ll first be exploring those shell commands and then will be checking them for Solaris and Fedora Linux systems. Let’s first obtain the pipe buffer size using bash:

$ yes produce_this_string_as_output | tee >(sleep 1) | wc -c

73728

Now, let’s examine a portable version of obtaining the pipe buffer size using bash: 

$ ( (sleep 1; exec yes produce_this_string_as_output) & echo $! ) | (pid=$(head -1); sleep 2; kill "$pid"; wc -c </dev/stdin)

65520

4.1. Solaris 10

On Solaris 10, ulimit prints 512:

$ ulimit -a | grep pipe

pipe size            (512 bytes, -p) 8

And this matches the output:

$ getconf -a |grep PIPE_BUF

PIPE_BUF                      4096
_POSIX_PIPE_BUF               4096

4.2. Fedora 25

On Fedora 25, our first yes method prints 73728 instead of the 64 KiB determined:

$ dd if=/dev/zero bs=4096 status=none | pv -bn | sleep 1

65536

5. Conclusion

In this article, we discussed the concept of pipe buffer capacity in Linux.

We focused on discussing methods to find the pipe buffer capacity in the initial part of the article. However, later, we discussed utilities that, when used directly, give us the limit on the total size (in pages) of all pipes.

Finally, we explored alternatives to evaluate the actual pipe buffer capacity using shell commands for Solaris and Fedora Linux systems.