GNU sed vs. BSD sed

1. Overview

sed (Stream Editor) is a tool that lets us transform and process text content. In the Linux userland, it’s one of the go-to tools for power users to manipulate text data quickly. However, some sed variants behave differently under certain use cases.

In this tutorial, we’ll discuss the differences between the two most used sed variants: GNU sed and BSD sed. We’ll learn the essential differences between the two with hands-on examples.

2. GNU sed vs. BSD sed

BSD sed is used on UNIX-based operating systems like FreeBSD, OpenBSD, NetBSD, and macOS. In addition, the macOS sed is derived from the FreeBSD sed.

In this section, we go through the differences between the GNU sed and BSD sed flavors. Most of the differences should also apply to other BSD variants of sed as well.

2.1. Extended Regular Expression (ERE)

ERE extends the Basic Regular Expression (BRE) by allowing additional pattern-matching abilities. In ERE, we don’t need to escape certain characters which carry out special operations.

In BSD sed, we enable the ERE option with -E:

$ sed -E 's/(apple|banana)/REMOVED/g' fruits.txt
REMOVED
REMOVED
orange
mango
tomato
guava

Before GNU sed 4.2, we enable the same option using the -r option:

$ gsed -r 's/(apple|banana)/REMOVED/g' fruits.txt
REMOVED
REMOVED
orange
mango
tomato
guava

However, in GNU sed 4.2+, we can use -E as well, which is an alias for -r. In the same way, in the newer versions of BSD sed (except OpenBSD sed), -r is an alias for -E.

2.2. The -i Option

-i lets us make modifications in place in a file. For instance, in GNU sed, we simply specify -i to replace text without having to create a backup file:

$ gsed -i 's/^apple$/avocado/' fruits.txt && cat fruits.txt 
avocado
banana
orange
mango
tomato
guava

Conversely, -i in BSD sed always expects an argument. The argument is a backup file extension for the specified input file:

$ /usr/bin/sed -i '.bak' 's/^avocado$/apple/' fruits.txt && ls -lH
total 16
-rw-r--r--@  1 hey  staff   39 Jan 21 17:40 fruits.txt
-rw-r--r--@  1 hey  staff   39 Jan 21 17:39 fruits.txt.bak

In this case, “.bak” is appended to the backup filename. However, if we don’t need to create the backup file, we simply supply an empty string:

$ sed -i '' 's/^avocado$/apple/' fruits.txt

In addition, if we want to achieve cross-compatibility, we need to supply -e alongside -i:

$ sed -i -e 's/^banana$/lime/' fruits.txt

The -e option specifies the expression. It should work the same in both GNU sed and BSD sed except FreeBSD sed. In FreeBSD sed, -i still expects an argument even if we pass it the -e option. So, once we run the command with FreeBSD sed, it creates a backup file with -e appended to its filename.

2.3. Escaping Sequences

GNU sed can interpret and process escape sequences like \t, \s, \n, \w, and so on. These sequences have special meanings. For instance, \t specifies the tab character:

$ echo "apple&lime" | gsed 's/&/\t/'
apple	lime

On the other hand, BSD sed has limited support for interpreting escape sequences:

$ echo "apple&lime" | sed 's/&/\t/'
apple\tlime

It only interprets \n (newline character) in the pattern and replacement parts of the expression. macOS sed is an exception in this case.

2.4. Newlines Between Expressions

In GNU sed, we can separate the expressions with a newline when surrounded by braces:

$ echo "apple" | gsed '{ s/apple/lemon/             
s/lemon/lime/ }'
lime

In contrast, it’s not permissive in BSD sed:

$ echo "apple" | sed '{ s/apple/lemon/
s/lemon/lime/ }'
sed: 2: "{ s/apple/lemon/
s/Hi/Hey/ }": bad flag in substitute command: '}'

2.5. The i, a, and c Commands

The i, a, and c commands specify how to add text:

• i inserts text before a specified line
• a appends text after the specified line
• c changes the entire line(s) matching a pattern with new text

In BSD sed, we need to add a backslash and a newline after the i, a, and c commands:

$ echo "apple" | sed '1i\
lime'
lime
apple

Alternatively, we can achieve the same result in a single-line expression:

$ echo "apple" | sed $'1i\\\nlime\n'
lime
apple

In GNU sed, we don’t need anything like that:

$ echo "apple" | gsed '1ilime'
lime
apple

GNU sed automatically inserts a newline after the text is inserted with the i, a, and c commands.

2.6. The I and M Modifiers

In regular expressions, modifiers change the behavior of pattern matching. The I modifier disables case sensitivity and the M modifier does multi-line mode matching.

In older macOS sed, the I and M modifiers aren’t supported. However, they’re supported by FreeBSD sed as well as GNU sed:

$ echo "Apple" | gsed -E '/apple/I s/a/A/'
Apple

Similarly, we can use the M modifier in GNU sed as well:

$ echo -e "apple\nlime" | gsed -n '/^apple$/M{N; s/\n/ /p;}'
apple lime

2.7. The -s, -u, and -z Options

macOS sed doesn’t support -s, -u, and -z options:

• -s or –separate treats input files as separate streams
• -u or –unbuffered flushes the output buffer after each line
• -z or –null-data allows the processing of null-terminated lines

While they work in GNU sed, we’ll encounter an error when we specify one of these options in BSD sed:

$ sed -s 's/file/replaced/' file1.txt file2.txt
sed: illegal option -- s

2.8. The -a Option

In BSD sed, -a makes the w command append to a file:

$ echo "plum" | sed -a 'w fruits.txt' && cat fruits.txt
apple
banana
orange
mango
tomato
guava
plum

But, it’s not supported by macOS sed. However, we can achieve the same result with the append to file redirect operator >>.

3. Additional Differences

In addition to the core differences, there are extra features that are solely implemented by GNU sed:

• Deleting lines using the + symbol
• Printing every nth line
• Quitting after the first occurrence of a pattern with Q
• Changing the case of the first letter
• Executing a shell command before processing a file
• Turning off line wrapping

While we can achieve the same result with BSD sed, it’s much more difficult compared to the built-ins provided in GNU sed. For that reason, we should use alternatives like awk or Perl.

4. Using GNU sed on macOS

Sometimes, it’s harder to keep track of the differences between the variants. Fortunately, we can install GNU sed on macOS. For that purpose, we can use Homebrew:

$ brew install gnu-sed

Once installed, we can use it by prefixing it with g:

$ gsed --version
gsed (GNU sed) 4.9
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.

5. Conclusion

In this article, we learned the core differences between the GNU sed and BSD sed. In addition to the core differences, we also briefly discussed the additional minor differences that are specific to GNU sed.

Finally, we saw how to install and use GNU sed on macOS.