1. Overview

The sed command is a versatile tool designed for stream editing and text processing tasks. Among its many capabilities, sed allows us to perform multiple substitutions on a file with a single command.

In this tutorial, we’ll explore various methods to replace different parts of a file with one sed line, enabling us to efficiently manipulate text files and automate editing tasks.

2. Sample Task

Let’s suppose we have a file named input_file, and we want to modify its contents:

$ cat input_file
   The executable is located at /usr/bin/program.
Visit http://example.com for more information.

In particular, we’d like to implement several changes to the contents of the file:

  1. remove leading spaces from each line
  2. replace occurrences of /usr/bin/ with /usr/local/bin/
  3. change http links to https

Let’s explore different approaches to implement these changes efficiently using sed.

3. Sequential Substitutions Using the -e Flag

One approach to carrying out multiple substitutions with sed involves using the -e flag. In particular, by providing multiple -e flags followed by substitution commands, each operation is sequentially applied to the file:

$ sed -e 's/^[[:space:]]*//g' -e 's:/usr/bin/:/usr/local/bin/:g' -e 's/http/https/g' input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.

Each of the substitution commands takes the form s/pattern/replacement/g where g indicates global substitution.

In the first substitution command, the pattern ^[[:space:]]* matches zero or more whitespace characters at the beginning of each line, which gets replaced with an empty string.

In the second substitution command, we use colons as delimiters (:) since the pattern and replacement both contain forward slashes. Alternatively, we could escape the forward slashes within a forward slash pattern. In any case, we replace /usr/bin/ instances with /usr/local/bin/.

Finally, in the third substitution command, we replace any http instance in the file with https.

By combining these substitution commands, we can effectively perform the desired modifications on the file using the sed utility.

4. Substitution Separators

When multiple -e statements are inconvenient, we can also turn to separators within a single statement.

4.1. Semicolon Separator

We can separate multiple substitution commands using semicolons within a single sed command:

$ sed 's/^[[:space:]]*//g; s:/usr/bin/:/usr/local/bin/:g; s/http/https/g' input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.

In this case, we apply our last solution but place a semicolon between each operation. As a result, the substitution commands execute sequentially giving the desired outcome.

4.2. Newline Separator

Alternatively, we can separate multiple substitution commands via newlines:

$ sed 's/^[[:space:]]*//g
>      s:/usr/bin/:/usr/local/bin/:g
>      s/http/https/g' input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.

Here also, the substitution commands execute sequentially as usual.

5. Script File and the -f Flag

For more complex substitution scenarios, creating a separate script file can streamline the process. For example, we place each substitution command without quotations on a new line within the script.sed script file:

$ cat script.sed
s/^[[:space:]]*//g
s:/usr/bin/:/usr/local/bin/:g
s/http/https/g

We then pass the script file to sed using the -f flag:

$ sed -f script.sed input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.

Additionally, the file for processing, input_file, is passed as an argument at the end of the sed command.

6. Script File With Shebang

Another way to use a sed script file is to incorporate a shebang directive (also known as a hashbang) into the script. This directive enables the script to execute directly with the sed executable.

So, let’s modify the script.sed file by adding a shebang on the first line:

$ cat script.sed
#!/usr/bin/sed -f
s/^[[:space:]]*//g
s:/usr/bin/:/usr/local/bin/:g
s/http/https/g

By adding a shebang header, we indicate that the script executes by running /usr/bin/sed -f script.sed followed by any additional arguments. To ensure the script can be executed directly, we need to grant execute permission to script.sed using the chmod command. Then, we can pass input_file as an argument to the script:

$ chmod u+x script.sed
$ ./script.sed input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.

As before, sed runs each of the substitution commands within the script and returns the desired output.

7. Conclusion

In this article, we’ve seen various ways to perform multiple substitutions using a single sed command. In particular, we’ve seen that we can use sequential substitutions with the -e flag, semicolon or newline separation, or sed script files. This way, we can streamline text editing tasks and automate repetitive operations when manipulating files.

Comments are closed on this article!