How to Edit and Patch Files Programmatically Using the Shell

1. Overview

In Linux, we sometimes need to modify files programmatically using shell commands. This can happen when there are no additional software packages installed or when we write portable scripts.

In this tutorial, we’ll focus on the main ways of changing files with mostly built-in or common tools. First, we’ll learn to update file content using the sed command. After that, we’ll look at how to create and apply patches.

2. Create a Sample File

To begin with, let’s create a sample text file that we’re going to edit.

In particular, we use a here document to create the text file quickly:

$ cat <<-EOF > test.txt
This is a sample file created in 2023 to demonstrate editing files via terminal commands. 
It will be used in 2023.
EOF

Let’s confirm that the text file was created correctly:

$ cat test.txt 
This is a sample file created in 2023 to demonstrate editing files via terminal commands. 
It will be used in 2023.

Now, we should be ready to edit the file via our shell of choice, Bash.

3. Using the sed Command

sed is a powerful stream editor tool that we can use to manipulate file contents.

Let’s cover the main ways of performing string substitution using the sed command.

3.1. Replace the First Occurrence of a String

To demonstrate how string substitution works with sed, we can replace the first occurrence of the year 2023 within a file with the current year 2024.

The syntax for this is fairly straightforward:

$ sed -zi 's/{OLD_STRING}/{NEW_STRING}/' {file}

Here, the option z separates text lines by the NULL character, and i means to edit the file in place.

The symbol s makes sed perform a substitution of an old string with a new string.

Now, we use this syntax to replace the year:

$ sed -zi 's/2023/2024/' test.txt

Let’s check that the first occurrence of 2023 is now replaced by 2024:

$ cat test.txt 
This is a sample file created in 2024 to demonstrate editing files via terminal commands. 
It will be used in 2023.

Indeed, the year is correct now.

3.2. Replace All Occurrences of a String

To replace all occurrences of a string in a file, we might need to add the g symbol at the end of the sed command:

$ sed -zi 's/2023/2024/g' test.txt

Let’s check the results now:

$ cat test.txt 
This is a sample file created in 2024 to demonstrate editing files via terminal commands. 
It will be used in 2024

We can see that the file no longer has the 2023 numbers because they’ve been replaced with the 2024.

3.3. Replace the Last Occurrence of a String

To replace the last occurrence of a string in the file, we use a different sed pattern:

$ sed -zi 's/\(.*\){OLD_STRING}/\1{NEW_STRING}/' {file}

Here, the combination of special sequences \(.*\) and \1 enables us to find and replace the last occurrence of an old string with a new one in the file.

Let’s replace the year 2024 on the last line of the file with the year 2025:

$ sed -zi 's/\(.*\)2024/\12025/' test.txt

Next, we can check the result:

$ cat test.txt 
This is a sample file created in 2024 to demonstrate editing files via terminal commands. 
It will be used in 2025.

As expected, the last line now contains the year 2025.

4. Using Patching

While sed is a powerful tool for file manipulation via the command line, it may not be handy when we need to modify files significantly or according to manual changes.

If this is the case, patching is the right tool to use. Let’s see how we create a patch and use it to modify a sample file.

4.1. Create New File

First, to patch the file, we need to create a new file which contains all modifications:

$ cat <<-EOF > new_test.txt
This is a sample file created in 2024 to demonstrate editing files via terminal commands.
It will be used in 2024 and 2025.
It has been updated using the patch.
EOF

As we can see, the file new_test.txt contains some additions to the previously used sample file.

4.2. Create Patch

Then, we create a patch that holds the modifications to test.txt based on the newly created file new_test.txt. For that, we leverage the fairly standard diff command:

$ diff test.txt new_test.txt > test.patch

Now, we should have the patch test.patch.

In addition, we can test the modifications that it can perform by looking at the patch file itself:

$ cat test.patch 
2c2,3
< It will be used in 2025.
---
> It will be used in 2024 and 2025.
> It has been updated using the patch.

The patch shows that it removes one line and replaces it with two new lines based on the content of new_test.txt.

4.3. Apply Patch

Finally, to modify the test.txt file, we apply the created patch using the patch command:

$ patch test.txt test.patch
patching file test.txt

Let’s check that the text.txt file has been modified correctly:

$ cat test.txt 
This is a sample file created in 2024 to demonstrate editing files via terminal commands.
It will be used in 2024 and 2025.
It has been updated using the patch.

Indeed, the file content is now as expected.

5. Conclusion

In this article, we learned how to modify the file using shell commands.

Initially, we looked at the sed command, where we studied how to replace the first, the last, and all occurrences of a string in the file. After that, we learned how to create and apply a patch to the text file.