Removing a Trailing Newline From a File

1. Overview

Trailing newlines can end up in our files due to various reasons, such as saving the output of a buggy command that emits extra newlines. In this tutorial, we’ll learn how to remove trailing newline characters from a file.

2. Removing a Single Trailing Newline

Let’s look at the commands used to remove a single trailing newline from a file.

In our examples below, we use the wc -l command to count the number of newlines.

2.1. Using perl

First off, let’s check the contents of our sample file:

$ cat bad.txt
one newline
$ wc -l bad.txt
1 bad.txt

Here, wc -l outputs 1 as the file contains a single trailing newline.

Now, we can use perl‘s chomp function to remove the last character if it’s a newline:

$ perl -p -i -e 'chomp if eof' bad.txt
$ wc -l bad.txt
0 bad.txt

As we can see, wc -l outputs 0 after removing the trailing newline. We use the -p flag to print the output and the -i flag to write the output to the file, while -e allows us to pass the Perl code as an argument.

The if eof statement indicates that the character is the last character, i.e., at the end of the file. This method is quite efficient since we only invoke a single perl command.

2.2. Using head and tail

We can use the tail command to test whether the last character is a newline and then use the head command to remove it:

$ cat bad.txt
one newline
$ wc -l bad.txt
1 bad.txt
$ if [ "$(tail -c 1 bad.txt | wc -l)" = 1 ]; then head -c -1 bad.txt > tmp; mv tmp bad.txt; fi
$ wc -l bad.txt 
0 bad.txt

Here, the output of wc -l is similar to the previous example. The -c 1 flag gets the last character of the file. We pipe this character into wc to check whether it’s a newline. If it is, we use head -c -1 to print the whole file except the last character, thus removing the newline.

This method uses a temporary file and overwrites the original file with it. Therefore, it is less efficient than the method using perl, as we invoke two commands instead of one.

3. Removing All Trailing Newlines

Instead of just removing the last trailing newline, we can remove all trailing newlines from a file.

First, let’s look at the file contents:

$ cat bad.txt
many newlines



$ wc -l bad.txt
4 bad.txt

Here, wc -l outputs 4 as the file contains four trailing newlines.

We can use Bash’s built-in printf command for removing the trailing newlines:

$ printf "%s" "$(< bad.txt)" > tmp
$ mv tmp bad.txt 
$ wc -l bad.txt
0 bad.txt

We can see that wc -l outputs 0 after removing the trailing newlines. We load the file into memory with the $(< ) operator and pass it to printf. Bash automatically strips trailing newlines when reading the file.

4. Conclusion

In this article, we learned to use various commands for removing single as well as multiple trailing newlines from the end of a file.