How to Add a UTF-8 BOM in Java

1. Introduction

In text encoding, the Byte Order Mark (BOM) is a special marker at the beginning of a file that indicates its byte order and encoding scheme. For UTF-8 encoding, the BOM is a sequence of three bytes: 0xEF, 0xBB, and 0xBF. Additionally, these bytes serve as a signal to software that the file is encoded using UTF-8.

In this tutorial, we’ll explore different methods to add a UTF-8 BOM to a file in Java, examining both byte-level and text-level approaches and ensuring consistency in handling and explaining the BOM.

2. Understanding the UTF-8 BOM

The UTF-8 BOM indicates that a file is encoded in UTF-8 through a special sequence of bytes. Although it isn’t mandatory, including the BOM can be crucial in certain situations, especially when working with older software or specific platforms that rely on it to detect the encoding format.

As we mentioned above, the UTF-8 BOM consists of three bytes in hexadecimal: 0xEF, 0xBB, and 0xBF.

Additionally, the Unicode character \uFEFF, known as the Zero-Width No-Break Space (ZWNBSP), also represents this sequence. This Unicode character signals the presence of the BOM and serves the same function as the byte sequence.

To ensure consistency in our code, we’ll define both the byte sequence and the Unicode representation as constants throughout this tutorial:

private static final byte[] UTF8_BOM = {(byte) 0xEF, (byte) 0xBB, (byte) 0xBF};
private static final String UTF8_BOM_UNICODE = "\uFEFF";

Throughout this tutorial, we’ll add the BOM to files using either raw bytes or with the Zero-Width No-Break Space character depending on whether we’re working with bytes or strings.

3. Using FileOutputStream and the write Method

One of the simplest methods to add a BOM to a file is to use Java’s FileOutputStream, which allows us to write raw bytes directly. This approach provides control over the exact byte sequence in the file, making it suitable for low-level, byte-oriented file operations.

First, let’s manually write the BOM bytes at the beginning of the file as 0xEF, 0xBB, and 0xBF, followed by the UTF-8 encoded content:

private static final String FILE_PATH_OUTPUT_STREAM = "output_with_bom.txt";
private static final String TEST_CONTENT = "This is the content of the file";

@Test
public void givenText_whenAddingBomWithFileOutputStream_thenBOMAdded() throws IOException {
    try (FileOutputStream fos = new FileOutputStream(FILE_PATH_OUTPUT_STREAM)) {
        fos.write(UTF8_BOM);
        fos.write(TEST_CONTENT.getBytes(StandardCharsets.UTF_8));
    }

    String result = Files.readString(Path.of(FILE_PATH_OUTPUT_STREAM), StandardCharsets.UTF_8);
    assertTrue(result.startsWith(UTF8_BOM_UNICODE));
    assertTrue(result.contains(TEST_CONTENT));
}

We first define the file path, content, and the byte array representing the UTF-8 BOM. Then, we open the file using FileOutputStream inside a try-with-resources block, ensuring the stream automatically closes after we finish with it.

Next, we write() the BOM bytes to the file, followed by the UTF-8 encoded content.

Finally, we read the file back using Files.readString() and ensure that the file starts with the BOM and contains the expected file content.

Note that this approach operates at the byte level. Reading the content back automatically converts the BOM bytes to its Unicode equivalent.

4. Writing UTF-8 with BOM Using Java Writers

When writing UTF-8 files with a BOM in Java, we can leverage writers that wrap output streams. BufferedWriter and PrintWriter allow us to add the BOM as we write the file content. These approaches handle encoding and provide higher-level abstractions for easier file output.

4.1. Using BufferedWriter and OutputStreamWriter

Using BufferedWriter with OutputStreamWriter offers a high-level approach for managing the BOM in UTF-8 files:

private static final String FILE_PATH_BUFFERED_WRITER = "output_with_bom_buffered.txt";

@Test
public void givenText_whenAddingBomWithBufferedWriter_thenBOMAdded() throws IOException {
    try (OutputStreamWriter osw = new OutputStreamWriter(
            new FileOutputStream(FILE_PATH_BUFFERED_WRITER), StandardCharsets.UTF_8);
         BufferedWriter writer = new BufferedWriter(osw)) {

        writer.write(UTF8_BOM_UNICODE);
        writer.write(TEST_CONTENT);
    }

    String result = Files.readString(Path.of(FILE_PATH_BUFFERED_WRITER), StandardCharsets.UTF_8);
    assertTrue(result.startsWith(UTF8_BOM_UNICODE));
    assertTrue(result.contains(TEST_CONTENT));
}

In this method, we open the file with a FileOutputStream and create a channel to write to it with an OutputStreamWriter, which also lets us specify the UTF-8 encoding.

Then, we wrap the stream in a BufferedWriter, which allows us to write to the file in a controlled manner. We write the BOM using the Unicode escape sequence UTF8_BOM_UNICODE at the start of the file, followed by the actual content.

Finally, we read the content back to verify that the BOM is at the start of the file followed by our contents.

This method is preferable for cases where text files and higher-level encoding management are the priority.

4.2. Using PrintWriter with OutputStreamWriter

Another option involves using PrintWriter with OutputStreamWriter. This approach offers a convenient text output format, especially for structured text:

private static final String FILE_PATH_PRINT_WRITER = "output_with_bom_print_writer.txt";

@Test
public void givenText_whenUsingPrintWriter_thenBOMAdded() throws IOException {
    try (PrintWriter writer = new PrintWriter(
            new OutputStreamWriter(
              new FileOutputStream(FILE_PATH_PRINT_WRITER), StandardCharsets.UTF_8))) {
        writer.write(UTF8_BOM_UNICODE);
        writer.println(TEST_CONTENT);
    }

    String result = Files.readString(Path.of(FILE_PATH_PRINT_WRITER), StandardCharsets.UTF_8);
    assertTrue(result.startsWith(UTF8_BOM_UNICODE));
    assertTrue(result.contains(TEST_CONTENT));
}

Here, the OutputStreamWriter specifies the UTF-8 encoding again while the PrintWriter provides a convenient method for writing structured text. We use write() to manually add the BOM using UTF8_BOM_UNICODE, followed by the println() method for the content.

5. Using Apache Commons IO

Apache Commons IO simplifies file handling, and we can leverage its utility methods to handle writing content with a BOM. While we still need to add the BOM manually, the library’s utility methods simplify writing and reading files:

private static final String FILE_PATH_COMMONS_IO = "output_with_bom_commons_io.txt";

@Test
public void givenText_whenUsingCommonsIO_thenBOMAdded() throws IOException {
    byte[] bomAndContent = ArrayUtils.addAll(
      UTF8_BOM,
      TEST_CONTENT.getBytes(StandardCharsets.UTF_8)
    );
    FileUtils.writeByteArrayToFile(new File(FILE_PATH_COMMONS_IO), bomAndContent);

    String result = FileUtils.readFileToString(
      new File(FILE_PATH_COMMONS_IO), StandardCharsets.UTF_8
    );
    assertTrue(result.startsWith(UTF8_BOM_UNICODE));
    assertTrue(result.contains(TEST_CONTENT));
}

We combine the BOM bytes with the content bytes in an array using ArrayUtils.addAll() from Apache Commons Lang. Then, we use FileUtils.writeByteArrayToFile() from Apache Commons IO to write the BOM and content in one step.

FileUtils.readFileToString() reads the entire file into a string, letting us verify the BOM and content. Note that we add the BOM as raw bytes, but it’s interpreted as the Unicode character when read back.

This approach is particularly effective for scenarios where Apache Commons libraries are already in use, as they provide efficient methods for file I/O while simplifying BOM management.

6. Conclusion

In this article, we’ve explored various methods for adding a UTF-8 Byte Order Mark (BOM) to a file in Java.

We started with the basic approach, using FileOutputStream to write the BOM bytes. Then we combined OutputStreamWriter with BufferedWriter or PrintWriter to manage the BOM.

Finally, we used third-party libraries like Apache Commons IO for simplified file handling.