Asserting Log Messages With JUnit

1. Overview

When developing Java applications, we use logging for debugging and monitoring. Capturing and validating these logs is an important part of ensuring our application functions as expected.

In this tutorial, we’ll explore how to assert log messages in our JUnit tests. We’ll cover two approaches: asserting logs in native Java tests and in Spring Boot tests.

2. Setting up Test Environment

Before asserting our log statements, we’ll need to include a few dependencies and configure our application correctly.

2.1. Dependencies

Let’s start by adding the logging dependency to our project’s pom.xml file:

<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.5.16</version>
</dependency>

We’re using SLF4J as the logging facade and Logback as its underlying implementation.

Next, to write assertions in our tests, we’ll include the assertj-core dependency:

<dependency>
    <groupId>org.assertj</groupId>
    <artifactId>assertj-core</artifactId>
    <version>3.27.2</version>
    <scope>test</scope>
</dependency>

AssertJ provides a fluent and expressive API for writing assertions in our tests.

Finally, If we’re following the Spring Boot approach for asserting logs, we’ll also need to include the spring-boot-test dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-test</artifactId>
    <version>3.4.1</version>
    <scope>test</scope>
</dependency>

This dependency provides the necessary testing classes for Spring Boot applications, including the OutputCaptureExtension which we’ll use later in the tutorial.

2.2. Logback Configuration

Next, to generate logs, we’ll create a logback.xml file in our src/test/resources folder:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <Pattern>
                %d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n
            </Pattern>
        </layout>
    </appender>

    <root level="error">
        <appender-ref ref="CONSOLE"/>
    </root>
</configuration>

We keep the logging configuration as simple as possible and redirect all logs to a CONSOLE appender using the specified pattern layout.

2.3. A Basic Business Function

Now, let’s create a class that will generate logs we can base our tests on:

class BusinessWorker {
    private static Logger LOGGER = LoggerFactory.getLogger(BusinessWorker.class);

    public void generateLogs(String msg) {
        LOGGER.trace(msg);
        LOGGER.debug(msg);
        LOGGER.info(msg);
        LOGGER.warn(msg);
        LOGGER.error(msg);
    }
}

Our BusinessWorker class exposes a single generateLogs() method which generates a log with the same content for each log level.

Although this method isn’t that useful in the real world, it’ll serve well for our testing purposes.

3. Asserting Logs in Native Java

Now that we have our test environment set up, let’s explore how to assert log messages in native Java applications.

3.1. Creating a MemoryAppender

First, let’s create a custom appender that keeps logs in memory. We’ll extend the ListAppender<ILoggingEvent> that logback offers, and we’ll enrich it with a few useful methods:

class MemoryAppender extends ListAppender<ILoggingEvent> {
    public void reset() {
        this.list.clear();
    }

    public boolean contains(String string, Level level) {
        return this.list.stream()
          .anyMatch(event -> event.toString().contains(string)
            && event.getLevel().equals(level));
    }

    public int countEventsForLogger(String loggerName) {
        return (int) this.list.stream()
          .filter(event -> event.getLoggerName().contains(loggerName))
          .count();
    }

    public List<ILoggingEvent> search(String string) {
        return this.list.stream()
          .filter(event -> event.toString().contains(string))
          .collect(Collectors.toList());
    }

    public List<ILoggingEvent> search(String string, Level level) {
        return this.list.stream()
          .filter(event -> event.toString().contains(string)
            && event.getLevel().equals(level))
          .collect(Collectors.toList());
    }

    public int getSize() {
        return this.list.size();
    }

    public List<ILoggingEvent> getLoggedEvents() {
        return Collections.unmodifiableList(this.list);
    }
}

The MemoryAppender class handles a List that is automatically populated by the logging system.

It exposes a variety of methods to cover a wide range of test purposes:

• reset() – clears the list
• contains(msg, level) – returns true only if the list contains an ILoggingEvent matching the specified content and severity level
• countEventForLoggers(loggerName) – returns the number of ILoggingEvent generated by the named logger
• search(msg) – returns a List of ILoggingEvent matching the specific content
• search(msg, level) – returns a List of ILoggingEvent matching the specified content and severity level
• getSize() – returns the number of ILoggingEvents
• getLoggedEvents() – returns an unmodifiable view of the ILoggingEvent elements

3.2. Asserting Log Statements

Next, let’s create a JUnit test for our business worker.

We’ll declare our MemoryAppender as a field and programmatically inject it into the log system. Then, we’ll start the appender.

For our tests, we’ll set the level to DEBUG:

private final MemoryAppender memoryAppender = new MemoryAppender();

@Before
void setup() {
    Logger logger = (Logger) LoggerFactory.getLogger(LOGGER_NAME);
    logger.setLevel(Level.DEBUG);
    logger.addAppender(memoryAppender);

    memoryAppender.setContext((LoggerContext) LoggerFactory.getILoggerFactory());
    memoryAppender.start();
}

Now we can create a simple test where we instantiate our BusinessWorker class and call the generateLogs method. We can then make assertions on the logs that it generates:

BusinessWorker worker = new BusinessWorker();
worker.generateLogs(MSG);
assertThat(memoryAppender.countEventsForLogger(LOGGER_NAME)).isEqualTo(4);
assertThat(memoryAppender.search(MSG, Level.INFO)).hasSize(1);
assertThat(memoryAppender.contains(MSG, Level.TRACE)).isFalse();

This test uses three features of the MemoryAppender:

• Four logs have been generated — one entry per severity should be present, with the trace level filtered
• Only one log entry with the content message with the level severity of INFO
• No log entry is present with content message and severity TRACE

If we plan to use the same instance of this class inside the same test class when generating a lot of logs, the memory usage will creep up. We can invoke the MemoryAppender.clear() method before each test to free memory and avoid OutOfMemoryException.

In this example, we’ve reduced the scope of the retained logs to the LOGGER_NAME package, which we defined as “com.baeldung.junit.log“. We could potentially retain all logs with LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME), but we should avoid this whenever possible as it can consume a lot of memory.

3.3. Handling Multiple Logs with Varying Levels

To validate application logs effectively in scenarios where multiple entries are generated at various levels (INFO, WARN, ERROR, and more), we can use methods such as counting, filtering, and asserting specific content:

BusinessWorker worker = new BusinessWorker();
worker.generateLogs("Transaction started for Order ID: 1001");

assertThat(memoryAppender.countEventsForLogger(LOGGER_NAME)).isEqualTo(4);
assertThat(memoryAppender.search("Transaction started", Level.DEBUG)).hasSize(1);
assertThat(memoryAppender.search("Transaction started", Level.INFO)).hasSize(1);
assertThat(memoryAppender.search("Transaction started", Level.WARN)).hasSize(1);
assertThat(memoryAppender.search("Transaction started", Level.ERROR)).hasSize(1);
assertThat(memoryAppender.search("Transaction started", Level.TRACE)).isEmpty();

In the @Before setup method, the logger’s level is programmatically set to DEBUG. A logger set to DEBUG will log everything from DEBUG to higher levels (INFO, WARN, ERROR), but it will exclude TRACE logs because TRACE has a lower priority than DEBUG.

3.4. Using Pattern Matching for Dynamic Log Content

When log messages contain dynamic data like timestamps, user IDs, or transaction IDs, we can use regular expressions to validate their content. This ensures that the structure and key details of the log messages are correct.

Let’s create a method for pattern-based validation:

boolean containsPattern(Pattern pattern, Level level) {
    return this.list.stream()
      .filter(event -> event.getLevel().equals(level))
      .anyMatch(event -> pattern.matcher(event.getFormattedMessage()).matches());
}

This utility method provides a clean way to check if any log message matches a given pattern at a specific level.

Let’s demonstrate how to validate log messages with dynamic content using regex:

BusinessWorker worker = new BusinessWorker();
worker.generateLogs("Order processed successfully for Order ID: 12345");

Pattern orderPattern = Pattern.compile(".*Order ID: \\d{5}.*");

assertThat(memoryAppender.containsPattern(orderPattern, Level.DEBUG)).isTrue();
// ... assert presence of pattern in INFO, WARN, and ERORR log levels as well 
assertThat(memoryAppender.containsPattern(orderPattern, Level.TRACE)).isFalse();

The regex .*Order ID: \\d{5}.* ensures the message contains an order ID followed by exactly 5 digits. With this, we can ensure the test remains valid even if the dynamic part of the log (order ID) changes.

In cases where multiple patterns are expected, we can create a containPatterns() method for matching multiple patterns:

boolean containsPatterns(List<Pattern> patternList, Level level) {
    return patternList.stream()
      .allMatch(pattern -> containsPattern(pattern, level));
}

Then, we can validate all of them in a single test:

BusinessWorker worker = new BusinessWorker();
worker.generateLogs("User Login: username=user123, timestamp=2024-11-25T10:15:30");

List patterns = List.of(
  Pattern.compile(".*username=user\\w+.*"),
  Pattern.compile(".*timestamp=\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}.*")
);

assertThat(memoryAppender.containsPatterns(patterns, Level.DEBUG)).isTrue();
// ... assert presence of patterns in INFO, WARN, and ERORR log levels as well
assertThat(memoryAppender.containsPatterns(patterns, Level.TRACE)).isFalse();

4. Asserting Logs in Spring Boot With OutputCaptureExtension

For Spring Boot applications, we have a built-in solution that we can use to assert logs in our tests.

Let’s start by extending our test class with the OutputCaptureExtension:

@ExtendWith(OutputCaptureExtension.class)
class BusinessWorkerUnitTest {
}

The OutputCaptureExtension captures the output written to System.out and System.err during the execution of each test method.

We can then inject the captured output into our test method using the CapturedOutput parameter:

@Test
void whenLogsGenerated_thenCapturedOutputContainsLogs(CapturedOutput capturedOutput) {
    String log = "Order processed successfully for Order ID: 12345.";
    BusinessWorker worker = new BusinessWorker();
    worker.generateLogs(log);

    assertThat(capturedOutput.getOut()).contains(log);
    assertThat(capturedOutput.getOut()).containsPattern(".*Order ID: \\d{5}.*");
}

Here, we first generate logs using our familiar generateLogs() method. Then, we assert the presence of our log message in the capturedOutput parameter.

To validate dynamic content in the logs, we use AssertJ’s built-in containsPattern() method.

It’s important to note that we don’t have an option to assert the log level in this approach. One might argue that it’s part of the underlying logging framework and we shouldn’t test it anyway.

The OutputCaptureExtension simplifies log assertions in Spring Boot tests by automatically capturing the output and providing it to our test methods. This eliminates the need for custom appenders or configuring the logging system.

5. Conclusion

In this article, we’ve explored asserting log messages in our JUnit tests, covering two approaches, both in native Java and Spring Boot applications.