Intro to Log4j2 – Appenders, Layouts and Filters

1. Overview

Logging events is a critical aspect of software development. While there are lots of frameworks available in Java ecosystem, Log4J has been the most popular for decades, due to the flexibility and simplicity it provides.

Log4j 2 is a new and improved version of the classic Log4j framework.

In this article, we’ll introduce the most common appenders, layouts, and filters via practical examples.

In Log4J 2, an appender is simply a destination for log events; it can be as simple as a console and can be complex like any RDBMS. Layouts determine how the logs will be presented and filters filter the data according to the various criterion.

2. Setup

In order to understand several logging components and their configuration let’s set up different test use-cases, each consisting of a log4J2.xml configuration file and a JUnit 4 test class.

Two maven dependencies are common to all examples:

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.19.0</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.19.0</version>
    <type>test-jar</type>
    <scope>test</scope>
</dependency>

Besides the main log4j-core package we need to include the ‘test jar’ belonging to the package to gain access to a context rule needed for testing of uncommonly named configuration files.

3. Default Configuration

ConsoleAppender is the default configuration of the Log4J 2 core package. It logs messages to the system console in a simple pattern:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <Console name="ConsoleAppender" target="SYSTEM_OUT">
            <PatternLayout 
              pattern="%d [%t] %-5level %logger{36} - %msg%n%throwable"/>
        </Console>
    </Appenders>
    <Loggers>
        <Root level="ERROR">
            <AppenderRef ref="ConsoleAppender"/>
        </Root>
    </Loggers>
</Configuration>

Let’s analyze the tags in this simple XML configuration:

• Configuration: The root element of a Log4J 2 configuration file and attribute status is the level of the internal Log4J events, that we want to log
• Appenders: This element is holding one or more appenders. Here we’ll configure an appender that outputs to the system console at standard out
• Loggers: This element can consist of multiple configured Logger elements. With the special Root tag, you can configure a nameless standard logger that will receive all log messages from the application. Each logger can be set to a minimum log level
• AppenderRef: This element defines a reference to an element from the Appenders section. Therefore the attribute ‘ref‘ is linked with an appenders ‘name‘ attribute

The corresponding unit test will be similarly simple. We’ll obtain a Logger reference and print two messages:

@Test
public void givenLoggerWithDefaultConfig_whenLogToConsole_thanOK()
  throws Exception {
    Logger logger = LogManager.getLogger(getClass());
    Exception e = new RuntimeException("This is only a test!");

    logger.info("This is a simple message at INFO level. " +
      "It will be hidden.");
    logger.error("This is a simple message at ERROR level. " +
    "This is the minimum visible level.", e);
}

4. ConsoleAppender With PatternLayout

Let’s define a new console appender with a customized color pattern in a separate XML file, and include that in our main configuration:

<?xml version="1.0" encoding="UTF-8"?>
<Console name="ConsoleAppender" target="SYSTEM_OUT">
    <PatternLayout pattern="%style{%date{DEFAULT}}{yellow}
      %highlight{%-5level}{FATAL=bg_red, ERROR=red, WARN=yellow, INFO=green} 
      %message"/>
</Console>

This file is using some pattern variables that gets replaced by Log4J 2 at runtime:

• %style{…}{colorname}: This will print the text in the first bracket pair (…) in a given color (colorname).
• %highlight{…}{FATAL=colorname, …}: This is similar to the ‘style’ variable. But a different color can be given for each log level.
• %date{format}: This gets replaced by the current date in the specified format. Here we’re using the ‘DEFAULT’ DateTime format, ‘yyyy-MM-dd HH:mm:ss,SSS’.
• %-5level: Prints the level of the log message in a right-aligned fashion.
• %message: Represents the raw log message

But there exists many more variables and formatting in the PatternLayout. You can refer them to the Log4J 2‘s official documentation.

Now we’ll include the defined console appender into our main configuration:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" xmlns:xi="http://www.w3.org/2001/XInclude">
    <Appenders>
        <xi:include href="log4j2-includes/
          console-appender_pattern-layout_colored.xml"/>
    </Appenders>
    <Loggers>
        <Root level="DEBUG">
            <AppenderRef ref="ConsoleAppender"/>
        </Root>
    </Loggers>
</Configuration>

The unit test:

@Test
public void givenLoggerWithConsoleConfig_whenLogToConsoleInColors_thanOK() 
  throws Exception {
    Logger logger = LogManager.getLogger("CONSOLE_PATTERN_APPENDER_MARKER");
    logger.trace("This is a colored message at TRACE level.");
    ...
}

5. Async File Appender With JSONLayout and BurstFilter

Sometimes it’s useful to write log messages in an asynchronous manner. For example, if application performance has priority over the availability of logs.

In such use-cases, we can use an AsyncAppender.

For our example, we’re configuring an asynchronous JSON log file. Furthermore, we’ll include a burst filter that limits the log output at a specified rate:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        ...
        <File name="JSONLogfileAppender" fileName="target/logfile.json">
            <JSONLayout compact="true" eventEol="true"/>
            <BurstFilter level="INFO" rate="2" maxBurst="10"/>
        </File>
        <Async name="AsyncAppender" bufferSize="80">
            <AppenderRef ref="JSONLogfileAppender"/>
        </Async>
    </Appenders>
    <Loggers>
        ...
        <Logger name="ASYNC_JSON_FILE_APPENDER" level="INFO"
          additivity="false">
            <AppenderRef ref="AsyncAppender" />
        </Logger>
        <Root level="INFO">
            <AppenderRef ref="ConsoleAppender"/>
        </Root>
    </Loggers>
</Configuration>

Notice that:

• The JSONLayout is configured in a way, that writes one log event per row
• The BurstFilter will drop every event with ‘INFO’ level and above if there are more than two of them, but at a maximum of 10 dropped events
• The AsyncAppender is set to a buffer of 80 log messages; after that, the buffer is flushed to the log file

Let’s take a look at the corresponding unit test. We’re filling the appended buffer in a loop, let it write to disk and inspect the line count of the log file:

@Test
public void givenLoggerWithAsyncConfig_whenLogToJsonFile_thanOK() 
  throws Exception {
    Logger logger = LogManager.getLogger("ASYNC_JSON_FILE_APPENDER");

    final int count = 88;
    for (int i = 0; i < count; i++) {
        logger.info("This is async JSON message #{} at INFO level.", count);
    }
    
    long logEventsCount 
      = Files.lines(Paths.get("target/logfile.json")).count();
    assertTrue(logEventsCount > 0 && logEventsCount <= count);
}

6. RollingFile Appender and XMLLayout

Next, we’ll create a rolling log file. After a configured file size, the log file gets compressed and rotated.

This time we’re using an XML layout:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <RollingFile name="XMLRollingfileAppender"
          fileName="target/logfile.xml"
          filePattern="target/logfile-%d{yyyy-MM-dd}-%i.log.gz">
            <XMLLayout/>
            <Policies>
                <SizeBasedTriggeringPolicy size="17 kB"/>
            </Policies>
        </RollingFile>
    </Appenders>
    <Loggers>
        <Logger name="XML_ROLLING_FILE_APPENDER" 
       level="INFO" additivity="false">
            <AppenderRef ref="XMLRollingfileAppender" />
        </Logger>
        <Root level="TRACE">
            <AppenderRef ref="ConsoleAppender"/>
        </Root>
    </Loggers>
</Configuration>

Notice that:

• The RollingFile appender has a ‘filePattern’ attribute, which is used to name rotated log files and can be configured with placeholder variables. In our example, it should contain a date and a counter before the file suffix.
• The default configuration of XMLLayout will write single log event objects without the root element.
• We’re using a size based policy for rotating our log files.

Our unit test class will look like the one from the previous section:

@Test
public void givenLoggerWithRollingFileConfig_whenLogToXMLFile_thanOK()
  throws Exception {
    Logger logger = LogManager.getLogger("XML_ROLLING_FILE_APPENDER");
    final int count = 88;
    for (int i = 0; i < count; i++) {
        logger.info(
          "This is rolling file XML message #{} at INFO level.", i);
    }
}

7. Syslog Appender

Let’s say we need to send logged event’s to a remote machine over the network. The simplest way to do that using Log4J 2 would be using it’s Syslog Appender:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        ...
        <Syslog name="Syslog" 
          format="RFC5424" host="localhost" port="514" 
          protocol="TCP" facility="local3" connectTimeoutMillis="10000" 
          reconnectionDelayMillis="5000">
        </Syslog>
    </Appenders>
    <Loggers>
        ...
        <Logger name="FAIL_OVER_SYSLOG_APPENDER" 
          level="INFO" 
          additivity="false">
            <AppenderRef ref="FailoverAppender" />
        </Logger>
        <Root level="TRACE">
            <AppenderRef ref="Syslog" />
        </Root>
    </Loggers>
</Configuration>

The attributes in the Syslog tag:

• name: defines the name of the appender, and must be unique. Since we can have multiple Syslog appenders for the same application and configuration
• format: it can be either set to BSD or RFC5424, and the Syslog records would be formatted accordingly
• host & port: the hostname and port of the remote Syslog server machine
• protocol: whether to use TCP or UPD
• facility: to which Syslog facility the event will be written
• connectTimeoutMillis: time period of waiting for an established connection, defaults to zero
• reconnectionDelayMillis: time to wait before re-attempting connection

8. FailoverAppender

Now there may be instances where one appender fails to process the log events and we do not want to lose the data. In such cases, the FailoverAppender comes handy.

For example, if the Syslog appender fails to send events to the remote machine, instead of losing that data we might fall back to FileAppender temporarily.

The FailoverAppender takes a primary appender and number of secondary appenders. In case the primary fails, it tries to process the log event with secondary ones in order until one succeeds or there aren’t any secondaries to try:

<Failover name="FailoverAppender" primary="Syslog">
    <Failovers>
        <AppenderRef ref="ConsoleAppender" />
    </Failovers>
</Failover>

Let’s test it:

@Test
public void givenLoggerWithFailoverConfig_whenLog_thanOK()
  throws Exception {
    Logger logger = LogManager.getLogger("FAIL_OVER_SYSLOG_APPENDER");
    Exception e = new RuntimeException("This is only a test!"); 

    logger.trace("This is a syslog message at TRACE level.");
    logger.debug("This is a syslog message at DEBUG level.");
    logger.info("This is a syslog message at INFO level. 
      This is the minimum visible level.");
    logger.warn("This is a syslog message at WARN level.");
    logger.error("This is a syslog message at ERROR level.", e);
    logger.fatal("This is a syslog message at FATAL level.");
}

9. JDBC Appender

The JDBC appender sends log events to an RDBMS, using standard JDBC. The connection can be obtained either using any JNDI Datasource or any connection factory.

The basic configuration consists of a DataSource or ConnectionFactory, ColumnConfigs, and tableName:

<JDBC name="JDBCAppender" tableName="logs">
    <ConnectionFactory 
      class="com.baeldung.logging.log4j2.tests.jdbc.ConnectionFactory" 
      method="getConnection" />
    <Column name="when" isEventTimestamp="true" />
    <Column name="logger" pattern="%logger" />
    <Column name="level" pattern="%level" />
    <Column name="message" pattern="%message" />
    <Column name="throwable" pattern="%ex{full}" />
</JDBC>

Now let’s try out:

@Test
public void givenLoggerWithJdbcConfig_whenLogToDataSource_thanOK()
  throws Exception {
    Logger logger = LogManager.getLogger("JDBC_APPENDER");
    final int count = 88;
    for (int i = 0; i < count; i++) {
        logger.info("This is JDBC message #{} at INFO level.", count);
    }

    Connection connection = ConnectionFactory.getConnection();
    ResultSet resultSet = connection.createStatement()
      .executeQuery("SELECT COUNT(*) AS ROW_COUNT FROM logs");
    int logCount = 0;
    if (resultSet.next()) {
        logCount = resultSet.getInt("ROW_COUNT");
    }
    assertTrue(logCount == count);
}

10. Dynamic File Name in the Appender

Sometimes, we want the log file name to be dynamic — for example, based on the environment, date, or application instance. In Log4j 2, we use attributes and the ${sys:…} lookup to reference system properties:

<Appenders>
    <File name="DynamicFileAppender" fileName="${sys:logfilename}.log">
        <PatternLayout pattern="%d [%t] %-5level %logger - %msg%n"/>
    </File>
</Appenders>

Before using the appender, set the system property:

System.setProperty("log4j.configurationFile", "src/test/resources/log4j2-dynamic.xml");
System.setProperty("logfilename", "log4j2-dynamic-log");
Logger logger = LogManager.getLogger("DYNAMIC_FILE_APPENDER");
logger.error("This is an ERROR log message to the same file.");

File file = new File("log4j2-dynamic-log");
assertTrue(file.exists());

String content = Files.readString(file.toPath());
assertTrue(content.contains("This is an ERROR log message to the same file."));

In Log4j 2, the ${sys:logfilename} expression reads the value of the system property logfilename. Using this value, the appender creates the log file dynamically at runtime, so the file name doesn’t need to be hardcoded in the configuration.

11. Conclusion

This article shows very simple examples of how you can use different logging appenders, filter and layouts with Log4J 2 and ways to configure them.