Get Log Output in JSON

1. Introduction

Most Java logging libraries today offer different layout options for formatting logs – to accurately fit the needs of each project.

In this quick tutorial, we want to format and output our log entries as JSON. We’ll see how to do this for the two most widely used logging libraries: Log4j2 and Logback.

Both use Jackson internally to represent logs in the JSON format.

For an introduction to these libraries take a look at our Introduction to Java Logging article.

2. Log4j2

Log4j2 is the direct successor of the most popular logging library for Java, Log4j.

As it’s the new standard for Java projects, we’ll show how to configure it to output JSON.

2.1. Maven Dependencies

First, we have to include the following dependencies in our pom.xml file:

<dependencies>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.19.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.19.0</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.17.2</version>
    </dependency>
</dependencies>

The latest versions of the previous dependencies can be found on Maven Central: log4j-api, log4j-core, jackson-databind.

2.2. Using JsonLayout

Then, in our log4j2.xml file, we can create a new Appender that uses JsonLayout and a new Logger that uses this Appender:

<Appenders>
    <Console name="ConsoleJSONAppender" target="SYSTEM_OUT">
        <JsonLayout complete="false" compact="false">
            <KeyValuePair key="myCustomField" value="myCustomValue"/>
        </JsonLayout>
    </Console>
</Appenders>

<Logger name="CONSOLE_JSON_APPENDER" level="TRACE" additivity="false">
    <AppenderRef ref="ConsoleJSONAppender"/>
</Logger>

As we can see in the example config, it’s possible to add our own values to the log using KeyValuePair, which even supports looking into the log context.

Setting the compact parameter to false will increase the size of the output and make it more human-readable.

Now, let’s test our configuration. In our code, we can instantiate our new JSON logger and make a new debug-level trace:

Logger logger = LogManager.getLogger("CONSOLE_JSON_APPENDER");
logger.debug("Debug message");

The debug output message for the previous code would be:

{
  "instant" : {
    "epochSecond" : 1696419692,
    "nanoOfSecond" : 479118362
  },
  "thread" : "main",
  "level" : "DEBUG",
  "loggerName" : "CONSOLE_JSON_APPENDER",
  "message" : "Debug message",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
  "threadId" : 1,
  "threadPriority" : 5,
  "myCustomField" : "myCustomValue"
}

2.3. Using JsonTemplateLayout

In the previous section, we saw how to use the JsonLayout property. Since version 2.14.0, the property has been deprecated and replaced by JsonTemplateLayout.

JsonTemplateLayout provides enhanced capabilities and improved efficiency, as it’s optimized to encode log events as fast as possible by default.

Additionally, it supports garbage-free logging giving it some performance advantage since garbage collector pauses can impact performance. To enable garbage-free logging, we need to set the log4j2.garbagefreeThreadContextMap and log4j2.enableThreadLocals properties to true:

-Dlog4j2.garbagefreeThreadContextMap=true 
-Dlog4j2.enableThreadlocals=true 

To use JsonTemplateLayout, let’s add to the log4j-layout-template-json dependency to pom.xml:

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-layout-template-json</artifactId>
    <version>2.24.3</version>
</dependency>

Next, let’s modify our Appender to use JsonTemplateLayout instead:

<Appenders> 
    <Console name="ConsoleJSONAppender" target="SYSTEM_OUT">
        <JsonTemplateLayout eventTemplateUri="classpath:JsonLayout.json">
            <EventTemplateAdditionalField key="myCustomField" value="myCustomValue"/>
        </JsonTemplateLayout>
    </Console>
</Appenders>

Here, we use JsonTemplateLayout and specify the JSON layout format using eventTemplateUri.  The eventTemplateUri defines the format of the JSON output. When eventTemplateUri is not specified, it uses an Elastic Common Schema (ECS) format by default (classpath:EcsLayout.json).

Other supported templates include the Graylog Extended Log Format (GELF) with value – classpath:GelfLayout.json.

The JsonLayout.json template is designed to easily transition from JsonLayout to JsonTemplateLayout. In our case, we use JsonLayout.json to maintain the initial format from the previous section example:

{
  "instant": {
    "epochSecond": 1736320992,
    "nanoOfSecond": 804274875
  },
  "thread": "main",
  "level": "DEBUG",
  "loggerName": "CONSOLE_JSON_APPENDER",
  "message": "Debug message",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.log4j.spi.AbstractLogger",
  "threadId": 1,
  "threadPriority": 5,
  "myCustomField": "myCustomValue"
}

Unlike JsonLayout which uses KeyValuePair properties to add custom key-value pairs, we use a property named EventTemplateAdditionalField instead to add keys and values.

3. Logback

Logback can be considered another successor of Log4j. It’s written by the same developers and claims to be more efficient and faster than its predecessor.

So, let’s see how to configure it to get the output of the logs in JSON format.

3.1. Maven Dependencies

Let’s include the following dependencies in our pom.xml:

<dependencies>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.4.8</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback.contrib</groupId>
        <artifactId>logback-json-classic</artifactId>
        <version>0.1.5</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback.contrib</groupId>
        <artifactId>logback-jackson</artifactId>
        <version>0.1.5</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.15.2</version>
    </dependency>
</dependencies>

We can check here for the latest versions of these dependencies: logback-classic, logback-json-classic, logback-jackson, jackson-databind.

3.2. Using JsonLayout

First, we create a new appender in our logback-test.xml that uses JsonLayout and JacksonJsonFormatter.

After that, we can create a new logger that uses this appender:

<appender name="json" class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
        <jsonFormatter
            class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
            <prettyPrint>true</prettyPrint>
        </jsonFormatter>
        <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
    </layout>
</appender>

<logger name="jsonLogger" level="TRACE">
    <appender-ref ref="json" />
</logger>

As we see, the parameter prettyPrint is enabled to obtain a human-readable JSON.

In order to test our configuration, let’s instantiate the logger in our code and log a debug message:

Logger logger = LoggerFactory.getLogger("jsonLogger");
logger.debug("Debug message");

With this – we’ll obtain the following output:

{
    "timestamp":"2017-12-14 23:36:22.305",
    "level":"DEBUG",
    "thread":"main",
    "logger":"jsonLogger",
    "message":"Debug message",
    "context":"default"
}

3.3. Using JsonEncoder

Another way of logging output in JSON is by using the JsonEncoder. It transforms a logging event into a valid JSON text.

Let’s add a new appender that uses JsonEncoder and also a new logger that uses this appender:

<appender name="jsonEncoder" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.classic.encoder.JsonEncoder"/>
</appender>

<logger name="jsonEncoderLogger" level="TRACE">
    <appender-ref ref="jsonEncoder" />
</logger>

Now, let’s instantiate the logger and call debug() to generate a log message:

Logger logger = LoggerFactory.getLogger("jsonEncoderLogger");
logger.debug("Debug message");

After executing this code, we get the following output:

{
    "sequenceNumber":0,
    "timestamp":1696689301574,
    "nanoseconds":574716015,
    "level":"DEBUG",
    "threadName":"main",
    "loggerName":"jsonEncoderLogger",
    "context":
        {
            "name":"default",
            "birthdate":1696689301038,
            "properties":{}
        },
    "mdc": {},
    "message":"Debug message",
    "throwable":null
}

Here, the message field represents the log message. Also, the context field shows the logging context. It’s typically default unless we’ve set up multiple logging contexts.

4. Conclusion

In this article, we saw how we can easily configure Log4j2 and Logback to have a JSON output format. We delegated all the complexity of the parsing to the logging library, so we don’t need to change any existing logger calls.