Structured Logging in Spring Boot

1.  Overview

Logging is an essential feature of any software application. It helps track the behavior of an application during runtime by recording errors, warnings, and other events.

By default, a Spring Boot application generates unstructured, human-readable logs. While these logs are useful for developers, they are not easily parsed or analyzed by log aggregation tools. Structured logging addresses this limitation.

In this tutorial, we’ll learn how to implement structured logging by leveraging features introduced in Spring Boot version 3.4.0.

2. Maven Dependencies

First, let’s bootstrap a Spring Boot project by adding spring-boot-starter to the pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
    <version>3.4.0</version>
</dependency>

The dependency above provides support for auto-configuration and logging in a typical Spring Boot application.

3. Spring Boot Default Log

Here’s a default Spring Boot log:

INFO 22059 --- [ main] c.b.s.StructuredLoggingApp  : No active profile set, falling back to 1 default profile: "default"
INFO 22059 --- [ main] c.b.s.StructuredLoggingApp   : Started StructuredLoggingApp in 2.349 seconds (process running for 3.259)

While these logs are informative, they cannot be easily ingested by tools like Elasticsearch or analyzed for metrics. Structured logging formats like JSON solve this problem by standardizing log content.

4. Configuration

Starting from Spring Boot version 3.4.0, structured logging is built-in and supports formats like Elastic Common Schema (ECS), Graylog Extended Log Format (GELF), and Logstash JSON.

We can configure structure logging directly in the application.properties file.

4.1. Elastic Common Schema

Elastic Common Schema (ECS) is a standardized JSON-based log format that integrates Elasticsearch and Kibana seamlessly. To configure ECS in our application, let’s add it property to our application.properties file:

logging.structured.format.console=ecs

Here’s an example output:

{
  "@timestamp": "2024-12-19T01:17:47.195098997Z",
  "log.level": "INFO",
  "process.pid": 16623,
  "process.thread.name": "main",
  "log.logger": "com.baeldung.springstructuredlogging.StructuredLoggingApp",
  "message": "Started StructuredLoggingApp in 3.15 seconds (process running for 4.526)",
  "ecs.version": "8.11"
}

The output contains key-value pairs that can easily be parsed in Elasticsearch and Kibana.

Furthermore, we can enhance ECS logs by adding fields like service name, environment, and node name to improve observability:

logging.structured.ecs.service.name=MyService
logging.structured.ecs.service.version=1
logging.structured.ecs.service.environment=Production
logging.structured.ecs.service.node-name=Primary

Here’s the new output:

{
  "@timestamp": "2024-12-19T01:25:15.123108416Z",
  "log.level": "INFO",
  "process.pid": 18763,
  "process.thread.name": "main",
  "service.name": "BaeldungService",
  "service.version": "1",
  "service.environment": "Production",
  "service.node.name": "Primary",
  "log.logger": "com.baeldung.springstructuredlogging.StructuredLoggingApp",
  "message": "Started StructuredLoggingApp in 3.378 seconds (process running for 4.376)",
  "ecs.version": "8.11"
}

The output contains the service information we defined in the application.properties file.

4.2. Graylog Extended Log Format

Graylog Extend Log Format (GELF) is another JSON-based supported structured log format. Let’s enable it in our application.properties file:

logging.structured.format.console=gelf

The GELF format is similar to the ECS format but with different property names:

{
  "version": "1.1",
  "short_message": "Started StructuredLoggingApp in 2.77 seconds (process running for 3.89)",
  "timestamp": 1734572549.172,
  "level": 6,
  "_level_name": "INFO",
  "_process_pid": 23929,
  "_process_thread_name": "main",
  "_log_logger": "com.baeldung.springstructuredlogging.StructuredLoggingApp"
}

Like ECS configuration, we can further enhance the output by defining the host and service version in the application.properties file:

logging.structured.gelf.host=MyService
logging.structured.gelf.service.version=1

This extends the log by adding host and service key-value pairs.

4.3. Logstash Format

The Logstash format is also supported out of the box. To structure our log to the format, let’s specify it in the application.properties:

logging.structured.format.console=logstash

Here’s the sample structured log:

{
  "@timestamp": "2024-12-19T02:49:33.017851728+01:00",
  "@version": "1",
  "message": "Started StructuredLoggingApp in 2.749 seconds (process running for 3.605)",
  "logger_name": "com.baeldung.springstructuredlogging.StructuredLoggingApp",
  "thread_name": "main",
  "level": "INFO",
  "level_value": 20000
}

The format above can be easily analyzed with a log aggregation that supports the Logstash format.

4.4. Additional Information

We can add more information to a structured log using the Mapped Diagnostic Context (MDC) class. For instance, we can add a userId to a log to filter logs by the userId:

private static final Logger LOGGER = LoggerFactory.getLogger(CustomLog.class);

public void additionalDetailsWithMdc() {
    MDC.put("userId", "1");
    MDC.put("userName", "Baeldung");
    LOGGER.info("Hello structured logging!");
    MDC.remove("userId");
    MDC.remove("userName");
}

In the code above, we remove each entry afterward to clean up the MDC context to prevent memory leaks.

Let’s see the log output with the user details:

{
  "@timestamp": "2024-12-19T07:52:30.556819106+01:00",
  "@version": "1",
  "message": "Hello structured logging!",
  "logger_name": "com.baeldung.springstructuredlogging.CustomLog",
  "thread_name": "main",
  "level": "INFO",
  "level_value": 20000,
  "userId": "1",
  "userName": "Baeldung"
}

Here, we add more stamps to the log message. We can easily filter a log based on the userId. We can add more properties to a log using the MDC class.

Also, we can use the fluent logging API to achieve a similar purpose:

public void additionalDetailsUsingFluentApi() {
    LOGGER.atInfo()
      .setMessage("Hello Structure logging!")
      .addKeyValue("userId", "1")
      .addKeyValue("userName", "Baeldung")
      .log();
}

This approach is more concise and it automatically handles context cleanup making it less error-prone.

4.5. Custom Log Format

Moreover, we can define our own custom structure log format and use it instead in the application.properties. This could be useful in a scenario where the supported log format doesn’t meet our use case.

First, we need to implement the StructuredLogFormatter interface and override its format() method:

class MyStructuredLoggingFormatter implements StructuredLogFormatter<ILoggingEvent> {
    @Override
    public String format(ILoggingEvent event) {
       return "time=" + event.getTimeStamp() + " level=" + event.getLevel() + " message=" + event.getMessage() + "\n";
    }
}

Here, our custom format is in text format and not a standard JSON. This provides flexibility in that we can structure our log based on any format, JSON, XML, etc.

Next, let’s define our custom configuration in the application.properties:

logging.structured.format.console=com.baeldung.springstructuredlogging.MyStructuredLoggingFormatter

Here, we define the fully qualified class name of MyStructuredLoggingFormatter.

Here’s the log output:

time=1734598194538 level=INFO message=Hello structured logging!

The output is in text format with key and value pair representing the log details.

Custom formats can be advantageous in case the supported formats aren’t tailored to our needs.

Furthermore, we can use JSONWriter to write a custom JSON formatted:

private final JsonWriter<ILoggingEvent> writer = JsonWriter.<ILoggingEvent>of((members) -> {
    members.add("time", ILoggingEvent::getInstant);
    members.add("level", ILoggingEvent::getLevel);
    members.add("thread", ILoggingEvent::getThreadName);
    members.add("message", ILoggingEvent::getFormattedMessage);
    members.add("application").usingMembers((application) -> {
        application.add("name", "StructuredLoggingDemo");
        application.add("version", "1.0.0-SNAPSHOT");
    });
    members.add("node").usingMembers((node) -> {
        node.add("hostname", "node-1");
        node.add("ip", "10.0.0.7");
    });
}).withNewLineAtEnd();

Next, let’s integrate the writer() method into the format() method:

@Override
public String format(ILoggingEvent event) {
    return this.writer.writeToString(event);
}

The outputted log is structured in JSON format:

{
  "time": "2024-12-19T08:55:13.284101533Z",
  "level": "INFO",
  "thread": "main",
  "message": "No active profile set, falling back to 1 default profile: \"default\"",
  "application": {
    "name": "StructuredLoggingDemo",
    "version": "1.0.0-SNAPSHOT"
  },
  "node": {
    "hostname": "node-1",
    "ip": "10.0.0.7"
  }
}

Depending on the use case, writing a custom format provides more flexibility to work with log aggregation not supported by default in Spring Boot.

4.6. Logging to File

Our previous examples log directly to the console. However, we can maintain the human log format in the console and write structured logs to a file by modifying our configuration:

logging.structured.format.file=ecs
logging.file.name=log.json

Here, we use the file property instead of the console property. This creates a log.json file with the structured log in our project root directory.

5. Conclusion

In this article, we learned how to configure our application for structure logging using the application.properties configuration. Also, we saw different examples of the supported structured log formats.

Finally, we saw how to write our custom format and use it through configuration.