Security Top

I just announced the new Learn Spring Security course, including the full material focused on the new OAuth2 stack in Spring Security 5:

>> CHECK OUT THE COURSE

1. Overview

With the large amount of data being logged, it's important to mask sensitive details of the users while logging. In the new GDPR-present world, among many concerns, we must give special attention to logging sensitive data of individuals.

In this tutorial, we'll see how to mask sensitive data in logs with Logback. Overall, this approach isn't the real way to solve the problem – it's kind of a last line of defense for our log files.

2. Logback

Logback is one of the most widely used logging frameworks in the Java Community. It's a replacement for its predecessor, Log4j. It offers a faster implementation than Log4j, and it provides more options for configuration and more flexibility in archiving old log files.

Sensitive data is any information that is meant to be protected from unauthorized access. This can include anything from personally identifiable information (PII), such as Social Security numbers, to banking information, login credentials, address, email, and others.

We'll mask the sensitive data that belongs to users while logging within our application.

3. Masking Data

Let's say we log user details in the context of a web request. We need to mask the sensitive data related to users. Let’s assume our application receives the following request or response that we logged:

{
    "user_id":"87656",
    "ssn":"786445563",
    "address":"22 Street",
    "city":"Chicago",
    "Country":"U.S.",
    "ip_address":"192.168.1.1",
    "email_id":"[email protected]"
 }

Here, we can see that we have sensitive data like ssn, address, ip_address, and email_id. Hence, we have to mask this data while logging.

We'll mask the logs centrally by configuring masking rules for all log entries produced by Logback. In order to do that, we have to implement a custom ch.qos.logback.classic.PatternLayout.

3.1. PatternLayout

The idea behind the configuration is to extend every Logback appender we need with a custom layout. In our case, we'll write a MaskingPatternLayout class as an implementation of PatternLayout. Each mask pattern represents the regular expression that matches one type of sensitive data.

Let's build the MaskingPatternLayout class:

public class MaskingPatternLayout extends PatternLayout {

    private Pattern multilinePattern;
    private List<String> maskPatterns = new ArrayList<>();

    public void addMaskPattern(String maskPattern) {
        maskPatterns.add(maskPattern);
        multilinePattern = Pattern.compile(maskPatterns.stream().collect(Collectors.joining("|")), Pattern.MULTILINE);
    }

    @Override
    public String doLayout(ILoggingEvent event) {
        return maskMessage(super.doLayout(event));
    }

    private String maskMessage(String message) {
        if (multilinePattern == null) {
            return message;
        }
        StringBuilder sb = new StringBuilder(message);
        Matcher matcher = multilinePattern.matcher(sb);
        while (matcher.find()) {
            IntStream.rangeClosed(1, matcher.groupCount()).forEach(group -> {
                if (matcher.group(group) != null) {
                    IntStream.range(matcher.start(group), matcher.end(group)).forEach(i -> sb.setCharAt(i, '*'));
                }
            });
        }
        return sb.toString();
    }
}

The implementation of PatternLayout.doLayout() is responsible for masking matched data in each log message of our application if it matches one of the configured patterns.

The maskPatterns list from logback.xml constructs a multiline pattern. Unfortunately, the Logback engine does not support constructor injection. If it comes as a list of properties, addMaskPattern is invoked for every config entry. So, we have to compile the pattern every time we add a new regex to the list.

3.2. Configuration

In general, we can use regex patterns for masking sensitive user details.

For example, for the SSN, we can use a regex like:

\"SSN\"\s*:\s*\"(.*)\"

And for the address, we can use:

\"address\"\s*:\s*\"(.*?)\" 

Furthermore, for the IP address data pattern (192.169.0.1), we can use the regex:

(\d+\.\d+\.\d+\.\d+)

Finally, for email, we can write:

(\[email protected]\w+\.\w+)

Now, we'll add these regex patterns in maskPattern tags inside our logback.xml file:

<configuration>
    <appender name="mask" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
           <layout class="com.baeldung.logback.MaskingPatternLayout">
	       <maskPattern>\"SSN\"\s*:\s*\"(.*?)\"</maskPattern> <!-- SSN JSON pattern -->
	       <maskPattern>\"address\"\s*:\s*\"(.*?)\"</maskPattern> <!-- Address JSON pattern -->
	       <maskPattern>(\d+\.\d+\.\d+\.\d+)</maskPattern> <!-- Ip address IPv4 pattern -->
	       <maskPattern>(\[email protected]\w+\.\w+)</maskPattern> <!-- Email pattern -->
	       <pattern>%-5p [%d{ISO8601,UTC}] [%thread] %c: %m%n%rootException</pattern>
            </layout>
        </encoder>
    </appender>
</ configuration>

3.3. Execution

Now, we'll create the JSON for the above example and use logger.info() to log the details:

Map<String, String> user = new HashMap<String, String>();
user.put("user_id", "87656");
user.put("SSN", "786445563");
user.put("address", "22 Street");
user.put("city", "Chicago");
user.put("Country", "U.S.");
user.put("ip_address", "192.168.1.1");
user.put("email_id", "[email protected]");
JSONObject userDetails = new JSONObject(user);

logger.info("User JSON: {}", userDetails);

After executing this, we can see the output:

INFO  [2021-06-01 16:04:12,059] [main] com.baeldung.logback.MaskingPatternLayoutExample: User JSON: 
{"email_id":"*******************","address":"*********","user_id":"87656","city":"Chicago","Country":"U.S.", "ip_address":"***********","SSN":"*********"}

Here, we can see that the user JSON in our logger has been masked:

{
    "user_id":"87656",
    "ssn":"*********",
    "address":"*********",
    "city":"Chicago",
    "Country":"U.S.",
    "ip_address":"*********",
    "email_id":"*****************"
 }

With this approach, we can only mask those data in log files for which we've defined regular expressions in maskPattern in logback.xml.

4. Conclusion

In this tutorial, we covered how to use the PatternLayout feature to mask sensitive data in application logs with Logback. Also, we saw how to add regex patterns in logback.xml for masking specific data.

As usual, code snippets are available over on GitHub.

Security bottom

I just announced the new Learn Spring Security course, including the full material focused on the new OAuth2 stack in Spring Security 5:

>> CHECK OUT THE COURSE
Generic footer banner
2 Comments
Oldest
Newest
Inline Feedbacks
View all comments
Comments are closed on this article!