How to Distinguish Between Field Absent vs. Null in Jackson

1. Introduction

In this tutorial, we’ll explore ways to configure Jackson’s ObjectMapper to handle serialization and deserialization for null and absent values. Finally, we’ll demonstrate a real-world scenario with a method for updating a record that treats null and absent values differently.

2. Difference Between Absent and Null Fields in JSON

When working with JSON data, it’s essential to distinguish between an absent field and a field explicitly set to null. While they might seem similar, they have different implications for data processing and API design. Let’s start with this simple POJO with a mix of primitive, List, and Object value types:

public class Sample {

    private Long id;
    private String name;
    private int amount;
    private List<String> keys;
    private List<Integer> values;

    // standard getters and setters
}

A field is absent when entirely missing from the JSON payload. For example, in this JSON, every field except the name field is absent:

{
    "name": null
}

When deserialized, absent fields take on the default value for their type (for example, null for objects or 0 for primitives). This distinction is critical in these scenarios:

1. Partial Updates — In APIs that support partial updates (for example, PATCH requests), an absent field might indicate “don’t change this value,” while a null field could mean “remove this value.”
2. Default Values — Applications might apply a default value when a field is absent. Conversely, explicitly setting a field to null signals the intention to clear its value.
3. Validation — Validation rules often differ for missing vs. null fields depending on business needs.

In our example, we’ll create methods to patch an existing object, considering different strategies for non-absent fields. So, understanding these nuances helps ensure predictable application behavior and adherence to JSON semantics. Also, we’ll include custom default values and simple JSON validation for primitives.

2.1. Default Jackson Behavior

Consider a scenario where having an amount of zero is invalid. We can set a default value for the amount field in the Sample class:

private int amount = 1;

When serializing a new Sample instance without calling any setters, the resulting JSON includes 1 for amount, while other fields are included with null values:

@Test
void whenSerializingWithDefaults_thenNullValuesIncluded() {
    Sample zeroArg = new Sample();
    Map<String, Object> map = new ObjectMapper()
      .convertValue(zeroArg, Map.class);

    assertEquals(1, map.get("amount"));
    assertTrue(map.containsKey("id"));
    assertNull(map.get("id"));
    // other fields ...
}

If the JSON payload explicitly sets the amount field to null, Jackson assigns the default primitive value (0) instead of using our custom default:

@Test
void whenDeserializingToMapWithDefaults_thenNullPrimitiveIsDefaulted() {
    String json = """
      {
        "amount": null
      }
    """;
    Sample sample = new ObjectMapper().readValue(json, Sample.class);

    assertEquals(0, sample.getAmount());
}

3. Customizing Jackson Deserialization

To ensure null values aren’t silently converted to defaults, we can enable the FAIL_ON_NULL_FOR_PRIMITIVES deserialization feature. With this configuration, setting null for a primitive will throw a MismatchedInputException:

@Test
void whenValidatingNullPrimitives_thenFailOnNullAmount() {
    ObjectMapper mapper = new ObjectMapper();
    mapper.enable(DeserializationFeature.FAIL_ON_NULL_FOR_PRIMITIVES);
    String json = """
      {
        "amount": null
      }
    """;

    assertThrows(MismatchedInputException.class,
      () -> mapper.readValue(json, Sample.class));
}

4. Customizing Jackson Serialization

For our patch methods, we want to exclude fields that are null, absent, or set to Java default values. Absent, in the context of Jackson, refers to an empty Optional. We can achieve all of this using the Include.NON_DEFAULT configuration. This setting reduces the payload size by omitting unnecessary fields.

Let’s convert an empty Sample instance into a map to verify that only the amount field will appear because of our custom default value:

@Test
void whenSerializingNonDefault_thenOnlyNonJavaDefaultsIncluded() {
    ObjectMapper mapper = new ObjectMapper();
    mapper.setSerializationInclusion(Include.NON_DEFAULT);

    Sample zeroArg = new Sample();
    Map<String, Serializable> map = mapper.convertValue(
      zeroArg, Map.class);

    assertEquals(zeroArg.getAmount(), map.get("amount"));
    assertEquals(1, map.keySet().size());
}

A lean serialization makes it easier to decide which fields to update when patching objects.

5. Patching Methods

Now, let’s take our understanding of how Jackson works with absent and null values and apply it to a real-world setting: partial updates.

Briefly, there are multiple ways to handle partial updates. Let’s look at two:

• Update only non-nulls because a null value means “this value didn’t change”
• Update all non-absent because a null and non-absent value means “this value should be set to null“

Let’s take a look at some concrete code that achieves each of these, departing from the usual “copy all properties” method, while taking advantage of our Jackson configuration.

5.1. Update Only Non-Nulls

Our first method involves ignoring every null value after deserialization. This way, when sending a patch, we only need to worry about the values that we want to be changed:

void updateIgnoringNulls(String json, Sample current) 
  throws JsonProcessingException {
    Sample update = MAPPER.readValue(json, Sample.class);

    if (update.getId() != null)
        current.setId(update.getId());

    if (update.getName() != null)
        current.setName(update.getName());

    current.setAmount(update.getAmount());

    if (update.getKeys() != null)
        current.setKeys(update.getKeys());

    if (update.getValues() != null)
        current.setValues(update.getValues());
}

This solution works great if we don’t need to worry about deleting existing values.

5.2. Test Non-Null Fields Update Strategy

Let’s add some setup to test this, starting with some default values in our Sample class:

public static Sample basic() {
    Sample defaults = new Sample();

    List keys = List.of("foo", "bar");
    List values = List.of(1, 2);

    defaults.setId(1l);
    defaults.setKeys(keys);
    defaults.setValues(values);

    return defaults;
}

Then, we test by only including the values field in our JSON input, checking if the field is updated and if one of the absent fields retains a value:

@Test
void whenPatchingNonNulls_thenNullsIgnored() {
    List<Integer> values = List.of(3);

    Sample defaults = Sample.basic();

    String json = """
      {
        "values": %s
      }
    """.formatted(values);

    updateIgnoringNulls(json, defaults);

    assertEquals(values, defaults.getValues());
    assertNotNull(defaults.getKeys());
}

5.3. Update All Non-Absent

Our next solution updates every field contained in the JSON input, even those that are null:

void updateNonAbsent(String json, Sample current) 
  throws JsonProcessingException {
    Map<String, Serializable> update = MAPPER.readValue(json, Map.class);

    if (update.containsKey("id"))
        current.setId((Long) update.get("id"));

    if (update.containsKey("name"))
        current.setName((String) update.get("name"));

    if (update.containsKey("amount"))
        current.setAmount((int) update.get("amount"));

    if (update.containsKey("keys"))
        current.setKeys((List<String>) update.get("keys"));

    if (update.containsKey("values"))
        current.setValues((List<Integer>) update.get("values"));
}

With this solution, explicitly including a null field means we want to clear this field when updating an existing object.

5.4. Test Non-Absent Fields Update Strategy

To test this, we’ll explicitly set the keys field to null and change the values field. We expect these to be the only affected fields, so we also check if an absent field remains unchanged:

@Test
void whenPatchingNonAbsent_thenNullsConsidered() {
    List<Integer> values = List.of(3);

    Sample defaults = Sample.basic();

    String json = """
      {
        "values": %s,
        "keys": null
      }
    """.formatted(values);

    updateNonAbsent(json, defaults);

    assertEquals(values, defaults.getValues());
    assertNull(defaults.getKeys());
    assertNotNull(defaults.getId());
}

6. Conclusion

In this article, we reviewed approaches that ensure flexible handling of null and absent values, depending on the application’s requirements. Whether ignoring nulls or treating them as significant, customizing Jackson’s behavior allows us to achieve the desired functionality while adhering to JSON semantics.