How to Resolve NaN Values in Micrometer Gauges in Prometheus

1. Overview

Micrometer Gauges are a convenient way to expose the state of in-memory objects as application metrics. However, when they aren’t configured correctly, they may produce unexpected results, such as returning NaN instead of the actual value. This affects how a metric appears in Prometheus or any other metrics backend.

In this tutorial, we’ll reproduce a common scenario where a gauge reports NaN due to garbage collection. Then, we’ll discuss the underlying cause and fix the issue by maintaining strong references to the monitored object.

2. Reproducing the NaN Issue

Let’s suppose we want to monitor the state of a Java object and expose it via the Spring Boot Actuator.

Firstly, let’s add the spring-boot-starter-actuator dependency to the project, which brings in the Micrometer dependency, too:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Now, we can inject MeterRegistry in the service class and use it to register a Gauge that observes an object. For example, we can create and register the Gauge when the Spring application is fully initialized by listening for the ApplicationReadyEvent:

@Service
class FooService {

    private final MeterRegistry registry;

    // constructor

    @EventListener(ApplicationReadyEvent.class)
    public void setupGauges() {
        setupWeakReferenceGauge();
    }

    private void setupWeakReferenceGauge() {
        Foo foo = new Foo(10);

        Gauge.builder("foo.weak", foo, Foo::value)
          .description("Foo value - weak reference (will show NaN after GC)")
          .register(registry);
    }
  
    record Foo(int value) {
    }

}

If we verify the foo.weak meter now, we could expect to see a value of ten. However, if we start the application locally and perform a GET request to /actuator/metrics/foo.weak, we may immediately or eventually see a value of NaN:

{
    "name": "foo.weak",
    "description": "Foo value - weak reference (will show NaN after GC)",
    "measurements": [
       {
          "statistic": "VALUE",
          "value": "NaN"
      }
    ],
    "availableTags": []
}

Thus, we get data from the gauge that isn’t useful.

3. Understanding the Root Cause

To understand why the Gauge returns NaN, we need to look at two things:

• how Java garbage collection works with different reference types
• how Micrometer implements gauges

In Java, we can reference an object in different ways. A strong reference is the normal type of reference and prevents the object from being garbage collected as long as it exists. Conversely, weak references don’t prevent garbage collection. When the JVM needs memory, it’s free to reclaim objects that are only weakly reachable.

Micrometer’s Gauge has a design that aims to avoid memory leaks. Therefore, it doesn’t keep a strong reference to the monitored object by default. Instead, Gauge stores a weak reference and calls the provided value function when the metric is scraped.

In the example above, the Foo instance is created inside the setupWeakReferenceGauge method and isn’t stored anywhere else. Once that method finishes, there are no strong references to the object. This means the Foo instance becomes eligible for garbage collection almost immediately. When the garbage collector removes the object, the weak reference held by the gauge is cleared. Later, when the actuator endpoint tries to read the metric value, Micrometer can no longer access the object, so it reports the value as NaN.

4. Using Strong References

To prevent the gauge from returning NaN, we need to make sure that the monitored object isn’t garbage collected. The simplest way to do this is to keep a strong reference to it.

For instance, we could store Foo as a field in the service class. On the other hand, we can keep the code as it is and simply leverage .strongReference(true) from Gauge.Builder API:

private void setupStrongReferenceGauge() {
    Foo foo = new Foo(10);

    Gauge.builder("foo.strong", foo, Foo::value)
      .description("Foo value - strong reference (will persist)")
      .strongReference(true)
      .register(registry);
}

That’s it! Let’s verify the value of the new meter using the path /actuator/metrics/foo.strong:

{
    "name": "foo.strong",
    "description": "Foo value - strong reference (will persist)",
    "measurements": [
       {
          "statistic": "VALUE",
          "value": 10
      }
    ],
    "availableTags": []
}

Now, we again see a useful value.

5. Conclusion

In this article, we explored why Micrometer gauges may return NaN values when monitoring in-memory objects and how this behavior is related to the Java garbage collection and Micrometer’s use of weak references.

We reproduced the issue using Spring Boot Actuator, analyzed the root cause, and demonstrated how to fix it by keeping a strong reference to the monitored object or explicitly enabling strong references in the Gauge builder. With these approaches, we can ensure that metrics remain accurate and reliable when exposed to Prometheus or any other monitoring system.

As always, the code presented in this article is available over on GitHub.