Why Do Local Variables Used in Lambdas Have to Be Final or Effectively Final?

1. Introduction

Java 8 gives us lambdas, and by association, the notion of effectively final variables. Ever wondered why local variables captured in lambdas have to be final or effectively final?

Well, the JLS gives us a bit of a hint when it says “The restriction to effectively final variables prohibits access to dynamically-changing local variables, whose capture would likely introduce concurrency problems.” But, what does it mean?

In the next sections, we’ll dig deeper into this restriction and see why Java introduced it. We’ll show examples to demonstrate how it affects single-threaded and concurrent applications, and we’ll also debunk a common anti-pattern for working around this restriction.

2. Capturing Lambdas

Lambda expressions can use variables defined in an outer scope. We refer to these lambdas as capturing lambdas. They can capture static variables, instance variables, and local variables, but only local variables must be final or effectively final.

In earlier Java versions, we ran into this when an anonymous inner class captured a variable local to the method that surrounded it – we needed to add the final keyword before the local variable for the compiler to be happy.

As a bit of syntactic sugar, now the compiler can recognize situations where, while the final keyword isn’t present, the reference isn’t changing at all, meaning it’s effectively final. We could say that a variable is effectively final if the compiler wouldn’t complain were we to declare it final.

3. Local Variables in Capturing Lambdas

Simply put, this won’t compile:

Supplier<Integer> incrementer(int start) {
  return () -> start++;
}

start is a local variable, and we are trying to modify it inside of a lambda expression.

The basic reason this won’t compile is that the lambda is capturing the value of start, meaning making a copy of it. Forcing the variable to be final avoids giving the impression that incrementing start inside the lambda could actually modify the start method parameter.

But, why does it make a copy? Well, notice that we are returning the lambda from our method. Thus, the lambda won’t get run until after the start method parameter gets garbage collected. Java has to make a copy of start in order for this lambda to live outside of this method.

3.1. Concurrency Issues

For fun, let’s imagine for a moment that Java did allow local variables to somehow remain connected to their captured values.

What should we do here:

public void localVariableMultithreading() {
    boolean run = true;
    executor.execute(() -> {
        while (run) {
            // do operation
        }
    });
    
    run = false;
}

While this looks innocent, it has the insidious problem of “visibility”. Recall that each thread gets its own stack, and so how do we ensure that our while loop sees the change to the run variable in the other stack? The answer in other contexts could be using synchronized blocks or the volatile keyword.

However, because Java imposes the effectively final restriction, we don’t have to worry about complexities like this.

4. Static or Instance Variables in Capturing Lambdas

The examples before can raise some questions if we compare them with the use of static or instance variables in a lambda expression.

We can make our first example compile just by converting our start variable into an instance variable:

private int start = 0;

Supplier<Integer> incrementer() {
    return () -> start++;
}

But, why can we change the value of start here?

Simply put, it’s about where member variables are stored. Local variables are on the stack, but member variables are on the heap. Because we’re dealing with heap memory, the compiler can guarantee that the lambda will have access to the latest value of start.

We can fix our second example by doing the same:

private volatile boolean run = true;

public void instanceVariableMultithreading() {
    executor.execute(() -> {
        while (run) {
            // do operation
        }
    });

    run = false;
}

The run variable is now visible to the lambda even when it’s executed in another thread since we added the volatile keyword.

Generally speaking, when capturing an instance variable, we could think of it as capturing the final variable this. Anyway, the fact that the compiler doesn’t complain doesn’t mean that we shouldn’t take precautions, especially in multithreading environments.

5. Avoid Workarounds

In order to get around the restriction on local variables, someone may think of using variable holders to modify the value of a local variable.

Let’s see an example that uses an array to store a variable in a single-threaded application:

public int workaroundSingleThread() {
    int[] holder = new int[] { 2 };
    IntStream sums = IntStream
      .of(1, 2, 3)
      .map(val -> val + holder[0]);

    holder[0] = 0;

    return sums.sum();
}

We could think that the stream is summing 2 to each value, but it’s actually summing 0 since this is the latest value available when the lambda is executed.

Let’s go one step further and execute the sum in another thread:

public void workaroundMultithreading() {
    int[] holder = new int[] { 2 };
    Runnable runnable = () -> System.out.println(IntStream
      .of(1, 2, 3)
      .map(val -> val + holder[0])
      .sum());

    new Thread(runnable).start();

    // simulating some processing
    try {
        Thread.sleep(new Random().nextInt(3) * 1000L);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }

    holder[0] = 0;
}

What value are we summing here? It depends on how long our simulated processing takes. If it’s short enough to let the execution of the method terminate before the other thread is executed it’ll print 6, otherwise, it’ll print 12.

In general, these kinds of workarounds are error-prone and can produce unpredictable results, so we should always avoid them.

6. Conclusion

In this article, we’ve explained why lambda expressions can only use final or effectively final local variables. As we’ve seen, this restriction comes from the different nature of these variables and how Java stores them in memory. We’ve also shown the dangers of using a common workaround.