Writing an Enterprise-Grade AWS Lambda in Java

1. Overview

It doesn’t require much code to put together a basic AWS Lambda in Java. To keep things small, we usually create our serverless applications with no framework support.

However, if we need to deploy and monitor our software at enterprise quality, we need to solve many of the problems that are solved out-of-the-box with frameworks like Spring.

In this tutorial, we’ll look at how to include configuration and logging capabilities in an AWS Lambda, as well as libraries that reduce boilerplate code, while still keeping things lightweight.

2. Building an Example

2.1. Framework Options

Frameworks like Spring Boot cannot be used to create AWS Lambdas. The Lambda has a different lifecycle from a server application, and it interfaces with the AWS runtime without directly using HTTP.

Spring offers Spring Cloud Function, which can help us create an AWS Lambda, but we often need something smaller and simpler.

We’ll take inspiration from DropWizard, which has a smaller feature set than Spring but still supports common standards, including configurability, logging, and dependency injection.

While we may not need every one of these features from one Lambda to the next, we’ll build an example that solves all of these problems, so we can choose which techniques to use in future development.

2.2. Example Problem

Let’s create an app that runs every few minutes. It’ll look at a “to-do list”, find the oldest job that’s not marked as done, and then create a blog post as an alert. It will also produce helpful logs to allow CloudWatch alarms to alert on errors.

We’ll use the APIs on JsonPlaceholder as our back-end, and we’ll make the application configurable for both the base URLs of the APIs and the credentials we’ll use in that environment.

2.3. Basic Setup

We’ll use the AWS SAM CLI to create a basic Hello World Example.

Then we’ll change the default App class, which has an example API handler in it, into a simple RequestStreamHandler that logs on startup:

public class App implements RequestStreamHandler {

    @Override
    public void handleRequest(
      InputStream inputStream, 
      OutputStream outputStream, 
      Context context) throws IOException {
        context.getLogger().log("App starting\n");
    }
}

As our example is not an API handler, we won’t need to read any input or produce any output. Right now, we’re using the LambdaLogger inside the Context passed to our function to do logging, though later on, we’ll look at how to use Log4j and Slf4j.

Let’s quickly test this:

$ sam build
$ sam local invoke

Mounting todo-reminder/.aws-sam/build/ToDoFunction as /var/task:ro,delegated inside runtime container
App starting
END RequestId: 2aaf6041-cf57-4414-816d-76a63c7109fd
REPORT RequestId: 2aaf6041-cf57-4414-816d-76a63c7109fd  Init Duration: 0.12 ms  Duration: 121.70 ms
  Billed Duration: 200 ms Memory Size: 512 MB     Max Memory Used: 512 MB 

Our stub application has started up and logged “App starting” to the logs.

3. Configuration

As we may deploy our application to multiple environments, or wish to keep things like credentials separate from our code, we need to be able to pass in configuration values at deployment or runtime. This is most commonly achieved by setting environment variables.

3.1. Adding Environment Variables to the Template

The template.yaml file contains the settings for the lambda. We can add environment variables to our function using the Environment section under AWS::Serverless::Function section:

Environment: 
  Variables:
    PARAM1: VALUE

The generated example template has a hard-coded environment variable PARAM1, but we need to set our environment variables at deployment time.

Let’s imagine that we want our application to know the name of its environment in a variable ENV_NAME.

First, let’s add a parameter to the very top of the template.yaml file with a default environment name:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: todo-reminder application

Parameters:
  EnvironmentName:
    Type: String
    Default: dev

Next, let’s connect that parameter to an environment variable in the AWS::Serverless::Function section:

Environment: 
  Variables: 
    ENV_NAME: !Ref EnvironmentName

Now, we’re ready to read the environment variable at runtime.

3.2. Read an Environment Variable

Let’s read the environment variable ENV_NAME upon the construction of our App object:

private String environmentName = System.getenv("ENV_NAME");

We can also log the environment when handleRequest is called:

context.getLogger().log("Environment: " + environmentName + "\n");

The log message must end in “\n” to separate logging lines. We can see the output:

$ sam build
$ sam local invoke

START RequestId: 12fb0c05-f222-4352-a26d-28c7b6e55ac6 Version: $LATEST
App starting
Environment: dev

Here, we see that the environment has been set from the default in template.yaml.

3.3. Changing Parameter Values

We can use parameter overrides to supply a different value at runtime or deploy time:

$ sam local invoke --parameter-overrides "ParameterKey=EnvironmentName,ParameterValue=test"

START RequestId: 18460a04-4f8b-46cb-9aca-e15ce959f6fa Version: $LATEST
App starting
Environment: test

3.4. Unit Testing with Environment Variables

As an environment variable is global to the application, we might be tempted to initialize it in a private static final constant. However, this makes it very difficult to unit test.

As the handler class is initialized by the AWS Lambda runtime as a singleton for the entire life of the application, it’s better to use instance variables of the handler to store the runtime state.

We can use System Stubs to set an environment variable, and Mockito deep stubs to make our LambdaLogger testable inside the Context. First, we have to add the MockitoJUnitRunner to the test:

@RunWith(MockitoJUnitRunner.class)
public class AppTest {

    @Mock(answer = Answers.RETURNS_DEEP_STUBS)
    private Context mockContext;

    // ...
}

Next, we can use an EnvironmentVariablesRule to enable us to control the environment variable before the App object is created:

@Rule
public EnvironmentVariablesRule environmentVariablesRule = 
  new EnvironmentVariablesRule();

Now, we can write the test:

environmentVariablesRule.set("ENV_NAME", "unitTest");
new App().handleRequest(fakeInputStream, fakeOutputStream, mockContext);

verify(mockContext.getLogger()).log("Environment: unitTest\n");

As our lambdas get more complicated, it’s very useful to be able to unit test the handler class, including the way it loads its configuration.

4. Handling Complex Configurations

For our example, we’ll need the endpoint addresses for our API, as well as the name of the environment. The endpoint might vary at test time, but it has a default value.

We can use System.getenv several times over, and even use Optional and orElse to drop to a default:

String setting = Optional.ofNullable(System.getenv("SETTING"))
  .orElse("default");

However, this can require a lot of repetitive code and coordination of lots of individual Strings.

4.1. Represent the Configuration as a POJO

If we build a Java class to contain our configuration, we can share that with the services that need it:

public class Config {
    private String toDoEndpoint;
    private String postEndpoint;
    private String environmentName;

    // getters and setters
}

Now we can construct our runtime components with the current configuration:

public class ToDoReaderService {
    public ToDoReaderService(Config configuration) {
        // ...
    }
}

The service can take any configuration values it needs from the Config object. We can even model the configuration as a hierarchy of objects, which may be useful if we have repeated structures like credentials:

private Credentials toDoCredentials;
private Credentials postCredentials;

So far, this is just a design pattern. Let’s look at how to load these values in practice.

4.2. Configuration Loader

We can use lightweight-config to load our configuration from a .yml file in our resources.

Let’s add the dependency to our pom.xml:

<dependency>
    <groupId>uk.org.webcompere</groupId>
    <artifactId>lightweight-config</artifactId>
    <version>1.1.0</version>
</dependency>

And then, let’s add a configuration.yml file to our src/main/resources directory. This file mirrors the structure of our configuration POJO and contains hardcoded values, placeholders to fill in from environment variables, and defaults:

toDoEndpoint: https://jsonplaceholder.typicode.com/todos
postEndpoint: https://jsonplaceholder.typicode.com/posts
environmentName: ${ENV_NAME}
toDoCredentials:
  username: baeldung
  password: ${TODO_PASSWORD:-password}
postCredentials:
  username: baeldung
  password: ${POST_PASSWORD:-password}

We can load these settings into our POJO using the ConfigLoader:

Config config = ConfigLoader.loadYmlConfigFromResource("configuration.yml", Config.class);

This fills in the placeholder expressions from the environment variables, applying defaults after the :- expressions. It’s quite similar to the configuration loader built into DropWizard.

4.3. Holding the Context Somewhere

If we have several components – including the configuration – to load when the lambda first starts, it can be useful to keep these in a central place.

Let’s create a class called ExecutionContext that the App can use for object creation:

public class ExecutionContext {
    private Config config;
    private ToDoReaderService toDoReaderService;
    
    public ExecutionContext() {
        this.config = 
          ConfigLoader.loadYmlConfigFromResource("configuration.yml", Config.class);
        this.toDoReaderService = new ToDoReaderService(config);
    }
}

The App can create one of these in its initializer list:

private ExecutionContext executionContext = new ExecutionContext();

Now, when the App needs a “bean”, it can get it from this object.

5. Better Logging

So far, our use of the LambdaLogger has been very basic. If we bring in libraries that perform logging, the chances are that they’ll expect Log4j or Slf4j to be present. Ideally, our log lines will have timestamps and other useful context information.

Most importantly, when we encounter errors, we ought to log them with plenty of useful information, and Logger.error usually does a better job at this task than homemade code.

5.1. Add the AWS Log4j Library

We can enable the AWS lambda Log4j runtime by adding dependencies to our pom.xml:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-lambda-java-log4j2</artifactId>
    <version>1.2.0</version>
</dependency>

We also need a log4j2.xml file in src/main/resources configured to use this logger:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration packages="com.amazonaws.services.lambda.runtime.log4j2">
    <Appenders>
        <Lambda name="Lambda">
            <PatternLayout>
                <pattern>%d{yyyy-MM-dd HH:mm:ss} %X{AWSRequestId} %-5p %c{1} - %m%n</pattern>
            </PatternLayout>
        </Lambda>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="Lambda" />
        </Root>
    </Loggers>
</Configuration>

5.2. Writing a Logging Statement

Now, we add the standard Log4j Logger boilerplate to our classes:

public class ToDoReaderService {
    private static final Logger LOGGER = LogManager.getLogger(ToDoReaderService.class);

    public ToDoReaderService(Config configuration) {
        LOGGER.info("ToDo Endpoint on: {}", configuration.getToDoEndpoint());
        // ...
    }

    // ...
}

Then we can test it from the command line:

$ sam build
$ sam local invoke

START RequestId: acb34989-980c-42e5-b8e4-965d9f497d93 Version: $LATEST
2021-05-23 20:57:15  INFO  ToDoReaderService - ToDo Endpoint on: https://jsonplaceholder.typicode.com/todos

5.3. Unit Testing Log Output

In cases where testing log output is important, we can do that using System Stubs. Our configuration, optimized for AWS Lambda, directs the log output to System.out, which we can tap:

@Rule
public SystemOutRule systemOutRule = new SystemOutRule();

@Test
public void whenTheServiceStarts_thenItOutputsEndpoint() {
    Config config = new Config();
    config.setToDoEndpoint("https://todo-endpoint.com");
    ToDoReaderService service = new ToDoReaderService(config);

    assertThat(systemOutRule.getLinesNormalized())
      .contains("ToDo Endpoint on: https://todo-endpoint.com");
}

5.4. Adding Slf4j Support

We can add Slf4j by adding the dependency:

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-slf4j-impl</artifactId>
    <version>2.13.2</version>
</dependency>

This allows us to see log messages from Slf4j enabled libraries. We can also use it directly:

public class ExecutionContext {
    private static final Logger LOGGER =
      LoggerFactory.getLogger(ExecutionContext.class);

    public ExecutionContext() {
        LOGGER.info("Loading configuration");
        // ...
    }

    // ...
}

Slf4j logging is routed through the AWS Log4j runtime:

$ sam local invoke

START RequestId: 60b2efad-bc77-475b-93f6-6fa7ddfc9f88 Version: $LATEST
2021-05-23 21:13:19  INFO  ExecutionContext - Loading configuration

6. Consuming a REST API with Feign

If our Lambda consumes a REST service, we can use the Java HTTP libraries directly. However, there are benefits to using a lightweight framework.

OpenFeign is a great option for this. It allows us to plug in our choice of components for HTTP client, logging, JSON parsing, and much more.

6.1. Adding Feign

We’ll use the Feign default client for this example, though the Java 11 client is also a very good option and works with the Lambda java11 runtime, based on Amazon Corretto.

Additionally, we’ll use Slf4j logging and Gson as our JSON library:

<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-core</artifactId>
    <version>11.2</version>
</dependency>
<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-slf4j</artifactId>
    <version>11.2</version>
</dependency>
<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-gson</artifactId>
    <version>11.2</version>
</dependency>

We’re using Gson as our JSON library here because Gson is much smaller than Jackson. We could use Jackson, but this would make the start-up time slower. There’s also the option of using Jackson-jr, though this is still experimental.

6.2. Defining a Feign Interface

First, we describe the API we’re going to call with an interface:

public interface ToDoApi {
    @RequestLine("GET /todos")
    List<ToDoItem> getAllTodos();
}

This describes the path within the API and any objects that are to be produced from the JSON response. Let’s create the ToDoItem to model the response from our API:

public class ToDoItem {
    private int userId;
    private int id;
    private String title;
    private boolean completed;

    // getters and setters
}

6.3. Defining a Client from the Interface

Next, we use the Feign.Builder to convert the interface into a client:

ToDoApi toDoApi = Feign.builder()
  .decoder(new GsonDecoder())
  .logger(new Slf4jLogger())
  .target(ToDoApi.class, config.getToDoEndpoint());

In our example, we’re also using credentials. Let’s say these are supplied via basic authentication, which would require us to add a BasicAuthRequestInterceptor before the target call:

.requestInterceptor(
   new BasicAuthRequestInterceptor(
     config.getToDoCredentials().getUsername(),
     config.getToDoCredentials().getPassword()))

7. Wiring the Objects Together

Up to this point, we’ve created the configurations and beans for our application, but we haven’t wired them together yet. We have two options for this. Either we wire the objects together using plain Java, or we use some sort of dependency injection solution.

7.1. Constructor Injection

As everything is a plain Java object, and as we’ve built the ExecutionContext class to coordinate construction, we can do all the work in its constructor.

We might expect to extend the constructor to build all the beans in order:

this.config = ... // load config
this.toDoApi = ... // build api
this.postApi = ... // build post API
this.toDoReaderService = new ToDoReaderService(toDoApi);
this.postService = new PostService(postApi);

This is the simplest solution. It encourages well-defined components that are both testable and easy to compose at runtime.

However, above a certain number of components, this starts to become long-winded and harder to manage.

7.2. Bring in a Dependency Injection Framework

DropWizard uses Guice for dependency injection. This library is relatively small and can help manage the components in an AWS Lambda.

Let’s add its dependency:

<dependency>
    <groupId>com.google.inject</groupId>
    <artifactId>guice</artifactId>
    <version>5.0.1</version>
</dependency>

7.3. Use Injection Where It’s Easy

We can annotate beans constructed from other beans with the @Inject annotation to make them automatically injectable:

public class PostService {
    private PostApi postApi;

    @Inject
    public PostService(PostApi postApi) {
        this.postApi = postApi;
    }

    // other functions
}

7.4. Creating a Custom Injection Module

For any beans where we have to use custom load or construction code, we can use a Module as a factory:

public class Services extends AbstractModule {
    @Override
    protected void configure() {
        Config config = 
          ConfigLoader.loadYmlConfigFromResource("configuration.yml", Config.class);

        ToDoApi toDoApi = Feign.builder()
          .decoder(new GsonDecoder())
          .logger(new Slf4jLogger())
          .logLevel(FULL)
          .requestInterceptor(... // omitted
          .target(ToDoApi.class, config.getToDoEndpoint());

        PostApi postApi = Feign.builder()
          .encoder(new GsonEncoder())
          .logger(new Slf4jLogger())
          .logLevel(FULL)
          .requestInterceptor(... // omitted
          .target(PostApi.class, config.getPostEndpoint());

        bind(Config.class).toInstance(config);
        bind(ToDoApi.class).toInstance(toDoApi);
        bind(PostApi.class).toInstance(postApi);
    }
}

Then we use this module inside our ExecutionContext via an Injector:

public ExecutionContext() {
    LOGGER.info("Loading configuration");

    try {
        Injector injector = Guice.createInjector(new Services());
        this.toDoReaderService = injector.getInstance(ToDoReaderService.class);
        this.postService = injector.getInstance(PostService.class);
    } catch (Exception e) {
        LOGGER.error("Could not start", e);
    }
}

This approach scales well, as it localizes bean dependencies to the classes closest to each bean. With a central configuration class building every bean, any change in dependency always requires changes there, too.

We should also note that it’s important to log errors that occur during start-up — if this fails, the Lambda cannot run.

7.5. Using the Objects Together

Now that we have an ExecutionContext with services that have the APIs inside them, configured by the Config, let’s complete our handler:

@Override
public void handleRequest(InputStream inputStream, 
  OutputStream outputStream, Context context) throws IOException {

    PostService postService = executionContext.getPostService();
    executionContext.getToDoReaderService()
      .getOldestToDo()
      .ifPresent(postService::makePost);
}

Let’s test this:

$ sam build
$ sam local invoke

Mounting /Users/ashleyfrieze/dev/tutorials/aws-lambda/todo-reminder/.aws-sam/build/ToDoFunction as /var/task:ro,delegated inside runtime container
2021-05-23 22:29:43  INFO  ExecutionContext - Loading configuration
2021-05-23 22:29:44  INFO  ToDoReaderService - ToDo Endpoint on: https://jsonplaceholder.typicode.com
App starting
Environment: dev
2021-05-23 22:29:44 73264c34-ca48-4c3e-a2b4-5e7e74e13960 INFO  PostService - Posting about: ToDoItem{userId=1, id=1, title='delectus aut autem', completed=false}
2021-05-23 22:29:44 73264c34-ca48-4c3e-a2b4-5e7e74e13960 INFO  PostService - Post: PostItem{title='To Do is Out Of Date: 1', body='Not done: delectus aut autem', userId=1}
END RequestId: 73264c34-ca48-4c3e-a2b4-5e7e74e13960

8. Conclusion

In this article, we looked at the importance of features like configuration and logging when using Java to build an enterprise-grade AWS Lambda. We saw how frameworks like Spring and DropWizard provide these tools by default.

We explored how to use environment variables to control configuration and how to structure our code to make unit testing possible.

Then, we looked at libraries for loading configuration, building a REST client, marshaling JSON data, and wiring our objects together, with a focus on choosing smaller libraries to make our Lambda start as quickly as possible.

As always, the example code can be found over on GitHub.