Prompt Caching Support in Spring AI with Anthropic Claude

1. Overview

When working with large prompts, repeatedly sending the same context to the model can increase both latency and costs. This becomes especially noticeable when applications reuse large system instructions, documents, or conversation context across requests.

Prompt Caching in Spring AI with Anthropic addresses this by allowing frequently reused parts of a prompt to be cached and reused across requests, reducing the amount of work the model needs to process each time. This can improve response latency while lowering input-token costs.

In this tutorial, we’ll explain how prompt caching works, the limitations and requirements for different Claude models, and how to use it with Spring AI. We’ll also cover the main configuration options and practical considerations when applying caching in an application.

2. Dependencies

Let’s start by defining the minimum dependencies possible for demonstrating the Prompt Caching in Spring AI. We’ll only need spring-boot-starter-web and the spring-ai-starter-model-anthropic. The first is for the basic Spring Boot Application and auto-configuration. The latter contains the tools we’re exploring in this article, including Spring AI. The minimum version for Prompt Caching is 1.0.3, so we can go with:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>3.5.13</version>
</dependency>
<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-anthropic</artifactId>
    <version>2.0.1</version>
</dependency>

 3. Prompt Caching

Let’s start by understanding what Prompt Caching is. Prompt Caching allows Claude to reuse a previously processed prefix of a prompt instead of processing the same content from scratch on every request. This is particularly useful for large system instructions, tool definitions, documents, and other context that remains unchanged across requests.

The main benefit we get is reduced latency and input-token cost. Anthropic currently charges cache reads at 10% of the regular input-token price for most Claude models. While the initial cache write costs 25% more than the base input price for a 5-minute cache, the savings increase as the same prompt prefix is reused.

There are, however, some limitations to consider:

• the minimum cacheable prompt size depends on the model
• the cache uses a limited TTL
• we can define only up to four cache breakpoints
• changes we make to the cached prefix can invalidate the cache
• a cache entry is also only available after the first response begins, which matters when sending concurrent requests

Next, we need to understand the Prompt Caching Hierarchy for other concepts, like Strategies, to make sense. Prompt caching follows the hierarchy: tools → system → messages. Claude processes these sections in this order, and each level builds on the previous one.

Invalidation also follows this hierarchy. If we change the tools, this invalidates the tools cache and everything below it. If we change the system prompt, this leaves the tools cache intact but invalidates the system and message caches. When we change messages, only the message cache gets invalidated.

The idea is therefore simple: keep stable content as high in the hierarchy as possible and frequently changing content as low as possible. This allows the largest possible portion of the prompt to remain cached between requests.

4. Prompt Caching Strategies

The hierarchy and invalidation rules naturally lead to different caching strategies. The goal is to place cache breakpoints where they provide the most reuse while minimizing the amount of content that gets invalidated when something changes.

A cache breakpoint tells Anthropic where to create a cache entry. Each breakpoint caches the prompt content up to that point, following the request hierarchy of tools → system → messages. Anthropic currently allows a maximum of four cache breakpoints per request, so they should be placed deliberately rather than on every possible section.

Spring AI provides five Prompt Caching Strategies through AnthropicCacheStrategy:

The number and placement of breakpoints also affect invalidation. With a single breakpoint at the system message, for example, a change to the tools invalidates the whole cached prefix. With SYSTEM_AND_TOOLS, Spring AI places separate breakpoints after the tools and system message, allowing the tool cache to remain valid when only the system prompt changes.

Choosing the right strategy therefore depends on how stable each part of the request is. The more frequently a section changes, the more carefully its breakpoint should be separated from stable content.

5. Prompt Caching In Practice

Last, let’s put what we’ve learned so far into practice. First, we’ll apply the theory in a Spring Boot application, and then we’ll run some tests to see prompt caching in action.

5.1. Spring Boot Application with Prompt Caching

As we’ve seen before, we can use Spring properties to set up the model:

spring:
  ai:
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        options:
          model: claude-sonnet-4-6
          max-tokens: 500
        cache-options:
          strategy: SYSTEM_ONLY

At this moment, claude-sonnet models before 3.5 are no longer available. We set 4-6 and max-tokens to 500 as a guardrail to control the cost, and we’ll use SYSTEM_ONLY as the strategy in our application. The minimum cacheable prompt for claude-sonnet-4-6 is 1024 tokens.

Then, all we need is a service using this model:

public class ChatWithPromptCachingService {

    private final ChatClient chatClient;

    // ... constructors, etc

    public ChatResponseWithMetadataDto chat(String userMessage, String systemPrompt) {
        ChatResponse response = chatClient
          .prompt()
          .system(systemPrompt)
          .user(userMessage)
          .call()
          .chatClientResponse()
          .chatResponse();

        if (response == null || response.getResult() == null) {
            throw new RuntimeException("Client response has no results");
        }

        return ChatResponseWithMetadataDto.fromChatResponse(response);
    }
}

The chat() method accepts a system prompt and a user message and requests our chat-client. Then, it returns the ChatResponseWithMetadataDto:

public record ChatResponseWithMetadataDto(
  String responseText,
  Integer promptTokens,
  Integer completionTokens,
  Long cacheReadInputTokens,
  Long cacheWriteInputTokens) {
    public static ChatResponseWithMetadataDto fromChatResponse(ChatResponse chatResponse) {
        // ... implementation of the mapper
    }
}

ChatResponseWithMetadataDto only holds the information we need: responseText, promptTokens, completionTokens, cacheReadInputTokens, and cacheWriteInputTokens.

5.2. Testing the Prompt Caching Application

To make sure that prompts are truly cached, let’s first have a test with strategy set to NONE and run the test:

@Test
void chat_whenPromptCachingDisabled_returnsResponse() {
    String chatMessage = "hello there";

    ChatResponseWithMetadataDto response = service.chat(chatMessage, PromptsUtils.LONG_SYSTEM_PROMPT);

    assertThat(response).isNotNull();
    assertThat(response.promptTokens()).isGreaterThan(1030);
    assertThat(response.cacheReadInputTokens()).isEqualTo(0);

    response = service.chat(chatMessage + " again", PromptsUtils.LONG_SYSTEM_PROMPT);

    assertThat(response).isNotNull();
    assertThat(response.promptTokens()).isGreaterThan(1030);
    assertThat(response.cacheReadInputTokens()).isEqualTo(0);
}

In this test, we request a dummy message and the default PromptsUtils.LONG_SYSTEM_PROMPT, which is just a bit longer than 1024 tokens. As expected, for caching strategy NONE, we see that the prompt tokens are a full 1030 tokens, and there is no cached token recorded.

Next, let’s try the same test with the default caching strategy of our application (SYSTEM_ONLY):

@Test
void chat_whenPromptCachingEnabled_returnsResponse() {
    String chatMessage = "hello there";
    UUID testId = UUID.randomUUID();

    ChatResponseWithMetadataDto response = service.chat(
      chatMessage,
      PromptsUtils.LONG_SYSTEM_PROMPT + "\nTEST_ID=" + testId);

    assertThat(response).isNotNull();
    assertThat(response.promptTokens()).isLessThan(50);
    assertThat(response.cacheWriteInputTokens()).isGreaterThan(1000);
    assertThat(response.cacheReadInputTokens()).isEqualTo(0);


    response = service.chat(chatMessage + " again", PromptsUtils.LONG_SYSTEM_PROMPT + "\nTEST_ID=" + testId);

    assertThat(response).isNotNull();
    assertThat(response.promptTokens()).isLessThan(50);
    assertThat(response.cacheWriteInputTokens()).isEqualTo(0);
    assertThat(response.cacheReadInputTokens()).isGreaterThan(1000);
}

Now we see Prompt Caching effects! In the first request, the response tells us that cacheWriteInputTokens is over 1000, which means it just wrote to the cache. Moreover, promptTokens is low, just the user message, and cacheReadInputTokens is 0 cause there was no cache hit.

In the second request, promptTokens is low, but this time cacheWriteInputTokens is 0 (no new tokens cached). CacheReadInputTokens is now over 1000, which shows there was a cache hit!

The prompt in this test also uses a TEST_ID to make sure no previous tests will affect the next execution. It’s a trick to use different caches between test executions.

6. Conclusion

In this article, we explained how Prompt Caching Support in Spring AI with Anthropic Claude works. We walked through how it works, configuration options, and the limitations. Then we saw the Prompt Caching Strategies. Finally, we used Spring AI to demonstrate it in practice.

As always, the source code of the examples can be found over on GitHub.