1. Overview
Spring Framework officially enabled the power of AI generative prompts with the Spring AI project. In this tutorial, we’ll provide a robust introduction to the generative AI integration in Spring Boot applications, and familiarize ourselves with the essential AI concepts.
We’ll also gain an understanding of how Spring AI interacts with the models, and create an application to demonstrate its capabilities.
2. Spring AI Main Concepts
Before we start, let’s review some key domain terms and concepts.
Spring AI initially focused on models designed to handle language input and generate language output. The idea behind the project was to provide developers with an abstract interface, the foundation for enabling generative AI APIs into the application as an isolated component.
One such abstraction is the interface AiClient, which has two basic implementations, OpenAI and Azure OpenAI:
public interface AiClient {
default String generate(String message);
AiResponse generate(Prompt prompt);
}
AiClient provides two options for the generative function. The simplified one, generate(String message), uses String as input and output, and could be used to avoid the extra complexity of the Promt and AiResponse classes.
Now let’s take a closer look at their differences.
2.1. Advanced Prompt and AiResponse
In the AI domain, prompt refers to a text message provided to AI. It consists of the context and question, and that model is used for the answer generation.
From the Spring AI project perspective, the Prompt is a list of parametrized Messages:Â
public class Prompt {
private final List<Message> messages;
// constructors and utility methods
}
public interface Message {
String getContent();
Map<String, Object> getProperties();
MessageType getMessageType();
}
Prompt enables developers to have more control over the text input. A good example is the prompt templates, constructed with a predefined text and set of placeholders. Then we can populate them with the Map<String, Object> values passed to the Message constructor:
Tell me a {adjective} joke about {content}.
The Message interface also holds advanced information about the categories of messages that an AI model can process. For example, OpenAI implementation distinguishes between conversational roles, effectively mapped by the MessageType. In other models, it could reflect the message format, or some other custom properties. For more details, please refer to the official documentation:
public class AiResponse {
private final List<Generation> generations;
// getters and setters
}
public class Generation {
private final String text;
private Map<String, Object> info;
}
The AiResponse consists of the list of Generation objects, each holding output from the corresponding prompt. In addition, the Generation object provides the metadata information of the AI response.
However, while the Spring AI project is still in beta, not all the features are finished and documented. We can follow the progress with the issues on the GitHub repository.
3. Getting Started With the Spring AI Project
First of all, AiClient requires the API key for all communications with the OpenAI platform. For that, we’ll create a token on the API Keys page.
The Spring AI project defines the configuration property spring.ai.openai.api-key. We can set it up in the application.yml file:
spring:
ai:
openai.api-key: ${OPEN_AI_KEY}
The next step is to configure a dependency repository. The Spring AI project provides artifacts in the Spring Milestone Repository.
Therefore, we’ll need to add the repository definition:
<repositories>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<releases>
<enabled>false</enabled>
</releases>
</repository>
</repositories>
After that, we’re ready to import open-ai-spring-boot-starter:
<dependency>
<groupId>org.springframework.experimental.ai</groupId>
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
<version>0.7.1-SNAPSHOT</version>
</dependency>
Please keep in mind that the Spring AI project is actively evolving, so check the official GitHub page for the latest version.
Now let’s put the concept into practice.
4. Spring AI in Action
Let’s write a simple REST API for demonstration purposes. It’ll consist of two endpoints that return poetry on whatever theme and genre we’d like:
- /ai/cathaiku — will implement the basic generate() method and return a plain string value with Haiku about cats
- /ai/poetry?theme={{theme}}&genre={{genre}} — will demonstrate the capabilities of the PromtTemplate and AiResponse classes
4.1. Injecting AiClient in Spring Boot Application
To keep things simple, let’s start with the cat haiku endpoint. With the @RestController annotation, we’ll set up PoetryController and add GET method mapping:
@RestController
@RequestMapping("ai")
public class PoetryController {
private final PoetryService poetryService;
// constructor
@GetMapping("/cathaiku")
public ResponseEntity<String> generateHaiku(){
return ResponseEntity.ok(poetryService.getCatHaiku());
}
}
Next, following the DDD concept, the service layer will define all domain logic. All we need to do to call the generate() method is inject the AiClient into the PoetryService. Now we can define the String prompt, where we’ll specify our request to generate the Haiku:
@Service
public class PoetryServiceImpl implements PoetryService {
public static final String WRITE_ME_HAIKU_ABOUT_CAT = """
Write me Haiku about cat,
haiku should start with the word cat obligatory""";
private final AiClient aiClient;
// constructor
@Override
public String getCatHaiku() {
return aiClient.generate(WRITE_ME_HAIKU_ABOUT_CAT);
}
}
The endpoint is up and ready to receive the requests. The response will contain a plain string:
Cat prowls in the night,
Whiskers twitch with keen delight,
Silent hunter's might.
It looks good so far; however, the current solution has a few pitfalls. The response of plain string isn’t the best solution for REST contracts in the first place.
Furthermore, there’s not much value in querying ChatGPT with the same old prompt all the time. So our next step is to add the parametrized values: theme and genre. That’s when PromtTemplate will serve us best.
4.2. Configurable Queries With PromptTemplate
In its nature, PromptTemplate works quite similar to a combination of StringBuilder and dictionary. Similar to the /cathaiku endpoint, we’ll first define the base string for the prompt. In contrast, this time we’ll define the placeholders populated with actual values by their names:
String promptString = """
Write me {genre} poetry about {theme}
""";
PromptTemplate promptTemplate = new PromptTemplate(promptString);
promptTemplate.add("genre", genre);
promptTemplate.add("theme", theme);
Next, we may want to standardize the endpoint output. For that, we’ll introduce the simple record class, PoetryDto, which will contain the poetry title, name, and genre:
public record PoetryDto (String title, String poetry, String genre, String theme){}
A further step is to register PoetryDto in the BeanOutputParser class; it provides functionality to serialize and deserialize OpenAI API output.
Then we’ll provide this parser to the promtTemple, and from now on, our messages will be serialized into the DTO objects.
In the end, our generative function will look like this:
@Override
public PoetryDto getPoetryByGenreAndTheme(String genre, String theme) {
BeanOutputParser<PoetryDto> poetryDtoBeanOutputParser = new BeanOutputParser<>(PoetryDto.class);
String promptString = """
Write me {genre} poetry about {theme}
{format}
""";
PromptTemplate promptTemplate = new PromptTemplate(promptString);
promptTemplate.add("genre", genre);
promptTemplate.add("theme", theme);
promptTemplate.add("format", poetryDtoBeanOutputParser.getFormat());
promptTemplate.setOutputParser(poetryDtoBeanOutputParser);
AiResponse response = aiClient.generate(promptTemplate.create());
return poetryDtoBeanOutputParser.parse(response.getGeneration().getText());
}
The response our client receives now looks much better, and more importantly, it fits into the REST API standards and best practices:
{
"title": "Dancing Flames",
"poetry": "In the depths of night, flames dance with grace,
Their golden tongues lick the air with fiery embrace.
A symphony of warmth, a mesmerizing sight,
In their flickering glow, shadows take flight.
Oh, flames so vibrant, so full of life,
Burning with passion, banishing all strife.
They consume with ardor, yet do not destroy,
A paradox of power, a delicate ploy.
They whisper secrets, untold and untamed,
Their radiant hues, a kaleidoscope unnamed.
In their gentle crackling, stories unfold,
Of ancient tales and legends untold.
Flames ignite the heart, awakening desire,
They fuel the soul, setting it on fire.
With every flicker, they kindle a spark,
Guiding us through the darkness, lighting up the dark.
So let us gather 'round, bask in their warm embrace,
For in the realm of flames, magic finds its place.
In their ethereal dance, we find solace and release,
And in their eternal glow, our spirits find peace.",
"genre": "Liric",
"theme": "Flames"
}
5. Error Handling
Spring AI project provides an abstraction over OpenAPI errors with the OpenAiHttpException class. Unfortunately, it doesn’t provide individual mapping of classes per error type. However, thanks to such abstraction, we can handle all exceptions with RestControllerAdvice in one handler.
The code below uses the ProblemDetail standard of the Spring 6 Framework. To become more familiar with it, please check the official documentation:
@RestControllerAdvice
public class ExceptionTranslator extends ResponseEntityExceptionHandler {
public static final String OPEN_AI_CLIENT_RAISED_EXCEPTION = "Open AI client raised exception";
@ExceptionHandler(OpenAiHttpException.class)
ProblemDetail handleOpenAiHttpException(OpenAiHttpException ex) {
HttpStatus status = Optional
.ofNullable(HttpStatus.resolve(ex.statusCode))
.orElse(HttpStatus.BAD_REQUEST);
ProblemDetail problemDetail = ProblemDetail.forStatusAndDetail(status, ex.getMessage());
problemDetail.setTitle(OPEN_AI_CLIENT_RAISED_EXCEPTION);
return problemDetail;
}
}
Now if the OpenAPI response contains errors, we’ll handle it:
{
"type": "about:blank",
"title": "Open AI client raised exception",
"status": 401,
"detail": "Incorrect API key provided: sk-XG6GW***************************************wlmi.
You can find your API key at https://platform.openai.com/account/api-keys.",
"instance": "/ai/cathaiku"
}
The complete list of possible exception statuses is on the official documentation page.
6. Conclusion
In this article, we familiarized ourselves with the Spring AI Project and its capabilities in the context of REST APIs. Despite the fact that, at the time this article was written, spring-ai-starter remained in active development and was accessible in a snapshot version, it provided a reliable interface for generative AI integration into the Spring Boot application.
In the context of this article, we covered both basic and advanced integrations with Spring AI, including how the AiClient works under the hood. As the proof of concept, we implemented a basic REST application that generates poetry. Along with a basic example of a generative endpoint, we provided a sample using advanced Spring AI features: PromtTemplate, AiResponse, and BeanOutputParser. In addition, we implemented the error handling functionality.
The complete examples are available over on GitHub.
