LLM Integration With Apache Camel OpenAI Component

1. Overview

The Apache Camel OpenAI component, introduced in Camel 4.17, enables integration of Java applications with OpenAI or OpenAI-compatible LLM (Large Language Model) servers such as Ollama or vLLM. Basically, it uses Camel routes for interaction with LLMs and handles message conversion between Camel Exchanges and OpenAI requests. It uses the OpenAI Java API library under the hood.

In this tutorial, we’ll discuss LLM integration with the Apache Camel OpenAI component and see several examples.

2. Maven Dependencies

We have to add the following Maven dependency to our pom.xml:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-openai</artifactId>
    <version>4.18.0</version>
</dependency>

3. Basic Interaction With an LLM

The component uses the standard Camel URI (Uniform Resource Identifier) format: openai:operation[?options]. The openai part of the URI is fixed.

The operation part corresponds to the URI’s path parameter. Two operations are supported: chat-completion and embeddings. chat-completion, which is of interest to us, generates chat responses using a model. The other operation, embeddings, is useful for creating vector embeddings, for example, in RAG (Retrieval-Augmented Generation) pipelines.

The options part corresponds to the URI’s query parameters. These are generally configuration options such as model choices, generation settings, and authentication parameters. For example, the model option specifies which LLM to use. We can also pass these options using message headers instead of the URI.

3.1. Chat Application

In this section, we’ll examine a simple chat application that interacts with an LLM. The application sends a prompt to the model and displays the model’s response.

Let’s start by using the Apache Camel’s Main class to configure, run, and handle the lifecycle of a standalone Apache Camel application:

Main main = new Main();
main.configure()
  .addRoutesBuilder(new ChatRoute());
main.start();

The addRoutesBuilder() method registers a custom routing class, ChatRoute, that implements the routing rules. The routing logic includes how Apache Camel takes a prompt, wraps it in an HTTP call to an LLM, and processes the response. The start() method of the Main class starts the runtime, instantiates the routes, and begins processing prompts.

Then, let’s send a message to an endpoint using the ProducerTemplate class:

ProducerTemplate template = main.getCamelContext()
    .createProducerTemplate();
String aiResponse = template.requestBody("direct:startChat", userPrompt, String.class);

We use the requestBody() method of ProducerTemplate to pass the prompt to the route. The first argument, “direct:startChat”, is the endpoint that the LLM exposes. direct: is an Apache Camel component providing synchronous invocation of routes. startChat is the name we give to the endpoint.

Our application uses this endpoint to communicate with the LLM. We pass the second argument, userPrompt, to the LLM and store its response in the variable aiResponse. The third argument, String.class, tells Apache Camel that we expect the response to be of type String. Therefore, the type of aiResponse is String.

3.2. The Routing Class

Next, let’s discuss the routing class. We extend the ChatRoute class from the RouteBuilder class to create routing rules using the Java DSL (Domain-Specific Language):

public class ChatRoute extends RouteBuilder {

    @Override
    public void configure() throws Exception {

        onException(Exception.class).handled(true)
          .log(org.apache.camel.LoggingLevel.INFO, "OpenAI Error: ${exception.message}")
          .setBody(simple("OpenAI request failed."));

        from("direct:startChat")
          .setHeader(
            OpenAIConstants.SYSTEM_MESSAGE,
            constant("You are a helpful, brief IT support assistant."))
          .setHeader(OpenAIConstants.TEMPERATURE, constant(0.1))
          .setHeader(OpenAIConstants.MODEL, constant("gpt-4o-mini"))
          .to("openai:chat-completion")
          .log("AI Response: ${body}");
    }
}

We override the configure() method, which acts as the entry point. from(“direct:startChat”) defines a synchronous endpoint named startChat. As we saw in the previous subsection, applications can send user prompts to this endpoint by calling template.requestBody(“direct:startChat”, userPrompt, String.class).

The three setHeader() calls attach OpenAI-specific message headers by injecting the following key-value pairs:

• SYSTEM_MESSAGE sets the system prompt to “You are a helpful, brief IT support assistant.” and therefore defines the behavior and tone of the model.
• MODEL picks gpt-4o-mini as the target LLM.
• TEMPERATURE sets the sampling temperature to 1, thereby making the model more deterministic.

to(“openai:chat-completion”) is the actual call to the OpenAI component. It extracts the message body (i.e., the user prompt) and the headers, performs the OpenAI chat completion request, and waits for the response.

Instead of using message headers above, we can also use the Camel URI. For example, we can set the model and temperature options using the URI:

.to("openai:chat-completion?model=gpt-4o-mini&temperature=0.1")

Finally, log(“AI Response: ${body}”) prints the model’s response.

We must authenticate our application while making requests to OpenAI’s developer platform. One method for authentication is to set the confidential OpenAI API key to the OPENAI_API_KEY environment variable from the terminal we’ll run the application in Linux:

$ export OPENAI_API_KEY=<our_confidential_actual_key_here>

Let’s see the response of the LLM when we send the prompt, “Give me information about Baeldung”:

AI Response: Baeldung is a popular online platform that provides tutorials and articles primarily focused on Java, 
Spring Framework, and related technologies.
...
The content is well-structured and often includes practical examples, 
making it a valuable resource for both beginners and experienced developers.

The output shows that we received a successful response to our prompt. We abbreviated the response as it’s long.

4. Using Other OpenAI-Compatible Endpoints

The Apache Camel OpenAI component allows us to switch easily to other OpenAI-compatible endpoints using the baseUrl option. The endpoint can point to local or third-party providers. For example, we just need to change the router code as follows to use a local Ollama server:

.setHeader(OpenAIConstants.MODEL, constant("qwen3.5:9b"))
.to("openai:chat-completion?baseUrl=http://localhost:11434/v1")

The default value of baseUrl is https://api.openai.com/v1. We used the default OpenAI endpoint in the previous section. Therefore, we didn’t specify it explicitly. However, by setting baseUrl to http://localhost:11434/v1, we send the requests to the local Ollama server instead of OpenAI’s servers.

We also set the model to qwen3.5:9b in our example. Before sending prompts to the model, the model must have been run locally using the ollama run command. We don’t need to use a real API key while using Ollama locally. But an API key is still required. So we can set it to a dummy value for compatibility with OpenAI. But third-party providers over the cloud, like Ollama’s cloud service, require a real API key.

5. Keeping Conversational Memory

By default, conversational memory isn’t retained by the OpenAI component; i.e., OpenAI doesn’t remember the responses to previous prompts. For example, if we tell an LLM our name in one prompt and then ask for it in the next prompt, it will usually respond that it doesn’t know our name.

To maintain conversational memory with the OpenAI component, we need to make two changes to the previous example. Firstly, we must reuse the same Exchange object on the application side:

Exchange exchange = template.getCamelContext()
  .getEndpoint("direct:startChatWithMemory")
  .createExchange();

for (String prompt : prompts) {
    exchange.getMessage()
      .setBody(prompt);
    template.send("direct:startChatWithMemory", exchange);
                
aiResponses.add(exchange.getMessage()
  .getBody(String.class));

Once the Exchange object is created, namely exchange in the code snippet above, we use the same object in a for loop for processing the prompts. The prompts variable is a list of strings containing the prompts:

List<String> prompts = List.of("My name is Burak and I am a Java developer.", "What is my name and what do I do?"
);

The first prompt specifies the user’s name and job, and the second prompt asks them.

Secondly, we must set the conversationMemory option to true on the router side:

.to("openai:chat-completion?conversationMemory=true")

When we run the application, the response to the first prompt, “My name is Burak and I am a Java developer.”, is as follows:

Hi Burak! How can I assist you today with your Java development?

Let’s see the response to the second prompt, “What is my name and what do I do?”:

Your name is Burak, and you are involved in Java development. How can I assist you further?

Therefore, reusing the same Exchange object while setting the conversationMemory option to true enables us to persist storage between calls.

6. Providing Structured Output

The OpenAI component can also present the LLM’s response in a deterministic format. For example, we can map the response to a POJO (Plain Old Java Object), which can be directed safely to a database. Let’s discuss an example that returns a book’s name, author, and summary:

public static BookInfo runChat(String userPrompt) throws Exception {
    ...
    BookInfo bookInfo = template.requestBody("direct:startBookChat", userPrompt, BookInfo.class);
    return bookInfo;
    ...
}

The runChat() member function in the above code snippet takes a user prompt as a string and passes it to the router by calling the requestBody() method of ProducerTemplate. The third argument of requestBody(), BookInfo.class, specifies the response to be a BookInfo object. The BookInfo class consists of three public member variables:

public class BookInfo {

    public String name;
    public String author;
    public String summary;
}

We also need to update the router class:

from("direct:startBookChat")
  .setHeader(OpenAIConstants.SYSTEM_MESSAGE, constant("You are a helpful, brief book advisor."))
  .setHeader(OpenAIConstants.MODEL, constant("gpt-4o-mini"))
  .setHeader(OpenAIConstants.TEMPERATURE, constant(0.1))
  .to("openai:chat-completion?outputClass=com.baeldung.apachecamel.BookInfo")
  .unmarshal()
  .json(JsonLibrary.Jackson, BookInfo.class)
  .log("AI Response: ${body}");

Here, we use the outputClass option to instruct the LLM to respond in JSON format that matches the BookInfo schema. The next calls, unmarhal() and json(JsonLibrary.Jackson, BookInfo.class), use the Apache Camel JSON Jackson component. They convert the raw JSON string in the message body and deserialize it to BookInfo.

Let’s see an example of asking for the author and summary of a book, The Time Regulation Institute:

String userPrompt = "Can you provide the author and summary of the following book: The Time Regulation Institute";
BookInfo book = runChat(userPrompt);
LOGGER.info("Book: {}", book.name);
LOGGER.info("Author: {}", book.author);
LOGGER.info("Summary: {}", book.summary);

Here is the abbreviated output:

Book: The Time Regulation Institute
Author: Ahmet Hamdi Tanpınar
Summary: A satirical novel that explores the absurdities of modernity and the clash between 
tradition and progress in Turkish society, focusing on the establishment of a time regulation 
institute and its impact on the characters' lives.

The output shows that the response was successfully mapped to the Java class BookInfo.

7. Conclusion

In this article, we discussed LLM integration with the Apache Camel OpenAI component.

Firstly, we learned how to send prompts to an LLM using Camel routes and implemented a chat completion application based on them. Then, we saw how to change the OpenAI-compatible endpoint. We also learned to maintain conversational memory so that the model can remember previous prompts throughout a chat. Finally, we discussed formatting an LLM’s responses in a predefined format.

Besides the applications we discussed, there are other use cases. For example, we can process an LLM’s responses chunk by chunk rather than waiting for the full completion of a response. We have to set the streaming option to true. Another example is the support for multi-modal inputs. We can use text files and images with vision-capable models.

As usual, the complete source code for the examples is available over on GitHub.