Leveraging Quarkus and LangChain4j

1. Overview

In this tutorial, we’ll learn about LangChain, a framework for developing applications powered by large language models (LLMs). More specifically, we’ll use langchain4j, a Java framework that simplifies the integration with LangChain and allows developers to integrate LLMs into their applications.

This framework is very popular for building Retrieval-Augmented Generation (RAG). In this article, we’ll understand all those terms and see how to leverage Quarkus to build such an application.

2. Large Language Models (LLMs)

Large language models (LLMs) are AI systems trained with massive amounts of data. Their goal is to generate human-like output. GPT-4 from OpenAi is probably one of the most well-known LLMs used nowadays. However, LLMs can answer questions and perform various natural language processing tasks.

Such models are the backbone for power chatbots, content generation, document analyses, video and image generation, and more.

2.1. LangChain

LangChain is a popular open-source framework that helps developers build LLM-powered applications. Initially developed for Python, it simplifies the process of creating multi-step workflows using LLMs such as:

• Integration with different LLMs: LangChain does not serve its own LLMs but rather provides a standard interface for interacting with many different LLMs, such as GPT-4, Lama-3, and more
• Integrate external tools: LangChain enables integration with other tools that are very important for such applications, like vector databases, web browsers, and other APIs
• Manage memory: Applications that require conversational flows need history or context persistence. LangChain has memory capabilities, allowing models to “remember” information across interactions

LangChain is one of the go-to frameworks for Python developers building AI applications. However, for Java developers, the LangChain4j framework offers a Java-like adaptation of this robust framework.

2.2. LangChain4j

LangChain4j is a Java library inspired by LangChain, designed to help build AI-powered applications using LLMs. The project creators aim to fill the gap between Java frameworks and the numerous Python and Javascript options for AI systems.

LangChain4j gives Java developers access to the same tools and flexibility that LangChain provides to Python developers, enabling the development of applications such as chatbots, summarization engines, or intelligent search systems.

Powerful applications can be quickly built with other frameworks, such as Quarkus.

2.3. Quarkus

Quarkus is a Java framework designed for cloud-native applications. Its ability to significantly reduce memory usage and startup time makes it highly suitable for microservices, serverless architectures, and Kubernetes environments.

Incorporating Quarkus into your AI applications ensures that your systems are scalable, efficient, and ready for production at an enterprise level. Moreover, Quarkus offers an easy-to-use and low learning curve to integrate LangChain4j into our Quarkus applications.

3. ChatBot Application

In order to demonstrate the power of Quarkus integration with LangChain4j, we’ll create a simple chatbot application to answer questions about Quarkus. For this, let’s use GPT-4. This will help us since it is trained with a lot of data from the internet, and therefore, we will not need to train our own model.

The application will only respond to questions regarding Quarkus and will remember the previous conversation in the particular chat.

3.1. Dependencies

Now that we have introduced the main concepts and tools, let’s build our first application using LangChain4j and Quarkus. First things first, let’s create our Quarkus application using Rest and Redis and then add the following dependencies:

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-openai</artifactId>
    <version>0.18.0.CR1</version>
</dependency>

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-memory-store-redis</artifactId>
    <version>0.18.0.CR1</version>
</dependency>

The first step is to add the latest quarkus-langchain4j-openai and quarkus-langchain4j-memory-store-redis dependencies to our pom.xml

The first dependency will introduce the LangChain4J version, which is compatible with Quarkus. Moreover, it’ll give us out-of-the-box components that configure LangChain4J to use OpenAI LLM models as the backbone of our service. There are many LLMs to choose from, but for the sake of simplicity, let’s use the current default, which is GPT-4o-mini. In order to use such a service, we need to create an ApiKey in OpenAI. Once this is done, we can proceed. We’ll talk about the second dependency later.

3.2. Setup

Now, let’s set up our application to allow it to communicate with GPT-4 and Redis so we can start implementing our chatbot. For that, we need to configure some application properties in Quarkus, as it has out-of-the-box components to do so. We only need to add the following properties to the application.properties file:

quarkus.langchain4j.openai.api-key=${OPEN_AI_KEY}
quarkus.langchain4j.openai.organization-id=${OPEN_AI_ORG}

quarkus.redis.hosts=${REDIS_HOST}

Quarkus has many other valuable configurations for this application. Please refer to Quarkus documentation; however, this will be enough for our chatbot.

Optionally, we can also use environment variables to set such properties as:

QUARKUS_REDIS_HOSTS: redis://localhost:6379
QUARKUS_LANGCHAIN4J_OPENAI_API_KEY: changeme
QUARKUS_LANGCHAIN4J_OPENAI_ORGANIZATION_ID: changeme

3.3. Components

LangChain4j offers a rich set of attractions to help us implement complex flows, such as document retrieval. Nonetheless, in this article, we’ll focus on a simple conversation flow. Having said that, let’s create our simple chatbot:

@Singleton
@RegisterAiService
public interface ChatBot {

    @SystemMessage("""
    During the whole chat please behave like a Quarkus specialist and only answer directly related to Quarkus,
    its documentation, features and components. Nothing else that has no direct relation to Quarkus.
    """)
    @UserMessage("""
    From the best of your knowledge answer the question below regarding Quarkus. Please favor information from the following sources:
    - https://docs.quarkiverse.io/
    - https://quarkus.io/
    
    And their subpages. Then, answer:
    
    ---
    {question}
    ---
    """)
    String chat(@MemoryId UUID memoryId, String question);
}

In the ChatBot class, we register a service that will use the LLM during the conversation. To do that, Quarkus offers the @RegisterAiService annotation, which abstracts the setup to integrate LangChain4J and GTP-4. Next, we apply some Prompt Engineering to structure how we want the Chatbot to behave.

In short, Prompt Engineering is a technique for shaping instructions that will help the LLMs adapt their behavior during the interpretation and processing of user requests.

So, using such prompts, we implemented the desired behavior for our bot. LangChain4J uses messages to do that, whereas:

• SystemMessage are internal instructions that influence the model responses but are hidden from the end-user;
• UserMessage are messages sent by the end-user. However, LangChain4J helps us by allowing us to apply templates on top of such messages to enhance the prompting.

Let’s discuss about @MemoryId next.

3.4. Memory

Despite being very good at generating text and providing relevant answers to questions, LLMs by themselves are not enough to implement chatbots. A critical aspect that LLMs cannot do themselves is to remember the context or data from previous messages. That is why we need memory capabilities.

LangChain4J offers a subset of abstract ChatMemoryStore and ChatMemory. These abstractions allow different implementations to store and manage the message list that represents a chat. Additionally, an identifier is needed to store and retrieve the chat memory, and @MemoryId is used to mark such ID.

Quarkus offers an out-of-the-box Redis-based implementation for chat memory, which is why we added the second dependency and the Redis setup.

3.5. API

Finally, the piece missing from our sample chatbot application is an interface to enable users to communicate with the system. Let’s create an API so users can send their questions to our chatbot.

@Path("/chat")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class ChatAPI {
    
    private ChatBot chatBot;

    public ChatAPI(ChatBot chatBot) {
        this.chatBot = chatBot;
    }

    @POST
    public Answer mesage(@QueryParam("q") String question, @QueryParam("id") UUID chatId) {

        chatId = chatId == null ? UUID.randomUUID() : chatId;

        String message = chatBot.chat(chatId, question);

        return new Answer(message, chatId);
    }

}

The API receives a question and, optionally, a chat ID. If the ID is not provided, we create it. This allows users to control whether they want to continue an existing chat or create one from scratch. Then, it sends both parameters to our ChatBot class, which integrates the conversation. Optionally, we could implement an endpoint to retrieve all messages from a chat.

Running the command below will send our first question to our chatbot.

# Let's ask: Does quarkus support redis?
curl --location --request POST 'http://localhost:8080/chat?q=Does%20quarkus%20support%20redis%3F'

So we get:

{
    "message": "Yes, Quarkus supports Redis through the Quarkus Redis client extension...",
    "chatId": "d3414b32-454e-4577-af81-8fb7460f13cd"
}

Note that since this was our first question, no ID was provided, so a new one was created. Now, we can use this id to keep track of our conversation history. Next, let’s test if the Chatbot is keeping our history as context for our conversation.

# Replace the id param with the value we got from the last call. 
# So we ask: q=What was my last Quarkus question?&id=d3414b32-454e-4577-af81-8fb7460f13cd
curl --location --request POST 'http://localhost:8080/chat?q=What%20was%20my%20last%20Quarkus%20question%3F&id=d3414b32-454e-4577-af81-8fb7460f13cd'

Excellent. As expected, our chatbot replied correctly.

{
    "message": "Your last Quarkus question was asking whether Quarkus supports Redis.",
    "chatId": "d3414b32-454e-4577-af81-8fb7460f13cd"
}

That means our simple chatbot application is ready only after creating a couple of classes and setting some properties.

4. Conclusion

Using Quarkus and LangChain4J simplifies the process of building a Quarkus-based chatbot that interacts with OpenAI’s language models and retains conversational memory. This powerful combination of Quarkus and LangChain4j enables the development of AI-powered applications with minimal overhead while still providing rich capabilities.

From this setup, we can continue to extend the chatbot’s capabilities by adding more domain-specific knowledge and improving its ability to answer complex questions. Quarkus and Langchain4j still provide many other tools for this.

In this article, we saw how much productivity and efficiency that combination of Quarkus and LangChain4J brings to the table for the development of AI systems using Java. Moreover, we presented the concepts involved in the development of such applications.