Spring AI’s Dynamic Tool Discovery

1. Overview

When we build AI-integrated systems, we often provide our AI clients with a large number of tools. On every request, we send the definitions of all available tools to the LLM so it can decide which ones to use. As a result, we waste a significant number of tokens before the model even processes the user query.  In this article, we explore how we solve this issue using the Tool Search Tool. 

2. How the Tool Search Tool Works

Using the Tool Search Tool, we don’t send all the tool definitions with the context. We only expose tools when the model actually needs them. First, we index all registered tools at startup. We store them inside the ToolSearcher, but we do NOT send them to the LLM. Next, we send only the Tool Search Tool in the initial request. This keeps the prompt small and focused. When the model needs a capability, it calls the Tool Search Tool using a natural-language query.

We treat this as a discovery signal and trigger a search over the indexed tools using the configured strategy. Next, we return only the most relevant matches from the ToolSearcher and inject their definitions into the next LLM request, so the model sees a focused set of tools instead of the full registry.

Once the relevant tools are available, the model selects and calls the actual tool. We execute it and send the result back to the LLM, which then uses it to generate the final answer.

3. Building a Travel Assistant Example

Let’s build a travel assistant that helps users plan trips. We connect multiple tools such as flights, hotels, weather, and attractions. We use the Tool Search Tool approach to avoid sending all tools to the LLM upfront. Instead, we discover tools dynamically at runtime.

3.1. Dependencies

We start by adding tool search dependency support:

<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>tool-search-tool</artifactId>
    <version>${tool-search-tool.version}</version>
</dependency>

Also, let’s add the regex searcher dependency:

<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>tool-searcher-regex</artifactId>
    <version>${tool-search-tool.version}</version>
</dependency>

Using it, we’ll have a regex tool search strategy. The other available strategies can be found in the project repository.

3.2. Flight Tools

Let’s create a simple FlightTools. We’ll use this tool to retrieve available flight options. In addition, we’ll create a bunch of artificial tools to simulate context overloading:

public class FlightTools {
    @Tool(description = "Searches available flights between two cities")
    public List<FlightOption> searchFlights(String from, String to, String departureDate) {

        return List.of(
          new FlightOption(
            "Romania Airlines",
            from,
            to,
            departureDate,
            249.99
          )
        );
    }
}

Here we return a single flight option.

3.3. TokenCounterAdvisor

Now let’s create a simple TokenCounterAdvisor that counts the number of tokens used to produce the final result. We’ll use it to compare token usage between different setups, with and without tool search enabled:

public class TokenCounterAdvisor implements BaseAdvisor {
    private static final Logger log = LoggerFactory.getLogger(TokenCounterAdvisor.class);

    private final AtomicInteger totalTokenCounter = new AtomicInteger(0);

    @Override
    public String getName() {
        return "TokenCounterAdvisor";
    }

    @Override
    public int getOrder() {
        return Ordered.LOWEST_PRECEDENCE - 1;
    }

    @Override
    public ChatClientRequest before(ChatClientRequest chatClientRequest, AdvisorChain advisorChain) {
        return chatClientRequest;
    }

    @Override
    public ChatClientResponse after(ChatClientResponse chatClientResponse, AdvisorChain advisorChain) {
        var usage = chatClientResponse.chatResponse().getMetadata().getUsage();

        totalTokenCounter.addAndGet(usage.getTotalTokens());

        log.info("Total tokens spent: {}", totalTokenCounter.get());

        return chatClientResponse;
    }
}

Here we store the number of tokens in an AtomicInteger field and log this information during execution. We attach this advisor to the maximum order, so it runs at the end of the processing pipeline. As a result, it captures the total token usage after all other advisors complete.

3.4. Configuration

Next, we add the TravelAssistantConfig implementation:

@Configuration
public class TravelAssistantConfig {

    @Bean
    ToolSearcher toolSearcher() {
        return new RegexToolSearcher();
    }

    @Bean
    ToolSearchToolCallAdvisor toolSearchToolCallAdvisor(ToolSearcher toolSearcher) {
        return ToolSearchToolCallAdvisor.builder()
          .toolSearcher(toolSearcher)
          .maxResults(5)
          .build();
    }

    @Bean
    ChatClient chatClient(ToolSearchToolCallAdvisor toolSearchToolCallAdvisor, OpenAiChatModel model) {
        return ChatClient.builder(model)
          .defaultTools(
            new FlightTools(),
            new RandomTools()
          )
          .defaultAdvisors(toolSearchToolCallAdvisor, new TokenCounterAdvisor())
          .build();
    }

    @Bean
    ChatClient chatClientWithoutToolsSearch(OpenAiChatModel model) {
        return ChatClient.builder(model)
          .defaultTools(
            new FlightTools(),
            new RandomTools()
          )
          .defaultAdvisors(new TokenCounterAdvisor())
          .build();
    }
}

We configure a travel assistant that uses dynamic tool discovery instead of loading all tools into the LLM. Next, we set up a ToolSearcher with a RegexToolSearcher implementation. This allows us to match tools based on naming patterns and fast keyword-like queries. Then, we create a ToolSearchToolCallAdvisor and connect it to the searcher. After that, we build the ChatClient with the flight tools registered.

By design, we’ve added RandomTools, which includes many unrelated tool definitions.  However, we do not send these tool definitions to the LLM initially. Instead, we only index them in the system. Finally, we expose only the Tool Search Tool to the model at the start. The model then uses it to discover which tools it actually needs for a given request. Additionally, we’ve configured a separate ChatClient bean that doesn’t use the ToolSearchToolCallAdvisor.

3.5. Call the TravelAssistant

Finally, let’s create a ToolsSearchToolLiveTest with similar test cases for both clients:

@SpringBootTest
@ActiveProfiles("toolsearchtool")
class ToolsSearchToolLiveTest {
    @Autowired
    private ChatClient chatClient;

    @Autowired
    private ChatClient chatClientWithoutToolsSearch;

    @Test
    void shouldFindFlightsBetweenRomaniaAndCroatiaUsingToolsSearch() {
        String response = getClientResponseString(chatClient);
        assetClientResponse(response);
    }

    @Test
    void shouldFindFlightsBetweenRomaniaAndCroatiaWithoutToolsSearch() {
        String response = getClientResponseString(chatClientWithoutToolsSearch);
        assetClientResponse(response);
    }
    
    private static void assetClientResponse(String response) {
        assertThat(response).isNotBlank();
        assertThat(response).containsIgnoringCase("Croatia");
        assertThat(response).containsIgnoringCase("flight");
    }

    private String getClientResponseString(ChatClient chatClientWithoutToolsSearch) {
        return chatClientWithoutToolsSearch.prompt()
          .user("""
                  Find available flights from Romania to Croatia next week.
                  """)
          .call()
          .content();
    }
}

We’ve called our travel advisor clients with the same prompt and obtained the same verified results. Now, let’s compare the token usage in both of them:

[2026-05-24 11:39:07] [INFO] [c.b.s.t.TokenCounterAdvisor] - Total tokens spent: 974 //With tools search tool
[2026-05-24 11:39:10] [INFO] [c.b.s.t.TokenCounterAdvisor] - Total tokens spent: 3685 //Without tools search tool

As we can see, the difference in token usage is crucial. The more tools we have in our system, the greater the token savings the Tool Search Tool will provide.

4. Conclusion

In this article, we reviewed the Tool Search Tool and demonstrated how it helps reduce token usage in real scenarios. Using it, we can build large AI-integrated systems with hundreds of attached tools and use them efficiently, without wasting tokens. Additionally, we can explore other tool search strategies, such as vector search, or even build our own custom strategy to make tool discovery even more efficient.

As always, the code is available over on GitHub.