Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion articles/flow/ai-support/listeners.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The response listener is called when the turn ends -- normally when the assistan
The event carries:

* [methodname]`getResponse()` -- the assistant's response text. On success, it may be empty when the model emitted only tool calls; on failure, it's always empty -- partial text received before the error isn't passed on, and the Message List shows a generic error message in its place. Empty responses are not appended to the conversation history.
* [methodname]`getError()` -- the failure cause, or an empty optional on success.
* [methodname]`getError()` -- the failure cause, or an empty optional on success. A turn stopped at a <<llm-providers#tool-call-limits,tool call limit>> fails with a [classname]`ToolCallLimitExceededException`.
* [methodname]`getMetadata()` -- the provider's metadata for the turn; see <<#response-metadata,Response Metadata>>.

A typical use is persisting the conversation after each exchange -- see <<conversation-history#,Conversation History & Session Persistence>>.
Expand Down
40 changes: 40 additions & 0 deletions articles/flow/ai-support/llm-providers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,46 @@ The provider manages its own conversation memory using a 30-message window.
Synchronous mode blocks the UI for the duration of each exchange; see <<#background-execution,Background Execution>>.


[[tool-call-limits]]
[role="since:com.vaadin:vaadin@V25.3"]
== Tool Call Limits

A turn that uses <<tool-calling#,tools>> is a loop: the model asks for tool calls, the provider runs them and calls the model again with the results, until the model answers. Both built-in providers bound that loop per turn. By default, the model may call any single tool at most 40 times and all tools together at most 150 times in one turn. Without a bound, a request the model can't satisfy keeps calling the model and the tools until the application is stopped.

When a limit is exceeded, the turn fails with a [classname]`ToolCallLimitExceededException`, whichever provider runs it. The Message List shows its generic error message, and the exception reaches the <<listeners#,response listener>> and [methodname]`AIController.onResponse()` as the error of the turn. Its message names the limit that was exceeded and, for a per-tool limit, the tool:

[source,java]
----
.withResponseListener(event -> {
event.getError()
.filter(ToolCallLimitExceededException.class::isInstance)
.ifPresent(error -> log.warn(
"Turn stopped: {}", error.getMessage()));
})
----

=== LangChain4j

[classname]`LangChain4JLLMProvider` runs the loop itself and enforces the limits. Change them with [methodname]`setMaxCallsPerTool()` and [methodname]`setMaxTotalToolCalls()`. A value of `0` removes that limit. The values are read when a turn starts, so a change applies from the next prompt.

[source,java]
----
LangChain4JLLMProvider provider = new LangChain4JLLMProvider(chatModel);
provider.setMaxCallsPerTool(10);
provider.setMaxTotalToolCalls(0); // No limit across all tools
----

The provider checks the limits before running the tool calls of a response. When a call would take a count past its limit, none of the tool calls in that response run, and the response isn't added to the provider's memory, so the next turn starts from a consistent conversation.

=== Spring AI

[classname]`SpringAILLMProvider` adds no limit of its own. Spring AI bounds the loop with the same defaults, 40 calls per tool and 150 in total per turn, with either constructor of the provider. When Spring AI stops the loop, the provider fails the turn with the [classname]`ToolCallLimitExceededException` described above, carrying Spring AI's own message about the limit. That message may remain in the chat memory, with either constructor: the provider doesn't rewrite what Spring AI's advisors stored.

The limits belong to the [classname]`ChatClient`, so tune or remove them there and pass the client to the [classname]`ChatClient` constructor of the provider; the [classname]`ChatModel` constructor builds a client with Spring AI's defaults. How to configure them is described under https://docs.spring.io/spring-ai/reference/api/tools.html#tool-call-limits[Tool Call Limits] in the Spring AI documentation.

A <<#custom-llm-providers,custom provider>> decides itself whether, and how, it bounds its loop.


[[background-execution]]
[role="since:com.vaadin:vaadin@V25.3"]
== Background Execution
Expand Down
4 changes: 4 additions & 0 deletions articles/flow/ai-support/tool-calling.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,10 @@ For a reusable set of tools that does not depend on a specific LLM framework's a
[NOTE]
Tool objects registered via [methodname]`withTools()` are executed by the vendor framework, whose own error handling decides what the LLM sees when a tool throws -- by default, both LangChain4j and Spring AI relay the raw message of any exception. To control what the LLM learns about failures, define the tool through a controller instead and throw a [classname]`ToolException` for messages the LLM is meant to see; see <<controllers#tool-error-handling,Tool Error Handling>>.

.Tool Call Limits
[NOTE]
The tool-calling loop of a turn is bounded: by default, the model may call any single tool at most 40 times and all tools together at most 150 times in one turn. Exceeding a limit fails the turn with a [classname]`ToolCallLimitExceededException`, with either built-in provider. See <<llm-providers#tool-call-limits,Tool Call Limits>>.


== Programmatic Prompts

Expand Down
Loading