Introduce MessagePart in AbstractMessage - #6997
Merged
dimitarproynov merged 3 commits intoSep 17, 2026
Merged
dimitarproynov merged 3 commits into
dimitarproynov merged 3 commits into
Conversation
Give AssistantMessage, UserMessage and ToolResponseMessage an ordered List<MessagePart> as their shared representation, so text, reasoning, tool calls, tool results and media can interleave and round-trip in the order a LLM provider produced them. Introduce the MessagePart hierarchy (TextPart, ReasoningPart, ToolCallPart, ToolResultPart, MediaPart, UnknownPart). ReasoningPart and ToolCallPart can carry an opaque, provider-tagged payload (OpaquePayload) for data that must be replayed verbatim, such as an Anthropic thinking signature or a Gemini thought signature. UnknownPart keeps the raw JSON of blocks an adapter does not model yet, so nothing is dropped silently. Parts serialize with Jackson 3 using a "type" discriminator, with Media round-tripping through a DTO that preserves its data kind (bytes, string or URL). The change is additive: existing constructors, builders and accessors (getText(), getToolCalls(), getMedia(), hasToolCalls(), getResponses()) keep working as views over the parts list, and messages built through legacy constructors or setters get their parts in the legacy order regardless of setter call order. Deprecated fields are kept for subclasses that read them directly. A message built without a text part has an empty, not null, text; ToolResponseMessage keeps its historical guarantee of a never-null text. Equality and hashCode now compare parts, which brings media into UserMessage equality. Media therefore gains value equality over id, MIME type and data, with byte arrays compared by content and the name excluded, since it defaults to a random value per instance. Streaming chat models can now deliver a part in increments. StreamingParts stamps a part with its provider content-block index and whether it is a delta; MessageAggregator merges indexed parts per block, appending deltas, replacing complete parts and payloads, and stripping the transport attributes from the aggregated message. Indexed parts are grouped by response id so that aggregating across tool-calling rounds does not merge unrelated groups together, with a heuristic fallback when a model stamps no id. Unindexed parts keep the previous concatenation behavior, including not carrying media on unindexed chunks. AOT runtime hints register the new part types for reflection, and AiRuntimeHints' nested-class discovery no longer recurses forever on nested classes that extend their enclosing Jackson type. Docs gain a Message Parts section and a 2.1.0 upgrade-notes section covering the parts model, behavioral changes, deprecations and the streamToolCallDeltas flag. Signed-off-by: Dimitar Proynov <dimitar.proynov@broadcom.com>
dimitarproynov
requested review from
ericbottard,
ilayaperumalg,
sdeleuze and
tzolov
September 15, 2026 12:52
Member
|
A couple of remarks/questions before I even read the code:
Is that really what we want (in the long run)? My understanding was that is broken (because we lost the interleaving). But maybe I'm not understanding what you're talking about here correctly.
I'm not sure we want to invest too much in that space. I see no reason for those classes to be keys in hashmaps. nor for them to be found in lists, etc by "semantic equality". NOT overriding Object.equals()/hc() should be fine IMO, but maybe I'm missing something here cc @tzolov |
Contributor
Author
In subsequent changes we will migrate all ChatModel implementations to use the MessagePart abstraction and not to use the legacy constructors. They are there to provide backward compatibility.
Equality is used in the isMemoryAlreadyInPrompt in the MessageChatMemoryAdvisor |
ericbottard
reviewed
Sep 16, 2026
Member
👍 |
Addresses review feedback on the MessagePart introduction. AssistantMessage.getToolCalls(), getReasoning() and ToolResponseMessage.getResults() are views over the parts instead of fields cached at construction, so they cannot go stale and every message no longer pays to extract parts the caller may never read. AbstractMessage.textContent is kept, now deprecated for removal, and its readers go through getText() so that removal is a single change later. joinText() moves to the Stream API, keeping an explicit fast path that hands back a lone TextPart's own String instead of copying it. equals() and hashCode() drop the text, which is derived from the parts they already compare; SystemMessage stops overriding them, as its text comparison could never fail once the parent compares parts and its instanceof narrowing made equals asymmetric. ReasoningPart drops the redacted flag, which restated what a null text and a redacted payload kind already carry. Signed-off-by: Dimitar Proynov <dimitar.proynov@broadcom.com>
ericbottard
approved these changes
Sep 16, 2026
ericbottard
left a comment
ericbottard
left a comment
Member
There was a problem hiding this comment.
LGTM for now, let's do a first iteration with this
tzolov
requested changes
Sep 16, 2026
tzolov
left a comment
tzolov
left a comment
Contributor
There was a problem hiding this comment.
please update the chatmodel.adoc and prompt.adoc and likely add a related section to the upgrade-notes.adoc.
Otherwise, it should be good for now!
Seal the MessagePart interface, rework the joinText implementation with Java Streams API Signed-off-by: Dimitar Proynov <dimitar.proynov@broadcom.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Give AssistantMessage, UserMessage and ToolResponseMessage an ordered List as their shared representation, so text, reasoning, tool calls, tool results and media can interleave and round-trip in the order a LLM provider produced them.
Introduce the MessagePart hierarchy (TextPart, ReasoningPart, ToolCallPart, ToolResultPart, MediaPart, UnknownPart). ReasoningPart and ToolCallPart can carry an opaque, provider-tagged payload (OpaquePayload) for data that must be replayed verbatim, such as an Anthropic thinking signature or a Gemini thought signature. UnknownPart keeps the raw JSON of blocks an adapter does not model yet, so nothing is dropped silently. Parts serialize with Jackson 3 using a "type" discriminator, with Media round-tripping through a DTO that preserves its data kind (bytes, string or URL).
The change is additive: existing constructors, builders and accessors (getText(), getToolCalls(), getMedia(), hasToolCalls(), getResponses()) keep working as views over the parts list, and messages built through legacy constructors or setters get their parts in the legacy order regardless of setter call order. Deprecated fields are kept for subclasses that read them directly. A message built without a text part has an empty, not null, text; ToolResponseMessage keeps its historical guarantee of a never-null text.
Equality and hashCode now compare parts, which brings media into UserMessage equality. Media therefore gains value equality over id, MIME type and data, with byte arrays compared by content and the name excluded, since it defaults to a random value per instance.
Streaming chat models can now deliver a part in increments. StreamingParts stamps a part with its provider content-block index and whether it is a delta; MessageAggregator merges indexed parts per block, appending deltas, replacing complete parts and payloads, and stripping the transport attributes from the aggregated message. Indexed parts are grouped by response id so that aggregating across tool-calling rounds does not merge unrelated groups together, with a heuristic fallback when a model stamps no id. Unindexed parts keep the previous concatenation behavior, including not carrying media on unindexed chunks.
AOT runtime hints register the new part types for reflection, and AiRuntimeHints' nested-class discovery no longer recurses forever on nested classes that extend their enclosing Jackson type. Docs gain a Message Parts section and a 2.1.0 upgrade-notes section covering the parts model, behavioral changes, deprecations and the streamToolCallDeltas flag.
Build and tests passed.
Integration tests for OpenAI - passed, others IT runs pending