feat(gateway): promote system messages to developer role#181
Open
M1k0t0 wants to merge 5 commits into
Open
Conversation
…-to-developer-simple
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Codex exposes only a Responses endpoint and does not accept
role: "system"messages inside Responsesinput. Floway can nevertheless produce such messages when requests arrive through native Responses, Chat Completions, or Anthropic Messages translation.Add a provider-owned
promote-system-to-developerflag and enable it by default for Codex. The gateway uses the flag to normalize system-role messages to developer-role messages before they reach the provider. The Codex provider then moves only the contiguous leading developer prefix into top-levelinstructions, while preserving later developer messages in their original chronological positions.This matches the request shape produced by the Codex client, which represents base instructions using either top-level
instructionsor leadingrole: "developer"input items:https://github.com/openai/codex/blob/1f17e7512f0e47625f2cad416f14870688a99814/codex-rs/core/src/client.rs#L829-L849
Implementation
Provider-owned behavior flag
Add the following optional provider behavior:
The flag means that system-role messages must be represented as developer-role messages when sent toward the selected upstream.
Provider defaults remain exhaustive:
ProviderModelreturned by Codex carries the resolved effective flag set.This keeps the compatibility decision provider-owned. A generic or OpenAI-compatible upstream is not assumed to support developer messages merely because Floway's canonical Chat Completions type supports the role.
Chat Completions source interceptor
Add
withPromoteSystemToDeveloperto the Chat Completions interceptor chain.When the selected candidate enables the flag, it rewrites every message with:
{ "role": "system" }to:
{ "role": "developer" }The promotion runs before the existing demotion compatibility interceptors. If an operator intentionally combines conflicting flags, the existing ordered chain remains authoritative:
That ordering continues to support stricter non-Codex upstream combinations.
Responses target interceptor
Add the corresponding
withPromoteSystemToDeveloperinterceptor to the Responses chain.When enabled, it walks the Responses
inputarray and rewrites only input items satisfying both conditions:This interceptor covers two different request shapes:
It runs before the existing developer/system demotion and interleaved-system compatibility stages, ensuring Codex-bound system messages become developer messages rather than being converted into user text.
Anthropic Messages behavior
The existing Messages-to-Responses translator retains ownership of normal protocol mapping.
A string or single text-block Messages top-level
systemfield maps directly to Responsesinstructions:This is existing generic translation behavior and does not use the new flag.
A multi-block Messages top-level
systemcannot be placed into the scalar Responsesinstructionsfield without losing block boundaries. The translator therefore preserves it as one leading Responses message containing oneinput_textpart per source block:Messages history entries with
role: "system"similarly become Responses system input messages during translation. The target Responses interceptor then promotes them to developer. If they occur after user or assistant history, the Codex boundary leaves them inline rather than moving them ahead of that history.Gemini behavior
The existing Gemini-to-Responses translator maps
systemInstructiondirectly to Responsesinstructions.No role conversion is needed for that field because it does not become a system-role input message. Any system-role Responses input introduced by other translated content would still be covered by the target Responses interceptor.
Codex provider boundary
Add a Codex-only Responses boundary interceptor before default-instruction injection and unsupported-field stripping.
The resulting order is:
Leading-prefix detection
The hoist considers only a contiguous prefix of Responses input items that are:
The scan stops at the first item that is:
This preserves chronology. A developer message after a user, assistant, reasoning, tool, or other input item remains exactly where the caller placed it.
Text representation
A developer message can be hoisted when its content is:
input_textandoutput_textparts.Messages containing images or other non-text content are not partially flattened. The scan stops before the unrepresentable item, and that item plus the remainder of input stays unchanged.
Text parts belonging to the same instruction message are joined with blank lines when lowered into scalar
instructions.Instruction merging
Hoisted text is merged in this order:
Each non-empty instruction segment is separated by
\n\n.For example:
{ "instructions": "caller instructions", "input": [ { "type": "message", "role": "developer", "content": "translated base instructions" }, { "type": "message", "role": "user", "content": "hello" } ] }becomes:
{ "instructions": "caller instructions\n\ntranslated base instructions", "input": [ { "type": "message", "role": "user", "content": "hello" } ] }The merge preserves the Responses distinction between omitted,
null, and empty instructions when there is no non-empty text to add. The following default-instructions interceptor remains responsible for replacing a missing, null, or empty final value with the Codex-required fallback.This scalar flattening remains local to
provider-codex. Generic translators continue preserving source block structure until a provider boundary requires a less expressive wire representation.Compatibility
The
developerrole is part of the current Chat Completions protocol, but not every OpenAI-compatible implementation necessarily accepts it. Promotion is therefore flag-gated rather than globally enabled.Only Codex enables the flag by default. Other providers retain their previous behavior unless an operator explicitly enables the override.
This change does not alter:
system → instructionstranslation;systemInstruction → instructionstranslation;