Skip to content

feat(gateway): promote system messages to developer role#181

Open
M1k0t0 wants to merge 5 commits into
Menci:mainfrom
M1k0t0:floway/codex-system-to-developer-simple
Open

feat(gateway): promote system messages to developer role#181
M1k0t0 wants to merge 5 commits into
Menci:mainfrom
M1k0t0:floway/codex-system-to-developer-simple

Conversation

@M1k0t0

@M1k0t0 M1k0t0 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Codex exposes only a Responses endpoint and does not accept role: "system" messages inside Responses input. Floway can nevertheless produce such messages when requests arrive through native Responses, Chat Completions, or Anthropic Messages translation.

Add a provider-owned promote-system-to-developer flag and enable it by default for Codex. The gateway uses the flag to normalize system-role messages to developer-role messages before they reach the provider. The Codex provider then moves only the contiguous leading developer prefix into top-level instructions, while preserving later developer messages in their original chronological positions.

This matches the request shape produced by the Codex client, which represents base instructions using either top-level instructions or leading role: "developer" input items:

https://github.com/openai/codex/blob/1f17e7512f0e47625f2cad416f14870688a99814/codex-rs/core/src/client.rs#L829-L849

Implementation

Provider-owned behavior flag

Add the following optional provider behavior:

promote-system-to-developer

The flag means that system-role messages must be represented as developer-role messages when sent toward the selected upstream.

Provider defaults remain exhaustive:

  • Codex enables the flag by default.
  • Azure, Claude Code, Copilot, custom, and Ollama default it off.
  • Operator overrides continue to participate in the normal provider flag resolution.
  • Every ProviderModel returned by Codex carries the resolved effective flag set.

This keeps the compatibility decision provider-owned. A generic or OpenAI-compatible upstream is not assumed to support developer messages merely because Floway's canonical Chat Completions type supports the role.

Chat Completions source interceptor

Add withPromoteSystemToDeveloper to the Chat Completions interceptor chain.

When the selected candidate enables the flag, it rewrites every message with:

{ "role": "system" }

to:

{ "role": "developer" }

The promotion runs before the existing demotion compatibility interceptors. If an operator intentionally combines conflicting flags, the existing ordered chain remains authoritative:

system → developer → system → user

That ordering continues to support stricter non-Codex upstream combinations.

Responses target interceptor

Add the corresponding withPromoteSystemToDeveloper interceptor to the Responses chain.

When enabled, it walks the Responses input array and rewrites only input items satisfying both conditions:

item.type === "message"
item.role === "system"

This interceptor covers two different request shapes:

  1. Native Responses requests, where system-role items already exist in the inbound payload.
  2. Requests translated from another protocol, where the translator produced a Responses system-role input message.

It runs before the existing developer/system demotion and interleaved-system compatibility stages, ensuring Codex-bound system messages become developer messages rather than being converted into user text.

Anthropic Messages behavior

The existing Messages-to-Responses translator retains ownership of normal protocol mapping.

A string or single text-block Messages top-level system field maps directly to Responses instructions:

Messages payload.system
  → Messages-to-Responses translator
  → Responses instructions

This is existing generic translation behavior and does not use the new flag.

A multi-block Messages top-level system cannot be placed into the scalar Responses instructions field without losing block boundaries. The translator therefore preserves it as one leading Responses message containing one input_text part per source block:

Messages multi-block system
  → leading multipart Responses system message
  → Responses system-to-developer interceptor
  → leading multipart developer message
  → Codex provider instruction hoist

Messages history entries with role: "system" similarly become Responses system input messages during translation. The target Responses interceptor then promotes them to developer. If they occur after user or assistant history, the Codex boundary leaves them inline rather than moving them ahead of that history.

Gemini behavior

The existing Gemini-to-Responses translator maps systemInstruction directly to Responses instructions.

No role conversion is needed for that field because it does not become a system-role input message. Any system-role Responses input introduced by other translated content would still be covered by the target Responses interceptor.

Codex provider boundary

Add a Codex-only Responses boundary interceptor before default-instruction injection and unsupported-field stripping.

The resulting order is:

1. hoistLeadingDeveloperToInstructions
2. injectDefaultInstructions
3. stripUnsupportedFields
4. Codex transport

Leading-prefix detection

The hoist considers only a contiguous prefix of Responses input items that are:

type: "message"
role: "developer"

The scan stops at the first item that is:

  • not a message;
  • not developer-role; or
  • not representable as plain instruction text.

This preserves chronology. A developer message after a user, assistant, reasoning, tool, or other input item remains exactly where the caller placed it.

Text representation

A developer message can be hoisted when its content is:

  • a string; or
  • an array containing only input_text and output_text parts.

Messages containing images or other non-text content are not partially flattened. The scan stops before the unrepresentable item, and that item plus the remainder of input stays unchanged.

Text parts belonging to the same instruction message are joined with blank lines when lowered into scalar instructions.

Instruction merging

Hoisted text is merged in this order:

existing top-level instructions
leading developer message 1
leading developer message 2
...

Each non-empty instruction segment is separated by \n\n.

For example:

{
  "instructions": "caller instructions",
  "input": [
    {
      "type": "message",
      "role": "developer",
      "content": "translated base instructions"
    },
    {
      "type": "message",
      "role": "user",
      "content": "hello"
    }
  ]
}

becomes:

{
  "instructions": "caller instructions\n\ntranslated base instructions",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": "hello"
    }
  ]
}

The merge preserves the Responses distinction between omitted, null, and empty instructions when there is no non-empty text to add. The following default-instructions interceptor remains responsible for replacing a missing, null, or empty final value with the Codex-required fallback.

This scalar flattening remains local to provider-codex. Generic translators continue preserving source block structure until a provider boundary requires a less expressive wire representation.

Compatibility

The developer role is part of the current Chat Completions protocol, but not every OpenAI-compatible implementation necessarily accepts it. Promotion is therefore flag-gated rather than globally enabled.

Only Codex enables the flag by default. Other providers retain their previous behavior unless an operator explicitly enables the override.

This change does not alter:

  • direct Messages string/single-block system → instructions translation;
  • direct Gemini systemInstruction → instructions translation;
  • generic translator block-preservation rules;
  • non-Codex provider defaults;
  • non-message Responses input items;
  • non-leading developer-message chronology;
  • client-facing response protocols.

@M1k0t0 M1k0t0 marked this pull request as ready for review July 11, 2026 23:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant