Keep JetBrains streaming overload errors out of assistant output

[sanchitmonga22]

Overview

Impact: High under overload. Priority: P1. Estimated effort: Medium.

The JetBrains OpenAI proxy converts streaming upstream429/503 refusals into an ordinary successful assistant answer (finish_reason:"stop", [DONE], HTTP200). The editor cannot reliably recognize overload or respect Retry-After, and it may retain the failure text in conversation as a model answer.

Current state and reproduction

Audited main 093cd44806ec64b2c79b42b557b4856259d2ea58:

• Streaming request commits chunked response before upstream result.
• Failure deliberately becomes content+stop+DONE.
• Nonstream preserves429 and Retry-After.

Hermetic loopback probe compiled unchanged openai_proxy.cpp and loopback_auth.cpp with pinned cpp-httplib0.46.1/nlohmann3.11.3. Only output and account/profile helpers were stubbed; no real credentials or API calls. Mock upstream /v1/chat/completions returned429 with Retry-After: 7 and {"error":{"message":"Too many parallel requests"}}. Authenticated request to real local proxy used stream:true.

Actual output:

JETBRAINS upstream429 -> status=200 retry_after= body=data: {"id":"chatcmpl-wally","object":"chat.completion.chunk","created":0,"model":"wally","choices":[{"index":0,"delta":{"role":"assistant","content":"{ error :{ message : Too many parallel requests }}"},"finish_reason":"stop"}]}

data: [DONE]

This proves transport/error rewriting. Exact retry/conversation behavior inside each supported JetBrains client remains to be qualified.

Proposed solution and implementation

• Do not classify upstream failure as successful assistant completion.
• Resolve upstream status/headers before committing downstream200 for pre-stream failures, preserving Retry-After and a compatible error body.
• For a failure after output starts, use a typed failure contract accepted by the supported editor, with explicit incomplete-stream behavior; do not silently emit stop/DONE.
• Add real loopback tests covering429/503 before tokens, refusal during streaming, auth refresh path, and normal stream success.
• Qualify the supported JetBrains client's handling against the fake overload endpoint.

Success criteria

Overload remains distinguishable from a successful inference, delay metadata survives, no failure text is stored/executed as model output, and an incomplete stream does not turn into a normal stop. Public API testing is deferred until access is supplied.

Related: #83 is the Anthropic shim's analogous metadata loss, but that shim at least emits a typed error rather than assistant content. #80/#82 cover connection pooling in this same proxy. This issue adds no release-blocker label; release triage is deferred.