Fix streaming request failover to retry on error status codes#7
Merged
Conversation
此前状态码级别的故障转移被 `!retryableStreamRequest` 一刀切挡掉, 导致流式请求(stream: true)即使在开流前收到 429 / 5xx,也不会按 priority 降级到下一个渠道,而是直接把错误透传给客户端(issue #6)。 错误状态码是在响应体转发给客户端之前返回的(随后会 cancel 掉上游 body),此时重试 / 回退是安全的;真正不可重试的是“已经开始流式响应体 后才出错”,而那种情况状态码是 2xx,本就不会命中 failover。因此去掉该 前置条件即可,无需新增开关。 新增两条流式 failover 端到端测试:开流前 429 重试成功、5xx 按 priority 回退到下一渠道。 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B1f8CF7YQmp9EU3m8yJYFr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the failover behavior for streaming requests to properly retry and fall back when the upstream returns error status codes before any response body is streamed to the client.
Key Changes
!retryableStreamRequestcondition that was preventing status-based failover for streaming requests. Error status codes are returned before the response body is streamed, making retries safe at that point.Implementation Details
The key insight is that when an upstream returns an error status code (4xx, 5xx), the response headers and status are sent before any body content. For streaming requests, this means the error status arrives before the first byte of the stream is sent to the client, so the response can be safely cancelled and retried/fallen back to another provider. Only after a successful (2xx) response has started streaming should retries be disabled, but those cases won't trigger failover anyway since they have success status codes.
https://claude.ai/code/session_01B1f8CF7YQmp9EU3m8yJYFr