Repository navigation
Conversation
added 2 commits
October 15, 2026 00:55
The access log only had TRANSPORT/TIMEOUT, lumping connection resets,
deadlines and other transport failures together, so 'unstable' could never be
attributed to upstream reset vs local disconnect vs a slow exit.
Add CodeReset ('TRANSPORT_RESET') and route the laneHTTP.Do failure through
classifyConnErr: net.Error.Timeout() -> CodeTimeout, reset/broken-pipe/
ECONNRESET -> CodeReset, else CodeTransport. CodeReset is added to every
behavior switch that handles CodeTransport (egressRetryable, switchableFailure,
the per-exit report class, the gateway 502 mapping), so retry/switch/budget
behavior is unchanged; egressBudgetFor already returns the slow budget for
anything but CodeQuota, so CodeReset is covered there too. The paths that RAISE
CodeTransport for an upstream cutting a stream (kilo/workbuddy/relay/probe)
are left alone.
订阅节点的出网(egress)在中途死亡、但本地 socks 入站仍然应答时,请求能 建立连接(能连接)却永远回不来(不能用)——每次尝试会一直挂到 responseHeaderTimeout(90s) 才超时。此前这类故障只写入 3~60s 的 (exit,model) 封禁且按模型键控,节点的 Alive 健康位从不随实时请求失败 下调;而 ProbeDead 每 2 分钟只复查 !Alive 的节点,于是死出口要等到下一轮 完整 ProbeAll(~30 分钟) 才会被发现,期间每个走它的请求都先白等 90s。 修复:把实时传输层失败反馈回节点健康。 - 新增 ReportDead(nodeID):连续 2 次传输层失败(CodeTimeout/CodeTransport/ CodeReset,在 failMemory 窗口内) 将节点 Alive 置为 false。 - Alive=false 立即让 Pick/nextNode 排除该节点,同时使其进入 ProbeDead 的 复查范围——真死的出口一两个请求内离场,仅偶发卡顿的会在下一轮探针自愈。 - Revive(真实请求成功) 重置失败计数并把 Alive 重新置 true(真实流量即活证)。 - 5xx 与空 2xx 体不算出口死亡(连接明明成功了),只走(exit,model) 封禁。 依赖前置提交对传输错误的 reset/timeout/other 细分,才能精确触发。 测试:新增 ReportDead 两次失败降级、记忆窗口重置计数、Revive 重新拉起死 节点,均通过;subs/lane 全量测试绿。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题背景
用户手动用「其他设备端 → 自定义订阅」时,出网(egress)在中途死亡、但本地 socks 入站仍然应答的节点会表现为:能建立连接(能连接),但请求永远回不来(不能用),用着用着突然卡死。
根因(与早先判断不同,过完代码后发现是实时反馈缺失):
Alive健康位只由后台探针路径写入,实时请求的传输层失败(CodeTimeout/CodeTransport/CodeReset)只写入 3~60s 的(exit,model)封禁且按模型键控,节点的Alive从不随实时失败下调。ProbeDead每 2 分钟只复查!Alive的节点;而ProbeAll约 30 分钟才一轮。于是死出口要等最长 ~30 分钟才会被发现。responseHeaderTimeout=90s才超时 —— 这就是反复“能连接但不能用”的卡顿来源。修复
把实时传输层失败反馈回节点级健康:
Rotator.ReportDead(nodeID):(exit,model)之外新增节点级失败计数,在failMemory记忆窗口(10min)内连续 2 次传输层失败 → 节点Alive=false。Alive=false立刻让Pick/nextNode排除该节点,同时使其进入ProbeDead复查范围 —— 真死的出口一两个请求内离场,偶发卡顿的会在下一轮探针自愈。Revive(真实请求成功)重置失败计数并把Alive重新置true(真实流量即活证)。CodeServer) 与空 2xx 体(CodeEmpty) 视为连接已成功,不算出口死亡,只走(exit,model)封禁。依赖
本 PR 建在
6c6b4e4 lane/gateway: split transport errors into reset / timeout / other(同分支内)之上 —— 只有把传输错误细分为 reset/timeout/other,才能精确触发本修复的降级逻辑。测试
internal/subs/ledger_test.go:新增ReportDead连续两次失败降级、failMemory窗口外重置计数、Revive重新拉起死节点。internal/lane/rotate_test.go:countRotatormock 补ReportDead以满足接口。go test ./internal/...全绿。关联