You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 16f8b00
Browse filesBrowse the repository at this point in the historyBrowse files
fix(llm): honour Retry-After and stop retrying in lockstep
Found by running brain init and the chat agent against a real rate-limited
Azure OpenAI deployment; neither failure is visible without a live provider.
Embeddings ignored the delay the service asked for
Azure replied 429 "retry after 4 seconds" while the local schedule slept
750ms then 1500ms, so all three attempts landed inside the cooldown. The
document was then dropped as "transient/non-fatal" and brain init still
reported COMPLETED — a partially embedded corpus with no error anywhere, the
silent RAG degradation CLAUDE.md warns about. Retrying early is not just a
wasted attempt: the attempt is itself counted by the limiter.
classify() already answered whether to retry; nothing answered when. Those
are separate questions and only the provider can answer the second, since
the delay arrives in a vendor-specific header or error string. So the
embedding SPI gains a default retryAfter(Throwable) returning Optional.empty,
implemented for OpenAI/Azure by reading Retry-After and falling back to the
delay stated in the body ("retry after 4 seconds", "try again in 1.5s").
EmbeddingService waits for whichever is longer, its own schedule or the
provider's, capped at 30s because the number is remote input.
Chat retried without jitter
INITIAL_BACKOFF_MS * (1L << attempt) is deterministic, so parallel agent
calls rejected together woke together and collided again — the retries
recreated the burst they were backing off from. Half the wait is now
randomised, with Retry-After as a floor and a 30s ceiling.
Measured over a live re-init: 33 waits honoured, 0 documents skipped, where
the previous build skipped repeatedly. Init is slower now because it waits
instead of giving up, which is the trade that was previously hidden.
Tests: 83 pass across the LLM layer, including a deterministic-backoff
regression check and cap/floor/negative handling for both paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
0 commit comments