Make MCP rate limits configurable and raise the global ceiling to 128 - #14
Merged
Merged
Conversation
Per-client limits now protect fairness between chat sessions, so the process ceiling is what binds under load. Default the global concurrency limit to 128, size anyio's synchronous tool thread pool from it (its default of 40 would otherwise cap throughput silently), and read all four limits from PWC_MCP_*_LIMIT environment variables for other deployments. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With per-chat-session identities (#13), the per-client limits already protect fairness; the process-wide ceiling of 32 concurrent requests is what binds when many chat turns run at once (each turn issues bursts of three parallel catalog calls). This raises the default global ceiling to 128, sizes anyio's synchronous tool thread pool from it (anyio's default of 40 tokens would otherwise cap throughput silently below the ceiling), and makes all four limits overridable:
PWC_MCP_REQUEST_LIMITPWC_MCP_SEMANTIC_LIMITPWC_MCP_CONCURRENCY_LIMITPWC_MCP_GLOBAL_CONCURRENCY_LIMITInvalid values fail at startup. README and SPEC updated. Pairs with huggingface/paperswithcode#252 (64-turn chat capacity tier).
Verification
uv run --project mcp_server pytest mcp_server/tests: 45 passed, including env parsing, defaults, thread-pool sizing, and a middleware-level check that the configured limits reach the rate limiter.🤖 Generated with Claude Code