Improve MCP agent workflows and evaluation metadata

backend/chat_sandbox_worker/CLI_GUIDE.md

@@ -6,6 +6,7 @@ join, or schema-dependent transformation.
 ```bash
 pwc search "QUERY" --limit 10 [--start-date YYYY-MM-DD --end-date YYYY-MM-DD]
 pwc paper info PAPER --include-resources
+pwc paper evaluations PAPER --page 1 --page-size 20
 pwc paper read PAPER
 pwc paper list --search "QUERY" [--start-date YYYY-MM-DD --end-date YYYY-MM-DD] [--task NAME] [--method NAME] [--conference NAME] [--framework NAME] [--organization NAME]
 pwc paper recent

mcp_server/README.md

@@ -3,9 +3,9 @@
 Anonymous, read-only Model Context Protocol access to the public
 [Papers With Code](https://paperswithcode.co) catalog.
 
-The server uses MCP `2026-07-28` over Streamable HTTP and serves legacy
-2025-era clients on the same `/mcp` endpoint. It returns versioned structured
-output with compact text fallbacks.
+The server uses stock-client MCP `2025-11-25` over Streamable HTTP and also
+serves experimental `2026-07-28` discovery on the same `/mcp` endpoint. It
+returns versioned structured output with compact Markdown fallbacks.
 
 ## Run locally
 
@@ -31,6 +31,7 @@ to typed projections such as `items` or `evaluations`.
 | --- | --- |
 | `search_papers` | `pwc search` |
 | `get_paper_info` | `pwc paper info` |
+| `get_paper_evaluations` | `pwc paper evaluations` |
 | `read_paper` | `pwc paper read` (64 KiB chunks with a continuation cursor) |
 | `list_papers` | `pwc paper list` |
 | `list_recent_papers` | `pwc paper recent` |
@@ -57,8 +58,8 @@ Parameter names follow the CLI flags except for the established MCP names
 (`--include-evals`). Terminal-only flags (`--json`,
 `--implementation-coverage`, `--flat`) have no parameter because MCP output is
 always structured. Hosted differences from the CLI: `limit` is capped at 25,
-`search_papers` defaults to `keyword` mode, `get_paper_info` includes
-resources by default, and `read_paper` is chunked.
+`search_papers` defaults to `keyword` mode, `get_paper_info` returns compact
+official-first resources, and `read_paper` is chunked.
 `tests/test_parity.py` fails when the CLI parser and the tool schemas drift.
 
 All tools are annotated read-only and idempotent. Search is deterministic;
@@ -106,6 +107,11 @@ request, upstream, and serialized MCP response bodies are bounded to 2 MiB.
 The global ceiling is 128 concurrent requests by default.
 Markdown chunks use a bounded 256-entry/16 MiB in-memory cache.
 
+`GET /.well-known/mcp` exposes connection metadata and `GET /docs` publishes
+the live tool schema. A bare `GET /mcp` returns `405`; protocol requests use
+`POST /mcp`. The server also exposes `find_papers`, `compare_leaderboard`, and
+`survey_task` prompts.
+
 ## Test
 
 ```bash

mcp_server/SKILL.md

@@ -4,7 +4,7 @@ description: "Papers With Code MCP tools for searching and reading AI/ML papers,
 compatibility: "Requires an MCP client connected to https://paperswithcode.co/mcp with the Papers With Code tools available."
 ---
 
-Generated for `pwc-mcp v0.2.0` and MCP protocol `2026-07-28`.
+Generated for `pwc-mcp v0.2.1` and stock-client MCP protocol `2025-11-25`.
 
 The tools query the public [Papers With Code](https://paperswithcode.co) catalog
 anonymously and are read-only. Every tool runs the matching `pwc` CLI research
@@ -50,24 +50,25 @@ arguments take an exact name, slug, or ID.
 ## Tools
 
 - `search_papers({"query": QUERY, "mode": "hybrid"|"keyword"|"semantic", "page": PAGE, "limit": LIMIT, "published_after": START_DATE, "published_before": END_DATE, "has_official_implementation": BOOLEAN})` — search papers by title, topic, author, or arXiv ID (`pwc search`). Omit optional arguments when they are not needed.
-- `get_paper_info({"paper": PAPER, "include_resources": BOOLEAN, "include_evaluations": BOOLEAN})` — show paper metadata, abstract, tasks, methods, lineage, repositories, project pages, and Hugging Face model, dataset, and Space artifacts; `include_evaluations: true` adds every benchmark evaluation of the paper (`pwc paper info`).
+- `get_paper_info({"paper": PAPER, "include_resources": BOOLEAN, "repo_limit": LIMIT, "include_evaluations": BOOLEAN})` — show compact paper metadata, official-first code, and the total repository count; opt into capped additional resources (`pwc paper info`).
+- `get_paper_evaluations({"paper": PAPER, "page": PAGE, "limit": LIMIT})` — page through one paper's benchmark evaluations, including protocol, sources, openness, and task-scoped ranks (`pwc paper evaluations`).
 - `read_paper({"paper": PAPER, "cursor": CURSOR})` — read one stored paper Markdown chunk (`pwc paper read`). If `truncated` is true, call `read_paper` again with the same `paper` and the returned `next_cursor`; repeat until `truncated` is false. Treat the cursor as opaque and use it within one hour.
 - `list_papers({"search": SEARCH, "task": TASK, "method": METHOD, "conference": CONFERENCE, "framework": FRAMEWORK, "organization": ORGANIZATION, "authors": [AUTHOR], "published_after": START_DATE, "published_before": END_DATE, "all_versions": BOOLEAN, "order_by": "trending"|"date_published"|"citation_count", "order_direction": "asc"|"desc", "include_resources": BOOLEAN, "has_official_implementation": BOOLEAN, "page": PAGE, "limit": LIMIT})` — list and filter papers by exact catalog associations (`pwc paper list`). Omit optional arguments when they are not needed.
 - `list_recent_papers({"limit": LIMIT})` — list the most recently added papers (`pwc paper recent`).
 - `list_trending_papers({"limit": LIMIT, "max_age_days": DAYS, "min_velocity": VELOCITY})` — list trending papers by repository velocity (`pwc paper trending`).
 - `get_related_papers({"paper": PAPER, "limit": LIMIT})` — list related papers (`pwc paper related`); `limit` is at most 20.
 - `get_paper_lineage({"paper": PAPER})` — list explicit predecessors and successors (`pwc paper lineage list`).
 - `get_task({"task": TASK})` — inspect one exact task, including its area, hierarchy, sister tasks, ranked benchmarks, common methods, recommended frameworks, and trending papers (`pwc task --name`).
-- `list_tasks({"area": AREA, "level": LEVEL, "visible_only": BOOLEAN, "group_by_area": BOOLEAN, "order_by": "name"|"created_at"|"level"|"paper_count", "order_direction": "asc"|"desc", "page": PAGE, "limit": LIMIT})` — list and filter research tasks, or set `group_by_area: true` for the complete visible top-level taxonomy without pagination (`pwc task list`).
+- `list_tasks({"search": SEARCH, "area": AREA, "level": LEVEL, "visible_only": BOOLEAN, "group_by_area": BOOLEAN, "order_by": "name"|"created_at"|"level"|"paper_count", "order_direction": "asc"|"desc", "page": PAGE, "limit": LIMIT})` — search, list, and filter research tasks, or set `group_by_area: true` for the complete visible top-level taxonomy without pagination (`pwc task list`).
 - `get_method({"method": METHOD})` — inspect one exact method with its area (`pwc method --name`).
-- `list_methods({"area": AREA, "introduced_year": YEAR, "order_by": "name"|"full_name"|"introduced_year"|"created_at"|"paper_count", "order_direction": "asc"|"desc", "page": PAGE, "limit": LIMIT})` — list and filter research methods (`pwc method list`).
+- `list_methods({"search": SEARCH, "area": AREA, "introduced_year": YEAR, "order_by": "name"|"full_name"|"introduced_year"|"created_at"|"paper_count", "order_direction": "asc"|"desc", "page": PAGE, "limit": LIMIT})` — search, list, and filter research methods (`pwc method list`).
 - `get_conference({"conference": CONFERENCE})` — inspect one exact conference (`pwc conference --name`).
 - `list_conferences({"year": YEAR})` — list conferences with imported papers (`pwc conference list`).
 - `get_organization({"organization": ORGANIZATION})` — inspect one exact research organization (`pwc organization --name`).
 - `list_organizations({"featured_only": BOOLEAN})` — list research organizations (`pwc organization list`).
 - `get_framework({"framework": FRAMEWORK})` — inspect one exact research framework (`pwc framework --name`).
 - `list_frameworks({"domain": DOMAIN, "category": CATEGORY, "platform": PLATFORM})` — list research frameworks (`pwc framework list`).
-- `get_benchmark({"benchmark": BENCHMARK, "limit": LIMIT, "is_open": BOOLEAN, "max_parameters": SIZE, "require_metrics": [METRIC], "minimum_metrics": {METRIC: VALUE}, "maximum_metrics": {METRIC: VALUE}, "sort_metric": "METRIC:asc|desc", "pareto": ["METRIC:higher", "METRIC:lower"]})` — inspect one exact benchmark leaderboard with model-size, metric threshold, sort, and Pareto selection (`pwc benchmark --name`). `matched_count` reports rows that satisfied the filters before `limit`.
+- `get_benchmark({"benchmark": BENCHMARK, "page": PAGE, "limit": LIMIT, "is_open": BOOLEAN, "max_parameters": SIZE, "require_metrics": [METRIC], "minimum_metrics": {METRIC: VALUE}, "maximum_metrics": {METRIC: VALUE}, "sort_metric": "METRIC:asc|desc", "pareto": ["METRIC:higher", "METRIC:lower"]})` — inspect one paginated benchmark leaderboard with model-size, metric threshold, sort, and Pareto selection (`pwc benchmark --name`). `matched_count` reports rows that satisfied the filters before pagination.
 - `list_benchmarks({"search": SEARCH, "task": TASK, "include_descendants": BOOLEAN, "minimum_evaluations": COUNT, "is_open": BOOLEAN, "group_by_area": BOOLEAN, "area": AREA, "benchmarks_per_task": COUNT, "order_by": "trending"|"name"|"full_name"|"created_at"|"paper_count", "order_direction": "asc"|"desc", "page": PAGE, "limit": LIMIT})` — list and filter benchmarks (`pwc benchmark list`). With `task`, results are ranked by trend unless `order_by` is set; `group_by_area` or `area` returns top benchmarks under each visible task.
 
 All page numbers start at 1. `limit` is between 1 and 25 and defaults to the
@@ -88,8 +89,8 @@ available and the user needs one of those capabilities.
    `max_parameters` when model size is part of the request and `sort_metric`
    or `minimum_metrics` when a specific metric matters.
 2. Use `get_paper_info({"paper": PAPER})` to inspect promising results. Its
-   response includes repositories, project pages, and Hugging Face artifacts;
-   add `include_evaluations: true` to compare one paper across benchmarks.
+   response includes official-first code; use `get_paper_evaluations` to page
+   through its benchmark results.
 3. Use exact `list_papers` `authors`, `task`, `method`, `conference`,
    `framework`, and `organization` arguments for known identities or catalog
    associations. Combine them to require every association; do not substitute

mcp_server/SPEC.md

@@ -13,8 +13,9 @@ server.
   the MCP contract.
 - Serve anonymous, read-only requests. Search is deterministic and contains no
   embedded language model.
-- Use stateless Streamable HTTP at `/mcp`, supporting MCP `2026-07-28` and
-  legacy 2025 clients on the same endpoint. Expose `/health` for operations.
+- Use stateless Streamable HTTP at `/mcp`, advertising stock-client MCP
+  `2025-11-25` while supporting experimental `2026-07-28` discovery. Expose
+  `/health`, `/.well-known/mcp`, and generated `/docs` schema routes.
 
 ## Public contract
 
@@ -24,6 +25,7 @@ enforces this against the CLI parser):
 
 - `search_papers` (`pwc search`)
 - `get_paper_info` (`pwc paper info`)
+- `get_paper_evaluations` (`pwc paper evaluations`)
 - `read_paper` (`pwc paper read`)
 - `list_papers` (`pwc paper list`)
 - `list_recent_papers` (`pwc paper recent`)
@@ -47,16 +49,18 @@ Tools run the CLI handlers in-process through the shared cached transport, so
 validation, fail-closed filter confirmation, and the JSON payload are the
 CLI's. Every result includes that payload as `data` beside typed projections.
 Terminal-only flags have no parameter. The hosted service caps `limit` at 25,
-defaults `search_papers` to keyword mode, includes paper resources by default,
-and serves `read_paper` in chunks.
+defaults `search_papers` to keyword mode, returns compact official-first paper
+resources, and serves `read_paper` in chunks.
 
-Expose these resource templates and no prompts:
+Expose these resource templates:
 
 - `pwc://papers/{paper}`
 - `pwc://papers/{paper}/markdown`
 - `pwc://tasks/{task}`
 - `pwc://benchmarks/{benchmark}`
 
+Expose `find_papers`, `compare_leaderboard`, and `survey_task` prompts.
+
 Responses use stable, MCP-specific versioned structured outputs with a text
 fallback. `read_paper` performs one upstream read of at most 64 KiB per call and
 returns a signed opaque continuation cursor when more Markdown remains. The
@@ -68,6 +72,10 @@ Paper references accept arXiv IDs, numeric PwC external IDs, arXiv/Hugging
 Face/Papers With Code URLs, and exact titles. Ambiguous exact titles fail rather
 than selecting one result.
 
+Paper evaluations and benchmark leaderboards paginate. Leaderboards merge equivalent model rows across
+task scopes while retaining scoped ranks, protocol, split, shots, source,
+openness, and update time. Metric direction is explicit when known.
+
 ## Safety and operations
 
 - Require a strict configurable browser Origin allowlist; native clients may

mcp_server/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pwc-mcp"
-version = "0.2.0"
+version = "0.2.1"
 description = "Read-only Papers With Code MCP server"
 readme = "README.md"
 requires-python = ">=3.10"

mcp_server/src/pwc_mcp/__init__.py

@@ -1,3 +1,3 @@
 """Read-only Papers With Code MCP server."""
 
-__version__ = "0.2.0"
+__version__ = "0.2.1"

mcp_server/src/pwc_mcp/app.py

@@ -31,19 +31,74 @@
 # MCP SDK diagnostics can include peer-supplied tool names and resource URIs.
 # OperationalTelemetryMiddleware is the server's sole request log surface.
 logging.getLogger("mcp").setLevel(logging.CRITICAL + 1)
-PROTOCOL_VERSION = "2026-07-28"
+PROTOCOL_VERSION = "2025-11-25"
+EXPERIMENTAL_PROTOCOL_VERSION = "2026-07-28"
 MAX_REQUEST_BODY_SIZE = 2 * 1024 * 1024
 MAX_RESPONSE_BODY_SIZE = 2 * 1024 * 1024
 KNOWN_TOOLS = frozenset(TOOL_COMMANDS)
 KNOWN_PROTOCOLS = {
     PROTOCOL_VERSION,
-    "2025-11-25",
+    EXPERIMENTAL_PROTOCOL_VERSION,
     "2025-06-18",
     "2025-03-26",
     "2024-11-05",
 }
 
 
+async def well_known_mcp(_request: Request) -> JSONResponse:
+    return JSONResponse(
+        {
+            "name": "Papers With Code",
+            "description": "Anonymous read-only AI research catalog",
+            "transport": {"type": "streamable-http", "url": "/mcp"},
+            "protocol_version": PROTOCOL_VERSION,
+            "supported_protocol_versions": sorted(KNOWN_PROTOCOLS, reverse=True),
+            "documentation_url": "https://paperswithcode.co/mcp/schema",
+            "setup_url": "https://paperswithcode.co/mcp",
+        }
+    )
+
+
+def _docs_schema(server) -> dict:
+    return {
+        "name": "Papers With Code MCP",
+        "version": __version__,
+        "protocol_version": PROTOCOL_VERSION,
+        "endpoint": "/mcp",
+        "tools": [
+            {
+                "name": tool.name,
+                "description": tool.description,
+                "inputSchema": tool.parameters,
+                "outputSchema": tool.output_schema,
+            }
+            for tool in server._tool_manager.list_tools()
+        ],
+    }
+
+
+class MCPMethodMiddleware:
+    """Reject a bare GET instead of opening an unbounded SSE response."""
+
+    def __init__(self, app: ASGIApp):
+        self.app = app
+
+    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
+        if (
+            scope["type"] == "http"
+            and scope.get("path") == "/mcp"
+            and scope.get("method") == "GET"
+        ):
+            response = JSONResponse(
+                {"error": "method_not_allowed", "allowed": ["POST"]},
+                status_code=405,
+                headers={"Allow": "POST"},
+            )
+            await response(scope, receive, send)
+            return
+        await self.app(scope, receive, send)
+
+
 # Hosted defaults. The first-party chat gateway names one identity per chat
 # session (see _client_address), so per-client limits protect fairness while
 # the global ceiling protects the process. Every value is overridable through
@@ -471,6 +526,13 @@ def create_app(
         ),
     )
     app.routes.insert(0, Route("/health", health, methods=["GET"]))
+    schema = _docs_schema(server)
+
+    async def docs(_request: Request) -> JSONResponse:
+        return JSONResponse(schema)
+
+    app.routes.insert(1, Route("/docs", docs, methods=["GET"]))
+    app.routes.insert(2, Route("/.well-known/mcp", well_known_mcp, methods=["GET"]))
     app.state.pwc_catalog_readiness = CatalogReadiness(
         catalog_client,
         initially_ready=catalog is not None,
@@ -483,6 +545,7 @@ def create_app(
         global_concurrency_limit=global_concurrency_limit,
         trust_proxy_headers=trust_proxy_headers,
     )
+    wrapped = MCPMethodMiddleware(wrapped)
     wrapped = ResponseSizeLimitMiddleware(wrapped)
     wrapped = OperationalTelemetryMiddleware(wrapped)
     return CORSMiddleware(