[Oseltamivir]

Summary

CollectiveX no longer runs a CI sweep: results are hand-run and committed to SemiAnalysisAI/InferenceX under collectivex/results/<sku>/<case_id>.json, with cases.json listing every requested case. The dashboard keeps serving from its Neon database, but that database is now filled from those committed files instead of the lazy GitHub-artifact ingest. This replaces #1353 (build-time snapshot); a database keeps the dashboard interactive and keeps run history.

Flow: a push to InferenceX main that touches collectivex/results/** sends repository_dispatch ingest-collectivex {sha} (SemiAnalysisAI/InferenceX#4592). The new .github/workflows/ingest-collectivex.yml runs the CollectiveX migrations, ingests that commit, and purges the collectivex cache scope. workflow_dispatch takes exactly one of sha, ref, backfill (optionally with prune) or delete_run.

Changes

• Added:
  • packages/db/src/collectivex/committed-results.ts: resolves a ref to the newest results commit on main, lists collectivex/results/ with the git trees API, reads each file from raw.githubusercontent.com, and validates it like the benchmark repo's collectivex/tests/test_results.py: filename matches case_id and the folder matches the pool, the case is runnable in cases.json, the EP version equals the cases.json version exactly, source_sha is 40 hex and source_dirty is false, and no string holds an IPv4 address and no key names a host. Errors name the file.
  • One cx_runs row per results commit with at least one measured result: run_id is the committer second, raised past the previous results commit's id on a same-second tie, computed from main's history so it is deterministic and unique; source_sha is the commit; matrix is cases.json. Commits with nothing measured (today every results commit on main only carries cases.json) are skipped, and the read queries also hide any run that measures nothing.
  • Every CollectiveX write runs in a transaction that first takes one fixed pg_advisory_xact_lock, then upserts the row and replaces its documents, so concurrent ingests of one commit cannot store its documents twice. The workflow uses one constant concurrency group that queues instead of cancelling.
  • ingest-collectivex.ts CLI: --sha / --ref / --backfill [--prune] / --delete-run <run_id>. A backfill skips commits GitHub answers 404 for; --prune then deletes every stored run the backfill did not store (commit left main, stopped validating, or measures nothing).
  • Migrations 004_committed_results.sql (drops the deleted_at tombstone) and 005_remove_lazy_ingest_runs.sql (deletes the lazy-ingest rows, whose GitHub run ids would outrank every committed run, and adds a CHECK keeping run_id below 10^10).
  • The dispatch workflow above, modelled on ingest-operatorx, with a Slack alert on failure.
• Changed:
  • Reader: takes cases.json as the matrix; unified swap-blocks cases no longer crash it, and swap pools come from cases.json. Pools with no results yet stay pending, and covered_skus lists only pools with a measured result.
  • Routes and UI: no lazy ingest, no polling, no delete; links point at the results tree in collectivex/.
  • API docs: EP tail percentiles (p90/p95/p99) of latencies and derived rates can be null (response-shape change, now documented); run-id and covered_skus semantics.
  • Docs (runbook: archive, then backfill with prune; no manual truncate, no ref=main), the route catalog, the skills reference and its integrity manifest.
• Deleted: the lazy artifact ingest, artifact selection, the runs/[runId] DELETE route and COLLECTIVEX_ADMIN_SECRET.

Rollout

1. Merge CollectiveX: run the MI300X and MI325X pools on the vLLM ROCm image / CollectiveX：在 vLLM ROCm 镜像上运行 MI300X 与 MI325X 资源池 InferenceX#4581 (first MI300X/MI325X results) and #4592.
2. Add the repo secret DATABASE_COLLECTIVEX_WRITE_URL (done).
3. Archive cx_runs / cx_run_docs with pg_dump, merge this PR, then run the workflow with backfill and prune checked. Its migration step removes the old rows (005), so no manual TRUNCATE is needed. The dashboard then shows the pools that have measured results; until a measured results commit is on main, the CollectiveX API returns no runs.

Verification

• typecheck, oxlint, oxfmt, unit tests (app, db with PGlite over the real migrations, skills) pass; actionlint is clean on the workflow.
• PGlite tests: a first commit with only mi300x/mi325x results serves those series with the other pools pending; a later commit adding pools becomes the newer run; re-ingest replaces; a conflicting run_id is refused; an unmeasured run is never served; overlapping upserts of one commit store each document once; prune and delete-run remove runs and their documents; migration 005 removes a legacy-id run that outranked a committed one and rejects new legacy ids.
• Unit tests: same-second and clock-skewed commits get distinct increasing ids; validation rejects a non-40-hex source_sha, a string version, an IPv4 address and a host key; a backfill skips 404 and unmeasured commits but stops on invalid results.
• Not run locally: Cypress, the full next build, zizmor, and any migration against the real database.

AI model disclosure

• Claude Opus 5.5 (claude-opus-5-5, run through Claude Code): the code, tests and this description.

中文

摘要

CollectiveX 不再运行 CI 扫描：结果改为手动运行，并提交到 SemiAnalysisAI/InferenceX 的 collectivex/results/<sku>/<case_id>.json，由 cases.json 列出所有请求的用例。仪表盘仍从 Neon 数据库读取，但数据库改由这些已提交文件填充，取代 GitHub artifact 懒加载。本 PR 取代 #1353（构建时快照）；使用数据库可以保持仪表盘的交互性并保留运行历史。

流程： 推送到 InferenceX main 且修改 collectivex/results/** 时，发送 repository_dispatch ingest-collectivex {sha}（SemiAnalysisAI/InferenceX#4592）。新的 .github/workflows/ingest-collectivex.yml 运行 CollectiveX 迁移、导入该提交并清除 collectivex 缓存范围。workflow_dispatch 需且只能提供 sha、ref、backfill（可附加 prune）或 delete_run 之一。

改动

• 新增：
  • packages/db/src/collectivex/committed-results.ts：将 ref 解析为 main 上最新的结果提交，用 git trees API 列出 collectivex/results/，从 raw.githubusercontent.com 读取每个文件，并按基准仓库的 collectivex/tests/test_results.py 校验：文件名与 case_id 一致、目录与资源池一致、用例在 cases.json 中可运行、EP 版本与 cases.json 版本严格相等、source_sha 为 40 位十六进制且 source_dirty 为 false、任何字符串都不含 IPv4 地址且没有指明主机的字段。错误信息会指出具体文件。
  • 每个至少包含一个已测量结果的结果提交对应一行 cx_runs：run_id 为提交者时间（秒），同一秒时递增到上一个结果提交之后，由 main 的历史计算，因此确定且唯一；source_sha 为该提交；matrix 为 cases.json。没有任何测量结果的提交（目前 main 上的结果提交都只包含 cases.json）会被跳过，读取查询也会隐藏没有测量结果的运行。
  • 所有 CollectiveX 写操作都在事务中先获取同一个固定的 pg_advisory_xact_lock，再写入运行行并替换其文档，因此同一提交的并发导入不会把文档存两遍。工作流使用单一固定的并发组，排队而不取消。
  • ingest-collectivex.ts CLI：--sha / --ref / --backfill [--prune] / --delete-run <run_id>。回填会跳过 GitHub 返回 404 的提交；--prune 随后删除回填未写入的所有已存储运行（提交已不在 main、不再通过校验或没有测量结果）。
  • 迁移 004_committed_results.sql（删除 deleted_at 墓碑字段）与 005_remove_lazy_ingest_runs.sql（删除懒加载旧行——其 GitHub 运行 ID 会压过所有已提交运行——并添加 CHECK 约束使 run_id 小于 10^10）。
  • 上述 dispatch 工作流，参照 ingest-operatorx，失败时发送 Slack 告警。
• 修改：
  • Reader： 以 cases.json 作为矩阵；统一格式的 swap-blocks 用例不再导致崩溃，swap 资源池来自 cases.json。尚无结果的资源池保持 pending，covered_skus 只列出有已测量结果的资源池。
  • 路由与界面： 移除懒加载、轮询和删除；链接指向 collectivex/ 下的结果目录。
  • API 文档： 延迟及其派生速率的 EP 尾部百分位（p90/p95/p99）可为 null（响应结构变更，现已写入文档）；补充运行 ID 与 covered_skus 的含义。
  • 文档（运维手册：先归档，再带 prune 回填；无需手动清空表，也不再使用 ref=main）、路由目录、skills 参考文档及其完整性清单。
• 删除： artifact 懒加载、artifact 选择、runs/[runId] DELETE 路由及 COLLECTIVEX_ADMIN_SECRET。

上线步骤

1. 合并 CollectiveX: run the MI300X and MI325X pools on the vLLM ROCm image / CollectiveX：在 vLLM ROCm 镜像上运行 MI300X 与 MI325X 资源池 InferenceX#4581（首批 MI300X/MI325X 结果）与 #4592。
2. 添加仓库密钥 DATABASE_COLLECTIVEX_WRITE_URL（已完成）。
3. 用 pg_dump 归档 cx_runs / cx_run_docs，合并本 PR，然后勾选 backfill 与 prune 运行工作流。其迁移步骤（005）会删除旧行，无需手动 TRUNCATE。仪表盘随后显示有已测量结果的资源池；在 main 上出现含测量结果的提交之前，CollectiveX API 不返回任何运行。

验证

• typecheck、oxlint、oxfmt、单元测试（app、基于真实迁移的 PGlite db 测试、skills）通过；工作流的 actionlint 无告警。
• PGlite 测试：仅含 mi300x/mi325x 结果的首个提交会展示这两个资源池，其余保持 pending；之后新增资源池的提交成为更新的运行；重新导入会替换；冲突的 run_id 被拒绝；没有测量结果的运行不会被返回；同一提交的重叠写入只存一份文档；prune 与 delete-run 会删除运行及其文档；迁移 005 删除压过已提交运行的旧 ID 运行，并拒绝新的旧式 ID。
• 单元测试：同一秒及时钟倒退的提交得到互不相同且递增的 ID；校验会拒绝非 40 位十六进制的 source_sha、字符串版本号、IPv4 地址和主机字段；回填会跳过 404 与无测量结果的提交，但遇到无效结果会停止。
• 未在本地运行：Cypress、完整的 next build、zizmor，以及任何针对真实数据库的迁移。

---

Note

Medium Risk
Large pipeline and schema cutover for CollectiveX (migrations, backfill/prune runbook, removed in-app deletion); runtime API surface is simpler but wrong rollout would empty or skew published runs until backfill succeeds.

Overview
CollectiveX moves from lazy ingest-on-read (GitHub Actions sweep artifacts) to a Neon database filled ahead of time from SemiAnalysisAI/InferenceX commits under collectivex/results/ (cases.json plus per-case JSON). Each measured results commit becomes one stored run with commit-derived run_id; workflow ingest-collectivex.yml and admin:db:ingest:collectivex handle migrate, ingest, cache purge, backfill/prune, and single-run delete.

App/API: CollectiveX routes are read-only DB—no ensure* discovery, no 502/503 fallback paths, discovery_complete always true, default CDN caching (no 60s/no-store polling). DELETE /api/v1/collectivex/runs/[runId], COLLECTIVEX_ADMIN_SECRET, and UI/admin delete flows are removed. Links and Cypress expectations point at collectivex/results instead of Actions runs; workflow conclusion badges go away. Chart/tooling treats withheld EP tail percentiles (p90/p95/p99) as null.

Ops/docs: .env.example and deployment docs drop CollectiveX lazy-ingest assumptions; migrations 004/005 retire tombstones and legacy GitHub run ids. API docs and route catalog reflect the new semantics.

^{Reviewed by Cursor Bugbot for commit 21f777a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
inferencemax-app	Ready	Preview	Oct 28, 2026 7:17am UTC

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 5437318. Configure here.}