CollectiveX: store results in the CollectiveX database only / CollectiveX：结果只保存在 CollectiveX 数据库中

collectivex/.gitignore

@@ -6,6 +6,8 @@ unsupported/
 .collx_backend/
 .collx_sources/
 /matrix_full.json
+# Results are uploaded to the CollectiveX database, not committed.
+/results/*/
 gpucore.*
 
 # Local plans and infrastructure inventory.

collectivex/README.md

@@ -2,16 +2,17 @@
 
 CollectiveX is an experimental MoE expert-parallel communication benchmark. It measures dispatch,
 combine and paired roundtrip latency across EP libraries and accelerator systems, and writes one
-result file per case into `results/`, where it is committed. A second suite, the
+result file per case into `results/` on the cluster that ran it. A second suite, the
 [vLLM `swap_blocks` benchmark](docs/swap-blocks.md), measures pinned CPU↔GPU and same-GPU block
 copies through the same matrix and pool launchers.
 
-CollectiveX is run by hand on each pool (see [Running By Hand](#running-by-hand)). Committing the
-results is the publish step: the dashboard builds from whatever `results/` holds on `main`. It does
-not rank, recommend, or select; any downstream display or comparison is the consumer's
-responsibility. [docs/methodology-summary.md](docs/methodology-summary.md) states the measurement
-rules, including each backend's engine contract, and [docs/methodology.md](docs/methodology.md)
-holds the evidence behind them.
+CollectiveX is run by hand on each pool (see [Running By Hand](#running-by-hand)). Uploading the
+results to the CollectiveX database is the publish step: the dashboard shows the latest stored run.
+Results are not committed to this repository. CollectiveX does not rank, recommend, or select; any
+downstream display or comparison is the consumer's responsibility.
+[docs/methodology-summary.md](docs/methodology-summary.md) states the measurement rules, including
+each backend's engine contract, and [docs/methodology.md](docs/methodology.md) holds the evidence
+behind them.
 
 ## Coverage
 
@@ -70,7 +71,7 @@ pools also need an operator document that names a stage directory (see
 
 These variables change what a row measures. The result records each one (in `kernel_generation`,
 `implementation.fp8_consume`, `implementation.rank_alignment` or `implementation.library_config`),
-but the `case_id` does not, so a run with one replaces the case's committed default-configuration
+but the `case_id` does not, so a run with one replaces the case's stored default-configuration
 result:
 
 | Variable | Default | Effect |
@@ -87,12 +88,15 @@ image) change only how a shard runs.
 
 Results record the commit they were measured at, so `run.sh` refuses to start while tracked files
 under `collectivex/` (outside `results/`) have uncommitted edits. `--allow-dirty` runs anyway and
-marks the results `source_dirty`, so they cannot be published. To publish, review
-`git diff collectivex/results/<sku>`, check it with
-`python3 -m unittest collectivex/tests/test_results.py`, and open a PR with the files. That test
-also runs in CI: every committed result must be a runnable case in `results/cases.json`, come from
-committed code, stay under 1 MiB, and contain no IP addresses or host-identifying fields.
-Regenerate `cases.json` whenever the configs change:
+marks the results `source_dirty`, so they cannot be published. `results/<sku>/` is gitignored. To
+publish, copy the pool's `results/<sku>/` and `results/cases.json` off the cluster and, from an
+InferenceX-app checkout with `DATABASE_COLLECTIVEX_WRITE_URL` set, run
+`bun run admin:db:ingest:collectivex --dir <copy>/results`. It lays the uploaded files over the
+latest stored run, replacing those cases, and stores the result as a new run. It rejects a file that
+is not a runnable case in `cases.json`, was measured from uncommitted code, or contains an IP
+address or host-identifying field, and every uploaded file must share one source SHA.
+`results/cases.json` stays committed; a unit test fails when it no longer matches the configs, so
+regenerate it whenever they change:
 
 ```bash
 python3 collectivex/sweep_matrix.py --suites ep,swap-blocks --cases-json collectivex/results/cases.json
@@ -160,4 +164,4 @@ collectivex/run.sh h200-dgxc --dry-run   # each shard's environment, without all
 ```
 
 Core paths are `run.sh`, `configs/`, `sweep_matrix.py`, `bench/`, `runtime/`, `launchers/`,
-`results/` and `tests/`. `summarize.py` and `bandwidth.py` read results and gate nothing.
+`results/cases.json` and `tests/`. `summarize.py` and `bandwidth.py` read results and gate nothing.

collectivex/docs/methodology-summary.md

@@ -192,5 +192,5 @@ Every row is gated. Any failed rank or point makes the case `invalid`.
 ## Provenance and publishing
 
 A result records its source SHA, image tag, library versions, `kernel_generation` and topology. Only
-results measured from committed code publish, by a reviewed commit of `results/<sku>/<case_id>.json`
-to `main` ([README](../README.md#running-by-hand)).
+results measured from committed code publish, by uploading `results/<sku>/<case_id>.json` to the
+CollectiveX database ([README](../README.md#running-by-hand)).

collectivex/docs/methodology.md

@@ -1,10 +1,10 @@
 # CollectiveX EP Benchmark Methodology
 
 CollectiveX runs expert-parallel (EP) communication benchmarks by hand on real accelerator
-allocations and writes one result document per case into `collectivex/results/`. Committing those
-files is the publish step. It does **not** rank, recommend, select, or decide what any consumer
-displays. The frontend reads `results/cases.json` and the committed result documents and makes its
-own coverage and display decisions. This document describes how a case is scheduled, measured,
+allocations and writes one result document per case into `collectivex/results/`. Uploading those
+files to the CollectiveX database is the publish step; they are not committed. It does **not** rank,
+recommend, select, or decide what any consumer displays. The frontend reads the stored
+`results/cases.json` and result documents and makes its own coverage and display decisions. This document describes how a case is scheduled, measured,
 checked, and recorded, not a publication or qualification contract.
 [methodology-summary.md](methodology-summary.md) states the same rules in brief; this document keeps
 the evidence behind each one. Where either disagrees with the code in `collectivex/bench/`, the code
@@ -895,7 +895,7 @@ so physical receive order is not treated as a correctness property.
 
 ## Result Document
 
-One raw case document, committed at `results/<sku>/<case_id>.json`, carries `record_type:
+One raw case document, written to `results/<sku>/<case_id>.json`, carries `record_type:
 "case-attempt"`, the sweep `version`, and a `generated_at` timestamp, and contains:
 
 - `identity`: `case_id`, `attempt_ordinal`, `case_factors` (the SKU and the scheduled case,
@@ -987,8 +987,8 @@ replacing the case's previous file, while unsupported or never-run cells produce
 Identifiers are readable factor strings:
 
 - `case_id`: `{sku}-{backend}-{workload}-{mode}-{phase}-ep{ep}-{routing}-{precision}`, each factor
-  slug-normalized. It names the result file, so a re-run replaces the committed result and git
-  history keeps the one it replaced.
+  slug-normalized. It names the result file, so a re-run replaces that case's result in the next
+  uploaded run, and earlier stored runs keep the one it replaced.
 - `attempt_ordinal`: `COLLX_ATTEMPT_ID`, 1 unless set.
 
 These IDs let a consumer group matched configurations and separate distinct ones. The backend does
@@ -1111,6 +1111,6 @@ concurrent allocation cannot lose files it is using.
 There is no results server or managed object store. Each shard runs one allocation and its per-case
 result documents are copied to `results/<sku>/` in the checkout that ran it. A case counts as
 successful on the benchmark's own return code, and failed or unsupported cells produce no synthetic
-record. Results reach the dashboard only through a reviewed commit to `main`; the unit-test job checks
-every committed result against `results/cases.json`, rejects ones measured from uncommitted code, and
-rejects IP addresses and host-identifying fields.
+record. Results reach the dashboard only through the InferenceX-app upload
+(`admin:db:ingest:collectivex --dir`), which checks every file against `results/cases.json`,
+rejects ones measured from uncommitted code, and rejects IP addresses and host-identifying fields.

collectivex/docs/swap-blocks.md

@@ -76,8 +76,8 @@ payload cap. `large-blocks` sweeps 257 B to 1 GiB blocks under a **1 GiB copied-
 larger products are excluded and recorded (294 measured points, 126 excluded); each buffer also
 holds two guard blocks, so a 1 GiB block case allocates 3 GiB per buffer plus CPU references.
 
-The two results land in `collectivex/results/<sku>/<case_id>.json`, where each case_id is
-`<sku>-swap-blocks-<profile>-<layout>`. Each records its case_id, profile and pool, the actual
+The two results land in `collectivex/results/<sku>/<case_id>.json` on the cluster, where each
+case_id is `<sku>-swap-blocks-<profile>-<layout>`. Each records its case_id, profile and pool, the actual
 GPU, framework versions, image, source SHA, correctness status, and measurements. The EP
 summary tables skip these documents. CPU CI is separate and does not establish GPU
 correctness.