tools/test_plan.py decides what a change must test. Cargo's graph supplies every Rust package,
every dependency edge and the split between the fast and the captured-fixture tier.
tools/testing/suites.toml holds only what Cargo cannot see. obc suites check validates the test
plan and runs on every pull request.
obc test -p obc-app # one package on nextest, then its doctests
obc test -p obc-app --lib # the library target alone
obc test fixtures -p obc-route # the captured-fixture tier, after a fixture sync
obc test affected --base origin/develop # the selection CI runs; --dry-run prints it
obc test full # cross-cutting changes only
obc check fmt clippy device docs # named CI gates; obc check full runs them all
obc suites check # plan drift and command resolution
obc suites select --base REF [--release] # the plan as text or json
obc suites prepare # the npm projects and wasm bridges of the web suitesobc test always needs a scope; it never expands to the workspace on its own. affected reads
the working tree, prints every selected unit with a reason, and stops at the first failure. Before
a suite runs, it installs each npm project of the suite's CI job whose lock file changed, and it
builds the wasm bridges when they are missing or the change reaches them. A
suite whose platforms exclude the host is reported as skipped, never as passed. Every Cargo
command needs the CI-pinned runner: cargo install cargo-nextest --version 0.9.143 --locked.
The Python suites need pip install -r tools/requirements-test.txt --group planner-snow --group forest-reference
(pip 25.1 or later).
route in tools/testing/suites.toml says when a unit runs:
| Route | Meaning |
|---|---|
ordinary |
the change selects it (the default for every Rust package) |
required |
it runs whenever one of its CI jobs starts |
manual |
its own command only: generators, probes, captured-source checks, iOS application tests |
live |
it contacts a live service; its own command only |
Selection is per suite, never per test function. A binary that mixes ordinary work with
captured-fixture, live or manual work is split into separate units. Captured fixtures are ordinary
work gated on required-features = ["external-fixtures"]; a missing package fails with the exact
obc fixtures sync command. Physical procedures have no route; they live in their issue.
Each Monday, test-weekly.yml runs manual iOS application tests and ordinary storage tests with
default features.
A [[package]] entry adds what Cargo cannot say about a package:
| Field | Meaning |
|---|---|
name |
the Cargo package |
route |
ordinary by default |
triggers |
paths whose edge no build graph carries |
fixtures |
true when a change under fixtures/ reaches the package |
command |
the local command for a standalone Cargo root |
platforms |
supported platforms, when restricted |
A [[suite]] entry is verification no Cargo package owns:
| Field | Meaning |
|---|---|
id |
stable identifier used by commands and reports |
route |
one of the four routes |
command |
one repository-root command, for local and CI use |
jobs |
the CI jobs that execute it (ordinary and required need at least one) |
triggers |
paths that select it, including its own test sources |
fixtures |
captured or production-shaped inputs it needs |
platforms, foundation, ci_only |
platform restriction; selected by manifest or toolchain changes; not reproducible locally |
scoped |
follows its own inputs instead of broad policy or unowned-deletion selection |
rust_packages |
linked libraries; Cargo supplies their non-development dependencies |
rust_excludes |
source paths outside the linked feature set |
package, targets |
Cargo test targets this suite owns; the fast tier runs them when the plan selects an ordinary or required owner, or with no plan (obc test full) |
Neither entry lists dependencies, test counts, durations or source files; Cargo and the result
artifacts supply those. The CI job table is in tools/test_plan.py; obc suites validate-filters
checks that it and .github/workflows/ci.yml describe the same jobs.
Selection fails closed: an unowned source path or a selected suite without a CI route is an error.
An error publishes no plan, so the ci aggregate fails. Foundation and policy changes select
unscoped suites across the relevant graph. An unowned deletion also selects that graph.
The iOS suites are scoped. They follow their source, tests, build inputs and linked Rust libraries.
Rust tests, examples, binaries and Markdown do not select a linked library build. A changed suite
entry selects that suite; an unrelated entry does not select iOS. Changes to the selector, aggregate
or CI workflow select scoped suites too. Full release verification includes every routed suite.
Within ios-unit, only selected camera, OBCKit and PMTiles suites execute.
Do not retry a flaky test; prefer observed state, a controllable clock or a protocol signal to a sleep. A suite that must wait on real time says why in a comment beside it.
Every job uploads its native results after success or failure; a skipped step uploads nothing,
and a missing expected file fails the upload. Download with
gh run download RUN_ID --pattern '<prefix>*' --dir test-results.
| Artifact | Holds |
|---|---|
rust-test-ATTEMPT, rust-fixtures-ATTEMPT |
nextest JUnit XML for the fast and fixture tiers; the fast artifact also holds doctests.log and formats-default.log |
python-repository-tools-ATTEMPT, python-firmware-tools-ATTEMPT, python-packer-ATTEMPT |
unittest and pytest XML |
web-builder-ATTEMPT |
Vitest JUnit XML |
web-builder-browser-ATTEMPT, web-demo-browser-ATTEMPT |
the Chromium journeys of the web app and of the landing page, with a screenshot and trace on failure |
ios-tests-coverage-ATTEMPT, ios-screenshots-ATTEMPT, ios-application-ATTEMPT |
.xcresult bundles; restore the suffix and open in Xcode |
The exit status and the CI log stay authoritative; an artifact does not prove a passing run.
web.planner-requests runs the web app in Chromium. Its area and Device journeys build a map in
the tab from a digest-pinned loopback catalog, through the real WebAssembly bridge, and send it to
the pretend device. web.demo-browser drives the landing page's save, view, reset and reload
controls. Each browser suite serves on a free loopback port, so separate worktrees run it at the
same time; OBC_BROWSER_PORT and the other OBC_*_BROWSER_PORT variables set one.