Skip to content

ci(e2e): run the tmachine conformance rows with the prebuilt openshell-conformance - #5332

Open
elezar wants to merge 2 commits into
mainfrom
ci/reuse-prebuilt-conformance/elezar
Open

elezar wants to merge 2 commits into
mainfrom
ci/reuse-prebuilt-conformance/elezar

Conversation

@elezar

@elezar elezar commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Summary

#5249 builds openshell-conformance once per architecture, but only the host lanes use it. The tmachine conformance rows in e2e-suites.yml still cross-build a nextest archive in every job (five builds per run). This runs them with the prebuilt program too.

Related Issue

None yet. Follow-up to #5249, part of #3954.

Changes

  • openshell-test-runner gets --program <path>: it runs a prebuilt testkit program in the guest (ready, then run --junit) instead of building a nextest bundle. It conflicts with --package, which only applies to the build. --filter (now repeatable) still applies: with --program it takes parts of test names, as the program does, instead of a nextest filterset. The program's exit codes map to the runner's (1 for a test failure, otherwise an error).
  • e2e-suites.yml gets a prebuilt-conformance input. When set, the conformance rows download the openshell-conformance-x86_64-unknown-linux-musl artifact and pass it with --program.
  • Branch E2E's conformance-integration needs build-conformance and sets the input. Release Qualification and Release Dev don't build the program, so their conformance rows are unchanged.
  • TESTING.md documents --program.

The results layout is unchanged: the program's output is still written to nextest.log so consumers of the results artifact see one layout.

Testing

  • cargo clippy --all-targets -- -D warnings and cargo test in tests/runner (new tests for the program bundle and exit-code mapping)
  • Workflow YAML parses
  • Not run: the tmachine lanes. They need test:e2e on this PR.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@elezar elezar added the test:e2e Requires end-to-end coverage label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Label test:e2e applied for 01e94e9. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

…l-conformance

The host lanes already share one openshell-conformance build per
architecture, but the tmachine conformance rows each cross-built a nextest
archive. Add --program to openshell-test-runner to run a prebuilt testkit
program in the guest, and have Branch E2E pass the build-conformance
artifact to the conformance rows of e2e-suites.yml.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
@elezar
elezar force-pushed the ci/reuse-prebuilt-conformance/elezar branch from 28e47c0 to 4146283 Compare November 6, 2026 17:13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant