Skip to content

Run Miri once per architecture, on the hosted runners - #1900

Merged
jprendes merged 2 commits into
hyperlight-dev:mainfrom
jprendes:split-build-test
Oct 8, 2026
Merged

jprendes merged 2 commits into
hyperlight-dev:mainfrom
jprendes:split-build-test

Conversation

@jprendes

@jprendes jprendes commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

build-test has been creeping towards its timeout, close enough that whether a
run finishes is becoming a matter of luck. A sizeable share of what is left
between the job and that limit is spent on Miri.

Miri interprets rather than executing. It reaches no hypervisor, reads no guest
binary, and asks nothing of a runner beyond the target it compiles for. It ran
on every Linux build-test job regardless: eight times over on X64 and twice on
aarch64, for two answers, each one occupying a pool the hypervisor jobs were
queuing for.

What this does

Miri runs once per architecture, in a job of its own, on the hosted runners.
build-test sheds the step, and the hypervisor pools stop spending slots on
work that never asked for one.

The architecture is the only axis its results depend on. just miri-tests takes
no arguments: its features are fixed in the recipe and no profile is passed, so
the hypervisor, the cpu vendor and the build configuration cannot change what it
covers. Should it ever take a profile or a feature, that becomes another axis
and the matrix has to grow with it, which is why the assumption is written down
beside the job.

Hosted runners

The shared setup is not used there. It reads /dev/kvm to report what the
runner offers, which a hosted one need not have, and installs a C toolchain and
spare Rust versions for builds Miri does not do. A toolchain, just and
cargo-nextest are what it wants.

miri-tests installs the pinned nightly it names rather than expecting the
runner to carry it. That is what let it assume a provisioned image before, and
it is also why the recipe now works on a fresh machine.

Verified

The Miri suite passes on a machine with no hypervisor infrastructure, which is
what the hosted runners are. bindgen and cc are the only build dependencies
that would need more than a Rust toolchain, and both sit behind the macOS and
hvf gate in the host's build script, so neither is reached here.

cargo-nextest was installed for build-test solely for Miri. Nothing else
uses it, so it goes with it.

Copilot AI balanced review requested due to automatic review settings October 7, 2026 22:55
@jprendes jprendes added C-enhancement For PRs adding features, improving functionality, docs, tests, etc. A-infra Concerns infrastructure rather than core functionality S-needs-review PR is ready for (re-)review labels Oct 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Nightly CI still repeats Miri across eight self-hosted jobs, and daily Miri unnecessarily waits for guest builds.

2 open findings
What changed in this PR

Moves Miri testing from repeated hypervisor jobs to dedicated hosted runners.

Changes:

  • Adds a reusable architecture-specific Miri workflow.
  • Removes Miri from the build-test matrix.
  • Installs the pinned Miri toolchain when needed.
File Description
Justfile Ensures the pinned nightly and Miri are installed.
.github/​workflows/​ValidatePullRequest.yml Adds dedicated X64 and arm64 Miri jobs.
.github/​workflows/​dep_miri.yml Defines the hosted-runner Miri workflow.
.github/​workflows/​dep_build_test.yml Removes repeated Miri execution.
.github/​workflows/​DailyArm64.yml Adds scheduled arm64 Miri coverage.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/DailyArm64.yml Resolved
danbugs
danbugs previously approved these changes Oct 7, 2026

@danbugs danbugs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ludfjig
ludfjig previously approved these changes Oct 7, 2026

@ludfjig ludfjig left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions github-actions Bot removed the S-needs-review PR is ready for (re-)review label Oct 7, 2026
@hyperlight-gh-bot

This comment has been minimized.

@jprendes jprendes added the S-needs-review PR is ready for (re-)review label Oct 8, 2026
jprendes and others added 2 commits October 8, 2026 10:14
Miri interprets rather than executing. It reaches no hypervisor, reads no guest
binary and asks nothing of a runner beyond the target it compiles for, yet it
ran on every Linux build-and-test job: eight times over on X64 and twice on
aarch64, for two answers, each on a pool the hypervisor jobs were queuing for.

It runs once per architecture now, in a job of its own on the hosted runners.
The build-and-test jobs shed a step that was a sizeable share of what was left
between them and their timeout, and the hypervisor pools stop spending a slot
on work that never asked for one.

The shared setup is not used there. It reads /dev/kvm to report what the runner
offers, which a hosted one need not have, and installs a C toolchain and spare
Rust versions for builds Miri does not do. A toolchain, just and nextest are
what it wants.

`miri-tests` installs the pinned nightly it names rather than expecting the
runner to carry it, which is what let it assume a provisioned image before. It
still takes no arguments, which is what makes one job per architecture enough.
Should it ever take a profile or a feature, that becomes another axis and the
matrix has to grow with it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>
The nightly musl matrix ran Miri in all eight of its entries, on the same pools
the pull request matrix was freed from. Its `TARGET_TRIPLE` never reaches the
recipe, which builds for the host with plain cargo, and the toolchain it pins
is the same one every other job uses, so the eight runs covered what one does.

The aarch64 daily run waited on the guest builds before starting. Miri reads no
guest binary, so it now waits for nothing and reports even when they fail.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>
@jprendes
jprendes dismissed stale reviews from ludfjig and danbugs via 58ffc74 October 8, 2026 09:14
@jprendes
jprendes enabled auto-merge (squash) October 8, 2026 09:17
@hyperlight-gh-bot

Copy link
Copy Markdown

Benchmark Results

Measured commit: 58ffc74cf7eb
Baseline commit: 21604ca82e8f

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 798.90 ns (➖ 1.05x slower)
vec_bytes 579.65 ns (➖ 1.02x faster)
373.36 µs (➖ 1.00x slower)

payload_allocation

slot_pool_segmented
262144 522.57 ns (➖ 1.01x faster)
65536 141.94 ns (➖ 1.00x faster)

sandboxes

create_initialized_and_drop
medium 80.26 ms (➖ 1.05x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
7.89 ns (➖ 1.01x slower) 7.73 ns (➖ 1.00x slower) 7.82 ns (➖ 1.01x slower)

snapshot_files

load_snapshot_unverified
small 93.24 µs (➖ 1.02x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.17 µs (➖ 1.02x faster) 6.99 µs (➖ 1.01x faster)
65536 2.05 µs (➖ 1.06x slower) 1.95 µs (➖ 1.09x faster)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 6.41 µs (➖ 1.12x faster) 6.24 µs (➖ 1.03x faster)
8192 1.06 µs (➖ 1.05x faster) 1.04 µs (➖ 1.05x faster)
262144 26.86 µs (➖ 1.06x faster)
kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 827.55 ns (➖ 1.04x slower)
vec_bytes 577.46 ns (➖ 1.02x faster)
801.81 µs (➖ 1.05x faster)

payload_allocation

slot_pool_segmented
262144 571.57 ns (➖ 1.00x faster)
65536 155.06 ns (➖ 1.00x slower)

sandboxes

create_initialized_and_drop
medium 83.20 ms (➖ 1.02x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
7.91 ns (➖ 1.01x faster) 7.89 ns (➖ 1.00x slower) 7.93 ns (➖ 1.02x faster)

snapshot_files

load_snapshot_unverified
small 49.55 µs (➖ 1.05x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 8.62 µs (➖ 1.01x slower) 8.57 µs (➖ 1.00x slower)
65536 2.45 µs (➖ 1.01x slower) 2.51 µs (➖ 1.04x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.06 µs (➖ 1.00x faster) 8.13 µs (➖ 1.00x slower)
8192 858.82 ns (➖ 1.01x slower) 826.34 ns (➖ 1.00x slower)
262144 34.07 µs (➖ 1.01x faster)
mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 945.46 ns (➖ 1.02x slower)
vec_bytes 719.89 ns (➖ 1.02x slower)
318.82 µs (➖ 1.28x faster)

payload_allocation

slot_pool_segmented
262144 670.57 ns (➖ 1.01x faster)
65536 181.92 ns (➖ 1.00x faster)

sandboxes

create_initialized_and_drop
medium 62.76 ms (➖ 1.02x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
9.91 ns (➖ 1.00x faster) 10.14 ns (➖ 1.02x slower) 9.96 ns (➖ 1.00x slower)

snapshot_files

load_snapshot_unverified
small 86.79 µs (➖ 1.01x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 9.31 µs (➖ 1.00x slower) 9.64 µs (➖ 1.06x slower)
65536 2.71 µs (➖ 1.01x slower) 2.47 µs (➖ 1.00x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.00 µs (➖ 1.03x faster) 8.04 µs (➖ 1.05x faster)
8192 1.38 µs (➖ 1.02x slower) 1.31 µs (➖ 1.00x slower)
262144 34.75 µs (➖ 1.01x faster)
mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 980.91 ns (➖ 1.05x slower)
vec_bytes 658.15 ns (➖ 1.00x faster)
681.10 µs (➖ 1.10x faster)

payload_allocation

slot_pool_segmented
262144 627.50 ns (➖ 1.01x slower)
65536 165.03 ns (➖ 1.02x faster)

sandboxes

create_initialized_and_drop
medium 66.88 ms (➖ 1.03x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
9.62 ns (➖ 1.01x slower) 8.76 ns (➖ 1.00x faster) 8.79 ns (➖ 1.00x faster)

snapshot_files

load_snapshot_unverified
small 45.54 µs (➖ 1.00x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.92 µs (➖ 1.02x faster) 8.00 µs (➖ 1.01x faster)
65536 2.30 µs (➖ 1.00x slower) 2.30 µs (➖ 1.00x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 7.55 µs (➖ 1.00x faster) 7.54 µs (➖ 1.01x faster)
8192 919.35 ns (➖ 1.04x faster) 941.51 ns (➖ 1.02x slower)
262144 39.48 µs (➖ 1.00x slower)
hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.13 µs (➖ 1.00x faster)
vec_bytes 746.63 ns (➖ 1.05x faster)
2.05 ms (➖ 1.03x slower)

payload_allocation

slot_pool_segmented
262144 803.48 ns (➖ 1.03x slower)
65536 232.09 ns (➖ 1.01x slower)

sandboxes

create_initialized_and_drop
medium 89.72 ms (➖ 1.14x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
10.54 ns (➖ 1.00x faster) 10.60 ns (➖ 1.12x faster) 10.54 ns (➖ 1.00x slower)

snapshot_files

load_snapshot_unverified
small 557.55 µs (➖ 1.59x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 8.81 µs (➖ 1.01x slower) 8.58 µs (➖ 1.01x faster)
65536 2.55 µs (➖ 1.01x faster) 2.52 µs (➖ 1.02x faster)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.74 µs (➖ 1.00x faster) 9.23 µs (➖ 1.07x slower)
8192 1.39 µs (➖ 1.01x faster) 1.39 µs (➖ 1.01x faster)
262144 37.68 µs (➖ 1.26x faster)
hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.21 µs (➖ 1.00x slower)
vec_bytes 790.98 ns (➖ 1.00x slower)
3.10 ms (➖ 1.07x slower)

payload_allocation

slot_pool_segmented
262144 735.31 ns (➖ 1.00x slower)
65536 211.21 ns (➖ 1.01x faster)

sandboxes

create_initialized_and_drop
medium 138.53 ms (➖ 1.28x slower)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
10.44 ns (➖ 1.02x faster) 10.23 ns (➖ 1.02x slower) 10.53 ns (➖ 1.00x faster)

snapshot_files

load_snapshot_unverified
small 614.22 µs (➖ 1.02x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.79 µs (➖ 1.04x faster) 7.78 µs (➖ 1.00x slower)
65536 2.34 µs (➖ 1.11x faster) 2.34 µs (➖ 1.00x faster)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.51 µs (➖ 1.00x slower) 8.66 µs (➖ 1.08x faster)
8192 1.19 µs (➖ 1.01x slower) 1.12 µs (➖ 1.02x slower)
262144 40.21 µs (➖ 1.03x faster)

Reported by cargo ci bench-report --candidate run:37755328715 --config-file bench_report.toml.

@jprendes
jprendes merged commit 73f8f3e into hyperlight-dev:main Oct 8, 2026
131 of 139 checks passed
@github-actions github-actions Bot removed the S-needs-review PR is ready for (re-)review label Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-infra Concerns infrastructure rather than core functionality C-enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants