Repository navigation
mcpp test re-does dependency-closure planning on every invocation — 15s to 2.5min of silent work before anything compiles #529
Description
Version: mcpp 2026.8.28.2, llvm@22.1.8, x86_64-linux-gnu, 8 cores.
On a workspace where every target is already built and cached, mcpp test -p <member>
still spends most of its wall time in a single silent phase before test compilation
starts. Nothing is recompiled — the phase produces no artefacts at all.
For our workspace the full test sweep costs ~17 minutes, and this phase is nearly all
of it. mcpp build over the same workspace, fully cached, finishes in 0.7s.
What the time is spent on
Timestamping each output line of a repeat run (member and test names elided):
0.00s Workspace building member '<member>'
0.09s Resolving toolchain
0.09s Resolved llvm@22.1.8 → .../bin/clang++
0.13s Target x86_64-unknown-linux-gnu
0.21s Compiling <member> v0.1.0 (.)
0.21s Cached <dep-a> v0.2.3 (7 units)
0.21s Cached <dep-b> v3.12.0 (1 unit)
15.84s Compiling <test-1> (test) ← 15.6s of silence
15.84s Compiling <test-2> (test)
... 7 more, all at 15.84s
15.86s <test-2> ... ok (0.02s) ← every binary already up to date
15.90s test result ok. 9 passed; 0 failed; finished in 15.92s (build 15.63s + run 0.06s)
Three observations about that 15.6s window:
- Nothing is written. Touching a marker file and then listing everything newer
than it under the member'starget/and under$MCPP_HOME/build-cacheyields
5.jsonfiles and 1.ninjafile. No.pcm, no.o, no executables. - No child processes. Sampling
ps --ppid <mcpp-pid>every 1.5s returns nothing
for the whole window, and/proc/<pid>/wchanreads0throughout — mcpp is
running on-CPU in-process. Neither clang nor ninja is spawned. - It is per-invocation, not a cold cache. Running the same command twice in a row
gives the same time both times, and interleaving other members does not change it.
How it scales
Not with the number of cached dependency units — with the module-interface surface of
the whole dependency closure, including workspace path dependencies (which are not
counted in the (N units) line):
| member | cached dep units | build phase |
|---|---|---|
| A — no dependencies | 0 | 3.3s |
| B — two small index deps | 8 | 15.7s |
| C — one vendored dep, 73 units | 73 | 60s (133s on a colder run) |
D — same 8 index units as B, plus a path dep on a large member |
8 | 154s |
D is the interesting row: same cached-unit count as B, ~10x the time, and the only
difference is a path dependency on a member with a few hundred module interfaces.
C measured 60s and 133s for the identical command on different runs, which looks like
page-cache sensitivity — consistent with the phase reading a lot of files.
Things that don't help
| attempt | result |
|---|---|
--profile dev (test defaults to release, build defaults to dev) |
identical, 15.5s |
--cache local |
43s — 3x worse |
| repeating the command / reordering members | no effect |
Why it matters
It puts a floor on the edit-test loop that is independent of how small the edit was.
For us it is the difference between a 7-minute and a 17-minute verification pass, and
it scales with workspace size rather than with change size, so it gets worse over time.
If the planning result were memoised — keyed on the resolved manifest set and the
toolchain, invalidated when either changes — a warm repeat invocation could skip it
entirely, the same way mcpp build already does.
Happy to put together a minimal reproducing workspace (a member with a large path
dependency seems to be enough to show it) if that would help.
Activity
fix: 六处「答案已记录、做决定的代码不读它」+ 明确拒绝 system 工具链 (#538)
yspbwx2010 commentedon Aug 31, 2026
Validated 2026.8.30.2 on the workspace the original numbers came from, as the design document requested.
Setup: same machine, same MCPP_HOME, only the mcpp binary swapped (2026.8.28.2 release vs 2026.8.30.2 release, both from GitHub Releases, checksums verified). For each member we first let the new binary do a full rebuild under its own fingerprint (real compilation observed in the output), then measured back-to-back warm
mcpp test -p <member>runs. The baseline was re-measured the same hour with the old binary, same warm state.Warm
mcpp test -p, wall clock:All tests pass under both binaries. The headline symptom we reported is gone on our tree; the remaining one-to-three-second floor is consistent with the items the design document lists as not shipped (no test fast path,
-pdisabling the fast path, per-member replanning under--workspace), and that floor is fine for us.One correction to our original report: we attributed the silent phase to dependency-closure replanning. Your root cause — the post-link verdicts being wiped by the per-invocation regeneration of resolution.json — is the correct mechanism; the scaling we observed matches it just as well. Thanks for the fix. From our side this issue can be closed.
yspbwx2010 commentedon Aug 31, 2026
Follow-up with the full-workspace number. We switched the tree's engine to the 2026.8.30.2 release binary (payloads untouched; the one-time full rebuild that follows from the version participating in the build fingerprint took about 35 minutes, as expected). A warm full-workspace
mcpp testacross our 7 members / 108 tests now completes in 24 seconds wall:Under 2026.8.28.2 the same run took about 17 minutes and could not finish inside a 10-minute foreground window at all. That closes the whole gap we reported, not just the per-member floor. Nothing further from our side.
speak-agent commentedon Aug 31, 2026
Fixed in 2026.8.30.2 (PR #538). Closing on the re-measurement you posted above rather than on ours.
Root cause, recorded here because the shape recurs: both post-link ELF passes were written with a read-back keyed on the artifact's stat, and the record they read lived in
resolution.json— whichprepare_buildrewrites from a fresh json object at the start of every run. Every invocation therefore began by deleting the memo its own backend was about to look for. Within a single invocation the read-back worked, which is why a one-command profile could not see it.What landed:
.mcpp-runtime-verdicts.json, which survives that rewrite;resolution.jsonkeeps publishing a copy.buildandtestshare one output directory with different link-unit sets and were deleting each other's records.MCPP_ALLOW_HOST_LIBS. Making a memo durable creates a staleness obligation that did not exist while the answer was recomputed on every run.Regression test:
tests/e2e/323_post_link_record_durability.sh. It asserts across processes AND across commands, because neither half is visible to a single repeated command, and it asserts the record's CONTENT rather than a wall-clock number — the timing is a machine property, the record is what makes the memo correct.Analysis and measurements:
.agents/docs/2026-08-30-issues-527-529-535-537-analysis-and-design.md.