Self-host M5: bootstrap gen-0 link prefers compiled driver when build/bin-compile-aot exists

[PurHur]

Milestone

M5 (retire Zend bootstrap) — parent #1492. Follow-up to closed #2842.

Problem

script/bootstrap-resolve-compile-invoke.sh can invoke compiled build/bin-compile-aot / build/bin-compile-aot-inventory, but a fresh harness tree (build/ empty) still uses Zend for routine M0/M2 compiles until driver-smoke populates artifacts:

bootstrap-compile-invoke: php …/bin/compile.php -o … (gen-0 Zend)

Tar-copy Docker (#2963) drops build/ between docker-exec invocations, so every backlog-architect run re-pays Zend on first link even when master logic supports compiled drivers.

Scope (implementation checklist)

1. script/bootstrap-resolve-compile-invoke.sh

• Resolution order (document in header):

  1. build/bin-compile-aot-inventory if present and probe requests inventory argv
  2. build/bin-compile-aot if present
  3. build/selfhost-native-compile-driver / helloworld compile driver when M3 link env set
  4. Zend php bin/compile.php only when BOOTSTRAP_GEN0_ZEND_ONLY=1 or no artifact and BOOTSTRAP_ALLOW_GEN0_ZEND=1 (default 1 for empty tree)

• Log line must say gen-0 compiled vs gen-0 Zend explicitly.

2. script/bootstrap-selfhost-link.sh + Makefile

• Optional target dependency: bootstrap-selfhost-driver-smoke once per tree or document that first north-star5-verify / loop probe seeds drivers.
• Do not require full M4 ladder for minimal link — only ensure bootstrap-ensure-inventory-argv-driver runs before spine if BOOTSTRAP_USE_INVENTORY_DRIVER=1.

3. script/docker-exec.sh tar-fallback (#2963)

• Persist minimum driver set to host build/ between invocations:
  • bin-compile-aot-inventory
  • selfhost-helloworld-compile (or gen-0 seed script output)
• Sync list in SYNC_BACK_PATHS auto-detection for bootstrap targets.

4. docs/bootstrap-generations.md

• Table: gen-0 Zend (empty tree) → gen-1 after driver-smoke → gen-2 loop.

Not in scope: vendor cold boot (#3022); cli_driver LLVM (#3023).

Repro

docker info >/dev/null
rm -rf build/
./script/docker-exec.sh -- bash -lc 'make bootstrap-selfhost-link' 2>&1 | grep bootstrap-compile-invoke
# Today: gen-0 Zend

./script/docker-exec.sh -- bash -lc '
  make bootstrap-selfhost-driver-smoke >/dev/null
  make bootstrap-selfhost-link
' 2>&1 | grep bootstrap-compile-invoke
# Today: may use compiled driver inside same container; host build/ still empty on next docker-exec

Done when

• Second docker-exec make bootstrap-selfhost-link on host with populated build/bin-compile-aot-inventory uses compiled driver (no Zend) without re-running driver-smoke.
• Fresh empty build/ still works (Zend fallback documented).
• Epic: Self-host critical path — compiler compiles itself (M3→M5) #1492 M0 row no longer lists gen-0 Zend as routine path when drivers exist on host.

Labels

M5 bootstrap generations — compiler-adjacent script work tied to real driver artifacts, not CI-only polish.

[PurHur]

Harness verification (2026-05-28, backlog-architect)

Observed on fresh tar-copy docker-exec tree (empty build/):

./script/docker-exec.sh -- bash -lc 'make bootstrap-selfhost-link' | grep bootstrap-compile-invoke
# bootstrap-compile-invoke: php …/bin/compile.php … (gen-0 Zend)

After BOOTSTRAP_M5_DRIVER_SMOKE=1 make bootstrap-selfhost-driver-smoke, a second make bootstrap-selfhost-link in the same container still prints gen-0 Zend unless build/bin-compile-aot is on the resolve path — driver-smoke populates artifacts but routine link target does not prefer them yet.

Implementation notes for assignee:

• script/bootstrap-resolve-compile-invoke.sh: check build/bin-compile-aot before Zend when file is executable
• script/bootstrap-selfhost-link.sh: optional bootstrap_resolve_compile_driver fast-path when artifact exists (mirror #2842)
• Bisect: BOOTSTRAP_GEN0_ZEND_ONLY=1 must keep today’s behavior

Blocks: #2901 vendor native invoker (closed — verify still needs gen-0 policy)

[PurHur]

Implementation-ready spec (2026-05-28)

Observed

./script/docker-exec.sh -- bash -lc 'make bootstrap-selfhost-link' 2>&1 | grep bootstrap-compile-invoke
# bootstrap-compile-invoke: php …/bin/compile.php … (gen-0 Zend)

Even when build/bin-compile-aot exists from a prior bootstrap-selfhost-driver-smoke in the same tree, routine link does not prefer it.

Scope

Path	Work
script/bootstrap-resolve-compile-invoke.sh	Prefer build/bin-compile-aot / build/selfhost-compile-driver when executable
script/bootstrap-selfhost-link.sh	Document / wire gen-1 driver for minimal bundle link
Makefile	Optional bootstrap-selfhost-link-compiled target

Repro

./script/docker-exec.sh -- bash -lc '
  source script/php-env.sh && make bootstrap-selfhost-driver-smoke
  make bootstrap-selfhost-link 2>&1 | grep gen-0
'

Done when

Second make bootstrap-selfhost-link in same tree prints gen-1 compiled invoke (no Zend) when build/bin-compile-aot is present.

[PurHur]

Harness evidence (2026-05-28, tar-copy Docker):

./script/docker-exec.sh -- bash -lc 'make bootstrap-selfhost-link'

Output includes:

bootstrap-compile-invoke: compiled driver(s) failed — falling back to Zend gen-0 (#2842)
bootstrap-compile-invoke: php /compiler/bin/compile.php -o /compiler/build/selfhost /compiler/test/selfhost/compiler_minimal/main.php (gen-0 Zend)
bootstrap-selfhost-link: OK /compiler/build/selfhost

Tracing shows this is currently “no native driver candidates exist in fresh tree” (none of build/bin-compile-aot, build/selfhost-compile-driver, build/selfhost-native-compile-driver are present), not an actual runtime failure.

[PurHur]

Observed in harness run (2026-05-28) via:

./script/docker-exec.sh -- bash -lc './script/north-star5-verify.sh --strict'

Step 4b still falls back to gen-0 Zend during lib spine smoke link:

=== north-star5-verify step 4b: lib spine smoke link ===
./script/bootstrap-selfhost-lib-spine-smoke-link.sh
bootstrap-compile-invoke: compiled driver(s) failed — falling back to Zend gen-0 (#2842)
bootstrap-compile-invoke: php /compiler/bin/compile.php -o /compiler/build/selfhost-lib-spine-smoke /compiler/test/selfhost/compiler_lib_spine_smoke/main.php (gen-0 Zend)
bootstrap-selfhost-lib-spine-smoke-link: OK /compiler/build/selfhost-lib-spine-smoke

This keeps the M5 ladder “Zend not in loop” from being true even when M4 is green.

[PurHur]

Probe note (2026-05-28): on a clean Docker tar-copy tree, we currently can’t reach the “prefer compiled driver when build/bin-compile-aot exists” behavior because the driver-smoke artifact required by the gen-0 ensure path isn’t produced.

Repro:

./script/docker-exec.sh -- bash -lc 'make bootstrap-selfhost-driver-smoke'

Fails at stage 4 with:

helloworld_compile_smoke: parseAndCompile returned null (parser/CFG spine)
helloworld_compile_smoke: native emit failed at phase=parseAndCompile
bootstrap-selfhost-driver-smoke: missing /compiler/build/selfhost-driver-smoke-emit-aot

Tracking the root symptom in #3004 and the fresh-tree bootstrap-selfhost-link fallout in #3005.