Skip to content

Require targeted pre-PR tests before vibe pull requests - #342

Merged
shafty023 merged 1 commit into
closedloop-ai:mainfrom
shafty023:fix/vibe-pre-pr-targeted-tests
Oct 23, 2026
Merged

shafty023 merged 1 commit into
closedloop-ai:mainfrom
shafty023:fix/vibe-pre-pr-targeted-tests

Conversation

@shafty023

Copy link
Copy Markdown
Collaborator

Purpose

Release vibe 1.0.80 with two prevention lessons for pull request preparation.

Lessons

Targeted tests before every vibe pull request

Class: A vibe pull request could open from broad or incomplete validation evidence because the final diff test inventory, exact targeted results and E2E capability choice were advisory instead of required.

The prevention ladder now has typed inventory and tier evidence, a blocking pull request evidence check, guard tests for final diff coverage and tier selection, and concise worker rules. A read-only worker inventories the final committed diff using the repository lane tooling, records exact files, cases, commands, result SHAs and E2E capability, and posts the validated evidence after the pull request opens. Isolated fixes rerun only failed or newly reached targets. Carried results are accepted only when covered file hashes still match.

Failing-first proof: the new tests failed on main because the tier selector and targeted evidence contract did not exist. The implemented guard refuses pull request readiness without the inventory and results artifact.

Evidence: symphony-alpha pull requests 9202, 9208, 9209 and 9210 merged after targeted pre-PR runs and their commits passed every post-merge batch lane on 2026-10-22 and 2026-10-23. This did not catch a hidden artifact slug that no test asserted, so the rule explicitly keeps visual QA mandatory.

Useful read-only work while the writer runs

Class: The orchestrator waited while its sole source writer ran even though independent read-only validation and environment preparation could proceed, adding avoidable delivery latency.

The prevention ladder now has a typed dispatch plan, a mechanical idle-path planner, guard tests for all five prep lanes and result hash validation, and a one-author rule. While the writer runs, the orchestrator drafts the evolving test inventory, warms a configured E2E runner with a fresh ref, runs stable lanes, prepares hidden local QA when the turn ends, and drafts pull request evidence. None of these lanes may edit product source or start another writer.

Failing-first proof: the planner test failed on main because no parallel prep dispatcher existed. The carried-result test also showed that a stale result could survive a covered file change. Both cases now fail closed.

Release metadata

  • vibe version: 1.0.80
  • Instruction text: 3 words shorter than 1.0.79
  • Auto-merge: off

Validation

  • Repository Node suite: 773 of 774 passed in the full run. The one unrelated EPIPE timing failure passed both tests in its isolated file on rerun.
  • Python suite: 2485 passed and 3 skipped in the full run. Two prose contracts exposed by the instruction compression were restored, and the affected 35 tests passed on isolated rerun.
  • vibe workers: 759 tests passed, with typecheck and build clean.
  • Design inventory: 393 tests passed, with typecheck and build clean.
  • Code review cost: 43 tests passed, with typecheck and build clean.
  • Guided manual QA: 50 tests passed, with typecheck and build clean.
  • Plugin manifests: 6 tests passed with typecheck clean.
  • Ruff and Pyright passed.
  • Diff check and the instruction word gate passed.

Risk

The pull request changes workflow policy and evidence validation, not product runtime code. The main compatibility risk is rejecting incomplete older evidence, which is intentional before opening a new vibe pull request.

@shafty023
shafty023 merged commit f96990b into closedloop-ai:main Oct 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant