story-040: No target-stack literal in harness source - #38
Merged
Conversation
Implemented by the l5 harness story workflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The harness is meant to work on any repository, so its only tie to a target's stack or layout should be in
.harness/. An audit found five ties outside it, two of them in a prompt — where no test would ever find them, because nothing executes a prompt.The scan
orchestration/harness_source.pyreports a matched line as aFinding, andtests/test_no_target_stack_in_harness_source.pyholds the lists and the assertions.Two rules, and the second is where the design question was:
TARGET_FACING_DIRS = ("prompts", "workflows").That second restriction is what makes the rule decidable. A scan cannot tell the harness's own
tests/from a target's — but a prompt is text an agent is given, and a workflow declaration names paths in the target, so a path literal there can only mean a target's. No allowlist needed to make it correct.Two lists, and the split is the point
TEMPORARY_TIESis a burn-down: mentions that name or assume a target's stack, whichthe-interpreter-is-not-assumed-to-be-pythonandthe-test-location-comes-from-configurationexist to remove. It reaching empty is those stories' completion signal.PERMANENT_MENTIONSis a dict whose values are the justification:Both are asserted equal to what
scan()reports against this repository, in both directions, and asserted to share no entry. Entries are keyed by path and the exact text of the matched line rather than by line number, so an unrelated edit above a tie does not churn the list and look like progress.The distinction is applied rather than mechanised:
story_coordinator.py's "the oldest supported Python" is on the temporary list, because it is an assumption about the target, while a shebang declaring the harness's own language is permanent.What it does not catch, stated and exercised
The module states its limits, and the tests then demonstrate them: a language absent from
STACK_TOKENSreally is invisible, and the layout rule really cannot seeorchestration/. The limits are known to match the behaviour rather than to overstate or undersell it — which is the failure mode of every scan that came before this one.The five audited ties are asserted present by running the scan, not by reading the list. A scan that cannot see the ties that motivated it has not been shown to work.
Verification
2248 passed, 52 written; verification passed on the first iteration, no retries; clean-clone green with the story committedtests/were permitted by the revert check — the suite fails with them revertedCaveat
"The scan is not tamper-proof and does not claim to be" is verified only as a stated limit in the module docstring; nothing in the suite detects the check being deleted. That is true of every standing check in this repository.
🤖 Generated with Claude Code