Skip to content

story-040: No target-stack literal in harness source - #38

Merged
jerodw merged 1 commit into
mainfrom
story/story-040
Aug 15, 2026
Merged

story-040: No target-stack literal in harness source#38
jerodw merged 1 commit into
mainfrom
story/story-040

Conversation

@jerodw

@jerodw jerodw commented Aug 15, 2026

Copy link
Copy Markdown
Owner

The harness is meant to work on any repository, so its only tie to a target's stack or layout should be in .harness/. An audit found five ties outside it, two of them in a prompt — where no test would ever find them, because nothing executes a prompt.

The scan

orchestration/harness_source.py reports a matched line as a Finding, and tests/test_no_target_stack_in_harness_source.py holds the lists and the assertions.

Two rules, and the second is where the design question was:

  • stack-token — a language or toolchain name, anywhere in harness source.
  • target-layout — a test-directory shape, but only in TARGET_FACING_DIRS = ("prompts", "workflows").

That second restriction is what makes the rule decidable. A scan cannot tell the harness's own tests/ from a target's — but a prompt is text an agent is given, and a workflow declaration names paths in the target, so a path literal there can only mean a target's. No allowlist needed to make it correct.

Two lists, and the split is the point

TEMPORARY_TIES is a burn-down: mentions that name or assume a target's stack, which the-interpreter-is-not-assumed-to-be-python and the-test-location-comes-from-configuration exist to remove. It reaching empty is those stories' completion signal.

PERMANENT_MENTIONS is a dict whose values are the justification:

('scripts/l5-run', '#!/usr/bin/env python3'):
    "the harness's own entry point is a Python program and says so to the kernel"

Both are asserted equal to what scan() reports against this repository, in both directions, and asserted to share no entry. Entries are keyed by path and the exact text of the matched line rather than by line number, so an unrelated edit above a tie does not churn the list and look like progress.

The distinction is applied rather than mechanised: story_coordinator.py's "the oldest supported Python" is on the temporary list, because it is an assumption about the target, while a shebang declaring the harness's own language is permanent.

What it does not catch, stated and exercised

The module states its limits, and the tests then demonstrate them: a language absent from STACK_TOKENS really is invisible, and the layout rule really cannot see orchestration/. The limits are known to match the behaviour rather than to overstate or undersell it — which is the failure mode of every scan that came before this one.

The five audited ties are asserted present by running the scan, not by reading the list. A scan that cannot see the ties that motivated it has not been shown to work.

Verification

  • 2248 passed, 52 written; verification passed on the first iteration, no retries; clean-clone green with the story committed
  • Every absence assertion carries a demonstration that it can fail, built against a throwaway repository root rather than by editing this one
  • Two implementer edits under tests/ were permitted by the revert check — the suite fails with them reverted

Caveat

"The scan is not tamper-proof and does not claim to be" is verified only as a stated limit in the module docstring; nothing in the suite detects the check being deleted. That is true of every standing check in this repository.

🤖 Generated with Claude Code

Implemented by the l5 harness story workflow.
@jerodw
jerodw merged commit a04420f into main Aug 15, 2026
3 checks passed
@jerodw
jerodw deleted the story/story-040 branch August 15, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant