Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .harness/docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,12 @@ Since story-032 the `create` value is **a path at or beneath** one of that stage

`technical_plan` deliberately carries no `type` keyword. story-001 and story-002 write it as a free-form block scalar and story-003 onward write it as a structured object; the validator subset has no union keyword, and those artifacts are committed and unedited. Omitting `type` keeps every nested constraint biting — `schema_validator` applies `properties`/`required` only when the value is a dict — so a malformed *object* form is still rejected while a block scalar is accepted. The reason is recorded in that property's own `description`.

One file in the inventory declares a *contract* rather than an artifact shape. Since story-039 the set of configuration keys the harness reads is declared in **`schemas/harness-config.schema.json`** — one property per key, thirteen of them (`allowed_tools`, `architecture_docs`, `base_branch`, `branch_prefix`, `clean_clone_python`, `logs_dir`, `model`, `permission_mode`, `runs_dir`, `standards_dir`, `stories_dir`, `test_command`, `workflow`), each typed as `load_config` produces it and described by what it governs and what it falls back to. It is in `schemas/manifest.json` like every other file there, and passes the same parametrized draft-2020-12, unsupported-keyword and no-`additionalProperties` checks, but no stage is asked to satisfy it because no agent produces a config file. `harness_config.declared_config_keys(harness_root=None)` is its **only reader**, resolving it relative to its own module through `schema_validator.load_schema` so `schemas/` keeps one reader, and raising `ValueError` naming the path on a missing, unparseable or wrong-shaped schema rather than degrading to an empty or partial tuple — a degraded return would make the coverage below vacuous instead of red.

The declaration is **not a run-time check**. Nothing calls `declared_config_keys` while a run executes, no target's `.harness/config.yaml` is validated against it, and no unknown key is refused — a target carrying an extra key runs exactly as it did before the file existed. `project` is the live example of a key a config file carries and nothing reads, and it is deliberately undeclared: the declared set *is* the set of keys the harness reads, so a key nothing reads is out of the set by construction rather than by oversight.

What the declaration is for is coverage, and the coverage is **set equality in both directions**, twice, in `tests/test_config_keys_are_obeyed.py`. Against `KEY_PROOFS` — a key declared with no proof fails, a proof naming an undeclared key fails — and against an AST scan of `orchestration/` and `scripts/` collecting every `config.get("...")` and `config["..."]`, extensionless scripts included, so a key the harness reads and the schema does not declare fails. Neither comparison is against a second maintained list. Every declared key carries a proof that **varies** it: the value is one the harness would never pick (each carries the token `xyzzy`, and an assertion checks that none coincides with the key's default or with this repository's own configured value), and ten keys are proven behaviourally while `model`, `permission_mode` and `allowed_tools` are proven on the invocation built for a fake runner, because those three are handed to the agent runner and observable nowhere else. `KEY_PROOFS` records which of the two each key gets, so a reader knows what *proven* means for it. A mutation control then replaces each key's read with its fallback literal in a throwaway copy of `orchestration/` and requires that key's proof to go red there, so a proof that sets a key and asserts nothing about its effect cannot survive. This subsumes but does not retire story-028's absence assertions: proving a literal is *absent* and proving the configured value *governs* are different claims, and both are kept.

### Prompts (`prompts/`)

One reusable template per agent role: `planner.md`, `implementer.md`, `tester.md`, `verifier.md`, `documenter.md`, `assist.md`. Each follows the five-layer structure: harness layer (durable rules shared by every agent), role layer (responsibilities and do-not boundaries), workflow layer (workflow priorities), stage layer (current objective), and runtime state layer (`{{placeholder}}` fields the coordinator fills at runtime). Optional placeholders render as `None` when nothing applies.
Expand Down Expand Up @@ -130,7 +136,7 @@ Outliving the session is what made **plan-time validation** possible, and story-

### Target-repository state (`.harness/`)

- `config.yaml` — repository-specific settings (branch prefix, model, permission mode, workflow name, `test_command`, and `clean_clone_python`), plus the optional `base_branch` key story-030 added, shipped as a commented example. Leaving it unset is the normal case: the base is then the repository's own default branch, read from `refs/remotes/origin/HEAD`. Set it only where the base is not the git default.
- `config.yaml` — repository-specific settings (branch prefix, model, permission mode, workflow name, `test_command`, and `clean_clone_python`), plus the optional `base_branch` key story-030 added, shipped as a commented example. The keys the harness reads are declared in `schemas/harness-config.schema.json` and each is proven to govern (see the `schemas/` section); the file itself is validated against nothing at run time. Leaving `base_branch` unset is the normal case: the base is then the repository's own default branch, read from `refs/remotes/origin/HEAD`. Set it only where the base is not the git default.
- `standards/` — repository standards (architecture, coding, testing) that verifiers evaluate against.
- `stories/` — approved story artifacts produced by `l5-plan`, and validated, committed and pushed by it (see the `plan_validation.py` and `plan_commit.py` bullets above). The directory is what `snapshot` watches; its location comes from the config's `stories_dir`, defaulting to `.harness/stories`.
- `runs/<story-id>/` — per-run state, events, and artifacts (not committed).
Expand Down
42 changes: 42 additions & 0 deletions orchestration/harness_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@
import sys
from pathlib import Path

import schema_validator

# The declaration of which keys the harness reads, beside the artifact
# schemas. It is a declaration and not a run-time check: nothing here
# validates a target's config file against it, and no unknown key is
# refused.
CONFIG_SCHEMA_NAME = "harness-config"


def find_target_root(start: Path) -> Path:
for candidate in [start, *start.parents]:
Expand Down Expand Up @@ -50,6 +58,40 @@ def load_config(target_root: Path) -> dict:
return config


def declared_config_keys(harness_root: Path | None = None) -> tuple[str, ...]:
"""The keys the harness reads, declared in schemas/harness-config.schema.json.

The schema ships with the harness code, so it is resolved relative to
this module's package exactly as the artifact schemas are, and read
through schema_validator.load_schema so schemas/ keeps one reader.

The declaration is not a run-time check. Nothing calls this while a run
is executing, no target's config file is validated against it, and no
unknown key is refused; what reads it is the coverage that asserts set
equality against the keys the harness actually reads.

A missing, unparseable or wrong-shaped schema raises ValueError naming
the path, rather than degrading to an empty or partial tuple, which
would silently make that coverage vacuous instead of red.
"""
path = schema_validator.schemas_dir(harness_root) / f"{CONFIG_SCHEMA_NAME}.schema.json"
try:
schema = schema_validator.load_schema(CONFIG_SCHEMA_NAME, harness_root)
except OSError as error:
raise ValueError(f"{path} could not be read: {error}") from error
except json.JSONDecodeError as error:
raise ValueError(f"{path} is not parseable as JSON: {error}") from error
if not isinstance(schema, dict):
raise ValueError(f"{path}: expected an object, found {type(schema).__name__}")
properties = schema.get("properties")
if not isinstance(properties, dict) or not properties:
raise ValueError(f"{path}: 'properties' must be a non-empty object of key declarations")
for key in properties:
if not isinstance(key, str) or not key:
raise ValueError(f"{path}: every declared property must be a key name")
return tuple(properties)


def load_workflow(harness_root: Path, name: str) -> dict:
path = harness_root / "workflows" / f"{name}.json"
return json.loads(path.read_text(encoding="utf-8"))
Expand Down
63 changes: 63 additions & 0 deletions schemas/harness-config.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "harness-config",
"description": "The set of keys the harness reads out of a target repository's .harness/config.yaml. The declared properties are the whole of that set: a key a target's config file carries and no harness code reads is out of the set by construction, which is why 'project' is absent. This file is a declaration rather than a run-time check — nothing validates a target's config file against it during a run and no unknown key is refused, so a target carrying an extra key runs exactly as it would without this schema. It is a contract rather than an artifact shape, so no stage is asked to produce or satisfy it; what holds it is coverage, asserted as set equality against declared_config_keys() in both directions. Every declared key carries a proof that varies it to a value the harness would never pick and observes the harness obey it. Types are the types harness_config.load_config produces: every scalar is a string, because the config parser never coerces, and a list-valued key is an array of strings.",
"type": "object",
"required": [],
"properties": {
"allowed_tools": {
"type": "array",
"description": "Bash command patterns stage agents may run without prompting. Passed to every agent invocation as --allowedTools, and injected into the shared harness-layer prompt partial as {{allowed_tools}}.",
"items": { "type": "string" }
},
"architecture_docs": {
"type": "array",
"description": "Repository-relative paths of the architecture documents assembled into every stage's context.",
"items": { "type": "string" }
},
"base_branch": {
"type": "string",
"description": "The branch a story branch is cut from and the branch l5-plan expects to commit a plan artifact onto. Unset, the base is resolved from refs/remotes/origin/HEAD and then from the literal main."
},
"branch_prefix": {
"type": "string",
"description": "Prepended to a story id to form the story branch name. Defaults to story/."
},
"clean_clone_python": {
"type": "string",
"description": "The interpreter the clean-clone and revert checks run the configured test command under. Unset, the check falls back to test_command's own leading word."
},
"logs_dir": {
"type": "string",
"description": "Directory, relative to the target root, holding raw agent output logs. Defaults to .harness/logs."
},
"model": {
"type": "string",
"description": "The Claude model each stage invocation is run with. Unset, the agent runner passes no model and the developer's default is used."
},
"permission_mode": {
"type": "string",
"description": "The permission mode each stage invocation is run under. Defaults to acceptEdits."
},
"runs_dir": {
"type": "string",
"description": "Directory, relative to the target root, holding per-run state, events and artifacts. Defaults to .harness/runs. Also what l5-status lists."
},
"standards_dir": {
"type": "string",
"description": "Directory, relative to the target root, holding the repository standards injected into every stage's context. Defaults to .harness/standards."
},
"stories_dir": {
"type": "string",
"description": "Directory, relative to the target root, holding approved story artifacts. Defaults to .harness/stories. Also the directory l5-plan snapshots to decide what a planning session produced."
},
"test_command": {
"type": "string",
"description": "The command the clean-clone and revert checks run inside a scratch clone. Read without a fallback: a target that omits it cannot run either check."
},
"workflow": {
"type": "string",
"description": "The name of the workflow definition under workflows/ that a run executes. Defaults to story-workflow."
}
}
}
1 change: 1 addition & 0 deletions schemas/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"changed-files",
"clean-clone-result",
"execution-history",
"harness-config",
"retry-guidance",
"retry-history",
"revert-check-result",
Expand Down
Loading
Loading