Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .changes/planning/feat-10039-acceptance-criterion-ids-3f9c2a71.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
bump: minor
---

### Added

- `check-plan-outcome.sh` grades a new `criteria` criterion: each acceptance criterion id (`AC1:`) must be named on some phase's `**Satisfies:**` line. It fails an unmapped id (`unmapped=`), an id no criterion defines (`unknown=`), a criterion without an id beside ones with ids, and an id used twice. A plan whose criteria carry no id passes with a warning.

### Changed

- `/planning:interview` writes each Brief acceptance criterion with an id, `- AC<n>: <criterion>`, with an EARS tag after the id under the `ears` format.
- `/planning:plan` numbers any Brief criteria that lack ids, gives a criterion it adds the next id, and has each phase name the ids it satisfies; its outcome gate maps them by script instead of by reading.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
bump: minor
---

### Added

- `/work-items:decompose` traces each source acceptance criterion id to the slices that carry it, reports every id no slice carries before approval, and keeps the id on each carried criterion in an item's `## Done when`.
76 changes: 73 additions & 3 deletions plugins/planning/scripts/check-plan-outcome.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,21 @@
# A plan with no such line passes. Failures are listed as
# `invalid=<line>:<reason>`, the reason one of kind, until, for,
# missing-phase, not-later, last-phase or outside-phase
# criteria every acceptance criterion id maps to a phase. A criterion is a
# list item under an `Acceptance criteria` heading; its id opens
# the item after any checkbox or bold, `AC` then digits
# (`- AC3: ...`). A phase names the ids it satisfies on a
# `**Satisfies:** AC1, AC3` line (a list marker before it is
# allowed); one outside a phase section maps nothing. Fails on a
# criterion id no phase satisfies (`unmapped=`), a satisfied id no
# criterion defines (`unknown=`), a criterion with no id once any
# has one (`without-id=<line>`) and an id defined twice
# (`duplicate=`). Pass prints `ids=<n> mapped=<n>`. A plan with no
# id anywhere passes `ids=0`, with a `warning=criteria-without-ids`
# line when it has criteria; an item reading `none ...` is not one
#
# "Every brief acceptance criterion maps to a phase, and no phase does
# Out-of-scope work" is judgment and stays in the skill's prose; this gate does
# not claim it.
# "No phase does Out-of-scope work, and every phase traces to a criterion" is
# judgment and stays in the skill's prose; this gate does not claim it.
#
# Headings inside fenced code blocks are ignored. A fence closes only on the same
# character with at least the opening length and no info string, so a three-backtick
Expand Down Expand Up @@ -153,6 +164,7 @@ structural="$(
/^#+[ \t]/ {
heading = $0
in_table = 0
in_ac = (tolower(heading) ~ /^#+[ \t]+acceptance criteria[ \t:]*$/)
if (heading ~ /^###[ \t]+Phase[ \t]+([0-9]+(\.[0-9]+)?[A-Za-z]?|[IVXLC]+)[ \t]*:/) {
flush_phase()
phases++
Expand Down Expand Up @@ -184,6 +196,26 @@ structural="$(
sub(/^[ \t]*([-*+][ \t]+)?\*\*Planned breakage:\*\*[ \t]*/, "", text)
breakage_text[breakages] = text
}
in_ac && /^ ?[-*+][ \t]+/ {
text = $0
sub(/^ ?[-*+][ \t]+/, "", text)
sub(/^\[[ xX]\][ \t]+/, "", text)
sub(/^[*_]+/, "", text)
if (match(text, /^AC[0-9]+/) && substr(text, RLENGTH + 1, 1) !~ /[A-Za-z0-9]/) {
id = substr(text, 1, RLENGTH)
if (id in ac_def) ac_dup = ac_dup " " id
else { ac_n++; ac_order[ac_n] = id; ac_def[id] = 1 }
} else if (tolower(text) !~ /^none([^a-z]|$)/) ac_noid = ac_noid " " NR
}
in_phase && /^[ \t]*([-*+][ \t]+)?\*\*Satisfies:\*\*/ {
text = $0
sub(/^[ \t]*([-*+][ \t]+)?\*\*Satisfies:\*\*/, "", text)
while (match(text, /AC[0-9]+/)) {
id = substr(text, RSTART, RLENGTH)
text = substr(text, RSTART + RLENGTH)
if (!(id in sat)) { sat_n++; sat_order[sat_n] = id; sat[id] = 1 }
}
}
{
if (in_phase && $0 ~ /Sanity Check/) phase_sanity = 1
if ($0 ~ /^[ \t]*\|[ \t]*Decision[ \t]*\|[ \t]*What it changes/) { in_table = 1; decisions_table = 1; next }
Expand Down Expand Up @@ -255,6 +287,18 @@ structural="$(
printf "id_without_row=%s\n", no_row
printf "row_without_tag=%s\n", no_tag
for (i = 1; i <= bare_n; i++) printf "bare_tag=%s\n", bare_tag[i]
for (i = 1; i <= ac_n; i++) {
if (ac_order[i] in sat) ac_mapped++
else ac_unmapped = ac_unmapped " " ac_order[i]
}
for (i = 1; i <= sat_n; i++) if (!(sat_order[i] in ac_def)) ac_unknown = ac_unknown " " sat_order[i]
printf "ac_ids=%d\n", ac_n
printf "ac_cited=%d\n", sat_n
printf "ac_mapped=%d\n", ac_mapped
printf "ac_unmapped=%s\n", ac_unmapped
printf "ac_unknown=%s\n", ac_unknown
printf "ac_noid=%s\n", ac_noid
printf "ac_dup=%s\n", ac_dup
}
' "$plan"
)" || die "could not parse: $plan"
Expand All @@ -273,6 +317,13 @@ id_without_row="$(field id_without_row)"
row_without_tag="$(field row_without_tag)"
breakages="$(field breakages)"
breakage_invalid="$(field breakage_invalid)"
ac_ids="$(field ac_ids)"
ac_cited="$(field ac_cited)"
ac_mapped="$(field ac_mapped)"
ac_unmapped="$(field ac_unmapped)"
ac_unknown="$(field ac_unknown)"
ac_noid="$(field ac_noid)"
ac_dup="$(field ac_dup)"

failed=0
report() {
Expand Down Expand Up @@ -337,6 +388,25 @@ else
report planned-breakage fail "invalid=${invalid// /,}"
fi

commas() { sed 's/^ //; s/ /,/g' <<<"$1"; }
if [[ "$ac_ids" -eq 0 && "$ac_cited" -eq 0 ]]; then
report criteria pass "ids=0"
if [[ -n "${ac_noid// /}" ]]; then
printf 'warning=criteria-without-ids count=%s (no acceptance criterion carries an AC<n> id, so none is traced to a phase)\n' "$(wc -w <<<"$ac_noid" | tr -d ' ')"
fi
else
detail="ids=$ac_ids mapped=$ac_mapped"
[[ -z "${ac_unmapped// /}" ]] || detail+=" unmapped=$(commas "$ac_unmapped")"
[[ -z "${ac_unknown// /}" ]] || detail+=" unknown=$(commas "$ac_unknown")"
[[ -z "${ac_noid// /}" ]] || detail+=" without-id=$(commas "$ac_noid")"
[[ -z "${ac_dup// /}" ]] || detail+=" duplicate=$(commas "$ac_dup")"
if [[ "$detail" == "ids=$ac_ids mapped=$ac_mapped" ]]; then
report criteria pass "$detail"
else
report criteria fail "$detail"
fi
fi

readonly max_path_hits=20

# /Users and /home must begin a path: not preceded by a path-continuation character.
Expand Down
44 changes: 44 additions & 0 deletions plugins/planning/scripts/check-plan-outcome.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,50 @@ good_plan | sed '/^## Brief$/a\
**Planned breakage:** build red until Phase 2 for src/: before any phase' >"$TMP/pb-outside.md"
run "a declaration outside any phase section fails" 1 "$TMP/pb-outside.md" '^criterion=planned-breakage status=fail invalid=[0-9]+:outside-phase$'

# criteria: each AC<n> id under an Acceptance criteria heading maps to a phase's
# **Satisfies:** line.
run "a plan with no criteria passes criteria" 0 "$TMP/good.md" '^criterion=criteria status=pass ids=0$'

traced_plan() {
good_plan | sed '/^Move dotfiles under chezmoi\.$/a\
\
### Acceptance criteria\
- AC1: `chezmoi apply` manages the shell config\
- AC2: [event-driven] WHEN chezmoi applies, the README names it' |
sed '/^### Phase 1:/a\
**Satisfies:** AC1' |
sed '/^### Phase 3:/a\
- **Satisfies:** AC2'
}

traced_plan >"$TMP/ac-traced.md"
run "every criterion id satisfied by a phase passes" 0 "$TMP/ac-traced.md" '^criterion=criteria status=pass ids=2 mapped=2$'

traced_plan | grep -v '^- \*\*Satisfies:\*\* AC2$' >"$TMP/ac-unmapped.md"
run "a criterion id no phase satisfies fails and names the id" 1 "$TMP/ac-unmapped.md" '^criterion=criteria status=fail ids=2 mapped=1 unmapped=AC2$'

traced_plan | sed 's/^\*\*Satisfies:\*\* AC1$/**Satisfies:** AC1, AC7/' >"$TMP/ac-unknown.md"
run "a satisfied id no criterion defines fails and names the id" 1 "$TMP/ac-unknown.md" '^criterion=criteria status=fail ids=2 mapped=2 unknown=AC7$'

traced_plan | sed 's/^- AC2: .*$/&\n- the README links the chezmoi docs/' >"$TMP/ac-noid.md"
run "a criterion with no id beside ones with ids fails" 1 "$TMP/ac-noid.md" '^criterion=criteria status=fail ids=2 mapped=2 without-id=[0-9]+$'

traced_plan | sed 's/^- AC2: .*$/&\n- AC1: a second criterion reusing the id/' >"$TMP/ac-dup.md"
run "an id defined twice fails" 1 "$TMP/ac-dup.md" '^criterion=criteria status=fail ids=2 mapped=2 duplicate=AC1$'

traced_plan | grep -v 'Satisfies' | sed 's/^- AC[0-9]*: /- /' >"$TMP/ac-bare.md"
run "criteria with no id anywhere pass with a warning" 0 "$TMP/ac-bare.md" '^criterion=criteria status=pass ids=0$'
run "the no-id warning counts the criteria" 0 "$TMP/ac-bare.md" '^warning=criteria-without-ids count=2 '

traced_plan | grep -v '^- \*\*Satisfies:\*\* AC2$' | sed 's/^## Blast radius$/&\n**Satisfies:** AC2/' >"$TMP/ac-outside.md"
run "a Satisfies line outside a phase maps nothing" 1 "$TMP/ac-outside.md" 'unmapped=AC2$'

traced_plan | sed 's/^- AC1: /- [ ] **AC1:** /; s/^\*\*Satisfies:\*\* AC1$/**Satisfies:** AC10, AC1/' | sed 's/^- AC2: .*$/&\n- AC10: a tenth criterion/' >"$TMP/ac-forms.md"
run "a checkbox or bold id maps, and AC10 is not AC1" 0 "$TMP/ac-forms.md" '^criterion=criteria status=pass ids=3 mapped=3$'

{ traced_plan; printf '\n```markdown\n### Acceptance criteria\n- AC9: quoted\n```\n'; } >"$TMP/ac-fenced.md"
run "a criterion inside a code fence is not graded" 0 "$TMP/ac-fenced.md" '^criterion=criteria status=pass ids=2 mapped=2$'

run "the default run does not require an Approval line" 0 "$TMP/good.md" '^phases=3 status=ok$'
run "the default run ignores an unapproved placeholder line" 0 "$TMP/ap-template.md" '^phases=3 status=ok$'

Expand Down
12 changes: 6 additions & 6 deletions plugins/planning/skills/interview/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,9 +261,9 @@ non-interactive, so never read an environment variable to decide this.
### EARS tags, when the convention selects them

Resolve `acceptance_criteria_format` per the ladder below. Under `free-text`, the default and every
degrade, emit criteria exactly as today: `- <testable criterion>`, untagged, byte-comparable in
shape to a run with no convention surface at all. Under `ears`, every emitted criterion carries a
bracketed pattern prefix on that same plain-bullet form:
degrade, emit criteria exactly as today: `- AC<n>: <testable criterion>`, untagged, byte-comparable
in shape to a run with no convention surface at all. Under `ears`, every emitted criterion carries a
bracketed pattern prefix after its id on that same plain-bullet form:

| Tag | Pattern |
|---|---|
Expand All @@ -274,8 +274,8 @@ bracketed pattern prefix on that same plain-bullet form:
| `[optional-feature]` | `WHERE <feature is included>, <response>` |

```text
- [event-driven] WHEN the upload completes, the manifest is rewritten
- [unwanted-behavior] IF the upload fails, THEN the partial manifest is discarded
- AC1: [event-driven] WHEN the upload completes, the manifest is rewritten
- AC2: [unwanted-behavior] IF the upload fails, THEN the partial manifest is discarded
```

The five names are exactly `ubiquitous`, `event-driven`, `state-driven`, `unwanted-behavior`, and
Expand All @@ -288,7 +288,7 @@ owed a sixth pattern.
Where the tags travel: they land in the Brief's `### Acceptance criteria`, and a consumer that
publishes the Brief verbatim as a tracker container body carries them with it. They do not reach
per-slice criteria, which are freshly authored by whatever decomposes the work, because a vertical
slice owns everything it grades.
slice owns everything it grades; a slice keeps only the criterion's id.

### Resolution ladder for `acceptance_criteria_format`

Expand Down
16 changes: 8 additions & 8 deletions plugins/planning/skills/interview/context/loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -461,7 +461,7 @@ Each section in the PLAN.md Brief captures a specific shape. Keep tight.
- **Ecosystem / stack constraints:** e.g. "no new languages", "no new mapping library"
- **Timeline constraints:** e.g. "must ship before mobile release freeze"

**Acceptance criteria:** bullet list. How to verify done. Each criterion must map to a test or verifiable observation:
**Acceptance criteria:** bullet list, each opening with its id (`AC1:`, `AC2:`, ...). How to verify done. Each criterion must map to a test or verifiable observation:

- ✅ "`GET /api/users/me` returns 401 when the session token is missing" (testable)
- ❌ "Authentication works correctly" (fuzzy)
Expand All @@ -482,11 +482,11 @@ Two behaviors attach here, both defined in the SKILL.md section "Acceptance-crit

```text
### Acceptance criteria
- [ubiquitous] The manifest is valid JSON at rest
- [event-driven] WHEN the upload completes, the manifest is rewritten
- [state-driven] WHILE a rebuild is in flight, reads are served from the previous manifest
- [unwanted-behavior] IF the upload fails, THEN the partial manifest is discarded
- [optional-feature] WHERE checksum verification is enabled, the manifest records a digest per entry
- AC1: [ubiquitous] The manifest is valid JSON at rest
- AC2: [event-driven] WHEN the upload completes, the manifest is rewritten
- AC3: [state-driven] WHILE a rebuild is in flight, reads are served from the previous manifest
- AC4: [unwanted-behavior] IF the upload fails, THEN the partial manifest is discarded
- AC5: [optional-feature] WHERE checksum verification is enabled, the manifest records a digest per entry
```

Spell them exactly this way: `ubiquitous`, `event-driven`, `state-driven`, `unwanted-behavior`, `optional-feature`. A downstream reader matches on the literal name, so a variant spelling is not a near miss; it is an untagged criterion that looks tagged.
Expand Down Expand Up @@ -526,7 +526,7 @@ Write this into `<memory_dir>/<topic-slug>/PLAN.md` (default `.work/`; the topic
- Quality requirement: <prompt>: <requirement>[; target: <measurable target>]

### Acceptance criteria
- <testable criterion>
- AC1: <testable criterion>

### Captured assumptions
- Quality bar: prototype | internal | production
Expand All @@ -547,6 +547,6 @@ Write this into `<memory_dir>/<topic-slug>/PLAN.md` (default `.work/`; the topic

**The Intent line keeps the user's own words.** Quote the user's stated reason, never a paraphrase, and write `none stated` on any of the three Goal lines the user gave nothing for.

**The criteria bullet stays a plain bullet.** `- <testable criterion>` is the emitted shape in both formats: `free-text` fills it as-is, `ears` fills it as `- [<pattern>] <criterion>`. Never a checkbox: `- [ ]` is decompose's slice shape, and a `[pattern]` prefix on a checkbox line is ambiguous with an unchecked box.
**The criteria bullet stays a plain bullet.** `- AC<n>: <testable criterion>` is the emitted shape in both formats: `free-text` fills it as-is, `ears` fills it as `- AC<n>: [<pattern>] <criterion>`. `AC<n>` is the criterion's id, numbered from `AC1` in order and never reused: `/planning:plan` maps each id to the phases that satisfy it and `/work-items:decompose` to the slices that carry it. Never a checkbox: `- [ ]` is decompose's slice shape, and a `[pattern]` prefix on a checkbox line is ambiguous with an unchecked box.

**The arbiter tag decides who resolves the question.** Default `/planning:plan` is fine for execution-shape decisions (orchestration shape, agent rosters, phase nesting) within already-approved scope. Use `USER-RESERVED` for any deferred question whose resolution could change the brief's acceptance criteria, out-of-scope list, or constraints. When in doubt, mark `USER-RESERVED` and let `/planning:plan` surface it at approval time.
2 changes: 1 addition & 1 deletion plugins/planning/skills/interview/context/surface.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ The event stream sends a `ping` every 15 seconds while idle. The page re-fetches
On the page surface, SKILL.md Step 3's confirmation gate runs through the page:

1. Restate the shared understanding with a `restate` op, `problem` first (the Brief's Problem fields per `problem-framing.md` "Problem fields", Gap first and `unknown` where no evidence exists, so the Confirm covers it), then `workSize`, `goal`, `constraints`, `decisions`, `acceptance`, `outOfScope`, `deferred`, and `planningOwned` (the decisions the interview hands to `/planning:plan` to make). The page's summary screen shows it with Confirm and Something's off. The restatement carries the same register-sourced recap as Step 3: one line per `Q<N>` (`Q<N> <status>: <question text> (<resolution>)`), in `decisions`, generated from `round.sh export-ledger --recap --ledger '<ledger>' --out '<data_dir>/recap.md'`, never from the transcript. The exporter's compact form keeps one line per live question, collapses every withdrawn row into one closing `Withdrawn (N): Q<n>, ...` line, and cuts long answers so the recap fits the 20,000-character restate cap; use its output as `decisions` without hand-trimming (it exits 1 after writing only when even the shortest cut cannot fit). The ledger's register lags the page until `round.sh sync-ledger` rewrites its rows, so sync it first, as Wrap-up step 1 does, then run the Step 3 procedure check and cite its exit code.
The `goal` section carries the outcome paragraph and then the Brief Goal's three lines, `- Intent (why, in the user's words):`, `- Vision (where this heads):` and `- When goals conflict, prefer:` ([`loop.md`](loop.md) "Brief template (the literal shape)"); the exporter folds goal text into one paragraph, so after `export-brief --into` writes PLAN.md, rewrite its `### Goal` so those three lines stand as separate `-` bullets.
The `goal` section carries the outcome paragraph and then the Brief Goal's three lines, `- Intent (why, in the user's words):`, `- Vision (where this heads):` and `- When goals conflict, prefer:` ([`loop.md`](loop.md) "Brief template (the literal shape)"); the exporter folds goal text into one paragraph, so after `export-brief --into` writes PLAN.md, rewrite its `### Goal` so those three lines stand as separate `-` bullets. The `acceptance` section writes one criterion per line, each opening with its id (`AC1: <criterion>`), which the export keeps.
2. Wait for a `confirm-understanding` event. `alt: confirm` whose `contentRev` equals the current restatement `rev` passes the gate: `handle` it. The server refuses a Confirm on an older `rev` as stale, so a passing event always names the current restatement.
3. `alt: off` means the gate has not passed. `note-reply` to its `text` with its `seq`, fix the understanding (re-ask or revise questions as needed), and post a new `restate`; the page shows the new one unconfirmed.

Expand Down
Loading
Loading