[spdd] Daily spec work plan - 2026-09-12

[github-actions]

Summary

Daily SPDD rotation processed 5 specs (files 10–14 of 19 in the tracked rotation list): specs/replace-label-spec.md, specs/safe-output-outcome-evaluation.md, specs/safe-outputs-scratchpad-removal.md, specs/security-architecture-spec-summary.md, and specs/security-architecture-spec-validation.md. These are mature, actively-maintained specs. Findings are mostly maintenance-grade: a hard scratchpad-removal deadline is now unblocked, two compliance-matrix gaps remain in the security architecture validation, and the outcome-evaluation spec has several not-started/partial evaluators that diverge from the "Implementation Priority" ordering.

Priority Work Queue

• P0: Delete scratchpad/safe-outputs-specification.md — zero remaining references confirmed (grep -r "scratchpad/safe-outputs-specification.md" docs/ .github/ returned no matches). Deadline 2026-09-21 is approaching.
• P1: Close the two open compliance-matrix gaps in specs/security-architecture-spec-validation.md (§12): Sandbox Isolation runtime host-visibility probe (tracked in #48686) and Threat Detection capability evidence (T-TD-002 to T-TD-007).
• P1: Add dedicated evaluators for not-started outcome types (autofix_code_scanning_alert, create_code_scanning_alert, link_sub_issue, update_project, update_release) that currently rely on evalGenericSticky fallback, per specs/safe-output-outcome-evaluation.md.
• P2: Reconcile specs/safe-output-outcome-evaluation.md "Implementation Priority" list (which only names 5 types) with the full 30-row acceptance map — update or remove the stale priority section.
• P2: Add a §12-style compliance test matrix cross-reference note in specs/replace-label-spec.md §9 pointing at any RL-0xx requirements lacking dedicated Go/JS test evidence.

SPDD Checklist

• /spdd-sync: Delete scratchpad/safe-outputs-specification.md and remove its removal-checklist entry from specs/safe-outputs-scratchpad-removal.md once deleted. Done when: the scratchpad file no longer exists and a repo-wide grep for its path returns zero hits.
• /spdd-generate: In pkg/cli/outcome_eval.go, implement dedicated evaluators for update_project and update_release (currently evalGenericSticky fallback per the acceptance map in specs/safe-output-outcome-evaluation.md). Done when: both types have named evaluator functions and matching entries change from "not-started" to "implemented"/"partial" in the spec's status table.
• /spdd-generate: In actions/setup/js/evaluate_outcomes.cjs, add type-specific logic for link_sub_issue and autofix_code_scanning_alert to replace the generic fallback noted in specs/safe-output-outcome-evaluation.md rows 23 and 21. Done when: the JS implementation-areas column for those rows names dedicated functions instead of "generic fallback".
• /spdd-analysis: Re-audit specs/security-architecture-spec-validation.md §12 Sandbox Isolation row against issue #48686 status; update the row's classification if the runtime host-visibility probe has landed. Done when: the row cites concrete test/evidence IDs or explicitly confirms the gap remains open with a current status note.
• /spdd-generate: Add dedicated evidence entries for T-TD-002 through T-TD-007 (prompt injection, secret leaks, malicious patches, custom prompt, engine override, workflow-failure-on-detection) in specs/security-architecture-spec-validation.md §5/§12, referencing concrete test names in pkg/workflow/. Done when: all six Threat Detection test IDs have named test evidence in the matrix.
• /spdd-reasons-canvas: Add a "Norms" subsection to specs/replace-label-spec.md §8 restating RFC 2119 keyword usage consistent with specs/safe-output-outcome-evaluation.md's Norms section, since §8 currently states requirements (RL-0xx) without an explicit terminology preface. Done when: §8 opens with an RFC 2119 interpretation note matching the style used elsewhere in the repo's safe-output specs.
• /spdd-sync: Update specs/safe-output-outcome-evaluation.md "Implementation Priority" section to either list all 30 types by current status or remove the stale 5-item priority list now that most types have evaluators. Done when: the section reflects the current not-started/partial/implemented counts from the status table above it.
• /spdd-generate: Add a Safeguards subsection to specs/security-architecture-spec-summary.md documenting fail-secure behavior for the two partially-evidenced compliance categories (Sandbox Isolation, Threat Detection) so summary consumers see the residual risk without reading the full validation doc. Done when: a "Known Gaps" or "Safeguards" callout exists in the summary linking to §12 of the validation doc.

Per-Spec Findings

specs/replace-label-spec.md — REASONS canvas

• Requirements: Well-formed W3C-style spec with numbered RL-0xx requirements across an 8-stage processing pipeline (schema validation → gate checks → REST update). Strong.
• Entities: Configuration schema and message schema are both defined with field tables (§4.1–4.2). Strong.
• Approach: Atomic single-REST-call design vs. sequential remove/add is clearly motivated (race-window elimination). Strong.
• Structure: Table of contents, 12 numbered sections, conformance classes present. Strong.
• Operations: 8-stage processing model (§5) is detailed and sequential. Strong.
• Norms: RFC 2119 keyword usage present in §2.2 but not restated at the point of the Security Considerations section (§8), unlike the outcome-evaluation spec's explicit per-section Norms block. Weak.
• Safeguards: §8 covers label allowlist/blocklist enforcement (RL-049) but no explicit compliance-test cross-reference table (unlike the security-architecture-spec-validation §12 matrix). Missing.

specs/safe-output-outcome-evaluation.md — REASONS canvas

• Requirements: RFC 2119 Norms section is explicit and well-scoped (4 MUST/MUST NOT rules for API failure handling). Strong.
• Entities: Outcome Categories table (6 outcomes) and Common OTel Attributes table are clear. Strong.
• Approach: "Same as a repository observer would check" principle is a good design anchor. Strong.
• Structure: 30 per-type sections plus Conformance Test Table — comprehensive but large (1061 lines); could benefit from splitting into per-type files. Moderate.
• Operations: Implementation status table (Current Default Acceptance Map) accurately tracks Go/JS split per type. Strong.
• Norms: Provenance Limits section clearly bounds bot-vs-human inference. Strong.
• Safeguards: MUST NOT emit accepted/rejected on API failure — good fail-safe rule. However, 5 types remain "not-started" (generic fallback only), which is a functional gap, not a spec gap. Weak coverage in implementation, spec itself is fine.

specs/safe-outputs-scratchpad-removal.md — REASONS canvas

• Requirements: Single clear requirement — remove deprecated file by 2026-09-21. Strong (minimal but complete).
• Entities: Two file paths named (deprecated scratchpad, canonical doc-site spec). Clear.
• Approach: Deprecation-then-delete pattern is standard and low-risk.
• Structure: Just a checklist, no headers beyond H1 — appropriate for its narrow scope.
• Operations: Checklist items reference exact grep commands for verification. Good actionability.
• Norms: No RFC 2119 language, but scope doesn't need it (single owner task).
• Safeguards: Verification step (grep for zero references) already confirmed clean in this run — safe to execute the deletion now.

specs/security-architecture-spec-summary.md — REASONS canvas

• Requirements: Summarizes 7 security guarantees (SG-01–SG-07) and 7-layer defense-in-depth model clearly. Strong.
• Entities: Conformance classes (Basic/Standard/Complete), WorkflowState formal model. Strong.
• Approach: TLA+/F*/Z3 formal verification approach is well-documented with a "Behavioral Coverage Map" (15 predicates) and generated test suite (16 tests). Strong.
• Structure: Extensive appendices (A–H), well organized. Strong.
• Operations: "Spec Maintenance Tasks" table tracks task history with dates — good operational hygiene, but every row shows "✅ Done"; no forward-looking open task exists in the table itself, meaning new gaps (like the §12 matrix gaps below) aren't yet reflected here.
• Norms: Versioning section (semver) is clear.
• Safeguards: No explicit callout of the two known partial-evidence compliance categories (Sandbox Isolation, Threat Detection) — a reader of only the summary would not know these gaps exist. Missing.

specs/security-architecture-spec-validation.md — REASONS canvas

• Requirements: Validates spec claims against compiled .lock.yml and JS implementation with concrete evidence per requirement ID (OI-01, PM-01/02, RS-01/02, etc.). Strong.
• Entities: Job architecture (pre_activation/activation/agent/detection/safe_outputs/conclusion) mapped precisely to spec sections. Strong.
• Approach: Executive Summary + Detailed Validation + §12 Gap Analysis structure is thorough and auditable.
• Structure: Numbered validation sections matching spec section references (e.g., "Section 5.2 - OI-01"). Strong.
• Operations: Re-validation Triggers section (Normative Triggers, Trigger #3 Decision Log, Failure Escalation) plus a Proposed CI Mechanism for automation. Strong.
• Norms: Overall Validation Grade "A" — but two categories remain "⚠️ PARTIALLY EVIDENCED": Sandbox Isolation (T-SI-001 to T-SI-007) and Threat Detection (T-TD-001 to T-TD-007).
• Safeguards: Gap Summary explicitly names remaining work and links issue #48686 for the sandbox probe — good traceability. Threat Detection gap (T-TD-002 to T-TD-007) has no tracking issue referenced yet. Missing issue link.

Sync Follow-ups

• After scratchpad/safe-outputs-specification.md is deleted, re-run grep -r "scratchpad/safe-outputs-specification.md" across the whole repo (not just docs/ and .github/) to confirm no stray references remain in specs/, pkg/, or README.md files, then close out the removal checklist in specs/safe-outputs-scratchpad-removal.md (or delete that checklist file too, since its purpose will be fulfilled).
• When new evaluators are added for update_project/update_release/link_sub_issue/autofix_code_scanning_alert in pkg/cli/outcome_eval.go, update the "Current Default Acceptance Map" and "Implementation" status tables in specs/safe-output-outcome-evaluation.md in the same PR to keep spec and code in lockstep.
• When issue #48686 (sandbox runtime host-visibility probe) is resolved, update both specs/security-architecture-spec-validation.md §8c/§12 and the "Spec Maintenance Tasks" table in specs/security-architecture-spec-summary.md to reflect the closed gap.
• Open a tracking issue for the Threat Detection capability-evidence gap (T-TD-002 to T-TD-007) mirroring the pattern used for #48686, and reference it from specs/security-architecture-spec-validation.md §12.

Context

Files reviewed this run (rotation indices 10–14 of 19):

• specs/replace-label-spec.md
• specs/safe-output-outcome-evaluation.md
• specs/safe-outputs-scratchpad-removal.md
• specs/security-architecture-spec-summary.md
• specs/security-architecture-spec-validation.md

Rotation state: last_index advanced from 9 → 14 (persisted to /tmp/gh-aw/cache-memory/spdd-daily/rotation.json). Total tracked spec files: 19. Next run will resume at index 15 (specs/security-architecture-spec.md).

Run URL: https://github.com/github/gh-aw/actions/runs/34703511196

Generated by 📋 Daily SPDD Spec Planner · copilot · auto · 50.6 AIC · ⌖ 7.55 AIC · ⊞ 7.4K · ◷

• expires on Sep 15, 2026, 7:58 AM UTC-08:00