Live state of the repository. Update after every meaningful work increment (sub-task done, blocker hit, decision made). Entries dated
YYYY-MM-DD. Newest first.
- Each dated section is a single working day (or session).
- Bullets are chronological inside a day.
- Each bullet states what changed, why, and what's next where relevant.
- After a session interruption, the last bullet of the latest day is the resume point.
- Added an explicit beginner distinction between the application's backend/SUT and the AQA admin/backend, with two-terminal commands and a bundled Bun API path that starts the target, runs AQA and opens the admin UI.
- Added a complete executable API scenario example showing risk/invariant references, an authenticated structured precondition, HTTP status/body oracles and replay-oriented evidence; the README now states the exact-once limitation of generic response assertions instead of overstating proof.
- Corrected the bundled Bun walkthrough: topological workspace build, example
directory initialization, generated
smokeprofile, explicit SUT base URL, and separate AQA admin startup. Pack authoring now explains manifest registration rather than suggesting loose scenario files are executable.
- Reworked the README around the shipped product truth: complete feature inventory, differentiator matrix versus conventional testing tools, explicit repository/provider evidence boundaries, three installation paths, junior first-project walkthrough, CI recipe and examples cookbook.
- Added examples for risk discovery, deterministic replay, external-result ingestion, mutation/holdout gates, disaster recovery and custom packs. No credentials or provider claims were added.
- Added
scripts/chaos-benchmark.mjsandbenchmark:chaos. It exercises the realRunnerQueuewith deterministic lease-expiry and worker-failure injection, terminal accounting, completion thresholds and zero-stranded-job assertions. Source attribution refuses a dirty worktree and the JSON output carries an explicit provider-scale evidence boundary. - Added
docs/operations/load-chaos-evidence.mdwith the fault model, local SLO thresholds, execution commands and the exact evidence that remains deferred to PostgreSQL/provider/production environments. - Next: commit and execute the probe, then connect the result to the roadmap audit before implementing mutation holdout governance and ecommerce failure journeys.
- Added
splitMutationCoverageHoldout()to@aqa/ingest. It produces a deterministic hash-ranked train/holdout split bound to a project/revision key, enforces minimum set sizes, and emits a plan digest. The primitive is provider-neutral and deliberately does not claim that a mutation producer or protected regression execution has run. - Added regression coverage for determinism, disjointness, sizing and invalid split keys. Next: expose the split in the mutation evidence contract and complete the ecommerce timeout/webhook/idempotency failure journey.
- Added
verifyCommerceFailureJourneys()and contract coverage for the local reference merchant. It executes one-stock/two-cart contention, verifies exactly-once idempotent retry, replays a signed webhook and rejects a conflicting event, then proves an approved financial mutation remainsunknownafter injected transport ambiguity. - Evidence is structured and explicitly bounded to
in_memory_reference_only; real provider, network and settlement execution remain final-gate evidence. Next: run the full hosted matrix and connect minimized failure artifacts to finding/replay records.
- Added
buildMinimizedCounterexampleReplay()to@aqa/reporter. It invokes the bounded methodology shrinker through a caller-owned failure predicate, emits a redactedreplay/counterexample.<finding-id>.min.jsonartifact and returns the exact evidence path to attach to the finding. No command, network call or candidate logging is performed by the helper. - Reporter coverage now verifies reduction, finding/scenario/run binding and secret redaction. Local reporter suite: 10/10 passed. Next: run the complete hosted matrix for the ecommerce/replay slice and then close the remaining repository-side evidence gaps.
- Reworked the inventory contention journey after review exposed that promise scheduling around a synchronous reference was only sequential. It now uses a deterministic async check/commit barrier with a post-barrier stock re-check, while separately asserting the real reference merchant's idempotent retry.
- The ambiguous mutation path now performs an observable local commit before
throwing the injected transport timeout; the journey asserts both committed
state and
unknownoutcome. Commerce suite remains 66 passed, 2 provider skips, 0 failed.
- Minimized replay artifacts now use key-aware
redactJson()before serialization and include the finding ID in the filename, preventing secret leakage and cross-finding overwrites. - The ecommerce journey now exercises two real reference carts in addition to the async overlap model, and still verifies exactly-once retry. Reporter and commerce suites remain green after the correction.
- Extended mutation regression evidence with an optional
plan_digestand addedevaluateMutationHoldoutRegressionEvidence(). The evaluator projects the report onto the immutable holdout set, requires an exact digest match, and fails closed on plan mismatch or contamination. - Ingest suite: 25/25 passed. This closes the repository-side holdout contract; actual producer-scale mutation execution remains an explicitly deferred final gate.
- Fixed the chaos CLI to exit nonzero when thresholds fail, rebuild the server artifact before loading it, and cover both passing and failing subprocess behavior. The runbook now makes the build requirement and artifact freshness explicit.
- PR #241 (
audit-retention-runbook) is merged onmainafter the complete hosted technical matrix passed: build, lint/typecheck, Bun/Node tests, PostgreSQL/S3/OCI integrations, live Prometheus/OTLP, CLI E2E and both Playwright admin journeys. It closes the retention runbook and multi-runner identity/provenance slice. - Remaining active repository work is now deliberately narrow: (1) bounded local load/chaos execution evidence and SLO/threshold aggregation, (2) methodology mutation-to-regression scale/holdout governance, and (3) complete ecommerce failure-injection journeys for timeout/unknown outcome, webhook replay, inventory/payment/idempotency races and reconciliation, plus the blueprint-required bug shrinking/minimization workflow and a versioned sizing/capacity-planning runbook.
- The following are explicitly deferred final gates, not silently marked done: real provider credentials and settlement/tax/shipping/WMS execution, KMS/Vault/WORM/PITR/RTO-RPO/IdP/mTLS deployment evidence, production-scale mutation producer evidence, penetration testing, SOC2/ISO, legal SLA, reference customers and independent assurance.
- Next slice: implement the bounded local load/chaos evidence contract with a real complete journey and explicit no-provider evidence boundary, then run the hosted matrix before moving to methodology scale closure.
- Added
docs/operations/sizing.mdwith reference Small/Medium/Large profiles, storage estimation, capacity dimensions, scaling rules and explicit limits of local evidence. Addedscripts/capacity-benchmark.mjsplusbenchmark:capacity, a reproducible provider-neutral queue lifecycle baseline that emits revision-bound JSON without secrets or customer data. - Added
shrinkJsonCounterexample()to@aqa/methodology. It performs deterministic bounded delta-style shrinking over JSON values through a caller-owned failure predicate, with depth/size/attempt/reduction limits and no command execution or candidate logging. Methodology suite: 37 passed. - Hosted CI exposed and then verified a legacy double-encoded JSONB audit projection path; the SQL kind filter now handles object and serialized-string shapes before pagination. The final PR rerun passed PostgreSQL, build, unit, CLI, Playwright, telemetry, OCI and S3 gates.
- Shrinking and benchmark generation are lazy/bounded, and the benchmark rejects dirty source attribution (apart from the environment lock file). Next: connect shrinking to replay/finding evidence, then implement the ecommerce failure-injection journey and mutation holdout governance.
- Propagated the host-owned worker identity through the real remote queue and
Kit lifecycle. Remote leases now bind the configured identity to the verified
control-plane subject, while the separate real-run OTLP journey verifies the
validated
aqa.runner_idon intermediate trace evidence. - Added the complete remote-worker assertion that dequeue/lease identity, persisted audit actor and trace-safe provenance remain correlated. Local evidence: observability 21/21 and Kit 196 passed / 0 failed / 2 platform skips.
- Next: add bounded local load/chaos evidence and methodology governance closure; protected provider and independent assurance gates remain deferred.
- Added
docs/operations/audit-retention-runbook.mdwith fail-closed procedures for scoped projections, checkpoint verification, legal holds, archive/purge reconciliation and incident handling. - Added ADR-284 to make the chain/checkpoint boundary explicit: local query retention is not WORM evidence, and destructive purge is not allowed without independent checkpoint coverage and provider read-back.
- Next repository slice: complete local multi-runner/trace provenance and load/chaos evidence; provider immutability, PITR and independent assurance remain deferred final gates.
- Merged PR #238 (
e68467f) after the full hosted matrix passed, including Playwright, CLI, PostgreSQL, S3, OCI and live Prometheus/OTLP journeys. A transientonnxruntime-nodenetwork timeout was rerun successfully. - Added
summarizeAuditEventsto the store contract and both adapters. The PostgreSQL implementation aggregates in SQL, while MemoryStore preserves semantic parity; both apply the same tenant, time-window and kind filters. - Added
GET /api/audit/summaryand a server contract proving that a tenant cannot observe another tenant's audit counts. Local evidence: store/server builds, 117/117 server tests, and Biome pass. - Admin wiring is implemented in PR #240; after merge, add bounded retention/aggregation operational controls. External WORM/KMS/PITR/IdP, provider credentials and independent assurance remain explicitly parked at the final production gate.
- Wired the admin Audit page to
/api/audit/summary, showing tenant-scoped event-kind counts while retaining fixture mode when the endpoint is absent. - Extended the Playwright operations journey to intercept both live requests and assert the rendered summary. Local evidence: admin build and 5/5 operations Playwright tests pass.
- PR #240 is awaiting hosted technical CI. Next after merge: implement bounded retention controls and the operational retention runbook.
- Audit events now derive authoritative
org/projectmetadata from the persisted run at the store boundary, with an explicit scope override for future producers. MemoryStore records the same provenance for parity. - Scoped audit projections now fail closed for legacy events without tenant metadata instead of treating them as globally visible. PostgreSQL applies the same exact-match predicate and finding status/verification audit writes inherit the owning run scope.
- Added MemoryStore and hosted-PostgreSQL contract coverage for same-tenant visibility, cross-tenant exclusion and legacy-row exclusion. Local evidence: store build and 21/21 store tests pass. Next: complete audit retention and aggregation projections, then wire the scoped admin journey.
- Added
publishMethodologyArtifactto the store contract and both adapters. Publication now re-reads the approved proposal under the artifact retention lock before writing the artifact and lifecycle, closing the stale-read race where purge could win between API validation and persistence. - Retention purge now scrubs staged copies from pending proposals as well as decided proposals. Scrubbed proposals disappear from operational lists and cannot be approved afterward; direct lookup remains available for audit-safe diagnosis without the artifact copy.
- Approval now uses the same artifact lock/re-read fence in MemoryStore and PostgreSQL. API conflict responses classify purged/changed proposals as 409.
- Added deterministic MemoryStore and hosted-PostgreSQL contract assertions for
pending-copy scrubbing and post-purge approval rejection. Local evidence:
store/server builds pass, 136 tests pass, and
git diff --checkis clean. Hosted PostgreSQL evidence is still required in CI before merging. Next: commit/push this slice, run the technical CI gate, then continue with the next roadmap block (durable audit projections).
- Corrected the PostgreSQL audit projection so
kindfiltering happens in SQL beforeLIMIT; previously a newer non-matching event could consume the limit and make a valid matching event disappear from the projection. - Added a durable contract assertion with a newer
infoevent and a boundedoracle_evaluatedquery. Next: explicit tenant metadata for all producer paths, retention/aggregation policy, and complete API/admin evidence without relying on lenient legacy rows.
- Hosted PostgreSQL proved artifact deletion but caught a remaining read-back
failure for approved/rejected proposal copies: the scrub ran after the
lifecycle/artifact transaction and could expose stale or concurrent state.
Proposal-copy scrubbing now executes inside the same transaction as lifecycle
and artifact deletion. The hosted rerun then showed that the JSONB path
predicate did not match the approved record reliably; scrubbing now locks
tenant-scoped proposal rows, parses them canonically, and matches terminal
status plus artifact identity/revision before removing the copy. The first
hosted attempt also exposed a provider JSONB double-encoding shape at this
read boundary; normalize one additional serialized layer before parsing.
The next hosted attempt exposed the symmetric write issue (
payload - keycannot operate on a JSONB scalar); the scrub now normalizes object/string payloads in SQL before removing the copy and preserves unexpected types. The resulting catalog contract is now explicit in both adapters: terminal proposals whose retained artifact was purged are omitted from operational lists, while direct lookup remains available without the sensitive copy. Additional concurrency hardening closes the remaining retention risks: publication rejects scrubbed proposals and always uses atomic artifact+ lifecycle persistence; MemoryStore serializes lifecycle/purge decisions; artifact-kind conflicts, tenant-scoped assertions, and PostgreSQL advisory lock ordering are enforced. Local store/server/methodology builds pass and the store contract remains 20/20. Next: rerun hosted CI, resolve only the now-obsolete review threads, and merge. Local store build and contract: 20 passed / 0 failed. Next: rerun hosted PostgreSQL and confirm the full CI gate.
-
The latest hosted PostgreSQL contract run found one remaining JSONB shape regression in legacy retention backfill: the artifact parser received the provider's object-shaped JSONB value instead of canonical serialized JSON. Normalized that read boundary and recorded the lesson; rerunning local gates and hosted CI is the next action.
-
The rerun then exposed a second hosted-only retention defect: purge returned one expired lifecycle but scoped artifact read-back still found the artifact. The deletion path now removes artifacts by tenant columns plus validated envelope identity/revision inside the same transaction; local gates and a fresh hosted contract run are required next.
-
The latest hosted attempt showed the operation must prefer its caller scope over nullable legacy row scope columns. Purge and proposal-copy scrubbing now use that authoritative scope with persisted columns as fallback.
-
Restart verification isolated the remaining issue to the reused PostgreSQL store instance: a fresh instance saw the purge while the original could read stale session state. The adapter now rotates its pool and reruns migrations after destructive reconciliation; hosted CI must revalidate this boundary.
- Added a live, tenant-scoped previous-revision comparison in the admin review
workspace. Revision
Nloads only the publishedN-1artifact; revision 1 and missing previous revisions fail closed with an explicit empty state. - Added the live
approved → publishaction. It sends the exact staged payload and proposal ID to the existing approval-bound publication API; mock mode remains unable to publish. Browser evidence: methodology review 3/3 passed, including previous-revision comparison and publication request. - Retention/archive/expiry controls are now implemented; the real provider/authenticated proposal-creation journey and external assurance remain deferred final gates.
- Technical hardening after automated review: publication now reports the
store durability boundary (
durablevsephemeral), revision 1 renders an explicit no-history state, the diff uses a real responsive two-column style, and browser evidence asserts mock read-only behavior, tenant headers and the exact published envelope. - Added hosted-journey coverage for revision-one no-history behavior and a post-publication refresh/read-back assertion; focused methodology browser evidence is now 5/5 passed.
-
Added validated tenant-scoped lifecycle records for published methodology revisions: active/archived state, bounded retention and archive deadlines, operator reason, and legal hold.
-
Added Memory/Postgres persistence, authenticated lifecycle inspection, archive, legal-hold and retention-reconcile routes. Expiry purge deletes the lifecycle and artifact only when the record is not under legal hold.
-
Evidence: methodology/store/server focused suite 139 passed / 0 failed; full adapter-hosted PostgreSQL retention evidence and external WORM/backup proof remain to be run or supplied.
-
Operator retention controls are now exposed in the admin review workspace; next is the real authenticated provider journey. External assurance remains a final promotion gate.
-
Automated technical review then exposed retention edge cases; hardened the slice with DLP-checked reasons, lifecycle binding/cloning, serialized in-memory transitions, transactional PostgreSQL publication and purge, terminal-proposal payload scrubbing, legacy-artifact backfill, explicit UI lifecycle errors and official package-test registration. Follow-up evidence: methodology 33 passed, store/server 136 passed, browser 5/5. Provider credentials, hosted WORM/restore evidence and external assurance remain deferred final gates by owner decision.
-
Follow-up review hardening: lifecycle transitions in PostgreSQL now lock and re-check the governing artifact inside the transaction, MemoryStore publish replay does not reset an existing lifecycle, and the DSN-backed store contract covers atomic publication, lifecycle read-back, expiry purge and terminal proposal scrubbing. Workspace typecheck and store tests pass; this follow-up is queued on PR #234 before merge.
-
Hosted PostgreSQL caught one real regression in the new transaction path: JSONB read-back from the
postgresdriver was not normalized before strict envelope parsing. The boundary now selectspayload::text, decodes it via the shared adapter helper and fails explicitly if the row disappears; local typecheck/build/store/server tests pass and CI must re-prove the fix.
- Added a durable
pending → rejectedtransition with an independent authenticated reviewer, a bounded reason, DLP validation, strict parsing and conflict-safe Memory/Postgres persistence. Rejected records retain the staged envelope and cannot also contain an approval. - Added the server route and live admin action; mock mode remains read-only. Local methodology 30 passed, store 21 passed / 1 intentional provider skip, server 147 passed / 0 failed / 1 intentional Postgres DSN skip; workspace typecheck/lint and admin build/typecheck passed.
- Next: implement previous-revision side-by-side diff and the live approve → publish journey, then retention/archive controls. Provider-backed evidence, penetration testing and independent assurance remain deferred final promotion gates by explicit owner decision.
- Hosted PostgreSQL initially rejected a timestamp-based test ID as a possible PAN through the DLP guard. The fixture now uses deterministic semantic IDs; the local store suite is 18 passed / 0 failed, and the required hosted PostgreSQL contract must re-run on this revision.
- Hardened the rejection slice after technical review: persisted decision invariants are revalidated on read, terminal UI actions are mutually disabled, rejection reasons survive normalization, notices identify the correct action, and proposal selection clears stale reasons. Added parser boundary tests, Postgres durability assertions and a Playwright mock-mode governance test. Local typecheck, admin build, methodology/store/server tests and the focused Playwright test pass.
- The hosted PostgreSQL rerun exposed and fixed a test-only fixture-scope error
in the reopen assertion; no production persistence defect was found. The
corrected local contract remains 18 passed / 0 failed and is queued for
hosted re-verification on commit
5fbf7a0. - The live browser test then exposed a real route-context wiring defect: the
shell mode was not passed into routed pages, so methodology review could show
Live while remaining action-disabled. The context now carries
mode; the focused live rejection and mock governance journeys both pass 2/2.
- PR #231 merged after protected hosted CI passed: PostgreSQL 16, S3, OCI, Bun, Node22, build, CLI smoke, admin Playwright and live telemetry all passed; branch protection remains strict with 12 required contexts.
- Completed in this slice: tenant-scoped proposal detail, bounded staged envelope, metadata-only queue, authenticated reviewer resolution, fail-closed payload loading and approval, and payload retention across the atomic approval transition.
- Remaining for this local workstream: authenticated complete browser journey through proposal creation → review → approve → publish, explicit reject reason/audit semantics, side-by-side previous-revision diff, retention and archive policy. External provider evidence and independent assurance remain deferred final promotion gates by explicit owner decision.
- Added the first control-plane review surface at
/methodology-review: tenant proposal queue, exact revision/digest/payload inspection, explicit human-gate warning, and live approval against the authenticated operator. Mock mode is intentionally read-only and cannot fabricate an approval. - Added the navigation smoke assertion. Local admin typecheck and production build pass. Next: wire a complete live browser journey with authenticated proposal creation → review → approval → artifact publication, then add retention/archive controls.
- Corrected the review boundary before delivery: pending proposals now retain a validated staging envelope, list responses omit the payload, and a tenant-scoped detail route supplies the exact content to reviewers. This prevents a UI-only preview from being mistaken for evidence.
- Hardened the live journey after adversarial review: admin requests now carry
tenant scope, unwrap the API record contract, resolve the authenticated
reviewer identity from
/api/session, and keep approval disabled until the staged payload is loaded. Store approval transitions retain the staged envelope for audit/review. Local server 147 passed, store 21 passed / 1 intentional provider skip, admin build/typecheck and workspace typecheck pass; hosted CI must re-prove this revision.
- Added ADR-281 and tenant-scoped methodology list/detail routes plus an
approval-bound publication route. The API validates the envelope and DLP
boundary, binds proposal kind/id/revision/digest, requires
risk-map:edit, checks independent approval, authenticated reviewer identity and a host-owned proposal-origin verifier, and keeps idempotent/conflicting revisions safe through the durable store. The HTTP adapter's query/params boundary is covered, including invalid kind filtering and digest conflicts. - Server evidence: 147 tests passed / 0 failed / 1 intentional Postgres DSN skip, typecheck and Biome passed.
- Added durable tenant-scoped proposal records and atomic Postgres approval transitions. Publication now accepts only an approved persisted proposal ID; proposals survive restart and same-ID conflicts fail closed.
- Evidence: store 21 passed / 1 intentional recovery skip, server 147 passed / 1 intentional Postgres DSN skip, methodology/build/typecheck and Biome passed. Next: admin review/diff UI and browser complete journey; external provider and assurance gates remain deferred-final-gate.
- Hosted CI caught and the local fix corrected a PostgreSQL
42P18parameter inference regression in the atomic approval update; the required provider contract must be rerun before this slice can merge. - A second hosted run exposed a JSONB status-predicate mismatch in the same atomic transition; the implementation now uses a full-record JSONB compare-and-swap. The hosted contract must pass on this revision before merge.
- Hardened the workflow after adversarial review: strict allowlisted proposal and approval parsers reject unknown/DLP-sensitive fields, new proposals must be pending, persisted records are revalidated on reads, human proposer identity cannot be delegated to the agent verifier, and the Postgres contract now races two approvals to prove a single winner. Local methodology/store/ server tests, full typecheck and lint pass; CI must re-prove this revision.
- Added ADR-279 and a provider-neutral envelope for methodology artifacts: bounded canonical payload, schema version, stable identity, revision, UTC timestamp, SHA-256 digest, and attack-tree validation on create/reload.
- Added tamper, schema-drift, malformed-tree, and round-trip tests. Durable Postgres/S3 persistence, tenant authorization, migrations, retention, and admin visualization remain the next local slices; external evidence gates remain deferred to final promotion.
- Added ADR-280 and
StoreProvidermethods for immutable methodology revisions.MemoryStoreandPostgresStorenow revalidate envelopes, isolate records by tenant scope, retain revisions, support idempotent replays, and reject same-revision digest conflicts atomically in Postgres. - Store package evidence: typecheck passed, 20 tests passed and 1 provider test skipped without DSN; Biome and diff checks passed. Next: authenticated API/control-plane publication and admin review UI, then retention/archive policy.
- Rejected methodology payloads that shared DLP would redact before digest acceptance; reads now revalidate payload/digest, MemoryStore returns defensive copies, and Postgres scopes listings by exact
org/projectcolumns instead ofLIKEprefixes. - Added DLP and Postgres tenant/restart regression coverage. Local methodology suite: 28 passed; store suite: 20 passed, 1 provider skip; workspace lint remains green. Next: authenticated publication API and admin approval/review UI.
- Aligned partial-scope semantics across adapters: a project-only query now returns that project across organizations in both MemoryStore and PostgresStore, with regression coverage.
-
Added methodology approval governance. Agent-generated risk maps, attack-tree, FMEA and coverage artifacts can now be represented as bounded digest/revision proposals and become approved only through an independent, time-bounded human approval bound to the exact artifact. Added canonical hashing, expiry/mismatch/self-approval fail-closed checks, 4 tests and ADR-278. Durable persistence and authenticated identity mapping remain host integration work; the local methodology contract is now closed.
-
Parked external promotion gates by explicit owner decision. Provider credentials/live deployment evidence and independent assurance (penetration test, SOC2/ISO, legal SLA, references and external sign-off) are now marked
deferred-final-gatein the roadmap audit. They remain mandatory for future production/customer promotion, but do not block side-project development; repository-testable contracts, local complete journeys and ecommerce gaps remain active. Next: resume the first still-open local workstream. -
Aligned review governance with the operator decision.
AGENTS.md,docs/RULES.mdand Copilot instructions now make Copilot review the default, while permitting an explicit documented operator opt-out; technical CI and the human branch-protection review remain mandatory. This prevents process docs from contradicting the active operating instruction. -
Enabled live repository governance controls.
mainnow requires the technical CI matrix and one pull-request approval, enforces linear history and conversation resolution, and disallows force-push/delete exceptions. Created the six protected provider/evidence Environments with required reviewer and protected-branch policy. Provider secrets remain intentionally absent, so this closes governance configuration but not provider execution evidence. -
Re-audited live external readiness before governance activation. The earlier snapshot found only the
copilotEnvironment, no provider credentials, and no protection onmain(404 Branch not protected). That snapshot is superseded by the governance controls recorded above; provider credentials and live provider execution evidence remain outstanding. No Stripe, gift-card, OIDC, mTLS, PostgreSQL-recovery, artifact/KMS or production DR workflow was started without an operator-owned protected environment. The exact result is recorded in the roadmap audit; this is an external configuration blocker, not a code failure. -
Bound mutation regression evidence to the workflow revision. The protected gate now passes
${github.sha}toaqa mutation regression, which rejects evidence whosesource_revisionbelongs to another commit. Added a regression test for cross-revision evidence and completed the full suite: 862 pass / 5 explicit skips / 0 fail. ADR-277. -
Connected protected mutation CI to regression execution evidence. The reusable
mutation-evidence-gate.ymlnow accepts optional producer and file inputs for observed mutant/scenario executions, validates their paths and invokesaqa mutation regressionin the protected Environment. Existing callers remain compatible when the optional artifact is absent; the absence is explicitly not treated as execution proof. Operations documentation was updated. Next: execute it with an operator-owned mutation producer and protected environment. -
Added mutation-to-regression execution evidence.
@aqa/ingestnow validates bounded evidence containing source revision, run ID and observed killed/survived outcome for every reviewed mutant/scenario pair. The newaqa mutation regressiongate fails closed on missing, unreviewed or contradictory observations and is covered by ingest and kit tests. This closes the repository-side mutation-to-regression automation contract; a protected producer job and scale execution remain deployment work. ADR-276. -
Added runtime stateful journey execution.
@aqa/methodologynow compiles legal graph paths into stable actor-bound plans and executes them through injected actor actions with authoritative state read-back, transition-bound temporal observers, bounded cancellation/timeout and guaranteed cleanup. Callback errors are reduced to safe codes and contexts are never returned. Added 18/18 methodology tests and ADR-273. -
Integrated stateful journeys into the real CLI/worker lifecycle.
runRun()andmakeKitWorker()now accept only host-owned compiled journey bindings, execute them through the canonical run directory and hash-chained audit events, and retain only digest/transition metadata and safe codes. Added a complete run-boundary test; kit-focused suite passes 53/53. Next: bind journey correlation IDs to the federated OTLP trace report. -
Added safe journey-to-trace correlation. The audit span observer now exports only allowlisted technical journey identifiers, state transitions, actor IDs, outcome and the SHA-256 plan digest; arbitrary payloads, secrets and PII are rejected before OTLP export. Added regression coverage and ADR-274. Collector sampling, exporter integrity and protected production provenance remain open.
-
Added holdout semantic calibration. Clustering now creates a deterministic SHA-256-digest-bound train/holdout split and evaluates the release policy only on unseen reviewed pairs. Added 10/10 clustering tests, package docs and ADR-275. This reduces threshold overfit but does not replace independent methodology validation or external governance.
-
Added bounded OTLP trace federation.
@aqa/ingestnow normalizes OTLP JSON span metadata with ID/time/size limits, redacts credential-like attributes and URLs, accepts optional OTLP status, and federates multiple sources with deterministic deduplication plus orphan/conflict reporting. Added regression coverage and ADR-272. Local ingest build, 21 ingest tests, lint and diff checks pass. This is a repository contract only; exporter integrity, sampling, clock quality, retention and production provenance still require protected deployment evidence. Next: implement runtime stateful journey compilation/execution. -
Added the protected mutation evidence workflow. The reusable
mutation-evidence-gate.ymlaccepts producer-uploaded mutation and reviewed coverage artifacts, enforces safe relative paths, runs in a protected GitHub Environment and invokes the fixed AQA coverage gate. Operations docs and ADR-271 define the producer/provenance boundary; a completed operator run is still required before this becomes live evidence. -
Added mutation-to-regression coverage. The reviewed manifest contract maps evaluated mutants to risk and regression scenario IDs, while
aqa mutation coveragegates mapping completeness and mapped kill rate and emits per-risk coverage. Ingest/kit tests pass locally; ADR-270 and package docs record that external mutation execution and provenance binding remain separate requirements. -
Closed the semantic clustering implementation gap. Added
clusterFindingsBySimilarity()with bounded deterministic token similarity or an operator-owned embedding callback, same-risk eligibility, explicit thresholds and auditable member/score/method edges. A regression found during implementation (semantic members being discarded by exact-cluster reduction) was fixed and covered. ADR-267 and package docs updated. Remaining gaps are threshold calibration/governance, persistent root-cause lifecycle and bulk fix/verify orchestration. -
Added semantic threshold calibration. Reviewed finding pairs can now be scored into a bounded confusion-matrix report with precision, recall, F1 and false-positive rate, then evaluated against an explicit policy. Added ADR-269, docs and a 9/9 clustering suite. Holdout/independent methodology validation remains external governance, not an unearned green state.
-
Added the bounded stateful journey graph contract.
@aqa/methodologynow validates versioned states/transitions, reachability, terminal states, non-terminal dead ends and legal transition paths without executing side effects. Added ADR-268, package docs and 14/14 methodology tests. This closes the static graph gap; real actor sessions, temporal observers, runtime cleanup and trace federation remain execution-level work. -
Added the first mutation-evidence release gate.
@aqa/ingestnow normalizes bounded flat/Stryker-style reports, preserves non-killed outcomes and computes a deterministic score;aqa mutation gate <report.json> --min-score Xapplies the explicit threshold without executing mutators. Added malformed/duplicate/status regression tests, package documentation and ADR-266. Local ingest and kit typecheck/build/test gates pass; next is full repository validation, PR/CI and then semantic root-cause clustering. -
Closed the operational oracle-calibration gap. Added
aqa oracle calibrate <corpus.json>with a versioned, bounded and fail-closed gold-corpus parser, Brier/ECE/reliability-bin output and an explicit--max-ecerelease gate. Unknown fields and private rationale leakage are rejected; the command never invokes an LLM. Kit tests, build, typecheck and lint pass. Remaining methodology work is mutation-scale execution, semantic root-cause clustering and independent calibration governance. -
Completed the formal roadmap audit. Added
docs/internal/roadmap-completion-audit-2026-09-20.mdand updated the README to replace the stale pre-production v1.9 summary with evidence-based status. The audit marks repository capabilities separately from protected provider execution and independent security/compliance obligations; it intentionally does not claim 100% production completion before those external controls run. Next: validate and merge the audit, then execute protected workflows when the operator environments are available. -
Added protected PostgreSQL recovery-target evidence. The manual
postgres-recovery-provider-evidence.ymlworkflow connects through the existing SELECT-only observer, requires recovery mode and transaction read-only state, and can pin server major/replay LSN without logging the DSN. It proves the recovered target posture only; cloud PITR/WAL/object restore, KMS, replication and RTO/RPO remain separate controls. Next: merge this slice, then complete the roadmap audit. -
Added protected mTLS and runner-token rotation evidence. The manual
mtls-runner-rotation-evidence.ymlworkflow writes CA/client certificate material only to a mode-700 temporary directory, verifies an HTTPS mTLS health endpoint, proves the old runner token is rejected and the new token is accepted, disables redirects and cleans up on every path. It never logs credentials or response bodies. This is deployment-boundary evidence only; IdP issuance, CA governance, revocation propagation and HA failover remain separate. Next: merge this slice, then perform the final roadmap audit and close any remaining implementable provider journey. -
Added protected read-only Stripe provider evidence. The manual
stripe-provider-evidence.ymlworkflow requires an operator-ownedsk_test_secret and PaymentIntent expectations in the protectedcommerce-provider-evidenceEnvironment, then exercises the real Stripe REST adapter and fails closed on currency, amount, received amount or status drift without logging credentials or response bodies. This proves provider authentication and typed retrieval only; checkout mutation, webhook, settlement, payout, refund and dispute evidence remain separate. Next: continue the final provider-evidence audit (KMS/WORM/PITR, mTLS/rotation and the remaining ecommerce provider journeys). -
Added protected full OIDC provider evidence. The manual
oidc-provider-evidence.ymlworkflow requires an operator-issued, single-use authorization code, PKCE verifier/nonce and client secret in theproduction-identity-evidenceEnvironment. It exercises discovery, exact endpoint policy, PKCE URL construction, token exchange, RS256/JWKS validation, UserInfo subject binding, role and optional MFA claims without logging credentials or tokens. -
Kept identity evidence explicit. The workflow proves only the configured authorization-code path at execution time; SCIM, mTLS, session persistence, rotation, failover and availability remain separate. Next: validate/merge this slice, then perform the final roadmap requirement audit.
-
Added protected live PostgreSQL provider evidence. The manual
postgres-provider-evidence.ymlworkflow requires a disposableproduction-database-evidenceEnvironment secret, refuses missing DSNs, and executes the syntheticpg_dump/isolatedpg_restore/digest journey without logging connection strings. It proves configured-provider restore compatibility only; cloud PITR/WAL, KMS, replication and RTO/RPO remain explicit DR evidence. -
Documented the operational safety boundary. The runbook and ADR-260 require an isolated target with temporary-database privileges and explicitly prohibit customer production DSNs. Next: validate and merge this slice, then continue IdP/mTLS execution evidence and final roadmap audit.
-
Added protected live artifact-provider evidence. The manual
artifact-provider-evidence.ymlworkflow requires aproduction-artifact-evidenceEnvironment, validates endpoint/region/ credentials and KMS-mode completeness before running, and exercises Object Lock plus optional exact KMS read-back for both artifact objects. Local MinIO remains explicitly separate from cloud/provider evidence. -
Made the S3 live journey configurable for real providers. Retention duration/mode and
AES256/KMS/DSSE encryption can now be supplied by the operator without committing credentials or weakening the default CI fixture. Next: run local artifact gates and merge this slice, then continue PITR, KMS rotation/IAM and IdP execution evidence. -
Added protected live gift-card provider evidence. The manual
gift-card-provider-evidence.ymlworkflow requires an operator-ownedcommerce-provider-evidenceEnvironment, fails closed on missing required settings, injects authorization only from an Environment secret, and runs a read-only typed endpoint/tenant/card/balance/status/expiry reconciliation. A successful run will be external provider evidence; it is not claimed until that Environment is configured and the job passes. -
Closed a provider error-leak path.
HttpGiftCardProviderno longer includes non-2xx response bodies in thrown errors, preventing provider diagnostics or tenant data from becoming logs or persisted artifacts. Next: run local gates, merge this sub-task, then continue provider-backed KMS/WORM/PITR and IdP execution evidence. -
Added the provider gift-card lifecycle boundary.
@aqa/commercenow exposesverifyGiftCardProviderJourney()and a schema-validated provider snapshot. It reconciles tenant/card identity, currency and exact balance against the merchant ledger and rejects impossible active/expired timestamp combinations. PR #195 merged ase852b27after the full technical matrix; local evidence was 827 passed, 1 skipped, 0 failed. This is adapter contract evidence, not a live issuer/provider execution claim. The remaining external KMS/WORM/PITR, IdP and real commerce-provider evidence stays explicit. -
Added the bounded HTTP gift-card provider adapter.
HttpGiftCardProvidernow connects the typed provider snapshot to a real HTTPS boundary with origin allowlisting, manual redirect handling, response limits, URL-credential rejection and explicit loopback-only HTTP for local tests. PR #196 merged as1f55196after both CI workflow runs passed the complete technical matrix; local evidence was 831 passed, 1 skipped, 0 failed. A transient hosted NuGet timeout was rerun successfully. Provider IAM, token rotation and live issuer execution remain deployment evidence. -
Hardened stored-value idempotency scope. Gift-card operation IDs are now unique within tenant + gift-card ownership rather than globally, in Memory and PostgreSQL lookup/index paths. Added a regression proving equal keys can be used independently by two tenants/cards; the durable migration promotes the entry identity to a scoped unique index. PR #193 merged as
eba363dafter the full technical matrix: 822 passed, 1 skipped, 0 failed locally, plus both CI workflow runs green for typecheck/lint, build, Bun/Node, PostgreSQL/S3/OCI, CLI smoke, Playwright admin UI, Helm, Cloudflare and live Prometheus/OTLP telemetry. Copilot remained the intentionally excluded failing check. The skip is the existing live S3 test withoutAQA_TEST_S3_ENDPOINT; no production credentials were introduced. -
Merged the atomic gift-card ledger boundary. PR #191 merged as
59d4e4aafter the full technical matrix, including both Playwright admin journeys, CLI smoke, Postgres/S3/OCI integrations and live telemetry.@aqa/commercenow provides deterministic Memory and PostgreSQL implementations for tenant-scoped credit, idempotent redeem and balance observation. Conflicting operation reuse, currency mixing and concurrent overspend fail closed; provider gift-card settlement/expiry remains separate evidence. Targeted suite: 4 passed, 0 failed, with the live PostgreSQL concurrency contract explicitly skipped withoutAQA_TEST_POSTGRES_DSN. ADR-255 records the boundary. Provider gift-card settlement/expiry and the missing production-evidence environment remain external next steps. -
Merged CI fail-closed hardening. PR #190 merged as
283d100. Unit, Node, build and Playwright jobs now execute their real commands unconditionally; a missing workspace or admin Playwright setup fails the job instead of printing a green skip notice. This closes a false-green governance gap in the enterprise release gate. -
Merged OIDC origin-policy hardening. PR #188 merged as
a466cfaafter the full technical matrix: Bun/Node tests, typecheck/lint, build, Helm, CLI smoke, Playwright admin UI, Postgres/S3/OCI integrations and live Prometheus/OTLP telemetry all passed. Copilot remained the intentionally excluded failing check. The remaining production-evidence gap is external: the GitHubproduction-evidenceEnvironment is missing/inaccessible, so live IdP issuance/rotation cannot be claimed. -
Exposed the enterprise OIDC endpoint-origin policy. The runtime now accepts
AQA_OIDC_ALLOWED_ENDPOINT_ORIGINSfor providers whose discovery, token, UserInfo or JWKS endpoints use additional HTTPS origins; the default remains issuer-origin-only. Helm exposes the same policy throughauth.oidc.allowedEndpointOriginsand validates the projected Secret env name/key at render time. Parser regressions and package gates pass; full repository gates and CI are the next evidence step. -
Production doctor now enforces the admin identity boundary. It reuses the same fail-closed OIDC environment parser as
aqa admin, reports whether sessions are shared, and fails when the CLI deployment would fall back to the local identity or has partial OIDC settings. Added complete/partial secret redaction regressions; the check still does not claim live IdP issuance or rotation. PR #187 merged asd28249cafter the technical CI matrix. -
Closed the OIDC deployment composition gap.
aqa adminnow resolves explicitAQA_OIDC_*settings, constructs the provider-neutral adapter and usesAQA_OIDC_SESSION_DSNfor the shared PostgreSQL PKCE/session store; partial settings or a missing secret fail closed and never fall back to the local identity. Helm now exposes Secret-backed OIDC settings with render-time validation and requires a PostgreSQL source for HA sessions. Evidence: environment parser regressions plus a non-loopback OIDC-configured admin boot test; local full gates and Helm CI are required before merge. ADR-254 records the boundary. -
Hardened ecommerce quote binding. Tax and shipping schemas now require
cart_id; the journey rejects cross-cart tax/shipping observations and shipping destinations that differ from the requested address. Added ADR-243 and regression coverage. Commerce suite: 54 passed, 0 failed, with the PostgreSQL-dependent contract skipped locally without DSN. This closes a false-green gap in provider-backed commerce evidence; tax-law correctness, final checkout price and real provider execution remain separate evidence. -
Added quote-to-checkout tax application evidence. When checkout is observable,
verifyTaxJourney()now performs a namespaced idempotent checkout and requires the final order tax to equal the provider quote; quote-only merchants remain explicitly bounded. Added ADR-244 and a regression for a tax quote silently dropped by checkout. Commerce suite: 55 passed, 0 failed locally; provider-specific tax-law and live merchant evidence remain separate. -
Bound fulfillment evidence to merchant identity. Fulfillment snapshots now require tenant and customer IDs, and post-purchase integrity rejects cross-scope delivery records even when
order_idmatches. Added ADR-245 and a regression test. Commerce suite: 55 passed, 0 failed locally; carrier, warehouse and physical-delivery evidence remain external boundaries. -
Added a provider-backed PostgreSQL backup/restore journey. The hosted PostgreSQL job now creates a unique synthetic canary, runs
pg_dump, restores withpg_restoreinto an isolated database, and compares the canonical data digest through a fresh connection, with bounded cleanup and no DSN/payload logging. Local execution is intentionally unavailable without a live PostgreSQL client/service; the hosted job is authoritative. This closes restore-integrity evidence for the CI provider but does not claim managed cloud PITR/WAL, KMS/WORM, replication or production RTO/RPO. -
Hardened commerce HTTP transport.
HttpCommerceAdapternow rejects external plaintext HTTP and embedded URL credentials, while allowing loopback HTTP only with explicitallowInsecureLocalHttp: true. Added ADR-247 and regressions for insecure external origins, unsafe allowlists and local journey compatibility. Commerce contract: 39 passed, 0 failed locally with the existing PostgreSQL-dependent skip. Merchant TLS, certificate rotation and network egress remain deployment evidence. -
Pinned OIDC discovery endpoints.
OidcAdapternow requires an HTTPS issuer, rejects URL credentials and validates authorization, token, UserInfo and JWKS origins against the issuer by default or an explicit HTTPSallowed_endpoint_originslist. Added ADR-248 and SSRF/misrouting regressions. Auth suite: 20 passed, 0 failed locally; live IdP certificate, DNS, egress and lifecycle evidence remains deployment-scoped. -
Made restore-drill timing operation-derived.
@aqa/compliancenow exposesmeasureRestoreDrill(), which records timestamps around the actual restore callback, computes bounded RTO and propagates provider failures. Added ADR-249 and regression coverage. Compliance suite: 26 passed, 0 failed locally. This improves evidence provenance but does not itself prove cloud PITR, KMS/WORM or artifact-provider execution. -
Reconciled governance with the live roadmap. Updated
AGENTS.mdanddocs/RULES.mdso the current priority is production-evidence hardening, and codified the complete-journey rule: fixtures and validators cannot be reported as provider execution evidence. This closes the stale bootstrap priority/documentation drift; cloud/provider exercises remain the next external-evidence action. -
OIDC signed-token boundary implemented on
task/oidc-jwks-rotation. The adapter now requires and validates RS256 ID tokens against discovered JWKS, checks issuer/audience/azp/iat/exp/nonce, binds UserInfosub, and refreshes keys once on provider rotation.OidcSessionManagergenerates a nonce with each one-time PKCE state; PostgreSQL migration rejects legacy pending rows that lack it. Local auth suite: 35 passed, 0 failed; hosted live IdP and PostgreSQL evidence remain open until CI/provider credentials are available. Next: run the full workspace gates, open the technical PR, then continue with provider-backed tax/shipping/fulfillment and recovery evidence.
-
Added a protected production-evidence CI handoff. Manual workflow
production-evidence-gate.ymlreads the signed envelope, inventory, restore-drill, trust root and key ID only from the protected GitHub Environment, runsaqa dr release-gate, and cleans a mode-700 temporary directory on every exit path. Added ADR-250 and operator documentation. It makes provenance/binding repeatable; provider execution evidence remains an explicit deployment obligation. -
Added fail-closed S3 server-side encryption verification.
S3ArtifactStorecan now request AES256, KMS or KMS DSSE encryption, pin an exact KMS key identity and verify the provider'sHeadObjectresponse for both the artifact and metadata sidecar. Added mismatch/validation tests, package docs and ADR-234. Artifact suite: 8 passed, 0 failed; package build and typecheck passed. This is an enforceable provider boundary, not proof of AWS IAM, key rotation or cross-region replication. Next: wire the production profile/deployment contract and continue PITR/identity evidence. -
Implemented signing-identity pinning for production evidence. Compliance verifiers now optionally require an exact
signature.key_id; the release gate requires--public-key-id, andaqa doctor --productionrequiresAQA_PRODUCTION_EVIDENCE_KEY_IDwhenever signed production evidence is configured. The restore-binding path applies the same pin to signed backup inventories. Added ADR-233, operational README updates and mismatch tests. Local evidence: 782 tests, 781 passed, 1 S3 platform skip, full workspace typecheck passed, Biome lint passed, build passed, CLI E2E passed andgit diff --checkpassed. Next: commit/push this slice, then continue with provider-backed KMS/WORM/PITR and identity execution evidence. -
Promoted production doctor binding policy to
main. Macro PR #158 merged asabf2466after CI run 35339493234 passed every technical gate, including Playwright, CLI E2E, live Prometheus/OTLP, PostgreSQL, S3, OCI, Bun/Node, build, lint/typecheck and Helm. The evidence chain is now executable from both the release CLI and production doctor. Remaining work is provider-backed execution evidence: KMS/WORM, PITR/restore and IdP/ identity exercises. -
Merged doctor restore-binding sub-task PR #157 into the macro branch. Hosted CI run 35338960631 passed typecheck/lint, Helm, Bun, Node 22, PostgreSQL, S3, OCI, build, CLI E2E, Playwright admin E2E and live Prometheus/OTLP telemetry. Next: promote the policy increment to
main, then continue real provider-backed KMS/WORM/PITR and identity exercises. -
Connected restore-drill binding to
aqa doctor --production. WhenAQA_PRODUCTION_DR_INVENTORY_PATHandAQA_PRODUCTION_DR_EVIDENCE_PATHare present, doctor now revalidates the inputs and reuses the canonical signed cross-document verifier; partial, unreadable or mismatched inputs fail, while absent paths remain an explicit warning. Kit suite: 172 passed, 0 failed, 2 platform skips; package typecheck and Biome pass. This closes a release-policy blind spot but still does not prove provider execution. Next: continue provider-backed KMS/WORM/PITR and identity proof. -
Promoted the production-evidence release gate to
main. Macro PR #156 merged as141de9aafter rerun 35337503239 passed every technical gate, including 144 Playwright tests, CLI E2E, live Prometheus/OTLP, PostgreSQL, S3, OCI, Bun/Node, build, lint/typecheck and Helm. Copilot was intentionally not used. The next increment is to enforce this binding from production doctor/release policy and then close provider-backed KMS/WORM, PITR and identity evidence gaps. -
Merged the DR release-gate sub-task PR #155 into the macro branch. The authoritative hosted CI run 35336917827 passed all technical gates: typecheck/lint, Helm, Bun, Node 22, PostgreSQL, S3, OCI, build, CLI E2E, Playwright admin E2E and live Prometheus/OTLP telemetry. A duplicate CI run hung in Playwright and was cancelled after the complete duplicate run had already passed; no code failure was observed. Next: promote the macro branch to
main, then continue provider-backed KMS/WORM/PITR and identity work. -
Added the executable DR release gate to the operator CLI.
aqa dr release-gate <inventory> <restore-evidence> <production-evidence> --public-key <pem>now reuses the canonical compliance verifier to validate the inventory, restore objectives, trusted production signature, drill reference and SHA-256 binding in one fail-closed command. Kit suite: 169 passed, 0 failed, 2 platform skips; kit typecheck, Biome and package build passed. This proves the evidence chain at the CLI boundary, not provider execution. Next: run full workspace gates, then continue provider-backed KMS/WORM/PITR and identity evidence. -
Merged PR #154 (
762e8ee) with the hosted production-evidence binding gate. CI run 35335845194 completed successfully across typecheck/lint, build, Bun and Node 22 tests, PostgreSQL, OCI, S3-compatible storage, CLI E2E, live Prometheus/OTLP telemetry, Playwright admin E2E and Helm validation. Copilot review was intentionally not used per the task instruction. The next roadmap gap is provider/deployment execution evidence, not another local contract-only claim. -
Added the complete production-evidence/restore-drill join verifier.
verifyProductionEvidenceRestoreBindingnow verifies the trusted signature, backup inventory, restore-drill contract, drill reference and recomputed SHA-256 digest as one fail-closed operation. Compliance suite: 24 passed, 0 failed; ADR-230 and lessons document the cross-document invariant. This proves evidence linkage, not the underlying provider execution. -
Made restore-drill digest production executable. Exported
canonicalRestoreDrillEvidenceandrestoreDrillEvidenceSha256from@aqa/compliance, both built on the same validated canonical record used by the evidence contract. Compliance suite: 23 passed, 0 failed; ADR-230, lesson and operator contract updated. This removes producer-side canonicalization drift; provider-side DR execution is still separate. -
Bound production recovery evidence to the exact restore drill.
database_recoverynow requires a validated lowercase SHA-256 digest of the canonical restore-drill record in addition to the operator reference ID; changing the referenced drill therefore invalidates the signed production envelope. Compliance suite: 22 passed, 0 failed; ADR-230 and the production-evidence example document the new contract. This improves provenance but still does not claim that the underlying PITR/object restore actually ran; that remains deployment evidence. -
Hardened recovery evidence against unsigned metadata. Backup inventories, restore-drill records, signed production-evidence envelopes and signature objects now reject unsupported fields at every nesting level before canonicalization or verification. This closes an audit ambiguity where ignored properties could appear alongside a valid signed record without being covered by its signature. Compliance suite: 21 passed, 0 failed; ADR-229 records the fail-closed schema-evolution rule. This remains a contract hardening change: real PostgreSQL PITR, object-store restore, KMS and WORM provider evidence are still deployment obligations.
-
Closed the webhook crash-recovery gap at the ledger boundary. Processing claims now carry a bounded lease; an abandoned in-memory or PostgreSQL claim can be atomically reclaimed, while completed effects remain duplicate-safe. The contract explicitly retains the requirement that business effects are idempotent across a crash/reclaim race. Commerce suite: 45 passed, 0 failed locally; PR #150 merged after hosted CI run 35332684961 passed PostgreSQL, Node 22, live observability and all E2E gates.
-
Closed the Stripe webhook side-effect safety gap. Added
StripeWebhookProcessorfor signature verification, body-size bounds, supported-event allowlisting and order/provider linkage before invoking a business effect. Hardened both webhook ledgers with processing/completed state and release-on-failure semantics, fixing the lost-retry case where an effect could fail after its claim was stored. Commerce suite: 44 passed, 0 failed; PR #149 merged after hosted CI run 35331906279 passed the PostgreSQL migration, full test matrix and E2E gates. -
Added a real Stripe payment boundary.
StripePaymentGatewaynow creates and retrieves PaymentIntents and creates refunds through Stripe's REST API, with strict secret-key/HTTPS validation, bounded response parsing, minor-unit money checks, mandatory write idempotency keys and typed provider observations. Commerce suite: 41 passed, 0 failed; no credentials are used in tests. PR #148 merged after hosted CI run 35330815340 passed the full technical matrix. This closes the repository adapter gap, but a live Stripe test-mode journey, signed webhook ingress, settlement/payout reconciliation and provider fault drills remain deployment evidence. -
Added the provider-neutral observability deployment bundle. Versioned an OpenTelemetry Collector pipeline, Prometheus recording/alert rules and a Grafana operations dashboard under
integrations/observability/. The bundle is credential-free, uses bounded processors and the metric names actually emitted by the registry; 20 observability tests passed, including asset structure and secret-safety checks. This closes the repository asset gap, but not live-cluster evidence: scrape, OTLP export, alert firing and dashboard population still require an operator environment. Added and closed a reproducible Docker journey: hosted CI run 35329751510 started disposable Prometheus and an OpenTelemetry Collector, scraped the real admin endpoint, queried the resulting series and verified OTLP span delivery; CLI, Playwright and all technical gates also passed. Local execution is explicitly skipped when Docker is unavailable. Remaining: provider-specific dashboards, alert routing and SRE ownership evidence in each target environment. -
Closed the complete remote worker production journey. PR #145 is merged on
mainas 95e5a18. The real Kit lifecycle now runs through the authenticated HTTP queue, lease renewal, token rotation, completion ACK and artifact publication; the lease watcher shutdown race found during hosted Node 22 execution is fixed. Hosted CI run 35327643173 passed typecheck, lint, Bun, Node 22, PostgreSQL, S3/MinIO, OCI, build/SBOM, CLI and Playwright gates. Remaining remote deployment evidence is AWS-specific KMS/replication, mTLS/revocation and operator/load-balancer behavior. Next: close the live observability deployment evidence slice. -
Added the complete remote worker run journey. The authenticated HTTP queue now drives the real
makeKitWorkerand canonicalrunRunpath against a local HTTP target, then provesevents.jsonl/findings.jsonlpublication, queue completion and a fresh JWT lookup after dequeue. Remote identity tests: 2 passed locally under Bun and Node 22 after hardening the lease watcher against in-flight remote queue probes during shutdown; hosted CI is required before marking the cross-process journey complete. -
Closed executable remote-worker token rotation.
aqa workernow selectsHttpRunnerQueuewheneverAQA_SERVER_URLis set, requires a token or token-file credential, and reads the token file for every queue request. Helm now requiresrunner.worker.tokenSecretRef, mounts the projected key, and fails closed during render when it is absent. Worker configuration tests: 4 passed; CI run 35326324592 passed Helm, Bun, Node 22, PostgreSQL, S3, OCI, build, CLI and Playwright gates. ADR-227 records the decision. Kubernetes Secret propagation timing, TLS/mTLS and issuer revocation remain operator/deployment evidence. -
Closed authenticated remote-runner identity evidence. Added
HttpRunnerQueue, which keeps enqueue/reaping/cancellation on the control plane and invokes a token source for every dequeue/get/renew/ACK/fail request. Added scope-checked HTTP get/renew routes and a realrunAdminHTTP journey proving RS256 issuer/audience validation, tenant isolation, lease renewal, ACK fencing and JWT rotation without restarting the runner. Local journey: 1 passed; CI run 35325413778 passed Bun, Node 22, PostgreSQL, S3, OCI, build, Helm, CLI E2E and Playwright gates. TLS/mTLS termination, revocation distribution and load-balancer behavior remain deployment evidence. Next: connect token-file rotation and server URL settings to the executableaqa workerdeployment path, then prove a full remote worker run lifecycle. -
Closed the real remote-artifact evidence slice. Added an ephemeral MinIO CI service and an endpoint-gated integration journey that creates an Object Lock-enabled bucket, writes a redacted tenant/run artifact through
S3ArtifactStore, verifies provider retention read-back on content and metadata, downloads with SHA-256 verification, and validates the metadata reference. ADR-225 records the boundary. CI run 35324437934 passed the real S3-compatible job plus Bun, Node 22, PostgreSQL, OCI, build, Helm, CLI and Playwright gates. Local package tests skip this provider journey whenAQA_TEST_S3_ENDPOINTis absent. AWS-specific KMS, replication, backup and restore evidence remains deployment-specific. Next: authenticate separate runner processes and prove identity rotation across the queue boundary. -
Closed the finding-transition audit gap.
MemoryStoreandPostgresStorenow route the legacyupdateFindingStatusAPI through the atomic transition path, preserving the operator reason in the hash-chainedfinding_status_changedevent. Added a regression test and corrected the admin/audit documentation that still claimed this was missing. Store suite: 16 passed, 0 failed. Next: audit cost-admission wiring and prove the remaining production journeys. -
Hardened the governed LLM boundary.
BudgetedLlmAdapternow exposes a bounded, prompt-free event sink for completed calls and budget denials, including authoritative usage and cost without leaking prompts or secrets. The adapter and cost suite pass; host wiring to the durable run event chain remains an explicit integration boundary rather than an implicit side effect. -
Wired audit metrics through the real run boundary.
aqa runnow accepts an injected boundedMetricsRegistry; persisted audit events map to low-cardinality run, scenario, finding, LLM usage and budget-denial counters, while OTLP tracing remains composable. The complete fixture run provesrun_started/run_finishedreach metrics state: 40 run tests and 16 observability tests passed. Live Prometheus scrape and collector delivery remain deployment evidence. -
Closed the exact-budget finalization edge case. When authoritative provider usage reaches the limit after admission,
BudgetedLlmAdapternow emits an explicitbudget_exceededevent before raising, so a run that stops immediately cannot lose the terminal cost signal. Adapter suite: 6 passed, 0 failed. -
M6 enterprise pack batch is complete and merged through PR #137. The desktop pack was the final originally missing M6 baseline; PRs #121–#137 passed the repository integrity, lint, build and unit gates. The pack contracts are now available on
main. Provider, cluster, OS and regulated production evidence remains intentionally operator/deployment-specific and is not conflated with pack schema validation. -
Closed the LLM budget-to-audit integration gap.
@aqa/runnernow exportsmakeBudgetEventSink, a structural bridge from the prompt-free events emitted byBudgetedLlmAdapterinto the run's hash-chainedllm_callandbudget_exceededrecords. The bridge bounds provider/model/reason fields and preserves numeric usage counters. A redaction regression was also fixed: canonical numeric token counters are retained for cost evidence while token strings remain redacted. Runner: 71 passed; observability: 17 passed. Host wiring still must explicitly provide the sink, and provider billing reconciliation remains deployment evidence. -
Hardened the budget audit boundary against malformed host input. The runner bridge now caps counters/costs, drops invalid numeric values and runs reason text through the central redactor before appending to the chain. Added regression coverage for overflow, negative/NaN/Infinity values and bearer leakage. Runner suite: 72 passed. This protects the audit artifact; provider billing reconciliation and distributed usage truth remain separate evidence.
-
Added the durable queue-to-kit complete journey. The PostgreSQL CI job now runs
packages/kit/test/run-cmd.test.tswith its real PostgreSQL service. The journey enqueues a unique job inPostgresRunnerQueue, executes it throughmakeKitWorkerand the real HTTPrunRunpath, verifies the durable job isdone, and checks both run artifacts exist. ADR-224 records the exact evidence boundary: this closes the durable queue/orchestrator gap, while deployed multi-process identity, remote artifact publication and provider effects remain separate production evidence.
The individual
Started pack-*bullets below are historical work-start notes retained for traceability. They are not current “next” items; the consolidated completion status above is authoritative.
-
Started
pack-desktop. It closes the M6 pack gap for Electron/Tauri with schema-valid IPC sandbox, signed auto-update and authenticated protocol contracts. Platform signing and OS evidence remain separate; next: validate and publish the pack. -
Started
pack-chaos. It adds opt-in, schema-valid contracts for bounded fault experiments, RTO/RPO recovery evidence and tenant blast-radius containment. Fault injection, provider recovery and production impact remain separate; next: validate and publish the pack. -
Started
pack-compliance-hipaa. It adds opt-in, schema-valid contracts for minimum-necessary PHI, purpose-bound access and tamper-evident incident evidence. Legal, administrative, physical and vendor controls remain separate; next: validate and publish the pack. -
Started
pack-mobile-native. It adds opt-in, schema-valid contracts for offline mutation idempotency, minimal permissions and authenticated deep-link session binding. Device, OS, store and push-provider evidence remain separate; next: validate and publish the pack. -
Started
pack-i18n-l10n. It adds opt-in, schema-valid contracts for locale coverage, commerce formatting and safe translation fallback/RTL behavior. Human linguistic, visual, tax and legal evidence remain separate; next: validate and publish the pack. -
Started
pack-realtime. It adds opt-in, schema-valid contracts for connection lifecycle, bounded backpressure and cursor-based replay ordering. Broker, proxy, device and production-capacity evidence remain operator supplied; next: validate and publish the pack. -
Started
pack-data-pipeline. It adds opt-in, schema-valid contracts for producer/consumer compatibility, data-quality promotion gates and bounded idempotent replay. Orchestrator, warehouse, lineage and provider recovery evidence remain operator supplied; next: validate and publish the pack. -
Started
pack-kubernetes. It adds opt-in, schema-valid contracts for workload hardening, admission policy decisions and rollout/disruption resilience. Cluster, image, network and production runtime evidence remain operator supplied; next: validate and publish the pack. -
Started
pack-performance. It adds opt-in, schema-valid contracts for p95/p99 latency, error-budget release gating and saturation visibility on critical journeys. Load-runner, cost and production-capacity evidence remain operator supplied; next: validate and publish the pack. -
Started
pack-infra-iac. It adds opt-in, schema-valid contracts for destructive plan visibility, drift ownership/correlation and policy-gated release decisions. Provider-backed apply, IAM, state-lock and recovery evidence remain separate; next: validate and publish the pack. -
Started
pack-accessibility-wcag. It adds opt-in, schema-valid contracts for keyboard operability, semantic names/roles/states and live-region announcements, with an explicit boundary against claiming WCAG certification. Next: validate and publish the pack. -
Started the next regulated e-commerce pack:
pack-compliance-pci. It defines opt-in, schema-valid contracts for payment-data redaction, tokenization boundaries and deny-by-default segmentation. It explicitly does not claim PCI-DSS certification or production payment evidence; next: run pack integrity and full gates. -
Started the next M6 enterprise pack:
pack-database-migrations. It adds opt-in, schema-valid contracts for expand/contract compatibility, rolling deploys and rollback/backfill reconciliation. Provider-specific migration runner, backup/PITR and production-database evidence remain intentionally outside this baseline; next: validate the pack through the loader and gates. -
Added the opt-in
pack-compliance-gdprenterprise pack (M6). It ships schema-valid DSAR access, consent withdrawal and erasure scenarios with privacy risks, bounded HTTP oracles, a junior-friendly README and loader regression coverage. It is deliberately not auto-enabled and does not claim legal compliance or provider/downstream deletion proof. Next: continue the remaining enterprise packs and live provider/deployment evidence. -
Added the M7 fixture management core.
aqa fixtures snapshot <dir>now creates bounded, versioned JSON fixtures under.aqa/fixtures/, with deterministic anonymization for common PII/credential fields.aqa fixtures restore <fixture> <dir>validates manifest paths, size limits and SHA-256 integrity before a no-overwrite restore. Absolute/traversal targets are rejected and only JSON is accepted by this safe baseline. Added CLI tests for anonymization, tampering and overwrite protection. Provider-specific staging extraction, distribution-preserving anonymization and ephemeral tenant provisioning remain deployment integrations. -
Closed the durable finding fix-verification loop (M5 core). Findings now retain a bounded
last_verificationrecord and supportfixed→regressedtransitions only when deterministic replay evidence matches the original failure fingerprint. Added tenant-scopedPOST /api/findings/:id/verification, atomic Memory/PostgreSQL persistence, hash-chained verification events, generated JSON Schema updates and complete API/store regression coverage. This records evidence but does not claim that the API itself executed a replay; CI/PR automation and scheduled retests remain open integrations. Next: continue M6/M7 and live provider evidence. -
Added freshness enforcement for signed production evidence. The compliance package now evaluates the bounded
captured_attimestamp against an operator-selectedAQA_PRODUCTION_EVIDENCE_MAX_AGE_HOURSbudget.aqa doctor --productionwarns for missing, stale or future-dated policy evidence and only passes when signature, completeness and freshness all hold. This is a temporal/provenance check, not live provider proof. Added unit coverage and updated ADR-206 plus the operator runbook. Next: continue the remaining runtime/provider-backed roadmap gaps. -
Added the signed production evidence boundary.
@aqa/compliancenow validates and signs bounded observations for KMS/Vault rotation, artifact-versioning/Object Lock, PostgreSQL PITR/WAL and IdP/OIDC/mTLS/ runner rotation.aqa doctor --productionoptionally verifies the envelope through an explicit public-key trust root and distinguishes missing, incomplete and invalid evidence. This makes the operational handoff automatable without claiming that a signed document alone proves live provider behavior. Added ADR-206 and 17 compliance tests. -
Added a live OCI complete journey for hardened kit runs. The container integration job now resolves an immutable Alpine digest and executes the real
release-gatepath with automatic sandbox selection and a shell probe, in addition to the lower-level sandbox contract. The local suite skips this journey when Docker/runtime credentials are unavailable; no local skip is counted as production evidence. -
Added bounded Playwright trace ingestion.
aqa ingest playwright <trace.zip>now reads only thetrace.traceJSONL member, rejects encrypted or traversal ZIP entries, bounds decompression, and persists action-level metadata without URLs, headers, payloads or screenshots. Added ADR-204 and parser tests for compressed traces and unsafe archives. This proves the ingestion boundary; it does not claim browser replay or provider runtime evidence. -
Propagated sandbox image readiness into operations.
aqa doctor --productionnow checks for a full immutableAQA_CONTAINER_IMAGEdigest without exposing its value. The Helm runner values/template expose the operator-owned sandbox image and fail rendering when an enabled worker has pinning required but no image configured. This validates configuration only; OCI runtime/socket, registry admission and live image provenance remain deployment evidence. Next: continue KMS/WORM, PITR/restore and IdP/mTLS. -
Required immutable images for auto-hardened sandbox profiles. The automatically selected OCI sandbox for
security/release-gatenow rejects mutable tags and requiresAQA_CONTAINER_IMAGE(or an injected equivalent) with a full SHA-256 digest. Invalid configuration returns a bounded run error before dispatch and never prints the digest. Evidence: sandbox pinning tests plus complete kit journey; live OCI execution remains hosted deployment evidence. Next: continue KMS/WORM, PITR/restore and IdP/mTLS. -
Connected hardened shell probes to the sandbox boundary. Orchestrator
securityandrelease-gateruns now create aContainerSandboxwhen the host has not supplied a driver; shell probes execute through the bounded OCI tool boundary, while HTTP remains origin-scoped and smoke profiles remain explicit/no-sandbox. Evidence: realaqa runshell journey with an injected sandbox executor passes, kit suite 38 passed / 0 failed, plus existing hosted OCI runtime coverage. This is not a claim of VM isolation or live Docker proof for every deployment. Next: enforce pinned production image and continue KMS/WORM, PITR/restore and IdP/mTLS evidence. -
Added an explicit scenario isolation policy. Profiles now support
parallel,groupedandserialscheduler modes; scenarios can declare anisolation_group, and grouped scenarios are serialized while independent groups retain bounded parallelism. Start events record the effective policy and key. This prevents a scheduler-level shared-state race but is explicitly not container/VM isolation or a tenant reset. Evidence: realaqa runcomplete journey proves two scenarios in one group never overlap; schema and kit tests 79 passed / 0 failed. Next: continue the remaining provider and deployment evidence (sandbox wiring, KMS/WORM, PITR/restore, IdP/mTLS). -
Fixed prefixed S3 Object Lock verification. Retention verification now checks the exact provider key for both the artifact and its metadata object; a configured prefix is no longer applied twice to the metadata check. The S3 retention journey asserts both
HeadObjectkeys and passes 6/6 locally. Next: continue the remaining deployment-grade evidence (real KMS/WORM, PITR/restore, IdP/mTLS and provider-backed commerce journeys). -
Made profile parallelism operational.
aqa runnow discovers scenarios before scheduling them through a bounded worker pool capped by the profile'sparallelism, while preserving unique finding seeds, cooperative cancellation, budget checks, and discovery-ordered run summaries. Audit events may interleave by completion time and retaindispatch_orderso the concurrency trade-off is explicit. Evidence: complete kit parallelism journey, workspace typecheck/build/lint and 729-test suite pass. Next: continue production evidence and external-provider validation. -
Added executable scenario preconditions.
Scenario.preconditionsnow accepts a backwards-compatible structured{ id, probe, oracle }contract.runScenario()executes those checks before scenario steps, records their probe/oracle evidence, runs cleanup after a failed setup check, and returnsblockedwithout emitting a security finding. Existing descriptive string preconditions remain valid but are explicitly non-executable. Evidence: runner and schema tests pass, schema JSON regenerated, Biome clean. Next: close the remaining execution-policy gap around configured parallelism. -
Connected signed-pack policy to execution.
aqa run --require-signed-packsnow refuses every discovered pack unless its canonical manifest digest, trusted Ed25519 signature and full file content digest all verify before any scenario runs. The CLI accepts only an operator public-key trust root fromAQA_PACK_TRUSTED_KEYS_JSON; local authoring remains opt-in. Kit typecheck, lint and targeted complete run tests pass. Next: continue deployment-grade evidence and real provider drills. -
Anchored replay to the original defect identity. Findings now persist a SHA-256
failure_fingerprintof the failed-oracle set, andaqa verifypasses it into every retry. A different oracle failure can no longer be reported as deterministic merely because it also produced a finding. Schema, runner and complete CLI verification tests cover the contract. Next: continue the remaining operational proof (real KMS/WORM/PITR/IdP and external security validation). -
Closed the audit-serving false-green boundary.
GET /api/runs/:id/eventsnow verifies the complete stored hash chain before returning evidence and returns boundedAUDIT_CHAIN_INVALIDon tampering, matching the existing risk-coverage projection. Evidence: server integration regression plus full local gate suite. Next: operational evidence and real infrastructure drills. -
Added an explicit production-readiness doctor mode.
aqa doctor --productionnow checks only configuration prerequisites for durable store, queue, S3/Object Lock retention, runner authentication, audit checkpoints and OTLP, never prints values and never confuses configuration with live provider evidence. Missing prerequisites fail (or warn for checkpoint/observability) with actionable remediation. Next: execute this contract against a real deployment and retain provider evidence. -
Hardened enterprise pack supply-chain enforcement. The scanner now has an explicit
requireSignaturepolicy; the admin/server import boundary enables it by default, rejects unsigned manifests before persistence, and preserves an explicit opt-out only for local/dev callers. Scanner, API and admin tests cover the policy. Next: continue the operational evidence layer (real trust root rotation, KMS/WORM/PITR and external security validation). -
Closed the custom pack-resource gap.
aqa runnow loads and validates every manifest-listed declarative probe/oracle definition before execution, rejects missing/duplicate/escaping/symlinked resources, and expands explicitkind: custom+with.refreferences into built-in contracts. Resource files are already inside the content-digest boundary; no executable pack code is introduced. Evidence: workspace typecheck, Biome lint and seven pack-loader regression tests pass. Next: integrate the branch and continue the remaining production-evidence audit (real KMS/WORM/PITR/IdP and external security validation). -
Started the evidence-integrity macro task. The server coverage projection now verifies every run's hash-chained audit events before deriving risk coverage and returns a bounded
AUDIT_CHAIN_INVALIDerror on tampering. The first server contract is green locally; full workspace gates and hosted PostgreSQL/HTTP evidence remain required. -
Hardened the DR evidence CLI after automated review. Restore evidence now must reconcile
observed_rto_minuteswith the rawstarted_atandcompleted_attimestamps, bundled-entrypoint tests cover both DR commands and missing--public-keyvalues, and CLI failures use the shared redaction-safe error formatter. Targeted compliance/kit tests are green; full repository gates are the next proof before pushing the PR update. -
Closed the DR envelope ambiguity found in the second review pass. The inventory parser now reserves top-level
inventory/signaturekeys so an unsigned document cannot be confused with a signed envelope; compliance and CLI regression tests cover the boundary. The refreshed hosted run is still required before merge. -
Added DR evidence CLI gates.
aqa dr inventorynow validates and hashes a machine-readable backup inventory, verifies signed inventory envelopes only with an explicit Ed25519 public key, andaqa dr restorevalidates a restore drill against the inventory's backup identity, artifact manifest digest, RPO/RTO objectives and required security checks. The DR runbook now points to these gates. Evidence:@aqa/kitsuite 151 pass / 0 fail / 2 platform skips, repository typecheck/lint/build, installed CJSaqa drsmoke and workspace tests 711 pass / 0 fail pass locally. Hosted CI is still required before merge; real PostgreSQL PITR, object-store restore, KMS and WORM controls remain deployment evidence. -
Closed the PostgreSQL trajectory migration race in code. Hosted CI showed that two trajectory-store instances could concurrently create the same PostgreSQL relation, despite
IF NOT EXISTS, producing apg_typeduplicate error. Production clients now bootstrap inside one transaction protected bypg_advisory_xact_lock; injected clients withoutbegin()are rejected unless they explicitly opt into the unsafe test-only migration path. Local runner evidence is 67 pass / 0 fail, typecheck and Biome are green. A post-fix hosted PostgreSQL run is required before this race boundary is considered verified. -
Added live PostgreSQL trajectory evidence to CI. The runner package test script now includes the trajectory suite, and the PostgreSQL integration job runs a two-instance trajectory contract with retry, immutable-conflict and read-back assertions when
AQA_TEST_POSTGRES_DSNis provided. The hosted run force11adfis evidence for the live trajectory contract before the transaction-scoped migration fix; the current PR must be validated by a newer hosted run before the race fix is closed. Backup/PITR/restore and WORM operations remain deployment evidence. -
Added cross-replica PostgreSQL trajectory persistence.
PostgresAgentTrajectoryStoreuses an immutable(run_id, scenario_id)primary key, atomic conflict-safe insert, bounded envelopes and digest plus invariant verification on read. Evidence: runner suite 65 pass / 0 fail including idempotent retry, conflict, tamper, traversal and injected-SQL cases; live PostgreSQL backup/PITR/restore and WORM evidence remain open. ADR-196 records the storage boundary. -
Added authenticated MCP HTTP transport.
McpHttpTransportnow mounts the bounded JSON request/response MCP surface through the Fetch API: every request is authenticated, sessions are principal-bound and capped, bodies are byte-limited, protocol versions are checked, idle sessions expire andDELETEterminates a session. Evidence: MCP tests 8 pass / 0 fail and server suite 140 pass / 0 fail, including auth binding, body limits, expiry and termination. SSE push, sticky/shared sessions and deployed TLS/IdP evidence remain open. ADR-195 records the transport boundary. -
Added durable immutable trajectory artifacts.
AgentTrajectoryStorevalidates before write, bounds UTF-8 size, writes through a temporary file and rename, refuses replacement under the same run/scenario identity, and rechecks digest plus trajectory invariants on read. Evidence: runner trajectory suite 9 tests pass / 0 fail (including tamper, traversal and size cases); remote WORM/Object-Lock, backup/restore and S3 deployment evidence remain open. ADR-194 records the storage boundary. -
Added semantic agent evaluation and calibration primitives.
evaluateAgentTrials()now requires bounded multi-judge/multi-trial evidence, distinct models when configured, and returnsinconclusivebelow the agreement floor.calibrateAgentJudges()reports Brier score and ECE over a gold corpus without storing rationale text. Evidence: runner suite 57 pass / 0 fail after the new contract tests; live provider quality, prompt pinning and human gold-set governance remain open. ADR-193 records the boundary. -
Bound MCP to the real queue/store.
createMcpRunPort(ctx)now uses the authoritativeRunnerQueueandStoreProvider: plan is read-only, start validates the tenant profile and uses scoped idempotency, status/cancel are tenant-fenced, and evidence returns bounded store-derived metadata. A real MemoryStore + RunnerQueue MCP journey passes alongside the 136-test server suite; PostgreSQL and authenticated streamable-HTTP deployment evidence remain required. -
Fixed the Node 22 CI compatibility regression in the MCP slice. The first hosted matrix caught a TypeScript parameter-property unsupported by Node's strip-only loader; the server class now uses explicit fields. This is a real runtime-compatibility fix, not a CI suppression. The PR will be re-pushed and the full Bun/Node matrix rerun.
-
Added the bounded MCP control surface.
AqaMcpServernegotiates an explicitly supported protocol version and exposes only tenant-scoped plan, start, status, cancel and metadata-only evidence tools. Permission checks, principal-derived scope, bounded inputs and idempotent starts are enforced before the host port is called. Evidence: server MCP suite 4 pass / 0 fail, package build/typecheck/Biome green. Streamable HTTP authentication, real queue/provider binding and deployed MCP journey remain required. ADR-192 records the transport-neutral boundary. -
Closed the trajectory verification loop.
verifyAgentTrajectory()now checks identity, contiguous sequence, SHA-256 digests, token reconciliation and (when supplied) one-to-one correspondence with agent hash-chain events. Evidence: trajectory verifier suite 4 pass / 0 fail including tamper and event-mismatch cases. ADR-191 records the privacy-preserving replay boundary; signatures, durable remote attestation and semantic grading remain separate. -
Added pinned opaque agent trajectory evidence.
AgentTrajectoryRecorderenforces contiguous step and token budgets, pins provider/model identity, and emitsllm_call/tool_callaudit events with SHA-256 digests and usage only. Evidence: trajectory + tool-guard tests 7 pass / 0 fail; raw prompt/tool data is absent from snapshots and events. ADR-190 records the boundary; provider semantic evaluation, durable storage and MCP protocol evidence remain open integrations. -
Added the provider-neutral agent tool guard.
AgentToolGuardenforces exact tool allowlists, call budgets, output byte limits and cooperative cancellation, while emittingtool_callaudit events containing only digests and bounded metadata. Evidence: runner guard + regression suite 33 pass / 0 fail with no raw secret in events. ADR-189 records the boundary; MCP/provider egress and model semantics remain deployment-scoped. -
Opened the agent execution boundary.
aqa runnow acceptsexecution_mode: agentonly with an explicit host-ownedagentRunner; the run fails closed before allocation when it is missing. Agent runs propagate agent actors to probe/oracle/finding evidence and mark findings with agent provenance. Evidence: runner + kit boundary suite 62 pass / 0 fail, workspace typecheck/lint green. ADR-188 records that provider-backed trajectories, model pinning and MCP/tool policy remain separate integrations. -
Added pre-execution scenario contract validation.
Scenarionow rejects unsupported HTTP probe fields, malformed named auth references, non-string headers, non-numeric HTTP status expectations, and incomplete JSONPath response comparators, including cleanup probes. Evidence: schemas typecheck and targeted validator/AJV suite 69 pass / 0 fail. ADR-187 records the boundary; full repository gates and hosted CI are still required before merge. -
Connected authenticated HTTP probes to the real
aqa runboundary.RunOptions.httpSecretsand the explicitAQA_HTTP_SECRET_<NAME>host mapping now feed named auth references without exposing values to packs, events or findings. A complete local journey (aqa run→ HTTP server → Authorization header → audit artifacts) passes; kit run-cmd contracts are 32 pass / 0 fail. ADR-186 records secret-manager/CI ownership and keeps provider-specific token rotation as separate live evidence. -
Closed two runner false-green paths. HTTP probes now reject unknown fields and resolve
auth: "${NAME}"only from host-injected secrets; missing secrets are execution errors, never anonymous requests. Theresponse_containsoracle now compares bounded JSONPath values with literal or prior-probe references such as@probe-1.body.id, instead of silently ignoringjsonpath/equals. Runner regression suite is 37 pass / 0 fail including missing-secret, secret-injection and cross-step equality cases. ADR-185 records the boundary; provider-specific token rotation remains a live integration journey. -
Added the executable dispute/chargeback journey.
verifyDisputeJourney()now requires provider-observed non-empty chargeback evidence, validates exact order/payment linkage, tenant/customer ownership, duplicate IDs, amount/currency bounds and opened-dispute evidence deadlines. Added the HTTP observer and explicit reference seeding path. Commerce contracts are 36 pass / 0 fail with explicit PostgreSQL skips. ADR-184 records that inducing a network dispute and proving representment remains live provider sandbox evidence. -
Added the executable subscription dunning journey. Versioned dunning attempts now expose contiguous retry history, amount/currency linkage and scheduling metadata.
verifyDunningJourney()requires a provider-observed failed renewal, validates subscription ownership and minimum attempts, and the HTTP/reference adapters implement the observation boundary. Commerce contracts are 34 pass / 0 fail with explicit PostgreSQL skips. ADR-183 records that inducing a failure and proving provider collection still needs a billing sandbox/staging integration. -
Added the executable ecommerce loyalty journey.
verifyLoyaltyJourney()now performs checkout, requires a checkout-linked earn event, validates tenant/customer ownership and reconciles the complete observed ledger. Added the HTTP observer path, reference merchant evidence and aggregate-suite support. Commerce contracts are 32 pass / 0 fail with one explicit PostgreSQL skip. ADR-182 documents the provider-neutral boundary; real loyalty vendors, expiration/promotions and durable atomicity remain live integration evidence. -
Added the shared LLM transport boundary. Every live provider adapter now fails closed on non-HTTPS endpoints, endpoint credentials/query/fragment data, literal private/local/metadata destinations and optional host allow-list violations. Local/private models require explicit
allowPrivateNetwork: true; added 20 adapter tests total and ADR-181. Evidence: adapter build/typecheck, 20 passed / 0 failed, Biome and diff checks. DNS-aware egress enforcement remains deployment evidence. -
Added executable subscription coverage.
verifySubscriptionJourney()now checks idempotent create retry, plan/amount/interval, period ordering, tenant/customer ownership and authoritative read-after-write observation. Commerce contracts remain 30 pass / 0 fail; billing provider, dunning, invoices, proration and tax integrations remain live evidence. -
Added executable post-purchase/RMA coverage.
verifyPostPurchaseJourney()now requires fulfillment observation and validates tracking/order quantity invariants; it can also create and retry a typed return request with strict order/currency bounds. Commerce contracts remain 30 pass / 0 fail; carrier/WMS/refund-on-return integrations remain external evidence. -
Added executable settlement reconciliation. Commerce adapters can expose
settlement_observer;verifySettlementJourney()now binds checkout payment identity to parsed refunds/chargebacks and applies the net-total invariant. The aggregate suite can include the check; the reference contract remains 30 pass / 0 fail locally. Acquirer/provider payout and live accounting evidence remain external. -
Added the paid-order cancellation journey. Commerce adapters can now expose an optional cancellation mutation;
verifyCancellationJourney()checks customer-scoped checkout, idempotent retry, accepted decision and compensating refund linkage. The aggregate suite can include it without treating unsupported providers as green. Commerce contracts remain 30 pass / 0 fail locally; provider void/refund, fulfillment release and durable external idempotency remain live evidence. -
Added the provider-neutral commerce journey suite gate.
verifyCommerceJourneySuite()now executes required checkout/refund flows and optional tax/shipping/webhook flows, prefixes their evidence, and keepsunsupporteddistinct frompass. The reference contract now proves the aggregate gate with 30 commerce tests passed / 0 failed. Real payment, tax, shipping, fulfillment and dispute providers remain external journey evidence. -
Closed the coverage evidence integrity gap.
aqa risk coveragenow verifies each persistedevents.jsonlhash chain before acceptingscenario_finishedobservations. Tampered but schema-valid events fail closed with no coverage report; the regression suite is 145 pass / 0 fail / 2 platform skips for the kit. Independent WORM/checkpoint publication and provider semantics remain separate evidence layers. -
Closed the local installable-artifact evidence gap. Added a complete
npm packjourney that prepares the publish manifest, rejects leaked@aqa/*dependencies, extracts the tarball in isolation, and runs the installed CLI through--version,--help,initandvalidate. CI now runs it after the production build. Evidence: local journey passed on Windows; hosted registry publication, fresh-machine network installation and real provider journeys remain separate gates. -
Connected risk coverage to the operator CLI. Added
aqa risk coverageto load RiskMap/pack scenarios and schema-validatedscenario_finishedrun events, then report covered/partial/gap/stale status. Invalid evidence fails with exit 1; valid incomplete coverage fails with exit 2, so zero findings cannot create a false green. Event-chain verification and real provider/ cluster semantics remain separate gates. -
Aligned air-gap image tags with the Helm release. The bundle script now derives server/runner image tags from
deploy/helm/Chart.yamlappVersionand fails closed when that value is absent, removing the previous0.6.0versus chart1.1.0drift. Shell syntax and Helm render CI remain the repository evidence; registry publication and a real offline cluster remain operational evidence. -
Aligned finding confidence with declared oracle weights. Runner findings now use a weighted mean of oracle agreement instead of an unweighted average; severity remains derived from the resolved risk. Evidence: runner build, typecheck, targeted tests including the 1:3 weighting regression, Biome and diff-check pass. Statistical calibration and project-specific weight quality remain separate evidence.
-
Fixed single-target agent bootstrap consistency. Claude, Gemini and Copilot adapters now emit the canonical
AGENTS.mdthey already reference, matching Codex behavior. Adapter tests cover every target's bootstrap and interoperableSKILL.mdlayout. Host-version discovery and execution remain separate operational evidence. -
Added bounded source reachability to risk discovery. The source method now filters JS/TS signals through a non-executing relative-import graph, preserves dependency manifests, and emits explicit
reachability:bounded-import-graphevidence. An unreachable-marker regression test prevents dead source files from becoming risks. Aliases, dynamic imports, generated code and non-JS/TS graphs remain intentionally outside this heuristic's proof boundary. -
Added human-gated AI risk hypotheses.
@aqa/generatornow exposesproposeRisksand a separateRiskReviewQueue: model candidates are DLP-redacted, schema-validated, provenance-hashed and kept pending until a named reviewer approves them; no activeRiskMapis mutated by generation. Evidence: generator tests cover valid/invalid output, redaction, provenance, approval and anonymous-review rejection. Remote identity, LLM quality and production persistence remain open evidence layers. -
Added bounded source-aware risk discovery.
aqa risk discover --method sourcescans up to 200 safe, bounded source/manifest files and emits only deterministic risks whose authentication, interpreter, outbound request or secret signals are observed. Each result carries source-aware and bounded evidence-path tags; no source leaves the project. Evidence: risk-discovery tests 6 passed / 0 failed, kit build/bundle/typecheck/ Biome/diff-check pass. This is an explainable heuristic baseline, not an autonomous LLM/security review. -
Source-aware risk discovery merged to
main. PR #70 was squash-merged as833d949after all 10 hosted CI jobs passed, including PostgreSQL 16, Node 22, CLI smoke and Playwright admin UI. The deterministic heuristic is now released; AST/dependency reachability, LLM-assisted hypotheses and human/security review remain separate evidence layers. -
Added W3C trace propagation across HTTP probe boundaries.
makeHttpProbeRunnernow accepts an explicit trusted trace context and propagatestraceparentto the target, overriding any scenario-supplied value. Invalid/untrusted scenario metadata cannot forge the run trace. Evidence: runner tests 25 passed / 0 failed, build/typecheck/Biome/ diff-check pass. Collector deployment and cross-process trace export remain operational evidence. -
W3C trace propagation merged to
main. PR #69 was squash-merged asc5ba579after all 10 hosted CI jobs passed, including PostgreSQL 16, Node 22, CLI smoke and Playwright admin UI. Trusted trace correlation now crosses the HTTP probe boundary; Collector/reverse-proxy preservation remains deployment evidence. -
Added restore-drill evidence validation.
@aqa/compliancenow checks a drill against the signed backup inventory: source and restored manifest identity, monotonic timestamps, observed RPO/RTO against approved objectives, tenant isolation, audit-chain, queue-fencing and secret-redaction controls. Drift, objective violations and incomplete controls fail closed. Evidence: compliance tests 15 passed / 0 failed, build/typecheck/Biome/diff-check pass. This validates submitted evidence; it does not execute a cluster restore. -
Restore-drill evidence merged to
main. PR #68 was squash-merged asd7d1298after all 10 hosted CI jobs passed, including PostgreSQL 16, Node 22, CLI smoke and Playwright admin UI. The evidence validator is now released; actual PITR/object restore and measured production RTO/RPO remain operational evidence. -
Hardened AI scenario generation provenance and fallback validity. Review queue items now retain provider/model/version plus SHA-256 hashes for the invariant, prompt and raw model response, while raw prompt/response text is not persisted. Also fixed a real generator bug where minimal valid JSON drafts were discarded because fallback
id/titlefields were absent. Evidence: generator tests 6 passed / 0 failed, build/typecheck/Biome/ diff-check pass. Human approval remains mandatory before activation. -
AI generation provenance merged to
main. PR #67 was squash-merged as0df8663after all 10 hosted CI jobs passed, including PostgreSQL 16, Node 22, CLI smoke and Playwright admin UI. The generated-draft audit trail and fallback regression are now released; model trust and human review remain explicit governance boundaries. -
Added Ed25519 signatures for DR inventory evidence. Backup inventories can now be signed over their canonical representation and verified against an explicitly supplied trusted public key; missing trust roots and tampered payloads fail closed. Evidence: compliance tests 13 passed / 0 failed, build/typecheck/Biome/diff-check pass. Key custody/rotation remains an external KMS/Vault responsibility.
-
DR inventory signatures merged to
main. PR #66 was squash-merged as4b38898after all 10 hosted CI jobs passed, including PostgreSQL 16, Node 22, CLI smoke and Playwright admin UI. The code now verifies signed recovery evidence; key custody, rotation and live restore remain external operational evidence. -
Added a machine-validatable DR backup inventory contract.
@aqa/compliancenow parses and canonicalizes a redacted recovery manifest containing PITR target/LSN, schema version, artifact manifest digest/object count, image digest, operator drill ID and RPO/RTO objectives. Invalid identifiers, digests, timestamps and zero objectives fail closed. Evidence: compliance tests 12 passed / 0 failed, build/typecheck/Biome/diff-check pass. This does not claim a live backup or restore drill. -
DR inventory contract merged to
main. PR #65 was squash-merged as0c42570after all 10 hosted CI jobs passed, including PostgreSQL 16, Node 22, CLI smoke and Playwright admin UI. The manifest format is now part of the released code; WAL/KMS/Object Lock/restore execution remains infrastructure evidence. -
Added provider-neutral commerce settlement reconciliation. The commerce contract now validates that captured payment, successful refunds and lost chargebacks are linked to the same payment and exactly reconstruct the provider-reported net settlement. Unresolved chargebacks and non-successful refunds fail closed. Evidence: commerce contracts 29 passed / 0 failed, build/typecheck/Biome/diff-check pass. Live PSP settlement remains external deployment evidence.
-
Settlement reconciliation merged to
main. PR #64 was squash-merged as5b801d4after all 10 hosted CI jobs passed, including PostgreSQL 16, Node 22, CLI smoke and Playwright admin UI. The implementation boundary is closed; live PSP/payout/fee/dispute reconciliation remains deployment evidence. -
Commerce refund integrity hardened. Payment validation now parses the runtime snapshot, requires
partially_refundedto carry a strictly positive partial amount, requires all refund states to reconcile with the captured amount, and rejects a successful refund larger than the observed cumulative refunded amount. Regression coverage: commerce contracts 28 passed / 0 failed; PostgreSQL provider coverage remains an external-DSN/CI journey. Next: run the complete hosted matrix on the PR. -
Commerce refund integrity merged to
main. PR #63 was squash-merged as2c87ca5after the complete hosted matrix passed: 10/10 jobs green, including PostgreSQL 16, Node 22, OCI sandbox, CLI smoke and Playwright admin UI. This closes the implementation slice; live payment settlement and provider reconciliation remain deployment evidence. -
Budget kill-switch API merged to
main. PR #62 was squash-merged asb81cdbcafter the complete GitHub CI matrix passed: 10/10 jobs green, including Node 22, PostgreSQL 16, OCI sandbox, CLI smoke and Playwright admin UI. Copilot review was intentionally waived per user instruction. The next evidence gap is a controlled deployed-operator exercise against a real durable budget store. -
Exposed the durable budget kill-switch through the admin API. Added tenant-derived
GET/POST /api/cost/haltroutes, an admin-onlycost:editpermission, fail-closed503behavior when the durable controller is absent, andAQA_BUDGET_DSNwiring throughaqa adminand Helm. The request can never choose another tenant's budget key. ADR-159 records the boundary. -
Evidence: server suite 132 passed / 0 failed, auth/server/kit typechecks pass, and Helm CI asserts the production DSN mapping. A deployed operator incident exercise remains operational evidence.
-
Promotion redemption slice merged to
main. PR #61 was squash-merged as4feefb7after the full CI matrix passed, including the PostgreSQL 16 two-client redemption race and Playwright/CLI acceptance jobs. This closes the implementation slice; real merchant promotion settlement and financial reconciliation remain provider/deployment evidence. -
Added atomic ecommerce promotion redemption.
PromotionRedemptionLedgernow modelsclaimed,duplicate,conflictandexhaustedoutcomes. The PostgreSQL implementation serializes each promotion code, enforces the hard redemption cap across replicas and preserves idempotent retry semantics; ADR-158 documents the provider-neutral boundary. -
Evidence: commerce typecheck and 29 tests passed locally; a two-client PostgreSQL race is wired into the CI persistence contract. Provider settlement and financial reconciliation remain deployment evidence.
-
Enterprise safety tranche merged to
main. PR #60 was squash-merged as4cdc3afafter the complete hosted CI matrix passed: Meta, Helm rendering, typecheck/lint, Bun and Node 22 tests, build plus CycloneDX SBOM, PostgreSQL 16 persistence contracts, OCI sandbox, CLI smoke and Playwright admin E2E. The PostgreSQL job included concurrent migration, durable idempotency response decoding, queue priority and the cross-client budget kill-switch contract. -
Verification boundary: this proves the repository's automated acceptance matrix, not a live merchant/payment provider, a real IdP ceremony, a restore drill or a production cluster upgrade. Those remain deployment evidence and are deliberately not marked complete.
-
Added a durable distributed LLM kill-switch. The optional
BudgetHaltControllerexposes bounded per-keyhalt()/getHaltReason()operations without breaking third-partyBudgetLedgerimplementations. PostgreSQL persists the stop inaqa_llm_budget_haltsand rejects later reservations across worker processes; the memory implementation remains an explicit local fallback. ADR-157 records the irreversible operator boundary. -
Evidence: cost typecheck and 14 tests passed. A live PostgreSQL multi-client halt/admission journey is now wired into the PostgreSQL CI contract; the hosted run is required before claiming HA production evidence.
-
Fixed PostgreSQL idempotency response decoding. The live multi-replica contract exposed that PostgreSQL's JSONB driver result can arrive as a JSON string, so a coalesced retry returned serialized body/headers instead of the same
ApiResponseshape as the original request. The durable adapter now decodes JSONB values before returning cached responses. -
Evidence: server typecheck and suite 131 passed / 0 failed locally; CI run
35229505062had all other gates green and isolated the PostgreSQL failure to this contract. A new hosted run is required for closure. -
Fixed a real PostgreSQL bootstrap race. Concurrent
PostgresEventBusreplicas could both passCREATE TABLE IF NOT EXISTSbut collide while PostgreSQL created the implicit identity sequence, failing CI withduplicate key ... aqa_live_events_sequence_seq. The schema and index DDL now run under a stable transaction-scoped advisory lock (ADR-147). -
Evidence: repository typecheck passed; the failing CI job was isolated to the concurrent PostgreSQL EventBus contract, while the other PostgreSQL contracts passed. The corrected branch still needs a fresh CI run to prove the fix in PostgreSQL 16.
-
Closed an HTTP dispatcher regression. The bundled admin server now parses
PATCHrequest bodies and advertisesPATCH,Idempotency-KeyandIf-Matchin CORS preflight responses. This restores the real HTTP path for SCIM patch and conditional/idempotent browser mutations; the route handlers alone had not exposed the adapter bug. -
Bounded webhook response reads before allocation.
HttpWebhookTransportnow consumes response streams incrementally and cancels as soon as the byte cap is exceeded, covering responses that omitContent-Length; a regression test proves the rejection path without contacting a provider. -
Added a concrete secret-manager integration.
VaultSecretResolvernow resolves Vault KV-v2 values lazily with an injected token provider, HTTPS endpoint enforcement, bounded timeout, safe secret-reference segments and generic provider errors (ADR-148). The queue persists onlysecret_ref; no real credentials or external calls are used in tests. -
Corrected a stale architecture claim.
docs/architecture/reference.mdno longer presents the historical v1.x task list as proof of GA or production readiness; it now points to the active v2.0 evidence matrix and distinguishes implemented, integrated and production-verified capabilities. -
Unified API mutation idempotency. Every non-GET route now accepts a validated
Idempotency-Keythrough one route boundary. The key is scoped to tenant/route, bound to params/body/conditional version, concurrent retries share the in-flight response, and mismatched reuse returns409without invoking the handler again.MemoryApiIdempotencyStoreis explicitly a single-process fallback; production multi-replica deployments must inject a shared durable implementation. The contract is covered on an organization mutation, not only on the existing runner queue path (ADR-149).PostgresApiIdempotencyStorenow provides the shared multi-replica claim and response store; the PostgreSQL CI contract exercises two clients competing for one mutation key. The local memory store remains an explicit fallback. -
Added connection-aware webhook egress.
NodePinnedHttpsWebhookTransportresolves DNS once, rejects every private/local/multicast answer before connecting, pins the selected IP in the TLS socket while retaining hostname SNI/Host, disables redirect behavior by construction and bounds response bytes. A regression test proves mixed public/private DNS answers are denied; non-Node runtimes must use an equivalent egress proxy policy. -
Closed a cancellation false-green path.
runScenario()now stops remaining steps when its signal is aborted, always executes declared cleanup, and returnsblockedwith failed execution instead of passing an empty-oracle scenario. A regression proves the remaining step is skipped and cleanup is still attempted. -
Persisted a canonical run terminal state.
run_finished.payload.run_statenow records the hash-protected terminal state at finalization. The shared schema derivation prefers this explicit state for new events and retains the counter-based fallback for legacy runs, so report/admin/other consumers share one authoritative state instead of independently guessing it. A CLI negative journey asserts a missing driver persistsrun_state=failed(ADR-151). -
Bounded cooperative probe execution. Every probe now receives a derived abort signal with its declared
timeout_ms; the runner converts a timeout or caller cancellation into an execution failure even if a cooperative driver returns a nominal response after abort. Runner tests now cover the timeout contract (47/47). Hard-killing an uncooperative third-party process still belongs to the OCI/container isolation boundary and is not claimed here. -
Fixed the second PostgreSQL EventBus CI defect. After the bootstrap race fix, the live replay query failed on PostgreSQL 16 when an optional project scope was absent because an untyped
NULLparameter could not be inferred. The nullable parameter now has an explicittextcast. A fresh CI run is required for final PostgreSQL evidence. -
Commerce assurance pack. Added the opt-in
pack-commerce-core, selected only for ecommerce/commerce/shop/storefront project tags. It defines five release-gate journeys (checkout idempotency, inventory oversell, tax/shipping money reconciliation, refund idempotency and webhook replay) plus five risks and invariants; it is provider-neutral and treats missing authoritative observations as incomplete rather than passing. -
Evidence: all five scenarios and the risk map parse through the real Zod schemas; bundled-pack integrity test 13/13 and kit bundle now contains 6 packs. ADR-130 records the opt-in boundary. Real merchant/provider journeys remain deployment evidence.
-
Replay artifact truthfulness. Playwright replay files now generate executable code for the runner's structured
url/actionscontract, and SQL replay files now contain a read-only transaction withPREPARE/EXECUTEand safely rendered parameters. External script-only browser probes are explicitly marked as requiring the original spec instead of being presented as a runnable reproduction. Reporter suite 9/9 passes. -
Evidence: reporter typecheck, replay tests and Biome pass. Provider-specific browser fixtures and database credentials remain external deployment inputs; ADR-131 records the artifact boundary.
-
Conditional admin edits. Profile, risk and scenario detail reads now emit strong content ETags; their PUT boundaries honor
If-Matchand reject stale writes with412 PRECONDITION_FAILEDbefore persistence. Profile, risk and scenario editors now capture the tag on open and send it on PUT; successful saves refresh the tag. A server regression proves the newer value survives a stale client update, and a Chromium journey proves the risk editor sends the captured tag. -
Evidence: server suite 123/123, admin typecheck/build, targeted Risk Edit browser suite 5/5, server/kit typechecks and Biome pass; ADR-129 records the backward-compatible migration boundary. A visible conflict-resolution flow now reloads the authoritative resource for profile, risk and scenario editors; targeted Risk Edit browser evidence is 6/6.
-
Admin live Runs/Findings integration. Runs and Findings now consume the tenant-scoped API and show an explicit
live APImarker; fixture fallback is retained only for an unavailable backend. The ecosystem fixture now declares its referenced risk/invariant, fixing a real scenario-parse failure. -
Evidence: rebuilt kit/admin and ran the complete ecosystem journey through a real SUT, CLI run, API server and Chromium: 2/2 Playwright tests passed (live audit chain plus live Runs/Findings and detail navigation). The run fixture now carries its tenant org and live finding normalization tolerates valid sparse evidence without crashing the UI. Restart/Postgres/tenant switching journeys remain open. ADR-128 records the boundary.
-
Documentation supply-chain audit remediated. The independent
docs-sitenpm lockfile had five high and one moderate advisory (Hugging Face/ONNX,sharp,adm-zip,protobufjsandlinkify-it) that the root Bun audit could not see. Upgraded the docs toolchain, regenerated the lockfile, and ignored its generated search index. Evidence:npm ci --ignore-scripts,npm run check,npm run build, andnpm audit --jsonall pass; audit result is 0 vulnerabilities. ADR-127 records the separate-gate rule. -
The historical PR review still records the pre-remediation advisory state; hosted CI/default-branch evidence must be refreshed after merge. The root audit is currently clean as well.
-
Report audit-chain verification.
aqa reportnow verifies every parsed event with@aqa/compliance.verifyEventChainbefore reconstructing state or writing Markdown/JSON output. Tampered payloads fail closed; fixtures now use production-compatible canonical SHA-256 sealing and include a negative regression. ADR-126 records the local-integrity boundary. -
Evidence: focused report suite 138 passed / 2 platform skips after the kit build. Full workspace gates are next; independent checkpoint/WORM completeness and live deployment evidence remain open.
-
HTTP error disclosure hardening. Added shared
safeErrorMessage()in@aqa/observabilityand wired it into server and bundled-admin failure responses. DSNs, bearer tokens, JWTs, cloud keys, PAN/IBAN values, secret assignments and control characters are redacted or bounded before HTTP exposure; ADR-123 and the threat model record the residual DLP gap. -
Evidence: observability test 13/13 passed, server and kit typechecks passed, and Biome lint passed. Full workspace regression is still required after this increment; PostgreSQL/browser/provider live journeys remain explicit deployment evidence gaps.
-
Centralized evidence DLP and PAN false-positive fix. Moved text/JSON redaction into
@aqa/observabilityand wired artifacts, findings, audit events, SQL, shell and browser evidence to the same policy. PAN masking now requires a valid Luhn checksum, preventing dates and run IDs from being rewritten; ADR-124 records the binary/provider-specific residual gap. -
Evidence: targeted observability 13/13, artifacts 6/6, commerce 23/23, and kit 137 passed / 2 platform skips. A full workspace test, typecheck and build gate is the next verification step.
-
Configurable DLP coverage expanded. The shared policy now redacts IPv4, contextual high-entropy secret assignments and operator-supplied regex patterns in addition to AWS/JWT/email/Luhn-PAN/IBAN values. Regression coverage explicitly proves timestamps and run IDs remain unchanged.
-
Evidence: observability 14/14 and repository lint passed. Binary screenshots/PDFs, IPv6/provider-specific formats and enterprise secret classifiers remain deployment/provider-specific gaps.
-
Replay identity hardening.
verifyScenario()now anchors all replay successes to the persisted original failure fingerprint when one is supplied, instead of accepting any later finding. Added a negative test for a mismatched original fingerprint and ADR-125; unsupported/no-finding attempts remain non-reproductions. -
Evidence: runner 45/45 tests, runner typecheck and repository lint passed. Full workspace regression is required before the increment is complete.
-
Runner capability preflight added.
runScenario()can now receive the configured driver's supported probe kinds and fails before executing steps or cleanup when a browser/SQL/shell/LLM/custom capability is absent. The gap is recorded as failed execution evidence and cannot emit a finding; concrete drivers and live journeys remain open. -
The kit orchestration boundary now forwards
supportedProbeKindsto every scenario, so integrations cannot accidentally configure preflight on the runner and then lose it ataqa run. -
CLI audit publication wired. A dedicated
AQA_AUDIT_CHECKPOINT_S3_*configuration now creates a separate compliance-retained S3 store for the final checkpoint. Partial configuration and non-COMPLIANCEretention fail before execution; credentials remain on the AWS provider chain. -
Controlled shell driver added. The runner now has an explicit
makeShellProbeRunner()withshell:false, executable allowlist, argv-only inputs, bounded/redacted output and timeout termination. It is deliberately not a sandbox; OCI/non-privileged deployment and explicit capability wiring remain required before production use. -
Read-only SQL driver added.
makeSqlProbeRunner()now exposes an injected database observer for ecommerce invariants with parameter binding, lexical mutation/multi-statement rejection, row limits and redacted rows. It is not a DB permission boundary; a production adapter still needs a dedicated read-only role, timeout, tenant views and live provider evidence. -
Canonical scenario outcomes added. Runner results now distinguish
pass,fail,error,blockedand reservednot_run, separating SUT assertion failures from provider/driver gaps. CLI/report persistence still needs to expose the field end-to-end for complete outcome coverage. -
Outcome persistence wired into the audit chain. Kit runs now emit
scenario_started/scenario_finishedand includescenario_outcomesinrun_finished, preservingblockedversuserrorin canonical evidence. Admin/report rendering remains the next integration gap. -
Outcome reporting completed. Reporter JSON now exposes validated
scenario_outcomes, Markdown renders each scenario state, andaqa reportreads the values from the terminal audit event with fail-closed validation. Admin live rendering and persisted server projections remain separate UI/API integration work. -
Real Playwright driver added. Runner now supports a persistent browser context through structured, origin-scoped actions with bounded/redacted text evidence and explicit close lifecycle. The fake-browser contract test is green; installed Chromium, live checkout journey and SUT/provider reconciliation remain required evidence.
-
Driver lifecycle wired into kit orchestration.
aqa runnow invokes an optional probe-runnerclose()before publishing replay/audit artifacts and turns shutdown failures into run errors. The Playwright journey has a real close assertion; hard process cancellation remains worker/sandbox work. -
Real Postgres SQL adapter added. The runner now provides a concrete Postgres implementation with read-only transactions, local statement timeout, bounded rows and lifecycle close. CI PostgreSQL now runs its contract test; local no-DSN runs are explicitly skipped. Merchant schema/RLS/provider reconciliation remains open.
-
Live HTTP commerce adapter added.
@aqa/commercenow maps typed carts, checkout, payments, refunds, tax, shipping and webhook observations to a configurable allowlisted HTTP merchant API. Responses are bounded and Zod parsed; tenant context and idempotency keys are explicit. A real merchant sandbox journey and provider-specific auth/webhooks remain deployment work. -
Commerce transport journey verified. A local HTTP server backed by the reference merchant now runs the full checkout journey through
HttpCommerceAdapter, including inventory-before/after and idempotent retry. This proves the transport contract, not a third-party merchant sandbox. -
S3 WORM verification strengthened. Production retention mode now performs a
HeadObjectread-back after both content and metadata writes and fails closed when Object Lock is absent or shorter than requested. This closes the provider-silent-ignore false-green path; independent checkpoint publication, bucket/versioning/KMS and restore drills remain deployment evidence. -
Independent checkpoint publication added.
runRun()now supports a separately injected audit checkpoint store, publishes the final checkpoint undercheckpoints/<run_id>.json, and records its digest in the canonical manifest. The journey test uses two distinct file stores; production still needs a separately administered WORM/KMS/backup domain and restore drill. -
CI SBOM provenance gate added. The Build job now runs a pinned Anchore Syft action after the workspace build, emits and retains a CycloneDX JSON inventory, and validates that it is non-empty and structurally valid. The root
npm sbomcommand was tested and rejected because this Bun monorepo has no rootpackage-lock.jsonand npm sees workspace/link dependencies as incomplete; it was deliberately not promoted into CI. Hosted CI evidence is still required, and this does not yet prove image-level or published-pack provenance. -
Durable SCIM rotation made atomic.
ScimTokenStorenow has an optional atomic rotation contract; the PostgreSQL store serializes revoke-and-issue in one transaction and the manager uses it when available. Lightweight stores retain the documented fallback. Existing auth tests and typecheck remain the next local gate; two durable store instances must still prove the path in CI. -
CI regressions fixed from complete-journey evidence. The Bun 1.3.11 job was executing PostgreSQL contracts after
t.skip()instead of stopping; absent-DSN contracts now emit an explicit visible skip and return. The CLI smoke harness now runs the live-target child asynchronously so its parent HTTP server can service the real request; local evidence is 537 Bun tests passed and all 5 CLI smoke checks passed. The hosted rerun is required to close the previous CI failure. -
Audit completeness checkpoint contract added.
@aqa/compliancenow creates and verifies a checkpoint binding one run’s contiguous sequence, event count, head hash and canonical event digest; optional Ed25519 signing adds operator-controlled authenticity. Compliance tests: 10/10. This isaqa runnow emitscanonical/checkpoint.jsonand references it from the canonical manifest. The CLI can sign it through paired checkpoint-key environment variables and rejects partial configuration. It is not yet stored in a WORM/Object-Lock domain; that external retention boundary remains open. -
S3 WORM configuration can now fail closed.
aqa runrejectsAQA_ARTIFACT_S3_REQUIRE_RETENTION=trueunless an ISO retain-until timestamp andGOVERNANCE/COMPLIANCEmode are both configured. This prevents a production-shaped deployment from silently using mutable S3 artifacts; the bucket’s actual versioning/Object Lock/KMS state still requires infrastructure evidence. -
RFC 6238 TOTP verification boundary added.
@aqa/authnow exposes a bounded, fail-closed verifier with constant-time code comparison, Base32 decoding, configurable period/digits/skew window and RFC vector coverage. This deliberately does not claim enrollment, secret persistence, recovery codes or WebAuthn; those remain separate security work. Evidence: auth suite 18 passed, 3 PostgreSQL-dependent skips, 0 failures, lint and diff-check pass. The RFC vector caught and fixed an unsigned HMAC truncation issue and an incorrect test secret encoding. -
SCIM abuse limiter wired. Added a bounded tenant-scoped sliding-window limiter, applied before bearer authorization on every SCIM operation, with
429 scim_rate_limitedresponses and a default admin-boot policy. Evidence: auth 19 passed/3 PostgreSQL skips, server typecheck and repository lint pass. The limiter is intentionally process-local; a shared Redis/Postgres implementation is required before multi-replica deployment. -
Shared SCIM limiter added for HA.
PostgresScimRateLimiternow uses a serialized schema migration and a transaction-scoped advisory lock per tenant, making the fixed-window admission decision atomic across replicas.aqa adminacceptsscimRateLimitDsn/AQA_SCIM_RATE_LIMIT_DSN, and Helm plus CI render assertions expose the DSN. Evidence: auth 19 passed, 4 PostgreSQL-dependent skips, server build/typecheck, workspace typecheck, lint and diff-check pass. The hosted PostgreSQL job must execute the new cross-instance contract before claiming live HA evidence. -
OTLP runtime wiring shipped.
aqa runaccepts--otlp-endpoint(orAQA_OTLP_ENDPOINT), maps redacted audit-event metadata to bounded spans, and drains the exporter before returning. Collector failures become a visible warning while the hash-chained audit remains authoritative. Evidence: the real run journey sends spans to a local HTTP Collector fixture and verifies delivery before process return; observability 12/12, kit 126 passed/2 platform skips, typecheck and lint pass. -
Durable webhook effect ledger added. Commerce now exposes an explicit
applyWebhookEffectOnceboundary plus in-memory and PostgreSQL ledgers. A unique logical effect key is claimed atomically; duplicate deliveries are skipped and a different event for the same effect fails closed. Evidence: commerce 17 passed, 1 PostgreSQL skip, typecheck, lint and diff-check pass. Live provider settlement and reconciliation remain separate. -
Trusted pack signature boundary added. Pack manifests support an operator-managed
key_idplus detached Ed25519 signature over the canonical unsigned digest. The server import boundary enforces the allowlist when configured, while legacy SHA-256 remains an integrity-only path. Evidence: pack-scanner 9/9 and server 111 pass/1 PostgreSQL skip; schemas regenerated, typecheck and lint pass. This is not yet keyless Sigstore/cosign bundle or certificate-chain verification. -
Stripe-style webhook signature boundary added. Commerce now verifies the raw request body,
t=timestamp, one or morev1=HMAC signatures and a positive replay tolerance using constant-time comparison. Evidence: commerce 16/16, typecheck, lint and diff-check pass. This is a provider verification primitive, not evidence of live Stripe/payment settlement or durable event idempotency. -
Formal risk coverage measurement shipped.
@aqa/methodologynow computes the M2 weighted score (invariant mapping 35%, oracle-backed scenarios 25%, deterministic replay 20%, 30-day pass rate 10%, flake health 10%), validates impossible observations and emitscovered/partial/gap/staleplus drift alerts. This is a pure aggregation boundary ready for store/API wiring; it does not invent missing run evidence. -
Commerce webhook assurance expanded. The adapter now supports optional signed-delivery observations and
verifyWebhookJourney()validates exact order linkage, delivered state, signature validity and bounded retries. The reference merchant emits one deterministicorder.createdobservation per idempotent checkout; real provider webhook ingress, signature keys and durable reconciliation remain explicitly open. -
Commerce tax/shipping assurance expanded.
@aqa/commercenow defines versioned address, tax quote and shipping rate contracts plusverifyTaxJourney()/verifyShippingJourney(). The reference merchant exposes deterministic zero-tax/standard-shipping observations; validators reject currency mismatches, negative amounts and duplicate rates. Real tax, carrier and fulfillment integrations remain provider-specific and are not claimed by the reference implementation. -
LLM registry truthfulness corrected. The registry no longer contains a stale “live adapter not implemented” scaffold after native OpenAI-compatible, Anthropic, Google, Cohere and Bedrock adapters shipped; invalid runtime provider values now fail explicitly and the threat model reflects the real vLLM/Ollama boundary.
-
Server identifier hardening. API job/event IDs now use Node
crypto.randomUUID()instead of a hand-rolledMath.random()generator; this removes predictable identifiers from queue and notification boundaries. -
Run enqueue idempotency and tenant boundary shipped.
POST /api/runsnow requires org/project scope, embeds that scope in the job payload, bindsIdempotency-Keyto a canonical request fingerprint, returns the same job for identical retries and returns409 IDEMPOTENCY_CONFLICTfor changed retries. Memory and PostgreSQL queue contracts cover the behavior; ADR-030 documents retention as a remaining operational concern. -
EventBus integrated into API/admin lifecycle.
ApiContextcan now emit store-firstrun.requestedandfinding.status_changednotifications, andaqa admincan inject or constructPostgresEventBusfromAQA_EVENT_BUS_DSN, closing it during shutdown. Notification failures remain fail-open by design; API tests prove the request event is emitted. -
Helm production DSN wiring corrected. The chart now maps an external or in-cluster PostgreSQL URL to the actual store, queue, shared OIDC-session and EventBus environment variables. CI renders a secret-ref production shape and asserts all four DSNs are present; the previous
AQA_POSTGRES_URLalone did not activate the production adapters. -
PostgreSQL event bus shipped.
@aqa/servernow exposesEventBus,MemoryEventBusandPostgresEventBuswith bounded envelopes, validated channels, cross-replicaLISTEN/NOTIFY, idempotent unsubscribe and isolated subscriber errors. The live cross-client contract is wired into the PostgreSQL CI job; durability remains intentionally owned by the store and runner queue. ADR-029 records the transport boundary and reconnect gap. -
OIDC store retention hardening. The PostgreSQL session backend now indexes pending-state expiry and opportunistically removes expired PKCE/session rows on writes, preventing abandoned login attempts from producing unbounded table growth while preserving fail-closed expiry checks.
-
Post-OIDC acceptance gates green.
bun run lint, workspace typecheck, full Bun test suite (444 pass, 1 skip), workspace/docs build andbun audit --jsonall pass locally. The one skip is deliberate: the livePostgresOidcSessionStorecontract requiresAQA_TEST_POSTGRES_DSNand is executed by the PostgreSQL CI service job; no local database was available. Build warnings remain limited to the known CJSimport.metabundle warning and the admin chunk-size warning. -
Shared OIDC session backend shipped.
OidcSessionManagernow supports an async shared store, andPostgresOidcSessionStoreprovides advisory-locked tables, atomic one-time PKCE state consumption, durable session lookup and revocation. The admin HTTP path uses async authentication/logout and closes the backend on shutdown; two manager instances are covered by auth tests (9/9). The local map remains the explicit dev default. ADR-028 records the injection and production TLS/TTL responsibilities. -
Air-gap deployment path completed. The installer now has a real
installflow: archive path/link safety checks, SHA-256 verification, optional required Cosign blob verification, Docker/Podman image loading and explicit Helm upgrade/install options. Terraform can optionally own a pinned Helm release while remaining cloud-provider neutral. CI runsbash -n; a live air-gap cluster remains deployment evidence, not a local claim. ADR-027 records the contract. -
Provider-neutral observability foundation shipped. New
@aqa/observabilityprovides W3C traceparent parsing/formatting, injectable spans, bounded Prometheus counters/gauges/histograms, label validation, cardinality limits and redacted structured logs.EventChainWriterexposes a non-blocking observer hook whose failures cannot invalidate persisted audit evidence. Tests: observability 5/5, runner 29/29. ADR-026 records the OTel/Prometheus integration boundary and remaining operator responsibilities. -
Legacy configuration migration shipped. Memory/Postgres stores now expose an explicit, fail-closed migration primitive for global packs, profiles, risks and scenarios. The admin-only
POST /api/admin/migrate-legacy-configurationendpoint derives its target only from tenant scope headers, preflights conflicts and never overwrites a namespaced record. PostgreSQL performs the move in a serialized transaction; ADR-025 records the operator workflow. It intentionally does not migrate runs/findings or identity data. -
Canonical run evidence publication shipped.
aqa runnow publishes the byte-preservedevents.jsonlandfindings.jsonlthrough the configuredArtifactStore, plus a digest-bearingcanonical/manifest.json. It usesputBytesso redaction cannot mutate already-redacted hash-chain evidence; any publication error fails the structured run result. Local metadata and byte equality are covered by the kit journey test. ADR-024 records the non-transactional partial-upload recovery responsibility. -
Admin durable control-plane wiring shipped.
aqa adminnow accepts an injectedStoreProviderand selectsPostgresStorefromAQA_STORE_DSN, independently of the durable queue selected byAQA_QUEUE_DSN; both clients close during graceful shutdown and conflicting injection/DSN options fail before boot. The default remainsMemoryStorefor local development. ADR-021 and package docs record the deployment contract. PostgreSQL restart durability remains CI-authoritative when a real DSN is supplied. -
Tenant legacy fallback closed. Scoped Memory/Postgres reads and listings no longer fall back to globally keyed legacy packs, profiles, risks or scenarios; unscoped operations remain available for explicit migration/admin work. ADR-022 documents the fail-closed rule and a regression proves a global profile is invisible to a scoped tenant request. Migration/import tooling is still required to assign old records to a tenant.
-
Commerce Assurance adapter/journey slice shipped.
@aqa/commercenow defines a provider-neutral asyncCommerceAdapter, explicit capability preflight, andverifyCheckoutJourney()/verifyRefundJourney()with structured evidence. It validates money/order/payment/inventory snapshots, exact stock effects, bounded partial refunds and idempotent checkout/refund retries; missing capabilities returnunsupported, malformed or inconsistent provider observations returnerror. The deterministic reference merchant exposes the same adapter contract. ADR-020 records the boundary. Commerce tests: 12/12. Real payment, tax, shipping, webhook and durable-provider integrations remain open and are not claimed. -
Native Anthropic adapter shipped.
@aqa/llm-adaptersnow supports the Messages API with timeout, max-token bound, pre-request/response redaction, usage parsing, tool-schema forwarding and model provenance hash. Registry tests keep Google/Cohere/Bedrock on their native provider adapters; Anthropic suite: 10/10. -
Native Google/Cohere adapters shipped.
@aqa/llm-adaptersnow maps Google GeminigenerateContentand Cohere v2chatcontracts with provider-specific roles/content, tool schemas, bounded output, timeout, redacted transport/errors and usage parsing. Tests use injected transports and no credentials: provider adapter suite 2/2. ADR-023 records the boundary. Bedrock is now covered by its native SigV4 adapter; live AWS runtime evidence remains deployment-scoped. -
Native Bedrock adapter shipped.
@aqa/llm-adaptersnow supports Bedrock Runtime Converse with AWS SigV4 signing, session-token forwarding, native content/tool mapping, bounded output, timeout, redaction, usage parsing and fail-closed credential validation. The signature contract is tested with deterministic injected transport and no live AWS credentials; IAM/region/private endpoint operations remain deployment responsibilities. -
Dependency audit remediated.
bun audit --jsonis clean after upgrading Vite 8 + React plugin 6, esbuild 0.28, and pinning the vulnerablefast-urirange to 3.1.6 via the root override. Admin production build, kit typecheck/test/build and bundle execution remain green; the existing non-fatal CJSimport.metawarnings are tracked separately. -
Tenant-scoped configuration resources implemented. Packs, profiles, risks and scenarios now accept
org/projectscope throughStoreProvider; new records use deterministic namespaced keys in Memory/PostgreSQL, API handlers propagate request scope, and legacy unscoped records remain readable for migration compatibility. Memory/store isolation and an HTTP API regression cover same-name profiles in separate projects; server suite: 101/101. -
PostgreSQL migrations serialized. Store schema/table/index creation and version bookkeeping now run in one transaction-scoped advisory lock, closing the multi-replica first-boot race that
IF NOT EXISTSalone does not prevent. The live PostgreSQL CI contract remains the authoritative evidence. -
S3/MinIO artifact adapter shipped.
@aqa/artifactsnow exposesS3ArtifactStorewith content redaction, SHA-256 verification on reads, tenant prefix support, sidecar references and optional S3 Object Lock retention. An injected-client contract test proves the behavior without credentials; bucket versioning, KMS, tenant authorization and production Object Lock policy remain operator/API concerns. -
Run integration selects durable artifacts.
aqa runnow selectsS3ArtifactStorewhenAQA_ARTIFACT_S3_BUCKETis configured, scopes keys under the configured prefix plus run ID, validates retention settings, and keeps filesystem behavior as the local default. Kit backend-selection tests cover both paths without credentials. -
Helm render gate corrected. The production-shaped CI render exposed that the runner DNS egress rule still emitted an unrestricted
namespaceSelector: {}even though the ingress selector was constrained. DNS egress now targets the Kubernetes system namespace label explicitly, so the chart-level assertion matches the intended network boundary. -
Finding status audit made atomic. The API now uses a store-level transition primitive that updates the finding and appends its hash-chained audit event in one transaction. PostgreSQL serializes the complete read/hash/write sequence with a transaction advisory lock; MemoryStore mirrors the same contract. Added concurrency assertions for unique sequence/hash values and a memory regression. Local store: 9/9; server: 100/100. PostgreSQL concurrency branch remains CI-only without
AQA_TEST_POSTGRES_DSN. -
Queue retry budget and DLQ implemented. Memory and PostgreSQL queues now persist
attempts,max_attempts,failed, and capped failure reason; lease expiry after the budget is terminal, explicit worker failures usePOST /api/runner/jobs/:id/fail, and stale tokens remain fenced. Server suite: 100/100; PostgreSQL live migration behavior is queued for CI evidence. -
Migrated synchronous
aqa reportMarkdown/JSON writes toFileArtifactStoresync APIs; report files now receive redaction, atomic replacement and SHA-256 metadata sidecars. The report journey asserts both sidecars; direct-file writers remain only for canonical event/finding JSONL streams. -
CI runtime findings fixed, not suppressed. Run
35169917083exposed two real integration issues: concurrent queue clients raced onCREATE TABLE, and the first Docker image pull exceeded Bun's default 5-second test timeout. PostgreSQL migration now uses an advisory lock; the OCI test has an explicit 120-second budget. The failed run is retained as negative evidence; a new run is required. -
Full acceptance after infrastructure fixes. Run
35170349159is green across typecheck/lint, build, Bun + Node 22, PostgreSQL store plus durable queue, real Docker sandbox, CLI smoke, and all 142 Playwright admin tests. This is the authoritative evidence fore215871. -
Playwright CI failure diagnosed as runner provisioning, not an application regression. GitHub run
35167371462passed typecheck/lint, workspace build, Bun + Node 22 unit tests, CLI smoke, and the real PostgreSQL 16 contract. The admin job failed before test execution because Playwright 1.60 downloaded Chromium but not its separately requiredchromium-headless-shell-1223executable. CI, root, and admin install commands now provision both browser artifacts explicitly. Next: rerun the gate, then add fresh-process/reconnect and concurrency evidence to the Postgres contract. -
Second Playwright CI failure narrowed to CLI-version drift. Run
35168002405installed revision 1243 throughbunxbut the workspace test runner requested revision 1223, causing all 142 tests to fail at browser launch. CI and root scripts now invoke the package-local@playwright/testbinary throughbun run, so install and execution resolve one lockfile version. This needs a fresh CI run before any UI regression conclusion. -
Full CI acceptance restored. Run
35168485693is green across Meta, typecheck/lint, build, PostgreSQL 16 integration (including reconnect and concurrent create), Bun tests, Node 22 tests, CLI smoke, and all 142 Playwright admin tests. This is the first complete journey evidence for the current branch after the Playwright toolchain mismatch. Remaining roadmap work is functional, not a CI false-green. -
Durable user directory implemented.
StoreProvider.upsertUsernow persists the authenticated IdP snapshot in both Memory and PostgreSQL stores; the bundled admin HTTP boundary records the user, role/status, andlast_active_atafter successful RBAC authentication.GET /api/userstherefore survives a PostgreSQL process restart instead of returning an empty production directory. Targeted admin/store tests: 18 passing; the next CI run must re-prove the Postgres user round-trip. -
Postgres cost aggregation corrected.
costSummary()now carries run token counts, USD totals, per-profile aggregation, and daily series instead of returning zero-valued placeholders. The live store contract includes a non-zero cost run and asserts the aggregate; local execution still skips that branch withoutAQA_TEST_POSTGRES_DSN, so CI remains the authoritative evidence. -
Pack import supply-chain gate added.
POST /api/packs/importnow runs@aqa/pack-scanner, rejects critical/high findings (including unsigned shell packs) before persistence, and verifies a declared SHA-256 signature against the submitted YAML. Added a server regression proving an unsigned shell pack is not stored. Sigstore/cosign trust-root verification and mandatory policy configuration remain separate follow-ups. -
Real container sandbox backend shipped.
ContainerSandboxnow executes shell calls through Docker/Podman with read-only root, no network by default, tmpfs/tmp, CPU/memory/PID caps, dropped capabilities,no-new-privileges, non-root UID, timeout/error handling, and pinned-image configuration. The executor is injectable for unit tests; 9 sandbox tests pass. A Docker runtime integration test remains required because this workstation has no Docker engine. -
OCI sandbox integration gate added. CI now runs a separate Docker-backed sandbox contract with
AQA_TEST_CONTAINER_RUNTIME=docker, asserting the real container reports UID 65532, cannot write/, and can read its image filesystem. Local test count is 10/10, with the real branch explicitly skipped when no runtime is configured. -
Durable runner queue implemented. Added
PostgresRunnerQueuewith idempotent schema creation, row-locking/SKIP LOCKED, visibility leases, attempt counters, fencing tokens, reconnect persistence, and stale-worker ACK rejection.ApiContextnow consumes theRunnerQueueLikecontract and awaits both memory and durable implementations. The PostgreSQL integration contract covers two queue clients plus reconnect; production boot wiring still needs an explicit DSN/configuration choice. -
Durable queue boot wiring completed.
aqa adminaccepts an injected queue orqueueDsnand honorsAQA_QUEUE_DSN, selecting PostgreSQL explicitly while preserving the memory queue for local development. Shutdown closes the durable queue. Targeted kit admin tests: 10 passing; CI must prove the new queue contract against PostgreSQL and the real OCI sandbox. -
PostgreSQL contract strengthened. The live integration now closes the first store, opens a fresh store instance against the same DSN, verifies the prior run survives process boundaries, and races eight
createProfilecalls to prove the database uniqueness boundary returns exactly one creator. Local execution without a DSN remains intentionally non-evidence; CI must prove this against PostgreSQL 16. -
Deep technical review completed; product fixes NOT implemented. Local
mainsynchronized/verified againstorigin/mainat044135cdee0aaaf2e865ca10381ef5e4052ed22e. Preserved pre-existing.claude/scheduled_tasks.lock. Review-only work: no push, PR, publish or changes to product source. -
Readiness correction: historical roadmap closure below is not evidence of enterprise readiness. Detailed review records 24 prioritized findings, including reproduced false-green gates/oracles, browser audit accepting tampered events, finding ID collisions, invalid verified status, failed runs reported successful, anonymous admin writes, volatile mutations and failing CJS run/admin commands. Real compiled UI + real loopback API used for the audit negative; no intercepted requests.
-
Evidence: Windows/Bun 1.3.14/Node 25.2.1; workspace build and typecheck pass; root build only builds documentation; lint fails; unit suite 366 pass / 2 fail (Windows symlink setup and unsupported test skip); CLI E2E 5 pass but does not exercise published CJS entrypoint. GitHub CI run 27881325266 on reviewed main fails lint with downstream jobs skipped. Full Playwright, Node 22/Linux, deployed enterprise journeys and installed registry tarball remain unverified. Diagnostic reproducer prints observations, not a passing regression suite.
-
Strategy and ecommerce design delivered: current AI/QA comparison with primary sources in review; Commerce Assurance proposal covers missing commerce packs, money/order/inventory invariants, ten complete journeys, provider/merchant adapter contracts and safe testing boundaries. Proposal only; implementation requires confirmed pilot constraints and approval.
-
Next: agree remediation scope, then fail-closed execution/oracles, truthful outcome/audit, bundle/build gates, durable state and authorization/tenant boundaries before enterprise expansion. No production readiness or ecommerce coverage claimed. Existing AGENTS/rules/ADRs were not silently rewritten to adopt proposed architecture.
-
Fase 0 started on
task/v2.0-enterprise-truth-safety: runner no longer fabricates a200when no probe driver exists; text oracles fail closed on transport errors; all declared cleanup probes execute and emit evidence even after a main-step exception. Added regression tests and preserved informational smoke semantics while release-gate remains red on findings. Targeted runner tests: 12/12; targeted kit contract tests: 6/6; typecheck passed. Next increment: canonical execution outcomes and report/admin alignment. -
Fase 0 outcome alignment implemented:
Run.deriveStateFromCompletion()is now the shared fail-closed state rule consumed byaqa reportand admin seeding. Completion counters with any error or zero scenarios producefailed; missing completion remainsrunning. Added schema-level contract tests; schemas/report/admin targeted tests: 62/62. Next: repair the published CJS bundle'simport.metapaths and add an installed-artifact journey. -
Fase 0 bundle/state increment verified: ESM/CJS asset resolution now supports both
dist/commands/*and the bundleddist/cli.cjslayout. Added bundle entrypoint tests and confirmed the external diagnostic run reportsstate: failedfor a failed release gate. Kit build succeeds and bundle tests are 5/5; esbuild still emits non-fatalimport.metawarnings from the ESM fallback. The long-running bundled admin process is intentionally timeout-terminated by the diagnostic and must be covered by a future HTTP lifecycle test. -
Fase 0 test-harness cleanup: Windows/Bun symlink tests now log an explicit non-executable limitation instead of throwing from unsupported
node:testt.skip(). Full local suite is now 376 pass / 0 fail; typecheck passed. This does not claim symlink coverage on the current machine; Node/Linux or Developer Mode remains required for that negative path. -
PR gate cleanup: CI exposed 16 repository Biome errors, including seven in the new diagnostic and nine pre-existing docs/config guard scripts. Applied formatter/import/template-only fixes to the exact files; local
bun run lintnow passes across 261 files. No application behavior was changed by this cleanup. The PR must rerun CI before any merge decision. -
Fresh-checkout build gate repaired: root
bun run buildnow builds docs and all workspaces; CI installsdocs-site's locked npm dependencies before invoking it. Localbun run buildcompletes across docs, admin, packages and bundled CLI. Vite chunk-size and esbuildimport.metawarnings remain non-fatal and are tracked separately; the previousdocmd: not found/ docs-only build mismatch is fixed in this branch.
- v1.9 macro closed — junior quick-start truthing. Five sub-task PRs merged into
task/v1.9-junior-quickstart-truthing:- PR #52 —
aqa install-agent-filesCLI verb. CablesrenderForTargets()(@aqa/adapters) into a real command.--targets <csv>,--project-name <slug>,--force,--dry-run. 14 tests. 2 Copilot iter passes (array-form trim + Windows trailing-space in test temp dir, help text path.github/copilot-instructions.md, extractedlastPathSegment+slugifytocli-utils.ts, then iter 2: slugify cap to 64 chars +KNOWN_TARGETSderived fromadaptersregistry). - PR #53 —
aqa reportCLI verb. Cables@aqa/reporter(md+json) readingevents.jsonl+findings.jsonl.--run-id,--format md|json|both. 24 tests. 3 Copilot iter passes covering: state derivation fromrun_finishedpayload counters (not "any-finished=succeeded"), mtime-based latest run (vs lexical, broken for--seedhashes), missing-artifact fail-fast,--run-idLongSlug regex (mirrors@aqa/schemasSlugPattern + 256-char cap),readJsonlrejects non-object lines (null/array/string/number), symlink protection on run dir AND per-file (report.md/report.json), reconstructed Run revalidated viaRun.Run.safeParse. - PR #54 —
aqa adminCLI verb. Bootsnode:httpserver in-process serving bundleddist/admin/SPA + delegating/api/*tomakeApi().--port,--host,/api/healthzalways-200. Path-traversal-safe static serving, SPA fallback for client routes. SeedsMemoryStorefrom.aqa/runs/. Newpackages/pack-author/package extracted to break the kit↔server build cycle. 7 admin tests + 2 pack-author tests. 3 Copilot iter passes (stale-cycle comment,--host 0.0.0.0security warning, pack-author README + named-export comment). - PR #55 — GitHub Packages publish pipeline. esbuild bundles every workspace+npm dep into
dist/cli.cjs(~570 KB CJS-in-.cjs).scripts/publish-prep.mjsswaps@aqa/kit→@padosoft/agentic-qa-kitAND strips@aqa/*deps (inlined in bundle)..github/workflows/publish.ymlruns onv*tag,npm publish --provenance --access publictohttps://npm.pkg.github.com.LICENSEcopied intopackages/kit/. 4 publish/bundle tests + POSIX exec-bit assertion. 2 Copilot iter passes (CRITICAL@aqa/*strip, stale doc comments, LICENSE missing). - PR #56 — docs refresh. README +
docs/getting-started.mdrewritten 1:1 with shipped verbs. Adds GH Packages auth.npmrcsnippet (P1 junior trap), 10-step quick-start,aqa adminsingle-command boot,bun run e2e:ecosystempointer. CHANGELOG[1.9.0]entry + backfilled[1.8.1]/[1.8.2]/[1.8.3].
- PR #52 —
- Strategy: 5 PRs in parallel to amortise Copilot review latency. Each merged in cascade: 52 → 53 → 54 → 55 → 56, with conflict resolution at each step on
packages/kit/package.json(deps + test scripts) andpackages/kit/src/cli/aqa.ts(case statements + VALUE_FLAGS). 12 packages, 270 tests pass locally after the cascade. - Bundle health snapshot:
node packages/kit/dist/cli.cjs --version+--helpboth work end-to-end; all 7 verbs (init / doctor / validate / install-agent-files / run / report / admin / pack new) listed. Bundle size 571 KB (admin SPA + makeApi inlined via dynamic import). - Next: macro PR
task/v1.9-junior-quickstart-truthing→main, thengit tag v1.9.0+ GitHub Release. The tag triggers.github/workflows/publish.ymlwhich publishes@padosoft/agentic-qa-kit@1.9.0to GitHub Packages.
- v1.x roadmap closure completed. Added a dedicated ecosystem Playwright smoke (
packages/admin/test/e2e/ecosystem-live.e2e.ts) with a single-command stack bootstrap (scripts/ecosystem-stack.mjs) that bootsexamples/bun-api, runs a realaqa run --profile smoke, serves live/api/*from@aqa/server.makeApi+MemoryStore, and drives the admin against that live backend (VITE_AQA_SERVER_URL). The test assertsfinding_emittedis visible from live/api/auditdata and that chain verification returnsCHAIN OK. Command:bun run e2e:ecosystem. - Docs/progress alignment closed for v1.x. The previous "in progress/deferred" notes for the v1.6→v1.8 follow-ups are now materially closed in code and release artifacts (
v1.8.2published). Historical bullets below remain as timeline context; current operational status is: no open PRs, no open issues, ecosystem smoke present and green. - v1.8.2 slice closed — CLI smoke now runs a real HTTP end-to-end path.
scripts/e2e-cli.mjsno longer stops at version/help/doctor/validate only: it now boots a local HTTP/healthztarget, seeds a schema-valid local smoke pack/profile, executesaqa run --profile smokewith the real HTTP probe runner, and asserts run artifacts are emitted under.aqa/runs/<run-id>/(events.jsonlnon-empty,findings.jsonlpresent). This closes the old “CLI smoke is command-only” gap and makes CI catch integration regressions earlier. - v1.8.1 slice closed — audit-chain canonical reconciliation. Aligned
@aqa/compliance.verifyEventChainwith@aqa/runner.EventChainWriter: hash recomputation now excludesprev_hashfrom canonical body (matches writer), and first-recordprev_hash: nullis now treated as canonical instead of expecting all-zero literal in the field. Updated compliance tests and removed stale divergence note in@aqa/kitrun smoke tests. - v1.x docs closure in progress — README/docs refresh pass started. Removed stale preview/stub wording from README, added the new How you use it section after the 7-word model, updated quick-start flow to the current shipped commands (including admin panel boot), and aligned
PACK-AUTHORING.mdwith the real HTTP probe runner now shipped in v1.8 (aqa runusesproject.sut.base_urlforhttpprobes). - v1.7 slice 4j closed — AuditChainViewer autoload from live initial chain. Removed the manual dependency on "Load good chain" for live audit data:
AuditChainViewernow consumesinitialChainreactively, resets verify state safely on incoming chain changes, and both Audit pages pass normalized/api/auditevents viainitialChain. Added e2e coverage proving/api/auditdata auto-loads and verifies toCHAIN OKwithout demo-button interaction. - v1.7 slice 4f closed — Admin section pages wired to existing endpoints. PR #40 (
4c93bb7). PageTokens fetchesGET /api/tokenswithx-aqa-org(adapts@aqa/schemasApiToken to the page's fixture shape:display_name → name,last_used_at → last_used, owner-prefix heuristic forkind). PageOrg fetchesGET /api/orgsand joins live slugs into the subtitle. PageAdminAudit shares the slice 4e/api/auditwire with admin-view copy via a newnormalizeAuditEventsForViewerhelper.fmtDate/fmtDateTimemade null-safe (em-dash for missing dates). Create-token modal scope chips switched from pre-schemaruns:write/packs:install/adminto the actualApiTokenScopeenum. Users/Roles/SSO deferred — no server scaffolding exists; out of scope. 4 new e2e tests inadmin-section.e2e.ts. 6 Copilot review iterations. - v1.7 slice 4e closed — Operations admin pages wired. PR #39 (
99633d5). PageAudit/PageQueue/PageCost/PageNotifications now read from existing/api/audit,/api/queue,/api/cost/summary,/api/notificationswith graceful fixture fallback. Schema-true normalizers (server EnqueuedJob → UI job shape, Event → AuditChainViewer demo shape, Notification → fixture title/body/unread). Cost passes explicitfrom/tofor MTD bounds. SELF resolves from SESSION_USER.id. Notifications filter list derives from serverNotificationKindenum. 5 new e2e tests inoperations.e2e.ts. 7 Copilot iterations. - v1.7 slice 4d closed — Agents page refactored around real data. PR #38 (
d97720a). New@aqa/schemas Agent, newagents:read/agents:editpermissions (legacyagents:installaliased), new server CRUD (GET /api/agents,GET /api/agents/:id,POST /api/agents/:id/install,POST /api/agents/:id/uninstall), newStore.listAgents/loadAgent/installAgent/uninstallAgent(Memory implements +__test_seedAgenttest-only; Postgres scaffold notImpl). PageAgents fetches the live list with fixture fallback; install/uninstall buttons call real endpoints with in-flight guard + toasts.Agent.filesvalidated asSafeRepoPath(rejects leading/, drive letters,..segments, UNC roots). Empty server list is authoritative. Header counts sourced from state. 5 new e2e tests inagents.e2e.ts. 7 Copilot iterations. - v1.7 slice 4c.7+4c.8 admin closed — Scenario edit/clone YAML wizard. PR #37 (
a1a0d0a). SharedScenarioYamlWizard(mode: 'edit' | 'clone') with a YAML textarea seeded from a schema-conforming stub (or, in edit mode, the persisted override / created body). Client parses YAML via the newyamladmin dep; debounced 150ms with sync-on-seed. UX errors block submit: parse error, body-not-an-object, missing/non-string id, Slug-regex violation (matches@aqa/schemasSlug), clone-empty/clone-same-as-source/collision, edit path/body id mismatch. App-levelupdatedScenarios+createdScenariosMaps +aqa:scenario-updated/-createdevents with prototype-pollution guard (safeMergeObjectstrips__proto__/constructor/prototype). Scenario fixtures migrated from dotted ids (api.tenant.cross_tenant_search) to dashed Slug-compliant ids; tree grouping switched to explicitcategoryfield. Spec preview re-renders from override via__aqaYamlStringify. 10 e2e tests inscenario-yaml.e2e.ts. 8 Copilot iterations. - v1.7 slices 4c.1 through 4c.8 closed. Profile Delete/Edit/Clone (PRs #29/#30/#31), Risk Delete/Edit (PRs #32/#33), Scenario Delete (PR #34), Scenario Edit/Clone server-side hardening (PRs #35/#36). All 8 micro-PRs squash-merged; the architecture lessons (
inFlightRefsynchronous guard, captured-submittedId stale-submit guard, render-time refs not effect-time, modal close-affordance inertness during submit, App-level lifted state withaqa:*CustomEvents, atomiccreateXin Store, mock-id migration to schema-conforming Slug) carried through every later slice. - Final v1.7 closing step. Tag
v1.7.0, GitHub release notes covering the full slice 4 surface (Operations + Admin sections wired to existing routes, Agents fully end-to-end, Scenario admin via YAML, Profile/Risk/Scenario CRUD complete). - v1.7 slice 4g closed — Users + Roles admin pages wired (post-v1.7.0 patch). PR #42 (
77f3b1c). NewGET /api/users(returns the store's directory snapshot,settings:read-gated) andGET /api/roles(returns the@aqa/authrolePermissionsmatrix +Permission.optionsenum for the "every permission" axis, so wildcard-only perms likesettings:editshow as rows in the live grid). NewStoreProvider.listUsers()+ sharedStoreUserDirectoryEntrytype (exported from@aqa/storeso adapters and API handlers agree on field names). PageUsers fetches with display_name→name / suspended→disabled adapter; PageRoles renders the live (permission × role) matrix withadmin:everythingas wildcard. Fixture fallbacks preserved. 4 e2e + 3 server tests. 2 Copilot iterations. SSO config wiring still deferred — needs a new schema, tracked as a future slice. - v1.7 slice 4h closed — SSO config wiring completed. Added
SsoConfigschema in@aqa/schemas(+ fixture + JSON schema emitter/export),StoreProvider.loadSsoConfig()in@aqa/store(Memory implementation + Postgres scaffold), newGET /api/sso/config(settings:read) in@aqa/serverwith schema-validation guard, and Admin SSO page wire-up to live config with fixture fallback. Added server/store tests and admin e2e coverage for live/fallback SSO rendering. - v1.7 slice 4i closed — SSO config write-path enabled. Added
StoreProvider.saveSsoConfig()(Memory implementation + Postgres scaffold), newPUT /api/sso/config(settings:edit) in@aqa/serverwithSsoConfigschema validation, and Admin SSO save wiring to persist live edits. Security hardening applied during review: removeddangerouslySetInnerHTMLin SSO alert, added explicitconfig: nullrendering state, and stabilized domain chip keys. Added store/server tests plus admin e2e forPUT /api/sso/config.
- Started
task/bootstrap-governancefrommain(commitc25dd4e= initial). Goal: stand up the process governance — branch strategy, validation loop, Copilot review automation, docs scaffolding, repo tooling — before any product code is written. - Task 0 closed. PR #1 merged (
d9cb4b3). Tagv0.0.1-governancepushed. 8 Copilot review iterations; 52 actionable comments addressed. - Task 1 closed. PR #2 merged (
3871cd1).@aqa/schemas— Zod source of truth + JSON Schema (Draft 2020-12) generated artifacts. Determinism contract from §3.1 codified inFinding. Hash-chained audit codified inEvent. 4 Copilot review passes; 29 actionable comments addressed. Follow-up #3 tracks remaining JSON-Schema parity work. - Task 2 closed. PR #4 merged (
895cec9).@aqa/kit—aqaCLI (init/doctor/validate) + project profiler. CI bun + Node 22 jobs aligned to per-package script runner; topological build added (run-workspace-script DFS sort) so downstream packages can resolve workspace imports through dist/. - Task 3 closed. PR #5 merged.
@aqa/pack-loader+ 5 baseline packs (core / api-core / web-ui / llm-agent / security). One Copilot review pass; 15 actionable comments addressed (slug placeholders, manifest descriptions, OWASP coverage scoped to v0.1.0 subset). - Task 4 closed. PR #6 merged.
@aqa/adapters— Claude, Codex, Gemini, Copilot adapters with per-target capability profiles and deterministicrender(ctx). - Task 5 closed. PR #7 merged.
@aqa/runner— RunLifecycle state machine, hash-chained EventChainWriter (end-to-end verified), FindingsWriter (in-run dedup), built-in oracles, runScenario orchestrator. - Task 6 closed. PR #8 merged.
@aqa/reporter— Markdown + JSON reporters + 3-level replay artifact generator (repro.sh, repro.curl, repro.playwright.ts). - Task 7 — admin panel bootstrap done.
packages/admin(@aqa/admin, private) — Vite + React 19 + TS strict scaffold with a 12-route sidebar shell (Dashboard, Runs, Findings, Risk map, Profiles, Packs, Scenarios, Agents, Replay, Audit log, Cost, Settings). Each route renders a typedScreenPlaceholderdocumenting what lands when. Vite build producesdist/(197 KB JS, gzip 62 KB). Full Tailwind 4 wiring, TanStack Router migration, and per-screen data wiring are deferred to Task 17 (task/admin-editing). 4 node:test tests; 86 repo-wide. - Repo health snapshot: 9 packages (schemas, kit, pack-loader, adapters, runner, reporter, admin + 5 packs), 86 tests passing under both Bun and Node 22, biome + tsc strict zero errors, hash-chained audit verified, JSON Schemas Draft 2020-12 compliant.
- Next: Task 8 —
docs/getting-started.md(junior 15-min onboarding),docs/architecture/reference.md(real diagram + component map),docs/methodology/agentic-qa.md(Risk/Invariant/Probe/Oracle), ADR-001..ADR-009, examples/bun-api, examples/nextjs-saas, then v0.1.0 release tag. Task 9 (FINAL) — knowledge consolidation across LESSON.md / RULES.md / agent files. - Tasks 8 — 22 closed. v0.1.0 through v0.6.0 tags pushed (#9..#16). Stack grew to 18 packages: schemas, kit, pack-loader, pack-scanner, adapters, llm-adapters, runner, reporter, admin, admin-core, auth, sandbox, store, generator, server, clustering, methodology, + 5 packs. Deploy scaffolds (
deploy/helm,deploy/terraform,scripts/air-gap-install.sh) shipped with explicit "v0.6 / v1.0" labels. - Task 23 — v1.0 readiness in progress.
@aqa/complianceships SOC2/ISO controls catalog (CONTROL_MAPPINGS,controlsCoverage) + hash-chain audit verifier (verifyEventChain,aqa-audit-verifyCLI).docs/compliance/soc2-iso-mapping.mdis the auditor-facing source of truth;docs/compliance/pen-test-scope.mdis the engagement contract. 7 new tests; 165 repo-wide. - v1.1 polish shipped (PR #18, tag
v1.1.0). README banner now points to a real PNG.deploy/helmis feature-complete (runner StatefulSet w/ per-pod PVC, optional Ingress + TLS, NetworkPolicy that confines runner egress, optional in-cluster Postgres subchart). Three examples:bun-api,nextjs-saas(session-cookie invariant),laravel-app(demonstrates language-agnostic targeting).docs/LESSON.mdconsolidated retrospective. GitHub Releases backfilled for every tag fromv0.0.1-governancethroughv1.1.0. README pre-alpha badge replaced with GA + Release badges.
-
Audit viewer truthfulness increment shipped locally. The admin now preserves the raw API audit record and verifies
prev_hashplus the canonical SHA-256 digest in the browser via Web Crypto. A live tampered record therefore rendersCHAIN BROKEN; the previous continuity-only check could accept altered payloads. The live Playwright fixture now computes real hashes and the reproduction harness asserts the built SPA against the realrunAdmin()HTTP server with zero API mocks. -
Evidence:
node docs/internal/reviews/2026-09-17-reproduce.mjsreportsaudit-verifier-backend.backendAcceptsTamper=falseandbuilt-ui-real-api-tamper.chainOk=false;bun run --filter @aqa/admin buildand admin typecheck pass. The ecosystem E2E remains environment-blocked by a Bun cacheENOENTwhile installinghono, not by an assertion failure. -
Next: tenant/auth enforcement at the HTTP adapter boundary, then durable store semantics and ecommerce journey packs. Copilot review explicitly waived by maintainer instruction for this roadmap run.
-
HTTP authorization boundary increment shipped.
aqa adminnow authenticates and enforces each route's declared RBAC permission before dispatching tomakeApi();AdminOptions.authenticateallows a real OIDC/JWT verifier to be injected. An unauthenticated/api/orgsrequest is covered by a live HTTP test and returns 401. The default localhost mode remains an explicitly local admin identity and is not an enterprise identity provider. -
Finding attestation increment shipped.
POST /api/findings/:id/statusnow requires tenant scope, loads the authoritative run, validates the complete candidate Finding through the Zod schema before mutation, and emits a hash-chained audit event containing actor, old/new status and reason. Invalidverified/duplicatetransitions are rejected without changing state. Server tests: 93 passing. -
Non-loopback admin hardening shipped.
aqa admin --host 0.0.0.0now refuses to boot unless an explicit authentication callback is supplied. This closes the unsafe LAN default while keeping localhost development convenient. -
Replay truth increment shipped.
verifyScenario()now computes a stable SHA-256 failure fingerprint from scenario and failed-oracle reasons, records it per attempt, and optionally compares it with the original finding fingerprint. Different failures no longer count as deterministic reproduction; runner tests: 26 passing. -
Replay artifact integration shipped.
aqa runnow materializesreplay/repro.shandreplay/repro.curlfor every HTTP finding before finalizingrun_finished; artifact count/errors are recorded in the audit event and write failures fail the run. Relative probes resolve against the configured SUT base URL and header values are shell-quoted. Kit/reporter replay and run-command tests: 29 passing. -
Queue fencing increment shipped.
RunnerQueue.dequeue()issues a fresh lease token and returns a snapshot;ack(id, token)rejects missing/stale worker tokens, while requeue/kill clear them. A real stale-worker interleaving test now proves the old worker cannot close the reassigned job. Server tests: 92 passing. -
Tenant finding isolation increment shipped.
GET /api/findingsnow requires project scope and resolves each finding through its authoritative run before returning it; findings from another project and findings with no resolvable run are excluded. Added a cross-project server regression; server tests: 93 passing. -
HTTP probe safety increment shipped. The real HTTP driver now defaults to the configured target origin, rejects non-allowlisted absolute URLs, does not follow redirects, bounds response evidence to 1 MiB, and reports oversized/invalid responses as execution errors. Negative origin and body-size tests pass; runner tests: 27 passing.
-
Cost safety increment shipped.
BudgetTrackerno longer treats an unknown model as free:charge()records a pricing error and exhausts the tracker, whilewouldExhaust()returns true. Existing known-model arithmetic is unchanged; cost tests: 5 passing. -
Pre-persistence redaction increment shipped. Audit payloads are recursively redacted before canonical hashing/persistence and finding text/evidence is sanitized before JSONL output. Bearer tokens, secret-like keys, PAN-shaped values and related sensitive strings are not retained; runner tests: 28 passing.
-
Replay command hardening shipped. Generated curl artifacts now resolve relative URLs against the SUT base URL, use
--data-raw, preserve expected 4xx responses (-sSwithout-f), and shell-escape method/URL/header/body arguments including embedded single quotes. Reporter tests cover the hostile quoting case. -
Commerce Assurance foundation shipped. Added new
@aqa/commercepackage with versionedMoney,CommerceContext,OrderSnapshot,PaymentSnapshot,InventorySnapshotandJourneyOutcomecontracts plus fail-closed currency/oversell invariants. Four contract tests pass afterbun install; this is the contract layer, not yet a merchant/payment adapter or full checkout journey. -
Commerce reference vertical slice shipped.
@aqa/commercenow includes a deterministic in-memory merchant for synthetic carts, checkout, captured payment snapshots, tenant/customer authorization, idempotency-key replay and no-backorder inventory fencing. Added tests for exactly-once retry behavior, incompatible-key rejection, last-item competition and cross-customer/tenant denial. This is a reference test merchant; it is not evidence for a real payment provider or durable production persistence. -
Refund integrity slice shipped. The reference merchant now models a refund ledger snapshot, cumulative refunded amount, partial/full refund states, strict currency matching, idempotent refund retries and over-refund rejection. Nine commerce tests pass; real provider reconciliation and durable transactional persistence remain open roadmap work.
-
OIDC authentication slice shipped.
OidcAdapternow performs provider discovery, Authorization Code + optional S256 PKCE token exchange, UserInfo retrieval, strictUser/AuthSessionvalidation and supported-role mapping; missing secret, endpoint, identity or role fails closed. Auth tests: 7 pass. Application-level CSRF state binding, session cookie/rotation and durable tenant membership are still required for production deployment. -
HTTP tenant-membership boundary shipped.
ApiContext.authorizeScopeand the bundled admin dispatcher now support server-side org/project membership checks after authentication/RBAC, with403on a valid-role user requesting an unauthorized scope. Added a real HTTP boot regression; server/admin boundary tests: 97 pass. The local loopback admin default remains development-only, and standalone deployments must provide the callback. -
Postgres persistence slice shipped. Replaced the
PostgresStorenot-implemented scaffold with a realpostgresdriver adapter: idempotent schema marker/table creation, durable JSONB records, tenant indexes, hash-addressed audit events, atomic profile/scenario creates, server-side filtering and graceful connection-close semantics. Store tests: 8 pass; the real-DB contract is conditional onAQA_TEST_POSTGRES_DSNand was not claimed locally without a database. -
Postgres CI evidence added.
.github/workflows/ci.ymlnow provisions PostgreSQL 16 as a service and runs the store integration contract with an ephemeral CI DSN after a fresh workspace build. Local Docker is unavailable, so the GitHub job remains the authoritative runtime evidence still to be observed. -
CI fresh-job build regression fixed. Package-only jobs now invoke
build:workspace; jobs that require the root docs build installdocs-sitedependencies withnpm cifirst. This removes the observeddocmd: not foundexit 127 from independent CI jobs. -
Postgres CI round-trip bug fixed. The first real PostgreSQL 16 run exposed that JSONB values returned through the unsafe driver path can be strings; centralized decoding now restores typed payloads before reads/event results. Local store tests remain 8/8; CI must re-prove the live contract.
-
Finding identity increment shipped. New runner findings use 20-digit UUID-derived numeric suffixes instead of the per-run scenario counter, preventing cross-run overwrites in stores keyed by
finding.id; historical four-digit IDs remain schema-compatible. A regression test and diagnostic prove two runs preserve two distinct findings. -
v1.7 slices 1+2 shipped — pack authoring tutorial +
aqa pack newCLI. PR #25 merged (6cc0013), prerelease tagv1.7.0-rc.1published. 19 review iterations with Copilot + Codex; the convergence pattern hit a sharp tail (5→1→4→2→1→2→0 real items per round) after Copilot started re-flagging the same ~13 already-addressed comments. Real issues caught and fixed before merge: slug-length validation against derived-ID schema cap (52-char limit), in-memory schema validation of generated Scenario/RiskMap/PackManifest before writing, symlink rejection at both packs/ parent and packDir, non-directory parent rejection, atomic backup-rename--force(failed scaffolds restore the original pack),package.json#filesmatching reality, scoped publish guidance, schema-valid profile snippet, integration test assertsscn-pack-demo-starteractually executed (rejects false-positives via bundled packs), honest NO_NETWORK_PROBE documentation. 54 tests in@aqa/kit(12 pack-new + 42 run-cmd). Still pending in v1.7: slice 3 (admin Create-pack wizard) and slice 4 (audit + wire/implement 81 silent admin placeholder buttons, plan indocs/internal/admin-placeholder-audit.md). Finalv1.7.0tag after those slices ship. -
v1.6 shipped —
aqa run+ bundled packs + ecosystem foundation. PR #24 merged (21d7b10), tagv1.6.0pushed, GitHub release published. The CLI now has the missingaqa runcommand that closes the loop betweenaqa initand a real audit trail. 21 review iterations with Copilot + Codex, every one surfacing a real bug or coverage gap (zero false alarms). 42 TDD tests inpackages/kit/test/run-cmd.test.tscover every behavior. Highlights: SUT-aware init pack selection, three-tier pack discovery (project / node_modules / kit-bundled — all 5 baseline packs now ship inside@aqa/kit's tarball viabundle-packs.mjs), atomic run-dir creation (TOCTOU-safe for concurrent seeded runs), path-traversal + symlink-escape rejection,applies_whenfiltering, manifest-name dedup with priority, legacy bare-slug aliasing, agent-mode rejection until that driver lands, unrelated-broken-pack tolerance with structuredwarnings, capped error strings (MAX_DETAIL_PER_KIND), detail samples inrun_finishedaudit event for auditors. Known scoped follow-ups: real HTTP probe runner (current is no-network stub → release-gate strict semantics deferred),EventChainWriter↔verifyEventChaincanonical-form reconciliation, browser-driven ecosystem smoke. -
Next macro task — v1.7 pack-authoring story. Per user confirmation: (a)
docs/PACK-AUTHORING.mdcommunity tutorial, (b)aqa pack new <slug>CLI scaffolding, (c) Admin "Create pack" wizard wired over the new CLI. PLUS: a full audit pass on every placeholder button/interaction in the admin panel — noonClick={() => {}}or no-op silent clicks. Each placeholder either gets wired to a real endpoint, gets a client-side implementation, or gets an explicit "decorative" doc note. -
v1.5 admin design integration shipped. PR #23 merged (
f7b879f), tagv1.5.0pushed, GitHub release created. The 30-screen hi-fi prototype from Claude Design is now the official admin web panel: bundled intopackages/admin/src/app.tsx(8.9k LOC,@ts-nocheck), token-driven CSS, Vite production build. NewE2E (Playwright, admin UI)CI job runs the full Playwright suite (*.e2e.ts) — per-screen smoke for all 19 nav routes + audit-chain verify (OK/tampered) + Findings views (Clusters/List/Kanban) + Replay tabs + risk-map matrix + theme + palette. Total 36 Playwright tests green in 1m27s. Known scoped tradeoffs (deferred): in-memory routing only (not URL-driven), live-mode still reads in-file mocks (no real fetch layer wired). Both intentional for the design port; will be picked up in v1.6. -
v1.5 lessons captured. Documented in
docs/LESSON.md: (a) bundled-prototype@ts-nocheckpattern with Biome ignore-list; (b)window.useTweaksfallback injection for design-tool-only hooks; (c) Playwright.e2e.tsextension to avoid Bun's test runner picking it up; (d) nav-item locator pattern (no$anchor, escape regex metas, target prototype's actual.replay-tab/.seg-btnclasses, notgetByRole('button')). -
Next macro task — v1.6 ecosystem end-to-end smoke. Full end-to-end ecosystem smoke via Playwright: boot server + runner pool + admin in a single command, drive a real
aqa runagainstexamples/bun-api, verify findings appear in the admin, verify audit chain remains valid. TDD: any broken path → failing test first, then fix. After that, the README/docs refresh closing step (see below). -
v1.4 admin API surface (in flight). Expanded
packages/server'smakeApi()from 4 to 28 routes covering everythingdocs/design/admin-panel-spec-v2.mdreferences: runs detail + events, finding status mutation, packs CRUD, profiles CRUD, risks CRUD, scenarios edit, audit query, cost summary, queue snapshot, notifications, saved views, API tokens, tenancy (orgs + projects).StoreProviderextended with matching methods;MemoryStoreimplements all of them (Postgres scaffold throwsnot implemented). New@aqa/schemasnamespaces:Notification,SavedView,ApiToken,CostSummary,Tenancy. Multi-tenant viax-aqa-org/x-aqa-projectheaders. 8 new tests; 184 repo-wide. -
Design brief for admin v2 shipped.
docs/design/admin-panel-spec-v2.md— self-contained enterprise-grade spec (tokens, 30 screens, full component library, interaction patterns, a11y, perf budget, deliverables checklist) so an external designer (or Claude Design) can build the React template in parallel. -
Next macro task (post-admin-design). After admin v2 design lands and integrates: full end-to-end ecosystem smoke via Playwright — boot server + runner pool + admin in a single command, drive a real
aqa runagainstexamples/bun-api, verify findings appear in the admin, verify audit chain remains valid. TDD: any broken path → failing test first, then fix. -
Issue #3 closed. Mirrored 3 remaining Zod superRefines into JSON Schema (Finding
status='duplicate' ⇒ duplicate_of, ReproLeveldeterministic=true ⇒ attempts >= 1, ProfilesFileprofile.name === keyvia$comment). Added Ajv 2020 round-trip test (packages/schemas/test/ajv-roundtrip.test.ts) that validates every fixture against the emitted schema — catches Zod/JSON-Schema divergence at build time. 204 tests repo-wide. Patches resolve#/definitions/<name>indirection emitted by zod-to-json-schema. -
PR #22 local gates verified (2026-05-18).
bun install✅,bun run build✅,bun run typecheck✅,bun run lint✅ (4 warnings, no errors),bun test✅ 204/204. -
BLOCKER — Copilot review request (PR #22). Both
gh pr edit --add-reviewer copilot-pull-request-reviewerand GraphQLrequestReviewsByLoginreturn HTTP 403 (DNS monitoring proxy blocks GitHub API). Action required: please open PR #22 in the GitHub UI and manually addcopilot-pull-request-reviewerfrom the Reviewers sidebar panel. -
Final closing step (after every macro task above is closed). README + docs refresh pass:
- Audit every
v0.x.xreference inREADME.md— replace stale ones with the current shipped surface or drop. - Quick-start section: remove the "preview of v0.1.0" disclaimer; write the definitive end-to-end junior flow that actually works today, including booting the web admin panel. No more "this will work in vX" hedging.
- Architecture section in
README.md: refresh diagram + component list to match the 18 packages shipped (schemas, kit, pack-loader, pack-scanner, adapters, llm-adapters, runner, reporter, admin, admin-core, auth, sandbox, store, generator, server, clustering, methodology, compliance). docs/: audit every file, prune obsolete content, keep only current/good. Anything that says "stub" or "lands in vX" must either be filled in or removed.- After "The mental model in 7 words" section, add a new section titled "How you use it" — clean, concise, written in the same rhythm as "7 words" — describing the end-to-end junior workflow:
aqa init(detect repo, scaffold.aqa/)- edit
risk-map.yaml(declare what matters) - install agent files for your coding agent
aqa run --profile smoke(skills + scenarios + runner + oracles)- open admin panel (
bun --filter @aqa/admin dev) - inspect findings, replay deterministically, verify audit chain
- iterate on risks + scenarios until release-gate green
- Tag the README/docs refresh PR as the official closure of the agentic-qa-kit v1.x line.
- Audit every
- v1.2 admin wired.
@aqa/adminmigrated from inline-style placeholder shell to a real SPA: Tailwind 4 + TanStack Router + TanStack Query + Zustand + lucide-react. 12 screens shipped end-to-end: Dashboard (KPIs), Runs (table), Findings (clustered via content-hash signature, async via Web Crypto), Risk map (grouped by category), Profiles, Packs (with signature badge), Scenarios (pack→scenario tree), Agents (per-agent instruction-file detection), Replay (per-finding repro.sh / repro.curl preview + verify button), Audit log (paste events.jsonl → re-walk the sha256 chain in-browser; "Load good chain" / "Load tampered chain" demo buttons), Cost (bar by profile), Settings (theme toggle). - Browser-side hash-chain verifier.
node:cryptois not Vite-safe, so the admin re-implementsverifyEventChain+signatureOfon top ofcrypto.subtle.digest. The CLI version in@aqa/complianceremains the SOC2 source of truth; the in-browser copy is a UX affordance only. Documented indocs/LESSON.md. - Build: 376 KB JS (116 KB gzip), Tailwind CSS 9.94 KB (2.92 KB gzip). 165 tests still pass.
- Closed the legacy
POST /api/packssupply-chain bypass: JSON manifests now pass schema validation and the same critical/high scanner gate as YAML imports. - Added duplicate protection (
409 EEXIST) with explicitforce=truereplacement semantics; malformed, unsigned shell, and invalid signed JSON manifests are rejected before persistence. - Added canonical parsed-JSON digest verification to
@aqa/pack-scanner; this is integrity verification only and does not establish Sigstore publisher trust. - Evidence:
bun run --filter @aqa/pack-scanner test(8 passed),bun run --filter @aqa/server test(96 passed),bun run --filter @aqa/server typecheck(passed). - Next: implement durable artifact storage and redaction-aware artifact lifecycle; remaining enterprise gaps include WORM audit transactionality, OIDC/Vault/S3/Sigstore trust roots, live LLM adapters, and ecommerce journey packs.
- Added
@aqa/artifactswithArtifactStoreand localFileArtifactStoreimplementation: content SHA-256 references, atomic temp-file writes, metadata sidecars, safe relative keys, traversal rejection, text/JSON pre-write redaction and explicit binary API. - Added canary tests for bearer/JWT/AWS/PAN/email redaction, JSON key redaction, binary byte preservation, deletion and traversal resistance; added ADR-013.
- Evidence:
bun run --filter @aqa/artifacts test(3 passed), typecheck and build passed. - Honest gap: existing runner/reporter/server paths still write directly to
.aqa/runs; wiring them to this contract plus S3/MinIO/WORM/tenant authorization is the next acceptance slice. - Wired
aqa runreplay generation throughFileArtifactStore; the real run journey now produces replay files plus content metadata sidecars through the redaction-aware boundary. The event/finding writers andaqa reportremain direct-file paths and are intentionally the next migration surface.
- Replaced the live-provider gap for OpenAI-compatible
openai,ollamaandvllmwith a real HTTP adapter: injectable fetch for deterministic tests, timeout cancellation, bounded output tokens, model-version hash, usage extraction, redacted request/response/error content and BYOK endpoint/key options. - Anthropic/Google/Cohere/Bedrock remain explicit scaffolds until provider-specific contracts, auth, regional routing and replay/fixture tests are implemented; this is not a claim of complete multi-vendor production readiness.
- Evidence:
bun run --filter @aqa/llm-adapters test(8 passed), typecheck and Biome passed.
- Added
OidcSessionManagerwith one-time state, S256 PKCE binding, short-lived sessions, HttpOnly cookie handling and explicit revoke; addedaqa admin/auth/login,/auth/callbackandPOST /auth/logoutroutes. - OIDC mode is fail-closed and cannot silently use the local admin identity. Loopback cookie security is configurable so HTTP development does not emit an unusable
Securecookie; non-loopback defaults toSecure. - Complete journey evidence: admin test performs login redirect, callback, authenticated API request, logout and post-logout denial. Auth tests: 8 passed; kit tests: 107 passed, 2 platform skips; kit typecheck and Biome passed.
- Remaining enterprise gap: the session map is process-local; multi-replica production still needs a shared encrypted session store and reverse-proxy TLS contract.
- Hardened the Helm chart: durable server audit PVC enabled by default, non-root UID/GID, RuntimeDefault seccomp, dropped capabilities, no service-account token, read-only root filesystem, termination grace period, server startup/readiness/liveness TCP probes, and constrained ingress-controller namespace selector instead of
{}. - Documented the operational limits: TCP probes are temporary until a deployable server wrapper exposes a dedicated health endpoint; audit PVC is not WORM or a backup; operators must set the real ingress namespace labels.
- Helm CLI is not installed in this Windows workspace, so chart rendering/lint remains a CI/operator-side verification gap for this increment.
- Added a GitHub
Validate Helm deployment chartjob to lint and render both default and ingress/TLS production-shaped values, assert the audit PVC is rendered, and reject an unrestricted ingress namespace selector. Local Helm remains unavailable, but the verification is now executable in CI.
- Added
POST /api/runner/jobs/:id/ack; workers must present the current fencinglease_token, and stale/unknown tokens return409without completing the job. - Added optional
runnerAuthorizeenforcement to both dequeue and ACK routes, preserving local test compatibility while allowing production deployments to require a dedicated runner credential rather than a user session. - Evidence: server suite 98/98 passed, server typecheck and Biome passed. PostgreSQL lease fencing remains covered by the existing live CI integration.
- Exposed the same
runnerAuthorizecallback throughrunAdmin, with a complete admin HTTP test proving unauthorized dequeue is401and the configured runner credential reaches the route.
- Added
aqa verify <finding-id>as the first executable fix→verify loop: it locates a persisted finding, resolves the owning scenario from project or installed packs, requires a real--base-urlor an injected probe runner, replays with bounded attempts, and writes a unique verification evidence artifact beside the run. - Deterministic replay returns exit code 0; completed but flaky replay returns exit code 2; missing finding/scenario or missing network boundary fails closed. The command deliberately does not auto-close findings or imply CI/PR/deployment verification.
- Evidence:
bun run --filter @aqa/runner build;bun run --filter @aqa/kit typecheck;bun run --filter @aqa/kit test(113 passed, 2 platform skips); andgit diff --checkall pass. Remaining slice: publish the command and connect verification evidence to durable finding status/audit events.
- Added
measureRiskCoverage()to derive the documented coverage score from a validated risk map, scenario risk/invariant links, oracle declarations, and timestamped run observations. It counts recent pass rate, deterministic replay, flaky scenario history, and stale evidence without treating missing artifacts as success. - The aggregation is persistence-agnostic so CLI, API, and warehouse adapters can share one deterministic boundary. It currently consumes supplied observations; store/API ingestion and an admin coverage endpoint remain open.
- Evidence:
bun run --filter @aqa/methodology typecheck;bun run --filter @aqa/methodology test(9/9);bun run lint.
- Extended
@aqa/clusteringwith stableroot_cause_idderivation and explainable priority (severity × confidence × blast_radius / cost_to_fix_estimate) while preserving conservative fingerprint-only grouping. Optional finding estimates are schema-validated and bounded; missing estimates default to neutral1. - Evidence and remaining gap: clustering tests/typecheck/lint prove deterministic grouping and prioritization. Cross-fingerprint semantic clustering, persistent root-cause lifecycle, and bulk status transitions remain separate work.
- Added
@aqa/ingestandaqa ingest junit|sast <file>. JUnit pass/failure/error/skip and Semgrep-compatible SAST results now normalize into a common bounded report with stable fingerprints; CLI writes through the redaction-aware artifact store under.aqa/ingest/. - JUnit rejects
DOCTYPE/ENTITYdeclarations and all inputs are capped at 10 MiB. Ingestion remains evidence-only: it does not auto-verify or close findings. Playwright trace, k6, mutation and provider-specific adapters remain open. - Evidence: ingest package typecheck and 4/4 tests; kit typecheck and 117 tests (115 passed, 2 platform skips); repository lint passed.
- Added
aqa risk discover --method stride [--scope <path>], producing six schema-validated STRIDE baseline risks with one explicit invariant each, mapped to AQA risk categories and tagged with scope. Existing maps are preserved unless--forceis explicit; symlink targets and traversal scopes fail closed. - This is a deterministic baseline, not autonomous source-code truth. FMEA, OWASP import, attack trees, AST/LLM-assisted discovery, and human approval/versioning remain open extensions.
- Evidence: kit typecheck; kit suite 120 tests (118 passed, 2 platform skips); repository lint and diff check passed.
- Added
GET /api/risk-coverage, requiring org/project scope andrisk-map:read. It loads tenant-scoped risks and scenarios plus persisted run oracle events, ignores incomplete scenario observations, and delegates scoring to@aqa/methodology. - Complete API evidence: server suite 108 tests (107 passed, 1 live PostgreSQL EventBus skip); server typecheck and repository lint passed. Admin UI projection and durable event retention/aggregation remain open.
- Added
evaluateSlo()to@aqa/observability: validates event counts and target, computes allowed bad events, remaining budget, burn rate and explicitno_data/within_budget/budget_warning/budget_exhaustedreason codes. - Evidence: observability typecheck and 7/7 tests; repository lint passed. Runtime metric wiring, OTel Collector export and operational dashboards/alerts remain open.
- Added optional
orgto the Run schema and generated JSON Schema, propagated org filtering through Memory/Postgres stores and all scoped run/finding API checks. Scoped reads now fail closed for legacy runs without an org and prevent same-project-slug cross-org leakage. - Evidence: schema suite 59/59, store build, server typecheck, server suite 108 (107 passed, 1 PostgreSQL EventBus skip), repository lint. Worker persistence must populate
Run.orgfrom the authenticated queue scope; unscoped local CLI runs remain intentionally local-only.
- Added validated per-organization/project admission limits for concurrent runs and declared scenario units to both the memory queue and the PostgreSQL queue. Idempotent retries return the existing job before quota evaluation; API callers receive a bounded
429 RESOURCE_QUOTA_EXCEEDEDresponse with no secret or payload echo. - PostgreSQL quota admission now serializes each scoped decision with a transaction-scoped advisory lock before reading active jobs and inserting. The queue still needs durable quota configuration/metrics and runtime kill-switch propagation, but the previous snapshot-plus-insert race is closed.
- Evidence: server typecheck, server suite (109 passed plus one PostgreSQL EventBus platform skip), repository lint. A live two-client contention test is included in
postgres-queue.test.tsand runs in CI with PostgreSQL 16; this Windows workspace has no local PostgreSQL DSN.
- Hardened
@aqa/costwith validated non-negative budgets/token counts, a fail-closedassertCanDispatch()boundary, and an explicit irreversible-in-instancehalt(reason)kill switch. Unknown model pricing remains blocked and no provider call is counted when admission rejects it. - Evidence: cost package typecheck and 8/8 tests; repository lint. Runtime worker integration, distributed halt persistence, provider reconciliation and auditable
budget_exceededevents remain open.
- Updated Claude, Codex, Gemini and Copilot adapters to render the standard directory form
<skills-root>/<skill-name>/SKILL.mdinstead of a flataqa-*.mdfile. Added a contract test that validates the path and frontmatter for every target. - This improves discovery compatibility but does not prove every installed host version discovers the files; host-version installation tests remain a separate complete-journey gap.
- Added a bounded
OtlpHttpSpanExporterwith OTLP/HTTP JSON payloads, explicit flush, failed-batch retry, queue limits, endpoint validation and span-attribute redaction. It is injectable throughfetcher, so tests do not require a Collector. - Evidence: observability typecheck and 9/9 tests; repository lint. Timer/shutdown integration, Collector deployment, Prometheus metric wiring and dashboards remain open.
- Extended
aqa risk discoverwith a deterministic ten-control OWASP baseline (--method owasp) covering access control, crypto, injection, design, configuration, dependencies, authentication, integrity, logging and SSRF. It uses the same schema validation, safe overwrite and scope tags as STRIDE. - Evidence: kit typecheck, risk-discovery tests including the 10-risk OWASP map, and repository lint. FMEA, attack trees, source-aware analysis, LLM-assisted hypotheses and human approval/versioning remain open.
- Added
aqa risk discover --method fmea, generating six bounded failure-mode hypotheses for ambiguous requirements, invalid input, dependency outage, concurrency races, configuration drift and detection gaps. Each has a stable ID, one invariant, framework tag, scope tag and schema validation. - Evidence: kit typecheck, risk-discovery tests including the FMEA map, and repository lint. Attack trees, source-aware analysis, LLM-assisted hypotheses and human approval/versioning remain open.
- Added bounded AND/OR attack-tree contracts to
@aqa/methodology: validation rejects duplicate IDs, malformed leaves, excessive depth and node counts; evaluation consumes only an explicit compromised-leaf set;attackTreeForRisk()creates an invariant-linked skeleton. - Evidence: methodology typecheck and 12/12 tests; repository lint. Persistence/schema integration, graph visualization, source-aware derivation and human-reviewed attack paths remain open.
- Added
MfaPolicy/enforceMfa()to@aqa/auth, optionalUser.mfa_verified, OIDCamrclaim mapping (mfa,otp,webauthn,hwk) and session-manager enforcement before persistence. Policies can target all users or selected roles; missing proof fails closed. - Evidence: auth typecheck and 12 tests (11 passed, one PostgreSQL session skip); repository lint. TOTP/WebAuthn enrollment, recovery codes, admin policy persistence and an external-IdP complete journey remain open.
- Added a tenant-bound SCIM 2.0 provisioner contract to
@aqa/authwith create, replace, patch (active/displayName), deactivate, get and list operations. It validates email/userName, assigns least-privilege viewer by default, rejects cross-tenant reads and keeps persistence behind an injected directory. - Evidence: auth typecheck and 14 tests (13 passed, one PostgreSQL session skip); repository lint. PostgreSQL tenant-aware user schema, bearer-token route exposure, PATCH filter semantics, SAML and complete IdP provisioning journey remain open.
- Extended
StoreProvider.listUsers/upsertUserwith optional org/project scope. MemoryStore namespaces user IDs with the same safe scope key used by other resources; PostgresStore persists the scope columns and queries through the existing scoped record index. The server users projection and OIDC admin snapshot now preserve the authenticated organization/project boundary. - Evidence: store 13/13 tests, store/server typechecks, server suite 109 passed plus one PostgreSQL EventBus skip, repository lint. Live PostgreSQL scoped-user round-trip remains CI evidence; SCIM bearer routes, audit events and legacy-user migration remain open.
- Added
/scim/v2/Usersand/scim/v2/Users/:idGET/POST/PUT/PATCH/DELETE routes. Every route requires an injected dedicated bearer verifier andx-aqa-org; DELETE deactivates rather than destructively erases the directory record. The journey preserves SCIMuserName, roles, active state and tenant isolation. - Complete HTTP evidence: server suite 111 tests (110 passed, one PostgreSQL EventBus skip), including unauthorized access, create, patch, list, cross-org denial and soft-delete; store build/typecheck and repository lint pass. SCIM token rotation/audit events, RFC filter pagination, SAML and live PostgreSQL SCIM round-trip remain open.
- Added provider-neutral commerce contracts for exact split-tender reconciliation across card, gift-card and store-credit instruments, including currency and duplicate-tender guards. Added commit-time promotion validation for currency, expiry and redemption limits.
- Evidence:
@aqa/commercetypecheck and 15/15 tests; repository lint andgit diff --checkpass. Durable atomic redemption, provider settlement, gift-card locking, tax treatment and financial reconciliation remain open.
- Added optional query parameters to the API request contract and wired SCIM
filter,startIndexandcountto a tenant-bound ListResponse. The endpoint now reports total results before pagination and preserves the existing dedicated bearer authorization. - Evidence: server typecheck, 111 tests (110 passed, one PostgreSQL EventBus skip), repository Biome check and
git diff --checkpass. Complex SCIM filter grammar, token rotation/audit events and live PostgreSQL provisioning remain open.
- Ran the workspace gates after the enterprise slices: typecheck passed,
bun testpassed with 490 tests and 4 environment-dependent PostgreSQL skips, and the documentation plus workspace build passed. - At that point the build still reported two known warning classes: the admin SPA emitted a 578.94 kB minified chunk and the CJS CLI reported
import.metacompatibility warnings. The latter was resolved in the follow-up CJS bundle runtime-path slice below; the chunk-size warning remains.
- Removed ESM-only
import.metapath resolution from the CLI run/admin commands. Runtime assets now resolve from the CJS entrypoint or recognized direct command path, with bounded package-relative fallback. - Evidence: kit typecheck and 121/123 tests passed (2 symlink-capability skips); the rebuilt bundle emits no
import.metawarning, contains noimport.metatoken, and completed a real temporary-project journey (init→run, 2 scenarios, 2 findings). The follow-up bundle suite now covers the real admin process health endpoint; the admin SPA chunk remains 578.94 kB.
- Added a process-level test that starts the published CJS bundle, waits for
/api/healthz, validates the JSON response, and terminates the child process in afinallyblock. This closes the prior asset-resolution smoke gap without claiming full authenticated UI coverage. - Evidence: kit typecheck and 122/124 tests passed (2 symlink-capability skips); the real bundled admin health journey passed.
- Added
ScimTokenManagerwith opaque high-entropy issuance, hash-only persistence, tenant-bound constant-time verification, expiry, revoke/rotate and injected audit events. Rejected attempts are classified without recording bearer material. - Evidence: auth typecheck and 14/15 tests (one PostgreSQL session skip), repository Biome check and diff check. Secret-manager wiring, atomic durable rotation, rate limiting and HTTP administration remain deployment work.
- Added bounded ingestion for k6 JSON summaries. p95 latency is preserved as duration evidence; request-error and check rates become explicit failed records, with stable fingerprints and fail-closed malformed-input handling.
- Evidence: ingest typecheck and 6/6 tests; repository Biome check and diff check pass. Threshold-policy evaluation, Locust ingestion and live performance execution remain open.
- Extended the performance ingestion boundary with Locust JSON statistics: request method/name, p95 latency, request/failure counts, stable fingerprints and bounded worker-error warnings. Non-negative numeric validation fails closed.
- Evidence: ingest typecheck and 8/8 tests; repository Biome check and diff check pass. Threshold-policy evaluation and live performance execution remain open.
- Added a provider-neutral SAML login boundary. An injected maintained XML signature/parser adapter feeds exact issuer/audience, required identity, time-window, atomic replay-claim and least-privilege role validation; no hand-written XML crypto is shipped.
- Evidence: auth typecheck and 16/17 tests (one PostgreSQL session skip); repository Biome check and diff check pass. Real IdP metadata/certificate rollover, durable replay storage and external SAML HTTP journey remain open.
- Added
docs/operations/dr-runbook.mdwith approved RPO/RTO inputs, Postgres/WAL and artifact-store backup contract, isolated restore sequence, digest/tenant/queue checks, quarterly drill criteria and explicit infrastructure boundary. Added ADR-052. - Evidence: documentation is present and scoped honestly; no live provider backup/restore was claimed. A real drill with KMS, WAL/PITR, object retention and measured RTO/RPO remains required.
- Added a combined stdout/stderr byte cap to
ContainerSandbox, propagated through the executor, with child termination and explicit fail-closed result when exceeded. This closes an unbounded-memory path not covered by call count or timeout limits. - Evidence: sandbox typecheck and 11/11 tests; repository Biome check and diff check pass. Real OCI fault/output stress and VM-level hostile-tenant isolation remain deployment evidence.
- Added optional OCI digest rendering and fail-closed
requireDigestswitches for server and runner images. Operator docs now state that production must pin digests and retain SBOM/provenance evidence; tag defaults remain development-only. - Evidence: chart/value/template diff validated by repository lint and diff check. Live
helm template/upgrade against a cluster remains an infrastructure gate.
- Workspace typecheck, test and lint completed after SAML, k6, DR, sandbox and Helm changes: 498 tests passed, 4 PostgreSQL integration tests skipped because no DSN was configured, and 0 failures. Helm render was not available locally because the
helmexecutable is not installed; this remains an explicit deployment gate.
- Added the
Risk coverageadmin screen and navigation entry. Mock mode renders explicit covered/partial/stale evidence states; live mode reads the tenant-scoped/api/risk-coverageprojection, exposes drift alerts, and links each row back to the risk editor. - Added a Playwright journey covering navigation, table rendering and representative coverage states. Admin typecheck and the focused Playwright test passed (1/1). Full workspace regression remains the next gate; the page does not claim live authenticated browser coverage without a configured server/identity provider.
- Added numeric measurement retention to k6/Locust ingestion and a separate
evaluatePerformanceThresholdsboundary for p95, failure-rate and check-rate gates. Parsing remains evidence-only; policy now produces deterministic violations without mutating imported records. - Evidence: ingest test 10/10; workspace typecheck, 502 tests passed, 4 PostgreSQL integration tests skipped without DSN, lint and diff-check passed. Live load execution and CI artifact wiring remain open.
- Extended
aqa ingestto accept k6 and Locust JSON summaries, preserving the same bounded/redacted artifact path and normalized report contract used by JUnit/SAST. Added a CLI-boundary test for both frameworks. - Evidence: workspace typecheck, full workspace test gate passed (506 tests, 4 PostgreSQL skips, 0 failures), lint and diff-check passed. Threshold-file CLI flags and live CI gate enforcement remain separate follow-up work.
- Added
aqa ingest k6|locust <file> --threshold-file <policy.json>. The report remains the primary redacted evidence artifact; the numeric policy and structured violations are persisted in a separate threshold artifact. A failed threshold returns exit code 2 while ingestion itself remains recorded, making CI behavior explicit and inspectable. - Evidence: kit suite 124 passed, 2 platform symlink skips; root lint and diff-check pass. CI workflow wiring and live load execution remain open.
- Added a CI step to the existing built CLI smoke job. It feeds a non-secret k6 JSON fixture and policy into the published CJS bundle, asserts the expected gate exit code
2, and checks that the separate threshold artifact was written. This proves CI wiring without pretending to execute a real load generator. - Evidence: workflow diff is syntactically scoped to the existing
e2e-clibuild job; local workspace build passed (admin chunk-size warning remains non-fatal). A GitHub Actions run is required for authoritative hosted-runner evidence.
- Corrected the k6 adapter so
http_req_failed.ratenormalizes tofailure_rateandchecks.ratetocheck_rate; the prior genericratemeasurement could make those explicit policies silently pass. - Evidence: ingest suite 11/11, workspace lint and diff-check pass. The CLI/CI journey now exercises the same normalized measurement contract.
- Added serialized flushes, stoppable bounded auto-flush and
shutdown()drain semantics toOtlpHttpSpanExporter. Failed deliveries remain queued; shutdown fails if the bounded queue cannot drain instead of silently dropping telemetry. - Evidence: observability suite 11/11, workspace lint and diff-check pass. Server/runner boot wiring to a configured Collector and deployed Collector availability remain environment work.
- Added the standard
Bearer <token-id>.<secret>verification helper and wiredScimTokenManagerintoaqa adminas an explicit option. The admin HTTP server now delegates/scim/v2/*routes to the API (the previous static-only path made the tested SCIM routes unreachable from the bundled server). - Evidence: auth 17 passed/1 PostgreSQL skip; kit 125 passed/2 platform skips; the real boot test covers unauthorized and authorized SCIM list requests, with lint/typecheck path green. Durable token storage and rate limiting remain deployment work.
- Added
PostgresScimTokenStorewith serialized schema migration, hash-only records, tenant/index support, expiry cleanup and explicit close. Helm now mapsAQA_SCIM_TOKEN_DSNin all PostgreSQL modes, and the CI Helm assertions cover the new production wiring. - Evidence: auth typecheck and 17 passed/2 environment skips (the new PostgreSQL round-trip is skipped without DSN); lint and diff-check pass. The hosted PostgreSQL job must execute the round-trip before claiming durable production evidence; atomic rotation/rate limiting remain open.
- Added
PostgresSamlReplayGuardwith serialized migration, expiry index and atomic single-claim semantics. The existing SAML boundary can now use a durable replay store without weakening its maintained signature-verifier adapter boundary. - Evidence: auth suite 17 passed/3 PostgreSQL-dependent skips without DSN, typecheck and lint pass. The hosted PostgreSQL job now invokes the combined SCIM/SAML persistence contract; real IdP metadata/certificate rollover and HTTP login wiring remain open.
- Workspace typecheck, full Bun suite and lint completed after durable SCIM/SAML, OTLP and performance changes: 509 passed, 6 PostgreSQL-dependent skips, 0 failures across 51 files.
git diff --checkalso passes. The six skips are now explicitly named: OIDC session, SCIM token, SAML replay, EventBus, queue idempotency and quota concurrency.
- Added
MfaLifecyclewith generated TOTP enrollment material,otpauthURI, confirmation before activation, protected-secret injection, and one-time recovery-code consumption. Raw TOTP secrets and recovery codes are never persisted by the lifecycle; production must provide a KMS/Vault-backed protector and durable store. - Added
PostgresMfaCredentialStorewith idempotent migration and composite tenant/user key; the store persists only protected secret material and recovery-code hashes. - Evidence: auth build/typecheck, lifecycle tests 2/2, PostgreSQL contract 1 skip without DSN, repository lint and diff-check pass. Provider-backed secret protection, rate limiting/audit integration and WebAuthn remain open deployment work.
- Separated probe execution state from oracle assertion state. Missing drivers, transport errors and cleanup failures now produce
execution_status=failed, blockaqa run, and do not create security findings.aqa runrecords bounded execution-error samples inrun_finished; success fixtures inject an explicit probe driver instead of relying on an implicit no-op. - Evidence: runner/kit journey tests 36/36 and pack-scaffold integration updated; full regression pending after this increment. This closes the false-green boundary but does not yet provide browser/SQL/shell drivers for every pack.
- Added a runtime-neutral canonical JSON implementation and a WebCrypto-only
@aqa/compliance/browserverifier. The admin audit viewer now consumes that package boundary instead of carrying a second hash implementation; prefix verification supports animated progress while rejecting tampered, reordered or partial chains. - Evidence: compliance browser/node contract 8/8, full workspace typecheck and lint, admin production build passed. The admin bundle still reports the existing non-fatal chunk-size warning; WORM storage, signed checkpoints and completeness attestations remain separate operational controls.
- Added optional
oracle.probe_idwith Scenario cross-field validation for unique step IDs and existing referenced steps. Runtime evaluation scopes the oracle to that observation and fails closed when the reference is missing; legacy scenarios without the field remain compatible. - Evidence: schema/runner targeted tests 41/41 after the new contracts, typecheck and lint pass. Existing legacy scenarios still use fallback semantics until packs are migrated to explicit references.
- Migrated all first-party scenario packs with oracle steps to explicit
probe_idreferences (API, web UI and LLM-agent packs). The legacy fallback remains only for third-party/older packs and is now visible as a migration concern rather than the default first-party contract. - Evidence: pack YAML remains schema-loadable and the full regression is the authoritative gate after this change.
- Added optional
signing.content_sha256, a deterministic digest over the unsigned canonical manifest plus every regular pack file. Symlinks and unsupported file types fail closed.aqa runverifies this digest after loading a pack and refuses tampered scenario/probe content before execution. - Evidence: pack-scanner/CLI targeted tests 37/37, workspace typecheck and lint pass. Existing packs without
content_sha256remain integrity-unpinned until a signing pipeline emits and verifies the field; Sigstore keyless publisher identity remains open.
- Added real Sigstore bundle verification through the maintained
sigstoreJavaScript client, with explicit certificate identity, OIDC issuer and transparency-log threshold policy. Server pack imports now reject declared bundles when no operator policy is configured and fail closed on malformed or unverifiable bundles. - Evidence: scanner/server targeted tests 112/112, pack-scanner/server builds, workspace lint/typecheck pass. No live Fulcio/Rekor bundle was claimed in local evidence; CI must provide a real signed bundle journey before marking keyless production trust complete.
- Hosted CI exposed a real packaging regression: the static Playwright import made the CLI bundle resolve optional browser internals (
chromium-bidi) even for HTTP-only runs. The driver now loads Playwright dynamically only when a browser journey is instantiated; injected browser factories remain available for deterministic tests. - Evidence: runner build/typecheck, runner tests 38/38, kit build (including CLI bundle) and kit typecheck pass locally;
git diff --checkpasses. Live Chromium installation and a hosted browser journey remain open deployment evidence.
aqa runnow enforcesprofile.budget_minutesat the scenario scheduler boundary. Once the deadline is reached, remaining scenarios are recorded asnot_runwithreason: budget_exceeded, the run emits bounded budget metadata and returnsok: false; partial coverage can no longer greenlight a gate. The clock is injectable only for deterministic embedding/tests and defaults toDate.now.- Evidence: kit build/bundle, typecheck and
run-cmdsuite 30 passed, 1 explicit platform skip. Hard cancellation of an already-running provider/browser request and LLM dollar/token budgets remain separate work.
- Moved finding status-transition validation to the shared schema/store write boundary. No-op and illegal transitions now fail with
409 INVALID_TRANSITIONat the API, preserve the original finding and append no audit event. Memory and PostgreSQL adapters both validate and parse the resulting finding, so callers cannot bypass the API contract through a direct store call. - Evidence: server + store builds, 115 tests passed, repository diff check passed. A full PostgreSQL transition round-trip remains hosted evidence when the CI DSN is available.
- Extended the shared run-state derivation so
budget_exceeded: trueis preserved as the terminalbudget_exceededstate in reports and admin projections, while execution/replay/canonical artifact errors remain fail-closed. This prevents a governed timeout from being flattened into an indistinguishable generic failure. - Evidence: schemas build and validator suite 35 passed; generated schema artifacts remain valid and
git diff --checkpasses.
aqa runnow loads.aqa/risk-map.yaml, ingests validated risk catalogs declared by selected packs, rejects scenarios with unresolvedrisk_refsas coverage errors, and passes the resolved risk into the runner. Findings deriverisk_idand severity from that declaration instead of hard-codinghigh.- Evidence: runner + kit build/bundle/typecheck, 47 targeted tests passed with one explicit symlink-capability skip. The default scaffold remains intentionally minimal; projects must add risks for custom scenarios, while first-party packs provide their own risk catalog files.
- Applied the same realpath containment rule used for scenario files to pack-declared risk catalogs, rejecting symlinks that escape the pack root before parsing or execution.
- Evidence: kit build/bundle and
run-cmdsuite 30 passed, with the existing explicit platform symlink skip.
- Hardened the real HTTP probe boundary: configured origins are normalized and validated, credential-bearing base/target URLs are rejected, redirects are never followed automatically, and redirect targets are checked against the same allowlist before being reported. Private origins remain usable only when explicitly allowlisted.
- Evidence: runner build and 18 tests passed, including credential and off-origin redirect denial. DNS pinning/rebinding defense and browser network interception remain separate deployment controls.
- Playwright contexts now install a route-level network policy before the first page is created. Every HTTP(S) request is checked for credentials and an allowlisted origin; non-HTTP schemes and off-origin requests are aborted. This covers redirects and subresources that structured action URL validation cannot see.
- Evidence: runner build and 18 tests passed, including an injected off-origin browser request that is aborted. Real Chromium/provider redirect evidence remains a hosted deployment journey.
- Added
cancelledas a terminal queue state in Memory and PostgreSQL adapters.POST /api/runs/:id/cancelrequires tenant scope, records a bounded reason, clears the lease token and publishesrun.cancelled; late worker ACKs are fenced and cross-tenant cancellation is indistinguishable from not-found. - Evidence: server build, 112 tests passed across API and queue suites. This is cooperative cancellation state: a worker already executing must observe the cancelled job and abort its driver; no false claim of process interruption is made.
- Extended the runner driver contract with an optional
AbortSignal.runScenarioforwards the worker/orchestrator signal to the selected probe runner, and the HTTP driver propagates it tofetchwhile removing its listener on completion. An aborted request is an execution failure, therefore it cannot produce a security finding. - Evidence: runner build, runner typecheck, 19 tests passed, and
git diff --checkpassed. Shell, SQL, PostgreSQL and Playwright drivers still need signal-aware cancellation; there is still no executable server worker that observes queue cancellation and interrupts an in-flight job.
- Added
RunnerWorker, a provider-neutral worker loop that dequeues a leased job, polls the queue for cancellation, propagates anAbortSignalto the handler, ACKs only successful non-cancelled work, and records bounded single-line failure reasons. Addedget(id)to both queue adapters so cancellation observation works with memory and PostgreSQL implementations. - Evidence: server build, 120 tests passed, including in-flight cancellation and bounded failure tests, and
git diff --checkpassed. The worker handler is intentionally injected: real payload-to-aqa runorchestration, lease heartbeats for long jobs, runner authentication/identity, and live PostgreSQL worker evidence remain open.
- Added
renew(id, lease_token)to memory and PostgreSQL queues.RunnerWorkerrenews the lease while a handler is running and aborts withlease_lostif the token is fenced or the job leavesin_flight; it never converts a lost lease into an ACK or a false failure write. - Evidence: server build, 122 tests passed, including current-token renewal and lease-loss fencing, and
git diff --checkpassed. Live PostgreSQL renewal/reconnect evidence and production heartbeat metrics remain open.
- The controlled shell driver now accepts the runner
AbortSignal, rejects an already-cancelled probe before spawn, kills the child process on cancellation, removes its listener, and reports cancellation as execution error. This preserves the no-finding rule for interrupted execution. - Evidence: runner build, 20 tests passed, including an actual long-running child cancellation, and
git diff --checkpassed. SQL cancellation depends on the injected adapter contract; Playwright/provider-specific cancellation and live sandbox process-tree cleanup remain open.
- Added
makeRunJobHandlerin@aqa/kit. It maps a validated queue payload to the fixed operator-configured project root and the canonicalrunRunimplementation, forwards the workerAbortSignal, and rejects invalid profile/seed types.RunOptionsnow stops scheduling after cancellation and returnsok: falsewith explicit cancellation evidence. - Evidence: kit build/bundle, typecheck, 56 filtered tests passed, and
git diff --checkpassed. A live server-to-worker HTTP process journey, artifact publication through a remote store, runner identity enforcement and multi-process PostgreSQL evidence remain open.
- Added
makeKitWorker, the official composition ofRunnerWorkerand the canonical kit handler. The integration journey now enqueues a real job, dequeues it through the worker, executes a local HTTP probe throughrunRun, writes the real.aqa/runsartifacts, and ACKs the queue job. - Evidence: kit build/bundle, typecheck, targeted journey tests 3/3 passed, and
git diff --checkpassed. This is in-process MemoryQueue evidence; a separate-process PostgreSQL/remote-artifact/runner-auth journey is still required for production sign-off.
runAdminnow refuses to boot withAQA_QUEUE_DSNunless a dedicatedrunnerAuthorizecallback orAQA_RUNNER_TOKENis configured. The environment-token fallback accepts onlyBearer <token>and compares SHA-256 digests withtimingSafeEqual; the raw token is never logged or persisted.- Evidence: kit build/bundle, typecheck, 14 admin tests passed, and
git diff --checkpassed. OIDC/mTLS/short-lived runner identity and rotation remain the production-grade credential path; the static token is a bounded bootstrap fallback.
- Browser probe runners now accept the worker signal, reject pre-cancelled probes, close the active page on abort, re-check cancellation after browser awaits, and remove listeners in
finally. Cancellation is reported as execution failure and cannot become an oracle finding. - Evidence: runner build/typecheck, 21 tests passed, including active-page closure, and
git diff --checkpassed. Native provider cancellation, browser context/process cleanup and hosted Chromium evidence remain open.
- SQL runner adapters now receive the worker
AbortSignal; pre-cancelled queries are rejected and cancellation is rechecked after adapter completion. PostgreSQL preserves its bounded read-only transaction and statement timeout, then reports cancellation if the signal arrived; the underlying driver has no claimed native abort primitive. - Evidence: runner build/typecheck, 22 tests passed, including signal propagation to an injected adapter, and
git diff --checkpassed. Native PostgreSQL query cancellation, live DSN evidence and resource cleanup remain open.
- Added
BudgetedLlmAdapter: it performs pre-dispatch admission using a configurable token estimate, charges authoritative provider usage after the call, exposes a budget snapshot, and blocks subsequent dispatches once the USD budget is exhausted. Unknown model pricing remains fail-closed throughBudgetTracker. - Evidence: LLM adapter build and 15 tests passed, including no-dispatch admission denial and post-call exhaustion. Default character-based estimation is conservative scaffolding; production should inject versioned tokenizer/pricing data and persist aggregate usage across workers/projects.
- Added
BudgetLedgerwith atomic Memory and PostgreSQL implementations. Reservations are keyed by org/project/run scope, account for in-flight estimated spend, and settle idempotently against actual usage.BudgetedLlmAdapteroptionally uses the ledger, preventing concurrent workers from dispatching past a shared budget. - Evidence: cost build and 9 tests passed, LLM adapter build and 16 tests passed, including concurrent shared-ledger admission. PostgreSQL live concurrency/settlement evidence, durable budget configuration APIs, pricing version distribution and reconciliation remain open.
- Hardening follow-up: PostgreSQL ledger migrations now use an advisory lock and existing keys reject changed budget limits, preventing replica configuration drift.
- Added TTLs to budget reservations and
reapExpired()to Memory/PostgreSQL ledgers. PostgreSQL reclaims expired rows withFOR UPDATE SKIP LOCKED, releases the reserved estimate and marks the reservation settled; repeated cleanup is safe. - Evidence: cost build/typecheck and 10 tests passed, including simulated worker crash recovery. Production still needs a scheduled reaper/metric/alert and a live PostgreSQL multi-client recovery journey.
- Added
BudgetReaper, a validated long-lived scheduler aroundreapExpired(). It prevents overlapping ticks, exposes a deterministicrunOnce()for job schedulers/tests, supports idempotent start/stop and routes failures through an injected callback. - Evidence: cost build/typecheck and 11 tests passed, including scheduled lifecycle behavior. Kubernetes CronJob/systemd wiring, metrics and alerts remain deployment work.
- Added the
aqa-budget-reaperone-shot server entrypoint and an optional HelmCronJobwithconcurrencyPolicy: Forbid, bounded retry history, non-root/read-only security context and DSN-from-Secret wiring. Enabling the chart without a PostgreSQL Secret fails at template time; the binary fails closed withoutAQA_BUDGET_LEDGER_DSN. - Evidence: server build, typecheck, 122 tests passed, CLI missing-DSN negative check, and CI Helm lint/template assertions updated. Helm is not installed in this Windows workspace, so rendered-chart evidence remains CI-authoritative.
- Added
PricingCatalogparsing with schema/version/effective timestamp validation, deterministic model ordering and SHA-256 identity.BudgetTrackercan consume the catalog and exposespricing_version/pricing_sha256in every snapshot, making cost evidence attributable to a precise listino. - Evidence: cost build/typecheck and 12 tests passed, including digest mismatch rejection. Signed catalog distribution, admin configuration/rotation and durable usage records carrying the catalog identity remain open.
- Extended the shared budget settlement contract so each completed reservation can persist provider model, authoritative input/output token counts, actual USD and the applied pricing catalog version/hash. PostgreSQL migration is additive and safe for existing reservation rows; orphan reaping remains metadata-free by design because no provider call completed.
- Evidence: cost build/typecheck with 12 tests passed and LLM adapter build with 17 tests passed, including catalog identity propagation at settlement. Live PostgreSQL schema migration/concurrency and an operator-facing usage query remain deployment/integration evidence.
- Added the shared
RunRequestschema and applied it before queue insertion. The public API now accepts only the supportedprofileand bounded deterministicseed; tenant org/project are server-derived and arbitrary fields such as filesystem roots are rejected before durable queue persistence. - Evidence: schemas/server build, server 123 tests passed, including an explicit rejection of an unsafe
rootfield. The worker still resolves the project root from operator configuration, never from the request payload.
- Tightened pricing catalog validation to require bounded catalog versions, non-empty model names and canonical ISO UTC timestamps (
Z), preventing timezone/date-only ambiguity in cost evidence. - Evidence: cost build/typecheck and 12 tests passed, including rejection of a date-only effective timestamp.
- Runner authentication can now return explicit
org/projectscopes. Memory and PostgreSQL dequeue apply those scopes before leasing; ACK/fail re-read the job and hide cross-tenant attempts as not-found. Boolean authorizers remain compatible as an intentionally unscoped bootstrap path. - Evidence: server build/typecheck and 125 tests passed, including scoped dequeue and cross-tenant ACK denial. Live PostgreSQL scoped dequeue and production mTLS/OIDC claim issuance remain deployment evidence.
- Added the real
aqa workerentrypoint: it requires a PostgreSQL queue DSN, operator-owned project root and explicit runner scopes, composesRunnerWorkerwith the canonical kit handler, handles SIGTERM/SIGINT and closes the queue cleanly. Helm can now enable the worker StatefulSet with Secret-backed DSN and scoped environment configuration; default scaffold remains disabled until configured. - Evidence: server/kit builds and worker configuration tests 3/3 passed. Helm is not installed in this Windows workspace; CI render/lint is authoritative for the chart path.
- Rejected malformed
org/scope entries; onlyorg/projectand explicitorg/*are accepted, preventing an incomplete deployment value from becoming an unintended organization-wide permission. - Evidence: kit build/typecheck and worker configuration tests 3/3 passed.
- Added
CommerceToolPolicyfor EC-10: read tools require an allowlist and same-tenant/customer target; write/financial tools require a single-use human approval bound to call ID, tenant, customer, cart revision, exact minor-unit total/currency and expiry. Prompt text or agent output cannot self-approve a mutation. - Evidence: commerce build/typecheck and 21 tests passed, including cross-tenant denial, missing approval, stale/expired approval and replay rejection. Durable approval storage, provider atomicity and a live merchant tool gateway remain required integration evidence.
- Added
CommerceApprovalLedgerwith in-memory and atomic PostgreSQL implementations.CommerceToolPolicy.authorizeAsync()claims an approval exactly once; when a durable ledger is configured, synchronous authorization fails closed instead of using process-local replay state. - Evidence: commerce build/typecheck and 22 tests passed, including duplicate approval consumption. Live PostgreSQL concurrency, approval issuance/audit and atomic merchant mutation still remain deployment evidence.
- Added
WebAuthnLifecyclewith HTTPS-origin validation, bounded random one-time challenges, user/origin/RP/credential binding, injected signature verification, monotonic-counter clone detection and explicit counterless-authenticator support. Exported the boundary from@aqa/authand documented the production integration limits in ADR-117. - Evidence: auth build/typecheck and 30 tests passed, including replay, wrong-user/origin/credential, expiry, signature failure, counter rollback and counterless credentials. Durable challenge/credential stores, a maintained cryptographic verifier and a real browser/provider passkey journey remain open.
- Evidence: repository
typecheck, 591 tests passed / 0 failed, workspace build and Biome lint all passed locally. The local run has noAQA_TEST_POSTGRES_DSN, so PostgreSQL-dependent tests remain explicit non-evidence skips; GitHub CI run35205836642has PostgreSQL integration and Bun tests passed, while Node 22 and Build were still running at ledger update time. No Copilot review was requested or awaited per user instruction.
- Added
RunnerJwtAuthorizerwith an RS256-only trust boundary, exact issuer/audience, required expiry, optional bounded not-before, runner subject and strictorg/projector explicitorg/*scopes.aqa adminnow wires the verifier fromAQA_RUNNER_JWT_PUBLIC_KEY,AQA_RUNNER_JWT_ISSUERandAQA_RUNNER_JWT_AUDIENCE, rejecting partial configuration; staticAQA_RUNNER_TOKENremains an explicit bootstrap fallback. - Evidence: auth build/typecheck and 33 tests passed, kit typecheck and its 136 passed / 2 platform skips suite passed, including partial JWT environment rejection. Live IdP token issuance/rotation, mTLS and multi-process server-to-worker JWT journey remain deployment evidence.
- Added
CommerceMutationGatefor agentic commerce writes. It composes durable approval authorization with a provider executor and preserves the distinction betweencommitted,not_committedandunknown; thrown/ambiguous provider outcomes never become a successful mutation or a safe retry. The executor contract requires merchant-side atomic revision/total/idempotency checks and reconciliation. - Evidence: commerce build/typecheck and 23 tests passed, including the ambiguous timeout path and approval replay denial. Real payment/merchant transaction atomicity, provider idempotency retention and fault-injected sandbox journeys remain deployment evidence.
- Added Ed25519 signing and trust-map verification for canonical versioned pricing catalogs. Budget configuration can now reject unknown operator keys, algorithm changes and catalog tampering before admission; the catalog digest/version remain the reproducibility identity.
- Evidence: cost build/typecheck and 13 tests passed, including trusted-key verification, tampering and unknown-signer rejection. KMS/Vault key lifecycle, admin distribution/rotation and provider invoice reconciliation remain deployment evidence.
- Evidence: repository typecheck, 599 tests passed / 0 failed, and Biome lint passed locally after JWT runner, commerce mutation, signed pricing catalog, durable WebAuthn and CORS changes. PostgreSQL-dependent branches remain explicit skips without
AQA_TEST_POSTGRES_DSN; hosted CI remains authoritative for those live contracts and for Node 22/OCI/browser execution.
- Added multi-replica WebAuthn persistence: advisory-locked migrations, atomic one-time challenge consumption, user-bound credential reads and conditional monotonic-counter updates. Credential registration is explicit and does not persist private keys.
- Evidence: auth build/typecheck and 34 tests passed, including a concurrent two-client counter contract (skipped without
AQA_TEST_POSTGRES_DSN). The hosted PostgreSQL run must execute this new contract; real browser/provider ceremony and verifier integration remain open.
- Removed the wildcard CORS default from
aqa admin. Cross-origin access is now opt-in through an explicit origin allowlist; allowed origins receive exact credentialed CORS headers, while disallowed preflights and state-changing requests return403before routing. - Evidence: kit build and 137 tests passed / 2 platform skips, including a real HTTP allowlist/preflight/mutation contract. CSRF token strategy, production reverse-proxy headers and live browser deployment remain separate controls.
- Fixed the CLI smoke fixture so its scenario risk/invariant references are backed by the generated project risk map; the smoke now validates the same resolution path as a real project.
- Fixed PAN redaction false positives for structured run/checkpoint identifiers whose timestamp digits happen to satisfy Luhn. Conventional contiguous/grouped card numbers remain redacted, while immutable artifact keys remain byte-stable.
- Evidence:
@aqa/observability15 passed,@aqa/kit140 tests / 138 passed / 2 platform skips, andbun run test:e2e-cli5/5 passed locally. The fix is ready for the next hosted CI run; production browser/provider, live merchant/payment atomicity and durable deployment evidence remain open by design.
- Corrected the PostgreSQL store contract to model concurrent finding decisions as serialized optimistic conflicts: one or both transitions may commit according to lock order, while an invalid terminal-state transition is an explicit rejected operation rather than an unhandled test failure.
- Evidence: local workspace regression remains green; the hosted PostgreSQL contract had exposed this order-dependent assertion and the fix is queued for CI revalidation.
- Added
GET /openapi.json, generated from the concrete server route table so methods, paths, operation IDs, bearer security and permission metadata cannot silently drift from the running API. Added server route-count coverage and a real admin HTTP assertion for the published document. - Evidence: server tests 124 passed, kit tests 140 total / 138 passed / 2 expected platform skips, repository typecheck, Biome lint and diff checks passed locally. Generic transport schemas remain intentionally provisional; domain payloads must be connected to the versioned schemas package before declaring SDK-generation readiness.
- Next: wire the admin SPA to the stream and add reconnect/cursor semantics for multi-replica gaps.
- Implemented
GET /api/events/streamin the admin Node adapter. It requires authenticatedruns:read, an org scope, optional project scope, server-side event filtering, SSE reconnect hints, heartbeat comments, and cleanup on client disconnect. The framework-neutral route table exposes the same permission metadata and returns an explicit 501 in adapters without stream support. - Evidence: real HTTP admin test passed with tenant-positive and tenant-negative events, initial SSE framing and reader cancellation; kit suite 141 total / 139 passed / 2 expected platform skips, typecheck, Biome lint and diff checks passed locally.
- Remaining: validate the browser EventSource journey through a deployed reverse proxy and add replay/cursor semantics for reconnect gaps in multi-replica deployments.
- Replaced the admin's SSE-labelled simulation gap with a real
EventSourceconnection whenVITE_AQA_SERVER_URLis configured. The UI exposes connecting/connected/reconnecting state and the last received event, while durable API reads remain authoritative. - Upgraded the ecosystem stack to use
MemoryEventBus, a real SSE adapter and a causal post-subscription fixture event. The complete browser journey now proves the stream is opened andrun.requestedis rendered by the SPA. - Evidence: ecosystem Playwright 3/3 passed, admin typecheck/build, repository Biome lint and diff checks passed. Remaining: durable cursor/replay after reconnect and reverse-proxy deployment evidence.
- Next: add bounded replay/cursor semantics to the event bus and recovery refetches on stream reconnect.
- Added an AsyncAPI 3.0 document generated from the supported live event
registry and exposed it from
GET /asyncapi.json. It documents the tenant-scopedBusEventenvelope,runs:readpermission, and the current run/finding notification types for SSE and future WebSocket adapters. - Evidence: server contract suite passed with the AsyncAPI operation assertion, kit admin HTTP test verifies content type and event operation, repository typecheck/lint/diff checks passed. Durable replay/cursor semantics remain the next eventing increment.
- Upgraded the generated OpenAPI contract from generic request/response objects
to versioned
@aqa/schemasreferences for runs, run requests, findings, profiles, scenarios, risk maps, projects, notifications, agents and SSO configuration, with typed list/detail envelopes where the API already has a stable shape. - Specialized endpoints that do not yet have a promoted domain envelope remain explicitly generic; this is a measured contract gap, not a false claim of complete SDK-generation readiness. Server tests 125 passed, typecheck, lint and diff checks passed locally.
- Added reconnect lifecycle handling to the admin
EventSource: a recovered connection dispatchesaqa:live-reconnected, and Runs/Findings refetch their tenant-scoped authoritative API projections. Received events dispatch the low-latencyaqa:live-eventrefresh signal as well. - Evidence: admin typecheck/build passed, ecosystem Playwright 3/3 passed, repository lint and diff checks passed. Server-side durable cursors/replay and reverse-proxy fault-injection remain the next eventing evidence gap.
- Added optional provider-neutral event replay with tenant/project scope and
bounded limits.
MemoryEventBusretains a deterministic 1,000-event history;PostgresEventBuspersists events beforeNOTIFYinaqa_live_eventsand replays after a monotonic cursor. - The SSE adapter subscribes before replay, deduplicates event IDs, emits
stream.gapfor expired cursors or replay failures, and keeps the existing authoritative reconnect refetch fallback. - Evidence: server suite 126 passed, kit suite 139 passed / 2 expected
platform skips, kit/server typecheck, repository lint and diff checks pass
locally. PostgreSQL persistence/replay remains skipped locally without
AQA_TEST_POSTGRES_DSN; CI/deployment evidence is still required. - Next: add outbound webhook delivery with retry/DLQ semantics and define PostgreSQL event-log retention/pruning before production rollout.
- Added
@aqa/integrationswith a provider-neutral delivery state machine: stable delivery IDs, exact-body HMAC-SHA256 signing, bounded exponential retry withRetry-After, per-integration rate limiting and explicit DLQ after five attempts. - Evidence: package tests 3 passed, package typecheck, repository lint and diff checks pass locally. The implementation uses an injectable transport and memory queue; no external vendor or secret was contacted.
- Remaining before production integrations: secret manager integration, redacted audit/metrics, destination allowlisting and real provider journeys.
- Added
PostgresWebhookQueuewithaqa_webhook_deliveries, atomicFOR UPDATE SKIP LOCKEDclaims, lease fencing, retry/DLQ state and explicitredrive(id). The queue stores onlysecret_ref; workers resolve the secret through an injected resolver before signing the exact body. - Added an optional PostgreSQL integration contract that runs when
AQA_TEST_POSTGRES_DSNis configured. Local evidence: package typecheck and three deterministic tests pass; the durable test is intentionally skipped without PostgreSQL credentials. - Remaining: wire a secret-manager implementation and authenticated admin endpoint/metrics, enforce transport-level DNS/private-IP protections, and prove a real provider journey. Durable origin allowlisting is now enforced.
- Added deterministic payload renderers for Slack, Teams Adaptive Cards, Jira issue creation and PagerDuty Events API. Payloads carry event metadata but no routing key, secret or credential; delivery remains owned by the signed, durable webhook queue.
- Evidence: integrations suite 5 pass locally, package typecheck/lint and repository diff checks pass. No external provider was contacted.
- Remaining: secret-manager implementation, audit/metrics observer, transport DNS/private-IP enforcement and real provider journeys.
- Added
HttpWebhookTransport: send-time HTTPS allowlist validation, no redirects, bounded timeout, bounded response handling andRetry-Afterparsing. It is injectable and never contacts a vendor in unit tests. - Evidence: integrations suite 6 pass, package typecheck/lint and diff checks pass locally. DNS rebinding/private-IP protection remains explicitly delegated to a connection-aware egress proxy/runtime.
- Remaining: wire host metrics/log adapters, secret manager and production egress policy, then prove real provider journeys.
- Closed the documented ADR-020 ecommerce gap by adding versioned
FulfillmentSnapshotandReturnRequestSnapshotcontracts plus shared invariants for order quantity, duplicate SKU, currency, tracking and shipped/delivered evidence. - Evidence:
@aqa/commercesuite 24 passed / 0 failed, typecheck, lint and diff checks pass locally. This proves the domain contract only; real WMS, carrier, RMA approval and settlement journeys remain adapter evidence. - Next ecommerce gaps: subscriptions, cancellations/chargebacks, loyalty and provider-backed fulfillment journeys, if required by the target merchant.
- Added versioned
SubscriptionSnapshotandChargebackSnapshotcontracts with invariants for billing period order, cancellation semantics, exact payment linkage, currency/amount bounds and dispute evidence deadlines. - Evidence:
@aqa/commercesuite 25 passed / 0 failed, typecheck, lint and diff checks pass locally. No billing or dispute provider was contacted. - Remaining: provider-backed subscription/dunning/dispute journeys and loyalty/cancellation adapters.
- Added
LoyaltyAccountSnapshotandLoyaltyTransactionSnapshotwith exact balance reconciliation, duplicate transaction detection, sign invariants and negative-balance protection. - Evidence:
@aqa/commercesuite 26 passed / 0 failed, typecheck, lint and diff checks pass locally. Provider redemption and durable ledger journeys remain adapter evidence. - Remaining: real billing/dispute/loyalty providers, cancellation workflows and complete ecommerce journey coverage.
- Added
CancellationSnapshotwith explicit decision states, decision timing, exact order linkage and compensatingrefund_idevidence for paid orders. - Evidence:
@aqa/commercesuite 27 passed / 0 failed, typecheck, lint and diff checks pass locally. Provider void/refund reconciliation remains adapter evidence. - Remaining: provider-backed billing, disputes, loyalty, fulfillment and full external ecommerce journeys.
- Added an optional
PostgresWebhookQueueobserver for attempt/outcome events. It exposes only stable delivery metadata and HTTP status, never URL, payload, secret or secret reference; observer failures are isolated from delivery. - Evidence: integrations suite 5 pass, package typecheck/lint and diff
checks pass locally. PostgreSQL observer assertion runs with the existing
AQA_TEST_POSTGRES_DSNcontract in CI. - Remaining: wire host metrics/log adapters, enforce transport DNS/private-IP protections and prove real provider journeys.
- Added
RunnerQueueLike.reapExpired()to reclaim orphaned PostgreSQL leases independently of worker availability. Expired jobs below the retry limit are requeued; jobs at the limit become terminally failed; both paths clear the fencing token and return bounded counts only. - Added
aqa-runner-reaperplus an optional Helm CronJob with Secret-backed DSN,Forbidconcurrency, non-root and read-only filesystem settings. ADR-152 records the retry/idempotency boundary: queue recovery cannot compensate an already-executed external side effect. - Evidence pending: local server typecheck/tests and Helm rendering; the live
PostgreSQL crash/retry journey requires
AQA_TEST_POSTGRES_DSN.
- Extended the chart NetworkPolicy so in-cluster runners can reach the
PostgreSQL subchart on TCP/5432 and the lease-reaper CronJob has a dedicated
database-only egress policy. Managed PostgreSQL remains an operator input via
networkPolicy.runnerExtraEgressCidrs; a DSN Secret cannot safely imply a Kubernetes network identity. - Evidence: chart CI renders/lints the production-shaped deployment; live managed-Postgres connectivity remains deployment-specific.
- Wired the existing bounded
MetricsRegistryintoaqa adminthrough an injectedGET /metricsendpoint. The endpoint is opt-in, emits only the registry's Prometheus-safe text, and requiresmetricsAuthorizebefore an off-loopback bind is allowed. Added ADR-153 and a real HTTP admin test. - Evidence: kit typecheck, Biome and admin journey suite 140 passed / 2 platform skips. Prometheus deployment, alert rules and host metric instrumentation remain operator/application integration work.
- Upgraded
examples/laravel-appfrom the unlocked Laravel 11 constraint to Laravel 12.69.2 and committed the Composer lockfile. This removes the two runtime Dependabot advisories without suppressing Composer security policy. - Evidence:
composer audit --lockedreports no security vulnerability advisories. The docs-site npm audit is also currently clean; GitHub's older default-branch alerts may take time to refresh.
- Added optional run priority
-10..10to the request/schema/API boundary and queue contracts. Memory and PostgreSQL queues lease higher priorities first, preserve FIFO ties, and PostgreSQL persists the value through an additive migration. ADR-155 documents that fairness/starvation SLOs are intentionally not inferred from priority alone. - Evidence: server suite 131 passed / 0 failed, server typecheck/Biome and
generated JSON Schema pass locally. PostgreSQL ordering remains a hosted
contract when
AQA_TEST_POSTGRES_DSNis available.
- Hosted CI exposed a real multi-client migration race: two
PostgresApiIdempotencyStoreinstances could concurrently create the table type despiteIF NOT EXISTS, yielding a duplicatepg_typefailure. Serialized table/index bootstrap now uses a PostgreSQL advisory lock with a guaranteed unlock. ADR-156 records the production-relevant fix. - Evidence: failure reproduced from CI logs; local typecheck/tests remain green. The next hosted PostgreSQL run is required before marking the regression closed.
- Extended the race fix to the PostgreSQL runner queue and LLM budget ledger.
All three adapters now run the complete DDL bootstrap and
pg_advisory_xact_lockon the same transaction client. This closes the pooled-session hazard where a session lock could be acquired, DDL routed to another connection, or unlock issued against the wrong session. - Evidence: code-level lock audit has no remaining session-scoped migration locks in the durable adapters. Local typecheck, Biome and package tests are still required; hosted PostgreSQL CI remains the authoritative concurrency proof.
- PR #141 exposed a real gap in the first durable worker journey: the live PostgreSQL dequeue predicate did not reliably match scoped JSONB payloads, and the CI database also proved that an unscoped worker could consume unrelated residual jobs. The journey now uses unique tenant/project values, passes the same RunnerScope to the worker, uses typed denormalized scope columns with migration backfill and an index, and the queue contract has an exact scoped-dequeue regression.
- Evidence: server typecheck, Biome and 144 local server tests pass. The hosted PostgreSQL rerun is still required to prove the live queue-to-kit journey; PR #141 is intentionally not mergeable while that gate is red.
- Next: push the typed predicate fix, rerun the full technical CI, then merge only after PostgreSQL proves enqueue → scoped dequeue → runRun → artifacts → ACK.
- PR #141 merged to main as 718993c after CI run 35322790731 passed typecheck/lint, Helm, Bun, Node 22, build, PostgreSQL persistence, OCI sandbox, CLI E2E and Playwright admin E2E.
- The durable journey is now proven through scoped PostgreSQL dequeue, makeKitWorker, the real runRun boundary, persisted events.jsonl and findings.jsonl, and fenced ACK. The PostgreSQL adapter now stores typed scope_org/scope_project columns, backfills legacy rows and indexes the ready scope path.
- Next macro task: prove deployment-grade external boundaries still open in the roadmap — separate runner processes with authenticated identity, remote artifact storage/retention, OIDC provider lifecycle, KMS/WORM/PITR restore, OTel/Prometheus deployment evidence, and real commerce provider reconciliation journeys.
- Added a provider-backed, SELECT-only PostgreSQL recovery observer with DSN connection lifecycle, replay LSN/timestamp validation, read-only target assertion and bounded redacted output. Unit coverage includes valid state, writable/non-recovery rejection and malformed provider output.
- This proves the recovered PostgreSQL target's observed state, not the cloud backup provider's PITR execution, object restore or KMS/WORM controls. Those remain separate evidence requirements.
- Next: run store gates, document the operator wiring, then add an end-to-end hosted PostgreSQL recovery observation job if the CI environment can expose a disposable recovery target.
- PR #162 merged as
22f26f6after the full technical matrix passed, including Playwright admin E2E, CLI E2E, PostgreSQL/S3/OCI integrations, telemetry, build, Bun and Node unit suites. - The artifact adapter now requests and reads back AES256, KMS or DSSE encryption metadata, failing closed on provider mismatch. Bucket-level KMS policy and WORM remain deployment evidence requirements.
- Closed a concrete distributed-worker gap: runner JWT
runner_idis now propagated from API authorization into queue leases. Renew, ACK and fail are fenced by both lease token and authenticated runner identity in memory and PostgreSQL; the worker supportsAQA_RUNNER_IDfor direct queue operation. - Added API and queue regressions proving a different runner in the same org/project cannot mutate the lease. Existing hosted PostgreSQL CI remains required before declaring the durable migration path promoted.
- Closed the default CLI wiring gap for the drivers that already had bounded
implementations:
runRunnow composes HTTP, SQL/PostgreSQL, Playwright and controlled shell drivers through an explicit host-ownedprobeDriversboundary, derives capability preflight, and closes lifecycle-aware drivers. - The CLI exposes opt-in environment configuration for PostgreSQL DSN, Playwright origin policy and shell executable allowlists. Packs cannot enable these capabilities; security/release-gate profiles retain sandbox precedence for shell execution.
- Evidence: kit typecheck, runner build, 175 kit tests passed / 0 failed; new real orchestration journeys cover injected read-only SQL and direct argv shell execution. A Windows path-separator regression was found and fixed in the shell allowlist comparison.
- Next: complete provider-backed production evidence (PITR/KMS/IdP/PSP/tax/ carrier/WMS) and run the protected evidence workflow in a configured GitHub Environment.
- Added
reconcileStripeRefunds()to the Stripe boundary. It reads back the PaymentIntent and bounded refund ledger, verifies provider capture state, currency and captured/refunded totals, and fails closed on incomplete pagination, non-successful refunds or drift. Evidence: commerce suite 47 passed / 0 failed locally; ADR-238 and package README updated. - This closes only provider payment/refund reconciliation. Disputes, payouts, taxes, fulfillment and durable merchant transaction joins remain separate production evidence requirements.
- Independent WORM/checkpoint publication now fails closed if the external artifact reference's SHA-256 or byte count differs from the canonical checkpoint. Added regression coverage for a transforming/drifting store and ADR-237. This proves cross-store content identity, not provider Object Lock or IAM configuration.
- Added
listDisputes()andreconcileStripeDisputes()to the Stripe boundary. The implementation verifies exact PaymentIntent linkage, bounded pagination, known dispute lifecycle states, evidence deadlines, currency and total exposure. Commerce suite now passes 49/49 locally; ADR-239 and the package README document the boundary. - This proves provider-observed dispute exposure only. Won/open disputes are not treated as settled chargebacks; payout timing, fees, representment and durable merchant joins remain separate production evidence requirements.
- Added
retrievePayout()andreconcileStripePayout()to read back the payout plus its linked balance transaction. The boundary preserves signed provider amounts and verifies payout amount, fee, net equation, source/type, currency and optional status. Commerce suite now passes 51/51 locally; ADR-240 and the package README document the production boundary. - This proves provider payout consistency only. It does not prove bank arrival, order inclusion, fee allocation or a durable merchant payment/order-to-payout join.
- Extended payout reconciliation with bounded
balance_transactions?payout=...read-back. When supplied, every merchant-persisted provider source ID must appear in the payout constituent ledger; missing sources, duplicates and incomplete pagination fail closed. Commerce suite now passes 53/53 locally; ADR-241 and package README document the explicit join boundary. - This proves provider-source inclusion only. Amount equality cannot fabricate an order join; bank arrival, tax/fee allocation and merchant durable transaction persistence remain separate evidence requirements.
- Propagated the explicit
AQA_PROBE_*host policy throughrunnerConfigFromEnv→makeKitWorker→makeRunJobHandler→runRun. Queue payloads cannot select credentials, origins or shell allowlists, and malformed opt-in policy prevents worker startup before leasing jobs. - Evidence: kit typecheck, Biome and 179 passed / 0 failed / 2 platform skips in the full kit suite; worker configuration regressions cover both policy propagation and fail-closed shell enablement. ADR-252 records the distributed boundary.
- Next: close the remaining provider-backed production evidence and execute the protected GitHub Environment gate with real operator-owned evidence.
- Added
runner.worker.probeDriversto the Helm chart. PostgreSQL DSNs are Secret-backed, Playwright requires an explicit origin list and shell requires an executable allowlist; all are disabled by default and queue payloads cannot override them. Render-time validation fails incomplete enabled policies. - Documented that managed database/SUT egress must still be declared through
networkPolicy.runnerExtraEgressCidrs; a DSN cannot imply network identity. ADR-253 records the deployment boundary. - Evidence: chart source review and
git diff --checkare clean. Helm is not installed in the local Windows environment, sohelm lint/render remain a CI gate and must pass before merge.
- Tightened the local methodology proposal/approval contract with runtime enum and schema validation, canonical UTC timestamps, bounded artifact hashing, independent reviewer enforcement, and expiry checks at both issuance and use time.
- Added regression tests for malformed untrusted JSON-shaped inputs and already-expired approvals; methodology package now passes 24 tests.
- The next local roadmap slice is durable storage/replay integration for approved methodology artifacts; external provider evidence and independent assurance remain deferred final gates.
- Added the final fail-closed boundary for remote runner provenance: the HTTP
API validates the authenticated subject before queue mutation, and the
exported
RunnerWorkervalidates identities even when instantiated directly outside Kit. - Updated ADR-227 to make token-file/JWT credentials the only supported remote worker contract; static bearer tokens cannot prove a lease subject and are rejected before dequeue.
- Local evidence: server build and Kit suite remain green (196 passed, 0 failed, 2 platform skips); lint is rerunning after import normalization.
- Next: push the correction, wait for the complete CI/E2E journey, merge PR #241, then implement bounded local load/chaos evidence and methodology regression-scale closure. Provider credentials, trust-root/mTLS, WORM/KMS, DR/PITR, penetration and independent assurance remain final deferred gates.
- Closed the final local review findings for PR #245: the unknown-outcome journey now performs a real reference checkout and rereads the committed order, while mutation holdout evaluation recomputes the full canonical split digest including link data.
- Normalized minimized replay paths to the finding-qualified form
replay/counterexample.<finding-id>.min.jsonin code and documentation. - Local evidence: commerce 66 passed / 2 provider skips, ingest 25/25, reporter 10/10, and the repository typecheck passed. Provider-backed credentials and independent assurance remain deferred final gates.
- Next: push the review fixes, wait for the complete hosted journey, merge PR
#245, then update the roadmap completion audit from
origin/main.
- Wired plan-bound mutation evidence through the supported
aqa mutation regressioncommand. Evidence carryingplan_digestnow requires--holdout-split, parses and revalidates the canonical digest, and evaluates only the immutable holdout links; the generic path remains available for legacy non-plan-bound evidence. - Evidence: ingest 25/25, kit 197 passed / 2 platform skips, and full repository typecheck passed locally. The final hosted CI rerun for PR #245 is still required before merge.
- Hardened direct and CLI holdout evaluation against overlapping train/holdout
IDs, zero-rate serializable plans, manifest/split mismatch, and the documented
space-separated
--holdout-split split.jsonsyntax. - Added process-bound success coverage and mismatch protection. Local evidence: ingest 25/25, kit 198 passed / 2 platform skips, full typecheck green.
- Reclassified v0.6 methodology and v2.2 stateful agentic QA as verified for repository/local scope after merging the chaos, holdout, shrink/replay and ecommerce failure-journey evidence. The audit now lists only provider/ deployment and independent-assurance work as final gates.
- The project is intentionally not labeled production-ready enterprise until those operator-owned gates are executed; this preserves the side-project decision and avoids treating repository CI as deployment evidence.