Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
3ed1eb6
docs(adr): propose 0003, plan kernel, tiers and an engine-neutral con…
claude Nov 17, 2026
0b68266
feat(proofs): state and check the ADR-0003 plan laws in Agda
claude Nov 17, 2026
6b86160
feat(proofs): gate the plan laws with canonical flags and negative co…
claude Nov 17, 2026
ae9640e
docs(status): record the ADR-0003 plan laws PL-1 to PL-19
claude Nov 17, 2026
ccca9af
docs(adr): correct ADR-0003 law 2, safety of a joined plan
claude Nov 17, 2026
c94eda9
feat(conformance): add the engine-neutral suite runner
claude Nov 17, 2026
64a03be
test(conformance): add 45 L0 kernel cases and their generator
claude Nov 17, 2026
7863633
test(conformance): add three negative-control kernels and selftest.sh
claude Nov 17, 2026
75d1dba
feat(kernel): add a Rust plan-kernel spike that passes every L0 case
claude Nov 17, 2026
100ef0d
docs(conformance): put the plan example's op members in JCS order
claude Nov 17, 2026
e683fe2
docs(conformance): document the optional case fields the runner reads
claude Nov 17, 2026
74e4a77
build(just): add conformance, kernel-build and kernel-check recipes
claude Nov 17, 2026
c91bd25
test(conformance): add residue-from-answer and the T6 literal-answer …
claude Nov 17, 2026
4cd0a7a
docs(conformance): record the first L0 evidence run and its roadmap line
claude Nov 17, 2026
21cff93
docs(adr): record owner decisions on 0003 and keep the mint-defect ca…
claude Nov 17, 2026
0aa489a
ci(conformance): replace the E2E job with the Conformance L0 check
claude Nov 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 18 additions & 182 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,204 +4,40 @@
# This workflow is managed by gh actions-lock.
# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
#
# RSR Standard E2E + Aspect + Benchmark Workflow Template
# Conformance workflow (owner decision 2026-11-17, ADR-0003).
#
# Covers ALL merge requirement test categories:
# - E2E (end-to-end pipeline tests)
# - Aspect (cross-cutting concern validation)
# - Benchmarks (performance regression detection)
# - Readiness (Component Readiness Grade: D/C/B)
# The required check is named after the highest conformance level the engine
# actually passes: `Conformance L0` today, later L1, L2, L3 as those levels
# exist (verification/conformance/README.adoc). There is no "E2E" job: it is
# retired until Scaffoldia can mint something end to end. The inherited
# Idris2/Zig harness tests/e2e.sh is no longer run here (docs/status/ROADMAP.adoc).
#
# INSTRUCTIONS: Uncomment and customise the section matching your stack.
# Delete sections that don't apply. See examples in each job.
# The commands live in verification/conformance/check-l0.sh, which
# `just conformance` also calls. Needs cargo, rustc, jq and coreutils, all on a
# stock ubuntu-latest image.

name: E2E + Aspect + Bench
name: Conformance
on:
push:
branches: [main, master, develop]
paths:
- 'src/**'
- 'ffi/**'
- 'tests/**'
- '.github/workflows/e2e.yml'
# No `paths:` filter on pull_request: a required check that is skipped by a
# path filter never reports, and the pull request would wait forever.
pull_request:
branches: [main, master]
paths:
- 'src/**'
- 'ffi/**'
- 'tests/**'
workflow_dispatch:
permissions: read-all
concurrency:
group: e2e-${{ github.ref }}
group: conformance-${{ github.ref }}
cancel-in-progress: true
jobs:
# ─── Default: repo smoke (always runs — not a silent no-op) ─────────
# Runs the repo's own E2E harness (tests/e2e.sh). Downstream work either
# extends tests/e2e.sh or uncomments a stack block below.
e2e:
name: E2E — smoke
conformance-l0:
name: Conformance L0
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v7.0.1
- name: Run E2E harness
- name: Build kernel, check fixtures, run L0, run negative controls
run: |
if [ -f tests/e2e.sh ]; then
bash tests/e2e.sh
else
echo "::notice::no tests/e2e.sh present — nothing to run"
fi

# ─── End-to-End Tests (downstream stack-specific examples) ─────────
# Uncomment ONE of the following e2e job blocks matching your stack.

## === RUST E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
# - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
# - run: cargo build --release
# - run: bash tests/e2e.sh
# # OR: cargo test --test end_to_end -- --nocapture

## === ZIG FFI E2E ===
# e2e:
# name: E2E — FFI Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: goto-bus-stop/setup-zig@abea47f85e598557f500fa1fd2ab7464fcb39406 # v2.2.1
# with:
# version: 0.15.1
# - run: cd ffi/zig && zig build test
# - run: bash tests/e2e.sh

## === ELIXIR E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: erlef/setup-beam@fc68ffb90438ef2936bbb3251622353b3dcb2f93 # v1.24.0
# with:
# otp-version: '27.0'
# elixir-version: '1.17'
# - run: mix deps.get && mix compile --warnings-as-errors
# - run: mix test test/integration/e2e_test.exs --trace

## === DENO/RESCRIPT E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: denoland/setup-deno@667a34cdef165d8d2b2e98dde39547c9daac7282 # v2.0.4
# with:
# deno-version: v2.x
# - run: deno install --node-modules-dir=auto
# - run: deno task res:build # ReScript compile
# - run: deno test tests/e2e/

## === PLAYWRIGHT (Browser E2E) ===
# e2e-playwright:
# name: Playwright — ${{ matrix.project }}
# runs-on: ubuntu-latest
# timeout-minutes: 20
# strategy:
# fail-fast: false
# matrix:
# project: [chromium-1080p, firefox-1080p, webkit-1080p]
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: denoland/setup-deno@667a34cdef165d8d2b2e98dde39547c9daac7282 # v2.0.4
# with:
# deno-version: v2.x
# - run: deno install --node-modules-dir=auto
# - run: npx playwright install --with-deps
# - run: npx playwright test --project=${{ matrix.project }}
# - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
# if: failure()
# with:
# name: playwright-traces-${{ matrix.project }}
# path: test-results/**/trace.zip
# retention-days: 7

## === HASKELL E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: haskell-actions/setup@cd0d9bdd65b20557f41bea4dbe43d0b5fbbfe553 # v2.11.0
# with:
# ghc-version: '9.6'
# cabal-version: '3.10'
# - run: cabal build all
# - run: bash tests/integration-test.sh

# ─── Aspect Tests ──────────────────────────────────────────────────
# Cross-cutting concerns: thread safety, ABI contracts, SPDX, dangerous patterns
# Uncomment and customise:

# aspect-tests:
# name: Aspect — Architectural Invariants
# runs-on: ubuntu-latest
# timeout-minutes: 10
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - run: bash tests/aspect_tests.sh

# ─── Benchmarks ────────────────────────────────────────────────────
# Performance regression detection. Uncomment matching stack:

## === RUST BENCH ===
# benchmarks:
# name: Bench — Performance Regression
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
# - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
# - run: cargo bench 2>&1 | tee /tmp/bench-results.txt
# - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
# if: always()
# with:
# name: benchmark-results
# path: /tmp/bench-results.txt
# retention-days: 30

## === ZIG BENCH ===
# benchmarks:
# name: Bench — Performance Regression
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: goto-bus-stop/setup-zig@abea47f85e598557f500fa1fd2ab7464fcb39406 # v2.2.1
# with:
# version: 0.15.1
# - run: cd ffi/zig && zig build bench

# ─── Readiness (CRG) ──────────────────────────────────────────────
# Component Readiness Grade: D (runs) → C (correct) → B (edge cases)

# readiness:
# name: Readiness — Grade D/C/B
# runs-on: ubuntu-latest
# timeout-minutes: 10
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
# - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
# - run: cargo test --test readiness -- --nocapture
set -euo pipefail
bash verification/conformance/check-l0.sh
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -123,3 +123,7 @@ dist/
# ...but never track Idris2 typecheck output. `idris2 --typecheck src/interface/abi.ipkg`
# writes compiled .ttc/.ttm under build/ttc/; these are generated artifacts.
/build/ttc/

# Agda interface files (written when agda is run by hand; the plan-law gate
# checks a fresh copy and writes none here)
*.agdai
27 changes: 16 additions & 11 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -120,12 +120,10 @@ clean-all: clean
# TEST & QUALITY
# ═══════════════════════════════════════════════════════════════════════════════

# Run all tests
test *args:
@echo "Running tests..."
idris2 --typecheck src/interface/abi.ipkg
cd src/interface/ffi && zig build test {{args}}
@echo "Tests passed!"
# Run the tests that gate this repository: the conformance suite at L0
# (the CI check `Conformance L0`). The inherited Idris2/Zig seam is not
# covered; see test-verbose, test-smoke and docs/status/ROADMAP.adoc (P0).
test: conformance

# Run tests with verbose output
test-verbose:
Expand All @@ -139,7 +137,8 @@ test-smoke:
idris2 --typecheck src/interface/abi.ipkg
cd src/interface/ffi && zig build

# Run end-to-end tests (full pipeline: build → run → verify)
# Inherited Idris2/Zig harness. NOT run by CI and not a gate (retired as a CI
# check 2026-11-17; docs/status/ROADMAP.adoc P0). Needs idris2 and zig.
e2e:
bash tests/e2e.sh

Expand Down Expand Up @@ -173,10 +172,10 @@ crg-badge:
esac; \
echo "[![CRG $$grade](https://img.shields.io/badge/CRG-$$grade-$$color?style=flat-square)](https://github.com/hyperpolymath/standards/tree/main/component-readiness-grades)"

# Run the full merge-requirement test suite (ALL categories)
# Per STANDING rule: P2P + E2E + aspect + execution + lifecycle + bench
test-all: test e2e aspect bench
@echo "All test categories passed — safe to merge!"
# Run the local test categories that exist: conformance, aspect, bench.
# This is not the merge gate; the required CI check is `Conformance L0`.
test-all: test aspect bench
@echo "Local test categories passed."

# Run all quality checks
quality: fmt-check lint test
Expand Down Expand Up @@ -665,6 +664,12 @@ help-me:

import? "build/just/proofs.just"

# ═══════════════════════════════════════════════════════════════════════════════
# PLAN KERNEL & CONFORMANCE SUITE — see build/just/conformance.just
# ═══════════════════════════════════════════════════════════════════════════════

import? "build/just/conformance.just"

# ═══════════════════════════════════════════════════════════════════════════════
# SESSION MANAGEMENT (THIN BINDINGS TO CENTRAL STANDARDS)
# ═══════════════════════════════════════════════════════════════════════════════
Expand Down
25 changes: 25 additions & 0 deletions build/just/conformance.just
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# SPDX-License-Identifier: MPL-2.0
# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
#
# PLAN KERNEL & CONFORMANCE SUITE (ADR-0003)
#
# Imported by ../../Justfile via `import? "build/just/conformance.just"`.
# The suite is engine-neutral (verification/conformance/README.adoc); the
# Rust spike under src/kernel/ is one engine it is run against. Needs: cargo,
# bash, jq, coreutils. Run via `just conformance`.

kernel_manifest := "src/kernel/Cargo.toml"

# Build the plan kernel (release, locked dependencies)
kernel-build:
cargo build --release --locked --manifest-path {{kernel_manifest}}

# Test, lint and format-check the plan kernel workspace
kernel-check:
cargo test --locked --manifest-path {{kernel_manifest}}
cargo clippy --locked --all-targets --manifest-path {{kernel_manifest}} -- -D warnings
cargo fmt --manifest-path {{kernel_manifest}} --all -- --check

# Fixtures match their generator; kernel passes L0; every negative control fails
conformance:
verification/conformance/check-l0.sh
16 changes: 15 additions & 1 deletion build/just/proofs.just
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# `just proof-check-all` for the full sweep.

# Check all formal proofs (Idris2 + Lean4 + Agda + Coq)
proof-check-all: proof-check-idris2 proof-check-lean4 proof-check-agda proof-check-coq proof-scan-dangerous
proof-check-all: proof-check-plan-laws proof-check-plan-laws-controls proof-check-idris2 proof-check-lean4 proof-check-agda proof-check-coq proof-scan-dangerous
@echo "=== All proof checks complete ==="

# Check Idris2 proofs (ABI, types, dependent type proofs)
Expand Down Expand Up @@ -87,6 +87,20 @@ proof-check-agda:
fi
echo "PASS: All Agda proofs verified"

# Runs agda --safe --without-K --cubical-compatible (standards PROOF-POLICY
# §2.1) on verification/proofs/agda/PlanLaws. Fatal if agda is absent or not
# the pinned version; also fails on a pragma, hole, postulate, warning or
# unchecked file. See the header of scripts/check-plan-laws.sh.
# Check the ADR-0003 plan laws (Agda; fatal if agda is absent)
proof-check-plan-laws:
@bash scripts/check-plan-laws.sh

# Each file in tests/reject/PlanLaws/Controls must be refused, for the reason
# its EXPECT-REFUSED line names.
# Run the negative controls of the plan-law gate
proof-check-plan-laws-controls:
@bash scripts/check-plan-laws.sh --controls

# Check Coq proofs
proof-check-coq:
#!/usr/bin/env bash
Expand Down
4 changes: 2 additions & 2 deletions docs/architecture/TOPOLOGY.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Scaffoldia consumes these schemas; it does not fork their definitions.
| `src/interface/Abi/` | Generic Idris2 foreign declarations and type/layout model | Not matched to Zig exports
| `src/interface/ffi/` | Zig exported scaffold functions and unit/integration tests | No composer operations; Zig-side tests only
| `verification/proofs/` | Example/skeleton proof artifacts | No linked composer assurance
| `tests/` | Shell E2E, aspect, workflow tests | No actual mint/retrofit acceptance test
| `tests/` | Conformance and workflow tests; inherited E2E harness (not run by CI), aspect tests | No actual mint/retrofit acceptance test
| `.github/workflows/` | 25 YAML workflows, actions.lock | Multiple pre-existing CI failures
| `.machine_readable/`, `build/`, `Justfile` | RSR substrate and task recipes | Some checks refer to removed/moved paths
|===
Expand Down Expand Up @@ -100,7 +100,7 @@ ADR-0002 leaves retention/retirement of this optional seam open.
permissions, allowlist and pinned reusable workflows separately.
* Some reusables still refer to `hyperpolymath/standards` while the live
canon is `metadatastician/standards`.
* The `e2e.yml` job invokes a script requiring Idris2 and Zig without
* (Resolved 2026-11-17: `e2e.yml` now runs the conformance suite as `Conformance L0`; `tests/e2e.sh` is no longer run by CI.) It used to invoke a script requiring Idris2 and Zig without
explicit toolchain provisioning.

Do not suppress failures to create a green summary. Fix at the owning layer
Expand Down
Loading
Loading