feat(harness): add Harness as Code package

.github/workflows/ci.yml

@@ -38,6 +38,26 @@ jobs:
       - run: npm ci
       - name: Run Vitest
         run: npm run test:ci
+      - name: Verify Harness DSL generated sources
+        run: npm run harness:generated
+      - name: Build Harness DSL package
+        run: npm run harness:build
+      - name: Test Harness DSL package
+        run: npm run harness:test
+      - name: Build Harness UI package
+        run: npm run harness-ui:build
+      - name: Test Harness UI package
+        run: npm run harness-ui:test
+      - name: Build Harness Studio package
+        run: npm run harness-studio:build
+      - name: Test Harness Studio package
+        run: npm run harness-studio:test
+      - name: Install Chromium for Harness Studio browser test
+        if: ${{ matrix.os == 'ubuntu-latest' && matrix.node == '22.20.0' }}
+        run: npx playwright install --with-deps chromium
+      - name: Test Harness Studio browser interaction
+        if: ${{ matrix.os == 'ubuntu-latest' && matrix.node == '22.20.0' }}
+        run: npm run harness-studio:test:browser
       - name: Upload JUnit test results
         if: ${{ always() }}
         uses: actions/upload-artifact@v4

.github/workflows/release.yml

@@ -2,6 +2,17 @@ name: Publish npm
 
 on:
   workflow_dispatch:
+    inputs:
+      package:
+        description: Package to publish
+        required: true
+        default: better-harness
+        type: choice
+        options:
+          - better-harness
+          - harness
+          - harness-ui
+          - harness-studio
 
 permissions:
   contents: read
@@ -20,8 +31,14 @@ jobs:
           registry-url: https://registry.npmjs.org
           package-manager-cache: false
       - run: npm ci
-      - run: npm test
-      - run: npm run pack:verify
-      - run: npm publish --access public
+      - run: npm run check
+      - name: Publish Better Harness
+        if: ${{ inputs.package == 'better-harness' }}
+        run: npm publish --access public
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+      - name: Publish selected Harness workspace
+        if: ${{ inputs.package != 'better-harness' }}
+        run: npm publish --workspace "@qoder-ai/${{ inputs.package }}" --access public
         env:
           NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.gitignore

@@ -18,4 +18,6 @@ skills/**/.plugin-eval/
 assets/wasm
 dev/terminal-demo/*.cast
 /.codex/better-harness/
+/harness-readme-compare-evidence/
+/harness-qoder-profile-evidence/
 design-qa.md

docs/ARCHITECTURE.md

@@ -21,6 +21,14 @@ the target journey, contract, evidence, governance, and DX-measurement system.
   supplied copy, but they do not compose or translate it; host UIs render the
   persisted semantic copy instead of rebuilding it from structural metadata.
 - Keep host evidence adapters separate from packaging, and keep all automation cross-platform.
+- Keep Harness as Code dependencies directed `core <- adapters <- devtools`:
+  the package root owns browser-safe grammar, IR, revision, and resolution;
+  source locking, host execution, compare execution, and highlighting are
+  explicit subpath boundaries. UI protocol layers accept injected executors.
+- Bridge persisted Harness runs into Inspector through the `harness-run`
+  session adapter. Harness owns revision and receipt schemas; session-analysis
+  owns `NormalizedToolActivityV1`; the bridge is one-way and never merges the
+  two contracts. See [ADR-0003](adrs/harness-run-evidence-bridge.md).
 - Keep stable host identity, display, home-option, and support-slice metadata in
   `scripts/host-support/`. Executable adapter imports and construction remain in
   capability-local registries; do not introduce a global host service locator.

docs/adrs/README.md

@@ -13,6 +13,10 @@ deprecated.
 | --- | --- | --- | --- | --- |
 | `ADR-0001` | [AI-Optimized Directory Structure](directory-structure.md) | Proposed | 2026-07-16 | Directory ownership, artifact routing, migration gates, and target-directory activation |
 | `ADR-0002` | [Developer Experience System](developer-experience-system.md) | Proposed | 2026-07-31 | Journeys, federated contracts, projections, native evidence, governance, support, and DX measurement |
+| `ADR-0003` | [Harness Run Evidence Bridge](harness-run-evidence-bridge.md) | Proposed | 2026-08-15 | Harness revision and run evidence ingestion into the Inspector normalization chain |
+| `ADR-0004` | [Harness Checkpoint Experiment Compare](harness-checkpoint-experiment-compare.md) | Proposed | 2026-08-17 | Checkpoint-anchored mixed-origin experiments, derived treatment axes, per-contrast verdicts, and the Studio experiment lifecycle |
+| `ADR-0005` | [Checkpoint-backed Compare Sources and Materialization](checkpoint-backed-compare-sources.md) | Proposed | 2026-08-17 | Source-neutral checkpoint discovery, request provenance, adapter projections, and per-lane materialization |
+| `ADR-0006` | [Session Notebook Trace and Outcome Projection](session-notebook-evidence-projection.md) | Proposed | 2026-08-18 | Ordered Turn evidence, response availability, evidence-bounded outcomes, and session-scoped patch requirements |
 
 ## Lifecycle
 

docs/adrs/checkpoint-backed-compare-sources.md

@@ -0,0 +1,120 @@
+# Checkpoint-backed Compare Sources and Materialization
+
+## Traceability
+
+- ADR ID: `ADR-0005`
+- Status: Proposed
+- Decision date: 2026-08-17
+- Related ADR: [Harness Checkpoint Experiment Compare](harness-checkpoint-experiment-compare.md)
+- Implementation spec: [Checkpoint-anchored multi-lane harness
+  experiments](../specs/2026-08-17-harness-studio-checkpoint-compare.md)
+
+## Context
+
+The first Harness Studio experiment runner uses a
+`session-execution-plan-v1` checkpoint. That plan happens to identify a Git
+repository, commit, tree, and agent-session position, and its current
+materializer creates detached Git worktrees. Those facts are sufficient for the
+first executable adapter, but they are not the product boundary.
+
+Compare must also be able to start from other resources that retain historical
+state. A presentation can have a versioned PPTX checkpoint and isolated file
+copies; a document, design, notebook, or remote workspace can expose different
+revision locators and materialization mechanisms. Naming the product or its
+browser contract `git-backed` would leak the first adapter into every future
+source and force non-Git histories to pretend to be repositories.
+
+The experiment layer also needs to keep three facts distinct:
+
+- the immutable checkpoint shared by the experiment;
+- the user request that fresh lanes execute, whether imported from history or
+  entered for a new comparison;
+- the runtime-owned isolated materialization created for each fresh lane.
+
+## Decision
+
+- **Harness Studio Compare is checkpoint-backed.** A Compare locks one
+  checkpoint and one request before execution. Git is the first checkpoint
+  source adapter, not the product identity or a required field in the generic
+  Studio model.
+- **`checkpointRef` remains opaque and authoritative.** The experiment manifest
+  continues to store the checkpoint plan reference and digest. It does not copy
+  repository, document, revision, slide, session, or worktree fields. The
+  checkpoint-owning adapter validates and interprets the referenced bytes.
+- **Source adapters project a generic descriptor for Studio.** A validated
+  adapter may expose display-oriented `adapter`, `resource`, `revision`,
+  optional `history`, and `materialization` facts. Labels and values belong to
+  the adapter: the Git adapter can say `Repository`, `Commit`, `Session
+  position`, and `Detached worktree`; a PPTX adapter can say `Presentation`,
+  `Version`, `Edit history`, and `Isolated document copy`. Studio renders the
+  descriptor and does not branch on those labels or require Git-shaped fields.
+- **The descriptor is a projection, not a second checkpoint.** It is derived
+  from a validated checkpoint for selection and preflight UX. Evidence and
+  execution continue to cite the checkpoint digest and adapter receipts rather
+  than trusting browser-authored display values.
+- **Discovery, lock, and materialization are separate phases.** A source adapter
+  may browse projects or document histories and resolve a mutable selection
+  into an immutable checkpoint. Locking creates no sandbox, worktree, document
+  copy, or result ref. Only execution invokes the adapter's materializer for
+  each fresh lane.
+- **Materialization is capability-based.** An adapter describes whether it can
+  create isolated copies, replay an observed history, preserve a result, and
+  clean up. The current Git adapter materializes a detached worktree at the
+  locked commit. A future PPTX adapter may copy a versioned file plus required
+  sidecars into an isolated directory. Neither mechanism changes what a
+  checkpoint means.
+- **Request provenance is independent of checkpoint provenance.** Historical
+  Replay locks the exact imported user request and its source locator; New
+  Request Compare locks newly entered request bytes. Editing an imported
+  historical request creates a new request identity instead of silently
+  rewriting the historical episode.
+- **Studio follows `Draft -> Lock -> Materialize -> Execute -> Evaluate`.** The
+  mutable Builder selects a scenario, checkpoint source, request source, and
+  variants. Lock produces the existing experiment/checkpoint references and a
+  comparability preview. The compact Workbench starts after lock and retains the
+  source identity while it streams ACP and result evidence.
+- **Comparison semantics remain those of ADR-0004.** Every fresh lane starts
+  from a materialization of the same checkpoint; observed history is contextual
+  unless its identity is complete; treatment axes and verdict strength remain
+  derived from evidence.
+
+## First Adapter Boundary
+
+The first implementation projects `session-execution-plan-v1` as:
+
+| Generic field | Git adapter value |
+| --- | --- |
+| Adapter | Git + agent session |
+| Resource | Repository display name |
+| Revision | Base commit, with tree as secondary identity |
+| History | Session id and selected entry |
+| Materialization | One detached worktree per fresh trial, created on Run |
+
+The projection deliberately does not become `GitCheckpointSourceV1` in the
+browser. Provider-specific validation, absolute paths, Git commands, locks, and
+cleanup stay in the Node adapter and runner.
+
+## Consequences
+
+Studio can explain the real Git/worktree plan today without making Git a
+permanent product dependency. Future versioned resources can join by supplying
+the same descriptor and lifecycle capabilities, while their canonical
+checkpoint formats remain adapter-owned. The extra boundary requires an
+adapter registry, explicit unsupported-checkpoint errors, and tests proving the
+browser UI contains no Git-specific branching. A generic projection cannot by
+itself guarantee that two historical environments are reproducible; checkpoint
+completeness and runtime receipts remain required evidence.
+
+## Rejected Alternatives
+
+- **Call the product Git-backed.** Rejected because it makes repository
+  vocabulary part of product identity and excludes versioned non-code
+  resources.
+- **Add a universal checkpoint object containing every source field.** Rejected
+  because optional Git, PPTX, document, design, and remote-workspace fields
+  would duplicate adapter contracts and weaken validation.
+- **Create one checkpoint per lane.** Rejected because it removes the shared
+  starting condition. Lanes receive separate materializations of one checkpoint.
+- **Create worktrees or file copies while the Builder is edited.** Rejected
+  because mutable selection is not execution authority and abandoned drafts
+  would leak resources.

docs/adrs/harness-checkpoint-experiment-compare.md

@@ -0,0 +1,127 @@
+# Harness Checkpoint Experiment Compare
+
+## Traceability
+
+- ADR ID: `ADR-0004`
+- Status: Proposed
+- Decision date: 2026-08-17
+- Related specs: [Harness Coding Compare](../specs/2026-08-15-harness-coding-compare.md),
+  [Session Checkpoint Executor PoC](../specs/2026-08-16-session-checkpoint-executor-poc.md),
+  [Harness UI Studio](../specs/2026-08-15-harness-ui-studio.md)
+- Implementation spec: [Checkpoint-anchored multi-lane harness
+  experiments](../specs/2026-08-17-harness-studio-checkpoint-compare.md)
+- Source abstraction: [Checkpoint-backed Compare Sources and
+  Materialization](checkpoint-backed-compare-sources.md)
+
+## Context
+
+Two evidence systems exist today and neither can express a mixed-origin,
+multi-lane experiment:
+
+- `harness-compare.v1` freezes a fixture, runs exactly two variants
+  (`baseline`/`candidate`) serially against a synthetic Git repository, and
+  derives a single verdict. The manifest carries one global `runtime.model`,
+  so per-lane model variation is inexpressible, and the treatment axis is
+  limited to `harness | runtime-profile`.
+- The session executor owns the checkpoint contract
+  (`session-execution-plan-v1`): base commit and tree plus session file
+  digest, entry id, and branch digest. Execution results land on
+  `refs/better-harness/session-executions` namespaced refs and never switch
+  the user's branch.
+
+Studio users want to pick a checkpoint, replay the observed historical
+trajectory, run two fresh agents (different harness or different model) from
+the same checkpoint in parallel, and compare the three trajectories side by
+side. The first executable adapter resolves a Git commit and materializes
+detached worktrees, but ADR-0005 keeps those mechanics out of the generic
+product contract. The temptation is to add a "sandbox checkpoint" type, copy
+source-specific fields into a new experiment schema, or let a single global
+verdict summarize a three-lane view. Each of those blurs provenance or
+fabricates attribution.
+
+## Decision
+
+- **One checkpoint definition per adapter.** The current session-executor plan
+  is the first checkpoint contract. Future source adapters may own other
+  versioned checkpoint formats under ADR-0005. A sandbox is a materialization
+  of the referenced checkpoint for one lane's execution, never a new checkpoint
+  type. Experiment documents hold a `checkpointRef` (path plus digest) and never
+  copy or reinterpret checkpoint fields.
+- **The `.harness` grammar does not change.** The upgrade lives entirely in
+  the experiment and compare layer. `harness-compare.v1` remains the frozen
+  fixture, two-variant path; it is not extended to cover checkpoints.
+- **New `harness-experiment.v1` manifest.** The version starts at v1; no
+  prior experiment schema exists. It declares one shared task (prompt hash,
+  grader) and N lanes. A lane has `origin: "observed"` (a recorded trajectory
+  reference plus its starting checkpoint digest; no sandbox is created) or
+  `origin: "execute"` (a harness id plus a per-lane runtime profile and model;
+  a sandbox is materialized from the shared checkpoint). The host, the visible
+  tool set, and the run policy are shared across lanes rather than per-lane, so
+  a lane cannot move the host and confound every contrast at once.
+- **Treatment axes are derived, never author-declared.** A contrast names
+  only the lanes it compares. The runner computes the axis by diffing the
+  lanes' harness id, runtime profile, and model. Exactly one differing axis
+  (`harness`, `runtime-profile`, or `model` — extending the existing
+  taxonomy) permits an attribution verdict; more than one yields
+  `multi-axis`, which is descriptive only and can never produce a harness
+  accept/reject.
+- **The statistical evidence bar is preserved.** Execute lanes carry a trial
+  count, and per-contrast verdicts are decided by the existing matched-pair
+  decision policy, including the two-matched-pair floor. A contrast whose
+  lanes ran once each is attributable in principle but reports
+  `insufficient_evidence`, never a promotion; the experiment schema is not a
+  bypass for the compare evidence bar.
+- **An identically configured pair is descriptive, not invalid.** Where
+  `harness-compare.v1` rejects a manifest whose variants move nothing, an
+  experiment accepts it and reports `no-axis-moved`. Two identical lanes from
+  one checkpoint measure run-to-run variance, which is the noise floor every
+  other contrast is read against.
+- **Checkpoint completeness is a gate, not an assumption.** A checkpoint
+  anchors a commit and tree, not the untracked files or dirty state the
+  historical trajectory may have started from. Materialization records a
+  completeness receipt: either a clean-tree assertion or a captured
+  dirty-state patch applied to every fresh lane. Without it, fresh lanes may
+  not claim to share the historical lane's starting condition.
+- **Observed lanes are contextual evidence by default.** A historical
+  trajectory participates in an attribution verdict only when its runtime,
+  model, harness revision, environment receipts, and task identity (prompt
+  and prior session context) all match the fresh lanes. Absent any of those
+  — and the prompt almost never matches — it is displayed as context in a
+  descriptive contrast. Grades are optional on observed lanes.
+- **Execution discipline for parallel lanes.** All lanes pass preflight
+  (checkpoint digest, base commit/tree, session digest) before any lane
+  starts. Worktree materialization is serialized to avoid Git lock
+  contention; execution then runs in parallel with `Promise.allSettled`, so
+  one lane's failure never discards another lane's evidence. Every event
+  carries `experimentId`, `laneId`, and `runId`. Each lane persists its own
+  `HarnessRevision`, runtime/model receipt, sandbox receipt, trajectory,
+  patch, and grade. Results stay on namespaced refs; adopting a result is a
+  later explicit action.
+- **Studio hosts experiments, not one global verdict.** The stateless
+  run-per-request `/agui` endpoint is insufficient. Studio gains a
+  server-side experiment registry (create an experiment, subscribe to
+  per-lane event streams, cancel), persists evidence per experiment
+  directory, previews which axes a configuration moves before running, and
+  renders one verdict per contrast. A three-lane view never shows a single
+  aggregate verdict.
+- **Tool-chain correlation is explicit evidence, not a similarity score.**
+  Studio normalizes each ACP-derived tool call into tool name, resource target,
+  and canonical arguments, then aligns lane sequences one-to-one and in order.
+  The UI labels a counterpart `exact`, `same-resource`, `same-tool`, or `none`
+  and shows the neighbouring calls around it. This makes a shared file read and
+  a shared Read → Edit → Test path visible without presenting fuzzy similarity
+  as provenance or causal proof.
+
+## Consequences
+
+Mixed-origin comparison becomes expressible without expanding the core
+`.harness` DSL or forking the checkpoint contract. The compare aggregate
+taxonomy gains a `model` axis and per-contrast decisions; consumers of
+`harness-compare-result.v1` are unaffected because `harness-compare.v1`
+remains frozen. Observed trajectories gain a principled, limited role:
+context by default, baseline only under full identity. The cost is a second
+manifest schema to maintain, a Studio server that now holds run state, and
+the obligation to keep the derived-axis rule and evidence floor enforced in
+the runner rather than trusted from the document. Implementation still
+requires the dated spec, acceptance scenarios, and test evidence mandated by
+`AGENTS.md`.