Feature/project datasource

.agents/skills/harness-creator/README.md

@@ -0,0 +1,83 @@
+# harness-creator
+
+A compact skill for building and auditing harnesses around AI coding agents.
+
+It helps a repository provide five things agents need: instructions, state, verification, scope boundaries, and lifecycle handoff.
+
+## Install
+
+```bash
+npx skills add walkinglabs/learn-harness-engineering --skill harness-creator
+```
+
+Or copy `skills/harness-creator/` into your skill path.
+
+## Use
+
+```bash
+node skills/harness-creator/scripts/create-harness.mjs --target /path/to/project
+node skills/harness-creator/scripts/validate-harness.mjs --target /path/to/project
+node skills/harness-creator/scripts/run-benchmark.mjs --target /path/to/project --html /path/to/report.html
+```
+
+The scripts use only Node.js built-in modules. They can be run after copying the skill directory into another repository.
+
+## What It Creates
+
+- `AGENTS.md` or `CLAUDE.md`
+- `feature_list.json`
+- `progress.md`
+- `init.sh`
+- `session-handoff.md`
+
+`create-harness.mjs` detects common project types and package managers. It supports Node/npm/pnpm/yarn/bun, Python, Go, Rust, Maven, Gradle, and .NET at a basic verification-command level.
+
+## What It Checks
+
+`validate-harness.mjs` scores the five harness subsystems:
+
+1. Instructions
+2. State
+3. Verification
+4. Scope
+5. Lifecycle
+
+The score is structural. It tells you whether the harness is present and coherent; it does not replace real before/after agent-session testing.
+
+## Status
+
+- [x] Minimal harness scaffolding
+- [x] Five-subsystem validation
+- [x] HTML assessment report
+- [x] Structural benchmark report
+- [x] 10 eval cases
+- [x] Generic verification detection for common stacks
+- [ ] Optional real before/after agent-session replay
+
+## Files
+
+```text
+harness-creator/
+├── SKILL.md
+├── metadata.json
+├── agents/openai.yaml
+├── scripts/
+│   ├── create-harness.mjs
+│   ├── validate-harness.mjs
+│   ├── render-assessment-html.mjs
+│   ├── run-benchmark.mjs
+│   └── lib/harness-utils.mjs
+├── templates/
+│   ├── agents.md
+│   ├── feature-list.json
+│   ├── feature-list.schema.json
+│   ├── init.sh
+│   ├── progress.md
+│   └── session-handoff.md
+├── references/
+└── evals/evals.json
+```
+
+## Boundaries
+
+This skill is for harness engineering, not model selection, prompt tuning alone, or app architecture. Keep project-specific facts in the target repository.

.agents/skills/harness-creator/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: harness-creator
+description: >-
+  Build, audit, and improve lightweight harnesses for AI coding agents: AGENTS.md/CLAUDE.md,
+  feature state, verification workflows, scope boundaries, lifecycle handoff,
+  memory persistence, context control, tool safety, and multi-agent coordination.
+license: MIT
+---
+
+# Harness Creator
+
+Use this skill to make a repository easier for coding agents to start, stay in scope, verify work, and resume across sessions. Keep the harness small enough that agents actually follow it.
+
+Not for model selection, prompt tuning in isolation, chat UI design, or general app architecture.
+
+## Core Model
+
+Every useful coding-agent harness has five subsystems:
+
+| Subsystem | Minimal artifact | Purpose |
+|---|---|---|
+| Instructions | `AGENTS.md` or `CLAUDE.md` | Startup path, working rules, definition of done |
+| State | `feature_list.json`, `progress.md` | Current feature, status, evidence, next step |
+| Verification | `init.sh` or documented commands | Tests/checks the agent must run before claiming done |
+| Scope | Feature dependencies and done criteria | Prevents overreach and half-finished work |
+| Lifecycle | `session-handoff.md`, end-of-session routine | Makes the next session restartable |
+
+## First Move
+
+1. Inspect what already exists: instruction files, feature/state files, verification commands, docs, package manifests.
+2. Ask only for missing context that cannot be inferred safely: target agent, desired file name, tolerance for structure, and whether overwriting is allowed.
+3. Prefer a minimal harness first. Add memory, tool safety, multi-agent, or benchmark details only when the user's problem calls for them.
+
+## Common Tasks
+
+### Create a harness
+
+Use the bundled script when working on a local repository:
+
+```bash
+node skills/harness-creator/scripts/create-harness.mjs --target /path/to/project
+```
+
+Options:
+
+- `--agent-file CLAUDE.md` for Claude-oriented projects.
+- `--package-manager npm|pnpm|yarn|bun` when detection is wrong.
+- `--commands "cmd one,cmd two"` for custom verification.
+- `--force` only after confirming overwrites are acceptable.
+
+Then explain what was created and how the user should replace placeholder feature entries.
+
+### Audit an existing harness
+
+Run:
+
+```bash
+node skills/harness-creator/scripts/validate-harness.mjs --target /path/to/project
+```
+
+Report the five subsystem scores, the lowest-scoring area, and the first 2-3 changes that would improve reliability. Treat the lowest score as a candidate bottleneck; confirm with failures, logs, or task outcomes before claiming causality.
+
+### Produce a report
+
+Use when the user wants a shareable assessment:
+
+```bash
+node skills/harness-creator/scripts/render-assessment-html.mjs --target /path/to/project
+node skills/harness-creator/scripts/run-benchmark.mjs --target /path/to/project --html /path/to/report.html
+```
+
+Be clear that this is a structural benchmark. Real effectiveness still needs before/after agent sessions on representative tasks.
+
+## When to Read References
+
+Load only the reference needed for the user's problem:
+
+- Memory across sessions: [Memory Persistence](references/memory-persistence-pattern.md)
+- Reusable workflows as skills: [Skill Runtime](references/skill-runtime-pattern.md)
+- Permissions, tools, concurrency: [Tool Registry & Safety](references/tool-registry-pattern.md)
+- Context budget and progressive disclosure: [Context Engineering](references/context-engineering-pattern.md)
+- Delegation and parallel agents: [Multi-Agent Coordination](references/multi-agent-pattern.md)
+- Hooks, startup, long-running work: [Lifecycle & Bootstrap](references/lifecycle-bootstrap-pattern.md)
+- Non-obvious failure modes: [Gotchas](references/gotchas.md)
+
+## Design Rules
+
+- Keep the root instruction file short: routing and invariants, not a full manual.
+- Put project facts in project docs, not in the skill.
+- Make verification commands explicit and runnable.
+- Require evidence before marking a feature done.
+- Use one active feature unless the harness has explicit multi-agent ownership boundaries.
+- Prefer append/update state files over relying on chat history.
+- Never hide destructive behavior in scripts; overwrites require explicit user approval.
+
+## Deliverable Checklist
+
+For a usable minimal harness, leave the target project with:
+
+- [ ] `AGENTS.md` or `CLAUDE.md`
+- [ ] `feature_list.json`
+- [ ] `progress.md`
+- [ ] `init.sh`
+- [ ] Optional `session-handoff.md` for multi-session work
+- [ ] Documented verification evidence or next action
+
+If you cannot create files, provide exact file contents and commands instead.

.agents/skills/harness-creator/SKILL.md.en

@@ -0,0 +1,107 @@
+---
+name: harness-creator
+description: >-
+  Build, audit, and improve lightweight harnesses for AI coding agents: AGENTS.md/CLAUDE.md,
+  feature state, verification workflows, scope boundaries, lifecycle handoff,
+  memory persistence, context control, tool safety, and multi-agent coordination.
+license: MIT
+---
+
+# Harness Creator
+
+Use this skill to make a repository easier for coding agents to start, stay in scope, verify work, and resume across sessions. Keep the harness small enough that agents actually follow it.
+
+Not for model selection, prompt tuning in isolation, chat UI design, or general app architecture.
+
+## Core Model
+
+Every useful coding-agent harness has five subsystems:
+
+| Subsystem | Minimal artifact | Purpose |
+|---|---|---|
+| Instructions | `AGENTS.md` or `CLAUDE.md` | Startup path, working rules, definition of done |
+| State | `feature_list.json`, `progress.md` | Current feature, status, evidence, next step |
+| Verification | `init.sh` or documented commands | Tests/checks the agent must run before claiming done |
+| Scope | Feature dependencies and done criteria | Prevents overreach and half-finished work |
+| Lifecycle | `session-handoff.md`, end-of-session routine | Makes the next session restartable |
+
+## First Move
+
+1. Inspect what already exists: instruction files, feature/state files, verification commands, docs, package manifests.
+2. Ask only for missing context that cannot be inferred safely: target agent, desired file name, tolerance for structure, and whether overwriting is allowed.
+3. Prefer a minimal harness first. Add memory, tool safety, multi-agent, or benchmark details only when the user's problem calls for them.
+
+## Common Tasks
+
+### Create a harness
+
+Use the bundled script when working on a local repository:
+
+```bash
+node skills/harness-creator/scripts/create-harness.mjs --target /path/to/project
+```
+
+Options:
+
+- `--agent-file CLAUDE.md` for Claude-oriented projects.
+- `--package-manager npm|pnpm|yarn|bun` when detection is wrong.
+- `--commands "cmd one,cmd two"` for custom verification.
+- `--force` only after confirming overwrites are acceptable.
+
+Then explain what was created and how the user should replace placeholder feature entries.
+
+### Audit an existing harness
+
+Run:
+
+```bash
+node skills/harness-creator/scripts/validate-harness.mjs --target /path/to/project
+```
+
+Report the five subsystem scores, the lowest-scoring area, and the first 2-3 changes that would improve reliability. Treat the lowest score as a candidate bottleneck; confirm with failures, logs, or task outcomes before claiming causality.
+
+### Produce a report
+
+Use when the user wants a shareable assessment:
+
+```bash
+node skills/harness-creator/scripts/render-assessment-html.mjs --target /path/to/project
+node skills/harness-creator/scripts/run-benchmark.mjs --target /path/to/project --html /path/to/report.html
+```
+
+Be clear that this is a structural benchmark. Real effectiveness still needs before/after agent sessions on representative tasks.
+
+## When to Read References
+
+Load only the reference needed for the user's problem:
+
+- Memory across sessions: [Memory Persistence](references/memory-persistence-pattern.md)
+- Reusable workflows as skills: [Skill Runtime](references/skill-runtime-pattern.md)
+- Permissions, tools, concurrency: [Tool Registry & Safety](references/tool-registry-pattern.md)
+- Context budget and progressive disclosure: [Context Engineering](references/context-engineering-pattern.md)
+- Delegation and parallel agents: [Multi-Agent Coordination](references/multi-agent-pattern.md)
+- Hooks, startup, long-running work: [Lifecycle & Bootstrap](references/lifecycle-bootstrap-pattern.md)
+- Non-obvious failure modes: [Gotchas](references/gotchas.md)
+
+## Design Rules
+
+- Keep the root instruction file short: routing and invariants, not a full manual.
+- Put project facts in project docs, not in the skill.
+- Make verification commands explicit and runnable.
+- Require evidence before marking a feature done.
+- Use one active feature unless the harness has explicit multi-agent ownership boundaries.
+- Prefer append/update state files over relying on chat history.
+- Never hide destructive behavior in scripts; overwrites require explicit user approval.
+
+## Deliverable Checklist
+
+For a usable minimal harness, leave the target project with:
+
+- [ ] `AGENTS.md` or `CLAUDE.md`
+- [ ] `feature_list.json`
+- [ ] `progress.md`
+- [ ] `init.sh`
+- [ ] Optional `session-handoff.md` for multi-session work
+- [ ] Documented verification evidence or next action
+
+If you cannot create files, provide exact file contents and commands instead.

.agents/skills/harness-creator/agents/openai.yaml

@@ -0,0 +1,7 @@
+interface:
+  display_name: "Harness Creator"
+  short_description: "Scaffold and audit coding-agent harnesses"
+  default_prompt: "Use $harness-creator to create, validate, and benchmark a production-ready coding-agent harness for this repository."
+
+policy:
+  allow_implicit_invocation: true