Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ docs/ # repo-level documentation
evals/ # eval fixtures and harnesses (not shipped)
```

Edit plugin components directly under `plugins/<plugin>/` (agents, skills, hooks, knowledge, templates, docs). See [`plugins/dev-team/CLAUDE.md`](plugins/dev-team/CLAUDE.md) for the orchestration pipeline.
Edit plugin components directly under `plugins/<plugin>/` (agents, skills, hooks, knowledge, templates, docs). See [`plugins/dev-team/CLAUDE.md`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/CLAUDE.md) for the orchestration pipeline.

## Local development

Expand Down Expand Up @@ -104,4 +104,4 @@ The docs use three diagram formats, each for a distinct purpose — match the co

## Releasing

Releases are managed by [release-please](https://github.com/googleapis/release-please): push [conventional commits](https://www.conventionalcommits.org/) to `main` and merge the release PR it opens. The full rules — the version-bump mapping, why every commit that lands on `main` must be conventional under rebase-merge, and how to recover a missed release with a `Release-As:` footer — are in [`CLAUDE.md`](CLAUDE.md#releasing).
Releases are managed by [release-please](https://github.com/googleapis/release-please): push [conventional commits](https://www.conventionalcommits.org/) to `main` and merge the release PR it opens. The full rules — the version-bump mapping, why every commit that lands on `main` must be conventional under rebase-merge, and how to recover a missed release with a `Release-As:` footer — are in [`CLAUDE.md`](https://github.com/bdfinst/agentic-dev-team/blob/main/CLAUDE.md#releasing).
2 changes: 1 addition & 1 deletion GETTING-STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ On this page:
- [Key Concepts](#key-concepts)
- [How to Use It](#how-to-use-it)
- [Common Workflows](#common-workflows)
- [Diagnostic & Audit Workflows](#diagnostic--audit-workflows)
- [Diagnostic & Audit Workflows](#diagnostic-audit-workflows)
- [Available Agents and Skills](#available-agents-and-skills)
- [Rules to Know](#rules-to-know)

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Every `git commit` is automatically gated by `/code-review`. A `PreToolUse` hook

Developing, testing, or releasing the plugins? See **[CONTRIBUTING.md](CONTRIBUTING.md)** — local-dev setup (including live installs via symlinks), the `/agent-eval` and `/agent-audit` test commands, the security comparative-testing harness, how to add agents and skills, and the release process.

**`/code-review` benchmark harness** ([`evals/code-review-benchmark/`](evals/code-review-benchmark/README.md)): checks out real, known-defect commits from Defects4J (Java) and BugsJS (JavaScript), runs `/code-review` against the buggy revision, and scores whether its findings actually cover the real defect — with a dedicated Missed Defects report. Adapters and scoring are unit-tested against real fetched dataset samples; a live sweep against the actual corpora requires a local Defects4J/BugsJS install (see the harness README for prerequisites and usage).
**`/code-review` benchmark harness** ([`evals/code-review-benchmark/`](https://github.com/bdfinst/agentic-dev-team/blob/main/evals/code-review-benchmark/README.md)): checks out real, known-defect commits from Defects4J (Java) and BugsJS (JavaScript), runs `/code-review` against the buggy revision, and scores whether its findings actually cover the real defect — with a dedicated Missed Defects report. Adapters and scoring are unit-tested against real fetched dataset samples; a live sweep against the actual corpora requires a local Defects4J/BugsJS install (see the harness README for prerequisites and usage).

## Documentation

Expand All @@ -122,7 +122,7 @@ Start here:
| --- | --- |
| [Getting Started](GETTING-STARTED.md) | Install, the workflow, suggested skills, worked examples |
| [Contributing](CONTRIBUTING.md) | Local development, testing, adding agents/skills, releasing |
| [Plugin Development Guide](CLAUDE.md) | Project North Star, repo structure, working rules |
| [Plugin Development Guide](https://github.com/bdfinst/agentic-dev-team/blob/main/CLAUDE.md) | Project North Star, repo structure, working rules |

Per-plugin docs: **[dev-team](plugins/dev-team/README.md)** · **[security-assessment](plugins/security-assessment/README.md)** · **[marketplace-dev](plugins/marketplace-dev/README.md)** — each plugin's README is the entry point to its architecture, commands, and deeper guides.

Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0022-reject-delegation-only-sweep-dispatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The experiment ran on 2026-07-18: 3 arms (solo session / released plugin
10.12.0 / treatment branch) × 3 task classes (one-file fix, 4-file feature
slice, 26-file review sweep) × 5 interleaved reps, 45/45 runs completed at
matched external acceptance checks. Evidence:
[`reports/orchestration-benchmark-2026-07-18.md`](../../reports/orchestration-benchmark-2026-07-18.md)
[`reports/orchestration-benchmark-2026-07-18.md`](https://github.com/bdfinst/agentic-dev-team/blob/main/reports/orchestration-benchmark-2026-07-18.md)
(raw per-run data alongside).

Measured, per class (median cost per run; quality = acceptance passes):
Expand Down
10 changes: 5 additions & 5 deletions docs/cloud-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,11 @@ fi
exit 0
```

[`.claude/cloud-setup.sh`](../.claude/cloud-setup.sh) is the actual work:
[`.claude/cloud-setup.sh`](https://github.com/bdfinst/agentic-dev-team/blob/main/.claude/cloud-setup.sh) is the actual work:
this repo's test/gate toolchain (`jq`, `shellcheck`, the Python dev deps, `gh`,
`uv`, `mutmut`, `adr`), Node 24+ plus `npm ci` for the git hooks
(`pre-commit`, `pre-push`, `commit-msg`), the plugin install/refresh, and a
closing [`scripts/verify_toolchain.py`](../scripts/verify_toolchain.py) pass.
closing [`scripts/verify_toolchain.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/verify_toolchain.py) pass.
It **refreshes** the plugin (re-pulls the catalog and `plugin update`) rather
than only installing — `plugin install` is a no-op once a version is cached, and
the reused snapshot would otherwise pin the first version forever (see
Expand All @@ -82,7 +82,7 @@ The Setup script must `exit 0` even when provisioning went wrong, so it can only
*report* a broken toolchain — it can never refuse to hand one over. Two things
close that gap:

- [`scripts/verify_toolchain.py`](../scripts/verify_toolchain.py) — runs every
- [`scripts/verify_toolchain.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/verify_toolchain.py) — runs every
tool instead of probing `PATH`, so an installed-but-unstartable tool fails
instead of passing. The Setup script calls it at the end; run it yourself any
time with `python3 scripts/verify_toolchain.py`.
Expand Down Expand Up @@ -128,15 +128,15 @@ The fix has three layers, all wired into `.claude/cloud-setup.sh` and the
bfinster` and `claude plugin update dev-team@bfinster` on every invocation, and
the SessionStart hook no longer no-ops when a version is already present.
- **Enable marketplace auto-update.** Both call the plugin's own
[`skills/upgrade/scripts/enable_autoupdate.py`](../plugins/dev-team/skills/upgrade/scripts/enable_autoupdate.py)
[`skills/upgrade/scripts/enable_autoupdate.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/skills/upgrade/scripts/enable_autoupdate.py)
(`--enable`), which sets `extraKnownMarketplaces.bfinster.autoUpdate: true` in
the config `settings.json` — the same flag the `/plugin` UI and `/upgrade`
toggle. This is the key lever: it makes the CLI re-pull and upgrade **at launch,
within the existing snapshot**, so a routine release lands without a snapshot
rebuild. `/upgrade` runs the very same script (its `--check`/`--enable` modes),
so there is one implementation of the flag, not two.
- **Drift advisory.** Both also run the plugin's
[`skills/upgrade/scripts/check_version_drift.py`](../plugins/dev-team/skills/upgrade/scripts/check_version_drift.py),
[`skills/upgrade/scripts/check_version_drift.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/skills/upgrade/scripts/check_version_drift.py),
which compares the installed version against the refreshed catalog and surfaces
a "v{installed} → v{latest}; restart or run `/upgrade`" advisory. This is the
safety net for the one case the refresh can't fix silently: a **restrictive
Expand Down
8 changes: 4 additions & 4 deletions docs/cloud-startup-prompt.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Cloud startup prompt

The Setup script ([`.claude/cloud-setup.sh`](../.claude/cloud-setup.sh)) provisions
The Setup script ([`.claude/cloud-setup.sh`](https://github.com/bdfinst/agentic-dev-team/blob/main/.claude/cloud-setup.sh)) provisions
the machine. This is the other half: the **first message** to send in a fresh
cloud session, so Claude confirms the environment is actually sound before it
starts changing code.
Expand Down Expand Up @@ -78,14 +78,14 @@ or a `SessionStart` hook. Two caveats if you do:
- A `SessionStart` hook runs *after* Claude boots, so it cannot fix the
plugin-loading problem in step 3 — that one genuinely needs the Setup script.
- Keep it fail-open and time-boxed, like the hooks already registered in
[`.claude/settings.json`](../.claude/settings.json). A verification step that
[`.claude/settings.json`](https://github.com/bdfinst/agentic-dev-team/blob/main/.claude/settings.json). A verification step that
hangs session startup is worse than the drift it was guarding against.

## See also

- [`cloud-setup.md`](cloud-setup.md) — the Setup script itself, plugin freshness,
and the snapshot/caching behavior that pins stale plugin versions.
- [`.claude/cloud-setup.sh`](../.claude/cloud-setup.sh) — what to paste into
- [`.claude/cloud-setup.sh`](https://github.com/bdfinst/agentic-dev-team/blob/main/.claude/cloud-setup.sh) — what to paste into
claude.ai/code → Environment → Setup script.
- [`scripts/verify_toolchain.py`](../scripts/verify_toolchain.py) — the verifier,
- [`scripts/verify_toolchain.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/verify_toolchain.py) — the verifier,
runnable on its own at any time (`--quiet` for failures only, `--json` to script it).
10 changes: 5 additions & 5 deletions docs/context-and-session-economy-playbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ answers on its own:

| Question | Use |
| --- | --- |
| How much did *that run* cost? | [`/cost-report`](../plugins/dev-team/skills/cost-report/SKILL.md) |
| How did *that run* go, step by step? | [`/run-report`](../plugins/dev-team/skills/run-report/SKILL.md) |
| What should we change, based on recent sessions? | [`/session-review`](../plugins/dev-team/skills/session-review/SKILL.md) |
| Which agents/routing have gone stale? | [`/harness-audit`](../plugins/dev-team/skills/harness-audit/SKILL.md) |
| Which skills and agents are unused? | [`/artifact-lifecycle`](../plugins/dev-team/skills/artifact-lifecycle/SKILL.md) |
| How much did *that run* cost? | [`/cost-report`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/skills/cost-report/SKILL.md) |
| How did *that run* go, step by step? | [`/run-report`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/skills/run-report/SKILL.md) |
| What should we change, based on recent sessions? | [`/session-review`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/skills/session-review/SKILL.md) |
| Which agents/routing have gone stale? | [`/harness-audit`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/skills/harness-audit/SKILL.md) |
| Which skills and agents are unused? | [`/artifact-lifecycle`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/skills/artifact-lifecycle/SKILL.md) |
| Is the context ceiling in the right place *right now*? | [`context_ceiling_report.py`](context-ceiling-validation.md) |

This playbook is the **longitudinal** layer over those: run them in a fixed
Expand Down
4 changes: 2 additions & 2 deletions docs/context-ceiling-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,11 +130,11 @@ fail is worse than no gate* rule applied to the measurement itself.
Every policy decision — which tools are gated, which skills are exempt, how a
model maps to a window, how occupancy is summed, what counts as a sidechain
row — is **imported** from
[`plugins/dev-team/hooks/context_ceiling_guard.py`](../plugins/dev-team/hooks/context_ceiling_guard.py)
[`plugins/dev-team/hooks/context_ceiling_guard.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/hooks/context_ceiling_guard.py)
rather than restated. A validator that measured the ceiling even slightly
differently from the guard would be validating a threshold nobody ships.

[`tests/scripts/test_context_ceiling_report.py`](../tests/scripts/test_context_ceiling_report.py)
[`tests/scripts/test_context_ceiling_report.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/tests/scripts/test_context_ceiling_report.py)
additionally pins the report's occupancy walk against the hook's own
`_measure_occupancy` on shared fixtures, in the same spirit as the hook
suite's utilization-formula equality test.
Expand Down
4 changes: 2 additions & 2 deletions docs/experiments/01-final-results.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
**Design + prior results:** [`01-experiment-prompt-3sizes-3arms.md`](agentic-workflow-evidence/01-experiment-prompt-3sizes-3arms.md),
`tdd-vs-test-after-experiment.md`,
`tdd-vs-test-after-consolidated-report.md` (prior campaign; not migrated into this docs set)
**Runner:** [`scripts/run_tdd_experiment.py`](../../scripts/run_tdd_experiment.py)
**Analyzer:** [`scripts/analyze_tdd_experiment.py`](../../scripts/analyze_tdd_experiment.py)
**Runner:** [`scripts/run_tdd_experiment.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_tdd_experiment.py)
**Analyzer:** [`scripts/analyze_tdd_experiment.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/analyze_tdd_experiment.py)
**Raw data:** [`agentic-workflow-evidence/data/3sizes-small-sonnet-2026-06-22.jsonl`](agentic-workflow-evidence/data/3sizes-small-sonnet-2026-06-22.jsonl) (small),
[`agentic-workflow-evidence/data/tdd-largetask-sonnet-2026-06-21.json`](agentic-workflow-evidence/data/tdd-largetask-sonnet-2026-06-21.json) (medium, folded in),
[`agentic-workflow-evidence/data/3sizes-large-sonnet-2026-06-22.jsonl`](agentic-workflow-evidence/data/3sizes-large-sonnet-2026-06-22.jsonl) (large),
Expand Down
2 changes: 1 addition & 1 deletion docs/experiments/02-final-results.md
Original file line number Diff line number Diff line change
Expand Up @@ -647,7 +647,7 @@ step is taken seriously.

*Report generated by `claude-sonnet-4-6` in a remote Claude Code session.*
*Raw data: [`docs/experiments/agentic-workflow-evidence/data/`](agentic-workflow-evidence/data/)*
*Analysis script: [`scripts/analyze_tdd_pays.py`](../../scripts/analyze_tdd_pays.py)*
*Analysis script: [`scripts/analyze_tdd_pays.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/analyze_tdd_pays.py)*

---

Expand Down
4 changes: 2 additions & 2 deletions docs/experiments/05-final-results.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ refactor-cadence/workflow-matrix experiment line (experiments 01–05); see

**Data:** [`agentic-workflow-evidence/data/refactor-workflow-matrix.jsonl`](agentic-workflow-evidence/data/refactor-workflow-matrix.jsonl)
**Model:** `claude-sonnet-4-6` (held fixed across all arms)
**Harness:** [`scripts/run_refactor_experiment.py`](../../scripts/run_refactor_experiment.py),
orchestrated by [`scripts/run_workflow_matrix.py`](../../scripts/run_workflow_matrix.py)
**Harness:** [`scripts/run_refactor_experiment.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_refactor_experiment.py),
orchestrated by [`scripts/run_workflow_matrix.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_workflow_matrix.py)

---

Expand Down
2 changes: 1 addition & 1 deletion docs/experiments/RECOMMENDATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ against this repo's own tasks or harness (this harness's runs are
clean-room `claude -p` dispatches with no plugin loaded either, so neither
side of the comparison is controlled on agent configuration — the gap is
the corpus and harness, not "our agents vs. theirs"). See
[`../../plugins/dev-team/knowledge/test-cadence-tradeoffs.md`](../../plugins/dev-team/knowledge/test-cadence-tradeoffs.md)
[`../../plugins/dev-team/knowledge/test-cadence-tradeoffs.md`](https://github.com/bdfinst/agentic-dev-team/blob/main/plugins/dev-team/knowledge/test-cadence-tradeoffs.md)
for the decision rule and what a local replication would require before this
recommendation changes.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Experiment Prompt: Build-Pipeline vs. TDD vs. Non-TDD across Small / Medium / Large Tasks

**Type:** Reusable experiment prompt (hand this whole file to Claude to execute)
**Harness:** [`scripts/run_tdd_experiment.py`](../../../scripts/run_tdd_experiment.py)
**Harness:** [`scripts/run_tdd_experiment.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_tdd_experiment.py)
**Design + prior results:** `tdd-vs-test-after-experiment.md`,
`tdd-vs-test-after-consolidated-report.md` (prior campaign; not migrated into this docs set)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Experiment Prompt: When Does TDD Actually Pay Off?

**Type:** Reusable experiment prompt (hand this whole file to Claude to execute)
**Harness:** [`scripts/run_tdd_experiment.py`](../../../scripts/run_tdd_experiment.py) — **must be extended** (see "Extend the harness")
**Harness:** [`scripts/run_tdd_experiment.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_tdd_experiment.py) — **must be extended** (see "Extend the harness")
**Motivation:** [`FAQ.md` Q1](../FAQ.md),
[`01-final-results.md`](../01-final-results.md)
**Prior run results:** [`02-final-results.md`](../02-final-results.md)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Experiment Prompt: Refactoring Granularity, the Test Safety Net, and Code/Test Authorship

**Type:** Reusable experiment prompt (hand this whole file to Claude to execute)
**Harness:** [`scripts/run_tdd_experiment.py`](../../../scripts/run_tdd_experiment.py) — **must be extended** (see "Extend the harness")
**Harness:** [`scripts/run_tdd_experiment.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_tdd_experiment.py) — **must be extended** (see "Extend the harness")
**Design + prior results:** [`02-final-results.md` § refactoring-cadence follow-up](../02-final-results.md#proposed-follow-up-refactoring-cadence-and-the-test-safety-net)

This run resolves the open question from the *When Does TDD Pay Off?* study: the two
Expand All @@ -13,7 +13,7 @@ real **5%** difference from scatter, and it pulls apart the candidate mechanisms
It judges every arm on three axes — **how modular and changeable the code is**, **how good
the tests are**, and **what that quality cost** — and reports quality both raw and
per-dollar, so the verdict names the efficient frontier rather than the most expensive arm.
The three axes and their sensors are defined in [What we measure](#what-we-measure--three-axes-define-these-before-running).
The three axes and their sensors are defined in [What we measure](#what-we-measure-three-axes-define-these-before-running).

---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Experiment Prompt: Which agentic workflow yields maintainable, well-tested code at minimum cost

**Type:** Reusable experiment prompt (hand this whole file to Claude to execute)
**Harness:** [`scripts/run_refactor_experiment.py`](../../../scripts/run_refactor_experiment.py)
**Harness:** [`scripts/run_refactor_experiment.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_refactor_experiment.py)
**Lineage:** the single, consolidated experiment for the refactor-cadence line. It supersedes
the earlier granularity / larger-corpus designs (former prompts 06 and the cadence-larger
prompt), fixes the variables they left open (refactoring on, specs clear), and re-scopes
Expand Down Expand Up @@ -169,7 +169,7 @@ variance before funding it. It is **not** part of the base run.
2. **1-task cost pilot** on the new arms (`--task fare --arm all-tests-first-single --arm
all-tests-first-split --trials 1`) to get real per-cell cost.
3. **Campaign:** one command —
[`scripts/run_workflow_matrix.py`](../../../scripts/run_workflow_matrix.py). It runs the 7 arms
[`scripts/run_workflow_matrix.py`](https://github.com/bdfinst/agentic-dev-team/blob/main/scripts/run_workflow_matrix.py). It runs the 7 arms
× 4 tasks × 6 base trials, then **sequentially extends** only the arms whose cost-efficiency
(quality-per-dollar) rank is still ambiguous, up to a 12-trial ceiling. Defaults to a **dry
plan** (no dispatch); pass `--go` to run, `--analyze-only` to print the efficiency frontier
Expand Down
Loading
Loading