diff --git a/2026-reproAI-MIT-STAMPED.html b/2026-reproAI-MIT-STAMPED.html new file mode 100644 index 00000000..83f78ea8 --- /dev/null +++ b/2026-reproAI-MIT-STAMPED.html @@ -0,0 +1,1542 @@ + + + + + + + + + STAMPED principles and their role in the age of AI-driven scientific software development + + + + + + + + + + + + + + + +
+
+ + + + + + + +
+
+ +

STAMPED principles and their role in the age of
AI-driven scientific software development

+

+ Agents often fail not for lack of "intelligence".
They fail for lack of structure to act on.
+

+ +
+ + + + + + + + + +
+ Yaroslav O. Halchenko
+ @yarikoptic + @yarikoptic@fosstodon.org +
+
Center for Open Neuroscience +
Department of Psychological and Brain Sciences +
Center for Cognitive Neuroscience
+ Dartmouth College
+ New Hampshire, USA
+ + +
+
+
+ + LLM Workshop — MIT — October 16th, 2026
+ Live slides/Sources: + https://datasets.datalad.org/centerforopenneuroscience/talks/2026-reproAI-MIT-STAMPED.html
+
+ + + + + + + + + + + +
+ + +
+
+ + + +
+

Reality Check

+ +
+ +
+ +
+

+ Disclaimer: I am not here to sell you a model, a vendor, or a workflow. +

+

+ I am here to argue that the boring infrastructure work we have been doing for 20 years
+ is exactly what decides whether agents help you or hurt you. +

+
+

+ Counts and names on these slides are links — follow along, click through later. +

+
+ + + +
+

Standing on two talks

+ + + + + +
+ Mine — ReproNim webinar, Feb 2026
+ ReproFlow & YODA: structure your studies + (video) +
    +
  • Version control everything
  • +
  • Look up you must not
  • +
  • Compose modularly you shall
  • +
  • … and preserve all artifacts as an AI-accessible knowledge base
  • +
+
+ Oscar Esteban — OHBM 2026 Educational
+ Using LLMs in Neuroimaging Research: LLMs for Coding +
    +
  • Agent = Model + Harness
  • +
  • AGENTS.md as the operational contract
  • +
  • Context engineering > prompt wording
  • +
  • Grounding beats a bigger search
  • +
+
+ +
+ Oscar showed how to build the harness.
+ This talk is about what the harness stands on — and how harnesses themselves become shareable. +
+ + +
+ + + +
+ +
+

It’s 2026. Agents write the code.

+ +
+ +
+

So the bottleneck moved

+ + + + + + + + + + + +
writing the code→verifying and situating it
can I express this?→is this right, and does it fit what already exists?
+ +
+ +
+

+ HBR, Feb 2026: “AI doesn’t reduce work, it intensifies it” +

+ +
+ +
+

Without structure, the agent has to guess

+ + + + + +
+ What an agent finds in most labs +
study_final2/
+  scans/
+    s01_ep2d_bold_moco_3.nii
+    s01_t1_mprage_FINAL.nii
+    s02_ep2d_bold_moco_3(1).nii
+  analysis_new/
+    run_it.sh
+    run_it_OLD.sh
+    results_use_this_one.mat
+  notes.docx
+
    +
  • Which file is which subject? Infer.
  • +
  • What are the acquisition parameters? Infer.
  • +
  • Which script made results_use_this_one.mat? Infer.
  • +
  • Was it run on the fixed data or the broken data? Infer.
  • +
+
+ What an agent finds in a BIDS dataset +
ds000001/
+  dataset_description.json
+  participants.tsv
+  sub-01/
+    anat/sub-01_T1w.nii.gz
+    anat/sub-01_T1w.json
+    func/sub-01_task-rest_bold.nii.gz
+    func/sub-01_task-rest_bold.json
+  derivatives/
+    mriqc/dataset_description.json
+
    +
  • Subject, session, task, run: in the filename
  • +
  • Parameters: in the sidecar
  • +
  • Provenance of derivatives: GeneratedBy
  • +
  • Correctness: a validator says so
  • +
+
+ +
+ Same model. Same prompt. Different affordances. + On the left the agent improvises confidently and is wrong. + On the right it can act, and the validator catches it when it does not. +
+

+ Not just rhetoric: a structured neuroimaging harness moved first-choice tool-selection accuracy from + 23.3% to 93.6% across seven models — + Chen, …, Poldrack, + Bringing analytic rigor to agentic AI for science, arXiv:2608.19902, 2026 + (brain-researcher.com). +

+ +
+ +
+

This problem already has a name

+ + + + + + +
+ +

YODA, 2018 — for humans composing studies

+
→ + +

STAMPED, 2026 — generalized, gradeable, domain-agnostic

+
+

+ We did not invent these for AI. That is the point — they were already the right properties. +

+
+ +
+ + + +
+ +
+ +

Seven properties a research object should have —
whoever, or whatever, picks it up next.

+ +
+ +
+

STAMPED in one table

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
PropertyIn one lineWhat it buys an agent
Self-containedEverything under one boundaryA bounded context window. Nothing it needs is “on Bob’s laptop”.
TrackedVersion control & provenanceA diff to review, and a record of who/what produced each change — including the agent.
ActionableExecutable & documented workflowsThe load-bearing one. A command it can run, not a paragraph it must interpret.
ModularIndependent & composable modulesA unit small enough to reason about, swap, and verify in isolation.
PortableRuns anywhere with explicit specsThe agent’s sandbox is not your laptop. If it only runs on your laptop, it does not run.
EphemeralTransient compute environmentThrow-away containers — exactly how agents already work. No accreted state to corrupt.
DistributableShareable, persistent, retrievableIt can get the inputs itself instead of asking you for a Dropbox link.
+

+ + A. Macdonald, C. Baker, I. To, Y. O. Halchenko. + STAMPED: principles for reproducible research objects. + Preprint, 2026. doi: 10.31222/osf.io/f3h82_v1 +  ·  + sources: stamped-principles/stamped-paper +  ·  + paper.stamped-principles.org +

+ +
+ +
+

Actionability is the one that agents live or die on

+
+

An actionable object answers, without a human in the loop:

+
    +
  • What is this? → a declared standard (BIDS, NWB), not a README paragraph
  • +
  • How do I run it? → a recorded command, not “see Methods”
  • +
  • Did I get it right? → a validator exit code, not a reviewer’s opinion
  • +
+
+ +
+ And standards are what make Actionability cheap. + You do not negotiate the layout per project — you inherit it.
+ “You’ve seen one BIDS dataset, you’ve seen them all” was a convenience for humans. + For a model that would otherwise have to infer the layout, it is the difference between acting and hallucinating. +
+ +

+ “All standards are ‘bad’, but some are used.” — D. Clunie, MICCAI 2017. + A used standard beats a correct one — and now it has a second constituency. +

+ +
+ +
+

… and AI changed what counts as actionable

+ + + + + + + + + + + + + + + + + + + + + +
“Actionable” required…… so what could act
BeforeAn exact command, API call, or documented CLI. Prose was not enough.Tools, and humans who already knew the tool. + Everything else stayed a paragraph in the Methods.
NowA procedure described in human language — + with scripts dropped in exactly where precision matters.Any agent, on first contact. + The interface moved from syntax to language, so vastly more of what we + already wrote down became executable.
NextRoutines that fire without being asked.The harness itself — CI, PR monitors, scheduled agents.
+ +
+ + + + + +
+ 1. Skills — logic written in language +
    +
  • Human language carries the judgement; scripts carry the precision
  • +
  • One skill, one job — modularity still applies
  • +
  • The procedure you did by hand three times, written down once
  • +
+
+ 2. Routines — the harness runs them for you +
    +
  • Claude Code on the web watching a repo: triage issues, review PRs
  • +
  • Scheduled agents; CI that does more than run tests
  • +
  • No human has to remember to invoke it
  • +
+
+
+ +
+ AI did not only make more things actionable. It made automation on top of + actionability cheap — which is the real multiplier. +
+ +
+ +
+

So grade them too: a skill is a research object

+

+ A skill that only runs in your repo, on your paths, undocumented and unversioned, + is a silo with a friendlier interface. +

+ + + + + + + + + + + + + + + + + + + +
Property… applied to a skill or an automation
Self-containedCarries its own references and scripts. Not “see our internal wiki”.
TrackedIn git, with a metadata.version and a history — so a change to the procedure is reviewable.
ActionableRunnable commands and scripts, not only prose. Recursive, and the point.
ModularOne skill, one job. Composable with the others instead of a mega-prompt.
PortableNo hardcoded paths, no “works on our cluster”. Declares its dependencies and versions.
EphemeralRuns in a fresh sandbox. No accreted state it silently depends on.
DistributablePublished where someone else can find and install it — not pasted into chat.
+ +
+ Good automation and good skills pass the STAMPED checklist. + That is not a purity test — it is precisely what makes them reusable + beyond the person who wrote them. +
+ +

+ Walk it yourself: checklist.stamped-principles.org + (MUST / SHOULD / MAY, item by item) · + examples · + principles · + paper source + (main.tex + — small enough to paste into a context window, which is how we actually grade a study today). +

+ +
+ +
+ + + +
+ +
+

What we built to make this real

+

Twenty years of it, in neuroimaging, before anyone said “agent”.

+
+ +
+

Standards are the grammar — BIDS as the worked example

+ +
+ 11 modalities · 25 active BEPs · + a machine-readable schema · + a validator · + BIDS-Apps with one calling convention +
+
+ The schema is the part that matters here: the standard is not prose an agent reads, + it is data an agent queries. And the validator is the type-checker that + turns a confident wrong answer into a failing build. +
+
+ But a layout only tells you what the files are. + It says nothing about how they came to be, or what it cost to make them. + Standard → datalad run → duct: + what it is → how it was made → what it took. + Three layers, and an agent needs all three. +
+ +

+ + K. J. Gorgolewski, …, Y. O. Halchenko, …, R. A. Poldrack. + The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. + Scientific Data 3:160044, 2016. +

+
+ +
+

Tracked: datalad run writes the record for you

+
$ datalad run -m "run brain extract workflow" \
+    --input "inputs/ds*/sub-01/anat/sub-01_T1w.nii.gz" \
+    --output "sub-*/anat" \
+    bash -e code/brain_extraction.sh inputs/ds*/sub-01/anat/sub-01_T1w.nii.gz
+
$ git show --stat
+commit 1223bc4bead3c7e6be15b78546c894c6465dd7a4
+
+    [DATALAD RUNCMD] run brain extract workflow
+
+    === Do not change lines below ===
+    {
+     "cmd": "bash -e code/brain_extraction.sh inputs/ds000001/sub-01/anat/sub-01_T1w.nii.gz ...",
+     "dsid": "b656ac80-973c-45ef-887e-1b8d6f6a51dd",
+     "exit": 0,
+     "inputs": ["inputs/ds*/sub-01/anat/sub-01_T1w.nii.gz"],
+     "outputs": ["sub-*/anat"],
+     "pwd": "."
+    }
+    ^^^ Do not change lines above ^^^
+
+ sub-001/anat.nii.gz      | 1 +
+ sub-001/anat_mask.nii.gz | 1 +
+
    +
  • Machine-readable, in the commit message — no sidecar database to lose
  • +
  • Clean git state in, clean git state out: no surprises
  • +
  • datalad rerun reads it back and re-executes
  • +
+

+ Example from the DataLad Handbook. + Found in the wild in ~15% of indexed DataLad repositories on GitHub. +

+
+ +
+

Provenance does not care who wrote the code

+

+ Agent-written or hand-written, the [DATALAD RUNCMD] record is the same shape. +

+

+ The tests do not know the code was AI-generated either. +

+
+ This is why Tracking is the principle that makes AI disclosure + mechanical instead of aspirational — and we will come back to that at the end. +
+ +
+ +
+

The missing piece: con/duct

+ +
+ Wrap any command. It records, next to the outputs: + wall time, CPU, peak RSS, per-process-tree samples, exit code, environment, full + stdout/stderr — as plain, self-contained files. +
+ +
+ +
+

… and that log is the provenance you need later

+ + + + + +
+ “Why did this get OOM-killed on the cluster?” +
  • Peak RSS over time, per process. Not your memory of it.
+ “How much do I request for 2,000 subjects?” +
  • Measured, from the run you already did.
+ “What did it print before it died?” +
  • Captured, versioned, next to the outputs.
+
+ +

con-duct plot — MRIQC on one subject/session

+
+ +
+ Why an AI audience should care: when you hand a failing pipeline to an agent, + it needs numbers and text it can read, not a human recollection. + duct turns “it crashed, I think it ran out of memory” into a file. + Self-contained (S), tracked (T), immediately actionable (A). +
+

+ Already wired in: set REPRONIM_USE_DUCT=1 with + ///repronim/containers + and every datalad containers-run gets a duct trace for free. +

+
+ +
+

All three layers, one real artifact — built with AI

+

+ dandi/compression-comparisons-t261-study + — a reproduction of + Buccino et al. 2023 on ephys compression, + plus a comparison against the new ITU-T T.261 codec. +

+ + + +
+ 1. The layout (what) +
dataset_description.json
+code/
+  compression-comparisons-tools/
+  run_sweep.sh
+sourcedata/
+  aind-ephys-compression/      # 523 GB
+derivatives/
+  compbench-*/<cell>/
+    metrics.json  manifest.json
+    duct-info.json  duct-usage.jsonl
+  reports-*/
+

BIDS-shaped, YODA-composed: data and tool are + subdatasets pinned by commit, not copies.

+
+ 2. The record (how) — and 3. the cost (what it took) +
$ git log --oneline
+7733465 [DATALAD RUNCMD] cell aind-634568__zstd...
+
+ "cmd": "duct --output-prefix .../duct- \
+      --sample-interval 1 --report-interval 5 \
+      compbench run --codec zstd ...",
+ "inputs":  ["sourcedata/aind-ephys-compression/.."],
+ "outputs": ["derivatives/../aind-634568__zstd.."],
+ "exit": 0
+

The recorded command is a duct + invocation — so every cell commits its own resource trace beside its own results.

+
+ + + + + + + + + + + + + + + +
commits[DATALAD RUNCMD]of those, wrapping ductcellsrecorded computeAI disclosure
126512161095912, 0 errors3838 h (peak RSS 120 GB)26 Co-Authored-By: Claude
+ +
+ +
+

So I tried to rerun it. Live, on this laptop.

+ + + +
+
$ datalad clone .../compression-comparisons-t261-study
+$ datalad get code/compression-comparisons-tools
+  ✗ fatal: repository not found
+
+$ # .gitmodules says:  url = ../..
+$ #  → resolves to https://github.com/ (not a repo)
+$ # fine on the author's disk; breaks for a cloner
+
+$ git clone .../dandi/compression-comparisons \
+     code/compression-comparisons-tools # by hand
+$ uv venv envs/compbench && uv pip install ...
+
+$ datalad rerun 42dc5cfa
+  run(ok): ... [duct --clobber --output-prefix ..]
+  Wall Clock Time: 0.549 sec   RSS peak: 7.8 MB
+
+ Did the science come back? + + + + + + + + + +
fieldoriginal vs. rerun
cr1.51429112246829 same
encoded_bytes5283 same
rmse0.0 same
round_trip_oktrue same
signal_std706.609486021761 same
*_time_sdiffer — as they must
manifestkernel 6.12→7.1,
python 3.13.15→3.13.5
+

+ Different machine, different kernel, six weeks later — + every scientific number came back bit-for-bit. + Only timings and the environment record moved. +

+
+ +
+ And the checklist earned its keep: the one thing that broke was + Distributable — a relative submodule URL that resolved only on + the author’s own disk. S, T, A and + M all held. You do not find that by reading your own repo; + you find it by being someone else. +
+ +
+ +
+ + + +
+ +
+

So what do we build next?

+

Not more code. Harnesses and skills.

+
+ +
+

Agent = Model + Harness

+

+ formulation borrowed from Oscar Esteban, OHBM 2026 +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Harness pieceWhat it isSTAMPED property
AGENTS.md / CLAUDE.mdThe project’s operational contract: conventions, commands, what not to doSelf-contained
SkillsA named, reusable procedure with its own references and scriptsModular, Actionable
Hooks / CIGuard rails that run whether or not anyone remembered toTracked
MCP serversThe port to external systems — archives, trackers, databasesDistributable
Container / sandbox
(con/yolo)
Where it is allowed to run, and how much it can breakPortable, Ephemeral
+
+ The harness is itself a research object. Which means it can be + self-contained, tracked, modular, portable — or it can be a pile of prompts in your notes app. +
+
+ + +
+

Where are you on the ladder?

+
+ +

+ Ladder figure & framing: + Jo Van Eyck, + “You are Not Ready: Agentic coding in 2026” +

+
+ + + + + + + + + + + + + + + + + + + + + +
LevelWhat it looks likeThe skill you have to acquire
L1 ChatBrowser tab; copy-pastePrompt & context engineering
L2 Mid-loopIn the IDE — “autocomplete on steroids”Critically evaluating alternatives, not accepting them
L3 In-the-loopThe babysitting phase — you watch it struggleDoom loops, guard rails, extracting reusable skills. 2–3 months, minimum.
L4 On-the-loopSpec it, hand off, verify the artifactEverything above plus a real harness: skills, tests, sandboxes, scanners
L5 Multi-agentOrchestration; the frontierEveryone is still figuring this out
+
+ L4–L5 is exactly where a bad harness stops being survivable — + which is why the harness, not the model, is the thing to invest in. +
+
+ +
+ +
+

And the loop that gets you up it

+ + + + + +
+
+%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '17px'}}}%% +flowchart TD + IDEA["Idea"] --> S1["1. Overall design / spec"] + S1 --> S2["2. Aspect design / prompt"] + S2 --> S3["3. AI work"] + S3 --> S4["4. Review / test"] + S4 -->|"iterate"| S2 + S4 --> S5["5. Compound into spec / SKILL"] + S5 -->|"next aspect"| S2 + S5 -->|"adjust / memorize"| S1 + + style IDEA fill:#9c27b0,color:#fff,stroke:#7b1fa2,stroke-width:2px + style S1 fill:#1565c0,color:#fff,stroke:#0d47a1,stroke-width:2px + style S2 fill:#2e7d32,color:#fff,stroke:#1b5e20,stroke-width:2px + style S3 fill:#e65100,color:#fff,stroke:#bf360c,stroke-width:2px + style S4 fill:#c62828,color:#fff,stroke:#b71c1c,stroke-width:2px + style S5 fill:#283593,color:#fff,stroke:#1a237e,stroke-width:3px +
+
+ + + + + + + + + + + + + + + +
Development modelStages usedWhat is missing
Vibe coding2 → 3 → 4 (loop)No spec, no condensing — the knowledge evaporates with the session
Spec-driven1 → 2 → 3 → 4 → 2…Loops back to the prompt; the spec should be updated too
Compound engineering1 → 2 → 3 → 4 → 5 → 1+2…Nothing — each cycle makes the next one cheaper
+
+ Stage 5 is where a skill is born. + Vibe coding throws the lesson away; compound engineering writes it down — + and what it writes down is exactly the artifact we just said must be STAMPED. +
+

+ Tooling for stage 1: spec-kit · + OpenSpec · + compound-engineering · + LAD + (in use in dandi-cli) +

+
+ +
+ +
+

That was the individual ladder. Here is the team one.

+ +

+ Figure reproduced from: + E. C. Johnson, T. T. Nguyen, B. K. Dichter, …, + M. E. Martone, F. Pestilli, D. Yatsenko. + SciOps: Achieving Productivity and Reliability in Data-Intensive Research. + arXiv:2401.00077, 2024. +

+
+ The preprint names BIDS, NWB, DataLad/git-annex, DANDI and brainlife.io as its own + Level-3 exemplars — and reserves Level 5 (AI + human in the loop) for the top. +
L5 presupposes L3 underneath it. You do not get closed-loop discovery + on top of study_final2/. +
+ +
+ +
+

Right now, everyone is rebuilding the same private harness

+

+ A hundred labs, each writing their own “how to work with BIDS” prompt.
+ None of them shared. None of them versioned. None of them tested. +

+

+ We have seen this failure mode before. It was called
+ “every lab has its own analysis scripts”, and we fixed it with standards and packages. +

+
+ Skills are the new reusable module. + Apply Modularity and Distributability to the harness, + or watch AI recreate the silo problem at machine speed. +
+ +
+ +
+

We are already doing this — upstream, in public

+

+ K-Dense-AI/scientific-agent-skills + — 177 scientific skills, + Agent Skills spec, MIT-licensed, scanned and reviewed. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
ContributionWhoWhat it carries
skills/bids
+ PR #125 — merged
ours (CON)Entity grammar & 35 entities · the + machine-readable schema shipped as a reference · + all 25 BEPs · + HeuDiConv/ReproIn/dcm2bids/BIDScoin conversion paths · + validation · derivatives & BIDS-Apps
skills/datalad
+ PR #227 — merged
Dylan Pulver,
with our upstream review
Clone/get semantics & the “pointers are not data” failure · + datalad run / rerun / containers-run as the provenance spine · + publishing to siblings · STAMPED landed in + references/provenance.md on our suggestion
PR #265 — openBrent McPherson
(McGill)
Maps all seven STAMPED properties onto concrete DataLad commands, + plus a read-only evidence probe per property + (datalad subdatasets -r for S/M, git log --grep '[DATALAD RUNCMD]' for T, + datalad rerun --report for A, datalad siblings for D…)
issue #217 — closedours (CON)“Should STAMPED be its own skill?” — and the still-open question underneath: + where should domain skills live? An upstream monorepo, or the project’s own repo, with discovery on top?
+

+ Note the shape: a community member wrote the DataLad skill, we reviewed it, and a third party + then extended it with STAMPED. That is exactly how a standard spreads — and it is + now happening to harnesses. +

+
+ +
+

And domain-agnostic ones: con/skills

+ + + + + + + + + + + + + + + + + +
SkillWhat it automates
introduce-reuse-complianceSPDX / REUSE licensing metadata — including for BIDS datasets
credit-contributionsCRediT author-contribution statements from git history
tinuous-analyzerCI-failure investigation across archived con/tinuous logs
bisect-and-patch-git-annexBisect a git-annex regression from CI logs and produce a red/green PR
issue-triage · pr-feedback-reviewTriage and review work, cross-referenced against the codebase
introduce-codespell · introduce-intuit-autoBootstrap spell-checking / automated releases into any project
+ +
+ Each of these started as a thing we did by hand three times and got tired of. + Writing it down as a skill is how the n-th time gets cheaper — for us, + and for whoever clones the repo. +
+

+ Deeper dive on the workflow around these: + “A few words of intro into AI assisted coding” + (CA Origami Retreat, 2026) — also in the appendix of this deck. +

+
+ +
+ + + +
+ +
+

Two regimes — and be honest about which one you are in

+

+ Either the system checks itself against goals and spec — + or the work items stay small enough that a human still can. +

+ + + + + +
+ A. Scaffolded self-verification
+ the agentic system keeps itself in sync +
    +
  • An executable spec the loop is checked against + (spec-kit, + OpenSpec, + LAD)
  • +
  • Tests and validators as the oracle — the loop must be able to fail
  • +
  • Provenance so drift is detectable at all + (datalad run, duct, CI history)
  • +
  • Automated gates before human eyes; humans review intent and architecture
  • +
+
+ Only then are large autonomous batches defensible. +
+
+ B. Small, human-reviewable, hand-tried
+ where almost every research project actually is +
    +
  • A diff a human can actually read is a design constraint, not a courtesy
  • +
  • One concern per change; small commits; mark them (@pytest.mark.ai_generated)
  • +
  • Run it by hand before you believe it. Agent-rate output, human-rate trust
  • +
  • Cheap to back out — the reversibility is the safety mechanism
  • +
+
+ Not a lesser mode — the correct default until A is actually built. +
+
+ +
+ There is no third regime where you ship large, unverified agent output safely. + And the way out of B is not a better model — it is + Actionable specs, Tracked provenance and + Modular units: the same STAMPED properties, again. +
+

+ “Well, less is more” — Robert Browning, Andrea del Sarto, 1855; + later the motto of Mies van der Rohe, who credited his mentor Peter Behrens rather than claiming it. +

+ +
+ +
+

We are not the only ones saying this

+ + + + + + + + + + + + + + + + + + + + + + + + + +
SourceWhat it found / argues
DORA 2025 + (Google Cloud)Higher AI adoption raises throughput and delivery instability. + Their prescription is literally regime A+B: + “working in small batches is a critical countermeasure to the risks of AI-assisted + development”, plus test automation and fast feedback — because + “verification is a fundamentally different cognitive task than creation”.
METR RCT, 202516 experienced OSS developers, 246 real issues, on repos they already maintained: + 19% slower with AI tools — while estimating they had been 20% faster. + The perception gap is the hazard, not the tooling. + (METR now labels this historical — tools moved on. The gap is the durable lesson.)
Addy Osmani“Proof over vibes.” Verification, not authoring, is the bottleneck; + break agent output into digestible commits; no PR without tests or a demo; + humans stay non-negotiable for security and architecture.
Simon Willison
+ ▶ Pragmatic Summit talk
+ ▶ Lenny’s Podcast
“Your job is to deliver code you have proven to work.” + The skill is instructing and then confidently verifying; a model cannot take accountability, you can. + And on regime B specifically — even with tests, he reports that + “I haven’t used it yet” still undermines his confidence in a change. + Trying it by hand is not superstition.
Faros telemetry + (~22k devs)PR size +51%, median time in review +441%, + incidents per PR +243%. Big agent diffs do not get reviewed — they get waved through.
+ +
+ Converging, from very different vantage points: + the safe size of an AI-assisted work item is set by the strength of your verification loop. + Build the loop, or keep the items small. Preferably both. +
+ +
+ +
+

HI ↔ AI — every project picks its own policy

+

+ Rows 1–2 live in regime B; rows 3–4 are only honest once + regime A is actually built. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
StanceOSS exemplarsCON / adjacent exampleLeans on
Reject AI-generated contentZig, Krita, Clojure, QEMUgit-annex upstream — pure-HI commitsAI stays ephemeral; never enters the tracked artifact
Accept with disclosureNumPy, Kubernetes, Linux kernel, DjangoDataLad, DANDI — + Co-Authored-By: Claude… trailers, @pytest.mark.ai_generatedTracking — provenance is non-negotiable
Spec-driven AI-generated
(HI specs, AI writes, HI reviews)
Common in practice; rarely a declared policyAnnexTube, + citations-collector, + parts of dandi-cliT + A + M — the spec is the contract
Autonomous agents in the loopBrain Researcher + (Poldrack lab) — agentic neuroimaging analysis; + K-Dense AI — “AI co-scientists” + (and the host of the skills library we contribute to)con/skills + + con/yolo for triage, review, dep updatesAll of STAMPED — the harness becomes the type-checker
+

+ Common ground, all four: AI cannot be an author + (ICMJE, Jan 2026); + humans retain full responsibility; AI use is disclosed inside the artifact. + STAMPED Tracking is what makes that mechanical rather than a promise. +

+

+ Catalog of declared OSS stances: + melissawm/open-source-ai-contribution-policies. + — Note what the row-4 exemplars are: harnesses. + Brain Researcher constrains the agent to + acceptable analyses and required validations rather than trusting it — + autonomy is bought with structure, not with a bigger model. +

+
+ +
+

Take home — what to do Monday

+
    +
  1. Make one object actionable. Pick one study. Put it in a declared standard + (BIDS, NWB) + and run the validator. The agent gets a grammar; you get a type-checker.
  2. +
  3. Record one command. Wrap your next analysis step in + datalad run + — or at minimum in duct. + A provenance record you did not have to remember to write.
  4. +
  5. Grade one thing against STAMPED. + checklist.stamped-principles.org, + fifteen minutes, and you will know which property is your weakest.
  6. +
  7. Write down your harness. An + AGENTS.md in the repo beats a prompt in your notes app. + Declare your AI policy there while you are at it.
  8. +
  9. Size the work to the loop you actually have. Until the project + self-checks, keep AI-assisted items small, reviewable, and tried by your own hand — + and say so out loud, so nobody mistakes regime B for regime A.
  10. +
  11. Push one skill upstream. The next person who needs your procedure should + clone it, not re-derive it. + scientific-agent-skills, + con/skills, or your own repo — but somewhere public.
  12. +
+
+ Standards and provenance were always the right answer. + AI did not change the answer — it changed the deadline. +
+
+ +
+ + + +
+

Acknowledgements

+ + +
+ + + + + + +
+

Thank you!

+

Structure your objects, and
let both intelligences work on them

+ +

+ stamped-principles.org · + checklist · + examples · + con/skills · + con/duct +

+
+ + + +
+ +
+

Appendix

+

Material cut for time — press ↓ to walk it, or follow the links.

+
+ +
+

STAMPED and FAIR do different jobs

+ + + + + + + + + + + + + + + + + +
FAIRSTAMPED
About sharing: can someone else find this and get it?About the object being operational on its own: can someone — or something — re-run it?
A FAIR dataset can still be a .zip of .mat files with a DOI.A STAMPED object carries its own layout, environment, and execution record.
Graded by a repository.Graded by a checklist you can walk, item by item.
+
+ Not competitors — different lenses on the same stack. FAIR tells you whether the world can reach it. + STAMPED tells you whether anything useful happens once it does. +
+
+ +
+

Portable + Ephemeral: the environment travels with the record

+ + + + + +
+
$ datalad containers-run \
+    -n repronim-mriqc \
+    -m "run MRIQC on sub-01" \
+    --input sub-01 --output derivatives/mriqc \
+    '{inputs}' '{outputs}' participant
+
    +
  • Container image is a tracked subdataset, not a latest tag that drifts
  • +
  • The commit records image and command together
  • +
  • Bind-mount only the current module — look up you must not
  • +
  • Throw the container away afterwards: that is Ephemerality, and it is already how agents run
  • +
+
+ +

ReproNim/containers — auto-rebuilt, versioned collection

+
+

+ “Depend on where you run, you should not.” An agent’s sandbox is not your laptop — + which makes portability a correctness property, not a courtesy. +

+
+ + +
+ +
+ +
+ +
+ +
+ + + + +
+
+ + + + + + + + + + diff --git a/2026-reproAI-MIT-STAMPED/PLAN.md b/2026-reproAI-MIT-STAMPED/PLAN.md new file mode 100644 index 00000000..6ee465ce --- /dev/null +++ b/2026-reproAI-MIT-STAMPED/PLAN.md @@ -0,0 +1,401 @@ +# PLAN — `2026-reproAI-MIT-STAMPED.html` + +**Venue**: LLM workshop, MIT — **Friday, 2026-10-16** +**Slot**: 25 min talk + 5 min Q&A +**Title**: *STAMPED principles and their role in the age of AI-driven +scientific software development* +(Dave's original framing — "STAMPED and how coding is transformed and +the new scientific framework for code/software development" — kept as +the speaker-note gloss, not on the slide.) + +**Live URL / QR**: +`https://datasets.datalad.org/centerforopenneuroscience/talks/2026-reproAI-MIT-STAMPED.html` +→ `pics/2026-reproAI-MIT-STAMPED-qrcode.png` (generated at authoring time) + +--- + +## 1. Thesis + +> Agents do not fail for lack of intelligence. They fail for lack of +> **structure to act on**. The same properties that made a research +> object reusable by a human — STAMPED — are what make it *actionable* +> by an agent. And the next thing we owe the commons is not more code: +> it is **shared harnesses and skills** that encode how to act on +> standardized objects. + +Three claims, in order: + +1. **What changed**: the bottleneck moved from *writing* code to + *verifying and situating* it. Agent-rate output demands agent-rate + review — which only provenance and standards can supply. +2. **What did not change**: STAMPED (and YODA before it) already named + the properties. **Actionability** is the load-bearing one for agents, + and **standards are what make Actionability cheap** — they are the + scaffolding *and* the shared vocabulary across HI ⇄ AI. +3. **What is next**: harnesses and skills become first-class, shareable, + *modular* research artifacts. We are already contributing them + (BIDS + DataLad skills upstream); the field should treat this as + infrastructure work, not as prompt-hoarding. + +## 2. Arc + +Hybrid of `SOUL.md` §7 arc 2 (Challenges → Solutions → Take-home) with +the AI-ladder flavor of arc 5. Parent decks for layout/slide theft: + +| Parent | What is borrowed | +| ----------------------------------------- | -------------------------------------------------------------------------- | +| `2026-repronim-YODA-BIDS-webinar.html` | Title-slide shape, `datalad run`/`rerun` record, `con/duct` slides, "Look up you must not", YODA-beyond-code-and-data | +| `2025-distribits-YODA.html` | Same YODA spine (grandparent) | +| `2026-ca-origami-retreat-aicoding.html` | Reality-Check GIF, AI-coding ladder (→ appendix), `con/skills` table, Thank-you closer | +| `2026-usrse-con-talk.html` | HI ⇄ AI policy table, "provenance doesn't care who wrote the code" beat, Acknowledgements | +| `2026-nih-bids2.0.html` | YODA → STAMPED transition, `stamped-principles-webshot_20260602.png` | +| `2024-distribits-datalad.html` | Canonical Acknowledgements layout | + +**External credit (explicit, on-slide):** +[Oscar Esteban, *Using LLMs in Neuroimaging Research: LLMs for Coding*, +OHBM 2026 Educational](https://www.nipreps.org/presentations/2026-OHBM-AI-Educational/) +— the "Agent = Model + Harness" formulation, `AGENTS.md` as the +operational contract, and the grounding-beats-search result. This talk +deliberately picks up *where Oscar's leaves off*: he showed how to build +the harness; this one argues about **what the harness stands on** and +**how harnesses themselves get shared**. + +## 3. Slide-by-slide (34 main slides + 7-slide appendix, for 25 min) + +### Act 0 — Frame (≈2 min) + +1. **Title slide** — standard `SOUL.md` §3 template, QR, logo strip. +2. **Reality Check** — `idiocracy-fixed.gif` disclaimer. + *(borrowed: aicoding deck §Reality Check)* +3. **Previously on… / standing on two talks** — ReproNim YODA webinar + (mine) + Oscar's OHBM AI Educational (his). States the hand-off. + +### Act I — What actually changed (≈4 min) + +4. **"It's 2026. Agents write the code."** — black beat slide. +5. **The bottleneck moved** — writing → verifying + situating. + `2026-ai-intensifies.png` (HBR) as the visual. +6. **Without structure, agents go wild** — the central claim slide. + Left column: what an agent sees in an ad-hoc `~/data/study_final2/`. + Right column: what it sees in a BIDS dataset. Same model, different + affordances. +7. **This has a name** — YODA (2018) → STAMPED (2026). + `pics/stamped-yoda.svg` / `pics/yoda-principles-reordered.png`. + +### Act II — STAMPED, and what AI did to Actionability (≈7 min) + +7b. **Section opener** — the STAMPED banner + (`pics/borrowed/stamped-banner_ver-3.svg`, the project's own artwork: + the seven words with their icons) replacing the former text-only + opener, with the "whoever, *or whatever*, picks it up next" line as a + fragment underneath. +8. **STAMPED in one table** — all seven, one line each, from + . Third column: *what it buys an + agent*. Paper + preprint DOI cited. +9. **Actionability is the one that agents live or die on** — and + standards are what make it cheap. "You've seen one BIDS dataset…" + lands twice as hard for a model that would otherwise *infer*. +10. **…and AI changed what counts as actionable** (NEW — the slide the + talk now pivots on). A three-row escalation: + - *Before*: an exact command / API / documented CLI. Tools and + humans-who-already-knew could act; everything else stayed a + paragraph in the Methods. + - *Now*: a procedure **described in human language**, with scripts + dropped in where precision matters → **any agent, on first + contact**. The interface moved from *syntax* to *language*. + - *Next*: routines that fire **without being asked** → the + **harness** acts (CI, PR monitors, scheduled agents). + Then two columns: **(1) Skills** — language carries the judgement, + scripts carry the precision, one skill one job (modularity still + applies); **(2) Routines** — Claude Code on the web watching a repo, + scheduled agents, CI that does more than run tests. + *Punchline:* AI did not only make more things actionable — it made + **automation on top of actionability cheap**, and that is the real + multiplier. +11. **So grade *them* too: a skill is a research object** (NEW — the + recursive turn). Seven-row table mapping every STAMPED property onto + a skill/automation (S: carries its own references, not "see our + wiki"; T: `metadata.version` + history; A: runnable, not only prose; + M: one skill one job; P: no hardcoded paths; E: fresh sandbox; + D: published, not pasted into chat). + *Punchline:* **good automation and good skills pass the STAMPED + checklist — that is precisely what makes them reusable beyond the + person who wrote them.** Footer carries the checklist / examples / + principles / `main.tex` links (absorbed from the old + "Operationalized, not aspirational" slide). + +### Act III — What makes it actionable in practice (≈6 min) + +12. **Standards = the agent's grammar** — BIDS as the worked example. + `BIDS-minder.svg`; the schema is *data an agent queries*, the + validator is the type-checker. *(marked SKIP CANDIDATE in-file)* + **Ends with the explicit jump into the next two slides**: *"a layout + only tells you what the files are — it says nothing about how they + came to be, or what it cost."* + **Standard → `datalad run` → `duct` = what it is → how it was made → + what it took.** Three layers, and an agent needs all three. This is + the connective tissue Act III previously lacked. +13. **Tracking: `datalad run`** — the RUNCMD commit record. +14. **Provenance doesn't care who wrote the code** — AI-written or + hand-written, same record shape; the tests don't know either. +15. **`con/duct`** — self-contained execution logs. + *(marked SKIP CANDIDATE in-file — the next slide carries the argument)* +16. **…and that log IS the provenance you need later** — three operator + questions answered by the trace rather than by recollection; + `REPRONIM_USE_DUCT=1`. +17. **All three layers, one real artifact — built with AI** (NEW): the + [`dandi/compression-comparisons-t261-study`](https://github.com/dandi/compression-comparisons-t261-study) + worked example, which *is* the BIDS→run→duct bridge in one object. + Left: the BIDS-shaped, YODA-composed layout (tool and 523 GB data as + **subdatasets pinned by commit**). Right: a real `[DATALAD RUNCMD]` + record whose `cmd` **is** a `duct` invocation, so every cell commits + its own resource trace beside its own results. + Verified counts (read from the repo 2026-10-06): **1265 commits · + 1216 `[DATALAD RUNCMD]` · 1095 of those wrapping `duct` · 912 cells, + 0 errors · 3838 h of recorded compute, peak RSS 120 GB · 26 + `Co-Authored-By: Claude` trailers.** +18. **So I tried to `rerun` it. Live, on this laptop.** (NEW) — the + receipt. See §4b below for what actually happened. + +### Act IV — Harnesses, the two ladders, and shared skills (≈8 min) + +19. **Agent = Model + Harness** — credited to Oscar's OHBM deck. + Anatomy table: `AGENTS.md` / skills / hooks / MCP / sandbox, each + mapped onto the STAMPED property it satisfies. +20. **Where are *you* on the ladder?** (NEW, promoted from the appendix) + `ai-ladder-skills.png` beside a condensed L1–L5 table (what it looks + like / the skill you have to acquire). L3 is the 2–3-month tax + nobody skips and is where reusable skills get extracted. + *Punchline:* **L4–L5 is exactly where a bad harness stops being + survivable — invest in the harness, not the model.** + Credit: Jo Van Eyck, *You are Not Ready: Agentic coding in 2026*. +21. **And the loop that gets you up it** (NEW, promoted): the 5-stage + mermaid beside the **development-model mapping** — vibe coding + (2→3→4, knowledge evaporates) / spec-driven (1→2→3→4→2…) / + compound engineering (…→**5**→1+2…, each cycle cheaper). + *Punchline:* **stage 5 is where a skill is born** — and what it + writes down is exactly the artifact slide 11 said must be STAMPED. +22. **That was the *individual* ladder. Here is the *team* one.** (NEW) + The SciOps maturity figure, converted from + `pics/borrowed/neuro-sciops.pdf` → `pics/borrowed/neuro-sciops-fig.svg` + (**vector**; the source PDF is pure vector — 0 rasters, 203 drawing + ops, text already outlined — so SVG keeps it sharp at any projector + scale and in the PDF export, and at 403 KB is *smaller* than the + 439 KB raster it replaced). + **Figure explicitly credited to the SciOps preprint** + (Johnson, Nguyen, Dichter, …, Martone, Pestilli, Yatsenko; + arXiv:2401.00077, 2024) — *not* presented as this talk's own. + The preprint names BIDS / NWB / DataLad / DANDI / brainlife.io as + its own L3 exemplars and reserves L5 for AI+human-in-the-loop. + *Punchline:* **L5 presupposes L3 — no closed-loop discovery on top + of `study_final2/`.** +23. **Everyone is rebuilding the same private harness** — the silo + failure mode, at machine speed. *(marked SKIP CANDIDATE in-file)* +24. **We are already doing this — upstream, in public** — the + `K-Dense-AI/scientific-agent-skills` contribution table. +25. **`con/skills`** — our own domain-agnostic set. + +### Act V — The honest thesis, and close (≈6 min) + +26. **Two regimes — and be honest about which one you are in** (NEW — + *the thesis slide*). Claim: **the safe size of an AI-assisted work + item is set by the strength of your verification loop, and nothing + else.** + - **A. Scaffolded self-verification** — an executable spec the loop + is checked against (spec-kit / OpenSpec / LAD); tests and + validators as the *oracle* (the loop must be able to **fail**); + provenance so drift is detectable; automated gates before human + eyes. *Only then* are large autonomous batches defensible. + - **B. Small, human-reviewable, hand-tried** — a diff a human can + actually read is a **design constraint, not a courtesy**; one + concern per change; **run it by hand before you believe it**; + cheap to back out. *Not a lesser mode — the correct default until + A is actually built.* + - *Punchline:* there is no third regime where large unverified agent + output ships safely; and the way out of B is not a better model + but **A**ctionable specs, **T**racked provenance, **M**odular + units — STAMPED again. + - Kicker, correctly attributed: *"Well, less is more"* — Robert + Browning, *Andrea del Sarto*, **1855**; later Mies van der Rohe's + motto, and he credited Peter Behrens rather than claiming it. +27. **We are not the only ones saying this** (NEW — the receipts; + marked SKIP CANDIDATE, ideal to hold for Q&A). See §4 below. +28. **HI ⇄ AI: every project picks its own policy** — 4-row table. + The *Autonomous* row now names real exemplars: + [**Brain Researcher**](https://brain-researcher.com/) (Poldrack lab; + arXiv:2608.19902) and [**K-Dense AI**](https://github.com/K-Dense-AI/) + (whose skills library we contribute to). Footnote makes the point + that both are **harnesses** — *autonomy is bought with structure, + not with a bigger model*. ICMJE Jan 2026 + melissawm catalog. + Now prefaced with the regime mapping: **rows 1–2 live in regime B; + rows 3–4 are only honest once regime A is actually built.** +29. **Monday checklist** — 6 actions (added: *size the work to the loop + you actually have, and say so out loud so nobody mistakes B for A*). +30. **Acknowledgements** — canonical layout. +31. **Thank you** — STAMPED/Yoda sign-off. + +### Appendix (vertical sub-deck, skipped live) + +- **STAMPED vs. FAIR** — different jobs (demoted from Act II). +- **Portable + Ephemeral** — `datalad containers-run` + + ReproNim/containers (demoted from Act III). +- **AI coding ladder, the long version** — per-level detail, time costs, + the "don't start at L5" warning. +- **BEP028** — provenance lifted into a community standard. +- **`con/yolo`** (ephemerality + portability applied to the agent itself) + and **`con/serve`**. +- **Related CON talks** table. + +## 4. External evidence used (and where) + +| Claim on a slide | Source | +| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | +| Structure beats scale: tool-selection 23.3% → 93.6% across 7 models | Chen, …, Poldrack, *Bringing analytic rigor to agentic AI for science*, [arXiv:2608.19902](https://arxiv.org/abs/2608.19902), 2026 — footnote on the hinge slide | +| SciOps maturity figure + L3 exemplars + L5 = AI-in-the-loop | Johnson *et al.*, [arXiv:2401.00077](https://arxiv.org/abs/2401.00077), 2024 — credited on-slide | +| Agent = Model + Harness; `AGENTS.md` as contract; grounding wins | [Oscar Esteban, OHBM 2026 Educational](https://www.nipreps.org/presentations/2026-OHBM-AI-Educational/) | +| AI coding maturity ladder L1–L5 | [Jo Van Eyck, *You are Not Ready: Agentic coding in 2026*](https://www.youtube.com/watch?v=6W_-YWHKwZ4) | +| Autonomous-stance exemplars | [brain-researcher.com](https://brain-researcher.com/), [K-Dense AI](https://github.com/K-Dense-AI/) | +| AI cannot be an author | [ICMJE, Jan 2026](https://www.icmje.org/recommendations/browse/artificial-intelligence/ai-use-by-authors.html) | + +### Backing for the two-regimes thesis (slide 27) + +| Source | What it supplies | +| ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| [DORA 2025](https://dora.dev/insights/balancing-ai-tensions/) (Google Cloud) | AI adoption raises throughput **and** delivery instability; *"working in small batches is a critical countermeasure to the risks of AI-assisted development"*; *"verification is a fundamentally different cognitive task than creation"* | +| [METR RCT](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/), 2025 | 16 experienced OSS devs, 246 real issues on repos they maintain: **19% slower** with AI while estimating **20% faster**. METR now labels it historical — cite it for the *perception gap*, not as a current benchmark | +| [Addy Osmani](https://addyo.substack.com/p/code-review-in-the-age-of-ai) | *"Proof over vibes"*; verification is the bottleneck; *break agent output into digestible commits*; no PR without tests or a demo | +| [Simon Willison](https://simonwillison.net/2025/Dec/18/code-proven-to-work/) | *"Your job is to deliver code you have proven to work"*; the skill is instructing **and then confidently verifying**; a model cannot take accountability | +| [Faros telemetry](https://www.faros.ai/blog/key-takeaways-from-the-dora-report-2025) (~22k devs) | PR size **+51%**, median time in review **+441%**, incidents per PR **+243%** | + +Attribution note for the kicker: *"less is more"* first appears in +Robert Browning's *Andrea del Sarto* (1855) — **not** Mies van der Rohe, +who adopted it as a motto and himself credited Peter Behrens +(AEG turbine factory façade, ~1907). The common misattribution to Mies +is worth getting right on a slide in a deck that otherwise insists on +sourcing. Robert Venturi's *"Less is a bore"* (1966) is the standard +rebuttal if anyone wants the counter-aphorism. + +## 4b. The rerun experiment (slide 18) — what was actually done + +Run on 2026-10-06, from a clean clone, by someone who is not the author. + +**It worked, and the science came back bit-for-bit.** + +| | | +| --------------------- | ------------------------------------------------------------------------------------------- | +| Cell chosen | `42dc5cfa` — `synthetic-tiny__blosc-zstd-level_3-shuffle_byte` | +| Why that one | `duct` recorded it at **0.5 s / 9 MB RSS**; its only input is a config file in the tool submodule, so none of the 523 GB source data is needed. *The record told me which cells were cheap — that is itself the argument for `duct`.* | +| Setup | `uv tool install datalad`; `uv venv envs/compbench`; `uv pip install './code/compression-comparisons-tools[pipeline]' con-duct` | +| Command | `datalad rerun 42dc5cfa` | +| Result | `run(ok)` — wall clock **0.549 s**, peak RSS **7.8 MB** | +| **Identical** | `cr` 1.5142911224682944 · `encoded_bytes` 5283 · `rmse` 0.0 · `round_trip_ok` true · `signal_std` 706.6094860217612 | +| **Differed** | `encode_time_s` / `decode_time_s` and their derived `*_xrt` — *as they must* | +| **Also differed** | `manifest.json`: timestamp, originating commit, kernel 6.12→7.1, python 3.13.15→3.13.5 | + +### The one STAMPED failure, and it is a real one + +`datalad get code/compression-comparisons-tools` **fails for any outside +cloner**. `.gitmodules` carries `url = ../..`, which git resolves against +the superproject URL to `https://github.com/` — not a repository. It worked +on the author's disk (`datalad-url = /home/yoh/proj/dandi/compression-comparisons`) +and nowhere else. Worked around by cloning +`https://github.com/dandi/compression-comparisons` into place by hand. + +- **S, T, A, M held. D did not.** +- One-line fix: `url = ../compression-comparisons`. +- This is the honest payoff of the whole deck: *you do not find that by + reading your own repo; you find it by being someone else.* Present it as + a finding, not a gotcha. + +**Caveat to state out loud:** I reproduced a 0.5-second smoke cell, not a +17-hour production cell. The claim is "the mechanism works end to end", +not "the whole 3838-hour study was re-verified". + +## 4c. Companion video playlist (ReproNim channel) + +A deliverable in its own right, specified in [`playlist.md`](playlist.md): +an 8-video playlist for the [ReproNim channel](https://www.youtube.com/@repronim) +collecting every video the talk references, so the audience can follow them +from one place. + +Metadata (title, channel, date, duration, **licence**) was read from YouTube +with `yt-dlp` on 2026-10-06 — licences are verified, not assumed, because +they decide what may be mirrored: + +- **4 videos are CC-BY** (all on the DataLad channel: Pragmatic YODA, + DataLad-Registry, git-annex-is-complete, compute-on-demand) → mirrorable. +- **1 is ReproNim's own** (the YODA webinar). +- **3 are Standard YouTube Licence** (Jo Van Eyck; Simon Willison ×2) → + **linked, never mirrored.** + +**ReproTube status: nothing outstanding.** Checked against + on 2026-10-06 — all five +mirrorable videos are *already archived* under their own channels, and the +archive is syncing current (a 2026-10-05 ReproNim video is present). + +One open design item, not urgent: `con/annextube` models **channels** and +mirrors each channel's own playlists; it has no notion of a curated +**cross-channel** playlist. Recommended default is to leave it — every +mirrorable video is already archived, and the playlist is a *view* +("dashboards visualise, don't own"). The alternative — a small annextube +feature storing an ordered id list plus symlinks into existing per-channel +dirs, skipping licence-forbidden videos — is written up in `playlist.md`. + +## 4d. PDF export + +Use the repo-level helper (**not** talk-specific): `tools/mkpdf.py`, +documented in `../README.md` alongside the manual `?print-pdf` and +decktape-via-Docker routes. + +```bash +uv run --with playwright playwright install chromium # once +uv run --with playwright python tools/mkpdf.py 2026-reproAI-MIT-STAMPED.html +``` + +Verified 2026-10-06: **41 pages for 41 slides**, 9.4 MB, SVGs stay +vector, mermaid and slide backgrounds render. Also checked against +`2024-distribits-datalad.html` (16:9, 45 pages) to confirm the page size +really is taken from each deck's own config rather than hardcoded. + +Do **not** commit the PDF — it is a derived artifact and the live HTML is +the thing to share. Generate on demand for anyone wanting an offline copy. + +## 5. Links discipline + +Per `~/.claude/CLAUDE.md` §Slides: every issue/PR/repo/paper reference +on a slide is a real ``; every count links to the live filter +that produces it (e.g. "177 skills" → the repo's `skills/` tree; "15.2% +of DataLad repos carry RUNCMD" → the usage dashboard). A one-liner on +the first content-dense slide tells the audience counts are links. + +## 6. Open TODOs before delivery + +- [ ] Confirm the workshop's official name for the title slide + (placeholder reads "LLM Workshop · MIT"). +- [ ] PR #265 may merge before Oct 16 — re-check its state and flip the + slide's "open" badge if so. +- [ ] Optional: webshot of `K-Dense-AI/scientific-agent-skills` + `skills/bids/SKILL.md` for slide 20 (currently text + links only). +- [ ] Optional: refresh the DataLad usage-registry RUNCMD percentage. +- [ ] `pics/borrowed/neuro-sciops-fig.svg` is **derived** from the annexed + `pics/borrowed/neuro-sciops.pdf`. Regenerate with: + `uv run --with pymupdf python -c "import pymupdf; open('pics/borrowed/neuro-sciops-fig.svg','w').write(pymupdf.open('pics/borrowed/neuro-sciops.pdf')[0].get_svg_image(text_as_path=True))"` +- [ ] Publish the companion YouTube playlist on the ReproNim channel — + see [`playlist.md`](playlist.md) for the ordered list, verified + licences, the creation recipe, and the ReproTube archival status. + Then backfill the playlist URL into this file, `playlist.md`, + `INDEX.md` and the deck's appendix. +- [ ] Optionally report the `.gitmodules` `url = ../..` bug upstream on + `dandi/compression-comparisons-t261-study` (one-line fix). Doing so + *before* the talk would be a nice "and it is already fixed" beat; + leaving it lets the slide stand as a live example. Speaker's call. +- [ ] Timing: **34 main slides for 25 min — this is now well over budget.** + Four slides carry a `SKIP CANDIDATE` note in their + ` + +
+
+
-
-

2016 — BIDS. Co-founding a community data standard.

-
+ +
+
-

Two decades, five verbs

- - - - - - - - - - - - - - - - - - - - - -
Reusejoin an upstream instead of writing one
Composesmall modules over silo’d monoliths
Extendstay on as maintainer of what you depend on
Standardizea common language — for humans and agents
Automateif you don’t, it doesn’t happen at scale
- + +
+ + + M. Hanke, Y. O. Halchenko, P. B. Sederberg, et al. PyMVPA: A Python toolbox for + multivariate pattern analysis of fMRI data. Neuroinformatics, 7:37–53, 2009. + doi: 10.1007/s12021-008-9041-y +
+
+

2007. Full test suite. CI. Tutorials. Before it was common.

+
+
+ +
+
+

REUSE

- - -
+

The cheapest reproducible thing is the one you didn’t have to build.

- - -
+
@@ -277,131 +267,123 @@

The cheapest reproducible thing is the one you didn’t have to build.

- - -
+

Born by joining Debian. Not forking it.

- - -
+

NeuroDebian: a textbook bridge

Eventually most of it dissolved upstream into Debian Med, Debian Science, and conda-forge. A successful bridge dissolves into the commons.

- - -
- -
- - - M. Hanke, Y. O. Halchenko, P. B. Sederberg, et al. PyMVPA: A Python toolbox for - multivariate pattern analysis of fMRI data. Neuroinformatics, 7:37–53, 2009. - doi: 10.1007/s12021-008-9041-y -
- - -
-

2007. Full test suite. CI. Tutorials. Before it was common.

-
- - -
-

duecredit

-

Automated scholarly credit tracking.

-
- -
-

con/duct

-

Process monitor. Wall time. Memory. Per task.

-
- -
-

citations-collector

-

Citation harvesting for your stack.

-
- - -
+

One upstream search pass saves a thousand local fixes.

- - -
+

REUSE

Debian DEP-5 → REUSE spec: machine-readable licensing, per file.

reuse.software

- - -
+
/introduce-reuse-compliance

We reuse the standard and automate its adoption.

github.com/con/skills

- - -
-

“What if AI attribution worked the same way?”

+
+

REUSE

- - -
-
SPDX-FileContributor: assisted-by-ai: Claude Code 2.1 / Claude Sonnet 4.6
-

File-level AI provenance that survives redistribution.

- +
+

Compose — small modules, common substrate

+

Sandwich, don't silo.

- - -
-

We filed the issue.

-

fsfe/reuse-website#127

+
+

(DE)COMPOSE

+
- - - - -
-

“Agents don’t search. They generate.”

+
+

Extend — stay on as maintainer

+

Ship pragmatic now, formalize upstream later.

+
+
+

EXTEND

+
+
+

Not an ending — a handover upwards

+
+ Debian + Debian Med + Debian Python Team +
+

+ Debian Med · + Debian Science · + Debian Python Team +

+

Small team, wide scope → big teams, each with a narrower one.
+ Packages we used to carry alone are now co-maintained: more hands, a bus factor above one.
+ Later — psychopy moves to Debian Med in 2020; + nifticlib, gifticlib, odin, cmtk follow over the next two years.

+
- -
-

Reuse requires asking “does this exist?” before the first keystroke.

+
+ +
+ DebConf10, New York City, 2010. This is what “joining” looks like. +
+
- -
-

REUSE

+
+
+ Later — DebConf15, Heidelberg, 2015. Still going. +
+
+
+ +
+
+

The DataLad sandwich

Layered tech that everyone already knows

@@ -429,9 +411,7 @@ 

Layered tech that everyone already knows

DataLad: distributed system for joint management of code, data, and their relationship. Halchenko et al., JOSS 2021.

- - -
+

DataLad extensions: domain-agnostic core, à la carte add-ons

 graph LR
@@ -460,294 +440,232 @@ 

DataLad extensions: domain-agnostic core, à la carte add-ons

datalad-extension-template — your domain plugs in, neuroimaging is just one of many.

- - -
+

Federate, don’t recentralize

registry.datalad.org indexes DataLad datasets across institutions and clouds — discovery over petabytes, no platform required.

+
+

Standardize — a language for HI and AI

+

All standards are “bad”, but some are used.

+

after D. Clunie, MICCAI 2017

+ +
+
+

Data: BIDS & NWB

+ +
+ + + Gorgolewski, K. J., Auer, T., …, Halchenko, Y. O., …, Poldrack, R. A. (2016). + The brain imaging data structure, a format for organizing and describing outputs of + neuroimaging experiments. Scientific Data, 3:160044. + +

You've seen one BIDS dataset, you've seen them all. That's the point.

+
+
+

BIDS as a meta-standard

+ +

Not a wire format — a contract. Validators, BIDS-Apps, BEPs all hang off it.

+
+
+

M

+

Schema. Machine-readable. Deterministically validatable.

+

Examples: BIDS, NWB, LinkML

+
+
+

C

+

Tools that operate on the Model.

+

Examples: BIDS-Apps, DataLad, HeuDiConv

+
+
+

V

+

Use-case views on the same data.

+

Examples: lab website, reports, discovery portals

+
+
+

+ Same M → endless C and V.
This is why standards compound. +

+ +
+
+
+ + +
+
+
+

ReproIn + HeuDiConv

Acquire reproducibly.

Sequence naming convention at the scanner → DICOM → BIDS, no human in the loop.

- -
-

NeuroConv + nwb2bids

-

Convert once, use everywhere.

-

Raw physiology → NWB; NWB → BIDS layout. Two flagship standards, one bridge.

-
- -
+

ReproNim/containers

Pinned environments. Everywhere.

Curated Singularity/Apptainer containers, auto-rebuilt and auto-snapshotted.

- - -
+

Small units. Common substrate.
No glue code you own alone.

- - -
+

“AI generates glue. Nobody owns the glue.”

- -
+

Deliberate interfaces outlast generated adapters.

- - -
-

COMPOSE

-
- -
- - - -
- -
-

Extend — stay on as maintainer

-

Ship pragmatic now, formalize upstream later.

-
- - -
-

Debian Med

-

Packaging. Still there.

-

pkg-exppsy → NeuroDebian → Debian Med & Debian Science → conda-forge.
- A successful bridge dissolves into the commons.

+
+

ReproMan

+

Run it there. Record what “there” was.

+

Started 2016 as NICEMAN. Submit to containers, AWS, SLURM/condor — and collect the + environment (Debian and conda packages, VCS revisions) in enough detail to re-create it.
+ M. Travers, R. Buccigrossi, C. Haselgrove, K. Meyer, and Y. O. Halchenko. + NICEMAN: NeuroImaging Computational Environments Manager, OHBM 2018.

+
-

BIDS Steering Group + WG-16

-

Advisory. Standards evolution.

-

Co-evolving the standard, not just consuming it. - BEP028: DataLad’s ad-hoc - RUNCMD provenance lifting into BIDS.

+

ReproMan’s afterlife: con/duct

+

+ ReproMan asked what did that remote run actually cost?
+ brainlife’s smon showed how to sample it.
+ 2024: duct <cmd> — no third-party dependencies, uniform records. +

+

Later, well out of this era — but it belongs to this story, not to 2024. + Already adopted inside ReproNim/containers and + ReproStim; RRID:SCR_025436.
+ Poster this week: Austin Macdonald, C. C. Baker, J. A. Lee, Y. O. Halchenko, + Agent-friendly Provenance Capture with con-duct. Go find Austin.

+
- -
+

NWB + Kitware

Academic RSE ↔ engineering industry.

Sustained collaboration on browse / analyze / visualize tooling.

- - -
-

From ad-hoc to standard, in one slide

-
-flowchart LR
-  RUNCMD["DataLad RUNCMD
(2015, ad-hoc JSON in commit msgs)"] --> BEP["BEP028
BIDS provenance"] - RUNCMD --> EXP["bids-prov-exporter
(WiP)"] - BEP --> COMMUNITY["BIDS standard
(community-owned)"] - EXP --> COMMUNITY - classDef orange fill:#ffa200,stroke:#333,stroke-width:1px; - classDef green fill:#40bf4c,stroke:#333,stroke-width:1px; - class RUNCMD orange - class BEP green - class COMMUNITY green -
-

- Take home: the next time you reach for your own JSON-in-a-commit-message format, - ask which upstream standard it should fold back into. -

+
+ +

NWB — Neurodata Without Borders

+ +

Standard for neurophysiology. Same principle.

- - -
-

“AI extends by adding. It rarely stays to maintain.”

+
+

STANDARDIZE

-
-

- Staying on as maintainer is the non-automatable contribution. -

-
+
+ +
+
+
+

NeuroConv + nwb2bids

+

Convert once, use everywhere.

+

Raw physiology → NWB; NWB → BIDS layout. Two flagship standards, one bridge.

- -
-

- Same M → endless C and V.
This is why standards compound. -

- +
+

BIDS Steering Group + WG-16

+

Advisory. Standards evolution.

+

Co-evolving the standard, not just consuming it. + BEP028: DataLad’s ad-hoc + RUNCMD provenance lifting into BIDS.

- - -
- -

For research objects, we wrote the standard ourselves. - stamped-principles.org

+
+

From ad-hoc to standard, in one slide

+
+flowchart LR
+  RUNCMD["DataLad RUNCMD
(2015, ad-hoc JSON in commit msgs)"] --> BEP["BEP028
BIDS provenance"] + RUNCMD --> EXP["bids-prov-exporter
(WiP)"] + BEP --> COMMUNITY["BIDS standard
(community-owned)"] + EXP --> COMMUNITY + classDef orange fill:#ffa200,stroke:#333,stroke-width:1px; + classDef green fill:#40bf4c,stroke:#333,stroke-width:1px; + class RUNCMD orange + class BEP green + class COMMUNITY green +
+

+ Take home: the next time you reach for your own JSON-in-a-commit-message format, + ask which upstream standard it should fold back into. +

- - -
-

“Without standards, AI produces cacophony at scale.”

+
+

“AI extends by adding. It rarely stays to maintain.”

- -
+

- Standards are the framing and validation layer for any AI endeavor. + Staying on as maintainer is the non-automatable contribution.

- - -
-

One instance of the pattern:
lab knowledge management.

-
- -
+

LinkML

Schemas that compose.

Built 2 hours north, at LBNL (Lawrence Berkeley National Laboratory).

- -
+

ORCID

Reused identifier, not invented.

- -
+

ROR

Reused identifier, not invented.

- -
+

DOI + PROV-O

Reused identifier, not invented.

- -
-

ORINOCO → ORINOCO-Lite

- -

INM-7 Jülich → CON, GitHub-native

-

Reusing ORINOCO (Michael Hanke / INM-7) and composing with GitHub Actions/Pages.

-
- -
-

- AI drafts candidate records.
Schema validates.
Humans curate via PR. -

-
- - -
-

- Posters this week:
- ORINOCO-Lite · STAMPED · con/duct
- Come find us. -

-
- - -
-

- Standards give both HI and AI a shared grammar. -

-
- - -
-

STANDARDIZE

-
- -
- - - -
- - -
+
EMBER @@ -755,81 +673,34 @@

STANDARDIZE

National archives. Built on shared standards.

- -
+

What's in DANDI

- -
+

DANDI ecosystem — integrate, don't re-implement

- -
- - - - - - -
- -
+

Automate — or it doesn’t happen at scale

If a human has to remember it, it’s already broken.

- - -
+

con/tinuous

“CI logs archived. Every PR. Every build.”

- -
-

git-annex daily tests

+
+

git-annex daily tests

“DataLad and git-annex: tested daily. Automatically.”

- -
-

ReproNim/containers

-

“Auto-rebuilt. Auto-snapshotted. Always current.”

-
- -
+

dandisets mirror

“Auto-mirrored. Petabytes. No human in the loop.”

- - -
+

Cost: harnesses, harnesses, harnesses

Each automation is itself code — CI YAML, hooks, @@ -843,83 +714,173 @@

Cost: harnesses, harnesses, harnesses

- -
+

con/skills + con/yolo: we automate the harness itself.

- - -
+

“The tests don’t know the code was AI-generated.”

- - -
+

“Agent-rate output demands agent-rate review.”

- - -
+

“con/duct watches every process. AI agents included.”

+
+

AUTOMATE

+
+
+
+ + +
+
+
+

duecredit

+

Automated scholarly credit tracking.

+
+ +
+

citations-collector

+

Citation harvesting for your stack.

+
+
+

“What if AI attribution worked the same way?”

+
+
+
SPDX-FileContributor: assisted-by-ai: Claude Code 2.1 / Claude Sonnet 4.6
+

File-level AI provenance that survives redistribution.

+ +
+
+

We filed the issue.

+

fsfe/reuse-website#127

+
+
+

“Agents don’t search. They generate.”

+
+
+

Reuse requires asking “does this exist?” before the first keystroke.

+
+
+ +

For research objects, we wrote the standard ourselves. + stamped-principles.org

+
+
+

“Without standards, AI produces cacophony at scale.”

+
+
+

+ Standards are the framing and validation layer for any AI endeavor. +

+
+
+

One instance of the pattern:
lab knowledge management.

+
+
+

ORINOCO → ORINOCO-Lite

+ +

INM-7 Jülich → CON, GitHub-native

+

Reusing ORINOCO (Michael Hanke / INM-7) and composing with GitHub Actions/Pages.

+
+
+

+ AI drafts candidate records.
Schema validates.
Humans curate via PR. +

+
+
+

+ Posters this week:
+ ORINOCO-Lite · STAMPED · con/duct
+ Come find us. +

+
+
+

+ Standards give both HI and AI a shared grammar. +

+
+
+

Reuse, in reverse

+

Your data is yours, even when someone else hosts it.

+
+
+

Pull content back: AnnexTube / mykrok / con/serve

+

YouTube → DataLad. Cloud silos → git-annex. AI sessions → subdatasets.

+
+

con/duct — wall time, CPU, memory, per task. Per agent run.

- -
+

“con/tinuous archives every run. AI-assisted runs included.”

- -
+

“datalad run records every transformation. Agent-written or hand-written.”

- - -
+

“Provenance doesn’t care who wrote the code.”

+
-
@@ -958,18 +919,7 @@

You are alone.

-
- - -
- -
- -
+
@@ -1101,16 +1051,24 @@

HI ↔ AI — every project picks its own policy

- Pryor, R. et al. (2026). arXiv:2607.13220 + Choudhury, S. et al. (2026). arXiv:2607.13220

-

- “Scale the connections, not just the agents.” +

+ Scale the connections, not just the agents. +

+

+ “The open problem is therefore not only how to scale models, but how to develop + ‘networked intelligence’, scaling the connections between humans and AI systems + so that a result or hypothesis produced in one context reaches another person, agent, + instrument or robot that can act on it.”

- +
@@ -1137,73 +1095,18 @@

HI ↔ AI — every project picks its own policy

- - +
-

Acknowledgements

- - - - - -
-
-
Software & communities
-
-
    -
  • Joey Hess (git-annex)
  • -
  • The DataLad team & contributors
  • -
  • Debian / Debian Med / Debian Science / conda-forge
  • -
  • BIDS & NWB communities
  • -
  • distribits community
  • -
-
-
Some slides origin:
-
- -
-
-
-
Funders
- - - - -
- - - - -
-
Collaborators
-
- - - - -
-
- - - - -
-
+

Acknowledgements

+
- -

Thank you!

Reuse. Compose. Extend. Standardize. Automate.

@@ -1215,12 +1118,9 @@

Reuse. Compose. Extend. Standardize. Automate.

Slides: datasets.datalad.org/…/2026-usrse-con-talk.html — CC-BY-SA

- - + diff --git a/2026-usrse-con-talk/ack-2004.html b/2026-usrse-con-talk/ack-2004.html new file mode 100644 index 00000000..97cd8eab --- /dev/null +++ b/2026-usrse-con-talk/ack-2004.html @@ -0,0 +1,34 @@ + + +
+Centroids +
+ \ No newline at end of file diff --git a/2026-usrse-con-talk/ack-2008.html b/2026-usrse-con-talk/ack-2008.html new file mode 100644 index 00000000..1971d454 --- /dev/null +++ b/2026-usrse-con-talk/ack-2008.html @@ -0,0 +1,44 @@ + + +
+Centroids +
+
+Collaborators +
+
+Large communities we take part in +
+Debian +
+ \ No newline at end of file diff --git a/2026-usrse-con-talk/ack-2012.html b/2026-usrse-con-talk/ack-2012.html new file mode 100644 index 00000000..e01619ed --- /dev/null +++ b/2026-usrse-con-talk/ack-2012.html @@ -0,0 +1,58 @@ + + +
+Centroids +
+
+Collaborators +
+
+Affiliated Faculty +
+
+Large communities we take part in +
+Debian +scikit-learn +PsychoPy +
+
+Funders and in-kind support +
+Dartmouth College +PBS +DBIC +
+ \ No newline at end of file diff --git a/2026-usrse-con-talk/ack-2016.html b/2026-usrse-con-talk/ack-2016.html new file mode 100644 index 00000000..7f475297 --- /dev/null +++ b/2026-usrse-con-talk/ack-2016.html @@ -0,0 +1,80 @@ + + + +
+Collaborators +
+
+Affiliated Faculty +
+
+Emeritus +
+
+Worked on directly +
+DataLad +ReproNim +BIDS +
+
+Large communities we take part in +
+Debian +scikit-learn +PsychoPy +
+
+Funders and in-kind support +
+Dartmouth College +PBS +CCN +DBIC +NSF +NIH +INCF +
+ \ No newline at end of file diff --git a/2026-usrse-con-talk/ack-2020.html b/2026-usrse-con-talk/ack-2020.html new file mode 100644 index 00000000..d88651db --- /dev/null +++ b/2026-usrse-con-talk/ack-2020.html @@ -0,0 +1,99 @@ + + + + + + +
+Worked on directly +
+DataLad +ReproNim +DANDI +BIDS +NWB +
+
+Large communities we take part in +
+Debian +Debian Med +scikit-learn +PsychoPy +HCP +
+
+Funders and in-kind support +
+Dartmouth College +PBS +CCN +DBIC +NSF +NIH +INCF +AWS Open Data +
+ \ No newline at end of file diff --git a/2026-usrse-con-talk/ack-2024.html b/2026-usrse-con-talk/ack-2024.html new file mode 100644 index 00000000..69ee0fef --- /dev/null +++ b/2026-usrse-con-talk/ack-2024.html @@ -0,0 +1,108 @@ + + + + + + +
+Worked on directly +
+DataLad +ReproNim +DANDI +EMBER +BIDS +NWB +
+
+Large communities we take part in +
+Debian +Debian Med +scikit-learn +PsychoPy +HCP +
+
+Funders and in-kind support +
+Dartmouth College +PBS +CCN +DBIC +NSF +NIH +INCF +AWS Open Data +
+ \ No newline at end of file diff --git a/2026-usrse-con-talk/ack-2026.html b/2026-usrse-con-talk/ack-2026.html new file mode 100644 index 00000000..ccbf0d15 --- /dev/null +++ b/2026-usrse-con-talk/ack-2026.html @@ -0,0 +1,109 @@ + + + + + + +
+Worked on directly +
+DataLad +ReproNim +DANDI +EMBER +BIDS +NWB +
+
+Large communities we take part in +
+Debian +Debian Med +scikit-learn +PsychoPy +HCP +
+
+Funders and in-kind support +
+Dartmouth College +PBS +CCN +DBIC +NSF +NIH +INCF +AWS Open Data +
+ \ No newline at end of file diff --git a/2026-usrse-con-talk/ack-now.html b/2026-usrse-con-talk/ack-now.html new file mode 100644 index 00000000..95c8bfe2 --- /dev/null +++ b/2026-usrse-con-talk/ack-now.html @@ -0,0 +1,109 @@ + + + + + + +
+Worked on directly +
+DataLad +ReproNim +DANDI +EMBER +BIDS +NWB +
+
+Large communities we take part in +
+Debian +Debian Med +scikit-learn +PsychoPy +HCP +
+
+Funders and in-kind support +
+Dartmouth College +PBS +CCN +DBIC +NSF +NIH +INCF +AWS Open Data +
+ \ No newline at end of file diff --git a/2026-usrse/2026-usrse-con-talk-associations-plan.md b/2026-usrse/2026-usrse-con-talk-associations-plan.md new file mode 100644 index 00000000..f1344e02 --- /dev/null +++ b/2026-usrse/2026-usrse-con-talk-associations-plan.md @@ -0,0 +1,412 @@ +# Plan: community associations as the backbone of the US-RSE'26 CON talk + +Status: **in progress.** Written 2026-10-10, state updated 2026-10-10. +See §0 for where things actually stand; the phases in §5 are the plan, §0 is +the truth. + +## 0. Current state + +Commits so far: + +| repo | commit | what | +| --------- | --------- | --------------------------------------------------------------------- | +| `talks` | `fe21f2c` | usrse deck adopts the dhmc acknowledgements slide verbatim | +| `talks` | `6a5be4f` | one "Funders and in-kind" row, historical order | +| `talks` | `79df3c7` | official PBS and DBIC marks replace the text placeholders | +| `talks` | `17f3903` | YOH back among the Centroids; plan state recorded | +| `talks` | `9a5d9fe` | generated project bands replace the hand-placed logo strip | +| `talks` | `d22d638` | INCF moves to in-kind; BIDS gets its real mark | +| `con.org` | `dd2cad5` | first association schema, harvester, validator (superseded) | +| `con.org` | `6c18ecf` | defect fixes against the first review | +| `con.org` | `0cf3004` | people and project records extracted, both round-trip clean | +| `con.org` | `c5840ca` | curated slide selection; decision to adopt the datalad-concepts shape | +| `con.org` | `3d7d5d3` | INCF to in-kind, BIDS logo override | +| `con.org` | *(head)* | records in the real schema; evidence as JSON-Pointer overlays | + +Phase status: + +| # | phase | state | +| - | -------------------------------------------------- | ---------------------------------------------- | +| 0 | branch check, mounts, this plan | done | +| 1 | association schema | **done** — see §3, it is not ours | +| 2 | evidence harvest | PsychoPy and CON converted and validated | +| 3 | project logos and group photos | logos done for the 10-project selection | +| 4 | `mk_whoweare.py` rewrite (YAML-driven, `--select`) | superseded by `scripts/render_ack_projects.py` | +| 5 | backbone slides from selections | acknowledgements band done; rest not started | +| 6 | playwright verification | harness in use throughout | + +### The finding that reshaped phases 1-2 + +`ORINOCO-Lite/con-site-specific` is not a future destination — it is a live +corpus of **24 `XYZProject`, 35 `XYZPerson`, 8 `XYZOrganization` and 9 +`XYZAgentRole` records for CON**, already carrying `associated_with` with +`object` and `roles`, plus `metadata/overlays/` and `curation-records/`. + +Everything the first association schema invented already existed there in +better form, and the one thing missing from it is exactly what we set out to +add: **temporal extent, and provenance of the date.** So the model was +replaced rather than patched — see `con.org:content/associations/SCHEMA.md`, +which is now the reference. §3 below is kept only as the record of what was +tried. + +### Defects cleared + +1. `psychopy.yaml` corrected (contributor phase to 2018-04-06, Hanke's 2008 + `UNRELEASED` entry attributed to him, the evidence-free `user` claim moved + to a `pending:` block that is explicitly not asserted). +2. The validator no longer crashes on an empty file, a non-mapping top level, + or malformed YAML; fault injection confirms nine distinct failures are + caught and a clean tree exits 0. +3. The bound rule was not just buggy, it was **backwards**: an artifact shows + the oldest or newest surviving trace, never the event, so a `started` from + a first commit is an `upper` bound and an `ended` from a last commit a + `lower` one. Six bounds in the PsychoPy evidence were wrong. The validator + now enforces direction per slot. +4. `supports: [start]` replaced by a JSON Pointer naming the exact slot, so + coverage is checked rather than a label trusted. + +### Still open + +1. **HED has no record anywhere on the site** (`grep` over `content/pages` + for hedtags / "hierarchical event descriptor" / `>HED<` returns zero). It + was explicitly asked for on the slide. Fix is to add it to `projects.html` + under Standards beside BIDS and NWB; commented out in the selection until + then so the omission is visible. +2. The `xyzrins:` namespace is literally the *demo* namespace, yet all 67 + con-site-specific records use it. Kept for byte-compatibility; needs a + decision. +3. Whether same-pid records merge across stores — decides whether the + additive `records/XYZOrganization/con.yaml` (pid plus one attribute) is + legitimate. +4. Whether a handover belongs in `End`'s Trigger slot. The one mapping made + by interpretation rather than by matching an existing use. +5. Associations still to write: JB ↔ INCF, Dave Kennedy ↔ NITRC, and the + remaining projects in the slide selection. +6. `published_anchor` from Pelican's `output/*.html` on every person/project + record; `anchor_id` demoted. + +### Evidence already in hand + +- `fsl/debian/changelog` bottoms out **2005-12-28** (Hanke); `pyepl` + **2006-06-18** (YOH) — corroborating the "2005 — pkg-exppsy" line already in + our decks. +- PsychoPy upstream `4178d2b09`, *"starting to add unit tests (for Yaro ;-) )"*, + dates the testing push to **2009-10-02 in upstream's own words**; `6048ffd0b` + credits a butterworth-filter contribution by name; `neurodebian/*` pull + requests were merged upstream 2015–2018. +- `neurodebian/sphinx/quotes-*.rst`: **50 dated testimonials, 2007–2011**, ten + from leads of roster projects (Perez/IPython, Peirce/PsychoPy, + Kleiner/Psychtoolbox, Varoquaux and Thirion/Parietal, Buccigrossi/NITRC). + Already public at `neuro.debian.net/testimonials.html`, so citable. Extracted + to `reports/neurodebian-testimonials.json`. **This is the strongest asset + found for the talk's thesis** — named project leads on the record about what + NeuroDebian meant to them. +- Dartmouth-side facts verified from primary sources: Haxby directed DBIC from + January 2008 "until June 2021", Tor Wager "In July of 2021"; Wager is + "Director, Center for Cognitive Neuroscience and Dartmouth Brain Imaging + Center"; CON is listed among CCN's Associated Programs. + +### Blocked on answers + +1. **CON's founding year**, with a citable artifact. The site repo only bounds it + to "by 2015-07-21". This decides every `actor: yoh` vs `actor: con` call. +2. Canonical location: embedded JSON-LD in `projects.html` (recommended) vs a + sidecar YAML. +3. Review vehicle: one PR on `con.org` vs per-project commits. +4. `2026-usrse-con-talk-enh` — merge, cherry-pick the Makefile, or drop. +5. Do Dartmouth / PBS / CCN / DBIC / AWS get `support` records? They are neither + collaborators nor funders in the same sense, and AWS's support is + per-archive (DANDI, OpenNeuro, EMBER), not to CON. + +### Side findings worth acting on separately + +- `projects.html` uses `id="dandi"` **three times** (DANDI, EMBER, OpenNeuro). + Pelican overwrites them so the live page is fine, but it is misleading in + source — records key off the generated slug instead. +- `dartmouthpbs.org` (PBS Knowledge, built by Cosanlab) describes the Center for + Open Neuroscience as *"The Center for Open **Science** …"*, and YOH's faculty + node is empty. Email drafted to Luke Chang; **sent**. +- CCN's staff page lists YOH as "Adjunct and Research Associate Professor"; PBS's + directory says "Research Professor". Email drafted to Courtney Rogers. + +## 1. Goal + +Make the talk's thesis — *community matters more in the age of AI, not less* — +visible rather than asserted, by positioning CON's output as dependent on long +ties to many open-source projects. + +Mechanically that means: + +1. Enrich the CON website with machine-readable metadata about **when and how** + CON (and Yarik personally) was associated with each community project, with + provenance for every claim. +2. Derive a reviewable YAML from that, and drive the acknowledgement slide from + the YAML instead of hand-tuned HTML. +3. Make the acknowledgement slide *selectable*, so the talk can walk its own + history: start with Yarik + Haxby + Hanke (the PyMVPA / NeuroDebian / DataLad + cradle), then expand into ReproNim / INCF / OpenNeuro, and so on. + +The online reference for the current hand-tuned slide: + + +## 2. Findings that shape the plan + +### 2.1 Branch state — nothing new on any remote + +`2026-usrse-con-talk` and local `master` are both `789f540`. The branch was +merged and **deleted on GitHub** (a `git fetch gh-con --prune` pruned it). +Divergence of local `master`, as `ahead / behind`: + +| remote | ahead / behind | +| ------------------------------------ | -------------- | +| `gh-con/master` | 4 / 0 | +| `hub-con/master` | 4 / 0 | +| `datasets.datalad.org/master` | 4 / 0 | +| `datasets.datalad.org/synced/master` | 6 / 0 | +| `smaug/master` | 6 / 0 | + +Zero behind everywhere. `smaug` and `datasets.datalad.org` are ssh remotes, so +those two rows are from the last fetch, not freshly verified. + +Loose end: **`2026-usrse-con-talk-enh`** is local-only and unmerged — 6 commits +adding `2026-usrse/Makefile` (md to HTML/PDF) and rewriting +`talk-proposal-draft.md` into ventilated prose. Tip message: *"Claude+I's +attempt to shorten it, not sure I like it yet"*. + +### 2.2 What already exists (do not duplicate) + +- `2026-usrse/community-slide-spec.md` — tiers 0-4, a `community-slide-data.yaml` + model with `people[].communities`, and an explicit "mass figures at the bottom" + intent. This plan **supersedes** it; fold it in rather than keep two specs. +- `ORINOCO-Lite/con-site-specific` is further along than "planned": it already + has `metadata/records/XYZProject/*.yaml` (25), `XYZPerson` (36), + `XYZOrganization` (9), `XYZAgentRole` (10), plus `sources/`, + `curation-records/` and provenance overlays. Associations exist as + `dlthings:Association` with `roles`, e.g. `bids.yaml` carries + `object: xyzrins:persons/yaroslav-halchenko, roles: [marcrel:ctb]`. + **What is missing there is exactly what we want to add: temporal extent and + provenance-of-the-date.** The YAML below must convert cleanly into these. +- `content/pages/projects.html` already has 31 entries across Software / + Initiatives / Meetings / Standards / Education / Infrastructure, each a + `
` with an `

` anchor and `class="centroid"` + person links. Several wanted additions (BIDS, EMBER, PyMVPA, distribits) are + already there; INCF and NITRC are under Partners on `whoweare`. So this is as + much **restructuring** as adding. + +### 2.3 Repositories + +| path | status | why it is needed | +| ---------------------------- | ---------- | ---------------------------------------------------------------- | +| `/home/yoh/proj/CON/talks` | present | slides, `tools/mk_whoweare.py`, these notes | +| `/home/yoh/proj/CON/con.org` | **absent** | the write target; all site enrichment happens here | +| neurodebian clone (path TBD) | **absent** | `debian/changelog` first-upload dates — the real dating evidence | +| `con-site-specific` clone | **absent** | shape the YAML to its records so migration is mechanical | + +All three are reachable read-only over the web (github, `neuro.debian.net`, the +copyright extracts), but writes need `con.org` mounted locally. + +## 3. Data model + +> **Superseded.** The model below was written before +> `con-site-specific`'s records were found. It is kept as the record of what +> was tried and why it was dropped; the live reference is +> `con.org:content/associations/SCHEMA.md`. In short: `phases` was a list of +> associations, `actor` was a reinvention of `Association.object`, `role` was +> already multivalued and IRI-typed, `transitioned_to` is `End`'s Trigger, +> and `supports:` is a JSON Pointer. + + +### 3.1 Associations are phases, not a start/end pair + +The recurring story is **transition**, not a clean stop. PsychoPy is the +archetype: inception-era involvement and the push to add unit testing (recalled +as a Yale hackathon — needs verification), then NeuroDebian packaging, then +offloaded to Debian Med, and we remain users. scikit-learn (SVN to git migration +via NeuroDebian, then veined off) and Jupyter/IPython (via NiPy around 2008, then +Debian/NeuroDebian, drifting to user-only somewhere 2015-2020) follow the same +shape. + +So each `(CON <-> project)` record holds a **list of phases**: + +```yaml +project: psychopy +phases: + - role: contributor # contributor | maintainer | packager | founder | user | steering + via: direct # direct | neurodebian | datalad | repronim | dandi | ... + start: 2009-?? # ISO 8601, partial allowed + end: ~ # null = ongoing + status: transitioned # active | transitioned | user-only | dormant + note: initiated unit testing + provenance: + - source: https://... + kind: recollection # debian-changelog | copyright-extract | site-git-history | + # publication | recollection + confidence: unverified # certain | approximate | unverified +``` + +**Rule: no date without at least one `provenance` entry.** Recollections are +recorded as recollections and flagged for human confirmation, never silently +promoted to fact. + +### 3.2 AI-contribution stance + +Researched rather than invented. The draft `ai-contribution-policy.yml` standard +defines four stance values verbatim; the 281-policy landscape study adds +`encouraged` as a distinct category (7.1% of projects actively promote AI use, +against 76.2% permit and 14.9% forbid). Combined, with one honest addition: + +| value | meaning | informal term | +| ------------------- | -------------------------------------- | -------------- | +| `encouraged` | policy actively promotes AI assistance | "AI embracing" | +| `unrestricted` | permitted, no disclosure expected | | +| `disclosed-allowed` | permitted if disclosed and attested | "AI cautious" | +| `restricted` | allowed only in stated circumstances | | +| `prohibited` | not accepted | "AI averse" | +| `unstated` | **no published policy** | — | + +`unstated` is non-negotiable: it is the majority case and must never be +collapsed into either extreme. + +Two framings worth borrowing from that literature, because they fit the talk: +**verification over detection** (attest "I ran the tests, I understand this +change"; never try to answer "was this AI?", which is unanswerable) and +**disclosure over bans** — the bans are what close contribution pathways that +took thirty years to open. + +### 3.3 Projects carry a stance; people carry quotes + +A project's stance is a published fact with a URL. A *person's* stance, absent a +public statement, is an inference about a named individual that would end up on a +public website and a conference slide, attached to people whose collaboration the +talk celebrates. Getting that wrong damages exactly the ties the talk is about. + +- **Projects** — `ai_contribution_stance` from the enum, with `source_url` and a + `captured` date (policies change fast; the landscape study tracked 196 + policy-change commits). No policy found means `unstated`. +- **People** — no scale. A list of *stated positions*: + `{stance, scope, quote, source_url, date}`, where `scope` records what they + were speaking for (their project, their lab, contributions they accept). + No public statement means the field is **absent**, not `prohibited`. + +Named examples still needing a citable artifact before they go in: Dimitri +Yatsenko / DataJoint and Satrajit Ghosh. Check whether the SciOps maturity model +paper already cited in these decks (arXiv 2401.00077; Johnson, ..., Halchenko, +..., Yatsenko) carries a usable statement, rather than asserting from reputation. + +CON is the one case where we *are* the authority: declaring CON's own stance — +possibly by adopting `ai-contribution-policy.yml` in CON repos — is both honest +and a talk beat. + +### 3.4 Where the canonical data lives + +Proposal (**decision 1 below**): a hidden +`