From b759b2db54289a2107364ad8279274901919eaa8 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Fri, 22 May 2026 13:55:52 -0400 Subject: [PATCH 01/27] Add talk-proposal-mvc.md (sole-authored US-RSE proposal) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Companion to talk-proposal-draft.md: where the history-arc proposal asks "how did we get here", this MVC-spined proposal asks "why does it compose, and what makes standards load-bearing?". Sole author (Y. Halchenko). Re-reads the CON stack as Model–View–Controller at the stack scale, with extra emphasis on metadata schemas (LinkML, pydantic, JSON Schema, SHACL, DataLad Concepts), schema-driven UIs (vjsf → DANDI meditor; shacl-vue from psychoinformatics-de), the con/validation harmonization recursion, and the "service-tied UI" anti-pattern contrast (Cockburn's Hexagonal Architecture; Evans's Smart UI; Fowler's Service Layer / Anemic Domain Model; GoF Adapter / Strategy / Façade). Closes with the six-lens table (MVC / STAMPED / FAIR / YODA / Purpose / SciOps) and a three-action Monday checklist. Abstract ~576 words, Connection ~270 words. Co-Authored-By: Claude Code 2.1.148 / Claude Opus 4.7 --- 2026-usrse/talk-proposal-mvc.md | 95 +++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 2026-usrse/talk-proposal-mvc.md diff --git a/2026-usrse/talk-proposal-mvc.md b/2026-usrse/talk-proposal-mvc.md new file mode 100644 index 00000000..37396d53 --- /dev/null +++ b/2026-usrse/talk-proposal-mvc.md @@ -0,0 +1,95 @@ +# Model–View–Controller at the Stack Scale: Standards, Schemas, and Why Open-Science Infrastructure Composes + +## Authors + +- Yaroslav O. Halchenko \, Department of Psychological and Brain Sciences, Dartmouth College / Center for Open Neuroscience, ORCID [0000-0003-3456-2493](https://orcid.org/0000-0003-3456-2493) + +## Keywords + +MVC, software architecture, hexagonal architecture, FAIR, metadata schemas, LinkML, SHACL, JSON Schema, pydantic, schema-driven UI, reproducibility, standards + +## Abstract + +This talk is the architectural companion to a separately submitted, history-arc proposal on two decades of RSEing open neuroscience at the **Center for Open Neuroscience (CON)**. +Where the history talk asks *how did we get here*, this talk asks *why does it compose*, and *what makes standards load-bearing*? +The answer, sharpened over two decades, is that the reusable open-science stack keeps converging on **Model–View–Controller** — not at the level of one app, but at the level of the whole stack. + +**Models — layered, standardized, schema-first.** +Dataset-layout models ([BIDS](https://bids.neuroimaging.io/), [YODA](https://github.com/myyoda/poster)) sit on top of per-file formats ([NWB](https://www.nwb.org/), [NGFF / OME-Zarr](https://ngff.openmicroscopy.org/), NIfTI, TSV, Parquet, HDF5, Zarr, DICOM), which sit on top of *metadata schemas* — the heart of this talk: [LinkML](https://linkml.io/), [pydantic](https://docs.pydantic.dev/), [JSON Schema](https://json-schema.org/), [SHACL](https://www.w3.org/TR/shacl/), and [DataLad Concepts](https://concepts.datalad.org/) define one source from which JSON Schema, OWL/SHACL, pydantic classes, and human-readable docs are *generated*. +Underneath everything sits a content-addressed storage model — `git` + `git-annex` + [DataLad](https://www.datalad.org/) — that any HTTP client can read with no service required. + +**Views — including ones the Model materializes.** +Archive UIs ([datasets.datalad.org](https://datasets.datalad.org/), [DANDI](https://dandiarchive.org/), [OpenNeuro](https://openneuro.org/), [EMBER](https://emberarchive.org/)), schema-driven editors and websites ([vjsf](https://vjsf.koumoul.com/) powering the DANDI [meditor](https://gui.dandiarchive.org/); [shacl-vue](https://github.com/psychoinformatics-de/shacl-vue) from M. Hanke's group rendering forms *and entire research-group websites* from SHACL shapes — see Hanke *et al.*, [LinkML metadata-driven workflow](https://datasets.datalad.org/repronim/ReproTube/DataLad/web/#/video/oF98hdaph1k?tab=local&wide=1&t=644&q=model&filter=1)), tabular surfaces ([Datasette](https://datasette.io/), [VisiData](https://www.visidata.org/)), programmatic APIs (PyBIDS, pynwb, `datalad.api`, fsspec, FUSE), and long-form narrative (the DataLad Handbook, per-archive docs, the specs themselves) — all derive from the same Model. +*The Model deserves many Views.* + +**Controllers — small, single-purpose.** +HeuDiConv, NeuroConv, ReproStim, ReproIn, BIDS-Apps, `datalad run` / `containers-run`, `git-annex` special remotes, [`con/duct`](https://github.com/con/duct), ReproMan. +Validation is itself a recursion: [`con/validation`](https://github.com/con/validation) harmonizes bids-validator (+ HED), pynwb, zarr, nwbinspector, and OME-Zarr into a single validation-result Model that [dandi-cli](https://github.com/dandi/dandi-cli) emits on every release. +*`con/validation` is the type-checker for your validators.* + +**The anti-pattern contrast.** +Most academic data infrastructure is a "service-tied UI": a bespoke LIMS / ELN, a "just a Flask app" portal, a Drupal/WordPress data site, a dashboard that *is* the data. +The Model is implicit (whatever the ORM happens to expose); the View is welded to the Controller; the Controller is welded to a running process. +*A bespoke backend dies the day its server stops.* +The design-pattern literature already names this: Evans's **Smart UI** anti-pattern ([DDD](https://www.domainlanguage.com/ddd/), 2003), Fowler's [**Anemic Domain Model**](https://martinfowler.com/bliki/AnemicDomainModel.html) and [**Service Layer**](https://martinfowler.com/eaaCatalog/serviceLayer.html), and GoF's **Adapter / Strategy / Façade** (1994). +The cure is Cockburn's [**Hexagonal Architecture / Ports & Adapters**](https://alistair.cockburn.us/hexagonal-architecture/) (2005) — the domain Model in the centre, UI, DB, and services as interchangeable Adapters around it. +That is MVC at the stack scale. + +**Many lenses, one stack.** +MVC is one of six overlapping lenses we keep using — Architectural (MVC), Procedural ([STAMPED](https://stamped-principles.org/); Macdonald *et al.*, 2026), Sharing (FAIR), Compositional (YODA), Purpose (acquisition / archive / analysis / governance), and Maturity ([SciOps](https://arxiv.org/abs/2401.00077) L1–L5; Johnson *et al.*, 2024). +Each picks out a different property of the same artefacts; disagreements between lenses surface real design tensions. + +**Punchline.** +Pick any cell — Model, View, or Controller — and swap it; the rest still works. +*That* is why a small RSE centre can reach population scale, why each piece stays small enough to maintain, and why a human or AI agent can pick up the stack cold. +**Take home:** name the M / V / C of your next project before you write it; if any column has one entry, you have a silo. + +## References + +1. *The Brain Imaging Data Structure (BIDS).* Gorgolewski *et al.* Sci. Data 2016. ; +2. *YODA — modular, look-up-you-must-not data organization.* +3. *Neurodata Without Borders (NWB).* +4. *NGFF / OME-Zarr — next-generation file formats for bioimaging.* +5. *LinkML — a linked-data modelling language.* +6. *pydantic — data validation using Python type hints.* +7. *JSON Schema.* +8. *SHACL — Shapes Constraint Language.* W3C Recommendation. +9. *DataLad Concepts — metadata schema definitions for DataLad-managed datasets.* +10. *DANDI Archive.* ; DANDI metadata editor (meditor) at ; *vjsf — Vue JSON Schema Form.* +11. *OpenNeuro.* Markiewicz *et al.* eLife 2021. ; +12. *EMBER Archive.* +13. *DataLad: distributed system for joint management of code, data, and their relationship.* Halchenko *et al.* JOSS 2021. ; +14. *shacl-vue — SHACL-driven Vue forms and websites.* M. Hanke's group, psychoinformatics-de. ; M. Hanke *et al.*, *LinkML metadata-driven workflow*, ReproTube: +15. *con/validation — a harmonized schema and CLI for validation results across BIDS, NWB, Zarr, OME-Zarr, nwbinspector, HED, etc.* +16. *dandi-cli — DANDI command-line client; harnesses con/validation across format validators.* +17. *Datasette — an open-source multi-tool for exploring and publishing data.* Simon Willison. +18. *VisiData — a terminal interface for exploring and arranging tabular data.* +19. *STAMPED principles for reproducible research objects.* A. Macdonald, C. C. Baker, I. To, Y. O. Halchenko. 2026. — **S**elf-containment, **T**racking, **A**ctionability, **M**odularity, **P**ortability, **E**phemerality, **D**istributability. +20. *SciOps: Achieving Productivity and Reliability in Data-Intensive Research.* E. C. Johnson *et al.* (incl. Y. O. Halchenko). 2024. — five-level Capability Maturity Model for rigorous scientific operations. +21. A. Cockburn. *Hexagonal Architecture (Ports & Adapters).* 2005. +22. E. Evans. *Domain-Driven Design: Tackling Complexity in the Heart of Software.* Addison-Wesley, 2003 — "Smart UI" anti-pattern (Ch. 4). +23. M. Fowler. *Anemic Domain Model.* ; *Service Layer*, *Patterns of Enterprise Application Architecture*. +24. E. Gamma, R. Helm, R. Johnson, J. Vlissides. *Design Patterns: Elements of Reusable Object-Oriented Software.* Addison-Wesley, 1994 — Adapter, Strategy, Façade. +25. *Center for Open Neuroscience.* + +## Connection to Mission, Goals, & Interests of US-RSE Community + +US-RSE audiences are fluent in Model–View–Controller from enterprise contexts — and yet that vocabulary is rarely brought to bear on *research* infrastructure, where the dominant idiom is still a bespoke server-rendered portal whose data evaporates the day the grant ends. +This talk gives RSEs a name for what they already feel: that "the View should be derived from the Model, not welded to a backend". +It is the architectural companion to a separately submitted history-arc proposal from the same author; together they offer two complementary views of the same stack — one diachronic, one structural. + +The bridging roles US-RSE foregrounds are central to the talk: + +- **RSE <-> engineering industry.** + Sustained collaboration with [Kitware](https://www.kitware.com/) on NWB browse/analyze/visualize tooling; close uptake of Koumoul's `vjsf` (commercial-OSS) for the DANDI metadata editor; and joint design work with [psychoinformatics-de](https://psychoinformatics.de/) (M. Hanke's group) on `shacl-vue`. + All three are textbook RSE-<->-industry partnerships built around *schemas*, not services. + +- **RSE <-> domain scientists.** + BIDS, NWB, LinkML, and DataLad Concepts are co-designed with the labs that produce the data; the schemas are the contract that lets RSEs and domain scientists negotiate without renegotiating each tool. + +- **RSE <-> global RSE community.** + Hexagonal Architecture, MVC, Adapter/Strategy/Façade — these are the names RSE practitioners across domains already share. + Reading the open-science stack through those names is the cheapest possible technology-transfer from RSE to research-data engineering. + +The talk closes with a concrete three-action Monday checklist that works in any domain: name your M / V / C; pick the cell you would swap within five years; publish your Model as a real artefact. + From 0edb9e674ee4687abcec22b28422e359b7a285db Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Sat, 30 May 2026 11:22:38 -0400 Subject: [PATCH 02/27] talk-proposal-mvc: flowing-prose abstract + per-section page breaks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-read the MVC proposal as continuous narrative, matching the rhythm of the companion talk-proposal-draft.md: - Convert the six bolded sub-headers in the abstract ("Models — ...", "Views — ...", "Controllers — ...", "The anti-pattern contrast.", "Many lenses, one stack.", "Punchline.") into prose paragraphs. The M/V/C terms still anchor each paragraph in **bold** but as in-sentence emphasis, not mini-titles, so the abstract now reads as one argument rather than a slide outline. - Same treatment for the Connection section: collapse the three bulleted RSE-bridging items into one paragraph with inline `**RSE <-> ...**` markers. - Drop in `{=latex} \newpage` raw-attribute fences so Abstract, References, and Connection each start on a fresh page in the PDF render (HTML drops them silently), mirroring the layout already used in talk-proposal-draft.md. All citations, named anti-patterns (Smart UI / Anemic Domain Model / Service Layer / Adapter-Strategy-Façade), the Hexagonal-Architecture cure, the six-lens table, and the Monday-checklist take-home are unchanged. Word counts: Abstract ~624 (was ~580 — small growth is the cost of de-bulleting), Connection ~298. Co-Authored-By: Claude Code 2.1.158 / Claude Opus 4.7 --- 2026-usrse/talk-proposal-mvc.md | 76 ++++++++++++++++----------------- 1 file changed, 37 insertions(+), 39 deletions(-) diff --git a/2026-usrse/talk-proposal-mvc.md b/2026-usrse/talk-proposal-mvc.md index 37396d53..bfe6c354 100644 --- a/2026-usrse/talk-proposal-mvc.md +++ b/2026-usrse/talk-proposal-mvc.md @@ -8,42 +8,45 @@ MVC, software architecture, hexagonal architecture, FAIR, metadata schemas, LinkML, SHACL, JSON Schema, pydantic, schema-driven UI, reproducibility, standards +```{=latex} +\newpage +``` + ## Abstract This talk is the architectural companion to a separately submitted, history-arc proposal on two decades of RSEing open neuroscience at the **Center for Open Neuroscience (CON)**. Where the history talk asks *how did we get here*, this talk asks *why does it compose*, and *what makes standards load-bearing*? -The answer, sharpened over two decades, is that the reusable open-science stack keeps converging on **Model–View–Controller** — not at the level of one app, but at the level of the whole stack. +The answer, sharpened over two decades of building and re-building infrastructure, is that the reusable open-science stack keeps converging on **Model–View–Controller** — not at the level of one application, but at the level of the whole stack. -**Models — layered, standardized, schema-first.** -Dataset-layout models ([BIDS](https://bids.neuroimaging.io/), [YODA](https://github.com/myyoda/poster)) sit on top of per-file formats ([NWB](https://www.nwb.org/), [NGFF / OME-Zarr](https://ngff.openmicroscopy.org/), NIfTI, TSV, Parquet, HDF5, Zarr, DICOM), which sit on top of *metadata schemas* — the heart of this talk: [LinkML](https://linkml.io/), [pydantic](https://docs.pydantic.dev/), [JSON Schema](https://json-schema.org/), [SHACL](https://www.w3.org/TR/shacl/), and [DataLad Concepts](https://concepts.datalad.org/) define one source from which JSON Schema, OWL/SHACL, pydantic classes, and human-readable docs are *generated*. -Underneath everything sits a content-addressed storage model — `git` + `git-annex` + [DataLad](https://www.datalad.org/) — that any HTTP client can read with no service required. +The **Models** are layered and schema-first. +Dataset-layout models ([BIDS](https://bids.neuroimaging.io/), [YODA](https://github.com/myyoda/poster)) sit on top of per-file formats ([NWB](https://www.nwb.org/), [NGFF / OME-Zarr](https://ngff.openmicroscopy.org/), NIfTI, TSV, Parquet, HDF5, Zarr, DICOM), which in turn rest on the *metadata schemas* that are the heart of this talk. +[LinkML](https://linkml.io/), [pydantic](https://docs.pydantic.dev/), [JSON Schema](https://json-schema.org/), [SHACL](https://www.w3.org/TR/shacl/), and [DataLad Concepts](https://concepts.datalad.org/) provide a single source from which JSON Schema, OWL/SHACL, pydantic classes, and human-readable docs are *generated* rather than maintained in parallel. +Beneath everything sits a content-addressed storage model — `git` + `git-annex` + [DataLad](https://www.datalad.org/) — that any HTTP client can read with no running service required. -**Views — including ones the Model materializes.** -Archive UIs ([datasets.datalad.org](https://datasets.datalad.org/), [DANDI](https://dandiarchive.org/), [OpenNeuro](https://openneuro.org/), [EMBER](https://emberarchive.org/)), schema-driven editors and websites ([vjsf](https://vjsf.koumoul.com/) powering the DANDI [meditor](https://gui.dandiarchive.org/); [shacl-vue](https://github.com/psychoinformatics-de/shacl-vue) from M. Hanke's group rendering forms *and entire research-group websites* from SHACL shapes — see Hanke *et al.*, [LinkML metadata-driven workflow](https://datasets.datalad.org/repronim/ReproTube/DataLad/web/#/video/oF98hdaph1k?tab=local&wide=1&t=644&q=model&filter=1)), tabular surfaces ([Datasette](https://datasette.io/), [VisiData](https://www.visidata.org/)), programmatic APIs (PyBIDS, pynwb, `datalad.api`, fsspec, FUSE), and long-form narrative (the DataLad Handbook, per-archive docs, the specs themselves) — all derive from the same Model. -*The Model deserves many Views.* +A well-defined Model deserves many **Views**, and several of those Views the Model itself materializes. +Archive UIs ([datasets.datalad.org](https://datasets.datalad.org/), [DANDI](https://dandiarchive.org/), [OpenNeuro](https://openneuro.org/), [EMBER](https://emberarchive.org/)), schema-driven editors and websites ([vjsf](https://vjsf.koumoul.com/) powering the DANDI [meditor](https://gui.dandiarchive.org/); [shacl-vue](https://github.com/psychoinformatics-de/shacl-vue) from M. Hanke's group rendering forms — and entire research-group websites — from SHACL shapes, as documented in Hanke *et al.*'s [LinkML metadata-driven workflow](https://datasets.datalad.org/repronim/ReproTube/DataLad/web/#/video/oF98hdaph1k?tab=local&wide=1&t=644&q=model&filter=1)), tabular surfaces ([Datasette](https://datasette.io/), [VisiData](https://www.visidata.org/)), programmatic APIs (PyBIDS, pynwb, `datalad.api`, fsspec, FUSE), and long-form narrative (the DataLad Handbook, per-archive docs, the specs themselves) all derive from the same Model — so when the Model evolves, every View follows for free. -**Controllers — small, single-purpose.** -HeuDiConv, NeuroConv, ReproStim, ReproIn, BIDS-Apps, `datalad run` / `containers-run`, `git-annex` special remotes, [`con/duct`](https://github.com/con/duct), ReproMan. -Validation is itself a recursion: [`con/validation`](https://github.com/con/validation) harmonizes bids-validator (+ HED), pynwb, zarr, nwbinspector, and OME-Zarr into a single validation-result Model that [dandi-cli](https://github.com/dandi/dandi-cli) emits on every release. -*`con/validation` is the type-checker for your validators.* +The **Controllers** stay small and single-purpose. +HeuDiConv, NeuroConv, ReproStim, ReproIn, BIDS-Apps, `datalad run` / `containers-run`, `git-annex` special remotes, [`con/duct`](https://github.com/con/duct), and ReproMan each do one slice of the acquisition-to-pipeline work and then compose. +Validation is itself a recursion of the same idea: [`con/validation`](https://github.com/con/validation) harmonizes bids-validator (+ HED), pynwb, zarr, nwbinspector, and OME-Zarr into a single validation-result Model that [dandi-cli](https://github.com/dandi/dandi-cli) emits on every release — `con/validation` is, in effect, the type-checker for your validators. -**The anti-pattern contrast.** -Most academic data infrastructure is a "service-tied UI": a bespoke LIMS / ELN, a "just a Flask app" portal, a Drupal/WordPress data site, a dashboard that *is* the data. -The Model is implicit (whatever the ORM happens to expose); the View is welded to the Controller; the Controller is welded to a running process. +Most academic data infrastructure does the opposite of all this. +The dominant pattern is a "service-tied UI" — a bespoke LIMS or ELN, a "just a Flask app" portal, a Drupal/WordPress data site, a dashboard that *is* the data — in which the Model is implicit (whatever the ORM happens to expose), the View is welded to the Controller, and the Controller is welded to a running process. *A bespoke backend dies the day its server stops.* -The design-pattern literature already names this: Evans's **Smart UI** anti-pattern ([DDD](https://www.domainlanguage.com/ddd/), 2003), Fowler's [**Anemic Domain Model**](https://martinfowler.com/bliki/AnemicDomainModel.html) and [**Service Layer**](https://martinfowler.com/eaaCatalog/serviceLayer.html), and GoF's **Adapter / Strategy / Façade** (1994). -The cure is Cockburn's [**Hexagonal Architecture / Ports & Adapters**](https://alistair.cockburn.us/hexagonal-architecture/) (2005) — the domain Model in the centre, UI, DB, and services as interchangeable Adapters around it. -That is MVC at the stack scale. +The design-pattern literature already names this trap: Evans's **Smart UI** anti-pattern ([DDD](https://www.domainlanguage.com/ddd/), 2003), Fowler's [**Anemic Domain Model**](https://martinfowler.com/bliki/AnemicDomainModel.html) and [**Service Layer**](https://martinfowler.com/eaaCatalog/serviceLayer.html), and GoF's **Adapter / Strategy / Façade** (1994). +The cure has a name too — Cockburn's [**Hexagonal Architecture / Ports & Adapters**](https://alistair.cockburn.us/hexagonal-architecture/) (2005), with the domain Model in the centre and the UI, DB, and services as interchangeable Adapters around it — which is exactly MVC at the stack scale. -**Many lenses, one stack.** -MVC is one of six overlapping lenses we keep using — Architectural (MVC), Procedural ([STAMPED](https://stamped-principles.org/); Macdonald *et al.*, 2026), Sharing (FAIR), Compositional (YODA), Purpose (acquisition / archive / analysis / governance), and Maturity ([SciOps](https://arxiv.org/abs/2401.00077) L1–L5; Johnson *et al.*, 2024). -Each picks out a different property of the same artefacts; disagreements between lenses surface real design tensions. +MVC is one of six overlapping lenses we keep using on the same stack: Architectural (MVC), Procedural ([STAMPED](https://stamped-principles.org/); Macdonald *et al.*, 2026), Sharing (FAIR), Compositional (YODA), Purpose (acquisition / archive / analysis / governance), and Maturity ([SciOps](https://arxiv.org/abs/2401.00077) L1–L5; Johnson *et al.*, 2024). +Each picks out a different property of the same artefacts, and disagreements between lenses are where real design tensions surface. -**Punchline.** -Pick any cell — Model, View, or Controller — and swap it; the rest still works. +The punchline is operational: pick any cell — Model, View, or Controller — and swap it; the rest still works. *That* is why a small RSE centre can reach population scale, why each piece stays small enough to maintain, and why a human or AI agent can pick up the stack cold. **Take home:** name the M / V / C of your next project before you write it; if any column has one entry, you have a silo. +```{=latex} +\newpage +``` + ## References 1. *The Brain Imaging Data Structure (BIDS).* Gorgolewski *et al.* Sci. Data 2016. ; @@ -72,24 +75,19 @@ Pick any cell — Model, View, or Controller — and swap it; the rest still wor 24. E. Gamma, R. Helm, R. Johnson, J. Vlissides. *Design Patterns: Elements of Reusable Object-Oriented Software.* Addison-Wesley, 1994 — Adapter, Strategy, Façade. 25. *Center for Open Neuroscience.* -## Connection to Mission, Goals, & Interests of US-RSE Community - -US-RSE audiences are fluent in Model–View–Controller from enterprise contexts — and yet that vocabulary is rarely brought to bear on *research* infrastructure, where the dominant idiom is still a bespoke server-rendered portal whose data evaporates the day the grant ends. -This talk gives RSEs a name for what they already feel: that "the View should be derived from the Model, not welded to a backend". -It is the architectural companion to a separately submitted history-arc proposal from the same author; together they offer two complementary views of the same stack — one diachronic, one structural. +```{=latex} +\newpage +``` -The bridging roles US-RSE foregrounds are central to the talk: - -- **RSE <-> engineering industry.** - Sustained collaboration with [Kitware](https://www.kitware.com/) on NWB browse/analyze/visualize tooling; close uptake of Koumoul's `vjsf` (commercial-OSS) for the DANDI metadata editor; and joint design work with [psychoinformatics-de](https://psychoinformatics.de/) (M. Hanke's group) on `shacl-vue`. - All three are textbook RSE-<->-industry partnerships built around *schemas*, not services. - -- **RSE <-> domain scientists.** - BIDS, NWB, LinkML, and DataLad Concepts are co-designed with the labs that produce the data; the schemas are the contract that lets RSEs and domain scientists negotiate without renegotiating each tool. +## Connection to Mission, Goals, & Interests of US-RSE Community -- **RSE <-> global RSE community.** - Hexagonal Architecture, MVC, Adapter/Strategy/Façade — these are the names RSE practitioners across domains already share. - Reading the open-science stack through those names is the cheapest possible technology-transfer from RSE to research-data engineering. +US-RSE audiences are fluent in Model–View–Controller from enterprise contexts, and yet that vocabulary is rarely brought to bear on *research* infrastructure, where the dominant idiom is still a bespoke server-rendered portal whose data evaporates the day the grant ends. +This talk gives RSEs a name for what they already feel — that the View should be derived from the Model, not welded to a backend — and a vocabulary for arguing it in their own organisations. +It is the architectural companion to a separately submitted, history-arc proposal from the same author; together they offer two complementary readings of the same stack, one diachronic and one structural, and either stands alone. -The talk closes with a concrete three-action Monday checklist that works in any domain: name your M / V / C; pick the cell you would swap within five years; publish your Model as a real artefact. +The bridging roles that US-RSE foregrounds are central to the talk. +On the **RSE <-> engineering industry** axis, the talk draws on sustained collaboration with [Kitware](https://www.kitware.com/) on NWB browse/analyze/visualize tooling, close uptake of Koumoul's commercial-OSS `vjsf` for the DANDI metadata editor, and joint design work with [psychoinformatics-de](https://psychoinformatics.de/) (M. Hanke's group) on `shacl-vue` — three textbook industry partnerships built around *schemas*, not services. +On the **RSE <-> domain scientists** axis, BIDS, NWB, LinkML, and DataLad Concepts are co-designed with the labs that produce the data, and the schemas are the explicit contract that lets RSEs and domain scientists negotiate once instead of renegotiating each tool. +On the **RSE <-> global RSE community** axis, Hexagonal Architecture, MVC, and Adapter/Strategy/Façade are names RSE practitioners across domains already share; reading the open-science stack through those names is the cheapest possible technology-transfer from RSE practice to research-data engineering. +The talk closes with a concrete three-action Monday checklist that works in any domain: name your M / V / C; pick the cell you would swap within five years; publish your Model as a real, citable artefact. From 57fa97b9d18d96eb0baa8459e19546b856b95c4e Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 7 Oct 2026 23:29:03 -0400 Subject: [PATCH 03/27] Initial sketches for MIT talk --- 2026-reproAI-MIT-STAMPED.html | 1577 ++++++++++++++++++++++ 2026-reproAI-MIT-STAMPED/PLAN.md | 401 ++++++ 2026-reproAI-MIT-STAMPED/playlist.md | 121 ++ INDEX.md | 282 ++++ pics/2026-reproAI-MIT-STAMPED-qrcode.png | 1 + 5 files changed, 2382 insertions(+) create mode 100644 2026-reproAI-MIT-STAMPED.html create mode 100644 2026-reproAI-MIT-STAMPED/PLAN.md create mode 100644 2026-reproAI-MIT-STAMPED/playlist.md create mode 120000 pics/2026-reproAI-MIT-STAMPED-qrcode.png diff --git a/2026-reproAI-MIT-STAMPED.html b/2026-reproAI-MIT-STAMPED.html new file mode 100644 index 00000000..9f36e41b --- /dev/null +++ b/2026-reproAI-MIT-STAMPED.html @@ -0,0 +1,1577 @@ + + + + + + + + + STAMPED principles and their role in the age of AI-driven scientific software development + + + + + + + + + + + + + + + +
+
+ + + + + + + +
+
+ +

STAMPED principles and their role in the age of
AI-driven scientific software development

+

+ Agents often fail not for lack of "intelligence".
They fail for lack of structure to act on. +

+ + + + LLM Workshop — MIT — October 16th, 2026
+ Live slides/Sources: + https://datasets.datalad.org/centerforopenneuroscience/talks/2026-reproAI-MIT-STAMPED.html
+
+ + + + + + + + + + + + + + +
+
+ + + +
+

Reality Check

+ +
+ +
+ +
+

+ Disclaimer: I am not here to sell you a model, a vendor, or a workflow. +

+

+ I am here to argue that the boring infrastructure work we have been doing for 20 years
+ is exactly what decides whether agents help you or hurt you. +

+
+

+ Counts and names on these slides are links — follow along, click through later. +

+
+ + + +
+

Standing on two talks

+ + + + + +
+ Mine — ReproNim webinar, Feb 2026
+ ReproFlow & YODA: structure your studies + (video) +
    +
  • Version control everything
    • +
  • Look up you must not
    • +
  • Compose modularly you shall
    • +
  • … and preserve all artifacts as an AI-accessible knowledge base
    • +
+ 		+ Oscar Esteban — OHBM 2026 Educational
+ Using LLMs in Neuroimaging Research: LLMs for Coding +
    +
  • Agent = Model + Harness
    • +
  • AGENTS.md as the operational contract
    • +
  • Context engineering > prompt wording
    • +
  • Grounding beats a bigger search
    • +
+
+ +
+ Oscar showed how to build the harness.
+ This talk is about what the harness stands on — and how harnesses themselves become shareable. +
+ + +
+ + + +
+ +
+

It’s 2026. Agents write the code.

+ +
+ +
+

So the bottleneck moved

+ + + + + + + + + + + +
writing the code→verifying and situating it
can I express this?→is this right, and does it fit what already exists?
+ +
+ +
+

+ HBR, Feb 2026: “AI doesn’t reduce work, it intensifies it” +

+ +
+ +
+

Without structure, the agent has to guess

+ + + + + +
+ What an agent finds in most labs +
study_final2/
+  scans/
+    s01_ep2d_bold_moco_3.nii
+    s01_t1_mprage_FINAL.nii
+    s02_ep2d_bold_moco_3(1).nii
+  analysis_new/
+    run_it.sh
+    run_it_OLD.sh
+    results_use_this_one.mat
+  notes.docx
+
    +
  • Which file is which subject? Infer.
    • +
  • What are the acquisition parameters? Infer.
    • +
  • Which script made results_use_this_one.mat? Infer.
    • +
  • Was it run on the fixed data or the broken data? Infer.
    • +
+ 		+ What an agent finds in a BIDS dataset +
ds000001/
+  dataset_description.json
+  participants.tsv
+  sub-01/
+    anat/sub-01_T1w.nii.gz
+    anat/sub-01_T1w.json
+    func/sub-01_task-rest_bold.nii.gz
+    func/sub-01_task-rest_bold.json
+  derivatives/
+    mriqc/dataset_description.json
+
    +
  • Subject, session, task, run: in the filename
    • +
  • Parameters: in the sidecar
    • +
  • Provenance of derivatives: GeneratedBy
    • +
  • Correctness: a validator says so
    • +
+
+ +
+ Same model. Same prompt. Different affordances. + On the left the agent improvises confidently and is wrong. + On the right it can act, and the validator catches it when it does not. +
+

+ Not just rhetoric: a structured neuroimaging harness moved first-choice tool-selection accuracy from + 23.3% to 93.6% across seven models — + Chen, …, Poldrack, + Bringing analytic rigor to agentic AI for science, arXiv:2608.19902, 2026 + (brain-researcher.com). +

+ +
+ +
+

This problem already has a name

+ + + + + + +
+ +

YODA, 2018 — for humans composing studies

+ 		→ + +

STAMPED, 2026 — generalized, gradeable, domain-agnostic

+
+

+ We did not invent these for AI. That is the point — they were already the right properties. +

+
+ +
+ + + +
+ +
+ +

Seven properties a research object should have —
whoever, or whatever, picks it up next.

+ +
+ +
+

STAMPED in one table

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
PropertyIn one lineWhat it buys an agent
Self-containedEverything under one boundaryA bounded context window. Nothing it needs is “on Bob’s laptop”.
TrackedVersion control & provenanceA diff to review, and a record of who/what produced each change — including the agent.
ActionableExecutable & documented workflowsThe load-bearing one. A command it can run, not a paragraph it must interpret.
ModularIndependent & composable modulesA unit small enough to reason about, swap, and verify in isolation.
PortableRuns anywhere with explicit specsThe agent’s sandbox is not your laptop. If it only runs on your laptop, it does not run.
EphemeralTransient compute environmentThrow-away containers — exactly how agents already work. No accreted state to corrupt.
DistributableShareable, persistent, retrievableIt can get the inputs itself instead of asking you for a Dropbox link.
+

+ + A. Macdonald, C. Baker, I. To, Y. O. Halchenko. + STAMPED: principles for reproducible research objects. + Preprint, 2026. doi: 10.31222/osf.io/f3h82_v1 +  ·  + sources: stamped-principles/stamped-paper +  ·  + paper.stamped-principles.org +

+ +
+ +
+

Actionability is the one that agents live or die on

+
+

An actionable object answers, without a human in the loop:

+
    +
  • What is this? → a declared standard (BIDS, NWB), not a README paragraph
    • +
  • How do I run it? → a recorded command, not “see Methods”
    • +
  • Did I get it right? → a validator exit code, not a reviewer’s opinion
    • +
+
+ +
+ And standards are what make Actionability cheap. + You do not negotiate the layout per project — you inherit it.
+ “You’ve seen one BIDS dataset, you’ve seen them all” was a convenience for humans. + For a model that would otherwise have to infer the layout, it is the difference between acting and hallucinating. +
+ +

+ “All standards are ‘bad’, but some are used.” — D. Clunie, MICCAI 2017. + A used standard beats a correct one — and now it has a second constituency. +

+ +
+ +
+

… and AI changed what counts as actionable

+ + + + + + + + + + + + + + + + + + + + + +
“Actionable” required…… so what could act
BeforeAn exact command, API call, or documented CLI. Prose was not enough.Tools, and humans who already knew the tool. + Everything else stayed a paragraph in the Methods.
NowA procedure described in human language — + with scripts dropped in exactly where precision matters.Any agent, on first contact. + The interface moved from syntax to language, so vastly more of what we + already wrote down became executable.
NextRoutines that fire without being asked.The harness itself — CI, PR monitors, scheduled agents.
+ +
+ + + + + +
+ 1. Skills — logic written in language +
    +
  • Human language carries the judgement; scripts carry the precision
    • +
  • One skill, one job — modularity still applies
    • +
  • The procedure you did by hand three times, written down once
    • +
+ 		+ 2. Routines — the harness runs them for you +
    +
  • Claude Code on the web watching a repo: triage issues, review PRs
    • +
  • Scheduled agents; CI that does more than run tests
    • +
  • No human has to remember to invoke it
    • +
+
+
+ +
+ AI did not only make more things actionable. It made automation on top of + actionability cheap — which is the real multiplier. +
+ +
+ +
+

So grade them too: a skill is a research object

+

+ A skill that only runs in your repo, on your paths, undocumented and unversioned, + is a silo with a friendlier interface. +

+ + + + + + + + + + + + + + + + + + + +
Property… applied to a skill or an automation
Self-containedCarries its own references and scripts. Not “see our internal wiki”.
TrackedIn git, with a metadata.version and a history — so a change to the procedure is reviewable.
ActionableRunnable commands and scripts, not only prose. Recursive, and the point.
ModularOne skill, one job. Composable with the others instead of a mega-prompt.
PortableNo hardcoded paths, no “works on our cluster”. Declares its dependencies and versions.
EphemeralRuns in a fresh sandbox. No accreted state it silently depends on.
DistributablePublished where someone else can find and install it — not pasted into chat.
+ +
+ Good automation and good skills pass the STAMPED checklist. + That is not a purity test — it is precisely what makes them reusable + beyond the person who wrote them. +
+ +

+ Walk it yourself: checklist.stamped-principles.org + (MUST / SHOULD / MAY, item by item) · + examples · + principles · + paper source + (main.tex + — small enough to paste into a context window, which is how we actually grade a study today). +

+ +
+ +
+ + + +
+ +
+

What we built to make this real

+

Twenty years of it, in neuroimaging, before anyone said “agent”.

+
+ +
+

Standards are the grammar — BIDS as the worked example

+ +
+ 11 modalities · 25 active BEPs · + a machine-readable schema · + a validator · + BIDS-Apps with one calling convention +
+
+ The schema is the part that matters here: the standard is not prose an agent reads, + it is data an agent queries. And the validator is the type-checker that + turns a confident wrong answer into a failing build. +
+
+ But a layout only tells you what the files are. + It says nothing about how they came to be, or what it cost to make them. + Standard → datalad run → duct: + what it is → how it was made → what it took. + Three layers, and an agent needs all three. +
+ +

+ + K. J. Gorgolewski, …, Y. O. Halchenko, …, R. A. Poldrack. + The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. + Scientific Data 3:160044, 2016. +

+
+ +
+

Tracked: datalad run writes the record for you

+
$ datalad run -m "run brain extract workflow" \
+    --input "inputs/ds*/sub-01/anat/sub-01_T1w.nii.gz" \
+    --output "sub-*/anat" \
+    bash -e code/brain_extraction.sh inputs/ds*/sub-01/anat/sub-01_T1w.nii.gz
+
$ git show --stat
+commit 1223bc4bead3c7e6be15b78546c894c6465dd7a4
+
+    [DATALAD RUNCMD] run brain extract workflow
+
+    === Do not change lines below ===
+    {
+     "cmd": "bash -e code/brain_extraction.sh inputs/ds000001/sub-01/anat/sub-01_T1w.nii.gz ...",
+     "dsid": "b656ac80-973c-45ef-887e-1b8d6f6a51dd",
+     "exit": 0,
+     "inputs": ["inputs/ds*/sub-01/anat/sub-01_T1w.nii.gz"],
+     "outputs": ["sub-*/anat"],
+     "pwd": "."
+    }
+    ^^^ Do not change lines above ^^^
+
+ sub-001/anat.nii.gz      | 1 +
+ sub-001/anat_mask.nii.gz | 1 +
+
    +
  • Machine-readable, in the commit message — no sidecar database to lose
    • +
  • Clean git state in, clean git state out: no surprises
    • +
  • datalad rerun reads it back and re-executes
    • +
+

+ Example from the DataLad Handbook. + Found in the wild in ~15% of indexed DataLad repositories on GitHub. +

+
+ +
+

Provenance does not care who wrote the code

+

+ Agent-written or hand-written, the [DATALAD RUNCMD] record is the same shape. +

+

+ The tests do not know the code was AI-generated either. +

+
+ This is why Tracking is the principle that makes AI disclosure + mechanical instead of aspirational — and we will come back to that at the end. +
+ +
+ +
+

The missing piece: con/duct

+ +
+ Wrap any command. It records, next to the outputs: + wall time, CPU, peak RSS, per-process-tree samples, exit code, environment, full + stdout/stderr — as plain, self-contained files. +
+ +
+ +
+

… and that log is the provenance you need later

+ + + + + +
+ “Why did this get OOM-killed on the cluster?” +
  • Peak RSS over time, per process. Not your memory of it.
+ “How much do I request for 2,000 subjects?” +
  • Measured, from the run you already did.
+ “What did it print before it died?” +
  • Captured, versioned, next to the outputs.
+ 		+ +

con-duct plot — MRIQC on one subject/session

+
+ +
+ Why an AI audience should care: when you hand a failing pipeline to an agent, + it needs numbers and text it can read, not a human recollection. + duct turns “it crashed, I think it ran out of memory” into a file. + Self-contained (S), tracked (T), immediately actionable (A). +
+

+ Already wired in: set REPRONIM_USE_DUCT=1 with + ///repronim/containers + and every datalad containers-run gets a duct trace for free. +

+
+ +
+

All three layers, one real artifact — built with AI

+

+ dandi/compression-comparisons-t261-study + — a reproduction of + Buccino et al. 2023 on ephys compression, + plus a comparison against the new ITU-T T.261 codec. +

+ + + +
+ 1. The layout (what) +
dataset_description.json
+code/
+  compression-comparisons-tools/
+  run_sweep.sh
+sourcedata/
+  aind-ephys-compression/      # 523 GB
+derivatives/
+  compbench-*/<cell>/
+    metrics.json  manifest.json
+    duct-info.json  duct-usage.jsonl
+  reports-*/
+

BIDS-shaped, YODA-composed: data and tool are + subdatasets pinned by commit, not copies.

+ 		+ 2. The record (how) — and 3. the cost (what it took) +
$ git log --oneline
+7733465 [DATALAD RUNCMD] cell aind-634568__zstd...
+
+ "cmd": "duct --output-prefix .../duct- \
+      --sample-interval 1 --report-interval 5 \
+      compbench run --codec zstd ...",
+ "inputs":  ["sourcedata/aind-ephys-compression/.."],
+ "outputs": ["derivatives/../aind-634568__zstd.."],
+ "exit": 0
+

The recorded command is a duct + invocation — so every cell commits its own resource trace beside its own results.

+
+ + + + + + + + + + + + + + + +
commits[DATALAD RUNCMD]of those, wrapping ductcellsrecorded computeAI disclosure
126512161095912, 0 errors3838 h (peak RSS 120 GB)26 Co-Authored-By: Claude
+ +
+ +
+

So I tried to rerun it. Live, on this laptop.

+ + + +
+
$ datalad clone .../compression-comparisons-t261-study
+$ datalad get code/compression-comparisons-tools
+  ✗ fatal: repository not found
+
+$ # .gitmodules says:  url = ../..
+$ #  → resolves to https://github.com/ (not a repo)
+$ # fine on the author's disk; breaks for a cloner
+
+$ git clone .../dandi/compression-comparisons \
+     code/compression-comparisons-tools # by hand
+$ uv venv envs/compbench && uv pip install ...
+
+$ datalad rerun 42dc5cfa
+  run(ok): ... [duct --clobber --output-prefix ..]
+  Wall Clock Time: 0.549 sec   RSS peak: 7.8 MB
+ 		+ Did the science come back? + + + + + + + + + +
fieldoriginal vs. rerun
cr1.51429112246829 same
encoded_bytes5283 same
rmse0.0 same
round_trip_oktrue same
signal_std706.609486021761 same
*_time_sdiffer — as they must
manifestkernel 6.12→7.1,
python 3.13.15→3.13.5
+

+ Different machine, different kernel, six weeks later — + every scientific number came back bit-for-bit. + Only timings and the environment record moved. +

+
+ +
+ And the checklist earned its keep: the one thing that broke was + Distributable — a relative submodule URL that resolved only on + the author’s own disk. S, T, A and + M all held. You do not find that by reading your own repo; + you find it by being someone else. +
+ +
+ +
+ + + +
+ +
+

So what do we build next?

+

Not more code. Harnesses and skills.

+
+ +
+

Agent = Model + Harness

+

+ formulation borrowed from Oscar Esteban, OHBM 2026 +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Harness pieceWhat it isSTAMPED property
AGENTS.md / CLAUDE.mdThe project’s operational contract: conventions, commands, what not to doSelf-contained
SkillsA named, reusable procedure with its own references and scriptsModular, Actionable
Hooks / CIGuard rails that run whether or not anyone remembered toTracked
MCP serversThe port to external systems — archives, trackers, databasesDistributable
Container / sandbox
(con/yolo)		Where it is allowed to run, and how much it can breakPortable, Ephemeral
+
+ The harness is itself a research object. Which means it can be + self-contained, tracked, modular, portable — or it can be a pile of prompts in your notes app. +
+
+ + +
+

Where are you on the ladder?

+
+ +

+ Ladder figure & framing: + Jo Van Eyck, + “You are Not Ready: Agentic coding in 2026” +

+ 		+ + + + + + + + + + + + + + + + + + + + + +
LevelWhat it looks likeThe skill you have to acquire
L1 ChatBrowser tab; copy-pastePrompt & context engineering
L2 Mid-loopIn the IDE — “autocomplete on steroids”Critically evaluating alternatives, not accepting them
L3 In-the-loopThe babysitting phase — you watch it struggleDoom loops, guard rails, extracting reusable skills. 2–3 months, minimum.
L4 On-the-loopSpec it, hand off, verify the artifactEverything above plus a real harness: skills, tests, sandboxes, scanners
L5 Multi-agentOrchestration; the frontierEveryone is still figuring this out
+
+ L4–L5 is exactly where a bad harness stops being survivable — + which is why the harness, not the model, is the thing to invest in. +
+
+ +
+ +
+

And the loop that gets you up it

+ + + + + +
+
+%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '17px'}}}%% +flowchart TD + IDEA["Idea"] --> S1["1. Overall design / spec"] + S1 --> S2["2. Aspect design / prompt"] + S2 --> S3["3. AI work"] + S3 --> S4["4. Review / test"] + S4 -->|"iterate"| S2 + S4 --> S5["5. Compound into spec / SKILL"] + S5 -->|"next aspect"| S2 + S5 -->|"adjust / memorize"| S1 + + style IDEA fill:#9c27b0,color:#fff,stroke:#7b1fa2,stroke-width:2px + style S1 fill:#1565c0,color:#fff,stroke:#0d47a1,stroke-width:2px + style S2 fill:#2e7d32,color:#fff,stroke:#1b5e20,stroke-width:2px + style S3 fill:#e65100,color:#fff,stroke:#bf360c,stroke-width:2px + style S4 fill:#c62828,color:#fff,stroke:#b71c1c,stroke-width:2px + style S5 fill:#283593,color:#fff,stroke:#1a237e,stroke-width:3px +
+ 		+ + + + + + + + + + + + + + + +
Development modelStages usedWhat is missing
Vibe coding2 → 3 → 4 (loop)No spec, no condensing — the knowledge evaporates with the session
Spec-driven1 → 2 → 3 → 4 → 2…Loops back to the prompt; the spec should be updated too
Compound engineering1 → 2 → 3 → 4 → 5 → 1+2…Nothing — each cycle makes the next one cheaper
+
+ Stage 5 is where a skill is born. + Vibe coding throws the lesson away; compound engineering writes it down — + and what it writes down is exactly the artifact we just said must be STAMPED. +
+

+ Tooling for stage 1: spec-kit · + OpenSpec · + compound-engineering · + LAD + (in use in dandi-cli) +

+
+ +
+ +
+

That was the individual ladder. Here is the team one.

+ +

+ Figure reproduced from: + E. C. Johnson, T. T. Nguyen, B. K. Dichter, …, + M. E. Martone, F. Pestilli, D. Yatsenko. + SciOps: Achieving Productivity and Reliability in Data-Intensive Research. + arXiv:2401.00077, 2024. +

+
+ The preprint names BIDS, NWB, DataLad/git-annex, DANDI and brainlife.io as its own + Level-3 exemplars — and reserves Level 5 (AI + human in the loop) for the top. +
L5 presupposes L3 underneath it. You do not get closed-loop discovery + on top of study_final2/. +
+ +
+ +
+

Right now, everyone is rebuilding the same private harness

+

+ A hundred labs, each writing their own “how to work with BIDS” prompt.
+ None of them shared. None of them versioned. None of them tested. +

+

+ We have seen this failure mode before. It was called
+ “every lab has its own analysis scripts”, and we fixed it with standards and packages. +

+
+ Skills are the new reusable module. + Apply Modularity and Distributability to the harness, + or watch AI recreate the silo problem at machine speed. +
+ +
+ +
+

We are already doing this — upstream, in public

+

+ K-Dense-AI/scientific-agent-skills + — 177 scientific skills, + Agent Skills spec, MIT-licensed, scanned and reviewed. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
ContributionWhoWhat it carries
skills/bids
+ PR #125 — merged		ours (CON)Entity grammar & 35 entities · the + machine-readable schema shipped as a reference · + all 25 BEPs · + HeuDiConv/ReproIn/dcm2bids/BIDScoin conversion paths · + validation · derivatives & BIDS-Apps
skills/datalad
+ PR #227 — merged		Dylan Pulver,
with our upstream review		Clone/get semantics & the “pointers are not data” failure · + datalad run / rerun / containers-run as the provenance spine · + publishing to siblings · STAMPED landed in + references/provenance.md on our suggestion
PR #265 — openBrent McPherson
(McGill)		Maps all seven STAMPED properties onto concrete DataLad commands, + plus a read-only evidence probe per property + (datalad subdatasets -r for S/M, git log --grep '[DATALAD RUNCMD]' for T, + datalad rerun --report for A, datalad siblings for D…)
issue #217 — closedours (CON)“Should STAMPED be its own skill?” — and the still-open question underneath: + where should domain skills live? An upstream monorepo, or the project’s own repo, with discovery on top?
+

+ Note the shape: a community member wrote the DataLad skill, we reviewed it, and a third party + then extended it with STAMPED. That is exactly how a standard spreads — and it is + now happening to harnesses. +

+
+ +
+

And domain-agnostic ones: con/skills

+ + + + + + + + + + + + + + + + + +
SkillWhat it automates
introduce-reuse-complianceSPDX / REUSE licensing metadata — including for BIDS datasets
credit-contributionsCRediT author-contribution statements from git history
tinuous-analyzerCI-failure investigation across archived con/tinuous logs
bisect-and-patch-git-annexBisect a git-annex regression from CI logs and produce a red/green PR
issue-triage · pr-feedback-reviewTriage and review work, cross-referenced against the codebase
introduce-codespell · introduce-intuit-autoBootstrap spell-checking / automated releases into any project
+ +
+ Each of these started as a thing we did by hand three times and got tired of. + Writing it down as a skill is how the n-th time gets cheaper — for us, + and for whoever clones the repo. +
+

+ Deeper dive on the workflow around these: + “A few words of intro into AI assisted coding” + (CA Origami Retreat, 2026) — also in the appendix of this deck. +

+
+ +
+ + + +
+ +
+

Two regimes — and be honest about which one you are in

+

+ Either the system checks itself against goals and spec — + or the work items stay small enough that a human still can. +

+ + + + + +
+ A. Scaffolded self-verification
+ the agentic system keeps itself in sync +
    +
  • An executable spec the loop is checked against + (spec-kit, + OpenSpec, + LAD)
    • +
  • Tests and validators as the oracle — the loop must be able to fail
    • +
  • Provenance so drift is detectable at all + (datalad run, duct, CI history)
    • +
  • Automated gates before human eyes; humans review intent and architecture
    • +
+
+ Only then are large autonomous batches defensible. +
+ 		+ B. Small, human-reviewable, hand-tried
+ where almost every research project actually is +
    +
  • A diff a human can actually read is a design constraint, not a courtesy
    • +
  • One concern per change; small commits; mark them (@pytest.mark.ai_generated)
    • +
  • Run it by hand before you believe it. Agent-rate output, human-rate trust
    • +
  • Cheap to back out — the reversibility is the safety mechanism
    • +
+
+ Not a lesser mode — the correct default until A is actually built. +
+
+ +
+ There is no third regime where you ship large, unverified agent output safely. + And the way out of B is not a better model — it is + Actionable specs, Tracked provenance and + Modular units: the same STAMPED properties, again. +
+

+ “Well, less is more” — Robert Browning, Andrea del Sarto, 1855; + later the motto of Mies van der Rohe, who credited his mentor Peter Behrens rather than claiming it. +

+ +
+ +
+

We are not the only ones saying this

+ + + + + + + + + + + + + + + + + + + + + + + + + +
SourceWhat it found / argues
DORA 2025 + (Google Cloud)Higher AI adoption raises throughput and delivery instability. + Their prescription is literally regime A+B: + “working in small batches is a critical countermeasure to the risks of AI-assisted + development”, plus test automation and fast feedback — because + “verification is a fundamentally different cognitive task than creation”.
METR RCT, 202516 experienced OSS developers, 246 real issues, on repos they already maintained: + 19% slower with AI tools — while estimating they had been 20% faster. + The perception gap is the hazard, not the tooling. + (METR now labels this historical — tools moved on. The gap is the durable lesson.)
Addy Osmani“Proof over vibes.” Verification, not authoring, is the bottleneck; + break agent output into digestible commits; no PR without tests or a demo; + humans stay non-negotiable for security and architecture.
Simon Willison
+ ▶ Pragmatic Summit talk
+ ▶ Lenny’s Podcast		“Your job is to deliver code you have proven to work.” + The skill is instructing and then confidently verifying; a model cannot take accountability, you can. + And on regime B specifically — even with tests, he reports that + “I haven’t used it yet” still undermines his confidence in a change. + Trying it by hand is not superstition.
Faros telemetry + (~22k devs)PR size +51%, median time in review +441%, + incidents per PR +243%. Big agent diffs do not get reviewed — they get waved through.
+ +
+ Converging, from very different vantage points: + the safe size of an AI-assisted work item is set by the strength of your verification loop. + Build the loop, or keep the items small. Preferably both. +
+ +
+ +
+

HI ↔ AI — every project picks its own policy

+

+ Rows 1–2 live in regime B; rows 3–4 are only honest once + regime A is actually built. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
StanceOSS exemplarsCON / adjacent exampleLeans on
Reject AI-generated contentZig, Krita, Clojure, QEMUgit-annex upstream — pure-HI commitsAI stays ephemeral; never enters the tracked artifact
Accept with disclosureNumPy, Kubernetes, Linux kernel, DjangoDataLad, DANDI — + Co-Authored-By: Claude… trailers, @pytest.mark.ai_generatedTracking — provenance is non-negotiable
Spec-driven AI-generated
(HI specs, AI writes, HI reviews)		Common in practice; rarely a declared policyAnnexTube, + citations-collector, + parts of dandi-cliT + A + M — the spec is the contract
Autonomous agents in the loopBrain Researcher + (Poldrack lab) — agentic neuroimaging analysis; + K-Dense AI — “AI co-scientists” + (and the host of the skills library we contribute to)con/skills + + con/yolo for triage, review, dep updatesAll of STAMPED — the harness becomes the type-checker
+

+ Common ground, all four: AI cannot be an author + (ICMJE, Jan 2026); + humans retain full responsibility; AI use is disclosed inside the artifact. + STAMPED Tracking is what makes that mechanical rather than a promise. +

+

+ Catalog of declared OSS stances: + melissawm/open-source-ai-contribution-policies. + — Note what the row-4 exemplars are: harnesses. + Brain Researcher constrains the agent to + acceptable analyses and required validations rather than trusting it — + autonomy is bought with structure, not with a bigger model. +

+
+ +
+

Take home — what to do Monday

+
    +
  1. Make one object actionable. Pick one study. Put it in a declared standard + (BIDS, NWB) + and run the validator. The agent gets a grammar; you get a type-checker.
    2. +
  2. Record one command. Wrap your next analysis step in + datalad run + — or at minimum in duct. + A provenance record you did not have to remember to write.
    3. +
  3. Grade one thing against STAMPED. + checklist.stamped-principles.org, + fifteen minutes, and you will know which property is your weakest.
    4. +
  4. Write down your harness. An + AGENTS.md in the repo beats a prompt in your notes app. + Declare your AI policy there while you are at it.
    5. +
  5. Size the work to the loop you actually have. Until the project + self-checks, keep AI-assisted items small, reviewable, and tried by your own hand — + and say so out loud, so nobody mistakes regime B for regime A.
    6. +
  6. Push one skill upstream. The next person who needs your procedure should + clone it, not re-derive it. + scientific-agent-skills, + con/skills, or your own repo — but somewhere public.
    7. +
+
+ Standards and provenance were always the right answer. + AI did not change the answer — it changed the deadline. +
+
+ +
+ + + +
+

Acknowledgements

+ + + + + +
+
+
STAMPED
+
+
    +
  • Austin Macdonald, Cody Baker, Isaac To
    • +
  • Brent McPherson (STAMPED → DataLad commands)
    • +
+
+
Software
+
+
    +
  • Joey Hess (git-annex)
    • +
  • The DataLad team & contributors
    • +
  • The BIDS, NWB & DANDI communities
    • +
+
+
Some slides origin:
+
+ +
+
+ 		+
Funders
+ + + + +
+ + + + +
+
Collaborators
+
+ + + + +
+
+ + + + +
+
+
+ + + +
+

Thank you!

+

Structure your objects, and
let both intelligences work on them

+ +

+ stamped-principles.org · + checklist · + examples · + con/skills · + con/duct +

+
+ + + +
+ +
+

Appendix

+

Material cut for time — press ↓ to walk it, or follow the links.

+
+ +
+

STAMPED and FAIR do different jobs

+ + + + + + + + + + + + + + + + + +
FAIRSTAMPED
About sharing: can someone else find this and get it?About the object being operational on its own: can someone — or something — re-run it?
A FAIR dataset can still be a .zip of .mat files with a DOI.A STAMPED object carries its own layout, environment, and execution record.
Graded by a repository.Graded by a checklist you can walk, item by item.
+
+ Not competitors — different lenses on the same stack. FAIR tells you whether the world can reach it. + STAMPED tells you whether anything useful happens once it does. +
+
+ +
+

Portable + Ephemeral: the environment travels with the record

+ + + + + +
+
$ datalad containers-run \
+    -n repronim-mriqc \
+    -m "run MRIQC on sub-01" \
+    --input sub-01 --output derivatives/mriqc \
+    '{inputs}' '{outputs}' participant
+
    +
  • Container image is a tracked subdataset, not a latest tag that drifts
    • +
  • The commit records image and command together
    • +
  • Bind-mount only the current module — look up you must not
    • +
  • Throw the container away afterwards: that is Ephemerality, and it is already how agents run
    • +
+ 		+ +

ReproNim/containers — auto-rebuilt, versioned collection

+
+

+ “Depend on where you run, you should not.” An agent’s sandbox is not your laptop — + which makes portability a correctness property, not a courtesy. +

+
+ + +
+ +
+ +
+ +
+ +
+ + + + +
+
+ + + + + + + + + + diff --git a/2026-reproAI-MIT-STAMPED/PLAN.md b/2026-reproAI-MIT-STAMPED/PLAN.md new file mode 100644 index 00000000..6ee465ce --- /dev/null +++ b/2026-reproAI-MIT-STAMPED/PLAN.md @@ -0,0 +1,401 @@ +# PLAN — `2026-reproAI-MIT-STAMPED.html` + +**Venue**: LLM workshop, MIT — **Friday, 2026-10-16** +**Slot**: 25 min talk + 5 min Q&A +**Title**: *STAMPED principles and their role in the age of AI-driven +scientific software development* +(Dave's original framing — "STAMPED and how coding is transformed and +the new scientific framework for code/software development" — kept as +the speaker-note gloss, not on the slide.) + +**Live URL / QR**: +`https://datasets.datalad.org/centerforopenneuroscience/talks/2026-reproAI-MIT-STAMPED.html` +→ `pics/2026-reproAI-MIT-STAMPED-qrcode.png` (generated at authoring time) + +--- + +## 1. Thesis + +> Agents do not fail for lack of intelligence. They fail for lack of +> **structure to act on**. The same properties that made a research +> object reusable by a human — STAMPED — are what make it *actionable* +> by an agent. And the next thing we owe the commons is not more code: +> it is **shared harnesses and skills** that encode how to act on +> standardized objects. + +Three claims, in order: + +1. **What changed**: the bottleneck moved from *writing* code to + *verifying and situating* it. Agent-rate output demands agent-rate + review — which only provenance and standards can supply. +2. **What did not change**: STAMPED (and YODA before it) already named + the properties. **Actionability** is the load-bearing one for agents, + and **standards are what make Actionability cheap** — they are the + scaffolding *and* the shared vocabulary across HI ⇄ AI. +3. **What is next**: harnesses and skills become first-class, shareable, + *modular* research artifacts. We are already contributing them + (BIDS + DataLad skills upstream); the field should treat this as + infrastructure work, not as prompt-hoarding. + +## 2. Arc + +Hybrid of `SOUL.md` §7 arc 2 (Challenges → Solutions → Take-home) with +the AI-ladder flavor of arc 5. Parent decks for layout/slide theft: + +| Parent | What is borrowed | +| ----------------------------------------- | -------------------------------------------------------------------------- | +| `2026-repronim-YODA-BIDS-webinar.html` | Title-slide shape, `datalad run`/`rerun` record, `con/duct` slides, "Look up you must not", YODA-beyond-code-and-data | +| `2025-distribits-YODA.html` | Same YODA spine (grandparent) | +| `2026-ca-origami-retreat-aicoding.html` | Reality-Check GIF, AI-coding ladder (→ appendix), `con/skills` table, Thank-you closer | +| `2026-usrse-con-talk.html` | HI ⇄ AI policy table, "provenance doesn't care who wrote the code" beat, Acknowledgements | +| `2026-nih-bids2.0.html` | YODA → STAMPED transition, `stamped-principles-webshot_20260602.png` | +| `2024-distribits-datalad.html` | Canonical Acknowledgements layout | + +**External credit (explicit, on-slide):** +[Oscar Esteban, *Using LLMs in Neuroimaging Research: LLMs for Coding*, +OHBM 2026 Educational](https://www.nipreps.org/presentations/2026-OHBM-AI-Educational/) +— the "Agent = Model + Harness" formulation, `AGENTS.md` as the +operational contract, and the grounding-beats-search result. This talk +deliberately picks up *where Oscar's leaves off*: he showed how to build +the harness; this one argues about **what the harness stands on** and +**how harnesses themselves get shared**. + +## 3. Slide-by-slide (34 main slides + 7-slide appendix, for 25 min) + +### Act 0 — Frame (≈2 min) + +1. **Title slide** — standard `SOUL.md` §3 template, QR, logo strip. +2. **Reality Check** — `idiocracy-fixed.gif` disclaimer. + *(borrowed: aicoding deck §Reality Check)* +3. **Previously on… / standing on two talks** — ReproNim YODA webinar + (mine) + Oscar's OHBM AI Educational (his). States the hand-off. + +### Act I — What actually changed (≈4 min) + +4. **"It's 2026. Agents write the code."** — black beat slide. +5. **The bottleneck moved** — writing → verifying + situating. + `2026-ai-intensifies.png` (HBR) as the visual. +6. **Without structure, agents go wild** — the central claim slide. + Left column: what an agent sees in an ad-hoc `~/data/study_final2/`. + Right column: what it sees in a BIDS dataset. Same model, different + affordances. +7. **This has a name** — YODA (2018) → STAMPED (2026). + `pics/stamped-yoda.svg` / `pics/yoda-principles-reordered.png`. + +### Act II — STAMPED, and what AI did to Actionability (≈7 min) + +7b. **Section opener** — the STAMPED banner + (`pics/borrowed/stamped-banner_ver-3.svg`, the project's own artwork: + the seven words with their icons) replacing the former text-only + opener, with the "whoever, *or whatever*, picks it up next" line as a + fragment underneath. +8. **STAMPED in one table** — all seven, one line each, from + . Third column: *what it buys an + agent*. Paper + preprint DOI cited. +9. **Actionability is the one that agents live or die on** — and + standards are what make it cheap. "You've seen one BIDS dataset…" + lands twice as hard for a model that would otherwise *infer*. +10. **…and AI changed what counts as actionable** (NEW — the slide the + talk now pivots on). A three-row escalation: + - *Before*: an exact command / API / documented CLI. Tools and + humans-who-already-knew could act; everything else stayed a + paragraph in the Methods. + - *Now*: a procedure **described in human language**, with scripts + dropped in where precision matters → **any agent, on first + contact**. The interface moved from *syntax* to *language*. + - *Next*: routines that fire **without being asked** → the + **harness** acts (CI, PR monitors, scheduled agents). + Then two columns: **(1) Skills** — language carries the judgement, + scripts carry the precision, one skill one job (modularity still + applies); **(2) Routines** — Claude Code on the web watching a repo, + scheduled agents, CI that does more than run tests. + *Punchline:* AI did not only make more things actionable — it made + **automation on top of actionability cheap**, and that is the real + multiplier. +11. **So grade *them* too: a skill is a research object** (NEW — the + recursive turn). Seven-row table mapping every STAMPED property onto + a skill/automation (S: carries its own references, not "see our + wiki"; T: `metadata.version` + history; A: runnable, not only prose; + M: one skill one job; P: no hardcoded paths; E: fresh sandbox; + D: published, not pasted into chat). + *Punchline:* **good automation and good skills pass the STAMPED + checklist — that is precisely what makes them reusable beyond the + person who wrote them.** Footer carries the checklist / examples / + principles / `main.tex` links (absorbed from the old + "Operationalized, not aspirational" slide). + +### Act III — What makes it actionable in practice (≈6 min) + +12. **Standards = the agent's grammar** — BIDS as the worked example. + `BIDS-minder.svg`; the schema is *data an agent queries*, the + validator is the type-checker. *(marked SKIP CANDIDATE in-file)* + **Ends with the explicit jump into the next two slides**: *"a layout + only tells you what the files are — it says nothing about how they + came to be, or what it cost."* + **Standard → `datalad run` → `duct` = what it is → how it was made → + what it took.** Three layers, and an agent needs all three. This is + the connective tissue Act III previously lacked. +13. **Tracking: `datalad run`** — the RUNCMD commit record. +14. **Provenance doesn't care who wrote the code** — AI-written or + hand-written, same record shape; the tests don't know either. +15. **`con/duct`** — self-contained execution logs. + *(marked SKIP CANDIDATE in-file — the next slide carries the argument)* +16. **…and that log IS the provenance you need later** — three operator + questions answered by the trace rather than by recollection; + `REPRONIM_USE_DUCT=1`. +17. **All three layers, one real artifact — built with AI** (NEW): the + [`dandi/compression-comparisons-t261-study`](https://github.com/dandi/compression-comparisons-t261-study) + worked example, which *is* the BIDS→run→duct bridge in one object. + Left: the BIDS-shaped, YODA-composed layout (tool and 523 GB data as + **subdatasets pinned by commit**). Right: a real `[DATALAD RUNCMD]` + record whose `cmd` **is** a `duct` invocation, so every cell commits + its own resource trace beside its own results. + Verified counts (read from the repo 2026-10-06): **1265 commits · + 1216 `[DATALAD RUNCMD]` · 1095 of those wrapping `duct` · 912 cells, + 0 errors · 3838 h of recorded compute, peak RSS 120 GB · 26 + `Co-Authored-By: Claude` trailers.** +18. **So I tried to `rerun` it. Live, on this laptop.** (NEW) — the + receipt. See §4b below for what actually happened. + +### Act IV — Harnesses, the two ladders, and shared skills (≈8 min) + +19. **Agent = Model + Harness** — credited to Oscar's OHBM deck. + Anatomy table: `AGENTS.md` / skills / hooks / MCP / sandbox, each + mapped onto the STAMPED property it satisfies. +20. **Where are *you* on the ladder?** (NEW, promoted from the appendix) + `ai-ladder-skills.png` beside a condensed L1–L5 table (what it looks + like / the skill you have to acquire). L3 is the 2–3-month tax + nobody skips and is where reusable skills get extracted. + *Punchline:* **L4–L5 is exactly where a bad harness stops being + survivable — invest in the harness, not the model.** + Credit: Jo Van Eyck, *You are Not Ready: Agentic coding in 2026*. +21. **And the loop that gets you up it** (NEW, promoted): the 5-stage + mermaid beside the **development-model mapping** — vibe coding + (2→3→4, knowledge evaporates) / spec-driven (1→2→3→4→2…) / + compound engineering (…→**5**→1+2…, each cycle cheaper). + *Punchline:* **stage 5 is where a skill is born** — and what it + writes down is exactly the artifact slide 11 said must be STAMPED. +22. **That was the *individual* ladder. Here is the *team* one.** (NEW) + The SciOps maturity figure, converted from + `pics/borrowed/neuro-sciops.pdf` → `pics/borrowed/neuro-sciops-fig.svg` + (**vector**; the source PDF is pure vector — 0 rasters, 203 drawing + ops, text already outlined — so SVG keeps it sharp at any projector + scale and in the PDF export, and at 403 KB is *smaller* than the + 439 KB raster it replaced). + **Figure explicitly credited to the SciOps preprint** + (Johnson, Nguyen, Dichter, …, Martone, Pestilli, Yatsenko; + arXiv:2401.00077, 2024) — *not* presented as this talk's own. + The preprint names BIDS / NWB / DataLad / DANDI / brainlife.io as + its own L3 exemplars and reserves L5 for AI+human-in-the-loop. + *Punchline:* **L5 presupposes L3 — no closed-loop discovery on top + of `study_final2/`.** +23. **Everyone is rebuilding the same private harness** — the silo + failure mode, at machine speed. *(marked SKIP CANDIDATE in-file)* +24. **We are already doing this — upstream, in public** — the + `K-Dense-AI/scientific-agent-skills` contribution table. +25. **`con/skills`** — our own domain-agnostic set. + +### Act V — The honest thesis, and close (≈6 min) + +26. **Two regimes — and be honest about which one you are in** (NEW — + *the thesis slide*). Claim: **the safe size of an AI-assisted work + item is set by the strength of your verification loop, and nothing + else.** + - **A. Scaffolded self-verification** — an executable spec the loop + is checked against (spec-kit / OpenSpec / LAD); tests and + validators as the *oracle* (the loop must be able to **fail**); + provenance so drift is detectable; automated gates before human + eyes. *Only then* are large autonomous batches defensible. + - **B. Small, human-reviewable, hand-tried** — a diff a human can + actually read is a **design constraint, not a courtesy**; one + concern per change; **run it by hand before you believe it**; + cheap to back out. *Not a lesser mode — the correct default until + A is actually built.* + - *Punchline:* there is no third regime where large unverified agent + output ships safely; and the way out of B is not a better model + but **A**ctionable specs, **T**racked provenance, **M**odular + units — STAMPED again. + - Kicker, correctly attributed: *"Well, less is more"* — Robert + Browning, *Andrea del Sarto*, **1855**; later Mies van der Rohe's + motto, and he credited Peter Behrens rather than claiming it. +27. **We are not the only ones saying this** (NEW — the receipts; + marked SKIP CANDIDATE, ideal to hold for Q&A). See §4 below. +28. **HI ⇄ AI: every project picks its own policy** — 4-row table. + The *Autonomous* row now names real exemplars: + [**Brain Researcher**](https://brain-researcher.com/) (Poldrack lab; + arXiv:2608.19902) and [**K-Dense AI**](https://github.com/K-Dense-AI/) + (whose skills library we contribute to). Footnote makes the point + that both are **harnesses** — *autonomy is bought with structure, + not with a bigger model*. ICMJE Jan 2026 + melissawm catalog. + Now prefaced with the regime mapping: **rows 1–2 live in regime B; + rows 3–4 are only honest once regime A is actually built.** +29. **Monday checklist** — 6 actions (added: *size the work to the loop + you actually have, and say so out loud so nobody mistakes B for A*). +30. **Acknowledgements** — canonical layout. +31. **Thank you** — STAMPED/Yoda sign-off. + +### Appendix (vertical sub-deck, skipped live) + +- **STAMPED vs. FAIR** — different jobs (demoted from Act II). +- **Portable + Ephemeral** — `datalad containers-run` + + ReproNim/containers (demoted from Act III). +- **AI coding ladder, the long version** — per-level detail, time costs, + the "don't start at L5" warning. +- **BEP028** — provenance lifted into a community standard. +- **`con/yolo`** (ephemerality + portability applied to the agent itself) + and **`con/serve`**. +- **Related CON talks** table. + +## 4. External evidence used (and where) + +| Claim on a slide | Source | +| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | +| Structure beats scale: tool-selection 23.3% → 93.6% across 7 models | Chen, …, Poldrack, *Bringing analytic rigor to agentic AI for science*, [arXiv:2608.19902](https://arxiv.org/abs/2608.19902), 2026 — footnote on the hinge slide | +| SciOps maturity figure + L3 exemplars + L5 = AI-in-the-loop | Johnson *et al.*, [arXiv:2401.00077](https://arxiv.org/abs/2401.00077), 2024 — credited on-slide | +| Agent = Model + Harness; `AGENTS.md` as contract; grounding wins | [Oscar Esteban, OHBM 2026 Educational](https://www.nipreps.org/presentations/2026-OHBM-AI-Educational/) | +| AI coding maturity ladder L1–L5 | [Jo Van Eyck, *You are Not Ready: Agentic coding in 2026*](https://www.youtube.com/watch?v=6W_-YWHKwZ4) | +| Autonomous-stance exemplars | [brain-researcher.com](https://brain-researcher.com/), [K-Dense AI](https://github.com/K-Dense-AI/) | +| AI cannot be an author | [ICMJE, Jan 2026](https://www.icmje.org/recommendations/browse/artificial-intelligence/ai-use-by-authors.html) | + +### Backing for the two-regimes thesis (slide 27) + +| Source | What it supplies | +| ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| [DORA 2025](https://dora.dev/insights/balancing-ai-tensions/) (Google Cloud) | AI adoption raises throughput **and** delivery instability; *"working in small batches is a critical countermeasure to the risks of AI-assisted development"*; *"verification is a fundamentally different cognitive task than creation"* | +| [METR RCT](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/), 2025 | 16 experienced OSS devs, 246 real issues on repos they maintain: **19% slower** with AI while estimating **20% faster**. METR now labels it historical — cite it for the *perception gap*, not as a current benchmark | +| [Addy Osmani](https://addyo.substack.com/p/code-review-in-the-age-of-ai) | *"Proof over vibes"*; verification is the bottleneck; *break agent output into digestible commits*; no PR without tests or a demo | +| [Simon Willison](https://simonwillison.net/2025/Dec/18/code-proven-to-work/) | *"Your job is to deliver code you have proven to work"*; the skill is instructing **and then confidently verifying**; a model cannot take accountability | +| [Faros telemetry](https://www.faros.ai/blog/key-takeaways-from-the-dora-report-2025) (~22k devs) | PR size **+51%**, median time in review **+441%**, incidents per PR **+243%** | + +Attribution note for the kicker: *"less is more"* first appears in +Robert Browning's *Andrea del Sarto* (1855) — **not** Mies van der Rohe, +who adopted it as a motto and himself credited Peter Behrens +(AEG turbine factory façade, ~1907). The common misattribution to Mies +is worth getting right on a slide in a deck that otherwise insists on +sourcing. Robert Venturi's *"Less is a bore"* (1966) is the standard +rebuttal if anyone wants the counter-aphorism. + +## 4b. The rerun experiment (slide 18) — what was actually done + +Run on 2026-10-06, from a clean clone, by someone who is not the author. + +**It worked, and the science came back bit-for-bit.** + +| | | +| --------------------- | ------------------------------------------------------------------------------------------- | +| Cell chosen | `42dc5cfa` — `synthetic-tiny__blosc-zstd-level_3-shuffle_byte` | +| Why that one | `duct` recorded it at **0.5 s / 9 MB RSS**; its only input is a config file in the tool submodule, so none of the 523 GB source data is needed. *The record told me which cells were cheap — that is itself the argument for `duct`.* | +| Setup | `uv tool install datalad`; `uv venv envs/compbench`; `uv pip install './code/compression-comparisons-tools[pipeline]' con-duct` | +| Command | `datalad rerun 42dc5cfa` | +| Result | `run(ok)` — wall clock **0.549 s**, peak RSS **7.8 MB** | +| **Identical** | `cr` 1.5142911224682944 · `encoded_bytes` 5283 · `rmse` 0.0 · `round_trip_ok` true · `signal_std` 706.6094860217612 | +| **Differed** | `encode_time_s` / `decode_time_s` and their derived `*_xrt` — *as they must* | +| **Also differed** | `manifest.json`: timestamp, originating commit, kernel 6.12→7.1, python 3.13.15→3.13.5 | + +### The one STAMPED failure, and it is a real one + +`datalad get code/compression-comparisons-tools` **fails for any outside +cloner**. `.gitmodules` carries `url = ../..`, which git resolves against +the superproject URL to `https://github.com/` — not a repository. It worked +on the author's disk (`datalad-url = /home/yoh/proj/dandi/compression-comparisons`) +and nowhere else. Worked around by cloning +`https://github.com/dandi/compression-comparisons` into place by hand. + +- **S, T, A, M held. D did not.** +- One-line fix: `url = ../compression-comparisons`. +- This is the honest payoff of the whole deck: *you do not find that by + reading your own repo; you find it by being someone else.* Present it as + a finding, not a gotcha. + +**Caveat to state out loud:** I reproduced a 0.5-second smoke cell, not a +17-hour production cell. The claim is "the mechanism works end to end", +not "the whole 3838-hour study was re-verified". + +## 4c. Companion video playlist (ReproNim channel) + +A deliverable in its own right, specified in [`playlist.md`](playlist.md): +an 8-video playlist for the [ReproNim channel](https://www.youtube.com/@repronim) +collecting every video the talk references, so the audience can follow them +from one place. + +Metadata (title, channel, date, duration, **licence**) was read from YouTube +with `yt-dlp` on 2026-10-06 — licences are verified, not assumed, because +they decide what may be mirrored: + +- **4 videos are CC-BY** (all on the DataLad channel: Pragmatic YODA, + DataLad-Registry, git-annex-is-complete, compute-on-demand) → mirrorable. +- **1 is ReproNim's own** (the YODA webinar). +- **3 are Standard YouTube Licence** (Jo Van Eyck; Simon Willison ×2) → + **linked, never mirrored.** + +**ReproTube status: nothing outstanding.** Checked against + on 2026-10-06 — all five +mirrorable videos are *already archived* under their own channels, and the +archive is syncing current (a 2026-10-05 ReproNim video is present). + +One open design item, not urgent: `con/annextube` models **channels** and +mirrors each channel's own playlists; it has no notion of a curated +**cross-channel** playlist. Recommended default is to leave it — every +mirrorable video is already archived, and the playlist is a *view* +("dashboards visualise, don't own"). The alternative — a small annextube +feature storing an ordered id list plus symlinks into existing per-channel +dirs, skipping licence-forbidden videos — is written up in `playlist.md`. + +## 4d. PDF export + +Use the repo-level helper (**not** talk-specific): `tools/mkpdf.py`, +documented in `../README.md` alongside the manual `?print-pdf` and +decktape-via-Docker routes. + +```bash +uv run --with playwright playwright install chromium # once +uv run --with playwright python tools/mkpdf.py 2026-reproAI-MIT-STAMPED.html +``` + +Verified 2026-10-06: **41 pages for 41 slides**, 9.4 MB, SVGs stay +vector, mermaid and slide backgrounds render. Also checked against +`2024-distribits-datalad.html` (16:9, 45 pages) to confirm the page size +really is taken from each deck's own config rather than hardcoded. + +Do **not** commit the PDF — it is a derived artifact and the live HTML is +the thing to share. Generate on demand for anyone wanting an offline copy. + +## 5. Links discipline + +Per `~/.claude/CLAUDE.md` §Slides: every issue/PR/repo/paper reference +on a slide is a real ``; every count links to the live filter +that produces it (e.g. "177 skills" → the repo's `skills/` tree; "15.2% +of DataLad repos carry RUNCMD" → the usage dashboard). A one-liner on +the first content-dense slide tells the audience counts are links. + +## 6. Open TODOs before delivery + +- [ ] Confirm the workshop's official name for the title slide + (placeholder reads "LLM Workshop · MIT"). +- [ ] PR #265 may merge before Oct 16 — re-check its state and flip the + slide's "open" badge if so. +- [ ] Optional: webshot of `K-Dense-AI/scientific-agent-skills` + `skills/bids/SKILL.md` for slide 20 (currently text + links only). +- [ ] Optional: refresh the DataLad usage-registry RUNCMD percentage. +- [ ] `pics/borrowed/neuro-sciops-fig.svg` is **derived** from the annexed + `pics/borrowed/neuro-sciops.pdf`. Regenerate with: + `uv run --with pymupdf python -c "import pymupdf; open('pics/borrowed/neuro-sciops-fig.svg','w').write(pymupdf.open('pics/borrowed/neuro-sciops.pdf')[0].get_svg_image(text_as_path=True))"` +- [ ] Publish the companion YouTube playlist on the ReproNim channel — + see [`playlist.md`](playlist.md) for the ordered list, verified + licences, the creation recipe, and the ReproTube archival status. + Then backfill the playlist URL into this file, `playlist.md`, + `INDEX.md` and the deck's appendix. +- [ ] Optionally report the `.gitmodules` `url = ../..` bug upstream on + `dandi/compression-comparisons-t261-study` (one-line fix). Doing so + *before* the talk would be a nice "and it is already fixed" beat; + leaving it lets the slide stand as a live example. Speaker's call. +- [ ] Timing: **34 main slides for 25 min — this is now well over budget.** + Four slides carry a `SKIP CANDIDATE` note in their + `