diff --git a/2026-dhmc-ieeg.html b/2026-dhmc-ieeg.html index 735a3a7..13817a7 100644 --- a/2026-dhmc-ieeg.html +++ b/2026-dhmc-ieeg.html @@ -342,44 +342,110 @@

Federally regulated but locally “managed”

- -

NWB: Neurodata Without Borders

- - +

NWB: one file format for all modalities + metadata

+ + +
+ +
+ + +
+ +
+ +

+ neuroconv.readthedocs.io +

+
- +
- -

neurosift: stream, don't download

+ +

Neurosift: stream, don't download

One known iEEG NWB example on DANDI: ds008610 (LFP only) — TODO: find/convert a second example with spectrograms if one exists.

+ +
+ +
+

Example 1: behavior, grouped by condition

+ +

+ Camera motion energy, aligned to auditory_cue_time, + grouped by gabor_stimulus_contrast + — DANDI:000409 +

+ +
+ +
+

Example 2: LFP aligned to reward

+ +

+ LFP aligned to reward_start_time, channels 0, 2, 4 + — DANDI:001539 +

+ +
+ +
+

Example 3: LFP around stimulus presentations

+ +

+ 40 Hz pulse-train presentations, probe D LFP, channels 5, 22, 25, 32 + — DANDI:001637 +

+
@@ -540,7 +606,7 @@

neurosift: stream, don't download

- +
-

dandi-compute

-
    -
  • TODO: dandi-compute demo — not yet documented anywhere we could pull from; - need a pointer/screenshot from whoever owns it
    • -
+ +

DANDI Compute

+ + +
+ +
+

DANDI Compute: orchestration

+
+ +
+
+ +
+

DANDI Compute: the spike-sorting pipeline

+
+ +
+
+ +
+

DANDI Compute: the derivative it produces

+
+ +
diff --git a/pics/2026-dhmc-ieeg-neuroconv-formats.svg b/pics/2026-dhmc-ieeg-neuroconv-formats.svg new file mode 100644 index 0000000..b53ea7f --- /dev/null +++ b/pics/2026-dhmc-ieeg-neuroconv-formats.svg @@ -0,0 +1,6405 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/pics/2026-dhmc-ieeg-nwb-convert-scale.svg b/pics/2026-dhmc-ieeg-nwb-convert-scale.svg new file mode 100644 index 0000000..11bcdd7 --- /dev/null +++ b/pics/2026-dhmc-ieeg-nwb-convert-scale.svg @@ -0,0 +1,11124 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/pics/2026-dhmc-ieeg-nwb-one-for-all.svg b/pics/2026-dhmc-ieeg-nwb-one-for-all.svg new file mode 100644 index 0000000..1b3b55f --- /dev/null +++ b/pics/2026-dhmc-ieeg-nwb-one-for-all.svg @@ -0,0 +1,20462 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/pics/dandi-compute-figure.svg b/pics/dandi-compute-figure.svg new file mode 100755 index 0000000..231a2fc --- /dev/null +++ b/pics/dandi-compute-figure.svg @@ -0,0 +1,248 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Reproducible spike sorting on archived data +DANDI Compute takes an asset already in the archive, runs it through an established spike-sorting pipeline +on institutional compute, and returns the results as a new, versioned derivative Dandiset. +dandi-compute/code +aind-ephys-pipeline +dandisets/001697 +1 +INPUT +DANDI Archive +Hundreds of TB of raw extracellular +electrophysiology available to the public. +Neuropixels sessions +NWB assets +2 +ORCHESTRATION +DANDI Compute +Decides what job to run on selected assets, using +which parameters and other configurations. +Prepare +Queue +Submit +3 +COMPUTATION +Spike-sorting pipeline +The containerized AIND Ephys pipeline runs on +HPC at MIT. +Nextflow +SpikeInterface +4 +OUTPUT +Dandiset 001697 +Sorted units, metrics and QC figures are saved +under a self-describing path. +Derivative +Versioned +2 +Inside DANDI Compute +Four responsibilities, each deliberately small. The infrastructure never touches the science — it decides what to run, and makes sure the record of that decision survives. +1 +Prepare +Point at an asset. Pick a named set of pipeline +parameters and a compute profile, and write +out everything needed to run it. +Named parameter sets +Compute profiles +2 +Queue +Prepared jobs pile up on disk, one self- +contained bundle per asset. Nothing has run +yet, so anything can still be pruned or rebuilt. +One bundle per asset +Inspect before you spend +3 +Submit +The bundle is handed to an institutional cluster, +where the pipeline claims a different slice of +hardware for each of its steps. +MIT Engaging HPC +Per-step resources +4 +Publish +Results, logs and the exact code are packaged +as a versioned derivative and pushed back out +where anyone can fetch them. +Version-controlled +DANDI + GitHub +Because the parameter set and the compute profile are both named, referenced things rather than typed-in flags, two runs that claim to be the same run really are (and two runs that differ say so in their identity). +3 +Inside the spike-sorting pipeline +A dispatch step fans the session out into one independent job per probe, shank and segment; the heavy work happens in parallel; the results are gathered back together at the end. +STEP 1 +Job dispatch +Splits work into jobs. +STEPS 2–6 · RUN IN PARALLEL +one job per probe · shank · segment +2 +Preprocess +Filter and correct drift. +3 +GPU +Spike sort +Detect and assign spikes. +4 +Postprocess +Waveforms and metrics. +5 +Curate +Label units by quality. +6 +Visualize +QC plots and views. +STEP 7 +Collect results +STEP 8 +Package as NWB +Parallelizing steps 2–6 makes the run affordable: only the sorting step needs a GPU, and only for one probe. Pipeline after +. +Buccino et al., 2026 +4 +Inside Dandiset 001697 +The output is a path you can read. Every level of the directory answers one question about where a result came from — nothing about the run is implicit, and nothing has to be looked up elsewhere. +001697/ +the derivative meta-Dandiset +└ +derivatives/ +└ +dandiset-<source>/ +WHICH DATA +└ +sub-<subject>/ +└ +ses-<session>/ +└ +pipeline-<id>/ +WHICH PIPELINE +└ +version-<id>/ +└ +params-<hash>_config-<hash>_attempt-<n>/ +WHICH RUN — SETTINGS, MACHINE, ATTEMPT +Re-run with different parameters and you get a sibling directory, not an overwrite — the old result stays where it was. +Inside every run directory +code/ +The exact code that produced this, not just a reference to it. +logs/ +What the run actually did, step by step. +output/ +Sorted units, waveforms and quality metrics. +visualizations/ +QC figures, viewable straight from the browser. +dataset_ +description +Machine-readable provenance for the whole run. +Each run directory stands on its own — enough to check the result and reproduce it. + The derivative sits alongside the data it came from, so the result and its inputs stay connected. +Published back to the archive. +Data in the archive +DANDI Compute infrastructure +Pipeline it relies on +The derivative it produces + + diff --git a/pics/neurosift-event-related-lfp-40hz-20261007.png b/pics/neurosift-event-related-lfp-40hz-20261007.png new file mode 100755 index 0000000..774c66e Binary files /dev/null and b/pics/neurosift-event-related-lfp-40hz-20261007.png differ diff --git a/pics/neurosift-event-related-lfp-reward-20261007.png b/pics/neurosift-event-related-lfp-reward-20261007.png new file mode 100755 index 0000000..81bb12b Binary files /dev/null and b/pics/neurosift-event-related-lfp-reward-20261007.png differ diff --git a/pics/neurosift-event-related-motion-energy-20261007.png b/pics/neurosift-event-related-motion-energy-20261007.png new file mode 100755 index 0000000..1fee4cd Binary files /dev/null and b/pics/neurosift-event-related-motion-energy-20261007.png differ