Evoked hypotheses, NBS edge-mask fix, and the 2026-09 audit remediation - #2
Merged
Merged
Conversation
…facet
roi_evoked and electrode_evoked were the last two modules with no path to the
declarative hypothesis layer -- HYPOTHESIS.md section 9 listed both as deferred for
"long-format DV". What ran instead was the R script's whole-brain
lmer(value ~ group * roi + (1|subject)) omnibus, which spends power across every
parcel when an auditory hypothesis names two, and cannot express a declared
contrast, omnibus or equivalence at all.
The deferral was real. Every other emmeans-tabular module exports one column per
DV and hands write_module_hypotheses a dv_cols vector; the evoked modules export
one `value` column with measure_name saying what it holds. One DV, twenty-odd
measures inside it.
The measure is a FACET, not a DV. The Python tabular adapter already carries
facet_cols, which runs an independent FDR family per facet across the spatial
grid -- and facet is exactly what dv_col is on the R side, where each DV gets its
own run_hypothesis() call and so its own family. That family boundary is the only
defensible one here: ITC is 0-1, ERSP is dB, ERP amplitude is signal units and
latency is seconds, so a family spanning them would pool quantities with no
common null. There is also no band axis -- each measure definition fixes its own
band and time window -- so the band coordinate is null rather than dropped.
Wired in analyses/_evoked_hypotheses.py, shared by both modules so they cannot
drift, the way _network_base is shared by the graph/NBS pair. roi_evoked scores
on roi, electrode_evoked on channel. Additive: the descriptive LMM in the R
scripts is untouched and still runs.
X49, found by the wiring and fixed here. Two different FDR families carried
byte-identical fdr_family labels. The label's `dv` coordinate read
facet_map.get("dv", "NA"), so it only picked up a facet literally named `dv` and
every real facet resolved to NA; the member hash could not separate them either,
because the members are the band x spatial cells and those are the same cells for
every facet. roi_network's 8 connectivity x 3 graph metrics produced 24 families
all labelled key=all|<hyp>|NA members=cell[N] hash=<same>. The label exists so
q-values from different families are provably non-comparable (REPORT_PLAN 10b)
and was asserting the opposite. q-values were never wrong -- BH is applied per
facet either way -- so this is label-only and no published number moves.
tests/test_evoked_hypotheses.py, 12 tests on planted synthetic signal: the
planted measure x ROI cell is recovered and neither a null ROI nor the null
measure is, the band coordinate is null, each measure is its own BH family, the
CSV fallback drives --steps statistics alone, a renamed design factor resolves,
and the X49 regression fails on the old code. 175 pass.
No study config is populated -- this is the mechanism only. The auditory
hypotheses themselves are Alex's to specify.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tten (X50) nbs_permutation_test allocated sig_edges under the comment "Build significant edge mask" and then returned it untouched, so significant_edges came back all-False for every call -- including a 99-edge component at p = 0.0, alongside a correct component_sizes=[99] and n_significant_components=1. The dataclass was self-contradictory as returned. Dormant in the shipped pipeline: nothing reads the field. _network_base uses component_nodes and component_sizes, and hypothesis/edge.py re-derives components itself, so no published number is affected. But it is a public field of a public return type, and any new consumer silently gets "no edges" -- which is what happened when surface-evoked's Phase 7 work read it to ask how many edges NBS declares significant and got 0 for every detection. The mask is now filled from each significant component's node set intersected with the suprathreshold graph. That is exactly a component's edge set: components are connected components of the suprathreshold graph, so they partition the nodes and every suprathreshold edge with both endpoints in one component's node set belongs to it. test_nbs_significant_edges_mask_is_populated checks the mask is non-empty for a significant component, contains no sub-threshold edges, and totals the significant components' sizes. It fails on the old code. 176 pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…code Every item in the 2026-09-04 repo audit was verified against the code and then fixed; the README claims it flagged as false are corrected too. Behaviour changes - Vertex `absolute` band power is a dB/Hz density like the ROI path (per-band constant shift; within-band statistics unchanged). - Vertex modules honour top-level and per-analysis `epoch_sampling` (global -> vertex block -> analysis block); vertex_cluster epoch-samples; `n_bootstrap: 0` means full timeseries on the vertex sampler too. - `--jobs`: explicit CLI value wins, omitted -> YAML `jobs:`, 0/-1 auto. - `--force` actually removes previous output (published dirs always, data/ when the process step runs); output paths use the canonical name. - vertex_spatial retired in Python (no loading, no R; empty tables + note). - roi_connectivity reads a `metrics:` list; R adapts to the metric columns present instead of requiring coherence. - roi_directed hypothesis tables renamed roi_directed_* and cover DTF; DTF-only runs no longer skip R. - fcd_comparison finds its two primaries across paradigm dirs (sensor_dir / source_dir overrides); metadata carries supplements/requires. Fixed - Evoked R scripts derive contrasts from design:/hypotheses: (tables were empty under modern configs); Python forwards --hypothesis. - vertex_evoked joins the hypothesis contract (SELECTABLE, perm adapter, vertex_evoked_hypotheses.csv, `list` heading). - AAC/PPC get an R hypothesis path (roi_cross_freq_edges_analysis.R). - aggregate_to_regions keeps delta_ref. - Package imports without mne (lazy TFR import, clear ImportError); matplotlib is a core dependency; R scripts ship in wheels/sdists and find_r_script_dir() looks there (plus SOURCE_ANALYTICS_R_DIR). - `init` writes a parseable design/hypotheses/paradigms config for both discovery layouts; `--output -` streams YAML to stdout. - Timeout log messages report the real timeout; aperiodic `about` text says 12-45 Hz; PSD `about` describes dB/Hz density. - Hard `source()` in the R modules (no silent tryCatch); nlme dropped. - Removed orphaned R/network_analysis.R, the dead *_network R lookups and run_connectivity_network.sh; added scripts/run_study.sh. Docs: README synced (init, figures off by default, real output trees, extras, R packages, catalog incl. fcd_comparison/electrode_signature, --jobs/--profile/--paradigm, no signed-STC fallback, no dB column); CLAUDE.md rewritten; CHANGELOG through 0.6.0 + this pass. Tests: 197 passed (new tests/test_audit_fixes.py, tests/test_r_scripts_smoke.py). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UukEcpPCEWnLHJHdT2fTLd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three commits on top of
main:f601c5c).roi_evoked/electrode_evokedrun thedesign:/hypotheses:spec through the tabular adapter with the measure as the FDR facet (sharedanalyses/_evoked_hypotheses.py).5176462). It was allocated and never written.faa6610). A repo audit (2026-09-04) compared README / CLAUDE.md against the code and found 24 defects plus a dozen false README claims. Every item was verified against the code before fixing; all are fixed here and the docs are synced.Behaviour changes (read before re-running a study)
absoluteband power is a dB/Hz density,10*log10(integral / bandwidth), matching the ROI/electrode path. Within-band statistics are unchanged (per-band constant shift); raw values and plots move by10*log10(bandwidth).epoch_sampling(global →vertex:block → analysis block).vertex_clusternow epoch-samples when enabled.n_bootstrap: 0= full timeseries on the vertex sampler too.--jobs: explicit CLI value wins (including1); omitted → YAMLjobs:;0/-1actually auto-parallelize now.--forcereally removes previous output (publishedtables/+figures/always; workingdata/when theprocessstep runs). Deprecated analysis names print/check the canonical output dir.vertex_spatialis retired in Python too: loads nothing, calls no R, writes empty tables + a note.roi_connectivityreads ametrics:list; the R report adapts to the metric columns present instead of requiring coherence.roi_directedhypothesis tables renamed toroi_directed_{global,directed_edges,region}_hypotheses.csvwith advcolumn coveringte/net_te/dtf; DTF-only runs no longer skip R.fcd_comparisonfinds its two primaries across paradigm dirs (resting+vertex);sensor_dir/source_diroverrides accepted. Metadata carriessupplements+requires.Fixed
config$contrasts(NULL under modern configs) so their tables came out empty; they now derive contrasts from the design spec and accept--hypothesis.vertex_evokedjoins the hypothesis contract (--hypothesis,vertex_evoked_hypotheses.csv,listheading).roi_cross_freqget an R hypothesis path (R/roi_cross_freq_edges_analysis.R, three tiers).aggregate_to_regions()keepsdelta_ref.mne(lazy TFR import with a clear ImportError);matplotlibis a core dependency; extras documented; lockfile refreshed.share/source-analytics/R);find_r_script_dir()checks there and honoursSOURCE_ANALYTICS_R_DIR.initwrites a parseabledesign:/hypotheses:/paradigms:config for both discovery layouts;--output -streams YAML to stdout.abouttext says 12–45 Hz; PSDaboutdescribes dB/Hz density.source()in the R modules (no silenttryCatch);nlmedropped from the retired script.R/network_analysis.R, the dead*_networkR lookups, andrun_connectivity_network.sh; addedscripts/run_study.sh.Docs
README synced to the code (
init, figures off by default, real analytics/results trees with paradigm + profile segments, install extras, R package list, catalog incl.fcd_comparison/electrode_signature/vertex_signature,--jobs/--profile/--paradigmsemantics, no signed-STC fallback, nodBcolumn). CLAUDE.md rewritten (it described the original PSD-only package). CHANGELOG filled through v0.6.0 plus this pass.Test plan
pytest: 197 passed (newtests/test_audit_fixes.py,tests/test_r_scripts_smoke.pywhich runs the real R scripts on synthetic CSVs)R/*.Rparses;Rscript tests/test_hypothesis.Rpassesuv build: wheel contains the 20 R scripts undershare/source-analytics/Rmneblockedscripts/run_study.sh) and diff vertexabsolutevalues against the expected per-band constant shift🤖 Generated with Claude Code
https://claude.ai/code/session_01UukEcpPCEWnLHJHdT2fTLd