Repository navigation
Kilosort import pipeline, interactive spike-sorter GUI, and the .vsw / KlustaKwik file formats #97
Description
Part of #90 (PR #61 catch-up). This is the largest single cluster of missing content and the one most worth scoping before starting.
What is missing
None of the following exist on main:
Kilosort import pipeline — src/ndi/fun/probe/import_/kilosort/: getInfo, labels, meanwaveform, probe, removeold, session, waveformdata, plus fun/probe/extracellularInfo.py and fun/probe/plotProbeGeometry.py. Ported from MATLAB +fun/+probe/+import/+kilosort/*. Depends on the apps/kilosort/kilosort_clusters document/schema pair (see the ndi_common issue).
Spike-sorter GUI — src/ndi/app/spikesorter_gui.py and src/ndi/app/spikesorter_clustermodel.py: an interactive PyQt sorter (graphical_mode=1) with lasso selection, persistent zoom, and the automatic clustering path (spike_sort + clusters2neurons). Built across PR #61 commits 27ed525, bdee33c, 5cf3ef4, c13ddd4, af0316c, 772ad65.
File formats — src/ndi/util/vhlspikewaveformfile.py (the real .vsw format, so spike storage is MATLAB-readable) and src/ndi/util/klustakwik.py. Note main already handles VHSB (tests/symmetry/element/vhsb_cases.py and the round-trip battery), so PR #61's src/ndi/util/vhsb.py is not wanted — only .vsw and KlustaKwik are.
What it would take
This is the cluster where "port the branch" is least applicable and a fresh implementation against current main is most warranted:
- The GUI is ~5 commits of iteration on top of an initial implementation, and carries a PyQt dependency the project does not otherwise take. Whether NDI-python should ship an interactive GUI at all is a product decision worth making before any porting —
mainhas meanwhile grown its own GUI direction (38995b3, the navigator pane stack, tracked in Port gaps vs NDI-matlab: ndi.gui.navigator and its pane stack, ndi.fun.ensemble, and five smaller items #78). These two should not proceed independently. - The Kilosort pipeline is the more clearly valuable half and has fewer entanglements: it is mostly self-contained under
fun/probe/, and its main external dependency is thekilosort_clustersdocument pair. .vswis a prerequisite for MATLAB-compatible spike storage regardless of what happens to the GUI, so it can land first and independently.
Suggested sequencing: .vsw → Kilosort pipeline → GUI decision.
Also deferred by PR #61 and still open: the Kilosort mean-waveform rework (S7) from MATLAB, which is a change to the pipeline rather than the initial port.
Validation
PR #61 validated the importer against a real Kilosort2.5/Phy export (commit cc37b71, tests/test_pr12_kilosort_real.py) rather than against fixtures alone. That is the right bar — a synthetic-fixture-only port of this pipeline will not tell you whether it works.
Activity
Merge pull request #196 from Waltham-Data-Science/claude/ndi-python-9…
stevevanhooser commentedon Sep 4, 2026
Part 4 status (Kilosort mean-waveform S7 rework)
The rework itself (VH-Lab/NDI-matlab#867) is already fully ported to this repo — no code work needed for part 4 as filed:
src/ndi/fun/probe/import_/kilosort/recalculate_mean_waveforms.py— the streaming pass, param-for-param with MATLAB (maxChunkBytes,chunkSamples,progressfcn,epochBounds,highpass/hp_cutoff/hp_order/hp_ripple,maxSpikes,verbose).recalculate_mean_waveform.py(single-cluster form kept as the readable reference).read_spikeglx_meta.pyandprompt_raw_binary.py(SpikeGLXnSavedChans,expectedSamplesstride validation).probe.pywires up the streaming call andexpectedSampleson the raw-file prompt.src/ndi/fun/probe/ndi_matlab_python_bridge.yamlalready documented all of the above in itsdecision_log.tests/test_fun_probe_import_kilosort.pycover it.What did need a small PR
Verifying part 4 turned up a real gap in the follow-on refactor VH-Lab/NDI-matlab@1b99d29 ("Use platform-independent folder names"): the Python
probe.pywas storingprobe_obj.elementstring()(e.g."ctx | 1") inkilosort_clusters.kilosort_directory, where MATLAB stores the sanitized folder name (e.g."ctx_-_1"). A MATLAB reader on a Python-written session would not find the folder. PR #197 fixes that (probe.py + session.py + bridge YAML sync-hash bump to1b99d29+ one focused regression test).#97 status
.vswfile format): done, merged in #196.main, covered by 77 tests.gui.app/gui.componentfoundation now that Port gaps vs NDI-matlab: ndi.gui.navigator and its pane stack, ndi.fun.ensemble, and five smaller items #78/Navigator port: close audit gaps for issue #78 task c #181 settled the PyQt toolkit); not in this PR.kilosort_directoryfield fix is in #197.Once #197 merges and item 3 lands separately, #97 can close.
Generated by Claude Code
Merge pull request #197 from Waltham-Data-Science/claude/ndi-python-9…
Merge pull request #198 from Waltham-Data-Science/claude/ndi-python-9…
claude commentedon Sep 4, 2026
All four items of this issue have landed on
main:.vswfile format — Port the VHL spike waveform (.vsw) file format (#97 items 1 & 2) #196 ("Port the VHL spike waveform (.vsw) file format").vhNDISpikeSorterandspikeSorterImporterhad landed earlier (commits 34f4b1f and 29d74bd) as part of the deferred Port gaps vs NDI-matlab: ndi.gui.navigator and its pane stack, ndi.fun.ensemble, and five smaller items #78ndi.gui.apptranche; Port ndi.gui.app.kiasort and pipelineEditor (closes item 3 of #97) #198 added the remainingkiasortandpipelineEditor, finishing that tranche at 11/11.VH-Lab/NDI-matlab@1b99d29(platform-safe folder names inkilosort_clusters.kilosort_directory), fixed in Kilosort import: use platform-safe folder name in the doc field (#97 part 4 follow-up) #197.Product decision the issue asked for ("whether NDI-python should ship an interactive GUI at all") was made implicitly when the deferred #78 tranche started landing app-by-app; #198 completes it. Closing.
🤖 Generated with Claude Code
https://claude.ai/code/session_013ZPKo8X5yuJcHp5UZHF8kU
Generated by Claude Code