You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Four correctness bugs still live on main: query De Morgan, ngrid column-major order, fail-open validation, unparseable schemas #93
Part of #90 (PR #61 catch-up). Each of these was fixed on catchup/ndi-python-2026-08 and each is still reproducible on main today. They are independent — four small patches, four regression tests.
1. ~(A & B) evaluates as (~A) & (~B), not (~A) | (~B)
src/ndi/query.py:312-332. __and__ builds a flat conjunction that DID evaluates with all(...); __invert__ negates each search_structure entry in place while preserving _composite_op = "and". So De Morgan is violated.
Concretely: a user asking for "every element that is not the probe named e1" silently loses every other probe (type='probe', name='e2') from the result set. This returns wrong data with no error.
Fix: when self._composite and self._composite_op == "and", distribute the negation over the OR of the negated sub-queries. The OR-composite path is already correct. Reference: commit d8a66bf (10 lines in query.py, 40 lines of test).
src/ndi/fun/data.py:71 — raw.reshape(data_size) with no order="F". MATLAB's fread fills column-major and fwrite serialises column-major; ndarray.tofile always writes C order. Every multi-dimensional ngrid written by Python is byte-incompatible with MATLAB, and every one read from MATLAB is transposed.
Fix: raw.reshape(data_size, order="F") on the read side, arr.ravel(order="F").tofile(...) on the write side. Reference: commit e95cd51.
3. validate() returns is_valid=True when the schema fails to parse
src/ndi/validate.py:115-117 returns None from _load_schema on JSONDecodeError, and validate() at :360-363 treats schema is None as "no schema found — can't validate" and returns a default ValidationResult with is_valid = True. Validation fails open: a document whose schema is corrupt is reported as valid.
Note the asymmetry already in the file — a missing superclass schema at :376-380 correctly sets is_valid = False. Only the document's own schema fails open.
Fix: distinguish "no schema declared" from "schema declared but unloadable", and fail closed on the latter. Reference: commit 8e4660c.
4. Two bundled schemas still do not parse
Confirmed by walking src/ndi/ndi_common/schema_documents/**/*.json on main:
apps/calculations/simple_calc_schema.json Expecting value: line 21 column 19 (char 426)
apps/markgarbage/valid_interval_schema.json Expecting value: line 20 column 19 (char 412)
(data/image_schema.json, the third one PR #61 repaired, parses on main now.)
These two are exactly the documents that item 3 silently waves through. Fix both, then add a test that walks the whole schema tree and asserts every file parses — that is the tripwire that keeps a third one from appearing. PR #61 also carried a deliberate quarantine + tripwire for an unparseable upstream calc schema (fee0561); worth deciding whether these two are ours to fix or upstream's.
Sequencing
Do 4 before 3, or 3's new fail-closed behaviour will turn two silent passes into two loud failures in the same commit and obscure which change caused what.
Activity
Merge pull request #178 from Waltham-Data-Science/claude/ndi-python-i…