Repository navigation
open_doc reports an out-of-bounds series slot as ManifestUnreadable, blaming a manifest that is fine #80
Description
Found while building the missing cross-language conformance vectors for series_member_of (the +makeArtifacts/+common/seriesMemberNames artifact DID-matlab produces and DID-python does not yet read).
Series slots are one-based, so slot 0 is out of bounds in the same way slot 99 is past the end. Python reports the two completely differently, and the report for slot 0 is wrong.
What happens
Against a live sparse series (demoSeries, members present at slots 2, 5 and 6):
| asked for | exist_doc |
open_doc |
|---|---|---|
chunkdata.bin_-1 |
False |
FileNotFoundError — not a file of this document |
chunkdata.bin_0 |
False |
DID:SQLITEDB:FileSeries:ManifestUnreadable |
chunkdata.bin_1 |
False |
DID:SQLITEDB:FileSeries:NoSuchMember (a gap — correct) |
chunkdata.bin_99 |
False |
DID:SQLITEDB:FileSeries:NoSuchMember (past the end — correct) |
The message for slot 0 reads:
Cannot read the manifest for series "chunkdata.bin" of document "<id>":
slot must be a positive integer (one-based)
The manifest is not damaged. It is read perfectly well a line later for slot 1. The caller asked for a slot that cannot exist, and was told their stored data is corrupt.
Why
_open_series_member folds the bounds check into the same except arm as a genuinely damaged file:
try:
member_uid, _count = read_series_manifest_uid(manifest_path, index)
except (OSError, SeriesManifestError, ValueError) as error:
raise FileAccessError(
"DID:SQLITEDB:FileSeries:ManifestUnreadable", ...
) from errorand read_series_manifest_uid opens with
if not isinstance(slot, int) or slot < 1:
raise ValueError("slot must be a positive integer (one-based)")So a caller-side range error arrives as evidence of a store-side corruption.
Why it matters
ManifestUnreadable and NoSuchMember are the two answers a caller branches on, and they point at opposite causes. NoSuchMember means ask for something else; ManifestUnreadable means your data is damaged, go and investigate the store. Sending someone after a corrupt manifest that does not exist is the expensive direction to be wrong in — and the guard added for DID-python#73 was about exactly this class of thing: an error that misdescribes what happened costs more than no error.
The other two callers already get it right, which is what makes this look like an oversight rather than a decision:
database._series_member_pathguardsindex < 1up front and returns a clean(False, None)— that is whyexist_docanswers honestly.database.series_hasswallows theValueErrorand returnsFalse, which is the right answer for "does the series record a member here".
MATLAB gets this right
did.implementations.sqlitedb/seriesMemberPath guards before touching the manifest:
if ~isscalar(index) || index < 1 || index ~= round(index), return, endIt returns a clean miss, and do_open_doc reports the member miss in its own words. So this is also a parity gap, not only an internal inconsistency.
Suggested fix
Bound-check in _open_series_member before the read, and report the miss as what it is:
if index < 1:
raise FileAccessError(
"DID:SQLITEDB:FileSeries:NoSuchMember",
f'No such member "{filename}" in series "{stem}" of document '
f'"{doc.id()}": member indices are one-based.',
)ManifestUnreadable then means only what it says. read_series_manifest_uid's own ValueError stays as it is — it is the right contract for a public function called directly.
Related
Turned up while resolving how the two languages should treat chunkdata.bin_-1, which MATLAB parses as index -1 and rejects downstream while Python's isdigit() declines to parse at all. That difference is cosmetic once this is fixed — both are out of bounds, both refused, and neither claims a corrupt manifest. It is recorded as a deviation rather than changed, since widening what counts as a member name to admit a negative index buys nothing.
Activity
stevevanhooser commentedon Sep 8, 2026
Fixed on
main(981cebc), in7ee31dc/ #81 — the suggested fix, as written here._open_series_membernow bound-checks before the read (sqlitedb.py:1708) and reportsDID:SQLITEDB:FileSeries:NoSuchMemberwith "member indices are one-based", so slot 0 is answered the same way slot 99 already was.ManifestUnreadableis left meaning only what it says, andread_series_manifest_uid's ownValueErroris unchanged — the right contract for a public function called directly. MATLAB'sseriesMemberPathguards the same way before touching the manifest, so the parity gap closes with the internal inconsistency.The
chunkdata.bin_-1question in the Related section is settled too, in #82: theseriesMemberNamesconformance reader records it as a checked deviation — MATLAB'sstr2numparses-1and rejects the index downstream, Python'sisdigit()declines and the name is not a member — rather than widening what counts as a member name.Verified against the code: removing the
index < 1guard reddenstest_slot_zero_is_no_such_member_not_a_damaged_manifest. Full suite green at 693 passed, 10 skipped, 1 xfailed.Closing as completed.
Generated by Claude Code