Resolve companion ranges across filename case differences - #19
Draft
imnasnainaec wants to merge 16 commits into
Draft
Resolve companion ranges across filename case differences#19imnasnainaec wants to merge 16 commits into
imnasnainaec wants to merge 16 commits into
Conversation
imnasnainaec
force-pushed
the
ranges-companion-filename-case
branch
3 times, most recently
from
August 18, 2026 20:23
cd9cb67 to
a565d67
Compare
imnasnainaec
marked this pull request as ready for review
August 20, 2026 12:41
imnasnainaec
marked this pull request as draft
August 25, 2026 16:22
A LIFT folder written on Windows can spell its pair inconsistently — Dict.LIFT beside Dict.lift-ranges, or the reverse — and load fine there, because the filesystem folds case. On Linux the sibling candidate is built from the .lift's own suffix, so it missed, the companion was skipped without a word, and every range it defined went absent. Candidates that match no file exactly now fall back to one whose name differs only in case. The fallback is reached only after an exact miss, so a case-folding filesystem never enters it and behaves as before; a case-sensitive one gets one directory read per folder, cached across the candidate list. Where several names fold together the lexicographically first wins. The choice is arbitrary but fixed, which matters more than which file it picks: directory order varies between filesystems and runs, and a companion that loads differently on consecutive reads would be worse than one that never loads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
0.1.0 has not shipped, so there is no released behavior for an Unreleased entry to be fixing — the tolerance is simply part of what companion discovery does in the first release. Fold it into that bullet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The directory listing the fallback builds called its files "entries", the word this module uses for a LIFT <entry> everywhere else — the same collision that keeps byte regions from being called spans. Name them files. Spell the surrounding prose the way the rest of the package does: a fallback that runs rather than fires, a name that matched no file rather than missed, a helper named for the filesystem it probes rather than abbreviating it, and fixture names deliberately not taken from the corpus file. Unpack the two densest clauses so each reads in one pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Companion lookup folds names with casefold() over NFC rather than lower(), so a Turkish-cased or NFD-spelled name resolves the way it does on the filesystem that wrote it — FLEx mixes normalization forms within one export. Ties break in code point order on every platform; sorting Path objects left the choice to directory order on Windows, where PurePath ordering is itself case-folded. An unstattable exact spelling now falls through to the folded lookup instead of giving up. A candidate that folds onto the .lift itself is skipped: RangesFile.load rejects a <lift> root, so a header href naming the lexicon in another case took the whole load down. One that folds onto a companion already tracked is skipped too — Path.resolve() leaves case alone on macOS, so a single file reached under two spellings was loaded and tracked twice, and written twice by save(). The sibling candidate is built with with_name, which agrees with with_suffix on every name that has an extension and does not raise on a name without one. Nothing upstream requires the .lift extension: parse_document never inspects it. dangling-ranges-href decides existence with that same lookup, so a companion spelled in another case is no longer reported missing on a case-sensitive filesystem. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 0.1.0 entry said companion names fold on case alone; they fold on Unicode normalization form as well. The folder guide listed the candidates tried but never mentioned the folding at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three helpers each stated a rule, defended it, then disclaimed it. What is left is the reasoning the code cannot show: casefold over lower, NFC, why only the final component folds, why code point order, and what the fold pre-check protects the inode comparison from. The folder guide drops the folding sentence outright — it describes behavior no reader acts on, in a paragraph otherwise about which candidate wins. The 0.1.0 entry keeps the fact and loses the justification, which now lives only in the docstrings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tion A candidate that exists as a directory now stops the lookup instead of falling through to the folded listing. An href of "" normalizes to the LIFT folder itself and one of "sub/" to a subfolder, and folding a folder's own name searches its *parent*: any file there spelled like the folder was returned as the companion, and RangesFile.load then rejected its root and failed the whole load. dangling-ranges-href also treats a match that is the .lift itself as no match. _resolve_ranges refuses to take the lexicon for its own companion, so a header href folding onto it resolves to nothing that supplies the range — the reference is as dangling as a missing file, and went unreported on a case-sensitive filesystem. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_same_file compared the spellings it was handed, so a caller had to canonicalize first to get a true answer. _resolve_ranges did; the dangling-ranges-href check did not, and an href reaching the .lift through a ".." segment or a symlink read there as some other file. The loader skipped that candidate as self-referential while validation counted it as a companion that exists, leaving the header range both unsupplied and unreported. Resolving inside _same_file makes the answer independent of how the caller spelled its arguments, and retires the loader's own pre-resolution — the duplicate that let the two drift apart. A path that will not resolve now compares false instead of falling back to the spelling as given; it would fail the samefile stat on the next line regardless. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_fold cited FLEx's mixed NFC/NFD content as its reason to normalize filenames. That is a different code path and no evidence for the names; decomposed filenames arrive from macOS, so say that instead. _existing_file described itself against the behavior it replaced, which reads oddly once nothing remembers that behavior. Lexicon.load's candidate list and its matching rules split into separate paragraphs, and "every one that exists is loaded" becomes "every distinct file among them" — candidates resolving to the lexicon, to a directory, or to a file already tracked all exist and are deliberately skipped. The dangling-ranges-href comment leads with what the check catches rather than closing with it, and loses a restatement of _existing_file's own docstring. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Docstrings and comments only, plus one rename. _existing_file claimed a case-folding filesystem never reaches the fallback. That is false on NTFS, which folds case but not normalization: an NFD companion misses the exact stat and only the fallback finds it. The guarantee that does hold everywhere — an exact hit is returned unchanged — leads instead. Its summary named the argument rather than the return value, and its motivation read as though LIFT folders can only be written on Windows, when what matters is whether the authoring filesystem folds case, as macOS also does. The with_name comment justified a choice against with_suffix rather than warning about it. Naming the hazard is what stops someone reaching for the tidier call and reintroducing the raise it avoids. _case_sensitive_filesystem returned a bool under a noun phrase that promises a filesystem; _same_file and _same_dir are the house pattern for a predicate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"Stable" names a specific property in sorting — preserving the relative order of equal elements — which is close enough to what is meant here to be read as a claim about the sort rather than about the outcome. The choice among fold-equal names is deterministic: same folder, same winner, on every platform and every run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Where the exact name missed and more than one file folded onto it, the lookup took the first in code point order. That puts the conventionally spelled Dict.lift-ranges last, behind any oddly cased twin beside it, and picks between files whose contents can differ — a stale export, or a rename git recorded as an add — with nothing said about it. Which one the name meant is not recoverable, so none of them is loaded now. Validation reports the collision as ambiguous-ranges-file, naming every spelling by code point: two can differ only by normalization and render identically. It walks the same candidates load does, so it covers the sibling name as well, which no header href reports on and which is what a folder authored on Windows most often collides over. One finding per colliding group, however many candidates fold onto it; a dangling href beside it says which range went unmet. A name that matches a file exactly still resolves to it. Folding runs only after that miss, so an odd-cased variant sitting beside an exactly named companion is no collision at all. The candidate list moves out of _resolve_ranges into _ranges_candidates for validation to walk, and now yields each path once: an href repeated across ranges, or agreeing with the sibling, was looked up as many times as it was written. A case-only collision cannot be created where the filesystem folds case, and there the exact-name stat folds too — so the test that runs everywhere separates its two companions by which of two accents is decomposed, the one difference NTFS and APFS keep. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
imnasnainaec
force-pushed
the
ranges-companion-filename-case
branch
from
August 25, 2026 17:25
0f6fdce to
86c9013
Compare
NFC is a normalization form, not a dimension along which two filenames differ: names collide because folding them applies NFC, and neither of the two need be in that form — a stem with two accents has four spellings that fold onto one key. So the problem code's description says the files answer to one name under case folding and NFC, rather than that they differ by it. Two spellings differ *in* case or normalization, not *by* it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The problem-code reference told the reader how folding works and why the collision is unresolvable. Both belong where the lookup lives; what a reader needs here is that the ranges go absent and that leaving one file brings them back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The comments restated the lookup helpers' docstrings and described the shape of the two loops below them. What is left is the part the code does not show: that the check walks every candidate because a collision on the sibling name has no href to report it, and why the spellings are named by code point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A candidate folding onto several files warned even when another candidate had named one of them exactly and loaded it — so the finding claimed that none of them is loaded while one was, and pointed a reader at renaming the very file supplying their ranges. Reachable whenever the sibling name resolves and a header href is spelled a third way: the sibling supplies the range, which also keeps dangling-ranges-href quiet, so the false finding was the only thing said about a folder that had lost nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A LIFT folder written on Windows can spell its pair inconsistently —
Dict.LIFTbesideDict.lift-ranges, or the reverse — and load fine there, because the filesystem folds case. On a case-sensitive filesystem the sibling candidate is built from the.lift's own name, so it missed: the companion was skipped silently, and every range it defined went absent.Zip packages inherit the fix:
load_zipextracts and callsLexicon.load, so_zip.pyneeds no change.Matching a companion filename
A candidate that matches no file exactly falls back to one whose name folds onto it (
_folded_matches).casefold()over NFC, notlower(). Decomposed filenames arrive from macOS, which stores and zips them that way, and neither NTFS nor ext4 folds the two forms.Three things the fallback must not resolve to
Each is reachable once a miss can match something other than the exact name:
.liftitself. A header href spellingDict.liftbeside aDict.LIFTfolds onto the lexicon, andRangesFile.loadrejects a<lift>root — failing the whole load rather than skipping one companion.""normalizes to the LIFT folder itself. Folding a folder's own name searches its parent, so any file up there spelled like the folder came back as the companion. A candidate that exists as a directory stops the lookup.Path.resolve()leaves case alone on macOS, so one file reached under two spellings was loaded twice and written twice bysave().An ambiguous name resolves to nothing
Where several files in the folder fold onto one candidate, which of them the name meant is not recoverable — the filesystem that wrote them could not have told them apart. None is loaded, and validation reports the collision as a new
ambiguous-ranges-filewarning:--strictpromotes the warning to an error, so a CI run can refuse such a folder outright.Validation and loading agree on what exists
dangling-ranges-hrefdecides existence with the same lookup and the same refusals, so a companion spelled in another case is no longer reported missing on a case-sensitive filesystem, and an href resolving to the.liftstill warns. Where a collision blocks the lookup both findings appear: one says why nothing resolved, the other which range went unmet._same_fileresolves its own arguments rather than trusting the caller's spelling, so the two sides cannot disagree about a path spelled two ways.The candidate list moved out of
_resolve_rangesinto_ranges_candidatesfor validation to walk, and yields each path once — an href repeated across ranges, or agreeing with the sibling, was looked up as many times as it was written. Building the sibling candidate withwith_namerather thanwith_suffixalso keeps a.liftloaded under a name with no extension from raising, whichparse_documenthas always accepted.Tests
Twelve in
test_ranges_folder.py. Two spellings of one name cannot coexist where the filesystem folds them together, so:python scripts/check.pygreen: 581 passed, 3 skipped, 97.55% coverage.Media hrefs have the same case problem and are deliberately untouched here — #34 carries the design question that makes them different from companions.
🤖 Generated with Claude Code
Devin review: https://app.devin.ai/review/sillsdev/python-sil-lift/pull/19
This change is