Add Paratext 9 interlinear test projects for import testing - #283
Open
imnasnainaec wants to merge 8 commits into
Open
Add Paratext 9 interlinear test projects for import testing#283imnasnainaec wants to merge 8 commits into
imnasnainaec wants to merge 8 commits into
Conversation
📝 WalkthroughWalkthroughAdded four Paratext 9 test projects with Scripture files, interlinear data, lexicons, settings, locale metadata, identifiers, and documentation for converter and manual import fixtures. ChangesParatext 9 test project fixtures
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: 🟡 Moderate · up to The PR adds Paratext fixtures for import testing, but one PIA fixture references a missing Spanish lexicon sense and the PIC project metadata conflicts with its PHP fixture filename, which can produce incomplete or misleading test results. Merge should wait until these fixture inconsistencies are corrected or explicitly accepted. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (7 skipped: 7 unsupported.) ✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Four minimal PT9 projects under test-data/pt9-projects/, for exercising the Paratext 9 interlinear import against a running platform - the part no unit test covers: the platform reading the files, the import service persisting the result, and the WebView presenting it. Each has one book and a handful of verses, so an import finishes immediately and its report reads whole. - PIA is the happy path: two gloss languages over one book, a phrase cluster, word parses, a word and parse cluster sharing one range, an approved verse hash, an excluded cluster, a punctuation entry, a repeated surface form for an ambiguous anchor, and every sense-resolution and bare-word-analysis outcome. - PIB carries all five clusterDrops reasons plus an interlinear file for a book the project has no text for. - PIC covers file identity: a canonical file beside a non-canonical twin, one file with no GlossLanguage and one with no BookId, a legacy language name, and two raw language values resolving onto one tag. - PID has a lexicon and word analyses but no interlinear book file, so the manifest is non-empty while the conversion reports no languages. - test-data/pt9-projects/README.md documents how to install and run them and tabulates what each covers. The root README's test-data line named a fixture removed in #272, so it now describes what the directory holds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PIA's text was two near-verbatim verses of Philippians followed by an unrelated English sentence - a blend that read as scripture without being it, and invited reading the glosses as a translation. Its three verses are now invented words throughout, and shorter: a form that repeats within a verse for the ambiguous anchor, a two-word run for the phrase cluster, and stem+suffix forms for the parses. Every feature the project covered it still covers - both gloss languages, the phrase, the paired word+parse range, the approved hash, the excluded cluster, the punctuation entry, all five sense-resolution outcomes, and all four bare-word-analysis outcomes. The README says the text is invented and why those shapes were chosen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Importing PIA looked like it produced no glosses. It had produced them - six records carried an es gloss - but the platform scans interlinear files in ordinal path order, that order becomes the project's analysisLanguages, and the WebView renders only analysisLanguages[0]. Interlinear_en sorted ahead of Interlinear_es, so the language on screen was the deliberately thin one: two glossed records, both in verse 1, and nothing at all in verses 2 and 3. PIA's second gloss language is now fr, so Interlinear_es sorts first and the es glosses are what the view shows. The thin file keeps its job - it still overlaps es on two tokens so their records merge, and still reads "daxes" as a single stem where es reads stem plus suffix, so the parses still conflict. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phrases were represented by one plain two-word cluster in PIA, so the branch that anchors them had only its happy path exercised. The rest of that branch now has fixtures too. PIA gains two verses rather than reworking the existing three, so nothing already covered shifts: - verse 4 repeats one two-word run twice and glosses it twice, so the first cluster is ambiguous between the two runs and the second, anchoring to the run the first left, is marked excluded; - verse 5 carries a three-word phrase, glossed by both languages, so two phrase records merge at one run. PIB gains the failures, beside a phrase that anchors as a control: one whose form matches no run of words, and one whose form is blank, which is the separate guard ahead of the run search. Both count as formMismatch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One `Hash` existed across all four projects, on a PIA verse holding no phrase. Since only an approved link renders - the phrase views read approved links alone, and a token falls back to the suggestion pool, which admits an analysis only once it holds an approved link - almost nothing these projects imported was visible in the app. PIA showed a single gloss, on `zav` in verse 2, and none of its four phrases; PIB's control phrase could be confirmed only from the report's count. - PIA 1:4 is approved, so its first `zim zam` lands visible and carries the low confidence its ambiguous anchor earned, beside the excluded twin that imports rejected. - A `zam zim` cluster overlaps that approved run on one token, which demotes it to candidate - the `approvedDemotedToCandidate` counter had no fixture. - PIA 1:5 is approved by `es` alone, so the phrase both languages gloss merges back down to suggested. That cross-language rule had no fixture either, and this one stays invisible on purpose. - PIA verse 3 gains `zav` and `flib` with no clusters of their own. Both forms carry an approved analysis in verse 2, so they exercise the suggestion pool, which nothing reached before: `zav` resolves a gloss and should show one, while `flib`'s dangling sense leaves its payload textless and filtered from the dropdown. - PIB's only verse is approved, so its control phrase lands somewhere a tester can see rather than only in a count. Verse text stays invented. Every cluster range still covers the word it names, checked against the edited verse text. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
imnasnainaec
force-pushed
the
pt9-import-test-projects
branch
from
September 2, 2026 18:30
721f9d9 to
409a6e6
Compare
No morpheme breakdown was linked to anything: every parse cluster sits in PIA 1:3, which carried no `Hash`, so all of them imported `suggested` and the catalog listed them as applied nowhere. The same held for most word shapes - of thirteen analyses, two were linked. Verse hashes now sit where each shape needs one: - es 1:1 approved, which covers a word cluster with a resolving sense, one whose sense dangles from an entry that exists (`gorp`), a second cluster on a repeated form so its anchor is a guess and it carries low confidence, and the `dax fep` phrase. - es 1:3 approved, which covers the stem-plus-suffix breakdowns and the word and parse clusters that share one range. - fr 1:1 approved too, so the records both languages contribute merge to approved rather than being reduced by the disagreement rule - one of them with a per-language sense, so the merged record resolves no single sense. - fr 1:4 is new: a one-morpheme parse on `kex`, which `es` leaves alone. A single-stem breakdown cannot be linked at a token that already carries a two-morpheme one, since the second approved record on a token demotes to candidate, so it needs a token of its own. Left deliberately unlinked, each the only instance of its case: the fr parse of `daxes` competing with the es parse on that token, `tomo` rejected for being excluded, and `nol pim tuk` in the verse only `es` approves. Every cluster range still covers the word it names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Verse 3 gained two forms that recur with no cluster of their own, so an analysis approved elsewhere has somewhere to be suggested, and the sentence listing what the text was invented for did not mention them. Also names the gloss language the five phrase clusters are counted in, since `fr` carries a sixth cluster for one of the same phrases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
imnasnainaec
marked this pull request as ready for review
September 2, 2026 19:40
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #5
Four minimal Paratext 9 projects under
test-data/pt9-projects/, for exercising the Paratext 9 interlinear import against a running platform, what no unit test covers:test-data/Pt9InterlinearProjectData.jsonalready covers the converter in isolation; this is its end-to-end counterpart.Each project has one book (PHP) and a handful of verses, so an import finishes immediately and its report is small enough to read whole.
test-data/pt9-projects/README.mddocuments how to install and run them and tabulates what each covers.Manual testing
Copy the four folders into the Platform.Bible project root, start the app with the extension, and run
interlinearizer.importPt9Projector Import from Paratext 9 on each. The projects load and import against a running build. The hash placement follows the converter's status rules and is confirmed as far as importing PIA and reading its analysis catalog; the per-shape linked counts have not each been eyeballed.Notes for review
en.ldmldiffs.*.ldml binaryrule in.gitattributes, not anything this PR sets.noglosslangis worth a look on each run.convertPt9Projectcounts a book as missing identity viaglossLanguage === undefined, and the C#Pt9InterlinearBookrecord carries noWhenWritingNullon that property, so anullrather than an omitted value would slip the check and make a null-language book its own language group.English,es-MX,es,es-MX), which is what the check holding looks like - a slipped null would add a fifth carrying no tag. Suggestive rather than conclusive.🤖 Generated with Claude Code
Devin review: https://app.devin.ai/review/sillsdev/interlinearizer-extension/pull/283
This change is
Summary by CodeRabbit
Documentation
Tests