From b5bf7503b9c9d18420866989f7937749fc39d2e9 Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Mon, 31 Aug 2026 09:11:56 -0400 Subject: [PATCH 1/8] Add Paratext 9 interlinear test projects for import testing Four minimal PT9 projects under test-data/pt9-projects/, for exercising the Paratext 9 interlinear import against a running platform - the part no unit test covers: the platform reading the files, the import service persisting the result, and the WebView presenting it. Each has one book and a handful of verses, so an import finishes immediately and its report reads whole. - PIA is the happy path: two gloss languages over one book, a phrase cluster, word parses, a word and parse cluster sharing one range, an approved verse hash, an excluded cluster, a punctuation entry, a repeated surface form for an ambiguous anchor, and every sense-resolution and bare-word-analysis outcome. - PIB carries all five clusterDrops reasons plus an interlinear file for a book the project has no text for. - PIC covers file identity: a canonical file beside a non-canonical twin, one file with no GlossLanguage and one with no BookId, a legacy language name, and two raw language values resolving onto one tag. - PID has a lexicon and word analyses but no interlinear book file, so the manifest is non-empty while the conversion reports no languages. - test-data/pt9-projects/README.md documents how to install and run them and tabulates what each covers. The root README's test-data line named a fixture removed in #272, so it now describes what the directory holds. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 2 +- test-data/pt9-projects/PIA/51PHPPIA.SFM | 8 + .../pt9-projects/PIA/InterlinearSetup.xml | 15 ++ .../PIA/Interlinear_en/Interlinear_en_PHP.xml | 19 ++ .../PIA/Interlinear_es/Interlinear_es_PHP.xml | 40 +++++ test-data/pt9-projects/PIA/Lexicon.xml | 169 ++++++++++++++++++ test-data/pt9-projects/PIA/Settings.xml | 31 ++++ test-data/pt9-projects/PIA/WordAnalyses.xml | 15 ++ test-data/pt9-projects/PIA/en.ldml | 1 + test-data/pt9-projects/PIA/unique.id | 1 + test-data/pt9-projects/PIB/51PHPPIB.SFM | 6 + .../pt9-projects/PIB/InterlinearSetup.xml | 9 + .../PIB/Interlinear_es/Interlinear_es_JAS.xml | 11 ++ .../PIB/Interlinear_es/Interlinear_es_PHP.xml | 24 +++ test-data/pt9-projects/PIB/Lexicon.xml | 33 ++++ test-data/pt9-projects/PIB/Settings.xml | 31 ++++ test-data/pt9-projects/PIB/en.ldml | 1 + test-data/pt9-projects/PIB/unique.id | 1 + test-data/pt9-projects/PIC/51PHPPIC.SFM | 6 + .../Interlinear_English_PHP.xml | 11 ++ .../Interlinear_es-MX_PHP.xml | 11 ++ .../PIC/Interlinear_es/Interlinear_es_PHP.xml | 11 ++ .../pt9-projects/PIC/Interlinear_es_PHP.xml | 11 ++ .../pt9-projects/PIC/Interlinear_esmx_alt.xml | 11 ++ .../pt9-projects/PIC/Interlinear_nobookid.xml | 11 ++ .../PIC/Interlinear_noglosslang.xml | 11 ++ test-data/pt9-projects/PIC/Lexicon.xml | 33 ++++ test-data/pt9-projects/PIC/Settings.xml | 31 ++++ test-data/pt9-projects/PIC/en.ldml | 1 + test-data/pt9-projects/PIC/unique.id | 1 + test-data/pt9-projects/PID/51PHPPID.SFM | 6 + test-data/pt9-projects/PID/Lexicon.xml | 25 +++ test-data/pt9-projects/PID/Settings.xml | 31 ++++ test-data/pt9-projects/PID/WordAnalyses.xml | 6 + test-data/pt9-projects/PID/en.ldml | 1 + test-data/pt9-projects/PID/unique.id | 1 + test-data/pt9-projects/README.md | 43 +++++ 37 files changed, 678 insertions(+), 1 deletion(-) create mode 100644 test-data/pt9-projects/PIA/51PHPPIA.SFM create mode 100644 test-data/pt9-projects/PIA/InterlinearSetup.xml create mode 100644 test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml create mode 100644 test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml create mode 100644 test-data/pt9-projects/PIA/Lexicon.xml create mode 100644 test-data/pt9-projects/PIA/Settings.xml create mode 100644 test-data/pt9-projects/PIA/WordAnalyses.xml create mode 100644 test-data/pt9-projects/PIA/en.ldml create mode 100644 test-data/pt9-projects/PIA/unique.id create mode 100644 test-data/pt9-projects/PIB/51PHPPIB.SFM create mode 100644 test-data/pt9-projects/PIB/InterlinearSetup.xml create mode 100644 test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_JAS.xml create mode 100644 test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml create mode 100644 test-data/pt9-projects/PIB/Lexicon.xml create mode 100644 test-data/pt9-projects/PIB/Settings.xml create mode 100644 test-data/pt9-projects/PIB/en.ldml create mode 100644 test-data/pt9-projects/PIB/unique.id create mode 100644 test-data/pt9-projects/PIC/51PHPPIC.SFM create mode 100644 test-data/pt9-projects/PIC/Interlinear_English/Interlinear_English_PHP.xml create mode 100644 test-data/pt9-projects/PIC/Interlinear_es-MX/Interlinear_es-MX_PHP.xml create mode 100644 test-data/pt9-projects/PIC/Interlinear_es/Interlinear_es_PHP.xml create mode 100644 test-data/pt9-projects/PIC/Interlinear_es_PHP.xml create mode 100644 test-data/pt9-projects/PIC/Interlinear_esmx_alt.xml create mode 100644 test-data/pt9-projects/PIC/Interlinear_nobookid.xml create mode 100644 test-data/pt9-projects/PIC/Interlinear_noglosslang.xml create mode 100644 test-data/pt9-projects/PIC/Lexicon.xml create mode 100644 test-data/pt9-projects/PIC/Settings.xml create mode 100644 test-data/pt9-projects/PIC/en.ldml create mode 100644 test-data/pt9-projects/PIC/unique.id create mode 100644 test-data/pt9-projects/PID/51PHPPID.SFM create mode 100644 test-data/pt9-projects/PID/Lexicon.xml create mode 100644 test-data/pt9-projects/PID/Settings.xml create mode 100644 test-data/pt9-projects/PID/WordAnalyses.xml create mode 100644 test-data/pt9-projects/PID/en.ldml create mode 100644 test-data/pt9-projects/PID/unique.id create mode 100644 test-data/pt9-projects/README.md diff --git a/README.md b/README.md index 5197d410..62956349 100644 --- a/README.md +++ b/README.md @@ -123,7 +123,7 @@ The general file structure for an extension is as follows: - `assets/descriptions/description-.md` contains a brief description of the extension in the language specified by `` - `contributions/` contains JSON files the platform uses to extend data structures for things like menus and settings. The JSON files are referenced from the manifest - `public/` contains other static files that are copied into the build folder -- `test-data/` contains sample interlinear XML (e.g. `Interlinear_en_MAT.xml`) for development and tests +- `test-data/` contains sample Paratext 9 interlinear data for development and tests: `Pt9InterlinearProjectData.json` is the served payload the converter's tests read, and `test-data/pt9-projects/` holds whole Paratext 9 projects for testing an import by hand (see its [README](test-data/pt9-projects/README.md)) - `.github/` contains files to facilitate integration with GitHub - `.github/workflows` contains [GitHub Actions](https://github.com/features/actions) workflows for automating various processes in this repo (e.g. **Test** and **Lint** on push/PR to main, release-prep, hotfix-\*; **Publish** and **Bump Versions** manual dispatch; **CodeQL** for security) - `.github/assets/release-body.md` is the template for the body of [releases published using GitHub Actions](#publishing). The Publish workflow substitutes its version placeholders into a copy outside the repo, and that copy plus a generated changelog becomes the release body diff --git a/test-data/pt9-projects/PIA/51PHPPIA.SFM b/test-data/pt9-projects/PIA/51PHPPIA.SFM new file mode 100644 index 00000000..2334d511 --- /dev/null +++ b/test-data/pt9-projects/PIA/51PHPPIA.SFM @@ -0,0 +1,8 @@ +\id PHP - PT9 Import A +\h PIA +\mt1 Philippians +\c 1 +\p +\v 1 Paul and Timothy and Silas greeted the saints in Christ Jesus. +\v 2 Grace and peace to you. +\v 3 The runners walked home. diff --git a/test-data/pt9-projects/PIA/InterlinearSetup.xml b/test-data/pt9-projects/PIA/InterlinearSetup.xml new file mode 100644 index 00000000..7d70cfff --- /dev/null +++ b/test-data/pt9-projects/PIA/InterlinearSetup.xml @@ -0,0 +1,15 @@ + + + + 12 + false + false + false + + + 12 + false + false + false + + diff --git a/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml new file mode 100644 index 00000000..5788be88 --- /dev/null +++ b/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml @@ -0,0 +1,19 @@ + + + + + PHP 1:1 + + + + + + + + PHP 1:3 + + + + + + diff --git a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml new file mode 100644 index 00000000..4ed4f74a --- /dev/null +++ b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml @@ -0,0 +1,40 @@ + + + + + PHP 1:1 + + + + + + + + + + + + + + + PHP 1:2 + + + + + true + + .; + + + + PHP 1:3 + + + + + + + + + diff --git a/test-data/pt9-projects/PIA/Lexicon.xml b/test-data/pt9-projects/PIA/Lexicon.xml new file mode 100644 index 00000000..a92868cb --- /dev/null +++ b/test-data/pt9-projects/PIA/Lexicon.xml @@ -0,0 +1,169 @@ + + + en + Charis SIL + 12 + + + greeted + + + + + + + + + + + + Pablo + Paul + + + + + + + + y + and + + + e + + + + + + + + Timoteo + + + + + + + + Silas + + + + + + + + el + + + + + + + + santos + + + saints + + + + + + + + en + + + + + + + + Gracia + + + + + + + + paz + + + + + + + + a + + + + + + + + ustedes + + + + + + + + casa + + + + + + + + Cristo Jesus + + + + + + + + correr + + + + + + + + -dores + + + + + + + + caminar + + + + + + + + -o + + + + + + + + casa + + + + + diff --git a/test-data/pt9-projects/PIA/Settings.xml b/test-data/pt9-projects/PIA/Settings.xml new file mode 100644 index 00000000..63addbd7 --- /dev/null +++ b/test-data/pt9-projects/PIA/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import A - clean + 65001 + T + + NFC + PIA + a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1 + Charis SIL + 12 + + + en::: + 41MAT + + PIA.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PIA/WordAnalyses.xml b/test-data/pt9-projects/PIA/WordAnalyses.xml new file mode 100644 index 00000000..414b72c7 --- /dev/null +++ b/test-data/pt9-projects/PIA/WordAnalyses.xml @@ -0,0 +1,15 @@ + + + + Stem:walkSuffix:ed + + + Stem:jumpSuffix:ed + + + brokenid + + + + + diff --git a/test-data/pt9-projects/PIA/en.ldml b/test-data/pt9-projects/PIA/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PIA/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PIA/unique.id b/test-data/pt9-projects/PIA/unique.id new file mode 100644 index 00000000..4fe808d8 --- /dev/null +++ b/test-data/pt9-projects/PIA/unique.id @@ -0,0 +1 @@ +aaaaaaaa-1111-4111-8111-aaaaaaaaaaaa diff --git a/test-data/pt9-projects/PIB/51PHPPIB.SFM b/test-data/pt9-projects/PIB/51PHPPIB.SFM new file mode 100644 index 00000000..517df09f --- /dev/null +++ b/test-data/pt9-projects/PIB/51PHPPIB.SFM @@ -0,0 +1,6 @@ +\id PHP - PT9 Import B +\h PIB +\mt1 Philippians +\c 1 +\p +\v 1 Alpha beta gamma delta. diff --git a/test-data/pt9-projects/PIB/InterlinearSetup.xml b/test-data/pt9-projects/PIB/InterlinearSetup.xml new file mode 100644 index 00000000..fe146674 --- /dev/null +++ b/test-data/pt9-projects/PIB/InterlinearSetup.xml @@ -0,0 +1,9 @@ + + + + 12 + false + false + false + + diff --git a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_JAS.xml b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_JAS.xml new file mode 100644 index 00000000..80897785 --- /dev/null +++ b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_JAS.xml @@ -0,0 +1,11 @@ + + + + + JAS 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml new file mode 100644 index 00000000..5578be55 --- /dev/null +++ b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml @@ -0,0 +1,24 @@ + + + + + PHP 1:1 + + + + + + + + + + + + + PHP 2:5 + + + + + + diff --git a/test-data/pt9-projects/PIB/Lexicon.xml b/test-data/pt9-projects/PIB/Lexicon.xml new file mode 100644 index 00000000..1219e75f --- /dev/null +++ b/test-data/pt9-projects/PIB/Lexicon.xml @@ -0,0 +1,33 @@ + + + en + Charis SIL + 12 + + + + + + + alfa + + + + + + + + beta + + + + + + + + delta + + + + + diff --git a/test-data/pt9-projects/PIB/Settings.xml b/test-data/pt9-projects/PIB/Settings.xml new file mode 100644 index 00000000..eaf6e0be --- /dev/null +++ b/test-data/pt9-projects/PIB/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import B - drops + 65001 + T + + NFC + PIB + b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2 + Charis SIL + 12 + + + en::: + 41MAT + + PIB.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PIB/en.ldml b/test-data/pt9-projects/PIB/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PIB/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PIB/unique.id b/test-data/pt9-projects/PIB/unique.id new file mode 100644 index 00000000..dc9b6f71 --- /dev/null +++ b/test-data/pt9-projects/PIB/unique.id @@ -0,0 +1 @@ +bbbbbbbb-2222-4222-8222-bbbbbbbbbbbb diff --git a/test-data/pt9-projects/PIC/51PHPPIC.SFM b/test-data/pt9-projects/PIC/51PHPPIC.SFM new file mode 100644 index 00000000..29f45944 --- /dev/null +++ b/test-data/pt9-projects/PIC/51PHPPIC.SFM @@ -0,0 +1,6 @@ +\id PHP - PT9 Import C +\h PIC +\mt1 Philippians +\c 1 +\p +\v 1 Uno dos tres. diff --git a/test-data/pt9-projects/PIC/Interlinear_English/Interlinear_English_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_English/Interlinear_English_PHP.xml new file mode 100644 index 00000000..78fb4c7c --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_English/Interlinear_English_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_es-MX/Interlinear_es-MX_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_es-MX/Interlinear_es-MX_PHP.xml new file mode 100644 index 00000000..673d720c --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_es-MX/Interlinear_es-MX_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_es/Interlinear_es_PHP.xml new file mode 100644 index 00000000..819b769a --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_es/Interlinear_es_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_es_PHP.xml new file mode 100644 index 00000000..c49c9e65 --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_es_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_esmx_alt.xml b/test-data/pt9-projects/PIC/Interlinear_esmx_alt.xml new file mode 100644 index 00000000..3209f124 --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_esmx_alt.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_nobookid.xml b/test-data/pt9-projects/PIC/Interlinear_nobookid.xml new file mode 100644 index 00000000..947ac9ab --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_nobookid.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_noglosslang.xml b/test-data/pt9-projects/PIC/Interlinear_noglosslang.xml new file mode 100644 index 00000000..d858c6c9 --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_noglosslang.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Lexicon.xml b/test-data/pt9-projects/PIC/Lexicon.xml new file mode 100644 index 00000000..37e794fd --- /dev/null +++ b/test-data/pt9-projects/PIC/Lexicon.xml @@ -0,0 +1,33 @@ + + + en + Charis SIL + 12 + + + + + + + uno + + + + + + + + dos + + + + + + + + tres + + + + + diff --git a/test-data/pt9-projects/PIC/Settings.xml b/test-data/pt9-projects/PIC/Settings.xml new file mode 100644 index 00000000..b2def3a0 --- /dev/null +++ b/test-data/pt9-projects/PIC/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import C - identity + 65001 + T + + NFC + PIC + c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3 + Charis SIL + 12 + + + en::: + 41MAT + + PIC.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PIC/en.ldml b/test-data/pt9-projects/PIC/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PIC/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PIC/unique.id b/test-data/pt9-projects/PIC/unique.id new file mode 100644 index 00000000..e0364517 --- /dev/null +++ b/test-data/pt9-projects/PIC/unique.id @@ -0,0 +1 @@ +cccccccc-3333-4333-8333-cccccccccccc diff --git a/test-data/pt9-projects/PID/51PHPPID.SFM b/test-data/pt9-projects/PID/51PHPPID.SFM new file mode 100644 index 00000000..8618c146 --- /dev/null +++ b/test-data/pt9-projects/PID/51PHPPID.SFM @@ -0,0 +1,6 @@ +\id PHP - PT9 Import D +\h PID +\mt1 Philippians +\c 1 +\p +\v 1 Solo texto aqui. diff --git a/test-data/pt9-projects/PID/Lexicon.xml b/test-data/pt9-projects/PID/Lexicon.xml new file mode 100644 index 00000000..62104c39 --- /dev/null +++ b/test-data/pt9-projects/PID/Lexicon.xml @@ -0,0 +1,25 @@ + + + en + Charis SIL + 12 + + + texto + + + + + + + + + + + + solo + + + + + diff --git a/test-data/pt9-projects/PID/Settings.xml b/test-data/pt9-projects/PID/Settings.xml new file mode 100644 index 00000000..77ef6ac7 --- /dev/null +++ b/test-data/pt9-projects/PID/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import D - lexicon only + 65001 + T + + NFC + PID + d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4 + Charis SIL + 12 + + + en::: + 41MAT + + PID.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PID/WordAnalyses.xml b/test-data/pt9-projects/PID/WordAnalyses.xml new file mode 100644 index 00000000..35c076c2 --- /dev/null +++ b/test-data/pt9-projects/PID/WordAnalyses.xml @@ -0,0 +1,6 @@ + + + + Stem:aquSuffix:i + + diff --git a/test-data/pt9-projects/PID/en.ldml b/test-data/pt9-projects/PID/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PID/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PID/unique.id b/test-data/pt9-projects/PID/unique.id new file mode 100644 index 00000000..830751f9 --- /dev/null +++ b/test-data/pt9-projects/PID/unique.id @@ -0,0 +1 @@ +dddddddd-4444-4444-8444-dddddddddddd diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md new file mode 100644 index 00000000..c87880e2 --- /dev/null +++ b/test-data/pt9-projects/README.md @@ -0,0 +1,43 @@ +# Paratext 9 interlinear test projects + +Four minimal Paratext 9 projects for exercising the Paratext 9 interlinear import by hand, end to +end, against a real running platform. Each has one book (PHP) and a handful of verses, so an import +finishes immediately and its report is small enough to read whole. + +These complement [`../Pt9InterlinearProjectData.json`](../Pt9InterlinearProjectData.json), which is +the served payload the converter's unit tests read directly. These are whole projects, for the part +no unit test covers: the platform reading the files, the import service persisting the result, and +the WebView presenting it. The XML schema they follow is documented in +[`src/parsers/pt9/pt9-xml.md`](../../src/parsers/pt9/pt9-xml.md). + +## Using them + +1. Copy the project folders into the Platform.Bible project root — on Windows + `%USERPROFILE%\.platform.bible\projects\Paratext 9 Projects\`, elsewhere + `~/.platform.bible/projects/Paratext 9 Projects/`. Alternatively point + `PLATFORM_BIBLE_PROJECT_ROOT_FOLDER` at a folder holding them, which leaves your own projects + untouched. +2. Start Platform.Bible with the extension (`npm start`). +3. Open the Interlinearizer on one of the projects and use **Import from Paratext 9**, or invoke + `interlinearizer.importPt9Project` with the project's ID directly. + +The platform serves this data through the `platformScripture.Pt9Interlinear` projectInterface, which +is advertised only on unpublished (editable) projects — so a project copied here must load as an +ordinary editable project, not as a resource. + +Project GUIDs and `unique.id` values are fixed rather than generated, so re-copying a project after +deleting it reuses the same Platform.Bible project ID and any interlinearizer state still keyed to +it. + +## What each project covers + +| Project | Purpose | Key features | +| ------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `PIA` | Happy path | Two gloss languages (`es`, `en`) over one book, so records merge across languages and one token's parses conflict; a phrase cluster; word parses; a word and a parse cluster sharing one range; an approved verse hash; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; every sense-resolution outcome; every bare-word-analysis outcome | +| `PIB` | Cluster drops | All five `clusterDrops` reasons — `formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound` — plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | +| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | +| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | + +For the remaining case — a project the import must refuse outright — use any project with no +`Lexicon.xml`, no `WordAnalyses.xml`, and no `Interlinear_*` file. Its manifest is empty, which is +what makes the import throw rather than create anything. From 6959146e1613e5c7ecbf5123072ce51047bbff01 Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Mon, 31 Aug 2026 11:09:15 -0400 Subject: [PATCH 2/8] Build PIA's verses from invented words rather than scripture PIA's text was two near-verbatim verses of Philippians followed by an unrelated English sentence - a blend that read as scripture without being it, and invited reading the glosses as a translation. Its three verses are now invented words throughout, and shorter: a form that repeats within a verse for the ambiguous anchor, a two-word run for the phrase cluster, and stem+suffix forms for the parses. Every feature the project covered it still covers - both gloss languages, the phrase, the paired word+parse range, the approved hash, the excluded cluster, the punctuation entry, all five sense-resolution outcomes, and all four bare-word-analysis outcomes. The README says the text is invented and why those shapes were chosen. Co-Authored-By: Claude Opus 5 (1M context) --- test-data/pt9-projects/PIA/51PHPPIA.SFM | 6 +- .../PIA/Interlinear_en/Interlinear_en_PHP.xml | 7 +- .../PIA/Interlinear_es/Interlinear_es_PHP.xml | 33 ++--- test-data/pt9-projects/PIA/Lexicon.xml | 137 ++++++------------ test-data/pt9-projects/PIA/WordAnalyses.xml | 8 +- test-data/pt9-projects/README.md | 5 + 6 files changed, 72 insertions(+), 124 deletions(-) diff --git a/test-data/pt9-projects/PIA/51PHPPIA.SFM b/test-data/pt9-projects/PIA/51PHPPIA.SFM index 2334d511..897049cb 100644 --- a/test-data/pt9-projects/PIA/51PHPPIA.SFM +++ b/test-data/pt9-projects/PIA/51PHPPIA.SFM @@ -3,6 +3,6 @@ \mt1 Philippians \c 1 \p -\v 1 Paul and Timothy and Silas greeted the saints in Christ Jesus. -\v 2 Grace and peace to you. -\v 3 The runners walked home. +\v 1 Wug blicket wug dax fep gorp. +\v 2 Tomo zav flib. +\v 3 Wugs daxes glorp. diff --git a/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml index 5788be88..97b78b8e 100644 --- a/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml @@ -4,15 +4,14 @@ PHP 1:1 - - - + + PHP 1:3 - + diff --git a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml index 4ed4f74a..8e80b10a 100644 --- a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml @@ -4,36 +4,29 @@ PHP 1:1 - - - - - - - - - - + + + + + PHP 1:2 - - - - true - - .; + true + + + .; PHP 1:3 - - - - + + + + diff --git a/test-data/pt9-projects/PIA/Lexicon.xml b/test-data/pt9-projects/PIA/Lexicon.xml index a92868cb..e2249dfe 100644 --- a/test-data/pt9-projects/PIA/Lexicon.xml +++ b/test-data/pt9-projects/PIA/Lexicon.xml @@ -5,163 +5,114 @@ 12 - greeted + flibs - - + + - + - - Pablo - Paul + + uno + one - - - - - - - y - and - - - e - - - - - - - - Timoteo - - - - - - - - Silas + + primero - + - - el + + dos - - - - - - - santos - - - saints - - - - - - - - en - - - - - - - - Gracia + + two - + - - paz + + tres cuatro - + - - a + + cinco - + - - ustedes + + seis - + - - casa + + siete - + - - Cristo Jesus + + ocho - + - - correr + + wug - + - - -dores + + -s - + - - caminar + + dax - + - - -o + + -es - + - - casa + + glorp diff --git a/test-data/pt9-projects/PIA/WordAnalyses.xml b/test-data/pt9-projects/PIA/WordAnalyses.xml index 414b72c7..4d089f40 100644 --- a/test-data/pt9-projects/PIA/WordAnalyses.xml +++ b/test-data/pt9-projects/PIA/WordAnalyses.xml @@ -1,10 +1,10 @@  - - Stem:walkSuffix:ed + + Stem:daxSuffix:es - - Stem:jumpSuffix:ed + + Stem:fepSuffix:ped brokenid diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md index c87880e2..bb6a2771 100644 --- a/test-data/pt9-projects/README.md +++ b/test-data/pt9-projects/README.md @@ -4,6 +4,11 @@ Four minimal Paratext 9 projects for exercising the Paratext 9 interlinear impor end, against a real running platform. Each has one book (PHP) and a handful of verses, so an import finishes immediately and its report is small enough to read whole. +The verse text is invented words, not scripture. Nothing here should be read as a translation of +anything, and the words are chosen only for the shapes the import has to handle — a form that +repeats within a verse, runs of words a phrase can span, and forms that divide into a stem and a +suffix. + These complement [`../Pt9InterlinearProjectData.json`](../Pt9InterlinearProjectData.json), which is the served payload the converter's unit tests read directly. These are whole projects, for the part no unit test covers: the platform reading the files, the import service persisting the result, and From 81fd3bfe69af40ab5188ae9d2b4b9048330b46ae Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Mon, 31 Aug 2026 12:45:40 -0400 Subject: [PATCH 3/8] Clean up new README --- test-data/pt9-projects/README.md | 41 ++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md index bb6a2771..0a9c7f3f 100644 --- a/test-data/pt9-projects/README.md +++ b/test-data/pt9-projects/README.md @@ -1,33 +1,38 @@ # Paratext 9 interlinear test projects -Four minimal Paratext 9 projects for exercising the Paratext 9 interlinear import by hand, end to -end, against a real running platform. Each has one book (PHP) and a handful of verses, so an import -finishes immediately and its report is small enough to read whole. +Four minimal Paratext 9 projects for exercising the Paratext 9 interlinear import against a real +running platform. Each has one book (PHP) and a handful of verses, so an import finishes immediately +and its report is small enough to read whole. -The verse text is invented words, not scripture. Nothing here should be read as a translation of -anything, and the words are chosen only for the shapes the import has to handle — a form that -repeats within a verse, runs of words a phrase can span, and forms that divide into a stem and a -suffix. +The verse text is invented for the shapes the import has to handle: a form that repeats within a +verse, runs of words a phrase can span, and forms that divide into a stem and a suffix. These complement [`../Pt9InterlinearProjectData.json`](../Pt9InterlinearProjectData.json), which is -the served payload the converter's unit tests read directly. These are whole projects, for the part -no unit test covers: the platform reading the files, the import service persisting the result, and -the WebView presenting it. The XML schema they follow is documented in +the served payload the converter's unit tests read directly. These are whole projects, for what no +unit test covers: + +- the platform reading the files; +- the import service persisting the result; +- the WebView presenting it. + +The XML schema they follow is documented in [`src/parsers/pt9/pt9-xml.md`](../../src/parsers/pt9/pt9-xml.md). ## Using them -1. Copy the project folders into the Platform.Bible project root — on Windows - `%USERPROFILE%\.platform.bible\projects\Paratext 9 Projects\`, elsewhere - `~/.platform.bible/projects/Paratext 9 Projects/`. Alternatively point - `PLATFORM_BIBLE_PROJECT_ROOT_FOLDER` at a folder holding them, which leaves your own projects - untouched. +1. Copy the project folders into the Platform.Bible project root. + + - On Windows: `%USERPROFILE%\.platform.bible\projects\Paratext 9 Projects\`. + - Elsewhere: `~/.platform.bible/projects/Paratext 9 Projects/`. + - Alternatively point `PLATFORM_BIBLE_PROJECT_ROOT_FOLDER` at a folder holding them, which leaves + your own projects untouched. + 2. Start Platform.Bible with the extension (`npm start`). 3. Open the Interlinearizer on one of the projects and use **Import from Paratext 9**, or invoke `interlinearizer.importPt9Project` with the project's ID directly. The platform serves this data through the `platformScripture.Pt9Interlinear` projectInterface, which -is advertised only on unpublished (editable) projects — so a project copied here must load as an +is advertised only on unpublished (editable) projects. So a project copied here must load as an ordinary editable project, not as a resource. Project GUIDs and `unique.id` values are fixed rather than generated, so re-copying a project after @@ -39,10 +44,10 @@ it. | Project | Purpose | Key features | | ------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `PIA` | Happy path | Two gloss languages (`es`, `en`) over one book, so records merge across languages and one token's parses conflict; a phrase cluster; word parses; a word and a parse cluster sharing one range; an approved verse hash; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; every sense-resolution outcome; every bare-word-analysis outcome | -| `PIB` | Cluster drops | All five `clusterDrops` reasons — `formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound` — plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | +| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`), plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | | `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | | `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | -For the remaining case — a project the import must refuse outright — use any project with no +For the remaining case, a project the import must refuse outright, use any project with no `Lexicon.xml`, no `WordAnalyses.xml`, and no `Interlinear_*` file. Its manifest is empty, which is what makes the import throw rather than create anything. From affa9aee4f6bc7fbfffcff37dd5c2d8a2040ac52 Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Tue, 1 Sep 2026 16:00:49 -0400 Subject: [PATCH 4/8] Make PIA's fullest gloss language the one the WebView shows Importing PIA looked like it produced no glosses. It had produced them - six records carried an es gloss - but the platform scans interlinear files in ordinal path order, that order becomes the project's analysisLanguages, and the WebView renders only analysisLanguages[0]. Interlinear_en sorted ahead of Interlinear_es, so the language on screen was the deliberately thin one: two glossed records, both in verse 1, and nothing at all in verses 2 and 3. PIA's second gloss language is now fr, so Interlinear_es sorts first and the es glosses are what the view shows. The thin file keeps its job - it still overlaps es on two tokens so their records merge, and still reads "daxes" as a single stem where es reads stem plus suffix, so the parses still conflict. Co-Authored-By: Claude Opus 5 (1M context) --- test-data/pt9-projects/PIA/InterlinearSetup.xml | 2 +- .../Interlinear_fr_PHP.xml} | 4 ++-- test-data/pt9-projects/PIA/Lexicon.xml | 6 +++--- test-data/pt9-projects/README.md | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) rename test-data/pt9-projects/PIA/{Interlinear_en/Interlinear_en_PHP.xml => Interlinear_fr/Interlinear_fr_PHP.xml} (84%) diff --git a/test-data/pt9-projects/PIA/InterlinearSetup.xml b/test-data/pt9-projects/PIA/InterlinearSetup.xml index 7d70cfff..270769ec 100644 --- a/test-data/pt9-projects/PIA/InterlinearSetup.xml +++ b/test-data/pt9-projects/PIA/InterlinearSetup.xml @@ -6,7 +6,7 @@ false false - + 12 false false diff --git a/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml similarity index 84% rename from test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml rename to test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml index 97b78b8e..7bdde168 100644 --- a/test-data/pt9-projects/PIA/Interlinear_en/Interlinear_en_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml @@ -1,11 +1,11 @@  - + PHP 1:1 - + diff --git a/test-data/pt9-projects/PIA/Lexicon.xml b/test-data/pt9-projects/PIA/Lexicon.xml index e2249dfe..3142ab5b 100644 --- a/test-data/pt9-projects/PIA/Lexicon.xml +++ b/test-data/pt9-projects/PIA/Lexicon.xml @@ -18,7 +18,7 @@ uno - one + un primero @@ -31,8 +31,8 @@ dos - - two + + deux diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md index 0a9c7f3f..6082019a 100644 --- a/test-data/pt9-projects/README.md +++ b/test-data/pt9-projects/README.md @@ -43,7 +43,7 @@ it. | Project | Purpose | Key features | | ------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `PIA` | Happy path | Two gloss languages (`es`, `en`) over one book, so records merge across languages and one token's parses conflict; a phrase cluster; word parses; a word and a parse cluster sharing one range; an approved verse hash; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; every sense-resolution outcome; every bare-word-analysis outcome | +| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; a phrase cluster; word parses; a word and a parse cluster sharing one range; an approved verse hash; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; every sense-resolution outcome; every bare-word-analysis outcome | | `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`), plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | | `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | | `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | From cd30f669c8df44861c4400c86299fc6d2bc6a36b Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Tue, 1 Sep 2026 16:49:20 -0400 Subject: [PATCH 5/8] Cover the phrase-anchoring cases the fixtures were missing Phrases were represented by one plain two-word cluster in PIA, so the branch that anchors them had only its happy path exercised. The rest of that branch now has fixtures too. PIA gains two verses rather than reworking the existing three, so nothing already covered shifts: - verse 4 repeats one two-word run twice and glosses it twice, so the first cluster is ambiguous between the two runs and the second, anchoring to the run the first left, is marked excluded; - verse 5 carries a three-word phrase, glossed by both languages, so two phrase records merge at one run. PIB gains the failures, beside a phrase that anchors as a control: one whose form matches no run of words, and one whose form is blank, which is the separate guard ahead of the run search. Both count as formMismatch. Co-Authored-By: Claude Opus 5 (1M context) --- test-data/pt9-projects/PIA/51PHPPIA.SFM | 2 ++ .../PIA/Interlinear_es/Interlinear_es_PHP.xml | 13 +++++++++++++ .../PIA/Interlinear_fr/Interlinear_fr_PHP.xml | 6 ++++++ test-data/pt9-projects/PIA/Lexicon.xml | 19 +++++++++++++++++++ .../PIB/Interlinear_es/Interlinear_es_PHP.xml | 3 +++ test-data/pt9-projects/PIB/Lexicon.xml | 8 ++++++++ test-data/pt9-projects/README.md | 12 ++++++------ 7 files changed, 57 insertions(+), 6 deletions(-) diff --git a/test-data/pt9-projects/PIA/51PHPPIA.SFM b/test-data/pt9-projects/PIA/51PHPPIA.SFM index 897049cb..e095520b 100644 --- a/test-data/pt9-projects/PIA/51PHPPIA.SFM +++ b/test-data/pt9-projects/PIA/51PHPPIA.SFM @@ -6,3 +6,5 @@ \v 1 Wug blicket wug dax fep gorp. \v 2 Tomo zav flib. \v 3 Wugs daxes glorp. +\v 4 Zim zam zim zam kex. +\v 5 Nol pim tuk vex. diff --git a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml index 8e80b10a..290f1e00 100644 --- a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml @@ -29,5 +29,18 @@ + + PHP 1:4 + + + true + + + + PHP 1:5 + + + + diff --git a/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml index 7bdde168..d3c7609d 100644 --- a/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml @@ -14,5 +14,11 @@ + + PHP 1:5 + + + + diff --git a/test-data/pt9-projects/PIA/Lexicon.xml b/test-data/pt9-projects/PIA/Lexicon.xml index 3142ab5b..25cc9d85 100644 --- a/test-data/pt9-projects/PIA/Lexicon.xml +++ b/test-data/pt9-projects/PIA/Lexicon.xml @@ -44,6 +44,25 @@ + + + + + nueve diez + + + + + + + + once doce trece + + + onze douze treize + + + diff --git a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml index 5578be55..e363ba52 100644 --- a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml +++ b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml @@ -12,6 +12,9 @@ + + + diff --git a/test-data/pt9-projects/PIB/Lexicon.xml b/test-data/pt9-projects/PIB/Lexicon.xml index 1219e75f..b73ab02d 100644 --- a/test-data/pt9-projects/PIB/Lexicon.xml +++ b/test-data/pt9-projects/PIB/Lexicon.xml @@ -29,5 +29,13 @@ + + + + + gama delta + + + diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md index 6082019a..5be2ae3f 100644 --- a/test-data/pt9-projects/README.md +++ b/test-data/pt9-projects/README.md @@ -41,12 +41,12 @@ it. ## What each project covers -| Project | Purpose | Key features | -| ------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; a phrase cluster; word parses; a word and a parse cluster sharing one range; an approved verse hash; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; every sense-resolution outcome; every bare-word-analysis outcome | -| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`), plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | -| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | -| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | +| Project | Purpose | Key features | +| ------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; four phrase clusters (a plain two-word one, one ambiguous between two matching runs, an excluded one, and a three-word one both languages gloss); word parses; a word and a parse cluster sharing one range; an approved verse hash; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; every sense-resolution outcome; every bare-word-analysis outcome | +| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`); two phrases that drop, one matching no run of words and one whose form is blank, beside a phrase that anchors as a control; plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | +| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | +| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | For the remaining case, a project the import must refuse outright, use any project with no `Lexicon.xml`, no `WordAnalyses.xml`, and no `Interlinear_*` file. Its manifest is empty, which is From 409a6e6c7e5c912fcdf21ae685ce1c3878876d37 Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Wed, 2 Sep 2026 14:08:38 -0400 Subject: [PATCH 6/8] Make the fixtures' imported records reach the screen One `Hash` existed across all four projects, on a PIA verse holding no phrase. Since only an approved link renders - the phrase views read approved links alone, and a token falls back to the suggestion pool, which admits an analysis only once it holds an approved link - almost nothing these projects imported was visible in the app. PIA showed a single gloss, on `zav` in verse 2, and none of its four phrases; PIB's control phrase could be confirmed only from the report's count. - PIA 1:4 is approved, so its first `zim zam` lands visible and carries the low confidence its ambiguous anchor earned, beside the excluded twin that imports rejected. - A `zam zim` cluster overlaps that approved run on one token, which demotes it to candidate - the `approvedDemotedToCandidate` counter had no fixture. - PIA 1:5 is approved by `es` alone, so the phrase both languages gloss merges back down to suggested. That cross-language rule had no fixture either, and this one stays invisible on purpose. - PIA verse 3 gains `zav` and `flib` with no clusters of their own. Both forms carry an approved analysis in verse 2, so they exercise the suggestion pool, which nothing reached before: `zav` resolves a gloss and should show one, while `flib`'s dangling sense leaves its payload textless and filtered from the dropdown. - PIB's only verse is approved, so its control phrase lands somewhere a tester can see rather than only in a count. Verse text stays invented. Every cluster range still covers the word it names, checked against the edited verse text. Co-Authored-By: Claude Opus 5 (1M context) --- test-data/pt9-projects/PIA/51PHPPIA.SFM | 2 +- .../PIA/Interlinear_es/Interlinear_es_PHP.xml | 5 +++-- test-data/pt9-projects/PIA/Lexicon.xml | 8 ++++++++ .../PIB/Interlinear_es/Interlinear_es_PHP.xml | 2 +- test-data/pt9-projects/README.md | 12 ++++++------ 5 files changed, 19 insertions(+), 10 deletions(-) diff --git a/test-data/pt9-projects/PIA/51PHPPIA.SFM b/test-data/pt9-projects/PIA/51PHPPIA.SFM index e095520b..f53af2b3 100644 --- a/test-data/pt9-projects/PIA/51PHPPIA.SFM +++ b/test-data/pt9-projects/PIA/51PHPPIA.SFM @@ -5,6 +5,6 @@ \p \v 1 Wug blicket wug dax fep gorp. \v 2 Tomo zav flib. -\v 3 Wugs daxes glorp. +\v 3 Wugs daxes glorp zav flib. \v 4 Zim zam zim zam kex. \v 5 Nol pim tuk vex. diff --git a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml index 290f1e00..fc539dcd 100644 --- a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml @@ -31,14 +31,15 @@ PHP 1:4 - + + true PHP 1:5 - + diff --git a/test-data/pt9-projects/PIA/Lexicon.xml b/test-data/pt9-projects/PIA/Lexicon.xml index 25cc9d85..ef93e5e4 100644 --- a/test-data/pt9-projects/PIA/Lexicon.xml +++ b/test-data/pt9-projects/PIA/Lexicon.xml @@ -52,6 +52,14 @@ + + + + + diez nueve + + + diff --git a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml index e363ba52..fa14f7f5 100644 --- a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml +++ b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml @@ -3,7 +3,7 @@ PHP 1:1 - + diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md index 5be2ae3f..3cc42a0e 100644 --- a/test-data/pt9-projects/README.md +++ b/test-data/pt9-projects/README.md @@ -41,12 +41,12 @@ it. ## What each project covers -| Project | Purpose | Key features | -| ------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; four phrase clusters (a plain two-word one, one ambiguous between two matching runs, an excluded one, and a three-word one both languages gloss); word parses; a word and a parse cluster sharing one range; an approved verse hash; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; every sense-resolution outcome; every bare-word-analysis outcome | -| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`); two phrases that drop, one matching no run of words and one whose form is blank, beside a phrase that anchors as a control; plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | -| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | -| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | +| Project | Purpose | Key features | +| ------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; five phrase clusters (a plain two-word one, one ambiguous between two matching runs, one overlapping an approved run so it demotes to `candidate`, an excluded one, and a three-word one both languages gloss); word parses; a word and a parse cluster sharing one range; three approved verse hashes, one of them on a verse only `es` approves, so the phrase both languages gloss merges down to `suggested`; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; two unanalyzed tokens whose forms carry an approved analysis elsewhere in the book, for the suggestion pool; every sense-resolution outcome; every bare-word-analysis outcome | +| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`); two phrases that drop, one matching no run of words and one whose form is blank, beside a phrase that anchors as a control, in an approved verse so it lands somewhere visible; plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | +| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | +| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | For the remaining case, a project the import must refuse outright, use any project with no `Lexicon.xml`, no `WordAnalyses.xml`, and no `Interlinear_*` file. Its manifest is empty, which is From 980c8c7abada28cba22c49a0d70621925158ad04 Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Wed, 2 Sep 2026 15:19:18 -0400 Subject: [PATCH 7/8] Give every analysis shape an approved instance No morpheme breakdown was linked to anything: every parse cluster sits in PIA 1:3, which carried no `Hash`, so all of them imported `suggested` and the catalog listed them as applied nowhere. The same held for most word shapes - of thirteen analyses, two were linked. Verse hashes now sit where each shape needs one: - es 1:1 approved, which covers a word cluster with a resolving sense, one whose sense dangles from an entry that exists (`gorp`), a second cluster on a repeated form so its anchor is a guess and it carries low confidence, and the `dax fep` phrase. - es 1:3 approved, which covers the stem-plus-suffix breakdowns and the word and parse clusters that share one range. - fr 1:1 approved too, so the records both languages contribute merge to approved rather than being reduced by the disagreement rule - one of them with a per-language sense, so the merged record resolves no single sense. - fr 1:4 is new: a one-morpheme parse on `kex`, which `es` leaves alone. A single-stem breakdown cannot be linked at a token that already carries a two-morpheme one, since the second approved record on a token demotes to candidate, so it needs a token of its own. Left deliberately unlinked, each the only instance of its case: the fr parse of `daxes` competing with the es parse on that token, `tomo` rejected for being excluded, and `nol pim tuk` in the verse only `es` approves. Every cluster range still covers the word it names. Co-Authored-By: Claude Opus 5 (1M context) --- .../PIA/Interlinear_es/Interlinear_es_PHP.xml | 4 ++-- .../PIA/Interlinear_fr/Interlinear_fr_PHP.xml | 8 +++++++- test-data/pt9-projects/PIA/Lexicon.xml | 8 ++++++++ test-data/pt9-projects/README.md | 2 +- 4 files changed, 18 insertions(+), 4 deletions(-) diff --git a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml index fc539dcd..e29045d6 100644 --- a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml @@ -3,7 +3,7 @@ PHP 1:1 - + @@ -22,7 +22,7 @@ PHP 1:3 - + diff --git a/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml index d3c7609d..4c512ab2 100644 --- a/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml +++ b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml @@ -3,7 +3,7 @@ PHP 1:1 - + @@ -14,6 +14,12 @@ + + PHP 1:4 + + + + PHP 1:5 diff --git a/test-data/pt9-projects/PIA/Lexicon.xml b/test-data/pt9-projects/PIA/Lexicon.xml index ef93e5e4..c19bbb0b 100644 --- a/test-data/pt9-projects/PIA/Lexicon.xml +++ b/test-data/pt9-projects/PIA/Lexicon.xml @@ -135,6 +135,14 @@ + + + + + quinze + + + diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md index 3cc42a0e..a55f06bd 100644 --- a/test-data/pt9-projects/README.md +++ b/test-data/pt9-projects/README.md @@ -43,7 +43,7 @@ it. | Project | Purpose | Key features | | ------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; five phrase clusters (a plain two-word one, one ambiguous between two matching runs, one overlapping an approved run so it demotes to `candidate`, an excluded one, and a three-word one both languages gloss); word parses; a word and a parse cluster sharing one range; three approved verse hashes, one of them on a verse only `es` approves, so the phrase both languages gloss merges down to `suggested`; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; two unanalyzed tokens whose forms carry an approved analysis elsewhere in the book, for the suggestion pool; every sense-resolution outcome; every bare-word-analysis outcome | +| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; five phrase clusters (a plain two-word one, one ambiguous between two matching runs, one overlapping an approved run so it demotes to `candidate`, an excluded one, and a three-word one both languages gloss); word parses, including a one-morpheme one on a token the other language leaves alone; a word and a parse cluster sharing one range; verse hashes placed so each analysis shape has at least one approved - and therefore linked - instance, since only an approved link is applied to anything; one verse approved by `es` alone, so the phrase both languages gloss merges back down to `suggested`; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; two unanalyzed tokens whose forms carry an approved analysis elsewhere in the book, for the suggestion pool; every sense-resolution outcome; every bare-word-analysis outcome | | `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`); two phrases that drop, one matching no run of words and one whose form is blank, beside a phrase that anchors as a control, in an approved verse so it lands somewhere visible; plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | | `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | | `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | From ab33bc6e79baadc384b3ad6f1f5edce40d3d7f25 Mon Sep 17 00:00:00 2001 From: Danny Rorabaugh Date: Wed, 2 Sep 2026 15:31:21 -0400 Subject: [PATCH 8/8] Say what the invented verse text now carries Verse 3 gained two forms that recur with no cluster of their own, so an analysis approved elsewhere has somewhere to be suggested, and the sentence listing what the text was invented for did not mention them. Also names the gloss language the five phrase clusters are counted in, since `fr` carries a sixth cluster for one of the same phrases. Co-Authored-By: Claude Opus 5 (1M context) --- test-data/pt9-projects/README.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md index a55f06bd..ae1cf209 100644 --- a/test-data/pt9-projects/README.md +++ b/test-data/pt9-projects/README.md @@ -5,7 +5,8 @@ running platform. Each has one book (PHP) and a handful of verses, so an import and its report is small enough to read whole. The verse text is invented for the shapes the import has to handle: a form that repeats within a -verse, runs of words a phrase can span, and forms that divide into a stem and a suffix. +verse, runs of words a phrase can span, forms that divide into a stem and a suffix, and forms that +recur with no cluster of their own, so an analysis approved elsewhere has somewhere to be suggested. These complement [`../Pt9InterlinearProjectData.json`](../Pt9InterlinearProjectData.json), which is the served payload the converter's unit tests read directly. These are whole projects, for what no @@ -41,12 +42,12 @@ it. ## What each project covers -| Project | Purpose | Key features | -| ------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; five phrase clusters (a plain two-word one, one ambiguous between two matching runs, one overlapping an approved run so it demotes to `candidate`, an excluded one, and a three-word one both languages gloss); word parses, including a one-morpheme one on a token the other language leaves alone; a word and a parse cluster sharing one range; verse hashes placed so each analysis shape has at least one approved - and therefore linked - instance, since only an approved link is applied to anything; one verse approved by `es` alone, so the phrase both languages gloss merges back down to `suggested`; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; two unanalyzed tokens whose forms carry an approved analysis elsewhere in the book, for the suggestion pool; every sense-resolution outcome; every bare-word-analysis outcome | -| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`); two phrases that drop, one matching no run of words and one whose form is blank, beside a phrase that anchors as a control, in an approved verse so it lands somewhere visible; plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | -| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | -| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | +| Project | Purpose | Key features | +| ------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; five phrase clusters in `es` (a plain two-word one, one ambiguous between two matching runs, one overlapping an approved run so it demotes to `candidate`, an excluded one, and a three-word one both languages gloss); word parses, including a one-morpheme one on a token the other language leaves alone; a word and a parse cluster sharing one range; verse hashes placed so each analysis shape has at least one approved - and therefore linked - instance, since only an approved link is applied to anything; one verse approved by `es` alone, so the phrase both languages gloss merges back down to `suggested`; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; two unanalyzed tokens whose forms carry an approved analysis elsewhere in the book, for the suggestion pool; every sense-resolution outcome; every bare-word-analysis outcome | +| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`); two phrases that drop, one matching no run of words and one whose form is blank, beside a phrase that anchors as a control, in an approved verse so it lands somewhere visible; plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | +| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | +| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | For the remaining case, a project the import must refuse outright, use any project with no `Lexicon.xml`, no `WordAnalyses.xml`, and no `Interlinear_*` file. Its manifest is empty, which is