diff --git a/README.md b/README.md index 5197d410..62956349 100644 --- a/README.md +++ b/README.md @@ -123,7 +123,7 @@ The general file structure for an extension is as follows: - `assets/descriptions/description-.md` contains a brief description of the extension in the language specified by `` - `contributions/` contains JSON files the platform uses to extend data structures for things like menus and settings. The JSON files are referenced from the manifest - `public/` contains other static files that are copied into the build folder -- `test-data/` contains sample interlinear XML (e.g. `Interlinear_en_MAT.xml`) for development and tests +- `test-data/` contains sample Paratext 9 interlinear data for development and tests: `Pt9InterlinearProjectData.json` is the served payload the converter's tests read, and `test-data/pt9-projects/` holds whole Paratext 9 projects for testing an import by hand (see its [README](test-data/pt9-projects/README.md)) - `.github/` contains files to facilitate integration with GitHub - `.github/workflows` contains [GitHub Actions](https://github.com/features/actions) workflows for automating various processes in this repo (e.g. **Test** and **Lint** on push/PR to main, release-prep, hotfix-\*; **Publish** and **Bump Versions** manual dispatch; **CodeQL** for security) - `.github/assets/release-body.md` is the template for the body of [releases published using GitHub Actions](#publishing). The Publish workflow substitutes its version placeholders into a copy outside the repo, and that copy plus a generated changelog becomes the release body diff --git a/test-data/pt9-projects/PIA/51PHPPIA.SFM b/test-data/pt9-projects/PIA/51PHPPIA.SFM new file mode 100644 index 00000000..21f7c674 --- /dev/null +++ b/test-data/pt9-projects/PIA/51PHPPIA.SFM @@ -0,0 +1,12 @@ +\id PHP - PT9 Import A +\h PIA +\mt1 Philippians +\c 1 +\p +\v 1 Wug blicket wug dax fep gorp. +\v 2 Tomo zav flib. +\v 3 Wugs daxes glorp zav flib. +\v 4 Zim zam zim zam kex. +\v 5 Nol pim tuk vex. +\v 6 Snee plovs vex plovs. +\v 7 Plovs snee vex plovs. diff --git a/test-data/pt9-projects/PIA/InterlinearSetup.xml b/test-data/pt9-projects/PIA/InterlinearSetup.xml new file mode 100644 index 00000000..270769ec --- /dev/null +++ b/test-data/pt9-projects/PIA/InterlinearSetup.xml @@ -0,0 +1,15 @@ + + + + 12 + false + false + false + + + 12 + false + false + false + + diff --git a/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml new file mode 100644 index 00000000..5aab570a --- /dev/null +++ b/test-data/pt9-projects/PIA/Interlinear_es/Interlinear_es_PHP.xml @@ -0,0 +1,64 @@ + + + + + PHP 1:1 + + + + + + + + + + PHP 1:2 + + true + + + .; + + + + PHP 1:3 + + + + + + + + + PHP 1:4 + + + + true + + + + PHP 1:5 + + + + + + PHP 1:6 + + + + + + + + + PHP 1:7 + + + + + + + + diff --git a/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml new file mode 100644 index 00000000..4c512ab2 --- /dev/null +++ b/test-data/pt9-projects/PIA/Interlinear_fr/Interlinear_fr_PHP.xml @@ -0,0 +1,30 @@ + + + + + PHP 1:1 + + + + + + + PHP 1:3 + + + + + + PHP 1:4 + + + + + + PHP 1:5 + + + + + + diff --git a/test-data/pt9-projects/PIA/Lexicon.xml b/test-data/pt9-projects/PIA/Lexicon.xml new file mode 100644 index 00000000..3c1c9ff8 --- /dev/null +++ b/test-data/pt9-projects/PIA/Lexicon.xml @@ -0,0 +1,174 @@ + + + en + Charis SIL + 12 + + + flibs + + + + + + + + + + + + uno + un + + + primero + + + + + + + + dos + + + deux + + + + + + + + tres cuatro + + + + + + + + nueve diez + + + + + + + + diez nueve + + + + + + + + once doce trece + + + onze douze treize + + + + + + + + cinco + + + + + + + + seis + + + + + + + + siete + + + + + + + + ocho + + + + + + + + catorce + + + decimocuarto + + + + + + + + wug + + + + + + + + -s + + + + + + + + dax + + + + + + + + -es + + + + + + + + quinze + + + + + + + + glorp + + + + + + + + plov + + + + + diff --git a/test-data/pt9-projects/PIA/Settings.xml b/test-data/pt9-projects/PIA/Settings.xml new file mode 100644 index 00000000..63addbd7 --- /dev/null +++ b/test-data/pt9-projects/PIA/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import A - clean + 65001 + T + + NFC + PIA + a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1 + Charis SIL + 12 + + + en::: + 41MAT + + PIA.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PIA/WordAnalyses.xml b/test-data/pt9-projects/PIA/WordAnalyses.xml new file mode 100644 index 00000000..4d089f40 --- /dev/null +++ b/test-data/pt9-projects/PIA/WordAnalyses.xml @@ -0,0 +1,15 @@ + + + + Stem:daxSuffix:es + + + Stem:fepSuffix:ped + + + brokenid + + + + + diff --git a/test-data/pt9-projects/PIA/en.ldml b/test-data/pt9-projects/PIA/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PIA/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PIA/unique.id b/test-data/pt9-projects/PIA/unique.id new file mode 100644 index 00000000..4fe808d8 --- /dev/null +++ b/test-data/pt9-projects/PIA/unique.id @@ -0,0 +1 @@ +aaaaaaaa-1111-4111-8111-aaaaaaaaaaaa diff --git a/test-data/pt9-projects/PIB/51PHPPIB.SFM b/test-data/pt9-projects/PIB/51PHPPIB.SFM new file mode 100644 index 00000000..517df09f --- /dev/null +++ b/test-data/pt9-projects/PIB/51PHPPIB.SFM @@ -0,0 +1,6 @@ +\id PHP - PT9 Import B +\h PIB +\mt1 Philippians +\c 1 +\p +\v 1 Alpha beta gamma delta. diff --git a/test-data/pt9-projects/PIB/InterlinearSetup.xml b/test-data/pt9-projects/PIB/InterlinearSetup.xml new file mode 100644 index 00000000..fe146674 --- /dev/null +++ b/test-data/pt9-projects/PIB/InterlinearSetup.xml @@ -0,0 +1,9 @@ + + + + 12 + false + false + false + + diff --git a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_JAS.xml b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_JAS.xml new file mode 100644 index 00000000..80897785 --- /dev/null +++ b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_JAS.xml @@ -0,0 +1,11 @@ + + + + + JAS 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml new file mode 100644 index 00000000..fa14f7f5 --- /dev/null +++ b/test-data/pt9-projects/PIB/Interlinear_es/Interlinear_es_PHP.xml @@ -0,0 +1,27 @@ + + + + + PHP 1:1 + + + + + + + + + + + + + + + + PHP 2:5 + + + + + + diff --git a/test-data/pt9-projects/PIB/Lexicon.xml b/test-data/pt9-projects/PIB/Lexicon.xml new file mode 100644 index 00000000..b73ab02d --- /dev/null +++ b/test-data/pt9-projects/PIB/Lexicon.xml @@ -0,0 +1,41 @@ + + + en + Charis SIL + 12 + + + + + + + alfa + + + + + + + + beta + + + + + + + + delta + + + + + + + + gama delta + + + + + diff --git a/test-data/pt9-projects/PIB/Settings.xml b/test-data/pt9-projects/PIB/Settings.xml new file mode 100644 index 00000000..eaf6e0be --- /dev/null +++ b/test-data/pt9-projects/PIB/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import B - drops + 65001 + T + + NFC + PIB + b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2 + Charis SIL + 12 + + + en::: + 41MAT + + PIB.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PIB/en.ldml b/test-data/pt9-projects/PIB/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PIB/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PIB/unique.id b/test-data/pt9-projects/PIB/unique.id new file mode 100644 index 00000000..dc9b6f71 --- /dev/null +++ b/test-data/pt9-projects/PIB/unique.id @@ -0,0 +1 @@ +bbbbbbbb-2222-4222-8222-bbbbbbbbbbbb diff --git a/test-data/pt9-projects/PIC/51PHPPIC.SFM b/test-data/pt9-projects/PIC/51PHPPIC.SFM new file mode 100644 index 00000000..29f45944 --- /dev/null +++ b/test-data/pt9-projects/PIC/51PHPPIC.SFM @@ -0,0 +1,6 @@ +\id PHP - PT9 Import C +\h PIC +\mt1 Philippians +\c 1 +\p +\v 1 Uno dos tres. diff --git a/test-data/pt9-projects/PIC/Interlinear_English/Interlinear_English_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_English/Interlinear_English_PHP.xml new file mode 100644 index 00000000..78fb4c7c --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_English/Interlinear_English_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_es-MX/Interlinear_es-MX_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_es-MX/Interlinear_es-MX_PHP.xml new file mode 100644 index 00000000..673d720c --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_es-MX/Interlinear_es-MX_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_es/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_es/Interlinear_es_PHP.xml new file mode 100644 index 00000000..819b769a --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_es/Interlinear_es_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_es_PHP.xml b/test-data/pt9-projects/PIC/Interlinear_es_PHP.xml new file mode 100644 index 00000000..c49c9e65 --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_es_PHP.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_esmx_alt.xml b/test-data/pt9-projects/PIC/Interlinear_esmx_alt.xml new file mode 100644 index 00000000..3209f124 --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_esmx_alt.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_nobookid.xml b/test-data/pt9-projects/PIC/Interlinear_nobookid.xml new file mode 100644 index 00000000..947ac9ab --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_nobookid.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Interlinear_noglosslang.xml b/test-data/pt9-projects/PIC/Interlinear_noglosslang.xml new file mode 100644 index 00000000..d858c6c9 --- /dev/null +++ b/test-data/pt9-projects/PIC/Interlinear_noglosslang.xml @@ -0,0 +1,11 @@ + + + + + PHP 1:1 + + + + + + diff --git a/test-data/pt9-projects/PIC/Lexicon.xml b/test-data/pt9-projects/PIC/Lexicon.xml new file mode 100644 index 00000000..37e794fd --- /dev/null +++ b/test-data/pt9-projects/PIC/Lexicon.xml @@ -0,0 +1,33 @@ + + + en + Charis SIL + 12 + + + + + + + uno + + + + + + + + dos + + + + + + + + tres + + + + + diff --git a/test-data/pt9-projects/PIC/Settings.xml b/test-data/pt9-projects/PIC/Settings.xml new file mode 100644 index 00000000..b2def3a0 --- /dev/null +++ b/test-data/pt9-projects/PIC/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import C - identity + 65001 + T + + NFC + PIC + c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3 + Charis SIL + 12 + + + en::: + 41MAT + + PIC.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PIC/en.ldml b/test-data/pt9-projects/PIC/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PIC/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PIC/unique.id b/test-data/pt9-projects/PIC/unique.id new file mode 100644 index 00000000..e0364517 --- /dev/null +++ b/test-data/pt9-projects/PIC/unique.id @@ -0,0 +1 @@ +cccccccc-3333-4333-8333-cccccccccccc diff --git a/test-data/pt9-projects/PID/51PHPPID.SFM b/test-data/pt9-projects/PID/51PHPPID.SFM new file mode 100644 index 00000000..8618c146 --- /dev/null +++ b/test-data/pt9-projects/PID/51PHPPID.SFM @@ -0,0 +1,6 @@ +\id PHP - PT9 Import D +\h PID +\mt1 Philippians +\c 1 +\p +\v 1 Solo texto aqui. diff --git a/test-data/pt9-projects/PID/Lexicon.xml b/test-data/pt9-projects/PID/Lexicon.xml new file mode 100644 index 00000000..62104c39 --- /dev/null +++ b/test-data/pt9-projects/PID/Lexicon.xml @@ -0,0 +1,25 @@ + + + en + Charis SIL + 12 + + + texto + + + + + + + + + + + + solo + + + + + diff --git a/test-data/pt9-projects/PID/Settings.xml b/test-data/pt9-projects/PID/Settings.xml new file mode 100644 index 00000000..77ef6ac7 --- /dev/null +++ b/test-data/pt9-projects/PID/Settings.xml @@ -0,0 +1,31 @@ + + usfm.sty + 4 + English + 8.0.100.76 + PT9 Import D - lexicon only + 65001 + T + + NFC + PID + d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4 + Charis SIL + 12 + + + en::: + 41MAT + + PID.SFM + Major::BiblicalTerms.xml + F + F + F + Test + Standard:: + + 3 + 000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000 + + diff --git a/test-data/pt9-projects/PID/WordAnalyses.xml b/test-data/pt9-projects/PID/WordAnalyses.xml new file mode 100644 index 00000000..35c076c2 --- /dev/null +++ b/test-data/pt9-projects/PID/WordAnalyses.xml @@ -0,0 +1,6 @@ + + + + Stem:aquSuffix:i + + diff --git a/test-data/pt9-projects/PID/en.ldml b/test-data/pt9-projects/PID/en.ldml new file mode 100644 index 00000000..3cc2aeaf --- /dev/null +++ b/test-data/pt9-projects/PID/en.ldml @@ -0,0 +1 @@ +left-to-right diff --git a/test-data/pt9-projects/PID/unique.id b/test-data/pt9-projects/PID/unique.id new file mode 100644 index 00000000..830751f9 --- /dev/null +++ b/test-data/pt9-projects/PID/unique.id @@ -0,0 +1 @@ +dddddddd-4444-4444-8444-dddddddddddd diff --git a/test-data/pt9-projects/README.md b/test-data/pt9-projects/README.md new file mode 100644 index 00000000..86dc0902 --- /dev/null +++ b/test-data/pt9-projects/README.md @@ -0,0 +1,58 @@ +# Paratext 9 interlinear test projects + +Four minimal Paratext 9 projects for exercising the Paratext 9 interlinear import against a real +running platform. Each has one book (PHP) and a handful of verses, so an import finishes immediately +and its report is small enough to read whole. + +The verse text is invented for the shapes the import has to handle: a form that repeats within a +verse, runs of words a phrase can span, forms that divide into a stem and a suffix, forms that recur +with no cluster of their own, so an analysis approved elsewhere has somewhere to be suggested, and +one form approved four times over, so its pool bucket holds competing payloads. + +Cluster `Range` values are character offsets into the verse text, so changing a verse means +recomputing every range at or after the change, in every gloss language for that book. + +These complement [`../Pt9InterlinearProjectData.json`](../Pt9InterlinearProjectData.json), which is +the served payload the converter's unit tests read directly. These are whole projects, for what no +unit test covers: + +- the platform reading the files; +- the import service persisting the result; +- the WebView presenting it. + +The XML schema they follow is documented in +[`src/parsers/pt9/pt9-xml.md`](../../src/parsers/pt9/pt9-xml.md). + +## Using them + +1. Copy the project folders into the Platform.Bible project root. + + - On Windows: `%USERPROFILE%\.platform.bible\projects\Paratext 9 Projects\`. + - Elsewhere: `~/.platform.bible/projects/Paratext 9 Projects/`. + - Alternatively point `PLATFORM_BIBLE_PROJECT_ROOT_FOLDER` at a folder holding them, which leaves + your own projects untouched. + +2. Start Platform.Bible with the extension (`npm start`). +3. Open the Interlinearizer on one of the projects and use **Import from Paratext 9**, or invoke + `interlinearizer.importPt9Project` with the project's ID directly. + +The platform serves this data through the `platformScripture.Pt9Interlinear` projectInterface, which +is advertised only on unpublished (editable) projects. So a project copied here must load as an +ordinary editable project, not as a resource. + +Project GUIDs and `unique.id` values are fixed rather than generated, so re-copying a project after +deleting it reuses the same Platform.Bible project ID and any interlinearizer state still keyed to +it. + +## What each project covers + +| Project | Purpose | Key features | +| ------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `PIA` | Happy path | Two gloss languages (`es`, `fr`) over one book, so records merge across languages and one token's parses conflict; five phrase clusters in `es` (a plain two-word one, one ambiguous between two matching runs, one overlapping an approved run so it demotes to `candidate`, an excluded one, and a three-word one both languages gloss); word parses, including a one-morpheme one on a token the other language leaves alone; a word and a parse cluster sharing one range; verse hashes placed so each analysis shape has at least one approved - and therefore linked - instance, since only an approved link is applied to anything; one verse approved by `es` alone, so the phrase both languages gloss merges back down to `suggested`; an excluded cluster; a punctuation entry; a repeated surface form, for an ambiguous anchor; two unanalyzed tokens whose forms carry an approved analysis elsewhere in the book, for the suggestion pool; one form (`plovs`) approved four times across two verses, once sentence-initially so only case folding files it under the same pool key, giving that key four competing payloads - two alike in gloss and morpheme breakdown, one with that gloss and no breakdown, one with that breakdown under a different gloss; every sense-resolution outcome; every bare-word-analysis outcome | +| `PIB` | Cluster drops | All five `clusterDrops` reasons (`formMismatch`, `duplicateCluster`, `lemmaOrOther`, `unparseableLexemeId`, `verseNotFound`); two phrases that drop, one matching no run of words and one whose form is blank, beside a phrase that anchors as a control, in an approved verse so it lands somewhere visible; plus an `Interlinear_es_JAS.xml` for a book the project has no text for, which reports `bookFound: false` | +| `PIC` | File identity | A canonical book file beside a non-canonical twin of it (`booksDroppedAsDuplicates`, and the canonical file's data is the data that must survive); one file with no `GlossLanguage` and one with no `BookId` (`booksMissingIdentity`); `GlossLanguage="English"` (`tagIsFallback`); `es-MX` and `es-mx`, which stay separate language groups but resolve onto one tag (`sameTagCollisions`) | +| `PID` | Lexicon only | A `Lexicon.xml` and `WordAnalyses.xml` with no interlinear book file at all: the manifest is non-empty, so the import is offered and succeeds, but the conversion has no book to report on and `report.languages` comes back empty | + +For the remaining case, a project the import must refuse outright, use any project with no +`Lexicon.xml`, no `WordAnalyses.xml`, and no `Interlinear_*` file. Its manifest is empty, which is +what makes the import throw rather than create anything.