Problem
Segment boundaries are persisted as a SegmentationDelta anchored to token refs of the form "<verse SID>:<charStart>" (e.g. "PSA 3:2:14"). On load, resegmentBook → effectiveStarts/normalize (src/utils/segmentation.ts) silently drop any anchor whose ref isn't found in the freshly-tokenized book, reverting that region to the default one-segment-per-verse segmentation.
This is the intended, non-destructive degradation — but it's silent. When the source text is reversified (classically the Psalms verse-0↔verse-1 superscription shift, which renumbers SIDs) or edited upstream (shifting char offsets), a user's hand-placed boundaries vanish with no signal. They lose work and may not notice.
Decisions made by Alex (up for debate)
Detect-and-warn — not remap. Reframed as a general "lost boundaries" safety net (reversification is just one trigger; ordinary edits are the same event to an anchor):
- Keep the existing silent drop (non-destructive). Add a warning.
- Detection signal: any non-empty
SegmentationDelta that loses anchors on load. Do not try to isolate reversification from ordinary edits — this deliberately sidesteps the host's versificationStr, which is documented as unreliable (InterlinearNavContext.tsx:51). A project with default/empty segmentation never warns.
- Do NOT prune the dead anchors from storage. Keeping them preserves the model's revival property: if the source reverts and the token ref reappears, the boundary comes back on its own.
- Explicitly out of scope: remapping boundaries across versifications, and any versification-mapping API. That's a much larger effort with poor cost/benefit for a rare event; not this issue.
Implementation notes (from a prior investigation)
- Set-difference site: the
book memo in src/components/InterlinearizerLoader.tsx (~line 272) is where the freshly-tokenized verseBook and the delta (draft?.segmentation) are both in scope. Detection must run in a useEffect, not the pure book memo (notifications are a side effect).
- Add a new pure helper (e.g.
lostAnchors(verseBook, delta): string[]) that intersects [...removedVerseStarts, ...addedStarts] against the book's token refs and returns the missing ones. Do not make effectiveStarts/normalize report drops — keep segmentation.ts pure. (Note: bookLookups in that file already computes an all token-ref set you can mirror.)
- Delivery:
papi.notifications.send({ message, severity: 'warning' }) (PlatformNotification; Severity = 'info' | 'warning' | 'error'). No frontend code sends notifications yet, but the mock already exposes papi.notifications.send.
- Copy: generic — "source text changed," not "versification changed" (we can't reliably attribute the cause). Optionally include a count. Add a key to
contributions/localizedStrings.json, resolve via useLocalizedStrings, interpolate the count with .replace('{count}', () => …) (pattern in MorphemeBox.tsx / TokenChip.tsx).
- Dedup: the extension re-tokenizes on many events (duplicate GetText re-fetches), so a naive per-load warning would spam.
useInterlinearizerBookData returns a reference-stable verseBook across duplicate USJ payloads (it dedupes by JSON), making the verseBook reference a natural dedup anchor (track the last-warned one in a useRef).
⚠️ Open decision — resolve before implementing
Dedup semantics were not settled:
- (A) Per-tokenization: warn again on each genuinely new re-tokenization that still has lost anchors; duplicate re-fetches suppressed via the reference-stable
verseBook. (Better matches "warn until something meaningful changes"; recommended in the prior investigation.)
- (B) Strict once-per-session: a
projectId+book Set that never re-warns for that book again this session — even across a reversify → revert → reversify cycle.
Pick one before building.
Acceptance / tests
- A lossy non-empty delta → warning sent (positive).
- A delta whose anchors all still resolve → no warning (negative).
- Default/empty segmentation → no warning (negative).
- Repeat render with the same
verseBook → no repeat warning (dedup).
- Dead anchors are not pruned — the persisted/draft segmentation is untouched by the warning path.
100% coverage enforced; npm run lint-fix at the end.
`
Problem
Segment boundaries are persisted as a
SegmentationDeltaanchored to token refs of the form"<verse SID>:<charStart>"(e.g."PSA 3:2:14"). On load,resegmentBook→effectiveStarts/normalize(src/utils/segmentation.ts) silently drop any anchor whose ref isn't found in the freshly-tokenized book, reverting that region to the default one-segment-per-verse segmentation.This is the intended, non-destructive degradation — but it's silent. When the source text is reversified (classically the Psalms verse-0↔verse-1 superscription shift, which renumbers SIDs) or edited upstream (shifting char offsets), a user's hand-placed boundaries vanish with no signal. They lose work and may not notice.
Decisions made by Alex (up for debate)
Detect-and-warn — not remap. Reframed as a general "lost boundaries" safety net (reversification is just one trigger; ordinary edits are the same event to an anchor):
SegmentationDeltathat loses anchors on load. Do not try to isolate reversification from ordinary edits — this deliberately sidesteps the host'sversificationStr, which is documented as unreliable (InterlinearNavContext.tsx:51). A project with default/empty segmentation never warns.Implementation notes (from a prior investigation)
bookmemo insrc/components/InterlinearizerLoader.tsx(~line 272) is where the freshly-tokenizedverseBookand the delta (draft?.segmentation) are both in scope. Detection must run in auseEffect, not the purebookmemo (notifications are a side effect).lostAnchors(verseBook, delta): string[]) that intersects[...removedVerseStarts, ...addedStarts]against the book's token refs and returns the missing ones. Do not makeeffectiveStarts/normalizereport drops — keepsegmentation.tspure. (Note:bookLookupsin that file already computes analltoken-ref set you can mirror.)papi.notifications.send({ message, severity: 'warning' })(PlatformNotification;Severity = 'info' | 'warning' | 'error'). No frontend code sends notifications yet, but the mock already exposespapi.notifications.send.contributions/localizedStrings.json, resolve viauseLocalizedStrings, interpolate the count with.replace('{count}', () => …)(pattern inMorphemeBox.tsx/TokenChip.tsx).useInterlinearizerBookDatareturns a reference-stableverseBookacross duplicate USJ payloads (it dedupes by JSON), making theverseBookreference a natural dedup anchor (track the last-warned one in auseRef).Dedup semantics were not settled:
verseBook. (Better matches "warn until something meaningful changes"; recommended in the prior investigation.)projectId+bookSetthat never re-warns for that book again this session — even across a reversify → revert → reversify cycle.Pick one before building.
Acceptance / tests
verseBook→ no repeat warning (dedup).100% coverage enforced;
npm run lint-fixat the end.`