Skip to content

test(docx): run the Word editing protocol over every document of the fidelity corpus - #847

Merged
DemchaAV merged 3 commits into
2.5-devfrom
test/docx-edit-protocol-corpus
Oct 4, 2026
Merged

DemchaAV merged 3 commits into
2.5-devfrom
test/docx-edit-protocol-corpus

Conversation

@DemchaAV

@DemchaAV DemchaAV commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Why

The DOCX export is held to the page line by line in Word and LibreOffice, but how a document edits was checked on one fixture only. edit-protocol-word.ps1 finds its targets by that fixture's text ("Lengthen this sentence…", "Storage growth reviewed"), so it cannot run on a CV, an invoice or a rota. An export can look exactly like the page and still fail the moment someone types in it, and nothing measured that across the corpus.

What changed

scripts/docx-visual/edit-protocol-corpus.ps1 runs Word's editing protocol over any exported DOCX. It chooses what to edit from the document itself:

  • the longest paragraph of body text, skipping frames and paragraphs holding a field, which Word refuses to cut in half;
  • the table with the most rows, nested tables included;
  • the first list;
  • the end of the document.

Each scenario edits a copy, saves it, reopens it from disk, and records what it can decide by itself:

Scenario Fails when
lengthen the paragraph about twofold the text is lost, the paragraph does not grow, it no longer ends above the block after it (after its row, in a table — a paragraph beside a taller cell grows inside its row), or it sits in a row of exact height
shorten it by half it stands taller, or the block after it comes lower
press Enter after it and type the new paragraph's letters are hidden, or not in the style, size and face of the letter typing continues from
restyle Normal by +2pt fewer than 90% of the paragraphs whose letters are at the body's size grow by those 2pt
add a table row, delete one the row count of the same table, or the table count, is off
continue a list with Enter the new paragraph is not a list item
type at the end, ten lines at a time, until a new page opens (at most 200 lines) the typed text went into a table cell, its letters are hidden, its exact lines are shorter than 1.1 times its size, or the page count does not grow; the footer the new page shows is recorded, not judged
save with no edit paragraphs, tables, characters or pages change

Enter and typing go through Word's Selection, as a person types. Word then carries the formatting forward from the caret; a script inserting text would set it instead. Letters' size, face and visibility are read from the text without its paragraph mark, which a range's font reports along with them. The data table is found again by its place among the document's tables, and a table with cells merged down its rows is passed over, since Word does not reach its rows one by one. Inputs are copied into the output directory before the first edit, so a build that exports the corpus again mid-run cannot remove them. The lengthened copy and the new-page copy are also written as PDFs, for a person to judge what the protocol cannot. CONTRIBUTING.md describes how to run it.

Verification

Ran on all 62 corpus documents of 2.5-dev's export in Word 16.0.20430, about a minute a document.

Scenario Pass
save with no edit 62 / 62
shorten a paragraph 62 / 62
add a table row 23 / 23
delete a table row 23 / 23
continue a list 14 / 14
lengthen a paragraph 62 / 62
restyle Normal 54 / 62
Enter and type 37 / 62
type at the end 20 / 62

The failures are the export's, and fixes follow separately:

  • Enter and type (25 documents). The new paragraph comes out in another size, for example 9pt after 10.5pt in NavySidebar and 14pt after 11pt in ClassicInvoice. Word continues a paragraph in its mark's formatting, and a paragraph written at an exact line height leaves its mark unstyled.
  • Type at the end (42 documents). Of these:
    • 33 end in a table, whose closing paragraph is a point tall with its mark hidden. The caret Word gives the end of the document lands in the table's last cell, and the typed lines run into it: CompactMono's is a narrow column, and the lines filled many more pages of it. Inserted at the very end without the caret, the same text goes into the hidden paragraph and does not show at all;
    • 9 end in a paragraph whose exact line is shorter than the size an unstyled mark gives what is typed (ClassicInvoice: 14pt letters on 9.25pt lines).
  • Restyle Normal (8 documents). The body text carries a direct size, so restyling Normal reaches none of it: ProposalEditorial, ProposalNorthline, CobaltRota, five invoices and VioletGrid.

Seen on the PDFs, not scored: drawings anchored to the page stay put when the text above them grows. NavySidebar's timeline icons ended up on the wrong lines after its summary was lengthened.

The script parses with PowerShell's own parser and has no tests of its own. Nothing in the build calls it.

Lane: test (scripts/docx-visual, CONTRIBUTING.md).

…graph mark, and keep each scenario to what the export decides
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant