Skip to content

fix(staging): validate encodings before truncating export files - #4835

Draft
Afloat16 wants to merge 1 commit into
Unstructured-IO:mainfrom
Afloat16:Afloat16/staging-encoding-preflight-20261121
Draft

Afloat16 wants to merge 1 commit into
Unstructured-IO:mainfrom
Afloat16:Afloat16/staging-encoding-preflight-20261121

Conversation

@Afloat16

@Afloat16 Afloat16 commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Export helpers currently open destinations in write mode before discovering an invalid encoding or an unencodable character. The exception therefore leaves an existing document empty.

Validate encoding incrementally before opening the destination, then keep the existing text-mode write. This protects all staging file exporters against encoding failures with bounded temporary encoded buffers. Operating-system write failures still propagate; this does not claim transactional filesystem writes.

Validation

Seventeen regression cases fail on main; 21 encoding controls pass before and after. All 123 new and staging-base tests pass. Cases cover an unknown codec, a non-ASCII character beyond the first preflight block, existing/new paths, UTF-16 and a stateful Japanese codec. The positional-argument test now uses a Latin-1-encodable multiplication sign, since its previous mocked write concealed a real encoding error. Whole-repository Ruff 0.15.10 lint and format checks pass. Full repository tests remain pending, so this is a draft.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant