Skip to content

feat: entities option for createSaxParser and XmlParseStream - #30

Merged
evoactivity merged 1 commit into
mainfrom
feat/sax-stream-entities
Oct 20, 2026
Merged

evoactivity merged 1 commit into
mainfrom
feat/sax-stream-entities

Conversation

@evoactivity

Copy link
Copy Markdown
Owner

createSaxParser() and XmlParseStream gain an entities option. With entities: true, text and attribute values are decoded the same way parse() decodes them. XML mode decodes the five XML entities plus numeric references, and html: true decodes the full HTML named set. CDATA, raw content (<script> and the like in HTML mode), comments, processing instructions and doctypes are left as written.

How it works

  • The internal saxEngine takes a decode function. Text is decoded after trimming, at the two non-raw text emit sites. Attribute values are decoded once per tag, just before onOpenTag fires, through setOwnProperty so the __proto__ guard still holds. Only strings containing & are passed to the decoder.
  • createSaxParser() and XmlParseStream resolve entities to decodeXML or decodeHTML from entities, matching parse().

Behaviour change in parse()

parse() used to decode text first and then judge whitespace on the decoded string. It now judges and trims whitespace on the source text and decodes afterwards, which is the order the SAX engine uses. Encoded whitespace now counts as text:

parse('<a> &#32;x </a>', { entities: true, trimWhitespace: true });
// before: ['x']
// after:  [' x']

With this, parse(xml, { entities: true, trimWhitespace: true }) produces the same tree as createSaxParser({ entities: true }) on every XML fixture, which the new parity test checks with the input in one chunk and in 7-character chunks. The separate decode branch in parse() is gone; the two existing text paths decode each kept string instead.

Performance

Median of five processes, each the median of nine runs, against a build of main:

main this branch, off this branch, on
SAX, catalog-10mb 31.0 ms 31.0 ms 34.2 ms
XmlParseStream, catalog-10mb 44.0 ms 44.1 ms 47.6 ms
SAX, 31 MB XMLTV file 115.8 ms 116.1 ms 130.6 ms
XmlParseStream, 31 MB XMLTV file 165.4 ms 165.0 ms 180.5 ms

With the option off there is no change. With it on the cost is similar to what parse() pays for decoding on the same file (173 to 190 ms).

Tests and docs

  • test/entities.test.ts covers the default, text and all attribute quoting styles, entities split across chunks, CDATA and raw content, XML and HTML entity sets, select, output: 'lossy', the trim order in all three APIs, and parity with parse().
  • docs/sax-parser.md, docs/web-streams.md, docs/html.md, docs/parsing.md and docs/limitations.md describe the option and the trim order.

Motivated by streaming XMLTV import in Ekto, where about 5.6% of programmes in a real feed carry entities.

🤖 Generated with Claude Code

Both streaming parsers take `entities: true` and decode text and
attribute values the same way parse() does, using the HTML entity set
in HTML mode. CDATA, raw content, comments, processing instructions
and doctypes are left as written.

Text is trimmed before it is decoded. parse() now follows the same
order, so encoded whitespace such as &#32; counts as text and is kept.
parse() with entities and trimWhitespace now produces the same text as
the streaming parsers.
@evoactivity evoactivity added the enhancement New feature or request label Oct 20, 2026
@evoactivity
evoactivity merged commit cfdb676 into main Oct 20, 2026
6 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant