Repository navigation
feat: entities option for createSaxParser and XmlParseStream - #30
Merged
Merged
Conversation
Both streaming parsers take `entities: true` and decode text and attribute values the same way parse() does, using the HTML entity set in HTML mode. CDATA, raw content, comments, processing instructions and doctypes are left as written. Text is trimmed before it is decoded. parse() now follows the same order, so encoded whitespace such as   counts as text and is kept. parse() with entities and trimWhitespace now produces the same text as the streaming parsers.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
createSaxParser()andXmlParseStreamgain anentitiesoption. Withentities: true, text and attribute values are decoded the same wayparse()decodes them. XML mode decodes the five XML entities plus numeric references, andhtml: truedecodes the full HTML named set. CDATA, raw content (<script>and the like in HTML mode), comments, processing instructions and doctypes are left as written.How it works
saxEnginetakes adecodefunction. Text is decoded after trimming, at the two non-raw text emit sites. Attribute values are decoded once per tag, just beforeonOpenTagfires, throughsetOwnPropertyso the__proto__guard still holds. Only strings containing&are passed to the decoder.createSaxParser()andXmlParseStreamresolveentitiestodecodeXMLordecodeHTMLfromentities, matchingparse().Behaviour change in parse()
parse()used to decode text first and then judge whitespace on the decoded string. It now judges and trims whitespace on the source text and decodes afterwards, which is the order the SAX engine uses. Encoded whitespace now counts as text:With this,
parse(xml, { entities: true, trimWhitespace: true })produces the same tree ascreateSaxParser({ entities: true })on every XML fixture, which the new parity test checks with the input in one chunk and in 7-character chunks. The separate decode branch inparse()is gone; the two existing text paths decode each kept string instead.Performance
Median of five processes, each the median of nine runs, against a build of
main:With the option off there is no change. With it on the cost is similar to what
parse()pays for decoding on the same file (173 to 190 ms).Tests and docs
test/entities.test.tscovers the default, text and all attribute quoting styles, entities split across chunks, CDATA and raw content, XML and HTML entity sets,select,output: 'lossy', the trim order in all three APIs, and parity withparse().docs/sax-parser.md,docs/web-streams.md,docs/html.md,docs/parsing.mdanddocs/limitations.mddescribe the option and the trim order.Motivated by streaming XMLTV import in Ekto, where about 5.6% of programmes in a real feed carry entities.
🤖 Generated with Claude Code