Skip to content

fix(meta-fetcher): decode HTML 4 named entities - #401

Open
08abhinav wants to merge 1 commit into
codercops:devfrom
08abhinav:fix/meta-fetcher-html-entities
Open

08abhinav wants to merge 1 commit into
codercops:devfrom
08abhinav:fix/meta-fetcher-html-entities

Conversation

@08abhinav

Copy link
Copy Markdown
Contributor

Description

decodeEntities() in src/lib/meta-fetcher.ts only knew ten named entities, so common ones like ’, —, …, é and © reached the preview checker as raw text. That inflated title lengths (a 59-character title was reported as 71 chars), showed ’ and — on every platform card, made the Google preview cut titles too early, and passed raw entities to Smart Autofill.

This PR:

  • Adds src/lib/html-entities.ts with the 252 HTML 4 named entities (Latin-1, symbol and special sets), with characters taken from the current WHATWG list (⟨ and ⟩ are U+27E8 and U+27E9). apos and the four uppercase names (AMP, LT, GT, QUOT) are kept.
  • Replaces the NAMED_ENTITIES table in meta-fetcher.ts with that import, keeping the case-sensitive exact lookup and the Object.hasOwn check from fix(meta-fetcher): ignore Object prototype properties when decoding named entities #391.
  • Changes the name pattern from [a-z]+ to [a-z][a-z0-9]* so names with digits (½, ²) decode. The numeric entity branch is unchanged.
  • Unknown names, wrong-case names (&Amp;,  ) and prototype names (&constructor;) are still left as they are.

Closes #396

How I Tested This

  • Added tests in tests/lib/meta-fetcher.test.ts:
    • the issue's example title now decodes to Don’t ship on Fridays — notes from 200 releases | Acme Blog and analyzeMeta() no longer reports a title-length warning
    • typographic, accented, symbol and digit-containing entities (…, é, É, ©, ™, €, ½, ², ⟨, ⟩) in attribute values
    • every entry in the table decodes in both <title> and a <meta> attribute value
    • the table has 257 entries (252 HTML 4 names + apos + 4 uppercase)
    • case-sensitivity (&EACUTE; and &Frac12; stay raw)
  • Confirmed the new tests fail on the old code (stashed my meta-fetcher.ts change) and pass with the fix.
  • Compared every name and character in html-entities.ts against https://html.spec.whatwg.org/entities.json with a temporary script (not committed). <result: e.g. "no mismatches">
  • Before the fix, the issue's parseMetaTags example returned Don&rsquo;t ship on Fridays &mdash; notes from 200 releases | Acme Blog. After, it returns Don’t ship on Fridays — notes from 200 releases | Acme Blog.
  • Ran npm run check, npm run test and npm run build: <paste result, e.g. "all pass">

AI Assistance

Claude helped me with a first draft of html-entities.ts and the tests, and debugging. I reviewed the code, verified the entity table against the WHATWG entities.json, and ran the tests and checks myself.

Type of Change

  • Bug fix
  • New template
  • New feature
  • Documentation
  • Tests
  • Refactoring
  • Performance improvement
  • Other (describe below)

Checklist

  • This PR targets dev
  • I was assigned the linked issue
  • npm run check passes
  • npm run test passes
  • For a bug fix: the new test fails without my change
  • Tested locally with npm run dev
  • Screenshots included for visual changes
  • New templates follow the Template Guidelines

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preview checker: named entities like &rsquo; and &mdash; show up as raw text

1 participant