Fix inflated post-page reading time; dedupe post metadata logic - #2027
Merged
Merged
Conversation
- calculateReadingTime/WordCount now drop rendered footnotes
(<section data-footnotes>, data-footnote-ref links), script/style, and tag
markup. Post pages count rendered HTML, so footnotes and attributes were
counted there but not on cards: 96 of 165 carded posts disagreed, up to
60 vs 28 min. Now 17 differ, by at most 2 min. Also fixes JSON-LD wordCount.
- KeepReading uses getPostMetadata instead of reimplementing URL/date/time
- feed.xml and llms.txt use getPublishedPosts({ sorted: true })
- llms.txt: drop the unreachable excerpt branch (description is required)
and strip bio HTML with stripHtmlTags
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2024. Merge that one first; this PR's base will then retarget to
main.Reading time was about 2x too high on long post pages
The audit flagged that cards and post pages disagree on reading time. I measured this across the whole build: 96 of 165 carded posts disagree, by as much as 60 min on the page vs. 28 on the card (Towards a more agile government).
The page is the side that's wrong:
PostLayoutcounts the rendered HTML.stripFootnotesonly recognizes Markdown[^1]:syntax, so rendered footnote sections were counted in full.With footnotes excluded, which is what the util already intends, that post is about 30 min. That matches the card.
calculateReadingTimeandcalculateWordCountnow also drop:<section data-footnotes>anddata-footnote-reflinks<script>and<style>contentsAfter the fix, 17 posts differ, each by at most 2 min. The remainder comes from code blocks and URLs, which the Markdown and HTML paths count slightly differently. JSON-LD
wordCountwas inflated the same way and is fixed too.Dedupe
KeepReadingnow usesgetPostMetadatainstead of reimplementing URL, date, and reading time.feed.xmlandllms.txtnow usegetPublishedPosts({ sorted: true })instead of hand-rolled sorts.llms.txtis byte-identical.llms.txt:description, so that branch could never run.stripHtmlTags.Verification
npm run check: clean.🤖 Generated with Claude Code