Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
6048bda
fix(extension-table): escape pipe characters in serialized table cells
lazerg Sep 1, 2026
aa5307a
refactor(extension-table): clarify pipe escaping helper name
lazerg Sep 1, 2026
504260b
perf(extension-list): stop scanning the whole document for every orde…
solomonaustin Sep 3, 2026
2812907
chore: format changeset frontmatter so vp check passes
solomonaustin Sep 3, 2026
399dc72
refactor(extension-list): address review on the ordered list scan
solomonaustin Sep 4, 2026
5a744de
Merge remote-tracking branch 'origin/main' into perf/ordered-list-lin…
solomonaustin Sep 5, 2026
c4480e8
Merge remote-tracking branch 'upstream/main' into fix/issue-8294-tabl…
lazerg Sep 6, 2026
cd7d7ab
fix(extension-table): stop leaking U+001F into table markdown
cursoragent Sep 6, 2026
7c8ad87
fix(markdown): choose an inline code fence the content cannot close
vingt-douze Sep 2, 2026
a90921e
fix(core): decode numeric HTML entities when parsing markdown
HwangRock Sep 20, 2026
c828d74
fix(core): decode HTML entities in a single pass
HwangRock Sep 21, 2026
916d135
fix(extension-table): keep empty cell paragraphs
Yahiro025 Sep 21, 2026
5c45a86
fix(core): make numeric HTML entity decoding CommonMark-conformant
HwangRock Sep 21, 2026
c9442ef
Merge pull request #8319 from Yahiro025/cursor/fix-table-markdown-sep…
arnaugomez Sep 22, 2026
3219c50
Merge pull request #8358 from HwangRock/fix/decode-numeric-html-entities
arnaugomez Sep 22, 2026
d5206a4
Merge branch 'main' into perf/ordered-list-linear-markdown-scan
arnaugomez Sep 22, 2026
508ed0c
Merge pull request #8302 from solomonaustin/perf/ordered-list-linear-…
arnaugomez Sep 22, 2026
fe63256
Merge pull request #8299 from vingt-douze/fix/code-span-backtick-fence
arnaugomez Sep 22, 2026
d50a45c
Merge main into fix/issue-8294-table-cell-pipe-escape
arnaugomez Sep 22, 2026
f878c05
Merge pull request #8295 from lazerg/fix/issue-8294-table-cell-pipe-e…
arnaugomez Sep 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/2026-09-02-fix-code-span-backtick-fence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@tiptap/markdown': patch
'@tiptap/extension-code': patch
---

Fix Markdown serialization of inline code containing backticks
5 changes: 5 additions & 0 deletions .changeset/2026-09-02-table-cell-pipe-escape.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tiptap/extension-table': patch
---

Table cells exported to Markdown now escape literal pipe characters, so the cell content survives when the output is read back.
5 changes: 5 additions & 0 deletions .changeset/2026-09-03-ordered-list-markdown-linear-scan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tiptap/extension-list': patch
---

Parsing large Markdown documents that contain ordered lists is much faster. A 578KB document went from about 19s to about 160ms.
5 changes: 5 additions & 0 deletions .changeset/2026-09-06-table-markdown-cell-separator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tiptap/extension-table': patch
---

Multi-block table cells no longer leak a U+001F control character into Markdown.
5 changes: 5 additions & 0 deletions .changeset/2026-09-20-decode-numeric-html-entities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tiptap/core': patch
---

Numeric HTML entities (`'`, `'`) are now decoded when parsing markdown, instead of showing up as literal text in the editor.
106 changes: 105 additions & 1 deletion packages/core/src/utilities/htmlEntities.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,110 @@ describe('decodeHtmlEntities', () => {
expect(decodeHtmlEntities('<')).toBe('<')
})

it('decodes decimal numeric entities like '', () => {
expect(decodeHtmlEntities('it's')).toBe("it's")
})

it('decodes hex numeric entities like '', () => {
expect(decodeHtmlEntities(''')).toBe("'")
})

it('decodes decimal numeric entities for non-ASCII characters like —', () => {
expect(decodeHtmlEntities('—')).toBe('\u2014')
})

it('handles doubly-encoded numeric entities like '', () => {
expect(decodeHtmlEntities(''')).toBe(''')
})

it('leaves decimal references over 7 digits as a literal (exceeds CommonMark digit limit)', () => {
expect(decodeHtmlEntities('�')).toBe('�')
})

it('leaves hex references over 6 digits as a literal (exceeds CommonMark digit limit)', () => {
expect(decodeHtmlEntities('�')).toBe('�')
})

it('leaves an 8-digit decimal reference as a literal (format exceeds the limit)', () => {
expect(decodeHtmlEntities('�')).toBe('�')
})

it('replaces a 7-digit decimal reference whose value exceeds 0x10FFFF with U+FFFD', () => {
expect(decodeHtmlEntities('�')).toBe('\uFFFD')
})

it('replaces a 6-digit hex reference whose value exceeds 0x10FFFF with U+FFFD', () => {
expect(decodeHtmlEntities('�')).toBe('\uFFFD')
})

it('matches a 7-digit decimal reference greedily and leaves the trailing digit as a literal', () => {
expect(decodeHtmlEntities('�8')).toBe('\uFFFD8')
})

it('replaces � (NUL) with U+FFFD', () => {
expect(decodeHtmlEntities('�')).toBe('\uFFFD')
})

it('replaces a lone low surrogate � with U+FFFD', () => {
expect(decodeHtmlEntities('�')).toBe('\uFFFD')
})

it('replaces a lone high surrogate � with U+FFFD', () => {
expect(decodeHtmlEntities('�')).toBe('\uFFFD')
})

it('decodes ퟿ normally (just below the surrogate range)', () => {
expect(decodeHtmlEntities('퟿')).toBe(String.fromCodePoint(0xd7ff))
})

it('decodes  normally (just above the surrogate range)', () => {
expect(decodeHtmlEntities('')).toBe(String.fromCodePoint(0xe000))
})

it('decodes  normally (the maximum valid code point)', () => {
expect(decodeHtmlEntities('')).toBe(String.fromCodePoint(0x10ffff))
})

it('remaps € (0x80) to the Windows-1252 euro sign', () => {
expect(decodeHtmlEntities('€')).toBe('\u20AC')
})

it('remaps — (0x97) to the Windows-1252 em dash', () => {
expect(decodeHtmlEntities('—')).toBe('\u2014')
})

it('remaps “ (0x93) to the Windows-1252 left double quote', () => {
expect(decodeHtmlEntities('“')).toBe('\u201C')
})

it('remaps ” (0x94) to the Windows-1252 right double quote', () => {
expect(decodeHtmlEntities('”')).toBe('\u201D')
})

it('remaps € (hex form) to the Windows-1252 euro sign', () => {
expect(decodeHtmlEntities('€')).toBe('\u20AC')
})

it('leaves  (0x81) as the raw C1 control character (not in the Windows-1252 table)', () => {
expect(decodeHtmlEntities('')).toBe('\u0081')
})

it('leaves  (0x8D) as the raw C1 control character (not in the Windows-1252 table)', () => {
expect(decodeHtmlEntities('')).toBe('\u008D')
})

it('does not re-scan replaced output for a second round of decoding', () => {
expect(decodeHtmlEntities(''')).toBe(''')
})

it('does not turn a decoded & into a second-pass < decode', () => {
expect(decodeHtmlEntities('<')).toBe('<')
})

it('does not turn a decoded & into a second-pass & decode', () => {
expect(decodeHtmlEntities('&')).toBe('&')
})

it('returns plain text unchanged', () => {
expect(decodeHtmlEntities('hello world')).toBe('hello world')
})
Expand Down Expand Up @@ -47,7 +151,7 @@ describe('roundtrip', () => {
expect(decodeHtmlEntities(encodeHtmlEntities(input))).toBe(input)
})

it('decode is a superset of encode – " decodes but " is not encoded', () => {
it('decode is a superset of encode - " decodes but " is not encoded', () => {
// " passes through encode unchanged, " decodes to "
expect(encodeHtmlEntities('"hello"')).toBe('"hello"')
expect(decodeHtmlEntities('"hello"')).toBe('"hello"')
Expand Down
64 changes: 56 additions & 8 deletions packages/core/src/utilities/htmlEntities.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,63 @@
const NAMED_ENTITIES: Record<string, string> = { lt: '<', gt: '>', quot: '"', amp: '&' }

/**
* Decode common HTML entities in text content so they display as literal
* characters inside the editor. The decode order matters: `&amp;` must be
* decoded **last** so that doubly-encoded sequences like `&amp;lt;` first
* survive the `&lt;` pass and then correctly become `&lt;` (not `<`).
* characters inside the editor. Runs as a single pass so a decoded entity
* (e.g. `&#38;` → `&`) is never re-scanned and decoded again.
*/
export function decodeHtmlEntities(text: string): string {
return text
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&amp;/g, '&')
// Digit limits per CommonMark: decimal refs are 1-7 digits, hex refs are 1-6.
return text.replace(
/&(?:#(\d{1,7})|#[xX]([0-9a-fA-F]{1,6})|(lt|gt|quot|amp));/g,
(match, dec, hex, name) => {
if (dec !== undefined) {
return decodeCodePoint(Number(dec))
}
if (hex !== undefined) {
return decodeCodePoint(parseInt(hex, 16))
}
return NAMED_ENTITIES[name]
},
)
}

// HTML5 maps numeric references in the C1 range (0x80-0x9F) to Windows-1252 characters.
const WINDOWS_1252 = new Map<number, number>([
[0x80, 0x20ac],
[0x82, 0x201a],
[0x83, 0x0192],
[0x84, 0x201e],
[0x85, 0x2026],
[0x86, 0x2020],
[0x87, 0x2021],
[0x88, 0x02c6],
[0x89, 0x2030],
[0x8a, 0x0160],
[0x8b, 0x2039],
[0x8c, 0x0152],
[0x8e, 0x017d],
[0x91, 0x2018],
[0x92, 0x2019],
[0x93, 0x201c],
[0x94, 0x201d],
[0x95, 0x2022],
[0x96, 0x2013],
[0x97, 0x2014],
[0x98, 0x02dc],
[0x99, 0x2122],
[0x9a, 0x0161],
[0x9b, 0x203a],
[0x9c, 0x0153],
[0x9e, 0x017e],
[0x9f, 0x0178],
])

// Replace invalid code points with U+FFFD, per the CommonMark spec.
function decodeCodePoint(codePoint: number): string {
if (codePoint === 0 || codePoint > 0x10ffff || (codePoint >= 0xd800 && codePoint <= 0xdfff)) {
return '\uFFFD'
}
return String.fromCodePoint(WINDOWS_1252.get(codePoint) ?? codePoint)
}

/**
Expand Down
22 changes: 20 additions & 2 deletions packages/extension-code/src/code.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,18 @@ export const pasteRegexMatch = (text: string): PasteRuleMatch[] => {
return matches
}

// A code span must be fenced by a backtick run its own content does not contain
function shortestUnusedBacktickRun(text: string): number {
const runs = new Set((text.match(/`+/g) || []).map(run => run.length))
let length = 1

while (runs.has(length)) {
length += 1
}

return length
}

/**
* This extension allows you to mark text as inline code.
* @see https://tiptap.dev/api/marks/code
Expand Down Expand Up @@ -128,12 +140,18 @@ export const Code = Mark.create<CodeOptions>({
return helpers.applyMark('code', [{ type: 'text', text: token.text || '' }])
},

renderMarkdown: (node, h) => {
renderMarkdown: (node, h, context) => {
if (!node.content) {
return ''
}

return `\`${h.renderChildren(node.content)}\``
// Children are a placeholder here, so the fence comes from the full span text
const markText = typeof context?.meta?.markText === 'string' ? context.meta.markText : ''
const fence = '`'.repeat(shortestUnusedBacktickRun(markText))
// CommonMark strips one space on each side, so padding keeps backticks apart
const padding = markText.startsWith('`') || markText.endsWith('`') ? ' ' : ''

return `${fence}${padding}${h.renderChildren(node.content)}${padding}${fence}`
},

addCommands() {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import type { OrderedListItem } from './utils.js'
import { collectOrderedListItems, ORDERED_LIST_ITEM_REGEX } from './utils.js'

const ORDERED_LIST_SCAN_WINDOW = 64

// Fewer than `max` lines means `source` ended.
function takeLines(source: string, max: number): string[] {
const lines: string[] = []
let start = 0

while (lines.length < max) {
const lineEnd = source.indexOf('\n', start)

if (lineEnd === -1) {
lines.push(source.slice(start))
return lines
}

lines.push(source.slice(start, lineEnd))
start = lineEnd + 1
}

return lines
}

/**
* Collect ordered list items from the start of `source`, reading only the lines the list needs.
*
* @param source Markdown beginning at a block boundary.
* @returns The items, how many lines they consumed, and the lines that were read.
* @example
* collectOrderedListItemsFromSource('1. one\n2. two\n\nAfter')
* // → [[item, item], 3, ['1. one', '2. two', '', 'After']]
*/
export function collectOrderedListItemsFromSource(
source: string,
): [OrderedListItem[], number, string[]] {
const firstLineEnd = source.indexOf('\n')
const firstLine = firstLineEnd === -1 ? source : source.slice(0, firstLineEnd)

// Every block start is offered here, so reject non-lists before reading further.
if (!ORDERED_LIST_ITEM_REGEX.test(firstLine)) {
return [[], 0, []]
}

let windowSize = ORDERED_LIST_SCAN_WINDOW
let lines = takeLines(source, windowSize)
let collected = collectOrderedListItems(lines)

// Consuming the whole window means the list may continue past it.
while (collected[1] === lines.length && lines.length === windowSize) {
windowSize *= 2
lines = takeLines(source, windowSize)
collected = collectOrderedListItems(lines)
}

return [collected[0], collected[1], lines]
}
11 changes: 3 additions & 8 deletions packages/extension-list/src/ordered-list/ordered-list.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,8 @@ import { Plugin } from '@tiptap/pm/state'

import { mergeAttributes, Node, wrappingInputRule } from '@tiptap/core'

import {
buildNestedStructure,
collectOrderedListItems,
parseListItems,
parsePlainTextOrderedListPaste,
} from './utils.js'
import { collectOrderedListItemsFromSource } from './collectOrderedListItemsFromSource.js'
import { buildNestedStructure, parseListItems, parsePlainTextOrderedListPaste } from './utils.js'

const ListItemName = 'listItem'
const TextStyleName = 'textStyle'
Expand Down Expand Up @@ -236,8 +232,7 @@ export const OrderedList = Node.create<OrderedListOptions>({
// `tokenize`, which would re-introduce the mid-line split.
start: () => -1,
tokenize: (src: string, _tokens, lexer) => {
const lines = src.split('\n')
const [listItems, consumed] = collectOrderedListItems(lines)
const [listItems, consumed, lines] = collectOrderedListItemsFromSource(src)

if (listItems.length === 0) {
return undefined
Expand Down
Loading
Loading