[siracozmen01]

Problem

The family conventions now say that opaque state only the library writes is checked by its cookie and by every index, slot and length in it being in bounds, so that no bit pattern makes a call read outside the state (Rawframe-Project/.github#119). muniScriptIterator and muniSegmentIterator carried no cookie, and their calls trusted the counts and flags in caller memory. A script iterator of 0x41 bytes made muniScriptIterator_NextRun copy 65 bytes into a 4-byte window on the stack, and 0x02 or 0x04 bytes made both iterators load a bool that was neither 0 nor 1.

Change

• Each init call writes a 16-bit cookie with its top bit set into the iterator's cursor, in two bytes that were its padding, and every call that takes an iterator back checks it, then checks every count against the storage it counts (the cursor's pending bytes and position, the bracket count, the override count), every class, state, hold kind and script index against the values the rules write, and every flag's byte as a 0 or 1 before the copy is read as a bool. It also checks the two cross-field facts the assertions downstream rely on: a waiting cursor holds less than a code point, and a hold lies before the cursor. The cookie only turns away an iterator no init call set up; the checks after it keep any other bit pattern from reaching outside the state too. They run once per call.
• An iterator that fails them is refused with muni_errorInvalid, the result each header already gives for a bad argument, and left as it was; muniSegmentIterator_GetLineBreakStrictness and muniSegmentIterator_GetWritingSystem give their defaults for it, as for NULL. Each header now names it, and says that a copy of an initialized iterator is one too. The pointers an iterator keeps (the text, the overrides, the caller's breaker) cannot be checked: an iterator forged there with the cookie kept stays outside the contract, as docs/internals/overview.md now says.
• Neither public iterator changes size: the cookie fills the cursor's padding on 32-bit and 64-bit targets alike, and a new static_assert beside muniSegmenter, like the script state's, keeps the iterator from growing a whole uint64_t more than the state needs. A first version put a 32-bit cookie in front of each state, which grew both iterators by 8 bytes; the segmenter's copies in and out of caller memory then passed 128 bytes and became memcpy calls, which cost a grapheme break about 8 ns.
• Since a call that was undefined behavior now refuses with a result the headers document, this is a feat, so the next release is 1.1.0. The ABI is unchanged.
• The segment check is inlined into each call and checks only the fields of the iterator's rule set; line's anywhere setting checks only the grapheme state it keeps. The flags' bytes are read from the caller's iterator rather than from the copy just made of it, so that no load waits on the copy's stores. The largest value each rule set writes in a state is named in src/segmenter.h and tied to the rule set's own enumeration by a static_assert.
• The script state moves to src/script_state.h so that the new fuzz target can reach its fields.
• docs/internals/segmentation.md describes the checks and where the cookie lives, and no longer asks for ! when the iterator grows, which 1.x forbids.

Who sees it: a caller that passed an iterator no init call set up, which was undefined behavior, now gets muni_errorInvalid. Every result for an initialized iterator is the same; no result of the results version changes.

Verification

• test_segment and test_script fill an iterator with 0x00 (zeroed, as a static one is), 0x41, 0x02 and 0x04 and pass it to every call that takes one back: each refuses it and changes nothing in it. On main the 0x41 script fill is a stack buffer overflow under AddressSanitizer and the 0x02 and 0x04 fills stop UBSan on a bool load.

• fuzz_iterators, a new fuzz target, sets up each kind of iterator over a fixed text, answers a few calls, then sets bytes of its state from the input, everywhere but the pointers and the lengths and counts that go with them, and makes every call again: none may read outside the state or the text or trip an assertion. 300 seconds of it ran 7,227,223 inputs with no finding.

• tools/check.sh changed and tools/check.sh pr pass: the generated tables, the HarfBuzz and AddressSanitizer presets with every suite, clang-tidy on every source in Release and Debug, and ten seconds of each fuzz target, the new one included.

• Speed, Release builds with clang 20, best of 15 passes over a text of 1 Mi code points, the two builds run eight times each in turn and the best of each kept. Every call here reports one run or boundary, the most calls a text can need, so the difference is the check's whole cost per call:

  Call, each reporting one run or boundary	main	this change	difference
  muniScriptIterator_NextRun, alternating Latin and Greek	31.1 ns	32.4 ns	+1.3 ns (4%)
  muniSegmentIterator_NextBreak, grapheme clusters of ASCII	17.0 ns	18.9 ns	+1.9 ns (11%)
  muniSegmentIterator_NextBreak, lines of "a " repeated	25.0 ns	29.9 ns	+4.9 ns (20%)

  The medians of the eight runs differ by the same: +0.6, +2.0 and +5.4 ns. perf stat over the whole bench counts the same branch misses (302 thousand) and L1 misses (1.5 million) for both builds and 29% more instructions, so the cost is the instructions of the check, which a line call runs more of: the cursor's checks and the line rules' classes, flags, number state and hold kind. The first version of this change, which called a validator per rule set out of line, cost +2.7 ns (16%) and +6.9 ns (28%) on the same two calls. On ordinary prose the cost is small: line NextBreak over the English preamble of the Universal Declaration of Human Rights, repeated to 1 MiB (168,979 breaks, one per 6.2 bytes), takes 107.2 ns per break on main and 108.2 ns with this change at best, +1.0 ns (1%), and 109.6 against 112.3 ns at the median, +2.6 ns (2%), over the same eight alternating runs.

Closes #263

[siracozmen01]

Review at d121e69. No changes needed. Read, not run. CI had just started when I looked, so this approval assumes it passes.

1. Every field is checked before it is used, in its rule set's arm, and every refusal reaches the shared exit. I compared muniSegmenter (src/segmenter.h:136-161) and its rule structs field by field with Load and RulesValid (src/segment.c):
   • Every call site does if (!Load(...)) return muni_errorInvalid; (or the default value in the two getters) before any read of the copy.
   • Top level: kind (an unknown kind falls to the switch's default: return false), phase, the four flag bytes and more, the cursor's pendingCount, position against length, text NULL with a length, and the waiting and hold cross-checks. complex.next is checked against its length when complex.text is set.
   • Grapheme: all four fields. Word and sentence: every field, including overrideCount against MAX_OVERRIDES and a NULL table. Line: lineSettings >> 3, actual, and then, by strictness, either the grapheme view (anywhere) or previous, before, lead, number, holdKind and the three flags. Sot is muni_lbXx (line_rules.h:47), so the <= muni_lbHh bounds take it.
   • Two line fields are not bounded: previousCodePoint and beforeCodePoint. Every use is a property lookup (IsEastAsian, IsWide, Category, muniLookupEmoji, line_rules.h:215-449), and the generated lookups return 0 for a code point at or past 0x10FFFE before indexing (e.g. east_asian_width.c:115-118). So any value stays inside the tables, and leaving them unchecked is safe.
   • The script iterator: Load (src/script_runs.c) checks the cookie, the four flag bytes, the cursor, bracketCount <= BRACKET_DEPTH, firstScript and every bracket's script against muniScriptTagCount or NO_SCRIPT, and no candidate bit past the last script, with at least one in a determined run.
2. There is no check-then-use gap from reading the flags in the caller's iterator. Within Load, nothing writes the caller's iterator between the memcpy and the byte reads, so each byte read is the byte copied. Everything after Load reads the stack copy, not the caller's iterator, including after the complex breaker's callback, during which the phase is Busy and other calls refuse. Only a write from another thread could make them differ, and the headers already rule that out. In the script Load, each flag's byte is checked before the copy's bool is read: waiting is checked before it is passed to muniCursorValid, which checks moreFollows' byte first.
3. The static_asserts' (int) casts are right. Each compares an enumerator of a private anonymous enum with one of segmenter.h's. The values are 2 and 4, so the casts change nothing, and they only keep an enum-compare warning away under -Werror.

feat is right at 1.x: a call that was undefined behaviour now refuses with a result the headers document, the ABI is unchanged, and results for an initialized iterator are the same.