[larroy]

split_phonemes measured chunks in bytes and left over-long unpunctuated stretches whole; _create_audio then cut them at 510 bytes, dropping audio and possibly splitting a UTF-8 character. Chunks are now at most 510 UTF-8 characters (<= 510 tokens), split at punctuation, then spaces, then after the last tone mark or at exactly 510 characters. The byte cut is replaced by a reported token cap. Closes ISSUES.md #1.

[larroy]

Review of commit b4a8ac0: request changes for two P2 findings.

1. Trailing punctuation can detach a Chinese tone mark (P2)

Location: src/PhonemeChunker.cpp:58–59.

The punctuation adjustment overrides the selected tone boundary by moving back only one character. When that character is a tone mark, it separates the tone from its syllable.

Reproduced with ni→ repeated 170 times, followed by ., using limit 510:

• First chunk: 509 characters, ending in ni without its tone.
• Second chunk: →., with no spoken phoneme.

The public synthesis API accepts →. and produces 0.675 seconds of PCM. This creates a separate inference call for the detached tone and punctuation instead of keeping the final syllable intact.

Suggested fix: retreat to the preceding syllable boundary when reserving room for trailing punctuation.

2. Flattening groups loses punctuation-first splitting (P2)

Location: src/PhonemeChunker.cpp:127–131.

Oversized groups are split into words before packing, then flattened alongside completed punctuation groups. pack() therefore cannot prefer a punctuation boundary over a later word boundary.

Reproduced with 100 a characters followed by . and three space-separated words of 199 b characters each. The first chunk contains the complete first sentence plus two words of the next sentence; the boundary falls before its third word despite the available period boundary. This violates the advertised punctuation-first policy and introduces an avoidable mid-sentence inference boundary.

Suggested fix: preserve group boundaries during packing; split an oversized group only after choosing whether to flush the preceding punctuation-terminated chunk.

Verification

• Chunker: 9 tests, 238 assertions passed, compiled from PR sources.
• Synthesis: 13 tests, 292 assertions passed, including the new long-text regression.
• Boundary probes and public-API synthesis smoke exercised.
• Inference ran on CPU; CUDA unavailable because cudnn64_9.dll is missing.

No repository source changes made.

Model used for this review: GPT-6.1-SOL.

[larroy]

Both findings from the review above are addressed in commit 446399a.

• Tone retention: trailing punctuation now reserves the final phoneme and its tone. The ni→ × 170 + . boundary case produces 169 complete syllables followed by ni→., rather than a detached →. chunk.
• Punctuation-first packing: punctuation groups remain intact during packing. An oversized group starts a new chunk before word or hard splitting, preserving the earlier punctuation boundary.

Added four regression cases covering all four tone marks, repeated punctuation, UTF-8 fallback, and boundary precedence. Updated the chunker API documentation and ARCHITECTURE.md.

Verification:

• Chunker: 13 tests, 295 assertions passed.
• Real-model CPU smoke: both corrected scenarios matched the sample counts from synthesis of their expected chunks, including repeated runs. PCM bytes vary even on identical repeated inputs, so byte equality was not used as proof.
• Synthesis: 12 cases passed in the combined run; the CPU-device case hit an allocation failure while loading another context, then passed separately (4 assertions).

Model used: GPT-6.1-SOL.