Algorithm to pad string is not Unicode aware #298

Activity

1. kgryte

   added

   BugSomething isn't working.Something isn't working.

   Help WantedExtra attention is needed.Extra attention is needed.

   on May 14, 2020
2. unworld11 commented on Feb 28, 2025

   

   unworld11

   on Feb 28, 2025

   More actions

   is this still open or fixed?
3. Shubham-Padkonde commented on Sep 17, 2026

   

   Shubham-Padkonde

   on Sep 17, 2026

   More actions

   I reproduced this with the published @stdlib/string-left-pad, @stdlib/string-right-pad, and @stdlib/string-pad packages at 0.2.3; the current develop implementations still use the same code-unit length/substrings:

   leftPad('\uD83D\uDE00', 3, 'a'); // "a😀": 2 grapheme clusters
   rightPad('a\u0308', 3, '.');     // "ä.": 2 grapheme clusters
   pad('a', 2, {rpad: '\uD83D\uDE00'}); // "a\ud83d": lone surrogate
   pad('\uD83D\uDE00', 1);             // "\ud83d": lone surrogate

   I would like to address all three APIs, including truncation and the centerRight tie rule, using the existing grapheme-boundary utilities rather than a Symbol/Intl-only implementation. Before preparing the RFC required for a breaking change, should this change the existing APIs' length semantics or introduce separate grapheme-aware variants?

   One edge case the contract should cover is padding that merges across a boundary: appending any number of \u0308 characters to a still produces one grapheme cluster. A loop trying to reach the requested visual length would never finish in that case. I propose explicitly documenting/rejecting padding that cannot increase the resulting grapheme count, rather than silently returning a shorter result or retrying without a bound.

   Investigation and this comment were prepared with Codex/GPT-6 assistance. No PR has been opened.
4. Abhist17 commented on Sep 18, 2026

   

   Abhist17

   on Sep 18, 2026

   Contributor

   More actions

   I looked into this. left-pad, right-pad (public + base/), and pad all use String#length/String#substring, so length and truncation are in UTF-16 code units, not user-perceived characters. pad has the sharper version of this: its truncation can land a substring cut inside a surrogate pair and emit an unpaired surrogate, e.g. pad('a', 2, {rpad: '😀'}) → 'a\ud83d'.

   @Shubham-Padkonde's comment above reproduced this precisely (thanks for the repro cases — reused them below to check my fix). I've opened #15319, which switches all five implementations to measure and truncate by grapheme cluster using this package's own num-grapheme-clusters/base/slice-grapheme-clusters, so pad('a', 2, {rpad: '😀'}) now returns 'a😀' and truncation never splits a cluster.

   On the combining-mark non-termination case you raised (lpad('a', N, '̈') never actually reaching N clusters since consecutive Extend characters merge under UAX #29 GB9) — confirmed it reproduces, and I've left it as a documented, tested limitation in the PR rather than rolled a fix in, since it's a distinct question from what's reported here. I asked in the PR whether a RangeError for a non-progressing pad string is the right follow-up, and whether this should go through the RFC process as a breaking change or land as a bug fix — your call to make, not mine. Happy to take this if it's unassigned.