Repository navigation
Decode non-BMP Java strings correctly and convert Swift strings with NewStringUTF - #24
Open
piercifani wants to merge 1 commit into
Open
piercifani wants to merge 1 commit into
piercifani wants to merge 1 commit into
Conversation
…NewStringUTF String(fromJNI:) decoded GetStringUTFChars with String(cString:), which repairs invalid UTF-8. GetStringUTFChars returns modified UTF-8: NUL is 0xC0 0x80 and, on HotSpot, characters outside the BMP are surrogate pairs, so those strings came back with U+FFFD replacement characters. Strings that fail UTF-8 validation are now decoded from UTF-16 via GetStringRegion. getJNIValue always copied the string into a UTF-16 array for NewString. Strings whose UTF-8 is also valid modified UTF-8 (no NUL, no 4-byte sequence) now go straight to NewStringUTF; the rest keep the UTF-16 path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes to
String'sJavaValueconformance inJavaValue+String.swift.Java → Swift: strings with emoji or NUL came back corrupted.
String(fromJNI:)decodedGetStringUTFCharswithString(cString:).GetStringUTFCharsreturns modified UTF-8: NUL is encoded as0xC0 0x80, and on HotSpot characters outside the BMP are encoded as surrogate pairs. Neither is valid UTF-8, andString(cString:)silently replaces them with U+FFFD, so"🎵🎶","Fed 🇺🇸 📉"and"before\0after"did not round-trip. Strings that failString(validatingCString:)are now decoded from UTF-16 viaGetStringLength+GetStringRegioninto a temporary buffer. Valid strings take the same path as before.ART encodes non-BMP characters as standard 4-byte UTF-8, so emoji were fine on Android (verified on a Pixel); the emoji case shows up on desktop JVMs (macOS, Linux, Windows, and Robolectric). The embedded-NUL case follows the JNI spec's
0xC0 0x80encoding everywhere.Swift → Java: every string was copied to UTF-16.
getJNIValuealways builtArray(utf16)and calledNewString(theFIXMEthis removes). Strings whose UTF-8 is also valid modified UTF-8 (no NUL, no 4-byte sequence) now go straight toNewStringUTFthroughwithCString, which uses the native buffer without a copy. The check scans the contiguous UTF-8 storage; strings without it, and strings that fail the check, keep the existing UTF-16 path.Numbers
Release build, macOS (Apple silicon), HotSpot, best of 9 runs of 100,000 iterations each:
Java → Swift for valid strings is unchanged within noise. The same change in Skip's swift-jni (skiptools/swift-jni#11) measured 1.56× on
Stringgetters on a Pixel 10 Pro Fold (ART, release build).Tests
New
JavaStringTests, run against an empty string, ASCII, Latin-1, CJK, emoji, a flag sequence, mixed BMP/non-BMP and an embedded NUL:roundTrip:String(fromJNI: s.getJNIValue(in:))equalss. Onmainit fails for the three emoji strings and the embedded NUL; with this change all eight pass.javaLengthMatchesUTF16: the Java string's length equalss.utf16.count, checking theNewStringUTFfast path never produces a different Java string.swift test: 21/21.swift format lint --strictis clean on the changed files.