Harden the generated FFM runtime and copy value types verbatim

README.md

@@ -150,8 +150,8 @@ Every test compiles Kotlin/Native → `libcalculator.so` (470+ exported symbols)
 | `Set<T>?` | &mdash; | ✅ 5t | &mdash; | &mdash; | &mdash; | -1 count = null sentinel |
 | `Map<K, V>` | ✅ 12t | ✅ 12t | &mdash; | ✅ 2t | ✅ 2t | String→Int, Int→String, Int→Int, String→String + merge/empty |
 | `Map<K, V>?` | &mdash; | ✅ 4t | &mdash; | &mdash; | &mdash; | -1 count = null sentinel |
-| `(T) -> R` (lambda) | ✅ 15t | &mdash; | &mdash; | &mdash; | &mdash; | persistent `Arena.ofShared()` |
-| `Flow<T>` | &mdash; | ✅ 50t+ | &mdash; | &mdash; | &mdash; | `channelFlow` + 3 callbacks (onNext, onError, onComplete), incl. `Flow<DataClass>` |
+| `(T) -> R` (lambda) | ✅ 15t | &mdash; | &mdash; | &mdash; | &mdash; | per-call upcall arena, exceptions propagate with their original type |
+| `Flow<T>` | &mdash; | ✅ 50t+ | &mdash; | &mdash; | &mdash; | static onNext/onError/onComplete stubs + credit-based backpressure, incl. `Flow<DataClass>` |
 
 ### Declarations
 
@@ -174,9 +174,9 @@ Every test compiles Kotlin/Native → `libcalculator.so` (470+ exported symbols)
 | Data classes (nativeMain) | ✅ | auto-generates JVM data class + field marshalling |
 | Data classes (commonMain) | ✅ | reuses existing JVM type, no proxy generated |
 | Suspend functions | ✅ | `suspendCancellableCoroutine` + bidirectional cancellation (110+ tests) |
-| Flow&lt;T&gt; return | ✅ | `channelFlow` + onNext/onError/onComplete callbacks (50+ tests) |
-| Exception propagation | ✅ | `try/catch` wrapping, `KotlinNativeException` on JVM |
-| Object lifecycle | ✅ | `Cleaner` for GC + `close()` for explicit release |
+| Flow&lt;T&gt; return | ✅ | onNext/onError/onComplete callbacks with backpressure, no element dropped (50+ tests) |
+| Exception propagation | ✅ | `try/catch` wrapping, `KotlinNativeException` on JVM (thread-local, safe under concurrency) |
+| Object lifecycle | ✅ | `Cleaner` for GC + `close()` for explicit release; calls after `close()` throw `IllegalStateException` |
 
 ### Suspend functions
 
@@ -193,7 +193,7 @@ suspend fun fetchData(query: String): String {
 val result = calc.fetchData("test")  // suspends the coroutine
 ```
 
-**How it works**: the native bridge launches a `CoroutineScope` with a `Job`, passes continuation + exception callbacks as FFM upcall stubs. The JVM proxy uses `suspendCancellableCoroutine` to suspend until the native coroutine completes.
+**How it works**: the native bridge launches a `CoroutineScope` with a `Job` and reports completion through two static FFM upcall stubs, dispatched by a per-call id. The JVM proxy uses `suspendCancellableCoroutine` to suspend until the native coroutine completes.
 
 **Cancellation**: JVM coroutine cancel → `Job.cancel()` on native side. Native `CancellationException` → JVM `CancellationException`. Bidirectional, automatic.
 
@@ -215,7 +215,7 @@ calc.countUp(100).toList()               // [1, 2, ..., 100]
 calc.infiniteFlow().take(3).toList()     // [0, 1, 2] — auto-cancelled
 ```
 
-**How it works**: 3 native callbacks (`onNext`, `onError`, `onComplete`) are passed as FFM upcall stubs. The native side collects the Flow in a `CoroutineScope` and calls `onNext` for each element. The JVM proxy uses `channelFlow { trySend(...); awaitClose { cancelJob() } }`.
+**How it works**: 3 static FFM upcall stubs (`onNext`, `onError`, `onComplete`) are dispatched by a per-call id. The native side collects the Flow in a `CoroutineScope` and only emits against credits granted as the JVM collector consumes elements, so a slow collector applies backpressure instead of losing elements.
 
 **Cancellation**: collecting only N elements (via `take`, `first`) automatically cancels the native Flow collection. Manual `Job.cancel()` also propagates.
 
@@ -241,7 +241,9 @@ desktop.memoryFlow(2000L).collect { info ->
 
 JVM lambdas cross the FFM boundary via upcall stubs. The plugin generates all the FFM infrastructure automatically.
 
-**Lifecycle**: each proxy object holds a persistent `Arena.ofShared()`. Upcall stubs live as long as the object &mdash; async callbacks (event handlers, listeners) work out of the box. The arena is freed on `close()` or GC.
+**Lifecycle**: upcall stubs are allocated in a per-call arena and released when the call returns, so a callback is valid **for the duration of the call only** (as with JNI). Native code must not store a callback and invoke it later; expose event streams as `Flow<T>` instead.
+
+**Exceptions**: an exception thrown by a JVM callback unwinds the native caller and is rethrown on the JVM with its original type and message.
 
 **Supported callback signatures**:
 - Params: `Int`, `Long`, `Double`, `Float`, `Boolean`, `Byte`, `Short`, `String`, `enum class`, `data class`
@@ -260,14 +262,16 @@ calc.onValueChanged { value -> println("Value: $value") }
 calc.transform { it * 2 }
 calc.formatWith { "Result: $it" }
 
-// Async callbacks work (e.g. native event listeners)
-desktop.setTrayClickCallback { index -> println("Clicked: $index") }
+// Event streams: use Flow rather than stored callbacks
+desktop.trayClicks().collect { index -> println("Clicked: $index") }
 ```
 
 ### Collections
 
 `List<T>`, `Set<T>`, and `Map<K, V>` cross the FFM boundary using flat arrays (pointer + size), inspired by swift-java's `[UInt8]` lowering.
 
+There is no size limit: results that do not fit the initial transfer buffers (4096 elements / 8 KB of packed strings) are parked natively and read back with a larger buffer, without calling the function twice. The same applies to large `String`, `ByteArray` and data class `String` fields.
+
 **Supported element types**: `Int`, `Long`, `Double`, `Float`, `Short`, `Byte`, `Boolean`, `String`, `enum class`
 
 ```kotlin
@@ -294,6 +298,8 @@ val meta = calc.getMetadata()              // {current=42, scale=3}
 
 Data classes are marshalled **by value** (field decomposition) &mdash; each field becomes a separate C ABI argument. Supported field types: all primitives + `String`.
 
+The JVM data class (and `enum class`) is a verbatim copy of the native declaration: default values, body members, companion objects, KDoc and annotations are preserved. Imports of native-only packages (`kotlinx.cinterop`, `platform.*`, `kotlin.native`) are dropped, so bodies must only use APIs available on the JVM.
+
 ```kotlin
 // Can be in commonMain or nativeMain
 data class Point(val x: Int, val y: Int)
@@ -407,6 +413,8 @@ Measured on Intel Core i5-14600 (20 cores), 45 GB RAM, Ubuntu 25.10, JDK 25 (Gra
 |---------|--------|-------------|
 | Private/internal/protected members | Only public API is exported | Use `public` modifier |
 | Expect/actual declarations | KMP's responsibility | Use platform-specific source sets |
+| Storing a JVM callback on the native side | Upcall stubs are released when the call returns | Expose events as `Flow<T>` |
+| NUL characters in `String` arguments | Native strings are NUL-terminated; rejected with `IllegalArgumentException` | Encode the data (e.g. `ByteArray`) |
 | `ByteArray` in collections | Buffer lifecycle complexity across FFM | Use `List<Int>` or Base64 String |
 | `ByteArray` as data class field | Out-param buffer not wired for DC fields | Use separate method or String |
 | `ByteArray` as callback param | Buffer lifecycle across callback boundary | Use String (Base64) |

examples/calculator/src/jvmTest/kotlin/com/example/calculator/BufferTortureTest.kt

@@ -0,0 +1,114 @@
+package com.example.calculator
+
+import kotlinx.coroutines.runBlocking
+import kotlin.random.Random
+import kotlin.test.Test
+import kotlin.test.assertContentEquals
+import kotlin.test.assertEquals
+import kotlin.test.assertFailsWith
+import kotlin.test.assertNull
+
+/** Every fixed-size transfer buffer of the generated bridges, pushed past its limit. */
+class BufferTortureTest {
+
+    private val sizes = listOf(0, 1, 8_190, 8_191, 8_192, 8_193, 100_000)
+
+    @Test
+    fun `data class string field beyond buffer`() {
+        Calculator(0).use { calc ->
+            for (n in sizes) {
+                val nv = calc.bigNamedValue(n)
+                assertEquals(n, nv.name.length, "n=$n")
+                assertEquals(n, nv.value)
+            }
+        }
+    }
+
+    @Test
+    fun `list of data classes with large strings`() {
+        Calculator(0).use { calc ->
+            val list = calc.bigNamedValues(50, 20_000)
+            assertEquals(List(50) { NamedValue("v$it-" + "x".repeat(20_000), it) }, list)
+        }
+    }
+
+    @Test
+    fun `data class bytearray field beyond buffer`() {
+        Calculator(0).use { calc ->
+            for (n in sizes) {
+                val p = calc.bigPayload(n)
+                assertEquals(n, p.name.length, "name n=$n")
+                assertContentEquals(ByteArray(n) { (it % 251).toByte() }, p.data, "data n=$n")
+            }
+        }
+    }
+
+    @Test
+    fun `nullable string beyond buffer`() {
+        Calculator(0).use { calc ->
+            assertNull(calc.bigNullableString(-1))
+            for (n in sizes) assertEquals("ü".repeat(n), calc.bigNullableString(n), "n=$n")
+        }
+    }
+
+    @Test
+    fun `string property beyond buffer`() {
+        Calculator(0).use { calc ->
+            for (n in sizes) {
+                val s = "λ".repeat(n)
+                calc.label = s
+                assertEquals(s, calc.label, "n=$n")
+            }
+        }
+    }
+
+    @Test
+    fun `extension function string return beyond buffer`() {
+        Calculator(0).use { calc ->
+            val prefix = "p".repeat(20_000)
+            assertEquals("$prefix: Calculator(current=0)", calc.describeWithPrefix(prefix))
+        }
+    }
+
+    @Test
+    fun `bytearray roundtrip beyond buffer`() {
+        Calculator(0).use { calc ->
+            for (n in sizes) {
+                val data = Random(n).nextBytes(n)
+                assertContentEquals(data, calc.echoBytes(data), "n=$n")
+            }
+        }
+    }
+
+    @Test
+    fun `maps with non-string keys`() {
+        Calculator(0).use { calc ->
+            for (n in listOf(0, 1, 5_000)) {
+                assertEquals((0 until n).associateWith { "v$it" }, calc.bigIndexed(n), "int keys n=$n")
+                assertEquals((0 until n).associate { it.toLong() * 3_000_000_000L to it }, calc.longKeyMap(n), "long keys n=$n")
+            }
+            assertEquals(Operation.entries.associateWith { it.ordinal * 10 }, calc.enumKeyMap())
+        }
+    }
+
+    @Test
+    fun `short and byte lists`() {
+        Calculator(0).use { calc ->
+            for (n in listOf(0, 1, 300, 5_000)) {
+                assertEquals(List(n) { (it - 30_000).toShort() }, calc.shortList(n), "shorts n=$n")
+                assertEquals(List(n) { (it - 128).toByte() }, calc.byteList(n), "bytes n=$n")
+                assertEquals(List(n) { (it - 30_000).toShort() }, runBlocking { calc.delayedShorts(n) }, "suspend shorts n=$n")
+            }
+        }
+    }
+
+    @Test
+    fun `closed object passed as parameter fails cleanly`() {
+        val manager = CalculatorManager()
+        val calc = Calculator(5)
+        assertEquals(6, manager.addWith(calc, 1))
+        calc.close()
+        assertFailsWith<IllegalStateException> { manager.addWith(calc, 1) }
+        assertFailsWith<IllegalStateException> { manager.describe(calc) }
+    }
+}