ci: try Blacksmith runners for performance benchmarks

[mrousavy]

Try Blacksmith for the performance workflow's Android and iOS builds and measurements, to measure whether its runners improve CI duration and reduce same-code timing variation.

• Android uses blacksmith-4vcpu-ubuntu-2404; iOS uses blacksmith-6vcpu-macos-26 (Apple Silicon).
• Keep API 36 / KVM, Xcode 26.5, the benchmark suite, calibration, sampling and per-case BASE → HEAD order unchanged. Both revisions still run on the same measurement host.
• Include the requested Blacksmith tier in raw result device descriptions. Preparation, collection and trusted reporting keep their existing GitHub runners.

Blacksmith documents macOS 26 and x64 Linux nested virtualization support.

Validation: all 77 performance-tool tests pass; actionlint and shellcheck pass with Blacksmith's custom runner labels allowed. The trial run, attempt 2 passed all build, measurement and collection jobs. Trusted reporting succeeded, and Nitro Modules Bot posted the comparison.

Job	Previous GitHub run (#1612)	Blacksmith trial
Build Android	10m01s	5m18s
Build iOS	11m03s	4m29s
Measure Android	16m17s	14m43s
Measure iOS	25m47s	14m03s

These are observed job durations, not a controlled hardware speedup estimate; cache and image state can differ. Both revisions were built and measured on Blacksmith, with required Android KVM and Xcode 26.5. Benchmark suite hash remains f34d9c1f3a289bdc8f9257a57ba955d72f2ca30cb632c9fcd9a554d964674b55.

The PR changes no Nitro implementation or benchmark code. It compares separately built base/head apps with different application IDs, not byte-identical apps. All 46 cases per platform completed with matching base/head operation and chunk counts.

Same-code base/head differences	Android	iOS
Median absolute change across cases	1.18%	1.09%
Cases with an absolute change of at least 5%	6/46	7/46
Largest absolute change	60.03%	14.33%

The iOS observations are encouraging compared with the earlier GitHub same-binary diagnostic (median absolute change 10.22%, 33/46 cases at least 5%), but this one run does not establish reliable detection of small regressions. Android still has a large outlier: C++ simpleFunc() reports +60.03%, with its head samples slowing approximately 92% between the first and last five. iOS's largest corresponding within-process drift was about 6.7%. No samples were discarded or thresholds adjusted.

Raw JSON and calibration/measurement records are attached to the successful CI run and linked from the bot comment. Existing compiler/React Native warnings remain, including the explicit-module compiler-selection warning addressed separately by #1610.

Attempt 1 was cancelled before platform jobs started, then retried after the Blacksmith installation was granted access to Nitro. The successful result is attempt 2 at 16367b7b5e3019a7489484c58c010c2893482c63.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
nitro-docs	Ready	Preview	Sep 7, 2026 5:20pm UTC

[nitro-modules-bot]

Performance Report

Report only: Measurements do not fail this PR. Each benchmark has one base/head process pair. Samples describe within-process variation; they do not establish repeatability between launches or statistical confidence.

iOS

Benchmark	Base p50	Head p50	Observed change
C++ small array (16)	851.2 ns	808.5 ns	-5.01% faster
C++ synchronousCallback()	181.3 ns	171.5 ns	-5.38% faster
C++ simpleFunc()	70.2 ns	63.5 ns	-9.5% faster
C++ number property get/set	123.5 ns	114.4 ns	-7.34% faster
Swift synchronousCallback()	239.1 ns	204.8 ns	-14.33% faster
Swift typed map (8 entries)	2.25 µs	2.10 µs	-6.75% faster
Swift number property get/set	130.5 ns	142.4 ns	+9.18% slower

All benchmarks and sample variation

Benchmark	Base p50	Head p50	Observed change
JavaScript addNumbers()	22.9 ns	22.7 ns	-0.56% faster
C++ bounce(1 MiB)	225.7 ns	223.8 ns	-0.86% faster
C++ bounce(4 KiB)	219.7 ns	220.0 ns	+0.14% slower
C++ bounce native-owned buffer (1 MiB)	3.49 µs	3.42 µs	-1.88% faster
C++ bounce native-owned buffer (4 KiB)	359.1 ns	359.5 ns	+0.11% slower
C++ copy(1 MiB)	54.80 µs	55.43 µs	+1.15% slower
C++ copy(4 KiB)	843.8 ns	864.2 ns	+2.42% slower
C++ large array (1,024)	41.27 µs	41.38 µs	+0.27% slower
C++ small array (16)	851.2 ns	808.5 ns	-5.01% faster
C++ synchronousCallback()	181.3 ns	171.5 ns	-5.38% faster
C++ create()	356.6 ns	365.7 ns	+2.56% slower
C++ returnExisting()	170.9 ns	171.1 ns	+0.11% slower
C++ typed map (8 entries)	1.50 µs	1.43 µs	-4.44% faster
C++ optional trailing string	128.1 ns	129.3 ns	+0.93% slower
C++ addNumbers()	73.6 ns	74.3 ns	+0.94% slower
C++ simpleFunc()	70.2 ns	63.5 ns	-9.5% faster
C++ deferred worker Promise (includes trigger call)	17.81 µs	17.55 µs	-1.45% faster
C++ immediatePromise()	722.9 ns	730.3 ns	+1.03% slower
C++ number property get/set	123.5 ns	114.4 ns	-7.34% faster
C++ short ASCII string	123.9 ns	123.4 ns	-0.44% faster
C++ Unicode string	245.4 ns	246.5 ns	+0.43% slower
C++ nested Car struct	2.09 µs	2.06 µs	-1.6% faster
C++ number	string variant	113.0 ns	112.3 ns
Swift bounce(1 MiB)	203.7 ns	202.0 ns	-0.8% faster
Swift bounce(4 KiB)	198.8 ns	198.7 ns	-0.06% faster
Swift bounce native-owned buffer (1 MiB)	3.08 µs	3.07 µs	-0.23% faster
Swift bounce native-owned buffer (4 KiB)	339.7 ns	328.8 ns	-3.23% faster
Swift copy(1 MiB)	43.89 µs	43.00 µs	-2.02% faster
Swift copy(4 KiB)	826.7 ns	803.9 ns	-2.76% faster
Swift large array (1,024)	44.46 µs	44.84 µs	+0.86% slower
Swift small array (16)	918.7 ns	912.5 ns	-0.68% faster
Swift synchronousCallback()	239.1 ns	204.8 ns	-14.33% faster
Swift create()	22.52 µs	21.96 µs	-2.48% faster
Swift returnExisting()	209.7 ns	213.5 ns	+1.79% slower
Swift typed map (8 entries)	2.25 µs	2.10 µs	-6.75% faster
Swift optional trailing string	142.5 ns	143.2 ns	+0.44% slower
Swift addNumbers()	69.6 ns	72.7 ns	+4.38% slower
Swift simpleFunc()	56.0 ns	56.2 ns	+0.37% slower
Swift deferred worker Promise (includes trigger call)	20.01 µs	19.94 µs	-0.35% faster
Swift immediatePromise()	1.15 µs	1.12 µs	-2.51% faster
Swift number property get/set	130.5 ns	142.4 ns	+9.18% slower
Swift short ASCII string	145.3 ns	145.7 ns	+0.28% slower
Swift Unicode string	315.3 ns	316.7 ns	+0.44% slower
Swift nested Car struct	2.29 µs	2.25 µs	-1.57% faster
Swift number	string variant	175.5 ns	179.4 ns
TurboModule addNumbers()	898.7 ns	890.2 ns	-0.95% faster

Benchmark	Base process p50	Head process p50	Paired changes	Sample MAD / p50 (base, head)
JavaScript addNumbers()	22.9 ns	22.7 ns	-0.56% faster	1.0%, 2.4%
C++ bounce(1 MiB)	225.7 ns	223.8 ns	-0.86% faster	2.5%, 2.6%
C++ bounce(4 KiB)	219.7 ns	220.0 ns	+0.14% slower	3.2%, 1.2%
C++ bounce native-owned buffer (1 MiB)	3.49 µs	3.42 µs	-1.88% faster	2.2%, 0.8%
C++ bounce native-owned buffer (4 KiB)	359.1 ns	359.5 ns	+0.11% slower	1.5%, 1.5%
C++ copy(1 MiB)	54.80 µs	55.43 µs	+1.15% slower	1.2%, 2.9%
C++ copy(4 KiB)	843.8 ns	864.2 ns	+2.42% slower	2.0%, 3.1%
C++ large array (1,024)	41.27 µs	41.38 µs	+0.27% slower	1.7%, 0.7%
C++ small array (16)	851.2 ns	808.5 ns	-5.01% faster	1.7%, 1.4%
C++ synchronousCallback()	181.3 ns	171.5 ns	-5.38% faster	1.7%, 1.5%
C++ create()	356.6 ns	365.7 ns	+2.56% slower	1.4%, 2.5%
C++ returnExisting()	170.9 ns	171.1 ns	+0.11% slower	1.6%, 1.8%
C++ typed map (8 entries)	1.50 µs	1.43 µs	-4.44% faster	1.0%, 1.2%
C++ optional trailing string	128.1 ns	129.3 ns	+0.93% slower	0.5%, 0.8%
C++ addNumbers()	73.6 ns	74.3 ns	+0.94% slower	1.1%, 1.4%
C++ simpleFunc()	70.2 ns	63.5 ns	-9.5% faster	2.7%, 2.8%
C++ deferred worker Promise (includes trigger call)	17.81 µs	17.55 µs	-1.45% faster	1.9%, 1.1%
C++ immediatePromise()	722.9 ns	730.3 ns	+1.03% slower	0.9%, 1.8%
C++ number property get/set	123.5 ns	114.4 ns	-7.34% faster	2.6%, 1.0%
C++ short ASCII string	123.9 ns	123.4 ns	-0.44% faster	0.7%, 0.8%
C++ Unicode string	245.4 ns	246.5 ns	+0.43% slower	2.1%, 1.8%
C++ nested Car struct	2.09 µs	2.06 µs	-1.6% faster	0.9%, 0.6%
C++ number	string variant	113.0 ns	112.3 ns	-0.58% faster
Swift bounce(1 MiB)	203.7 ns	202.0 ns	-0.8% faster	3.4%, 1.6%
Swift bounce(4 KiB)	198.8 ns	198.7 ns	-0.06% faster	1.8%, 1.0%
Swift bounce native-owned buffer (1 MiB)	3.08 µs	3.07 µs	-0.23% faster	0.8%, 0.7%
Swift bounce native-owned buffer (4 KiB)	339.7 ns	328.8 ns	-3.23% faster	1.0%, 0.7%
Swift copy(1 MiB)	43.89 µs	43.00 µs	-2.02% faster	1.4%, 1.2%
Swift copy(4 KiB)	826.7 ns	803.9 ns	-2.76% faster	1.6%, 1.9%
Swift large array (1,024)	44.46 µs	44.84 µs	+0.86% slower	0.4%, 0.9%
Swift small array (16)	918.7 ns	912.5 ns	-0.68% faster	1.2%, 2.2%
Swift synchronousCallback()	239.1 ns	204.8 ns	-14.33% faster	1.2%, 2.2%
Swift create()	22.52 µs	21.96 µs	-2.48% faster	1.6%, 0.9%
Swift returnExisting()	209.7 ns	213.5 ns	+1.79% slower	1.9%, 1.0%
Swift typed map (8 entries)	2.25 µs	2.10 µs	-6.75% faster	2.0%, 1.4%
Swift optional trailing string	142.5 ns	143.2 ns	+0.44% slower	1.2%, 0.8%
Swift addNumbers()	69.6 ns	72.7 ns	+4.38% slower	1.7%, 2.2%
Swift simpleFunc()	56.0 ns	56.2 ns	+0.37% slower	0.8%, 0.6%
Swift deferred worker Promise (includes trigger call)	20.01 µs	19.94 µs	-0.35% faster	1.5%, 1.4%
Swift immediatePromise()	1.15 µs	1.12 µs	-2.51% faster	0.9%, 0.9%
Swift number property get/set	130.5 ns	142.4 ns	+9.18% slower	1.1%, 0.8%
Swift short ASCII string	145.3 ns	145.7 ns	+0.28% slower	0.8%, 0.7%
Swift Unicode string	315.3 ns	316.7 ns	+0.44% slower	1.4%, 0.8%
Swift nested Car struct	2.29 µs	2.25 µs	-1.57% faster	1.2%, 1.3%
Swift number	string variant	175.5 ns	179.4 ns	+2.23% slower
TurboModule addNumbers()	898.7 ns	890.2 ns	-0.95% faster	2.9%, 2.0%

p50 is the median of timed batch averages in ns/op, not individual-call latency. MAD describes sample spread; ordered raw samples retain within-process drift.

Android

Benchmark	Base p50	Head p50	Observed change
C++ copy(1 MiB)	20.65 µs	21.94 µs	+6.25% slower
C++ synchronousCallback()	380.1 ns	403.2 ns	+6.06% slower
C++ create()	670.5 ns	618.7 ns	-7.74% faster
C++ simpleFunc()	82.9 ns	132.7 ns	+60.03% slower
C++ short ASCII string	194.2 ns	151.3 ns	-22.08% faster
Kotlin immediatePromise()	2.63 µs	2.44 µs	-7.34% faster

All benchmarks and sample variation

Benchmark	Base p50	Head p50	Observed change
JavaScript addNumbers()	23.5 ns	23.5 ns	-0.37% faster
C++ bounce(1 MiB)	525.1 ns	513.2 ns	-2.26% faster
C++ bounce(4 KiB)	515.7 ns	509.6 ns	-1.19% faster
C++ bounce native-owned buffer (1 MiB)	3.59 µs	3.74 µs	+4.09% slower
C++ bounce native-owned buffer (4 KiB)	476.8 ns	473.0 ns	-0.8% faster
C++ copy(1 MiB)	20.65 µs	21.94 µs	+6.25% slower
C++ copy(4 KiB)	872.1 ns	852.2 ns	-2.29% faster
C++ large array (1,024)	37.45 µs	37.08 µs	-0.97% faster
C++ small array (16)	819.1 ns	812.4 ns	-0.81% faster
C++ synchronousCallback()	380.1 ns	403.2 ns	+6.06% slower
C++ create()	670.5 ns	618.7 ns	-7.74% faster
C++ returnExisting()	236.9 ns	234.5 ns	-1.03% faster
C++ typed map (8 entries)	1.80 µs	1.76 µs	-2.31% faster
C++ optional trailing string	156.7 ns	156.6 ns	-0.07% faster
C++ addNumbers()	97.0 ns	97.3 ns	+0.25% slower
C++ simpleFunc()	82.9 ns	132.7 ns	+60.03% slower
C++ deferred worker Promise (includes trigger call)	24.29 µs	24.56 µs	+1.13% slower
C++ immediatePromise()	925.2 ns	896.2 ns	-3.14% faster
C++ number property get/set	166.5 ns	167.2 ns	+0.47% slower
C++ short ASCII string	194.2 ns	151.3 ns	-22.08% faster
C++ Unicode string	268.0 ns	269.0 ns	+0.39% slower
C++ nested Car struct	2.22 µs	2.29 µs	+2.95% slower
C++ number	string variant	137.4 ns	141.0 ns
Kotlin bounce(1 MiB)	1.36 µs	1.37 µs	+1.13% slower
Kotlin bounce(4 KiB)	1.35 µs	1.37 µs	+1.32% slower
Kotlin bounce native-owned buffer (1 MiB)	4.34 µs	4.32 µs	-0.39% faster
Kotlin bounce native-owned buffer (4 KiB)	1.40 µs	1.40 µs	-0.24% faster
Kotlin copy(1 MiB)	299.84 µs	294.65 µs	-1.73% faster
Kotlin copy(4 KiB)	5.15 µs	5.23 µs	+1.42% slower
Kotlin large array (1,024)	43.04 µs	43.46 µs	+0.99% slower
Kotlin small array (16)	1.10 µs	1.10 µs	+0.09% slower
Kotlin synchronousCallback()	1.30 µs	1.28 µs	-1.61% faster
Kotlin create()	1.85 µs	1.83 µs	-1.18% faster
Kotlin returnExisting()	625.1 ns	632.2 ns	+1.14% slower
Kotlin typed map (8 entries)	7.01 µs	6.95 µs	-0.83% faster
Kotlin optional trailing string	356.5 ns	354.3 ns	-0.62% faster
Kotlin addNumbers()	121.3 ns	124.8 ns	+2.91% slower
Kotlin simpleFunc()	102.8 ns	102.1 ns	-0.73% faster
Kotlin deferred worker Promise (includes trigger call)	49.11 µs	48.31 µs	-1.63% faster
Kotlin immediatePromise()	2.63 µs	2.44 µs	-7.34% faster
Kotlin number property get/set	239.0 ns	247.7 ns	+3.64% slower
Kotlin short ASCII string	478.8 ns	478.7 ns	-0.01% faster
Kotlin Unicode string	785.2 ns	794.1 ns	+1.13% slower
Kotlin nested Car struct	4.92 µs	4.90 µs	-0.43% faster
Kotlin number	string variant	437.0 ns	450.5 ns
TurboModule addNumbers()	256.1 ns	244.6 ns	-4.46% faster

Benchmark	Base process p50	Head process p50	Paired changes	Sample MAD / p50 (base, head)
JavaScript addNumbers()	23.5 ns	23.5 ns	-0.37% faster	0.5%, 0.3%
C++ bounce(1 MiB)	525.1 ns	513.2 ns	-2.26% faster	10.1%, 9.3%
C++ bounce(4 KiB)	515.7 ns	509.6 ns	-1.19% faster	11.9%, 11.7%
C++ bounce native-owned buffer (1 MiB)	3.59 µs	3.74 µs	+4.09% slower	1.3%, 2.4%
C++ bounce native-owned buffer (4 KiB)	476.8 ns	473.0 ns	-0.8% faster	0.9%, 0.6%
C++ copy(1 MiB)	20.65 µs	21.94 µs	+6.25% slower	2.3%, 3.7%
C++ copy(4 KiB)	872.1 ns	852.2 ns	-2.29% faster	3.8%, 3.0%
C++ large array (1,024)	37.45 µs	37.08 µs	-0.97% faster	0.2%, 0.7%
C++ small array (16)	819.1 ns	812.4 ns	-0.81% faster	0.5%, 0.7%
C++ synchronousCallback()	380.1 ns	403.2 ns	+6.06% slower	15.5%, 15.3%
C++ create()	670.5 ns	618.7 ns	-7.74% faster	15.2%, 6.9%
C++ returnExisting()	236.9 ns	234.5 ns	-1.03% faster	1.0%, 0.7%
C++ typed map (8 entries)	1.80 µs	1.76 µs	-2.31% faster	0.6%, 0.7%
C++ optional trailing string	156.7 ns	156.6 ns	-0.07% faster	0.6%, 1.2%
C++ addNumbers()	97.0 ns	97.3 ns	+0.25% slower	0.7%, 0.7%
C++ simpleFunc()	82.9 ns	132.7 ns	+60.03% slower	0.9%, 23.8%
C++ deferred worker Promise (includes trigger call)	24.29 µs	24.56 µs	+1.13% slower	2.7%, 6.4%
C++ immediatePromise()	925.2 ns	896.2 ns	-3.14% faster	1.0%, 1.0%
C++ number property get/set	166.5 ns	167.2 ns	+0.47% slower	1.0%, 0.6%
C++ short ASCII string	194.2 ns	151.3 ns	-22.08% faster	2.1%, 0.3%
C++ Unicode string	268.0 ns	269.0 ns	+0.39% slower	1.5%, 0.8%
C++ nested Car struct	2.22 µs	2.29 µs	+2.95% slower	0.9%, 2.0%
C++ number	string variant	137.4 ns	141.0 ns	+2.63% slower
Kotlin bounce(1 MiB)	1.36 µs	1.37 µs	+1.13% slower	13.4%, 4.9%
Kotlin bounce(4 KiB)	1.35 µs	1.37 µs	+1.32% slower	3.9%, 4.9%
Kotlin bounce native-owned buffer (1 MiB)	4.34 µs	4.32 µs	-0.39% faster	2.0%, 1.1%
Kotlin bounce native-owned buffer (4 KiB)	1.40 µs	1.40 µs	-0.24% faster	7.7%, 7.9%
Kotlin copy(1 MiB)	299.84 µs	294.65 µs	-1.73% faster	3.6%, 2.2%
Kotlin copy(4 KiB)	5.15 µs	5.23 µs	+1.42% slower	6.1%, 4.0%
Kotlin large array (1,024)	43.04 µs	43.46 µs	+0.99% slower	1.0%, 0.6%
Kotlin small array (16)	1.10 µs	1.10 µs	+0.09% slower	0.7%, 1.0%
Kotlin synchronousCallback()	1.30 µs	1.28 µs	-1.61% faster	6.0%, 3.4%
Kotlin create()	1.85 µs	1.83 µs	-1.18% faster	5.4%, 2.8%
Kotlin returnExisting()	625.1 ns	632.2 ns	+1.14% slower	0.2%, 0.9%
Kotlin typed map (8 entries)	7.01 µs	6.95 µs	-0.83% faster	0.5%, 0.3%
Kotlin optional trailing string	356.5 ns	354.3 ns	-0.62% faster	0.6%, 0.9%
Kotlin addNumbers()	121.3 ns	124.8 ns	+2.91% slower	0.7%, 0.7%
Kotlin simpleFunc()	102.8 ns	102.1 ns	-0.73% faster	0.9%, 0.8%
Kotlin deferred worker Promise (includes trigger call)	49.11 µs	48.31 µs	-1.63% faster	1.9%, 2.4%
Kotlin immediatePromise()	2.63 µs	2.44 µs	-7.34% faster	1.6%, 1.8%
Kotlin number property get/set	239.0 ns	247.7 ns	+3.64% slower	0.7%, 0.2%
Kotlin short ASCII string	478.8 ns	478.7 ns	-0.01% faster	0.8%, 0.7%
Kotlin Unicode string	785.2 ns	794.1 ns	+1.13% slower	1.0%, 0.5%
Kotlin nested Car struct	4.92 µs	4.90 µs	-0.43% faster	0.5%, 0.6%
Kotlin number	string variant	437.0 ns	450.5 ns	+3.1% slower
TurboModule addNumbers()	256.1 ns	244.6 ns	-4.46% faster	0.2%, 0.3%

p50 is the median of timed batch averages in ns/op, not individual-call latency. MAD describes sample spread; ordered raw samples retain within-process drift.

Benchmarking Code Diff 3303893f...16367b7b (view CI run)

Raw measurements: performance-report-2 (JSON artifact). Run 34117178949, attempt 2. Download requires GitHub access.

Android: measurements, attempt 2, apps, attempt 2.
iOS: measurements, attempt 2, apps, attempt 2.