Use XJB algorithm for writing Floats/Doubles by pjfanning · Pull Request #1656 · FasterXML/jackson-core

pjfanning · 2026-08-12T09:57:37Z

See No-allocation DoubleToDecimal / FloatToDecimal #1652
Aimed as a replacement for the Schubfach code
One of the main benefits is that the XJBWriter can avoid String allocations if you provide the byte or char array to write to
benchmarked in https://github.com/pjfanning/double-reader-writer - similar toString as Schubfach but the fact that you can get it to write directly to a byte array makes it a very useful addition for the Utf8JsonGenerator - this saves on String and array allocations and makes a big contribution to the performance improvement
has been enhanced to support writing directly to char arrays too
Android doesn't support VarHandles so we end up with some messy Reflection. For me the better solution is to take advantage of the fact that jackson-core is already a Multi-Release Jar because FastDoubleParser embedded code needs that. Android ignores the classes in META-INF/versions, so we could put the VarHandle supporting code in a class in META-INF/versions/17. JVM users will get to use that version while Android users will use the fallback version that is in the main code.

pjfanning · 2026-08-15T12:29:00Z

@plokhotnyuk feel free to ignore this but if you time, it would be nice to get your input on the XJBWriter. I added support for writing to char arrays as well as byte arrays because this is useful in some Jackson code paths. The char stream code may not be as heavily optimised as the byte stream path.

plokhotnyuk · 2026-08-15T13:09:42Z

@plokhotnyuk feel free to ignore this but if you time, it would be nice to get your input on the XJBWriter. I added support for writing to char arrays as well as byte arrays because this is useful in some Jackson code paths. The char stream code may not be as heavily optimized as the byte stream path.

Great work! Have you used AI agents or translated it manually to Java?

The next step could be using James Anhalt's algorithm for faster writing of mantissas: https://jk-jeon.github.io/posts/2022/02/jeaiii-algorithm/

Or you can leverage 3 digit LUT table to reduce cycles as it is done for integer numbers.

pjfanning · 2026-08-15T13:41:13Z

@plokhotnyuk mainly AI agent work but it was pretty iterative so possibly not a great time saver in this case.

pjfanning · 2026-09-05T12:39:12Z

@cowtowncoder if this does get merged in time for 3.3.0, the CSV module could also benefit by uptaking the new NumberOutput APIs that this PR adds.

cowtowncoder · 2026-09-08T04:54:55Z

@pjfanning Hoping to get to this next.

cowtowncoder · 2026-09-11T20:43:53Z

Hmmmh. I have some concerns based on Claude-assisted code review.

First is LICENSE:

  1. License and NOTICE
     - META-INF/jackson-core-NOTICE still describes Schubfach, and Schubfach-LICENSE is still bundled even though that code is removed.
     - There is no license text or NOTICE entry for jsoniter-scala (MIT, Andriy Plokhotnyuk), which the code is ported from. MIT requires shipping its copyright and permission notice.
     - If any of it traces back to the xjb reference code (Apache 2.0), that license and notice need to ship too.

which may be easy to address. Will add other notes too, but first things first.

cowtowncoder · 2026-09-11T20:48:01Z

Second: does XJB require use of VarHandles? If not, maybe first version could just use added ByteArrayUtil access methods, and we could consider move to VarHandles (and class-loading issues) separately?

HOWEVER, also: Claude claims:

 Tests pass and the speed regression is gone with direct VarHandle usage, but animal-sniffer flags the
  signature-polymorphic set/get calls as a false positive since VarHandle itself is available in Android 33+.
  I'll add a narrow ignore rule scoped just to java.lang.invoke.VarHandle, without excluding source or ignoring
  MethodHandle. 

...

 VarHandle is part of the Android
    API from 33, but animal-sniffer can't resolve calls like VarHandle.set(byte[], int, int) and reports them as missing.

pjfanning · 2026-09-11T21:02:27Z

@cowtowncoder AnimalSniffer failures are what sent me down the route of using indirection for the VarHandles. The docs that I found suggest that Android should support VarHandles at the version we support but AnimalSniffer doesn't appear to agree.

pjfanning · 2026-09-11T21:20:27Z

Hmmmh. I have some concerns based on Claude-assisted code review.

First is LICENSE:

  1. License and NOTICE
     - META-INF/jackson-core-NOTICE still describes Schubfach, and Schubfach-LICENSE is still bundled even though that code is removed.
     - There is no license text or NOTICE entry for jsoniter-scala (MIT, Andriy Plokhotnyuk), which the code is ported from. MIT requires shipping its copyright and permission notice.
     - If any of it traces back to the xjb reference code (Apache 2.0), that license and notice need to ship too.

which may be easy to address. Will add other notes too, but first things first.

@cowtowncoder I updated the licenses in META-INF - c729911

pjfanning · 2026-09-11T21:29:11Z

@cowtowncoder I'd prefer to keep the VarHandles as they really help the performance. For instance, we make use of them in the binary dataformats now.
I updated the PR to use ByteArrayUtil for the fallback when VarHandles are not available in the JRE.
b5d5d61

cowtowncoder · 2026-09-11T21:55:01Z

@pjfanning Ok. Yes, I know about binary (and some textual?) backends wrt recent PRs.

One last concern code review mentioned: some of serializations change for numbers -- not in wrong way (they still round-trip), but differ from what JDK produces.
Specifically for something called "smallest subnormals".

 With the feature enabled, comparing XJB against the Schubfach code it replaces, I found 17 differing values
  out of 40M random plus targeted ranges. All of them are the smallest subnormals:

  ┌──────────────────┬──────────────────────────────────────────┬──────────────────────────────────────────┐
  │       Bits       │                XJB (new)                 │             Schubfach (3.2)              │
  ├──────────────────┼──────────────────────────────────────────┼──────────────────────────────────────────┤
  │ double 0x1 …     │ 5.0E-324, 1.0E-323, 5.0E-323, 6.0E-323,  │ 4.9E-324, 9.9E-324, 4.9E-323, 5.9E-323,  │
  │ 0x14 (8 values)  │ 7.0E-323, 8.0E-323, 9.0E-323, 1.0E-322   │ 6.9E-323, 7.9E-323, 8.9E-323, 9.9E-323   │
  ├──────────────────┼──────────────────────────────────────────┼──────────────────────────────────────────┤
  │ float 0x1 … 0x47 │ 1.0E-45, 3.0E-45, 4.0E-45, 6.0E-45,      │ 1.4E-45, 2.8E-45, 4.2E-45, 5.6E-45,      │
  │  (9 values)      │ 8.0E-45, 1.0E-44, 3.0E-44, 4.0E-44,      │ 8.4E-45, 9.8E-45, 2.9E-44, 4.1E-44,      │
  │                  │ 1.0E-43                                  │ 9.9E-44                                  │
  └──────────────────┴──────────────────────────────────────────┴──────────────────────────────────────────┘

  The cause is that XJB emits the shortest decimal that round-trips, while the JDK and Schubfach add a second
  digit when the shortest has only one.

This may be fine, just wanted to mention; at least it can trigger test failures downstream.

pjfanning · 2026-09-11T22:02:18Z

@cowtowncoder The writer is opt-in but I understand the worry about the different output for some numbers.

@plokhotnyuk what do you think about the issue described in #1656 (comment) ?

plokhotnyuk · 2026-09-12T03:24:15Z

@plokhotnyuk what do you think about the issue described in #1656 (comment) ?

Originally XJB formats without dot when mantissa is just one digit (for instance 1e-45 - the shortest representation).

But for Java there is a formatting rule that dot and at least one digit after are required, so that why 1.0E-45 reappears like on 17 JDK.

cowtowncoder · 2026-09-12T03:52:23Z

@pjfanning Yes, opt-in is probably sufficient to allow change.

- Add char[] overloads to XJBWriter (writeFloat, writeDouble) - Add MAX_FLOAT_CHARS and MAX_DOUBLE_CHARS constants to NumberOutput - Add outputFloat and outputDouble char[] methods to NumberOutput - Update WriterBasedJsonGenerator to use char[] fast path - Add NumberOutputFloatDoubleTest and XJBWriterCharTest test files

XJBWriter throws ArithmeticException for non-finite values. When WRITE_NAN_AS_STRINGS is disabled, NaN/Infinity must still be written as raw strings (not quoted). Check Double.isFinite/Float.isFinite before calling the fast path, falling back to toString() for non-finite values.

- Add non-finite checks using bits already available (e2IEEE == 0xFF/0x7FF) - NaN writes 'NaN', Infinity writes 'Infinity'/'-Infinity' to buffer - Remove Double.isFinite/Float.isFinite checks from generators - Remove now-unused illegalNumberError methods - Update tests to expect correct output instead of ArithmeticException

writeFloat/writeDouble now handle NaN/Infinity directly, so toString no longer needs its own checks.

write3Digits used a 4-byte setInt store for the exponent, the last write of the longest output (-2.2250738585072014E-308, 24 chars), so it touched 25 bytes. UTF8JsonGenerator reserves exactly MAX_DOUBLE_BYTES, giving an AIOOBE when the value landed 24 bytes from the end of the output buffer. Write exactly 3 bytes instead. Also size the toString() scratch buffers from MAX_*_BYTES rather than MAX_*_CHARS, and refresh the NumberOutput javadoc: non-finite values are handled now, and the notFinite links / param types on the output* overloads were wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qqdq57uPQx3jxcg3WsoUiq

…s (24/15) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qqdq57uPQx3jxcg3WsoUiq

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qqdq57uPQx3jxcg3WsoUiq

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…ssors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

github-actions · 2026-09-14T16:19:16Z

📈 Overall Code Coverage

Metric	Coverage	Change
Instructions	84.72%	📈 +0.300%
Branches	77.83%	📉 -0.010%

Overall project coverage from JaCoCo test results. Change values compare against the latest base branch build.

pjfanning mentioned this pull request Aug 12, 2026

No-allocation DoubleToDecimal / FloatToDecimal #1652

Open

pjfanning marked this pull request as draft August 12, 2026 10:14

pjfanning mentioned this pull request Aug 12, 2026

UTF8JsonGenerator: writeNumber fast path that avoids extra String allocation #1657

Merged

pjfanning marked this pull request as ready for review August 14, 2026 17:21

pjfanning changed the title ~~[WIP] use XJB algorithm for writing Floats/Doubles~~ use XJB algorithm for writing Floats/Doubles Aug 14, 2026

pjfanning force-pushed the xjb branch 2 times, most recently from 7d395b3 to 34c3829 Compare August 14, 2026 22:02

pjfanning mentioned this pull request Aug 22, 2026

Use SWAR in SmileParser FasterXML/jackson-dataformats-binary#757

Merged

pjfanning force-pushed the xjb branch from 9cb5b74 to 296a45f Compare September 5, 2026 00:48

cowtowncoder changed the title ~~use XJB algorithm for writing Floats/Doubles~~ Use XJB algorithm for writing Floats/Doubles Sep 8, 2026

pjfanning mentioned this pull request Sep 11, 2026

Add NumberOutput.outputDouble/outputFloat(char[]), use in WriterBasedJsonGenerator #1704

Merged

pjfanning added 5 commits September 14, 2026 17:06

start work on java port of xjb algorithm

dc285ae

refactor and add tests

58aa1da

varhandle

0c5017e

Update XJBWriter.java

58903b6

android issue

56d5d79

pjfanning and others added 21 commits September 14, 2026 17:06

Update pom.xml

4853e8f

Update NumberOutput.java

3acfa8c

add tests

23580a7

remove schubfach code

dfcd26d

Update license information in package-info.java

0b20f4e

Update NumberOutput.java

77e2f86

rename test

a7c0397

fix tests

5bb1038

Update pom.xml

ef0d340

javadoc

8a5bad8

Update StreamWriteFeaturesTest.java

779a64d

Remove redundant NaN/Infinity checks from XJBWriter.toString

3da6211

writeFloat/writeDouble now handle NaN/Infinity directly, so toString no longer needs its own checks.

Add release notes for FasterXML#1652

5e7a9e9

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qqdq57uPQx3jxcg3WsoUiq

Tighten NumberOutput.MAX_DOUBLE_CHARS/MAX_FLOAT_CHARS to actual bound…

4e94bf8

…s (24/15) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qqdq57uPQx3jxcg3WsoUiq

Update USE_FAST_DOUBLE_WRITER and XJBWriter Javadoc for XJB

677c387

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qqdq57uPQx3jxcg3WsoUiq

Replace Schubfach license with jsoniter-scala and xjb licenses/notices

bd47d9b

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Use ByteArrayUtil as XJBWriter non-VarHandle fallback; add short acce…

b269c08

…ssors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

pjfanning force-pushed the xjb branch from dedaf23 to b269c08 Compare September 14, 2026 16:17

Uh oh!

Uh oh!

Conversation

pjfanning commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pjfanning commented Aug 15, 2026

Uh oh!

plokhotnyuk commented Aug 15, 2026

Uh oh!

pjfanning commented Aug 15, 2026

Uh oh!

pjfanning commented Sep 5, 2026

Uh oh!

cowtowncoder commented Sep 8, 2026

Uh oh!

cowtowncoder commented Sep 11, 2026

Uh oh!

cowtowncoder commented Sep 11, 2026

Uh oh!

pjfanning commented Sep 11, 2026

Uh oh!

pjfanning commented Sep 11, 2026

Uh oh!

pjfanning commented Sep 11, 2026

Uh oh!

cowtowncoder commented Sep 11, 2026

Uh oh!

pjfanning commented Sep 11, 2026

Uh oh!

plokhotnyuk commented Sep 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cowtowncoder commented Sep 12, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

📈 Overall Code Coverage

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pjfanning commented Aug 12, 2026 •

edited

Loading

plokhotnyuk commented Sep 12, 2026 •

edited

Loading