Skip to content

Use SWAR + VarHandle for 7-bit binary Smile encode/decode #774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cowtowncoder merged 3 commits into from
Sep 9, 2026
Merged

Use SWAR + VarHandle for 7-bit binary Smile encode/decode #774

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9c8d225
Use SWAR + VarHandle for Smile 7-bit binary encode/decode
pjfanning Sep 7, 2026
34347ef
Merge branch '3.x' into smile-7bit-binary-swar
cowtowncoder Sep 9, 2026
ac5e7ad
Update javadoc for `SmileByteShiftUtil` removal
cowtowncoder Sep 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions smile/src/main/java/tools/jackson/dataformat/smile/Smile7BitBinaryCodec.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
package tools.jackson.dataformat.smile;

/**
* Helper for Smile's "7-bit safe" binary encoding, which packs 7 payload bytes
* into 8 output bytes of 7 significant bits each (see
* {@link SmileWriteFeature#ENCODE_BINARY_AS_7BIT}, enabled by default).
*<p>
* Both directions are done with SWAR bit manipulation over a single 8-byte load
* and a single 8-byte store, replacing the byte-at-a-time shifting the callers
* would otherwise do. The load and store go through {@link SmileVarHandleUtil};
* where that is unusable this deliberately does NOT fall back to
* {@link tools.jackson.core.util.ByteArrayUtil}: composing the 8-byte load and
* store out of shifts costs more per chunk than the per-byte loop it would be
* replacing (measured on JDK 17, 0.79x on encode and 0.72x on decode, i.e. a
* slowdown), so both methods simply report failure and the caller runs its own
* loop, which stays the faster path there.
*<p>
* Note that the SWAR forms mask each input byte to 7 bits. For well-formed Smile
* content, where the high bit is always clear, that is a no-op. For corrupt
* content the two paths can differ -- but so does the value they decode to,
* which is meaningless either way.
*
* @since 3.3
*/
final class Smile7BitBinaryCodec
{
/**
* Whether {@code VarHandle}-based array access is usable on this runtime;
* probed once at class load, using the same pattern as
* {@code SmileParserBase._decodeQuad()}.
*/
private final static boolean VARHANDLE_AVAILABLE = _checkVarHandleAvailable();

private static boolean _checkVarHandleAvailable() {
// NOTE: this call is what first loads `SmileVarHandleUtil`, and that class
// names `VarHandle` in its field/method signatures. On a runtime without
// `java.lang.invoke.VarHandle` (some Android builds) loading it raises
// `NoClassDefFoundError` -- an Error, not an Exception -- so `Throwable`
// is what has to be caught here.
try {
return SmileVarHandleUtil.isAvailable();
} catch (Throwable t) {
return false;
}
}

private Smile7BitBinaryCodec() { }

/**
* Decodes 8 encoded bytes at {@code inPtr} into the 7 payload bytes they
* represent, writing them at {@code outPtr}.
*<p>
* The store writes a full 8 bytes: the 7 wanted ones plus a trailing zero.
* That byte is either overwritten by the next chunk or lies past the end of
* the decoded content, but it still has to be inside {@code out} -- hence
* the bounds check, which also rejects a final chunk that ends flush with
* the output array.
*
* @return {@code true} if the 7 bytes were written; {@code false} if the
* caller must decode this chunk itself
*/
static boolean decodeChunk(byte[] in, int inPtr, byte[] out, int outPtr)
{
if (!VARHANDLE_AVAILABLE
|| ((inPtr + 8) > in.length) || ((outPtr + 8) > out.length)) {
return false;
}
long v = SmileVarHandleUtil.getLongBE(in, inPtr);
// Compact 8 x 7 bits down to 56, doubling the field width each round
v = ((v & 0x7F007F007F007F00L) >>> 1) | (v & 0x007F007F007F007FL);
v = ((v & 0x3FFF00003FFF0000L) >>> 2) | (v & 0x00003FFF00003FFFL);
v = ((v & 0x0FFFFFFF00000000L) >>> 4) | (v & 0x0FFFFFFFL);
// Left-align the 56 bits so they land in the first 7 bytes written
SmileVarHandleUtil.setLongBE(out, outPtr, v << 8);
return true;
}

/**
* Encodes the 7 payload bytes at {@code inPtr} as the 8 bytes of 7
* significant bits they become, writing them at {@code outPtr}.
*<p>
* The load reads a full 8 bytes and discards the last, so -- as in
* {@link #decodeChunk} -- the extra byte has to be inside {@code in}.
*
* @return {@code true} if the 8 bytes were written; {@code false} if the
* caller must encode this chunk itself
*/
static boolean encodeChunk(byte[] in, int inPtr, byte[] out, int outPtr)
{
if (!VARHANDLE_AVAILABLE
|| ((inPtr + 8) > in.length) || ((outPtr + 8) > out.length)) {
return false;
}
// Drop the 8th byte: we want the 56 bits of the 7 payload bytes
long v = SmileVarHandleUtil.getLongBE(in, inPtr) >>> 8;
// Spread 56 bits back out into 8 x 7, halving the field width each round
v = ((v & (0x0FFFFFFFL << 28)) << 4) | (v & 0x0FFFFFFFL);
v = ((v & 0x0FFFC0000FFFC000L) << 2) | (v & 0x00003FFF00003FFFL);
v = ((v & 0x3F803F803F803F80L) << 1) | (v & 0x007F007F007F007FL);
SmileVarHandleUtil.setLongBE(out, outPtr, v);
return true;
}
}
70 changes: 40 additions & 30 deletions smile/src/main/java/tools/jackson/dataformat/smile/SmileGenerator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2297,21 +2297,26 @@ protected void _write7BitBinaryWithLength(byte[] data, int offset, int len) thro
if ((_outputTail + 8) >= _outputEnd) {
_flushBuffer();
}
int i = data[offset++]; // 1st byte
_outputBuffer[_outputTail++] = (byte) ((i >> 1) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 2nd
_outputBuffer[_outputTail++] = (byte) ((i >> 2) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 3rd
_outputBuffer[_outputTail++] = (byte) ((i >> 3) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 4th
_outputBuffer[_outputTail++] = (byte) ((i >> 4) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 5th
_outputBuffer[_outputTail++] = (byte) ((i >> 5) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 6th
_outputBuffer[_outputTail++] = (byte) ((i >> 6) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 7th
_outputBuffer[_outputTail++] = (byte) ((i >> 7) & 0x7F);
_outputBuffer[_outputTail++] = (byte) (i & 0x7F);
if (Smile7BitBinaryCodec.encodeChunk(data, offset, _outputBuffer, _outputTail)) {
offset += 7;
_outputTail += 8;
} else {
int i = data[offset++]; // 1st byte
_outputBuffer[_outputTail++] = (byte) ((i >> 1) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 2nd
_outputBuffer[_outputTail++] = (byte) ((i >> 2) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 3rd
_outputBuffer[_outputTail++] = (byte) ((i >> 3) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 4th
_outputBuffer[_outputTail++] = (byte) ((i >> 4) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 5th
_outputBuffer[_outputTail++] = (byte) ((i >> 5) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 6th
_outputBuffer[_outputTail++] = (byte) ((i >> 6) & 0x7F);
i = (i << 8) | (data[offset++] & 0xFF); // 7th
_outputBuffer[_outputTail++] = (byte) ((i >> 7) & 0x7F);
_outputBuffer[_outputTail++] = (byte) (i & 0x7F);
}
len -= 7;
}
// and then partial piece, if any
Expand Down Expand Up @@ -2378,21 +2383,26 @@ protected int _write7BitBinaryWithLength(InputStream in, int bytesLeft, byte[] b
if ((_outputTail + 8) >= _outputEnd) {
_flushBuffer();
}
int i = buffer[inputPtr++]; // 1st byte
_outputBuffer[_outputTail++] = (byte) ((i >> 1) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 2nd
_outputBuffer[_outputTail++] = (byte) ((i >> 2) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 3rd
_outputBuffer[_outputTail++] = (byte) ((i >> 3) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 4th
_outputBuffer[_outputTail++] = (byte) ((i >> 4) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 5th
_outputBuffer[_outputTail++] = (byte) ((i >> 5) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 6th
_outputBuffer[_outputTail++] = (byte) ((i >> 6) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 7th
_outputBuffer[_outputTail++] = (byte) ((i >> 7) & 0x7F);
_outputBuffer[_outputTail++] = (byte) (i & 0x7F);
if (Smile7BitBinaryCodec.encodeChunk(buffer, inputPtr, _outputBuffer, _outputTail)) {
inputPtr += 7;
_outputTail += 8;
} else {
int i = buffer[inputPtr++]; // 1st byte
_outputBuffer[_outputTail++] = (byte) ((i >> 1) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 2nd
_outputBuffer[_outputTail++] = (byte) ((i >> 2) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 3rd
_outputBuffer[_outputTail++] = (byte) ((i >> 3) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 4th
_outputBuffer[_outputTail++] = (byte) ((i >> 4) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 5th
_outputBuffer[_outputTail++] = (byte) ((i >> 5) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 6th
_outputBuffer[_outputTail++] = (byte) ((i >> 6) & 0x7F);
i = (i << 8) | (buffer[inputPtr++] & 0xFF); // 7th
_outputBuffer[_outputTail++] = (byte) ((i >> 7) & 0x7F);
_outputBuffer[_outputTail++] = (byte) (i & 0x7F);
}
bytesLeft -= 7;
}

Expand Down
125 changes: 70 additions & 55 deletions smile/src/main/java/tools/jackson/dataformat/smile/SmileParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1561,24 +1561,29 @@ private void _readBinaryEncoded(OutputStream out, int length, byte[] buffer) thr
if ((_inputEnd - _inputPtr) < 8) {
_loadToHaveAtLeast(8);
}
int i1 = (_inputBuffer[_inputPtr++] << 25)
+ (_inputBuffer[_inputPtr++] << 18)
+ (_inputBuffer[_inputPtr++] << 11)
+ (_inputBuffer[_inputPtr++] << 4);
int x = _inputBuffer[_inputPtr++];
i1 += x >> 3;
int i2 = ((x & 0x7) << 21)
+ (_inputBuffer[_inputPtr++] << 14)
+ (_inputBuffer[_inputPtr++] << 7)
+ _inputBuffer[_inputPtr++];
// Ok: got our 7 bytes, just need to split, copy
buffer[outPtr++] = (byte)(i1 >> 24);
buffer[outPtr++] = (byte)(i1 >> 16);
buffer[outPtr++] = (byte)(i1 >> 8);
buffer[outPtr++] = (byte)i1;
buffer[outPtr++] = (byte)(i2 >> 16);
buffer[outPtr++] = (byte)(i2 >> 8);
buffer[outPtr++] = (byte)i2;
if (Smile7BitBinaryCodec.decodeChunk(_inputBuffer, _inputPtr, buffer, outPtr)) {
_inputPtr += 8;
outPtr += 7;
} else {
int i1 = (_inputBuffer[_inputPtr++] << 25)
+ (_inputBuffer[_inputPtr++] << 18)
+ (_inputBuffer[_inputPtr++] << 11)
+ (_inputBuffer[_inputPtr++] << 4);
int x = _inputBuffer[_inputPtr++];
i1 += x >> 3;
int i2 = ((x & 0x7) << 21)
+ (_inputBuffer[_inputPtr++] << 14)
+ (_inputBuffer[_inputPtr++] << 7)
+ _inputBuffer[_inputPtr++];
// Ok: got our 7 bytes, just need to split, copy
buffer[outPtr++] = (byte)(i1 >> 24);
buffer[outPtr++] = (byte)(i1 >> 16);
buffer[outPtr++] = (byte)(i1 >> 8);
buffer[outPtr++] = (byte)i1;
buffer[outPtr++] = (byte)(i2 >> 16);
buffer[outPtr++] = (byte)(i2 >> 8);
buffer[outPtr++] = (byte)i2;
}
length -= 7;
// ensure there's always room for at least 7 bytes more after looping:
if (outPtr > lastSafeOut) {
Expand Down Expand Up @@ -2973,24 +2978,29 @@ private final byte[] _read7BitBinaryWithLength() throws JacksonException
_reportIncompleteBinaryRead7Bit(byteLen, ptr);
}
}
int i1 = (_inputBuffer[_inputPtr++] << 25)
+ (_inputBuffer[_inputPtr++] << 18)
+ (_inputBuffer[_inputPtr++] << 11)
+ (_inputBuffer[_inputPtr++] << 4);
int x = _inputBuffer[_inputPtr++];
i1 += x >> 3;
int i2 = ((x & 0x7) << 21)
+ (_inputBuffer[_inputPtr++] << 14)
+ (_inputBuffer[_inputPtr++] << 7)
+ _inputBuffer[_inputPtr++];
// Ok: got our 7 bytes, just need to split, copy
result[ptr++] = (byte)(i1 >> 24);
result[ptr++] = (byte)(i1 >> 16);
result[ptr++] = (byte)(i1 >> 8);
result[ptr++] = (byte)i1;
result[ptr++] = (byte)(i2 >> 16);
result[ptr++] = (byte)(i2 >> 8);
result[ptr++] = (byte)i2;
if (Smile7BitBinaryCodec.decodeChunk(_inputBuffer, _inputPtr, result, ptr)) {
_inputPtr += 8;
ptr += 7;
} else {
int i1 = (_inputBuffer[_inputPtr++] << 25)
+ (_inputBuffer[_inputPtr++] << 18)
+ (_inputBuffer[_inputPtr++] << 11)
+ (_inputBuffer[_inputPtr++] << 4);
int x = _inputBuffer[_inputPtr++];
i1 += x >> 3;
int i2 = ((x & 0x7) << 21)
+ (_inputBuffer[_inputPtr++] << 14)
+ (_inputBuffer[_inputPtr++] << 7)
+ _inputBuffer[_inputPtr++];
// Ok: got our 7 bytes, just need to split, copy
result[ptr++] = (byte)(i1 >> 24);
result[ptr++] = (byte)(i1 >> 16);
result[ptr++] = (byte)(i1 >> 8);
result[ptr++] = (byte)i1;
result[ptr++] = (byte)(i2 >> 16);
result[ptr++] = (byte)(i2 >> 8);
result[ptr++] = (byte)i2;
}
}
// and then leftovers: n+1 bytes to decode n bytes
int toDecode = (result.length - ptr);
Expand Down Expand Up @@ -3032,25 +3042,30 @@ protected byte[] _finishBinary7BitLong(final int expLen) throws JacksonException
_reportIncompleteBinaryRead7Bit(expLen, bb.size() + bufPtr);
}
}
int i1 = (_inputBuffer[_inputPtr++] << 25)
+ (_inputBuffer[_inputPtr++] << 18)
+ (_inputBuffer[_inputPtr++] << 11)
+ (_inputBuffer[_inputPtr++] << 4);
int x = _inputBuffer[_inputPtr++];
i1 += x >> 3;
int i2 = ((x & 0x7) << 21)
+ (_inputBuffer[_inputPtr++] << 14)
+ (_inputBuffer[_inputPtr++] << 7)
+ _inputBuffer[_inputPtr++];
// Ok: got our 7 bytes, just need to split, copy
// NOTE: lgtm cannot deduce the checks but a single bounds check IS enough here
buffer[bufPtr++] = (byte)(i1 >> 24);
buffer[bufPtr++] = (byte)(i1 >> 16); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)(i1 >> 8); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)i1; // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)(i2 >> 16); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)(i2 >> 8); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)i2; // lgtm [java/index-out-of-bounds]
if (Smile7BitBinaryCodec.decodeChunk(_inputBuffer, _inputPtr, buffer, bufPtr)) {
_inputPtr += 8;
bufPtr += 7;
} else {
int i1 = (_inputBuffer[_inputPtr++] << 25)
+ (_inputBuffer[_inputPtr++] << 18)
+ (_inputBuffer[_inputPtr++] << 11)
+ (_inputBuffer[_inputPtr++] << 4);
int x = _inputBuffer[_inputPtr++];
i1 += x >> 3;
int i2 = ((x & 0x7) << 21)
+ (_inputBuffer[_inputPtr++] << 14)
+ (_inputBuffer[_inputPtr++] << 7)
+ _inputBuffer[_inputPtr++];
// Ok: got our 7 bytes, just need to split, copy
// NOTE: lgtm cannot deduce the checks but a single bounds check IS enough here
buffer[bufPtr++] = (byte)(i1 >> 24);
buffer[bufPtr++] = (byte)(i1 >> 16); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)(i1 >> 8); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)i1; // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)(i2 >> 16); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)(i2 >> 8); // lgtm [java/index-out-of-bounds]
buffer[bufPtr++] = (byte)i2; // lgtm [java/index-out-of-bounds]
}
if (bufPtr >= buffer.length) {
bb.write(buffer, 0, bufPtr);
bufPtr = 0;
Expand Down
Loading
Loading