You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary
Adds ZSTD block-compression support to client-v2 (and by extension jdbc-v2), fixing issue #3105 where every compressed read fails against ClickHouse 26.9+ because the server's default network codec switched from LZ4 to ZSTD(3) and the client's ClickHouseLZ4InputStream hard-asserted the LZ4 magic byte. The old LZ4-only stream classes are renamed/generalized (CompressedBlockInputStream, CompressedBlockOutputStream with LZ4/ZSTD subclasses, CompressedBlockEntity): the input stream now auto-detects the codec per block header (LZ4 0x82, ZSTD 0x90, NONE 0x02), and the output side selects a codec from a new compression.method config (new CompressionMethod enum, new Client.Builder.compressionMethod() API) whose default is ZSTD. zstd-jni is promoted from test to compile scope (with native libs bundled into the -all JDBC package). Numerous tests across clickhouse-client, clickhouse-http-client, clickhouse-jdbc, and client-v2 are updated/added, several pinning network_compression_method=lz4 for legacy paths.
What this impacts
client-v2 compressed block wire path — both decompression (all compressed reads) and compression (all compressed inserts) for every user with default settings.
Public API surface: new CompressionMethod enum, new Client.Builder.compressionMethod(), new compression.method config key.
Packaging: new runtime dependency zstd-jni 1.5.7-20 (cloud classifier) with native libraries added to the -all JDBC artifact.
Test suites in clickhouse-client / http-client / jdbc / client-v2 adjusted for the new server default codec.
Concerns
High rules fired: changes client-v2 block reader/writer (wire (de)serialization layout handling) and the diff is well over 400 lines (1146+/641−) — consider reviewing the stream rewrite separately from the test churn, or splitting.
Default outgoing block compression changes from LZ4 to ZSTD (COMPRESSION_METHOD defaults to ZSTD) — a changed default that affects all inserts against older servers; no LZ4-default fallback flag beyond manually setting the new option.
Possible buffer-sizing regression: CompressedBlockOutputStream now allocates the compressed buffer as bufferSize + 25, dropping the previous compressor.maxCompressedLength(bufferSize) + header sizing — incompressible input can exceed bufferSize for both LZ4 and ZSTD and overflow/fail; deserves explicit review or a test with incompressible data.
New runtime native-code dependency (zstd-jni) bundled into -all — platform/size implications for consumers.
Minor intent drift: title says jdbc-v2 but no jdbc-v2 sources change (only clickhouse-jdbc tests); checklist items in the PR body are unchecked.
CI checks (Compile JDK 8, JDBC driver matrix, SonarCloud) were still pending at triage time — verify green before merge (not a risk-promoting factor).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
chernser
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Summary
Default block compression method on ClickHouse was switched from
LZ4toZSTD. This caused client to fail as it lacks new method support. One of workarounds was settingnetwork_compress_method(see https://clickhouse.com/docs/reference/settings/session-settings/network#network_compression_method) but is not possible for read-only user profiles.Now client detects compression method set in block header and (de)compresses it properly.
Closes #3105
Checklist
Delete items not relevant to your PR: