[Feature Request]: Add read() support to Java ClickHouseIO (currently write-only) #40048

What would you like to happen?

Motivation

The Java ClickHouseIO connector currently supports writing data to ClickHouse but does not provide a corresponding read transform.

Adding native read support would allow Apache Beam pipelines to consume ClickHouse data using Beam's standard I/O abstractions. Since ClickHouse is designed for large analytical workloads, the initial implementation should support distributed parallel reads rather than requiring all data to pass through a single Beam reader.

Proposal (v1)

Add ClickHouseIO.read() with the following capabilities:

• Bounded reads using SQL SELECT queries.
• Parallel reads across independently assigned, non-overlapping data ranges.
• Configurable partition column, range boundaries, and number of read partitions.
• Native Beam Row output using a caller-supplied Beam Schema, with explicit ClickHouse-to-Beam type mappings for an initially supported subset of types.
• Support for a user-supplied SQL query in single-reader mode and a table-oriented API for partitioned reads.
• Reuse of the existing ClickHouseIO connection configuration and Java client where practical.
• Incremental result processing without buffering the entire result set in memory.
• Connection lifecycle management and error handling.
• Unit tests, integration tests, Javadocs, and usage examples.

Proposed read architecture

For parallel reads, the connector would divide a configured range into non-overlapping partitions and distribute those partitions across Beam workers.

Each worker independently executes a bounded ClickHouse query for its assigned partition and emits Beam Row elements.

The initial partitioning approach would use a supported partition column and explicitly configured bounds. Automatic partition discovery and shard-aware optimization can follow later.

The implementation must document partition coverage, null handling, data consistency, and retry semantics.

Non-goals (v1)

The following are deferred:

• ClickHouseIO.readAll() for distributed execution of user-supplied query requests.
• Automatic ClickHouse shard/partition discovery.
• Dynamic partition splitting and advanced skew handling.
• Comprehensive support for all ClickHouse-specific and nested data types.
• Streaming/change-data-capture support.
• SchemaTransform/Managed I/O integration.

Benefits

• Adds native read functionality to the existing Java ClickHouseIO connector.
• Enables distributed data extraction from ClickHouse into Beam pipelines.
• Supports large analytical workloads without requiring a single source reader.
• Provides Beam Row output for downstream transformations.
• Establishes a foundation for additional ClickHouseIO capabilities.

Testing plan

• Unit tests for configuration, partition generation, boundary validation, schema conversion, and error handling.
• Integration tests using the existing ClickHouse Testcontainers infrastructure.
• Verify that parallel reads return the expected records without overlapping or missing configured ranges on a static dataset.
• Test empty partitions, boundary values, null handling, and uneven data distributions.
• Validate execution with Beam DirectRunner.
• Run the existing module's Gradle validation tasks.

Design discussion

A design proposal will be shared on dev@beam.apache.org for community feedback before implementation.