Apache Parquet has become the de-facto interchange format for analytical/columnar data (Spark, Trino, Athena, DuckDB, Arrow, Iceberg/Delta table formats all read and write it).
Today, JVM applications that want to produce or consume Parquet go through the parquet-java libraries directly (org.apache.parquet:parquet-hadoop and friends), which means hand-writing WriteSupport/ReadSupport plumbing or converting through another object model (parquet-avro, parquet-protobuf). Point solutions exist for narrow slices (see prior art below), but there is no first-class Jackson backend: no JsonFactory-level parser/generator pair giving bidirectional streaming, databind (POJOs, Maps) and tree-model (JsonNode) access under the ObjectMapper programming model applications already use for JSON, CBOR, Avro, etc.
Additionally, we've internally observed that adding Hadoop as a dependency ends up blowing the dependencies' size, forcing a lot of teams to drop AWS Lambdas (or using containerised lambdas)
A jackson-dataformat-parquet backend would close that gap: the same ObjectMapper
programming model, applied to the dominant analytics file format.
Prior art
- https://github.com/getyourguide/parquet-json — Parquet
WriteSupport driven by Jackson JsonNode:
overlaps the write path proposed here (no read path, and no JsonFactory-level
parser/generator or databind integration)
- https://github.com/jerolba/parquet-carpet — Java-records read/write binding over parquet-java,
demonstrating demand for a friendlier binding layer (records only; not a general
Jackson backend)
parquet-avro in parquet-java itself — the "bind through another object model"
pattern this module would generalize to Jackson's- Searched this tracker,
jackson-future-ideas and the org: no existing
Parquet request (only a passing mention in #9).
Apache Parquet has become the de-facto interchange format for analytical/columnar data (Spark, Trino, Athena, DuckDB, Arrow, Iceberg/Delta table formats all read and write it).
Today, JVM applications that want to produce or consume Parquet go through the parquet-java libraries directly (
org.apache.parquet:parquet-hadoopand friends), which means hand-writingWriteSupport/ReadSupportplumbing or converting through another object model (parquet-avro,parquet-protobuf). Point solutions exist for narrow slices (see prior art below), but there is no first-class Jackson backend: noJsonFactory-level parser/generator pair giving bidirectional streaming, databind (POJOs,Maps) and tree-model (JsonNode) access under theObjectMapperprogramming model applications already use for JSON, CBOR, Avro, etc.Additionally, we've internally observed that adding Hadoop as a dependency ends up blowing the dependencies' size, forcing a lot of teams to drop AWS Lambdas (or using containerised lambdas)
A
jackson-dataformat-parquetbackend would close that gap: the sameObjectMapperprogramming model, applied to the dominant analytics file format.
Prior art
WriteSupportdriven by JacksonJsonNode:overlaps the write path proposed here (no read path, and no
JsonFactory-levelparser/generator or databind integration)
demonstrating demand for a friendlier binding layer (records only; not a general
Jackson backend)
parquet-avroin parquet-java itself — the "bind through another object model"pattern this module would generalize to Jackson's
jackson-future-ideasand the org: no existingParquet request (only a passing mention in #9).