Repository navigation

GH-50628: [C++][Parquet] Rework RecordReader to introduce flat optional optimization - #50629

Open

AntoinePrv wants to merge 67 commits into

apache:mainfrom

AntoinePrv:levels-bitmap

Open

GH-50628: [C++][Parquet] Rework RecordReader to introduce flat optional optimization#50629
AntoinePrv wants to merge 67 commits into
apache:mainfrom
AntoinePrv:levels-bitmap

Commits on Oct 21, 2026
- Move TypedColReaderImpl::Skip
  AntoinePrv
  committed
  6b750f4
- Break TypedRecordReader inheritance on TypedColumnReader
  AntoinePrv
  committed
  17f1c9a
- Add ReadValuesBuffer
  AntoinePrv
  committed
  51b0169
- Split ColumnReaderImplBase definition and declaration
  AntoinePrv
  committed
  a1f7c22
- Add DefRepLevelsDecoder
  AntoinePrv
  committed
  aa31656
- Explicit int32_t for LevelDecoder
  AntoinePrv
  committed
  737cd10
- Change ValuesBuffer API
  AntoinePrv
  committed
  4ef6eef
- Add StackedLevelDecoder and prepare for modularity
  AntoinePrv
  committed
  93b069a
- Remove DefRepLevelsDecoder
  AntoinePrv
  committed
  3ce765e
- Split TypedRecordReader implementation
  AntoinePrv
  committed
  05d6018
- First RequiredTypedRecordReader
  AntoinePrv
  committed
  b1baa1c
- Remove RecordReader protected members
  AntoinePrv
  committed
  9f17b93
- Fully use RequiredRecordReader
  AntoinePrv
  committed
  1edbd7d
- Rename ReadValues -> ValueSink
  AntoinePrv
  committed
  5e79853
- Split ReserveValuesAndIsValid
  AntoinePrv
  committed
  5c413aa
- Devirtualize hooks with CRTP
  AntoinePrv
  committed
  7f40e5e
- Simplify template
  AntoinePrv
  committed
  218bdc1
- Remove StackedLevelDecoder
  AntoinePrv
  committed
  6f718c0
- Remove mark_as_written
  AntoinePrv
  committed
  9eeb3a6
- Make some of ColumnReaderImplBase private
  AntoinePrv
  committed
  180cc80
- Fix horrible dictionnary handling
  AntoinePrv
  committed
  c8f3c80
- Simplify and rename ColumnChunkReader
  AntoinePrv
  committed
  7db94f2
- Add RleBitPackedToBitmapDecoder::CountUpTo
  AntoinePrv
  committed
  24ae596
- Add BitPackedToBitmapDecoder
  AntoinePrv
  committed
  e9907da
- Add RleBitPackedToBitmapDecoder::Advance
  AntoinePrv
  committed
  53466bd
- Add LevelToBitmapDecoder
  AntoinePrv
  committed
  80cc713
- Add ValiditySinkBuffer
  AntoinePrv
  committed
  878fa3f
- Auto reserve space in sinks:
  AntoinePrv
  committed
  e01f5fa
- Fix integer types
  AntoinePrv
  committed
  b352b07
- Nit in ColumnChunkReader
  AntoinePrv
  committed
  544e4b4
- Eagerly read dict
  AntoinePrv
  committed
  52dcda9
- Cleanups
  AntoinePrv
  committed
  82f1a03
- Add FlatOptionalTypedRecordReader
  AntoinePrv
  committed
  144e8fd
- Add some doc
  AntoinePrv
  committed
  cc1b422
- Factorize ReadDictionary
  AntoinePrv
  committed
  59b612f
- Simplify read_dense_for_nullable
  AntoinePrv
  committed
  3377da7
- Factor Sinks with DataSinkBuffer
  AntoinePrv
  committed
  3f55b14
- Use LevelBufferSink in TypedRecordReader
  AntoinePrv
  committed
  290adcd
- Relax record reader level tests flat optional
  AntoinePrv
  committed
  6c0e310
- Fix compiler errrors
  AntoinePrv
  committed
  83ab573
- Safe decoded number check and func rename
  AntoinePrv
  committed
  4966170
- Explicit values_written contract
  AntoinePrv
  committed
  1a42d41
- Fix LevelDecoder::max_level type
  AntoinePrv
  committed
  7efe675
- Fix integer convertion
  AntoinePrv
  committed
  902379e
- Factorize level decoders
  AntoinePrv
  committed
  d83e19e
- Fixt typos
  AntoinePrv
  committed
  d9cf482
- Move level decoder utility and rename file
  AntoinePrv
  committed
  0681d64
- Move level decoder adapters to level_decoder.h
  AntoinePrv
  committed
  bbd546d
- Fix small typos
  AntoinePrv
  committed
  c757c1e
- Fix issues in level counting
  AntoinePrv
  committed
  725da03
- Fix extra alloc
  AntoinePrv
  committed
  1606ccf
- Add RecordReader::Make flag
  AntoinePrv
  committed
  e33fdc7
- Throw on FlatOptional def_levels
  AntoinePrv
  committed
  608d40a
- Always call RecordReader with flat optiona optimization
  AntoinePrv
  committed
  6b8c906
- Addition flat optional test
  AntoinePrv
  committed
  dff7065
- Detemplate kdic
  AntoinePrv
  committed
  841608f
- Add flat optiona bse in flba and byte array
  AntoinePrv
  committed
  e3a2654
- Ad more flba and array test
  AntoinePrv
  committed
  c44b2eb
- Make RecordReader::Make params struct public
  AntoinePrv
  committed
  352210c
- Run RecordReader tests with and without optimizations
  AntoinePrv
  committed
  a26f8d9
- Rename flat_optiona_optimization as allow_levels_optimization
  AntoinePrv
  committed
  7823021
- Add level optim option to RowGroupReader
  AntoinePrv
  committed
  0df1e9b
- Reuse validity map
  AntoinePrv
  committed
  bca2f09
- Lazily pop dictionnary
  AntoinePrv
  committed
  7e595f7
- Address some review comments
  AntoinePrv
  committed
  911a392
- Fix typo
  AntoinePrv
  committed
  3d77c49
Commits on Oct 22, 2026
- Add missing gTest PrintTo
  AntoinePrv
  committed
  b4a44de

Commits

Commits on Oct 21, 2026

Commits on Oct 22, 2026