Repository navigation
Optimize Row construction, RowWithGetters field caching, and Row utilities - #40791
sjvanrossum wants to merge 13 commits into
Conversation
… RowWithGetters equals/hashCode
…ls/hashCode contract
… SchemaCoderHelpers
There was a problem hiding this comment.
📋 Review Summary
This PR significantly optimizes Row construction, SchemaVerification.verifyFieldValue, RowWithGetters field caching, and Row/SchemaUtils helper methods by removing intermediate RowPosition, FieldAccessDescriptor, RowFieldMatcher, CapturingRowCases, TreeMap, and Stream allocations on hot paths. It also fixes several correctness issues uncovered during optimization, including Row.Equals.deepHashCode for logical types, RowWithGetters equals/hashCode contract consistency with RowWithStorage, nested Row schema validation, ByteBuffer slice/offset handling, and DATETIME-based logical type handling in SchemaCoderHelpers.
🔍 General Feedback
- Replacing
RowFieldMatcherandCapturingRowCaseswith direct staticRowUtils.captureRowandRowUtils.captureValuemethods is much cleaner and eliminates substantial per-row and per-field allocation overhead. - Replacing the
TreeMap<Integer, Object>inRowWithGetterswith atransient Object[]using aNULL_VALUEsentinel and extending caching toROWandLOGICAL_TYPEfields is well-designed and avoids repeated wrapper/conversion work. - The added unit tests and JMH benchmarks thoroughly cover the bug fixes and performance improvements.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
fa6bb38 to
0b10478
Compare
Optimizes
Rowconstruction, field verification,RowWithGettersfield caching, andRowutility methods by eliminating intermediate object allocations (RowPosition,FieldAccessDescriptor,RowFieldMatcher,CapturingRowCases,TreeMap, andStreampipelines) on hot paths.Previously,
Row.Builder.build()wrapped its positionalList<Object>into aFieldOverridestree (allocating aFieldAccessNode, aFieldAccessDescriptorper field, and aRowPositionper field/nested element) and traversed the schema throughRowFieldMatcher+CapturingRowCases. Similarly,SchemaVerification.verifyFieldValueconstructed a single-fieldSchema,FieldOverrides,CapturingRowCases,RowFieldMatcher, andRowPositionon every single field verification call. ReplacedRowFieldMatcherandCapturingRowCaseswith direct staticRowUtils.captureRow(Schema, List<Object>)andRowUtils.captureValue(FieldType, Object, String)methods, and simplifiedFieldOverrides(Row.FieldValueBuilder) to apply overrides directly viacaptureValue.Replaced the
TreeMap<Integer, Object>(and per-lookup boxedIntegerkey + anonymousFunctionallocation incomputeIfAbsent) inRowWithGetterswith a flatObject[]indexed byfieldIdx(using aNULL_VALUEsentinel to cachenullresults without re-invoking getters). ExtendedRowWithGetters.cacheFieldTypeto also cacheROWandLOGICAL_TYPEfields in addition toMAP,ARRAY, andITERABLE, avoiding repeated nestedRowWithGetters/ logical type conversions when a field is accessed multiple times.Pre-sized
Row.Builder's internalArrayListtoschema.getFieldCount()and usedCollections.addAllinaddValues(Object...). ReplacedIntStream/Streampipelines and name-based lookups with indexed loops inRow.getBaseValues(),Row.sorted(),Row.toSnakeCase(),Row.toCamelCase(),Row.nullRow(), andSchemaUtils.toPrettyRowString().Added JMH benchmarks covering flat and complex
Rowconstruction,Rowequals/hashCode, repeatedRowWithGettersfield reads, andRowCoder/SchemaCoderencoding and decoding.Note
This PR stacks on top of #40789 and is part of a series of patches driving the following improvements:
ops/us)ops/us)buildFlatRow1.070 ± 0.01126.722 ± 0.174buildComplexRow0.860 ± 0.0129.550 ± 0.195complexRowEqualsAndHashCode1.481 ± 0.03612.391 ± 0.400readFlatRowBoxedGetters148.151 ± 2.005501.611 ± 2.892readFlatRowPrimitiveGetters148.151 ± 2.005503.336 ± 3.551encodeFlatRow7.987 ± 0.06819.758 ± 0.280decodeFlatRow11.425 ± 0.35112.524 ± 0.068encodeComplexRow8.217 ± 0.13012.388 ± 0.538decodeComplexRow3.954 ± 0.0954.178 ± 0.047encodePojoViaSchemaCoder4.932 ± 0.0537.412 ± 0.159decodePojoViaSchemaCoder2.616 ± 0.0303.755 ± 0.019readRowWithGettersRepeatedly27.326 ± 0.642121.171 ± 1.810