Skip to content

Optimize Row construction, RowWithGetters field caching, and Row utilities - #40791

Open
sjvanrossum wants to merge 13 commits into
apache:masterfrom
sjvanrossum:row-coder-opt-1
Open

sjvanrossum wants to merge 13 commits into
apache:masterfrom
sjvanrossum:row-coder-opt-1

Conversation

@sjvanrossum

Copy link
Copy Markdown
Contributor

Optimizes Row construction, field verification, RowWithGetters field caching, and Row utility methods by eliminating intermediate object allocations (RowPosition, FieldAccessDescriptor, RowFieldMatcher, CapturingRowCases, TreeMap, and Stream pipelines) on hot paths.

Previously, Row.Builder.build() wrapped its positional List<Object> into a FieldOverrides tree (allocating a FieldAccessNode, a FieldAccessDescriptor per field, and a RowPosition per field/nested element) and traversed the schema through RowFieldMatcher + CapturingRowCases. Similarly, SchemaVerification.verifyFieldValue constructed a single-field Schema, FieldOverrides, CapturingRowCases, RowFieldMatcher, and RowPosition on every single field verification call. Replaced RowFieldMatcher and CapturingRowCases with direct static RowUtils.captureRow(Schema, List<Object>) and RowUtils.captureValue(FieldType, Object, String) methods, and simplified FieldOverrides (Row.FieldValueBuilder) to apply overrides directly via captureValue.

Replaced the TreeMap<Integer, Object> (and per-lookup boxed Integer key + anonymous Function allocation in computeIfAbsent) in RowWithGetters with a flat Object[] indexed by fieldIdx (using a NULL_VALUE sentinel to cache null results without re-invoking getters). Extended RowWithGetters.cacheFieldType to also cache ROW and LOGICAL_TYPE fields in addition to MAP, ARRAY, and ITERABLE, avoiding repeated nested RowWithGetters / logical type conversions when a field is accessed multiple times.

Pre-sized Row.Builder's internal ArrayList to schema.getFieldCount() and used Collections.addAll in addValues(Object...). Replaced IntStream/Stream pipelines and name-based lookups with indexed loops in Row.getBaseValues(), Row.sorted(), Row.toSnakeCase(), Row.toCamelCase(), Row.nullRow(), and SchemaUtils.toPrettyRowString().

Added JMH benchmarks covering flat and complex Row construction, Row equals/hashCode, repeated RowWithGetters field reads, and RowCoder / SchemaCoder encoding and decoding.

Note

This PR stacks on top of #40789 and is part of a series of patches driving the following improvements:

Benchmark Base (ops/us) Patch series (ops/us) vs base
buildFlatRow 1.070 ± 0.011 26.722 ± 0.174 24.97x (+2,397.4%)
buildComplexRow 0.860 ± 0.012 9.550 ± 0.195 11.10x (+1,010.5%)
complexRowEqualsAndHashCode 1.481 ± 0.036 12.391 ± 0.400 8.37x (+736.7%)
readFlatRowBoxedGetters 148.151 ± 2.005 501.611 ± 2.892 3.39x (+238.6%)
readFlatRowPrimitiveGetters 148.151 ± 2.005 503.336 ± 3.551 3.40x (+239.7%)
encodeFlatRow 7.987 ± 0.068 19.758 ± 0.280 2.47x (+147.4%)
decodeFlatRow 11.425 ± 0.351 12.524 ± 0.068 1.10x (+9.6%)
encodeComplexRow 8.217 ± 0.130 12.388 ± 0.538 1.51x (+50.8%)
decodeComplexRow 3.954 ± 0.095 4.178 ± 0.047 1.06x (+5.7%)
encodePojoViaSchemaCoder 4.932 ± 0.053 7.412 ± 0.159 1.50x (+50.3%)
decodePojoViaSchemaCoder 2.616 ± 0.030 3.755 ± 0.019 1.44x (+43.5%)
readRowWithGettersRepeatedly 27.326 ± 0.642 121.171 ± 1.810 4.43x (+343.4%)

@github-actions github-actions Bot added java core size/xl XL: >=1000 lines changed labels Oct 25, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR significantly optimizes Row construction, SchemaVerification.verifyFieldValue, RowWithGetters field caching, and Row/SchemaUtils helper methods by removing intermediate RowPosition, FieldAccessDescriptor, RowFieldMatcher, CapturingRowCases, TreeMap, and Stream allocations on hot paths. It also fixes several correctness issues uncovered during optimization, including Row.Equals.deepHashCode for logical types, RowWithGetters equals/hashCode contract consistency with RowWithStorage, nested Row schema validation, ByteBuffer slice/offset handling, and DATETIME-based logical type handling in SchemaCoderHelpers.

🔍 General Feedback

  • Replacing RowFieldMatcher and CapturingRowCases with direct static RowUtils.captureRow and RowUtils.captureValue methods is much cleaner and eliminates substantial per-row and per-field allocation overhead.
  • Replacing the TreeMap<Integer, Object> in RowWithGetters with a transient Object[] using a NULL_VALUE sentinel and extending caching to ROW and LOGICAL_TYPE fields is well-designed and avoids repeated wrapper/conversion work.
  • The added unit tests and JMH benchmarks thoroughly cover the bug fixes and performance improvements.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

@github-actions

Copy link
Copy Markdown
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core java size/xl XL: >=1000 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant