Skip to content

isthmus: the FP window-offset round-trip guard accepts a value it cannot represent #1229

Description

@nielspardon

integralLiteralOfType in WindowBoundConverter guards its FP branches with a narrowing round trip — (long) asDouble == value for fp64 and (long) asFloat == value for fp32. A narrowing doublelong cast in Java saturates rather than wrapping, so the guard returns true for Long.MAX_VALUE: (double) Long.MAX_VALUE is 2^63 = 9223372036854775808, and casting that back yields Long.MAX_VALUE = 9223372036854775807. The comparison succeeds even though the value changed.

Measured on fb6a54ad: a RANGE BETWEEN <Long.MAX_VALUE> PRECEDING bound over a DOUBLE ordering column converts to Preceding{FP64Literal{9.223372036854776E18}}, an offset one greater than the value in the plan. Every long above 2^53 that is not exactly representable as a double takes the same path, and the fp32 branch has the same shape above 2^24 — rangeOffsetFailingFloatRoundTripThrows covers 2^24+1, just below the magnitude where saturation starts masking the mismatch.

Comparing in the wider direction rejects these instead: BigDecimal.valueOf(asDouble).compareTo(BigDecimal.valueOf(value)) == 0, or a magnitude bound of 2^53 for fp64 and 2^24 for fp32.

This is pre-existing behavior, but #1206 makes the branch throw when it rejects rather than fall through to the literal's own type, so a false accept now silently emits a wrong offset where a true reject would be loud.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions