You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reading through the optimised varlen cast macros in gdv_string_function_stubs.cc I noticed the date64 one looks under-sized. GDV_FN_CAST_VARLEN_TYPE_FROM_DATE64 reserves an arena buffer of min(len, 10) bytes with the comment "Output is always YYYY-MM-DD = 10 chars max", and the shared formatter suffix then copies min(len, size) bytes from StringFormatter into it. That ten-byte assumption does not hold: the formatter widens to "-YYYYY-MM-DD" (up to 12 bytes) for years outside four digits, and for a millisecond value past its supported year range it returns "<value out of range: >" (up to 42 bytes). date64 values are just int64 milliseconds, so with a cast length above 10 the copy runs past the ten-byte slot and corrupts the following arena allocation. It is reachable from CAST(d AS VARCHAR) / CAST(d AS VARBINARY) over untrusted date64 column data.
I confirmed it by replaying the macro's allocation and copy under ASan for the value 9000000000000000 ms with length 100: a 38-byte memcpy ("<value out of range: 9000000000000000>") into the 10-byte region. The sibling integer macro (digits10 + 2) and float macro (24, which is the double-conversion shortest maximum, I checked the formatter never exceeds it) are both sized correctly, so only the date64 path needs changing.
What changes are included in this PR?
Raise max_date_str_len from 10 to 42, the longest rendering the formatter can produce ("<value out of range: " is 21 bytes, plus up to 20 int64 digits, plus ">"). The min(len, max) behaviour is otherwise unchanged, so a smaller requested length still truncates as before. This covers both helpers the macro generates, VARCHAR and VARBINARY.
Are these changes tested?
Yes. Added TestGdvFnStubs.TestCastVARCHARFromMillisecondsOutOfRange, which casts an out-of-range date64 and then does a second cast into the same arena chunk; with the old ten-byte reservation the second allocation overlaps and corrupts the first result, so the test fails before this change and passes after. The existing date64 cast tests still pass.
Are there any user-facing changes?
No. Only casts that previously overflowed are affected.
This PR contains a "Critical Fix". It fixes a heap buffer overflow reachable from casting an untrusted date64 column to VARCHAR/VARBINARY in Gandiva.
No worries, it's an easy one to miss since ordinary dates really do always come out as 10 chars. The out-of-range rendering only turned up when I was reading through what StringFormatter falls back to.
Reviewed this against #50540. The 42 bound is correct for the current formatter, and the test reproduces the overlap nicely. The tradeoff is that the constant has to stay in sync with StringFormatter<Date64Type>, while #50540 sizes the block from the formatted output inside the callback and drops the guessed constants from both the date64 and float macros. Since the reviewer preferred exact sizing, I'd suggest landing #50540 and closing this one as a duplicate, if that works for you.
Agreed, exact sizing is the better shape here. I had a read through #50540 and left a review there: allocating inside the callback from min(len, size) means nothing has to track the formatter any more, and the year 10000 and int64 min expectations in TestCastVARCHARFromMillisecondsLongOutput line up with what FormatYYYY_MM_DD and FormatOutOfRange actually produce, so my 9000000000000000 case is covered by the same out-of-range path. Happy for this one to be closed as a duplicate once #50540 goes in.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Reading through the optimised varlen cast macros in gdv_string_function_stubs.cc I noticed the date64 one looks under-sized. GDV_FN_CAST_VARLEN_TYPE_FROM_DATE64 reserves an arena buffer of
min(len, 10)bytes with the comment "Output is always YYYY-MM-DD = 10 chars max", and the shared formatter suffix then copiesmin(len, size)bytes from StringFormatter into it. That ten-byte assumption does not hold: the formatter widens to "-YYYYY-MM-DD" (up to 12 bytes) for years outside four digits, and for a millisecond value past its supported year range it returns "<value out of range: >" (up to 42 bytes). date64 values are just int64 milliseconds, so with a cast length above 10 the copy runs past the ten-byte slot and corrupts the following arena allocation. It is reachable from CAST(d AS VARCHAR) / CAST(d AS VARBINARY) over untrusted date64 column data.I confirmed it by replaying the macro's allocation and copy under ASan for the value 9000000000000000 ms with length 100: a 38-byte memcpy ("<value out of range: 9000000000000000>") into the 10-byte region. The sibling integer macro (digits10 + 2) and float macro (24, which is the double-conversion shortest maximum, I checked the formatter never exceeds it) are both sized correctly, so only the date64 path needs changing.
What changes are included in this PR?
Raise max_date_str_len from 10 to 42, the longest rendering the formatter can produce ("<value out of range: " is 21 bytes, plus up to 20 int64 digits, plus ">"). The
min(len, max)behaviour is otherwise unchanged, so a smaller requested length still truncates as before. This covers both helpers the macro generates, VARCHAR and VARBINARY.Are these changes tested?
Yes. Added TestGdvFnStubs.TestCastVARCHARFromMillisecondsOutOfRange, which casts an out-of-range date64 and then does a second cast into the same arena chunk; with the old ten-byte reservation the second allocation overlaps and corrupts the first result, so the test fails before this change and passes after. The existing date64 cast tests still pass.
Are there any user-facing changes?
No. Only casts that previously overflowed are affected.
This PR contains a "Critical Fix". It fixes a heap buffer overflow reachable from casting an untrusted date64 column to VARCHAR/VARBINARY in Gandiva.
Was AI used for this PR?
PR code and description written by:
Reviewed before submission by: