Skip to content

fix(datetime): error on out-of-range date_bin timestamp bins - #5

Open
benbellick wants to merge 2 commits into
mainfrom
fix/date-bin-overflow-errors
Open

benbellick wants to merge 2 commits into
mainfrom
fix/date-bin-overflow-errors

Conversation

@benbellick

@benbellick benbellick commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Which issue does this PR close?

Closes apache#26605.

Rationale for this change

date_bin returns NULL for unrepresentable timestamp bins. Return an error instead, matching PostgreSQL.

I'm not sure whether DataFusion has a specific policy on returning NULL versus an error here, but the PostgreSQL precedent seems clear. Returning NULL because we cannot compute a bin for a non-null timestamp silently hides a computation failure rather than representing missing input.

This prepares date_bin for avoiding aggregate repartitioning in #25345, similar to the date_trunc optimization in #26449. It follows the error behavior established for date_trunc in #25131.

What changes are included in this PR?

Make scalar and array timestamp-binning failures return DATE_BIN timestamp out of range. Preserve NULL inputs and successful wider-arithmetic results.

What is the testing strategy for this PR?

Unit tests and SQLLogicTests cover overflow errors, scalar/array consistency, NULL preservation, and representable results across timestamp precisions.

Are there any user-facing changes?

Previously NULL-producing timestamp-binning failures now fail the query.


This PR was developed with AI assistance. All changes have been reviewed, and I take full responsibility for this contribution.

}

#[test]
fn test_date_bin_preserves_timestamp_nullness() {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benbellick
benbellick marked this pull request as ready for review October 22, 2026 21:11
}

#[test]
fn test_date_bin_timestamp_scalar_and_array_agree() {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to the date_trunc scalar/array consistency tests

/// Evaluates `date_trunc(granularity, value)` once on a scalar and once on a
/// one-row column holding the same value.
fn date_trunc_scalar_and_array(
granularity: &str,
value: ScalarValue,
) -> (
datafusion_common::Result<ScalarValue>,
datafusion_common::Result<ScalarValue>,
) {
let invoke = |arg: ColumnarValue| {
let data_type = value.data_type();
let args = ScalarFunctionArgs {
args: vec![ColumnarValue::Scalar(ScalarValue::from(granularity)), arg],
arg_fields: vec![
Field::new("a", DataType::Utf8, false).into(),
Field::new("b", data_type.clone(), true).into(),
],
number_rows: 1,
return_field: Field::new("f", data_type, true).into(),
config_options: Arc::new(ConfigOptions::default()),
};
match DateTruncFunc::new().invoke_with_args(args)? {
ColumnarValue::Scalar(result) => Ok(result),
ColumnarValue::Array(result) => ScalarValue::try_from_array(&result, 0),
}
};
(
invoke(ColumnarValue::Scalar(value.clone())),
invoke(ColumnarValue::Array(value.to_array().unwrap())),
)
}
/// A timestamp beyond the nanosecond range is truncated the same way as a
/// scalar and as a column: neither converts it to nanoseconds first.
#[test]
fn scalar_and_array_accept_timestamps_beyond_nanosecond_range() {
// 2286-11-20T17:46:40, after the last nanosecond timestamp in 2262
let seconds = 10_000_000_000;
let utc: Option<Arc<str>> = Some("UTC".into());
let cases = [
// Seconds, truncated to a minute, an hour and a day
(
ScalarValue::TimestampSecond(Some(seconds + 59), None),
"minute",
),
(
ScalarValue::TimestampSecond(Some(seconds + 1), None),
"hour",
),
(ScalarValue::TimestampSecond(Some(seconds + 1), None), "day"),
// The granularity is the input's own unit
(ScalarValue::TimestampSecond(Some(seconds), None), "second"),
// With a time zone
(
ScalarValue::TimestampSecond(Some(seconds + 59), utc.clone()),
"minute",
),
// Milliseconds and microseconds, truncated to a coarser unit
(
ScalarValue::TimestampMillisecond(Some(seconds * 1_000 + 999), None),
"second",
),
(
ScalarValue::TimestampMicrosecond(Some(seconds * 1_000_000 + 999), None),
"millisecond",
),
// Before the epoch, with a time zone
(
ScalarValue::TimestampMicrosecond(Some(-seconds * 1_000_000 - 1), utc),
"second",
),
];
for (value, granularity) in cases {
let (scalar, array) = date_trunc_scalar_and_array(granularity, value.clone());
let scalar = scalar.unwrap_or_else(|e| {
panic!("scalar date_trunc('{granularity}', {value:?}) failed: {e}")
});
assert_eq!(
scalar,
array.unwrap(),
"date_trunc('{granularity}', {value:?})"
);
}
// `date_trunc('second', to_timestamp_seconds(10000000000))` returns its input
// rather than an out of range error.
// See <https://github.com/apache/datafusion/issues/25432>.
let (scalar, _) = date_trunc_scalar_and_array(
"second",
ScalarValue::TimestampSecond(Some(seconds), None),
);
assert_eq!(
scalar.unwrap(),
ScalarValue::TimestampSecond(Some(seconds), None)
);
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

date_bin should error when the computed timestamp bin is out of range

1 participant