Skip to content

TIMESTAMP_NTZ is scanned as a zoned TIMESTAMP #506

Open

Description

@jakubhrncar

TIMESTAMP and TIMESTAMP_NTZ both come back from rows.Scan as time.Time, and ColumnTypeDatabaseTypeName reports TIMESTAMP for both. A zone-less value cannot be told apart from an instant, and it is passed through a location conversion.

On the Arrow batch the two types are different. Both are timestamp[us]. The TIMESTAMP field has a time zone set. The TIMESTAMP_NTZ field has an empty TimeZone. Nothing in the driver reads that. GetDBTypeName returns the Thrift primitive name with _TYPE stripped, and there is no TIMESTAMP_NTZ handling in the driver. Every *arrow.TimestampType uses one converter, built from arrow.FixedWidthTypes.Timestamp_us.
Reproducer, against a SQL warehouse:

SELECT
  TIMESTAMP '2024-06-15 09:15:00.123456' AS zoned,
  TIMESTAMP_NTZ '2024-06-15 09:15:00.123456' AS ntz
var zoned, ntz time.Time
rows.Next()
rows.Scan(&zoned, &ntz)
// both are time.Time
// ColumnTypeDatabaseTypeName(0) == "TIMESTAMP"
// ColumnTypeDatabaseTypeName(1) == "TIMESTAMP"

Expected: TIMESTAMP_NTZ keeps the wall clock it was given and is reported as TIMESTAMP_NTZ. TIMESTAMP stays an instant https://docs.databricks.com/aws/en/sql/language-manual/data-types/timestamp-ntz-type.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions