Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Release History

## Unreleased
- Add default-enabled kernel-backed GEOMETRY / GEOGRAPHY results, selected with `WithEnableGeoSpatialSupport` or the `enableGeoSpatialSupport` DSN parameter. Enabled mode returns EWKB with its per-row SRID and logical geo column metadata from `database/sql`; disabled mode returns WKT / EWKT reported as `STRING`. Arrow batch access retains the canonical `struct<srid,wkb>` value with separate OGC WKB in enabled mode.
- Add opt-in kernel-backed GEOMETRY / GEOGRAPHY results, selected with `WithEnableGeoSpatialSupport` or the `enableGeoSpatialSupport` DSN parameter (default `false`). Enabled mode returns EWKB with its per-row SRID and logical geo column metadata from `database/sql`; disabled mode returns WKT / EWKT reported as `STRING`. Arrow batch access retains the canonical `struct<srid,wkb>` value with separate OGC WKB in enabled mode.
- **Behavior change:** `m2m.NewAuthenticatorWithScopes` now requests exactly the given scopes instead of always appending `all-apis`, so service principals with scoped OAuth secrets (e.g. `sql` only) can authenticate (databricks/databricks-sql-go#476). Empty scopes and `m2m.NewAuthenticator` still request `all-apis`; callers that relied on it being appended should pass no scopes, or add `all-apis` to their list if they need both. The SEA/kernel backend now forwards custom M2M scopes instead of rejecting them.
- Let the kernel telemetry feature flag decide enablement when `enableTelemetry` is unset; explicit `true` and `false` continue to override it.

Expand Down
6 changes: 3 additions & 3 deletions CONNECTION_PARAMETERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,14 +155,14 @@ ignored with a warning and still stripped from the SEA wire.
|---|---|:---:|:---:|---|---|
| `useArrowNativeDecimal` | `WithArrowNativeDecimal` | ✅ | ⚠️ | `false` | Thrift: return DECIMAL as native Arrow `decimal128` (lossless string when scanned via `database/sql`). The kernel path already renders DECIMAL as the exact string regardless, so the flag is inert there. |
| | `WithKernelDecimalAsFloat(b)` | ❌ | ✅ | `false` | Scan top-level DECIMAL as lossy `float64` instead of the exact string. |
| `enableGeoSpatialSupport` | `WithEnableGeoSpatialSupport(b)` | ❌ | ✅ | `true` | `true`: return top-level GEOMETRY / GEOGRAPHY as EWKB `[]byte` with the per-row SRID embedded and logical geo column metadata; `GetArrowBatches` retains the full `{srid,wkb}` struct with OGC WKB. `false`: return WKT / EWKT strings reported as `STRING`. |
| `enableGeoSpatialSupport` | `WithEnableGeoSpatialSupport(b)` | ❌ | ✅ | `false` | `true`: return top-level GEOMETRY / GEOGRAPHY as EWKB `[]byte` with the per-row SRID embedded and logical geo column metadata; `GetArrowBatches` retains the full `{srid,wkb}` struct with OGC WKB. `false`: return WKT / EWKT strings reported as `STRING`. |

Apart from the geospatial representation selected above, results render **byte-for-byte
identically** on both backends (scalars,
DECIMAL as exact string, TIMESTAMP / TIMESTAMP_NTZ shifted into the session time zone,
INTERVAL, nested ARRAY / MAP / STRUCT and VARIANT as JSON, and BINARY as
`sql.RawBytes`). Set `enableGeoSpatialSupport=false` on the kernel path to match
the Thrift path's WKT / EWKT geospatial values.
`sql.RawBytes`). The default `enableGeoSpatialSupport=false` on the kernel path
matches the Thrift path's WKT / EWKT geospatial values.

## Cloud Fetch

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ session parameter on both backends.
|---|---|---|---|---|
| `useArrowNativeDecimal` | `WithArrowNativeDecimal` | Thrift only (inert on kernel) | `false` | Thrift: return DECIMAL as native Arrow `decimal128` (lossless string when scanned via `database/sql`). The kernel path already renders DECIMAL as the exact string regardless. |
| | `WithKernelDecimalAsFloat(b)` | SEA only | `false` | Scan top-level DECIMAL as lossy `float64` instead of the exact string. |
| `enableGeoSpatialSupport` | `WithEnableGeoSpatialSupport(b)` | SEA only | `true` | `true`: return top-level GEOMETRY / GEOGRAPHY as EWKB `[]byte` with the per-row SRID embedded and logical geo column metadata; `GetArrowBatches` retains the full `{srid,wkb}` struct with OGC WKB. `false`: return WKT / EWKT strings reported as `STRING`. |
| `enableGeoSpatialSupport` | `WithEnableGeoSpatialSupport(b)` | SEA only | `false` | `true`: return top-level GEOMETRY / GEOGRAPHY as EWKB `[]byte` with the per-row SRID embedded and logical geo column metadata; `GetArrowBatches` retains the full `{srid,wkb}` struct with OGC WKB. `false`: return WKT / EWKT strings reported as `STRING`. |

See [Cloud Fetch](#cloud-fetch), [TLS](#tls), and [Proxy](#proxy) for the remaining
groups. Telemetry parameters are covered under [Telemetry](#telemetry).
Expand Down Expand Up @@ -363,8 +363,8 @@ Apart from the configurable geospatial representation, results render **byte-for
identically** on both backends. Scalars, DECIMAL (exact string), TIMESTAMP /
TIMESTAMP_NTZ (shifted into the session time zone), INTERVAL, and nested ARRAY / MAP /
STRUCT and VARIANT (as JSON) are supported. BINARY is returned as `sql.RawBytes`.
Thrift returns GEOMETRY / GEOGRAPHY as WKT / EWKT; the kernel defaults to EWKB and
matches Thrift when `enableGeoSpatialSupport=false`.
Both backends return GEOMETRY / GEOGRAPHY as WKT / EWKT by default.
Set `enableGeoSpatialSupport=true` on the kernel path to opt into EWKB.

Metadata is reached through SQL (`SHOW`, `DESCRIBE`, `information_schema`) on both
backends — the driver exposes no `GetCatalogs`/`GetSchemas`/`GetTables`/`GetColumns` API
Expand Down
7 changes: 4 additions & 3 deletions connector.go
Original file line number Diff line number Diff line change
Expand Up @@ -676,10 +676,11 @@ func WithKernelDecimalAsFloat(asFloat bool) ConnOption {
}

// WithEnableGeoSpatialSupport selects how the kernel path exposes top-level
// GEOMETRY and GEOGRAPHY result values. True (the default) returns EWKB bytes
// GEOMETRY and GEOGRAPHY result values. True returns EWKB bytes
// ([]byte), including the per-row SRID, and reports the logical GEOMETRY /
// GEOGRAPHY type through database/sql metadata. False returns WKT / EWKT strings and reports STRING, matching the
// existing string-mode driver contract. The per-row SRID remains present in the
// GEOGRAPHY type through database/sql metadata. False (the default) returns
// WKT / EWKT strings and reports STRING, matching the existing string-mode
// driver contract. The per-row SRID remains present in the
// Arrow struct returned by the driver's GetArrowBatches API in enabled mode;
// that raw Arrow value retains Databricks' separate SRID and OGC WKB children.
//
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/kernel/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,7 @@ func (k *KernelBackend) applyMaxConnections(cfg *C.KernelSessionConfig) error {
}

// applyEnableGeoSpatialSupport selects the kernel's client-side representation
// for GEOMETRY / GEOGRAPHY results. The connector supplies the default-enabled
// for GEOMETRY / GEOGRAPHY results. The connector supplies the default-disabled
// value explicitly, and an explicit false crosses the ABI unchanged.
func (k *KernelBackend) applyEnableGeoSpatialSupport(cfg *C.KernelSessionConfig) error {
if err := call(func() C.KernelStatusCode {
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/kernel/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ type Config struct {
// EnableGeoSpatialSupport selects the local GEOMETRY / GEOGRAPHY result
// representation. True requests the canonical struct<srid,wkb> Arrow
// representation; false requests WKT / EWKT strings. The connector always
// sets this field and defaults it to true.
// sets this field and defaults it to false.
EnableGeoSpatialSupport bool

// TokenCacheEnabled controls the kernel's on-disk OAuth U2M token-cache persistence
Expand Down
4 changes: 2 additions & 2 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ type KernelExperimentalConfig struct {
// EnableGeoSpatialSupport selects the kernel's client-side GEOMETRY /
// GEOGRAPHY result representation. True requests the canonical
// struct<srid,wkb> Arrow value; false requests WKT / EWKT strings. A pointer
// preserves the default-enabled value versus explicitly false.
// distinguishes the omitted value (default false) from an explicit setting.
EnableGeoSpatialSupport *bool
}

Expand Down Expand Up @@ -314,7 +314,7 @@ type UserConfig struct {
TokenCacheEnabledDSN bool
// EnableGeoSpatialSupportDSN is a DSN-only carrier for
// enableGeoSpatialSupport=true|false. Nil means omitted (the default is
// true); the connector copies a present value into KernelExperimental.
// false); the connector copies a present value into KernelExperimental.
EnableGeoSpatialSupportDSN *bool
}

Expand Down
6 changes: 3 additions & 3 deletions kernel_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -176,9 +176,9 @@ func buildKernelConfig(cfg *config.Config, kauth kernel.Auth) kernel.Config {
// this binding's system identity for its own runtime.
Telemetry: kernelTelemetryConfig(cfg),
DriverSystemConfiguration: kernelDriverSystemConfiguration(cfg),
// Match the public contract and the kernel default. An explicit false
// below selects WKT / EWKT instead.
EnableGeoSpatialSupport: true,
// Preserve string results by default, including automatic kernel recovery.
// An explicit true below opts into native geospatial values.
EnableGeoSpatialSupport: false,
}
if cfg.ClientQueryTimeout != nil {
timeout := *cfg.ClientQueryTimeout
Expand Down
6 changes: 3 additions & 3 deletions kernel_config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -559,11 +559,11 @@ func TestBuildKernelConfig(t *testing.T) {
}
})

t.Run("geospatial support defaults true and preserves explicit values", func(t *testing.T) {
t.Run("geospatial support defaults false and preserves explicit values", func(t *testing.T) {
c := baseKernelConfig()
kc := buildKernelConfig(c, kernel.Auth{Mode: kernel.AuthPAT, Token: "dapi-x"})
if !kc.EnableGeoSpatialSupport {
t.Fatal("omitted EnableGeoSpatialSupport = false, want default true")
if kc.EnableGeoSpatialSupport {
t.Fatal("omitted EnableGeoSpatialSupport = true, want default false")
}

WithEnableGeoSpatialSupport(false)(c)
Expand Down
2 changes: 1 addition & 1 deletion kernel_geospatial_e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ func TestKernelE2EGeospatialRepresentations(t *testing.T) {
native bool
extra []ConnOption
}{
{"default", true, nil},
{"default", false, nil},
{"enabled", true, []ConnOption{WithEnableGeoSpatialSupport(true)}},
{"disabled", false, []ConnOption{WithEnableGeoSpatialSupport(false)}},
} {
Expand Down
Loading