You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit afc6cb8
Browse filesBrowse the repository at this point in the historyBrowse files
feat: support geospatial results on the kernel path (#990)
* feat: support geospatial results on the kernel path
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* feat: align geospatial support configuration
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* test: handle pending kernel geospatial release
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* test: import kernel compatibility error
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* fix(thrift): report geospatial cursor metadata
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* fix: preserve geospatial result metadata and shapes
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* chore: update geospatial kernel pin
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* fix: default Python geospatial results to strings
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* docs: correct geospatial support default
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* test: match kernel table-type filters by exact case
Align Kernel E2E with case-sensitive filtering from databricks-sql-kernel#393. Check uppercase matches and lowercase/mixed-case empty results, and update stale filter comments.
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
* ci: refresh merge-queue check records
Refresh status contexts after GitHub reported the Kernel E2E requirement as expected despite matching successful checks. The source tree is unchanged from the validated fix in 3c800a5.
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
---------
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Copy file name to clipboardExpand all lines: CHANGELOG.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,7 @@
1
1
# Release History
2
2
3
3
# Unreleased
4
+
- Add the kernel-only `enable_geospatial_support` connection option (default `False`). GEOMETRY / GEOGRAPHY results are exposed as `{"srid": int, "wkb": bytes}` values when true or WKT / EWKT strings when false.
4
5
- Add the `escape_catalog_name` connection parameter (default `False`) to opt in to escaping unescaped `_` in metadata catalog names on Thrift and kernel connections. Existing escapes and `%` wildcards are preserved.
Copy file name to clipboardExpand all lines: CONNECTION_PARAMETERS.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -153,6 +153,7 @@ to change without notice.
153
153
|`max_download_threads`|`int`| ✅ | ❌ |`10`| Worker threads for cloud-fetch downloads. Not forwarded to the kernel. |
154
154
|`enable_query_result_lz4_compression`|`bool`| ✅ | ❌ |`True`| LZ4-compress result payloads. Not forwarded; the kernel handles compression internally. |
155
155
|`_disable_pandas`|`bool`| ✅ | ✅ |`False`| Skip the pandas-based Arrow→row deserialization and materialize rows directly with PyArrow. This is a **Python-side** result-conversion toggle, not a wire option: the kernel returns results as Arrow (`RecordBatch`es) and the connector runs the *same*`_convert_arrow_table` for both backends, so the flag is honored on the kernel path too. Affects only row fetches (`fetchone`/`fetchmany`/`fetchall`); the `fetch*_arrow` methods return the Arrow table unchanged regardless of this flag. |
156
+
|`enable_geospatial_support`|`bool`| ❌ | ✅ |`False`| Return GEOMETRY / GEOGRAPHY as `{"srid": int, "wkb": bytes}` values when `True`, or as WKT / EWKT strings when `False`. This is a local result conversion and is never forwarded to SEA. |
156
157
|`_use_arrow_native_complex_types`|`bool`| ✅ | ✅ |`True`| Return `ARRAY`/`MAP`/`STRUCT` as native Arrow types instead of JSON strings. Forwarded to the kernel. |
157
158
|`_use_arrow_native_decimals`|`bool`| ✅ | ❌ |`True`| Thrift wire encoding for `DECIMAL`: `True` → native Arrow `decimal128`, `False` → Arrow string. **No value-level effect**, though: the connector unconditionally re-casts the column back to `decimal128` (`convert_decimals_in_arrow_table`, `thrift_backend.py`), so both `fetchall()` and `fetchall_arrow()` yield `Decimal` / `decimal128(p,s)` either way (verified live). Not forwarded to the kernel, which always returns native Arrow decimals. |
158
159
|`_use_arrow_native_timestamps`|`bool`| ✅ | ❌ |`True`| Thrift wire encoding for `TIMESTAMP`: `True` → native Arrow timestamp (→ Python `datetime`), `False` → Arrow string (→ Python **`str`**). **Unlike decimals there is no re-cast**, so `False` genuinely surfaces strings — and `cursor.description` still reports the type code as `'timestamp'`, a mismatch to watch for (verified live). Note the connector always also sends the `spark.thriftserver.arrowBasedRowSet.timestampAsString=false` conf, but the `timestampAsArrow=False` flag wins. Not forwarded to the kernel, which always returns native Arrow timestamps. |
0 commit comments