Repository navigation
Commit da2030b
LAV-3746: Support WITH LOCATION for external tables (#4098)
Accept the optional WITH token in the Snowflake external-table clause loop,
using the existing LOCATION parser and registry-driven creation path. Preserve
WITH TAG and WITH ROW ACCESS POLICY parsing, and leave equals whitespace and
shared stage grammar untouched.
Add 14 metadata-only Cloud-captured compat cases for paired LOCATION spellings,
unqualified/schema/database/quoted names, root and nested subpaths, missing and
internal stages, following format/refresh options, and inline/named-format
GET_DDL recreation with identical DESC and INFORMATION_SCHEMA metadata.
Normalize named external-table references through the existing stage parser
and identifier renderer before storing them. Keep unqualified stages relative
so existing default GET_DDL snapshots remain unchanged. Report the resolved
internal stage's leaf name rather than the supplied namespace.
Verification:
- Cloud capture: TEST_TARGET=SNOWFLAKE_CLOUD PYTEST_XDIST_ARGS='' make test-compat
PYTEST_PATH=tests/queries/external_tables/test_external_table_with_location.py
PYTEST_ARGS='--snapshot-update' (14 passed).
- Before implementation, the unqualified WITH case failed on unexpected WITH;
the bare root case also exposed Cloud's canonical trailing slash.
- make check passed (clippy, warnings-as-errors, formatting).
- make test-vendored-sqlparser passed, including shared stage parser tests.
- Serial make test-compat over the new file, external_table_get_ddl,
create_stage_core_parity, and copy_into_quoted_stage_location: 37 passed.
- RUN_LOCALSTACK_TESTS=1 serial make test-compat over test_external_tables.py
with -k test_location_internal_stage_rejected: 1 passed.
- All paired Cloud recorded-content values are identical; each recorded
roundtrip's DDL, DESC and metadata before/after values are identical.
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
## Deviations
New Cloud snapshots exposed existing root-path, schema-qualified, and redundant
identifier-quote rendering mismatches, plus a qualified internal-stage error
mismatch. Fixed these in external-table creation/error consumers; shared stage
grammar, whitespace parsing, and metadata projections remain unchanged.
## Test matrix
LOCATION / unqualified root x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[unqualified-LOCATION]
WITH LOCATION / unqualified root x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[unqualified-WITH LOCATION]
LOCATION / schema-qualified subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[schema-subpath-LOCATION]
WITH LOCATION / schema-qualified subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[schema-subpath-WITH LOCATION]
LOCATION / database-qualified nested subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[database-nested-subpath-LOCATION]
WITH LOCATION / database-qualified nested subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[database-nested-subpath-WITH LOCATION]
LOCATION / quoted qualified names x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[quoted-LOCATION]
WITH LOCATION / quoted qualified names x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[quoted-WITH LOCATION]
LOCATION / missing stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[missing-LOCATION]
WITH LOCATION / missing stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[missing-WITH LOCATION]
LOCATION / internal stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[internal-LOCATION]
WITH LOCATION / internal stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[internal-WITH LOCATION]
Following inline FILE_FORMAT + false REFRESH_ON_CREATE/AUTO_REFRESH x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[unqualified-WITH LOCATION]
Preceding inline FILE_FORMAT + GET_DDL recreation x metadata equality -> tests/queries/external_tables/test_external_table_with_location.py::test_with_location_get_ddl_roundtrip[inline]
Preceding named FILE_FORMAT + GET_DDL recreation x metadata equality -> tests/queries/external_tables/test_external_table_with_location.py::test_with_location_get_ddl_roundtrip[named]
Existing LOCATION / default GET_DDL x regression -> tests/queries/external_tables/test_external_table_get_ddl.py::test_get_ddl_inline_json_options
Existing LOCATION / internal stage x error regression -> tests/queries/external_tables/test_external_tables.py::test_location_internal_stage_rejected
Shared stage parser / quoted case-sensitive name x regression -> tests/queries/stages/test_copy_into_quoted_stage_location.py::test_unload_quoted_target_case_sensitive_inner
Shared stage parser / stage clauses x regression -> tests/queries/stages/test_create_stage_core_parity.py::test_create_stage_with_bare_tag
Matrix scope is the optional LOCATION modifier and its externalStage input
production from SF:SQL:STATEMENT:CREATE EXTERNAL TABLE. Equals whitespace remains
owned by LAV-3612; unrelated audited clauses and reader option slices are not
implemented or changed here. No existing test bodies or snapshots were edited.
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>1 parent c566847 commit da2030b
1 file changed
Lines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2605 | 2605 | | |
2606 | 2606 | | |
2607 | 2607 | | |
2608 | | - | |
| 2608 | + | |
| 2609 | + | |
| 2610 | + | |
2609 | 2611 | | |
2610 | 2612 | | |
2611 | 2613 | | |
| |||
0 commit comments