Skip to content

Commit da2030b

Browse files
localstack-spiral[bot]spiralhovaesco
authored
LAV-3746: Support WITH LOCATION for external tables (#4098)
Accept the optional WITH token in the Snowflake external-table clause loop, using the existing LOCATION parser and registry-driven creation path. Preserve WITH TAG and WITH ROW ACCESS POLICY parsing, and leave equals whitespace and shared stage grammar untouched. Add 14 metadata-only Cloud-captured compat cases for paired LOCATION spellings, unqualified/schema/database/quoted names, root and nested subpaths, missing and internal stages, following format/refresh options, and inline/named-format GET_DDL recreation with identical DESC and INFORMATION_SCHEMA metadata. Normalize named external-table references through the existing stage parser and identifier renderer before storing them. Keep unqualified stages relative so existing default GET_DDL snapshots remain unchanged. Report the resolved internal stage's leaf name rather than the supplied namespace. Verification: - Cloud capture: TEST_TARGET=SNOWFLAKE_CLOUD PYTEST_XDIST_ARGS='' make test-compat PYTEST_PATH=tests/queries/external_tables/test_external_table_with_location.py PYTEST_ARGS='--snapshot-update' (14 passed). - Before implementation, the unqualified WITH case failed on unexpected WITH; the bare root case also exposed Cloud's canonical trailing slash. - make check passed (clippy, warnings-as-errors, formatting). - make test-vendored-sqlparser passed, including shared stage parser tests. - Serial make test-compat over the new file, external_table_get_ddl, create_stage_core_parity, and copy_into_quoted_stage_location: 37 passed. - RUN_LOCALSTACK_TESTS=1 serial make test-compat over test_external_tables.py with -k test_location_internal_stage_rejected: 1 passed. - All paired Cloud recorded-content values are identical; each recorded roundtrip's DDL, DESC and metadata before/after values are identical. Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud> ## Deviations New Cloud snapshots exposed existing root-path, schema-qualified, and redundant identifier-quote rendering mismatches, plus a qualified internal-stage error mismatch. Fixed these in external-table creation/error consumers; shared stage grammar, whitespace parsing, and metadata projections remain unchanged. ## Test matrix LOCATION / unqualified root x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[unqualified-LOCATION] WITH LOCATION / unqualified root x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[unqualified-WITH LOCATION] LOCATION / schema-qualified subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[schema-subpath-LOCATION] WITH LOCATION / schema-qualified subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[schema-subpath-WITH LOCATION] LOCATION / database-qualified nested subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[database-nested-subpath-LOCATION] WITH LOCATION / database-qualified nested subpath x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[database-nested-subpath-WITH LOCATION] LOCATION / quoted qualified names x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[quoted-LOCATION] WITH LOCATION / quoted qualified names x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[quoted-WITH LOCATION] LOCATION / missing stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[missing-LOCATION] WITH LOCATION / missing stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[missing-WITH LOCATION] LOCATION / internal stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[internal-LOCATION] WITH LOCATION / internal stage x error -> tests/queries/external_tables/test_external_table_with_location.py::test_location_stage_errors[internal-WITH LOCATION] Following inline FILE_FORMAT + false REFRESH_ON_CREATE/AUTO_REFRESH x success -> tests/queries/external_tables/test_external_table_with_location.py::test_location_spellings[unqualified-WITH LOCATION] Preceding inline FILE_FORMAT + GET_DDL recreation x metadata equality -> tests/queries/external_tables/test_external_table_with_location.py::test_with_location_get_ddl_roundtrip[inline] Preceding named FILE_FORMAT + GET_DDL recreation x metadata equality -> tests/queries/external_tables/test_external_table_with_location.py::test_with_location_get_ddl_roundtrip[named] Existing LOCATION / default GET_DDL x regression -> tests/queries/external_tables/test_external_table_get_ddl.py::test_get_ddl_inline_json_options Existing LOCATION / internal stage x error regression -> tests/queries/external_tables/test_external_tables.py::test_location_internal_stage_rejected Shared stage parser / quoted case-sensitive name x regression -> tests/queries/stages/test_copy_into_quoted_stage_location.py::test_unload_quoted_target_case_sensitive_inner Shared stage parser / stage clauses x regression -> tests/queries/stages/test_create_stage_core_parity.py::test_create_stage_with_bare_tag Matrix scope is the optional LOCATION modifier and its externalStage input production from SF:SQL:STATEMENT:CREATE EXTERNAL TABLE. Equals whitespace remains owned by LAV-3612; unrelated audited clauses and reader option slices are not implemented or changed here. No existing test bodies or snapshots were edited. Co-authored-by: spiral <spiral@localhost> Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
1 parent c566847 commit da2030b

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

‎src/dialect/snowflake.rs‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2605,7 +2605,9 @@ fn parse_create_external_table(
26052605
Expr::Tuple(exprs)
26062606
};
26072607
builder = builder.partition_by(Some(Box::new(expr)));
2608-
} else if parser.parse_keyword(Keyword::LOCATION) {
2608+
} else if parser.parse_keywords(&[Keyword::WITH, Keyword::LOCATION])
2609+
|| parser.parse_keyword(Keyword::LOCATION)
2610+
{
26092611
let _ = parser.consume_token(&Token::Eq);
26102612
builder = builder.location(Some(parse_external_table_location(parser)?));
26112613
} else if parser.parse_keyword(Keyword::FILE_FORMAT) {

0 commit comments

Comments
 (0)