Skip to content

GH-53872: [Format][FlightSQL][C++] Add Substrait dialect SqlInfo codes - #54360

Open
tokoko wants to merge 2 commits into
apache:mainfrom
tokoko:flight-sql-substrait-dialect
Open

tokoko wants to merge 2 commits into
apache:mainfrom
tokoko:flight-sql-substrait-dialect

Conversation

@tokoko

@tokoko tokoko commented Oct 18, 2026

Copy link
Copy Markdown

Rationale for this change

Flight SQL can already report whether a server accepts Substrait plans and which versions it accepts (FLIGHT_SQL_SERVER_SUBSTRAIT, _MIN_VERSION, _MAX_VERSION). It can't report which part of Substrait the server accepts. In practice no consumer implements the whole spec, so a producer only learns that a plan is unsupported when execution fails.

Substrait now has a standard format for this: dialect files. A dialect is a YAML document that lists the types, relations, expressions and function signatures a system supports. substrait-java already publishes one for Spark (spark_dialect.yaml).

What changes are included in this PR?

Two new SqlInfo codes in FlightSql.proto, mirrored in the C++ SqlInfoOptions::SqlInfo enum:

  • SQL_SUBSTRAIT_DIALECT = 577: a UTF-8 string holding a Substrait dialect YAML document. Should only be returned if FLIGHT_SQL_SERVER_SUBSTRAIT is true. If it isn't returned, the server doesn't describe which parts of Substrait it accepts; that doesn't mean it accepts all of Substrait.
  • SQL_SUBSTRAIT_DIALECT_HASH = 578: an opaque string identifying the dialect's content, for client-side caching. It must change whenever the dialect changes (e.g. a hex-encoded SHA-256 digest of the dialect). Clients compare it for equality only.

The hash code exists because of size. A dialect covering every function in the standard Substrait extensions plus all types, relations and expressions is about 34 KiB of YAML (about 3.6 KiB gzipped). Real dialects are subsets of that. This is fine to fetch once per connection, but chatty clients can fetch the hash first and only refetch the dialect when the hash changes.

Both codes are in the [500, 1000) range with the other SQL syntax codes, since they describe the syntax the server accepts. Because ADBC reuses Flight SQL's info codes, ADBC gets the same codes without a separate change.

Are these changes tested?

No. The change only adds enum values and documentation; no behavior changes.

Are there any user-facing changes?

Yes. Two new SqlInfo codes are available to Flight SQL servers and clients. The change is backwards-compatible: no existing codes or types change.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #53872 has been automatically assigned in GitHub to PR creator.

@lidavidm lidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread cpp/src/arrow/flight/sql/types.h Outdated
Comment thread format/FlightSql.proto
@github-actions github-actions Bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants