Skip to content

Add SQLAlchemy 2.1 support - #489

Open
Claire-ovo wants to merge 12 commits into
agronholm:masterfrom
Claire-ovo:fix/nested-type-arguments
Open

Claire-ovo wants to merge 12 commits into
agronholm:masterfrom
Claire-ovo:fix/nested-type-arguments

Conversation

@Claire-ovo

@Claire-ovo Claire-ovo commented Oct 23, 2026 •

Copy link
Copy Markdown

Changes

Adds SQLAlchemy 2.1 support and runs the test suite against both SQLAlchemy 2.0 and 2.1.

  • _render_column_type_value() now renders a MetaData argument as the generated module's metadata reference instead of MetaData(). SQLAlchemy 2.1 attaches PostgreSQL DOMAIN types to the table's metadata automatically, so generated code referenced an unimported MetaData in declarative output (NameError) and detached the domain's DDL from the generated tables.
  • Tests whose expected output differs between 2.0 and 2.1 (DOMAIN, JSON, GeoAlchemy2) are split into _sa20 / _sa21 variants, selected by a new sqlalchemy_version fixture that skips the variant not matching the installed version.
  • CI matrix now runs every Python version against both SQLAlchemy>=2.0.29,<2.1 and SQLAlchemy>=2.1,<2.2 (2.1 excluded on Python 3.10, which it does not support).
  • New tests/test_domain_metadata.py checks, for all four generators, that the generated DOMAIN shares the generated metadata and that create_all / drop_all emit the domain DDL exactly once.

The ClickHouse nested type argument rendering (#469) was split out of this PR.

Checklist

  • Added tests which fail without the patch.
  • Added a changelog entry.

@Claire-ovo

Copy link
Copy Markdown
Author

CI has finished: Python 3.10 and pre-commit.ci pass; Python 3.11–3.14 fail in the six existing DOMAIN/GeoAlchemy tests noted in the PR description. The eight new regression cases pass.

I reproduced all six failures locally with SQLAlchemy 2.1.6, then loaded the unmodified generator from master c7eef1f in the same environment and reran those six tests: they fail identically. With SQLAlchemy 2.0.55 the full suite passes (171 tests).

There is also a concrete runtime consequence in the existing DOMAIN path: after a DOMAIN is attached to a table, inherited constructor introspection emits metadata=MetaData() without a corresponding import in declarative output. Executing that output raises NameError. This reproduces on unmodified master as well; the JSON/GeoAlchemy annotation differences are a separate part of the 2.1 compatibility work.

I have kept this PR focused on nested type arguments and have not changed dependency constraints or refreshed unrelated snapshots to hide these failures. I can investigate the 2.1 compatibility work in a separate follow-up if that fits the project's direction.

@sheinbergon

Copy link
Copy Markdown
Collaborator

@Claire-ovo thank you for your contribution.

If you want to move forward, please stop using AI for communication. Neither @agronholm nor I should have to go through some semi readable heap of text just to get to the point.

Tests should pass before we even begin discussing this PR.

@coveralls

coveralls commented Oct 23, 2026 •

Copy link
Copy Markdown

Coverage Status

coverage: 97.933% (+0.08%) from 97.857% — Claire-ovo:fix/nested-type-arguments into agronholm:master

@sheinbergon sheinbergon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should split the work between supporting SA 2.1 and supporting Clickhouse nested imports. Up to you which do you want to tackle 1st.

As for SA 2.1, should be implemented differently as commented

10x

Comment thread tests/test_generator_declarative.py Resolved
@sheinbergon sheinbergon changed the title Fix rendering and imports of nested type arguments Add SQLAlchemy 2.1 support Nov 9, 2026
sheinbergon added a commit to Claire-ovo/sqlacodegen that referenced this pull request Nov 9, 2026
…lm#489

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sheinbergon

Copy link
Copy Markdown
Collaborator

@agronholm this is a welcomed addition

@loora-idans

Copy link
Copy Markdown

@agronholm WDYT?

@agronholm

Copy link
Copy Markdown
Owner

This almost escaped my attention (I get more PRs I can handle these days). I thought we already supported SQLA 2.1.

@agronholm

Copy link
Copy Markdown
Owner

Adding support for newer SA is good, but I'll need to review this in depth. I'll try to get that done within the next couple of days.

Comment thread tests/test_generator_declarative.py Resolved
Comment thread tests/test_generator_tables.py Resolved
@agronholm

Copy link
Copy Markdown
Owner

I feel that some of the tests that are currently duplicated should probably not be. I find it hard to believe that SQLAlchemy 2.1 is so fundamentally different to require that many changes.

@sheinbergon
sheinbergon force-pushed the fix/nested-type-arguments branch from 379ae3b to cfa486d Compare November 18, 2026 18:58
sheinbergon and others added 4 commits November 18, 2026 21:02
The SQLAlchemy 2.1 behavior is already covered by the version-parametrized
generator tests.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The pin made the SQLAlchemy 2.1 CI jobs fail at dependency resolution.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
SQLAlchemy 2.1 uses a sentinel default for SchemaType's schema argument and
sets the table's metadata on the type when its column is attached to a table,
which made both look like non-default arguments. Neither is needed in the
generated code, so the output for the plain domain tests is the same on 2.0
and 2.1 again and their version-specific copies are gone.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@sheinbergon

Copy link
Copy Markdown
Collaborator

I feel that some of the tests that are currently duplicated should probably not be. I find it hard to believe that SQLAlchemy 2.1 is so fundamentally different to require that many changes.

Yes, you were right. Only 3 pairs actually required duplication (domain mapping related)

sheinbergon and others added 2 commits November 18, 2026 21:19
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@sheinbergon

Copy link
Copy Markdown
Collaborator

@agronholm fixed applied. Looks like we have a lot of small fixes/improvents to release. Please share your thoughts on this PR at it's current state

@agronholm agronholm left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of minor comments worth considering. Otherwise I'm okay with this.

Comment on lines +15 to +19
sqlalchemy: [">=2.0.29,<2.1", ">=2.1,<2.2"]
exclude:
# SQLAlchemy 2.1 requires Python 3.11 or later.
- python-version: "3.10"
sqlalchemy: ">=2.1,<2.2"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This gives us 9 CI jobs. Do we need all of them? We could just add the SQLA 2.0 jobs as inclusions (instructions here).

Comment on lines +32 to +33
- name: Show dependency version
run: python -c "import sqlalchemy; print(sqlalchemy.__version__)"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we need this for? The previous step already shows clearly which version is installed.

@agronholm

Copy link
Copy Markdown
Owner

Dropping Python 3.10 might also be worth considering, though it's out of scope for this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants