.transform(strict=) and sqlite-utils transform --strict/--no-strict

[simonw]

Refs:

• .transform(strict=True/False) and sqlite-utils transform --strict/--no-strict #787

---

📚 Documentation preview 📚: https://sqlite-utils--788.org.readthedocs.build/en/788/

[simonw]

use uv run python -c and manually exercise the new .transform(strict=) option, see if you can find any edge-cases or bugs

GPT-5.6 Sol then found some edge-cases:

Manual probing found two bugs directly relevant to the new option.

1. Explicit strict=True silently fails on unsupported SQLite

Simulating an older SQLite version produced:

t.transform(strict=True)
t.strict  # False

create_table_sql() only appends STRICT when db.supports_strict is true (db.py). An explicit conversion should probably raise an informative error instead of succeeding as a no-op.

2. The Table object retains stale strict defaults

After:

t = db.table("items", strict=True)
t.create({"id": int})
t.transform(strict=False)

The schema becomes non-strict, but t._defaults["strict"] remains True. Dropping and recreating through the same object unexpectedly creates another strict table. The reverse happens when converting from non-strict to strict.

The default should be updated after a successful transformation, including storing False. It should remain unchanged if the transformation rolls back.

Other notable edge cases:

• SQLite ANY columns are introspected as float, so transforming a native strict table containing ANY can fail or change it to REAL. This is an existing transform/introspection limitation, but applies to strict=None and strict=True.
• An incompatible declared default does not prevent conversion. An empty INTEGER DEFAULT 'not-an-integer' table becomes strict successfully, but later inserting the default raises IntegrityError. This is SQLite behavior.
• Triggers are dropped during transformation. This is existing documented behavior, not introduced by this change.
• Losslessly convertible values work: "123" and "123.0" become integer 123.
• Fractional numbers, incompatible text, and BLOBs correctly fail and roll back.
• Indexes and foreign keys were preserved.
• Failed conversion inside an existing db.atomic() transaction rolled back only the transform; the outer transaction remained usable.
• keep_table correctly retained the original non-strict table alongside the new strict table.
• Repeated None → True → None → False → None transitions behaved correctly.

No files were changed during this diagnostic pass.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.55%. Comparing base (6531a57) to head (989729d).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #788   +/-   ##
=======================================
  Coverage   95.54%   95.55%           
=======================================
  Files           9        9           
  Lines        3793     3798    +5     
=======================================
+ Hits         3624     3629    +5     
  Misses        169      169           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[simonw]

I'm going to raise errors if you attempt to convert to STRICT with a SQLite version that fails the db.supports_strict test.

[simonw]

We don't have any mechanism to support ANY columns at the moment. Open question how to deal with that. Options include:

• Ignore the problem entirely
• Add a sqlite_utils.ANY constant which can be used in create table calls, e.g. db.create_table("t", {"id": int, "name": str, "misc": sqlite_utils.ANY}) - would have to be handled in add_column() and transform() and a bunch of other places too.
• Don't support them in create_table/etc but DO support them in introspection, since that's part of how transform() works

[simonw]

Here's a reproduction of the problem where the default strict value for a table is not correctly updated:

from sqlite_utils import Database

db = Database(memory=True)

table = db.table("items", strict=True)
table.create({"id": int})

table.transform(strict=False)
assert table.strict is False

# Recreate using the same Table object's stale strict=True default:
table.create({"id": int}, replace=True)

assert table.strict is True  # Unexpectedly strict again