Task 12: end-to-end integration test

[roachitect-aman]

Runs the shipped script as a subprocess against a real PostgreSQL 16 and checks the bundle it produces. Skipped unless DBPROFILER_POSTGRES_TEST_URL and DBPROFILER_TOKEN_KEY are both set, and named so python3 -m unittest never discovers it — the offline suite stays offline.

What it covers

50 tests over one disposable schema: bundle structure and checksums, collection scope, table and column shape, foreign keys and fan-out, Tier 1 telemetry, tokenization, and the negative assertion.

The fixtures are seeded so the right answer is computable rather than guessed. Hot orders go to five customer ids disjoint from the evenly spread range, with a stride coprime to it, so exactly 405 parents are referenced. org_id and site_id are correlated into five pairs from two orgs and five sites, so reading the extended statistics gives 5 where assuming independence would give 10 — before that change the composite fan-out test would have passed either way.

Asserting the negative

Four values are planted in the fixtures, each reaching the statistics by a different route: a most-common value, a histogram bound, a most-common value on a composite parent, and a utility statement's verbatim text in pg_stat_statements (which PostgreSQL does not normalize to $1). None may appear in the archive as stored or in any member decompressed. Two further tests prove each plant really is in the source, so a fixture that silently failed to insert cannot make every assertion of absence pass for the wrong reason.

Three corrections to the draft

Wrong about the tool rather than the tool being wrong, and each worth recording:

• data_type is format_type output — timestamp with time zone, numeric(12,2) — not the internal type name.
• Fan-out is children per referenced parent, so it divides by the child column's distinct count, not the parent's row count.
• p99 is read at the rank the distinct count implies, so a single hot parent among 376 sits past the 99th percentile and does not lift it. The seed now uses five hot parents.

Also: the plan's "unsupported JSON/array columns" was wrong. jsonb and text[] are both supported, so a fixture built from them would have asserted nothing. The exotic table carries an int4range and a CREATE DOMAIN column instead, and the suite asserts both directions.

Guarding the guard

TestIntegrationSuiteScope parses integration_test.py with ast and fails if any mutating statement does not name {SCHEMA}, if the schema name stops being unique per run, if drop_fixtures stops being called from both the success and the failure path, if a credentialed URL or a new token key appears, or if the file ever becomes discoverable by plain unittest. It earned its place immediately: a mutation-testing step left DROP SCHEMA IF EXISTS public CASCADE in the file, and the guard caught it from a static parse with no server involved.

The token test is itself guarded. Asserting an overlap between the parent's and the child's token lists would also pass if both sides were tokenized under a domain that merely happened to match, so the suite computes the expected token from the key and asserts it is present, then asserts the token the child's own domain would produce is absent.

Mutation testing

• Tokenizer returns raw values → 14 failures, including every negative assertion.
• Child columns tokenize under their own domain → exactly the two tests written for that property, nothing else.
• Composite fan-out assumes independence → exactly the composite test.
• Unqualified DROP SCHEMA → the new static guard, with no server present.

Rule change

.claude/rules/development.md exempted only ANALYZE for disposable fixtures, but multicolumn statistics need CREATE STATISTICS. The exemption now names the DDL, DML, ANALYZE, CREATE STATISTICS and COUNT(*) this one file needs, scopes it to integration_test.py and the schema it creates and drops, and says why — the tool reads statistics rather than computing them, so something has to compute them first. It also forbids comparing the profiler's estimates against a count queried at assertion time, since such a count would agree with an estimator broken in the same direction.

Verification

• python3 -m unittest — 371 passed (was 363).
• python3 -m unittest integration_test — 50 passed against PostgreSQL 16.15; 50 skipped when unconfigured.
• python3 dbprofiler.py --check-safety — OK, 14 SQL constants.
• ruff check — clean. Parses under the 3.9 grammar.
• public schema of the test database verified intact; no fixture schema left behind.

README.md gains a "What this release does not do" section listing the deferred scope, and the status note now says the tool is exercised end to end.