Skip to content

Task 10: orchestration and the postgres subcommand - #10

Merged
roachitect-aman merged 1 commit into
mainfrom
task-10-orchestration
Aug 25, 2026
Merged

Task 10: orchestration and the postgres subcommand#10
roachitect-aman merged 1 commit into
mainfrom
task-10-orchestration

Conversation

@roachitect-aman

Copy link
Copy Markdown
Contributor

run_postgres() wires the collectors together and postgres --output now produces a bundle end to end: probe the server version → check the client → fingerprint the catalog → dump the schema → read the catalog → read the statistics views → re-fingerprint → normalize → publish.

A bug the tests found

build_postgres_config resolved the whole output path, and Path.resolve() follows a symlinked final component. By the time task 9's publication-time symlink check ran there was no symlink left to see, so the tool would have written straight through it. The config now resolves the parent directory and keeps the final component as given. Two regression tests: the symlink survives into the config, and a .. in the directory part is still normalized away.

Fail early, fail completely

The tokenization key and the destination are both validated before a connection is opened — a missing DBPROFILER_TOKEN_KEY discovered after collection costs the operator a full pass over a production catalog, and so does an unwritable destination. Two tests assert those failures happen with zero child processes spawned.

An empty pg_dump is fatal: it exits zero when the role can see no objects, which would otherwise publish a bundle with an empty schema.sql and no other complaint. REQUIRED_BUNDLE_PATHS covers the same class of failure for profile.json and the three catalog CSVs — those may not degrade to a warning, and a bundle missing one of them looks complete and is not.

SchemaDrift is its own error class, so concurrent DDL is distinguishable from a command failure by a caller and not only by message text. KeyboardInterrupt exits 130; write_bundle already removes its own temporary file, so the handler has something to say and nothing to clean up.

Every failure path asserts the same three things: the exit code, that nothing was left in the destination directory, and that an existing bundle survives untouched.

Reporting

Progress goes to stderr; stdout is the bundle path and nothing else, so the run composes in a shell substitution. Source.collected_schemas is derived from what the catalog actually returned rather than from --schema-include, so a bundle that covered less than the operator expected says so on its face.

Testing

The fake child process dispatches on the SQL constant a call carries, not on call position. A positional side_effect list needs renumbering every time a step moves, and asserting "17 calls happened" proves nothing about their order; keying on the query makes the order assertion one comparison against the documented sequence, and a list-valued override is consumed one entry per call, which is how the two fingerprint reads are given different answers to simulate concurrent DDL.

Secrecy is asserted for five values — URL, password, user, host, tokenization key — against stdout, stderr and the bundle bytes, on the success path and on two failure paths where a server error deliberately echoes the URL and the password back. Two further tests assert the credentials reach the child through env= and never through argv, and that neither DBPROFILER_* variable is passed down.

As in task 9, the guarantees were mutation-tested: dropping the after-collection fingerprint comparison, neutering the comparison itself, publishing an incomplete bundle, and deferring the key load until after collection are each caught by the suite.

Verification

  • 354 tests pass
  • python3 dbprofiler.py --check-safety — OK, 14 SQL constants
  • ruff check — clean
  • source parses under Python 3.9

run_postgres() wires the collectors together: probe the server version, check
the client, fingerprint the catalog, dump the schema, read the catalog, read
the statistics views, re-fingerprint, normalize, and publish.

Fixes a bug the orchestration tests found. build_postgres_config resolved the
whole output path, and Path.resolve() follows a symlinked final component, so
by the time the publication-time symlink check ran there was no symlink left
to see -- the tool would have written straight through it. The config now
resolves the parent directory and keeps the final component as given.

The tokenization key and the destination are both validated before a
connection is opened. A missing DBPROFILER_TOKEN_KEY discovered after
collection costs the operator a full pass over a production catalog, and so
does an unwritable destination.

An empty pg_dump is fatal: pg_dump exits zero when the role can see no
objects, which would otherwise publish a bundle with an empty schema.sql and
no other complaint. REQUIRED_BUNDLE_PATHS covers the same class of failure for
the profile and the three catalog CSVs, which may not degrade to a warning.

Source.collected_schemas is derived from what the catalog returned rather than
from --schema-include, so a bundle that covered less than the operator
expected says so on its face.

Progress goes to stderr and stdout carries the bundle path alone, so the run
composes in a shell substitution. KeyboardInterrupt exits 130; write_bundle
already removes its own temporary file, so the handler only has something to
say.

The test fake dispatches on the SQL constant a call carries rather than on
call position, so the order assertion is one comparison against the documented
sequence and an override names the step it replaces. Secrecy is asserted for
five values across stdout, stderr and the bundle bytes, on the success path
and on two failure paths where a server error deliberately echoes the URL and
the password back.

As in task 9, the guarantees were mutation-tested: dropping the
after-collection fingerprint comparison, neutering the comparison, publishing
an incomplete bundle, and deferring the key load until after collection are
each caught.

354 tests pass; --check-safety and ruff are clean; the source parses under
Python 3.9.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@roachitect-aman
roachitect-aman merged commit ffbdc1f into main Aug 25, 2026
7 checks passed
@roachitect-aman
roachitect-aman deleted the task-10-orchestration branch August 25, 2026 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant