Skip to content

Collect Tier 1 workload telemetry - #6

Merged
roachitect-aman merged 1 commit into
mainfrom
task-6-workload-telemetry
Aug 25, 2026
Merged

Collect Tier 1 workload telemetry#6
roachitect-aman merged 1 commit into
mainfrom
task-6-workload-telemetry

Conversation

@roachitect-aman

Copy link
Copy Markdown
Contributor

Task 6 of the MVP plan.

Three best-effort statistics sources: per-table activity (pg_stat_user_tables), per-index scan counts and size (pg_stat_user_indexes), and top statements (pg_stat_statements).

Each source degrades independently. A role that can read the catalog but not a statistics view gets a bundle with that section omitted and a warning beside it — the operator running the profile is often not the person who can grant the missing privilege. Four warning codes distinguish not installed from installed but unreadable; only the second is worth retrying with more access. pg_stat_statements_info failing costs the reset timestamp, not the counters.

Top-N is ranked in SQL with two window functions in one pass — slowest and most frequent are different questions, and a migration plan needs both. Capped at 400 rows, scoped to the profiled database (pg_stat_statements is cluster-wide; without the filter the profile would describe a workload that never touched the target), deduplicated by queryid in Python keeping the most expensive occurrence.

The forbidden-token check now matches ANALYZE on word boundaries. pg_stat_user_tables exposes last_analyze, analyze_count, autoanalyze_count and n_mod_since_analyze, and a substring match rejected all four. Those columns are how a reader knows whether the source statistics are stale enough to make the rest of the profile untrustworthy — dropping them would trade real signal for the appearance of rigour. ANALYZE public.users and VACUUM ANALYZE are still rejected; \bCOUNT\( still catches pg_catalog.count(*) and now correctly ignores the tail of autovacuum_count. Six guard tests, both directions.

pg_database and pg_extension join the relation allowlist alongside the collectors that read them. The extension probe runs first, so a database without pg_stat_statements costs one query rather than two failures.

Query text is collected raw into a record that is never serialized; tokenization lands in task 7 where the HMAC key handling lives.

Verification: 185 tests pass, --check-safety clean over 14 SQL constants, ruff check clean, 3.9 grammar verified.

Adds three best-effort statistics sources: per-table activity from
pg_stat_user_tables, per-index scan counts and size from
pg_stat_user_indexes, and the top statements from pg_stat_statements.

Each source degrades independently. A role that can read the catalog but
not a statistics view gets a bundle with that section omitted and a warning
beside it, because the operator running the profile is often not the person
who can grant the missing privilege. Four warning codes distinguish "not
installed" from "installed but unreadable": only the second is worth
retrying with more access.

Statements are ranked in SQL with two window functions -- slowest and most
frequent are different questions -- capped at 400 rows, scoped to the
profiled database, and deduplicated by queryid in Python.

The forbidden-token check now matches ANALYZE on word boundaries.
pg_stat_user_tables exposes last_analyze, analyze_count, autoanalyze_count
and n_mod_since_analyze, and a substring match rejected all four. Those
columns are how a reader knows whether the source statistics are stale
enough to make the rest of the profile untrustworthy, so dropping them
would have traded real signal for the appearance of rigour. ANALYZE as a
statement is still rejected; there are guard tests in both directions.

pg_database and pg_extension join the relation allowlist alongside the
collectors that read them.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@roachitect-aman
roachitect-aman merged commit e369cb3 into main Aug 25, 2026
7 checks passed
@roachitect-aman
roachitect-aman deleted the task-6-workload-telemetry branch August 25, 2026 00:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant