Collect system.query_views_log: materialized-view execution

[CamiloSierraH]

Why

A materialized view that throws, or that "succeeds" while writing nothing, is invisible in query_log — the parent INSERT reports success either way. system.query_views_log is the only place an MV's own execution is recorded, and the bundle explicitly listed it under "deliberately does not contain".

Reviewing the closed escalations that touch query_views_log turned up five recurring questions, each already answered by support engineers running ad-hoc hourly aggregates against this table:

Pattern	Signature
Silent row drop	QueryFinish, exception_code = 0, read_rows > 0, written_rows = 0
MV push fails after the base part committed	status = 'ExceptionWhileProcessing' (60 / 241 / 252 / 341)
Per-MV memory regression after an upgrade	peak_memory_usage median/max vs flat written_rows
MVs burning CPU	view_duration_ms
Divergence between hops of a chain	written_rows per view per bucket

What

1. The collector. system.query_views_log_3_days in all three modes, 1-hour buckets keyed by view, target, type, status and exception_code. Per bucket: executions, zero_write_executions, row/byte sums, view duration sum + max, peak memory as avg/median/max, and a 500-char sampled exception.

2. The pre-pass reads it. Six checks in inspect_bundle.py, so MV failures reach triage instead of waiting for someone to go looking.

3. Skill docs. A file-guide.md entry, HC-5.9 and HC-7.6/7.7/7.8, patterns P-35 (MV runs clean but writes nothing) and P-36 (MV push throws after the base part committed), a bundle-layout.md row, P-33/P-34 upgraded from hedged references to the real file, and query_views_log struck from the never-collected list.

Keeping the query light

query_views_log is one row per (insert × view) and rivals part_log in size, so the shape matters more than the window:

• Aggregate, never raw rows. GROUP BY state stays bounded because every key is low-cardinality. initial_query_id and the exception text are deliberately not keys — that is the part_name regression part_log documents.
• No uniqExact on the exception string. exception_code is a group key, so any(exception) is already representative; a hash set of full messages would buy nothing and cost real CPU.
• stack_trace, view_query and ProfileEvents are never read — the bulk of the table's bytes.
• gov hashes in the outer select, over the aggregated result, so SHA256 runs once per output row instead of once per source row.

The pre-pass checks

The two that required care:

• Failures are attributed, not counted. A failed INSERT writes an ExceptionWhileProcessing row for every view in the pipeline with the same message. Counting rows per view reports four broken MVs when one is broken. The check names the highest-count view as the culprit and reports the rest as blast radius.
• Zero-write is read as a change, not an absolute. A filtering MV legitimately writes nothing forever, so the flat > 0.9 ratio in HC-7.7 is not what got implemented. Only a view with a mixed bucket history — it wrote, then stopped — warns. Views that never wrote get one info line.

The rest: an HC-0 coverage gate (MVs exist in system.tables but no query_views_log → log_query_views = 0; without this, "no MV findings" is indistinguishable from "no telemetry"), a per-view memory step change (≥ 3× median peak across the window halves with writes flat, ≥ 6 buckets required), an mv failures N flag on the existing incident timeline, and a top-10 views table — the only output that describes the MV topology on a bundle where nothing is wrong.

Three things found by testing rather than reading

• view_uuid is always the zero UUID. It was in the design as a recreate-detector; on 26.7.5.10 the server writes zeros into it for every view, TO-target and inner-table alike, while system.tables carries the real one. Dropped. For an MV declared without TO, view_target is db.`.inner_id.<uuid>` and that changes on a recreate.
• The pipeline-wide exception marking above — reproduced with four MVs where only one was faulty.
• merge() with no matching table fails with 636 CANNOT_EXTRACT_TABLE_STRUCTURE, not 60. That is the recognisable "not collected" when <query_views_log> is unconfigured.

merge(system, '^query_views_log') rather than the bare table so a post-upgrade query_views_log_0 still counts — the pre-upgrade baseline is exactly what a memory regression needs.

Verification

SQL against 22.8.21.38 (the floor) and 26.7.5.10:

• all six files execute, including GROUP BY ALL, clusterAllReplicas and the gov hashing
• onprem, cloud and gov collect end-to-end; the 23.11 rung is selected correctly
• both signatures reproduced in a purpose-built MV workload — a filtering MV at zero_write_executions = 5/5, a broken JOIN at exception_code = 60
• gov output contains no raw identifiers and no exception text

Pre-pass against four bundles: a real onprem one (correct culprit attribution across four views where one was faulty), a gov one (hashed labels, identical ratios), one with the file removed (coverage gate fires), and a synthetic fixture for the stopped-writing and memory-step branches — where a steady view correctly raised nothing. --json mode checked on all four.

make test passes, including TestShippedQueryDefaultWindows and TestGovQueries_NoRawIdentifiersOrDDL.

Review follow-up

Six findings, all addressed in the third commit and each checked against a live server first:

#	Finding	Verdict	Fix
1–2	gov split view_target on every dot; .inner_id.<uuid> targets hashed to a bare backtick	real	split at the first dot, strip backticks; hashes verified to equal system.tables's for TO and inner targets
3	QueryStart rows counted as executions	theoretical — the server writes no QueryStart rows for views (checked)	skipped anyway, as the query_log pass does
4	attribution by row count ties on every sibling	real — and the fix is exact, not heuristic: every sibling's message says while pushing to view <culprit>	parsed from the text; strict count lead as fallback; otherwise no view is named
5	stopped-writing lost bucket order — a view that started writing fired too	real	ordered per-hour history: last write, then ≥ 2 later read-only hours; fixture with a started-writing view stays silent
6	HC-7.7 documented the flat ratio the code deliberately doesn't implement	real	row rewritten to the ordered rule, with the reason the ratio is not it

Re-run on the real bundle (mv_join named from the text, three siblings correctly reported as blast radius) and on two fixtures (--json clean); make test passes.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved moderate issues affect collector correctness, diagnostics, and documentation alignment.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 5 · 1

Open (6)

• Incorrect parsing of dotted inner MV targets · New
• Incorrect parsing of dotted inner MV targets · New
• QueryStart rows are incorrectly counted as executions · New
• Failed view attribution ignores exception details · New
• Write-stop detection loses bucket ordering · New
• Documented warning condition differs from implemented check · New

What changed in this PR

Adds system.query_views_log collection and materialized-view diagnostics across ClickHouse deployment modes, including failure, zero-write, memory, and coverage analysis.

Changes:

• Adds versioned collectors for on-prem, cloud, and government environments.
• Extends bundle inspection and reporting with MV health checks.
• Updates documentation, patterns, bundle layout, and default-window tests.

Unresolved issues remain in collector target parsing, execution accounting, exception attribution, temporal zero-write detection, coverage interpretation, and health-check documentation alignment.

File	Summary
skills/​clickhouse-diagnostic/​SKILL.md	Documents MV telemetry coverage.
skills/​clickhouse-diagnostic/​scripts/​inspect_bundle.py	Adds MV analysis and reporting.
skills/​clickhouse-diagnostic/​references/​running-the-tool.md	Documents the MV collection window.
skills/​clickhouse-diagnostic/​references/​known-patterns.md	Adds MV failure and zero-write patterns.
skills/​clickhouse-diagnostic/​references/​health-checks.md	Adds MV health-check guidance.
skills/​clickhouse-diagnostic/​references/​file-guide.md	Documents the new bundle file.
skills/​clickhouse-diagnostic/​references/​bundle-layout.md	Defines MV archive schema and redaction.
README.md	Lists the collector and window.
queries.onprem/​system.query_views_log_3_days.sql	Adds the baseline on-prem collector.
queries.onprem/​23.11.1.0/​system.query_views_log_3_days.sql	Adds the versioned on-prem collector.
queries.gov/​system.query_views_log_3_days.sql	Adds the redacted government collector.
queries.gov/​23.11.1.0/​system.query_views_log_3_days.sql	Adds the versioned redacted government collector.
queries.cloud/​system.query_views_log_3_days.sql	Adds the cloud collector.
queries.cloud/​23.11.1.0/​system.query_views_log_3_days.sql	Adds the versioned cloud collector.
internal/​query/​window_test.go	Verifies the new default window.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[CamiloSierraH]

Fixed in 7b210e7 — correct. Verified live: for an MV declared with an ENGINE, view_target is db.\.inner_id.`andsplitByChar('.', …)[2]was a bare backtick, so every inner target hashed to one value that matched nothing. Both gov files now split at the first dot and strip backticks; checked that the hashed halves equalhex(SHA256(database))/hex(SHA256(name))of the matchingsystem.tables` row for a TO target and an inner target alike.

[CamiloSierraH]

Fixed in 7b210e7 — correct. Verified live: for an MV declared with an ENGINE, view_target is db.\.inner_id.`andsplitByChar('.', …)[2]was a bare backtick, so every inner target hashed to one value that matched nothing. Both gov files now split at the first dot and strip backticks; checked that the hashed halves equalhex(SHA256(database))/hex(SHA256(name))of the matchingsystem.tables` row for a TO target and an inner target alike. (same fix, root file.)

[CamiloSierraH]

Guarded in 7b210e7. For the record it was theoretical: the server writes only QueryFinish / ExceptionWhileProcessing for views — checked SELECT status, count() FROM system.query_views_log GROUP BY status on 26.7, no QueryStart rows exist. Skipped anyway, as the query_log pass does, so a future change in what the server logs cannot double-count.

[CamiloSierraH]

Fixed in 7b210e7 — right, and the data makes it exact rather than heuristic. Every sibling's row carries the same message, and that message ends with … while pushing to view db.mv naming the view that threw — at position ~200 of 287 chars in the real bundle, inside the 500-char cap. The pre-pass now parses it. Without text (gov) it names a view only on a strict count lead; with neither it lists the failing views and says attribution is unavailable. Fixture: three siblings at 7 failures each → the named one is picked, not the first in input order.

[CamiloSierraH]

Fixed in 7b210e7 — correct, the counters lost order. The pre-pass now keeps the per-hour history and looks for the last hour with writes followed by ≥ 2 later hours that read rows and wrote none; the message quotes both hours. A view that started writing (zeros first, writes later) has an empty trailing run and is not flagged — fixture-tested. I kept the ≥ 2 trailing-hour threshold rather than 1: a single zero-write hour after a write is what any bursty MV looks like at the end of the window.

[CamiloSierraH]

Fixed in 7b210e7 — the row now describes what the code implements (the ordered stop rule, info for a view that never wrote) and says explicitly why the flat ratio is not the rule: it fires on every correctly filtering MV. HC-7.6 also gained the attribution method.