cpu_attribution on the top-CPU rankings: what fraction of the box the ranking explains (#2320)

CHANGELOG.md

@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **`cpu_attribution` on the top-CPU rankings: what fraction of the box the ranking explains** ([#2320]) - the last unshipped item from #2235's wishlist. `get_top_queries_by_cpu` and `get_top_procedures_by_cpu` (both SKUs) now return the returned rows' summed CPU-seconds, the SQL process's measured CPU-seconds for the same window (avg `cpu_utilization` % x core count x window - both stores already collect every piece), and `attributed_cpu_ratio`. Pre-#2290 the reads explained ~10% of the box and nothing said so - a caller chased the visible tenth assuming it was everything; and the ratio catches impossible claims at a glance (an external comparison died the moment its worker_time sum divided out to 137% of the box's available CPU-seconds - above the process's own measured consumption the note now says to distrust the numbers rather than presenting them). Below half, a note explains where unattributable CPU goes (evictions between snapshots, rows outside the top-N, zero-cost rows, non-query CPU). The degrade rule is explicit: missing CPU series, missing core count, or a series covering under 90% of the window omits the ratio rather than inventing one. One computation in `PerformanceMonitor.Common` (`CpuAttribution`), pinned by the same decision table in both test projects; the denominator read windows on `collection_time` with the same bounds as the rankings, so numerator and denominator share collection gaps.
 - **`get_query_store_health`: the MCP read for the new collector, both SKUs** ([#2319]) - the promised follow-up to the `query_store_health` collector: one browsable tool (beside `get_database_scoped_config`, whose latest-snapshot shape it mirrors) returning per-database actual vs desired state with the mismatch pre-folded into `state_matches_desired`, `readonly_reason` both raw and decoded, storage used vs cap with `pct_of_cap`, cleanup mode/thresholds, and the runtime-stats interval length; also exposed as a `/api/read` web endpoint. The `readonly_reason` bit table now lives ONCE in `PerformanceMonitor.Common` (`QueryStoreReadonlyReason`) and both viewers' grids and both MCP servers decode through it - the labels were miswritten from memory once during #2319 review, so a single source is the fix. While counting the tools for the instructions doc, the census sentence turned out to have silently drifted (it said ninety tools while the server exposed one hundred); it is rewritten with accurate digit counts (101 total / 76 shared with Lite / 25 Darling-only) and a new cross-app pin test parses it against the scanned inventory so it can never drift again.
 - **Per-database Query Store health: a new `query_store_health` collector, both SKUs, both stores** ([#2319]) - `database_config` knows exactly one bit (`is_query_store_on = true`), which cannot answer the questions an investigation like #2312 needed: is Query Store actually WORKING (the classic silent failure is desired_state READ_WRITE with actual_state READ_ONLY after the storage cap hit - `readonly_reason` says why), how close to the cap is it, and what interval grain is it aggregating at. The new collector reads `sys.database_query_store_options` per database - the same proven enumeration idiom as `database_scoped_config` (list accessible ONLINE primaries, then `[db].sys.sp_executesql` per database), deliberately NOT filtered to QS-on databases: the options view answers one row even when Query Store is off, so OFF is recorded as OFF and an absent row can only mean "not collected". Hourly rather than the config family's on-load cadence, because unlike operator-changed knobs these values change BY THEMSELVES and the cap-hit transition is the whole point of collecting them. Every column exists on 2016+, so there are no version gates. Surfaced as a Query Store sub-tab on the Configuration tab in both apps (V76 store table + `v_query_store_health` passthrough keep the two viewers' SQL byte-identical; Lite's table and archive view generate from the catalog); a `get_query_store_health` MCP read follows separately. The issue asked for the fields on `database_config` itself; they land as a sibling enumerating collector instead because `database_config` is a single `sys.databases` scan and these fields need per-database context - bolting an enumeration onto it would change its execution model and failure isolation, and the codebase already has the per-database config member in `database_scoped_config` to mirror.
 - **V75 gives plan CONTENT its own retention horizon, because the fact-coupled one cannot bound a young store** ([#2316]) - the payload dimensions' GC deliberately follows the widest dim-feeding fact retention (90 days) so nothing a live fact references is ever deleted, and that guarantee has a blind spot measured on the dogfood fleet: `query_plan_dim` reached **127 GB - 63% of the store - in its first 22 days**, growing ~6 GB/day of parameter-sniffing recompile churn (344k distinct plan XMLs per day from 5,327 plan SHAPES - 65 variants per shape, the worst single shape producing 57,402 in one day), with the coupled GC unable to delete a single row until the horizon crossed the dimension's birth date - roughly a month AFTER the projected disk-full. Orphan pruning already existed and was healthy; compression was already spent (every row app-gzipped); the inflow is legitimate distinct content by the #1767 design, so the remaining lever is lifetime. The new `config_service.plan_content_retention_days` knob (default 21, clamps [7,365], 0 = disabled = the old behavior byte-for-byte) sets how long a stored plan XML outlives its last sighting: the dimension cutoff becomes the NEWER of the fact-coupled cutoff and `now - (knob + 1)` - the same one-day margin as the measured floor, for the same hourly `last_seen` refresh guard. Facts keep their full retention (metrics, hashes and text stay analyzable); a plan older than the window renders as the missing plan every reader already handles. The horizon governs the PLAN dimension only - query text keeps the fact-coupled cutoff (it is ~40 MB against the plan dim's 127 GB, and shortening it would break "text stays analyzable" for nothing) - and the Query Store plan map's prune learns the knob too, keeping the dimension-outlives-the-map ordering under every knob value so a live map row can never resolve to deleted content. A knob wider than the fact horizon is deliberately a no-op - it must not become a way to keep XML nothing can reference. Deliberately NOT done: shape-keyed latest-wins storage would shrink this 65x but breaks the historical-fact-to-exact-XML contract #1767 preserves on purpose - parameter-variant plans are the product's diagnostic bread and butter.
@@ -2797,6 +2798,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 [#2246]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/2246
 [#2319]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/2319
 [#2317]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/2317
+[#2320]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/2320
 [#2316]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/2316
 [#2324]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/2324
 [#2300]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/2300

Darling/Darling.Tests/CpuAttributionTests.cs

@@ -0,0 +1,156 @@
+/*
+ * Copyright (c) 2026 Erik Darling, Darling Data LLC
+ *
+ * This file is part of the SQL Server Performance Monitor.
+ *
+ * Licensed under the MIT License. See LICENSE file in the project root for full license information.
+ */
+
+using System;
+using PerformanceMonitor.Common;
+using Xunit;
+
+namespace Darling.Tests;
+
+/// <summary>
+/// Decision-table pins for the shared <see cref="CpuAttribution"/> (#2320) — the attributed-CPU
+/// disclosure both SKUs' get_top_queries_by_cpu / get_top_procedures_by_cpu serve. The contract under
+/// pin: the ratio is measured-or-omitted (never invented — missing samples, missing core count, or
+/// thin coverage all degrade to null + a reason), the low note fires under half, and above the
+/// process's own measured CPU the note calls the number impossible rather than presenting it —
+/// the 137%-of-the-box claim is the whole reason the marker exists. This SAME table is pinned
+/// identically in Lite.Tests so the two SKUs cannot drift.
+/// </summary>
+public sealed class CpuAttributionTests
+{
+    private static readonly DateTime Start = new(2026, 8, 18, 0, 0, 0, DateTimeKind.Utc);
+    private static readonly DateTime End = Start.AddHours(1);
+
+    /// <summary>Full coverage, healthy ratio: 25% of 8 cores over an hour = 7,200 CPU-seconds;
+    /// 5,000 ranked seconds is 0.694 — present, rounded to 3, no note.</summary>
+    [Fact]
+    public void HealthyRatio_NoNote()
+    {
+        var result = CpuAttribution.Compute(
+            rankedCpuSeconds: 5000, Start, End,
+            sampleCount: 60, firstSampleUtc: Start, lastSampleUtc: End, avgSqlCpuPercent: 25, cpuCount: 8);
+
+        Assert.Equal(5000, result.RankedCpuSeconds);
+        Assert.Equal(7200, result.SqlCpuSecondsInWindow);
+        Assert.Equal(0.694, result.AttributedCpuRatio);
+        Assert.Null(result.Note);
+    }
+
+    /// <summary>The pre-#2290 shape this feature exists for: the ranking explains ~10% of the box,
+    /// and now something says so instead of letting the caller chase the visible tenth.</summary>
+    [Fact]
+    public void LowRatio_SaysNotTheWholeStory()
+    {
+        var result = CpuAttribution.Compute(720, Start, End, 60, Start, End, 25, 8);
+
+        Assert.Equal(0.1, result.AttributedCpuRatio);
+        Assert.NotNull(result.Note);
+        Assert.Contains("10%", result.Note, StringComparison.Ordinal);
+        Assert.Contains("not the whole story", result.Note, StringComparison.Ordinal);
+    }
+
+    /// <summary>The 137% case — worker_time summing to more CPU than the process consumed is an
+    /// impossible claim, and the note must say to distrust the numbers, not decorate them.</summary>
+    [Fact]
+    public void OverAttribution_IsFlaggedImpossible()
+    {
+        var result = CpuAttribution.Compute(9864, Start, End, 60, Start, End, 25, 8);
+
+        Assert.Equal(1.37, result.AttributedCpuRatio);
+        Assert.NotNull(result.Note);
+        Assert.Contains("137%", result.Note, StringComparison.Ordinal);
+        Assert.Contains("impossible-claim", result.Note, StringComparison.Ordinal);
+    }
+
+    /// <summary>Just above 1.0 is sampling noise between two independent series, not a lie —
+    /// the impossible flag waits for the slack threshold.</summary>
+    [Fact]
+    public void SlightlyOverOne_CarriesNoNote()
+    {
+        var result = CpuAttribution.Compute(7500, Start, End, 60, Start, End, 25, 8);
+
+        Assert.Equal(1.042, result.AttributedCpuRatio);
+        Assert.Null(result.Note);
+    }
+
+    [Fact]
+    public void NoSamples_OmitsRatio_AndSaysWhy()
+    {
+        var result = CpuAttribution.Compute(5000, Start, End, 0, null, null, null, 8);
+
+        Assert.Equal(5000, result.RankedCpuSeconds);
+        Assert.Null(result.SqlCpuSecondsInWindow);
+        Assert.Null(result.AttributedCpuRatio);
+        Assert.Contains("no cpu_utilization samples", result.Note, StringComparison.Ordinal);
+    }
+
+    [Fact]
+    public void NoCoreCount_OmitsRatio_AndSaysWhy()
+    {
+        var result = CpuAttribution.Compute(5000, Start, End, 60, Start, End, 25, cpuCount: 0);
+
+        Assert.Null(result.AttributedCpuRatio);
+        Assert.Contains("core count unavailable", result.Note, StringComparison.Ordinal);
+    }
+
+    /// <summary>#2320's explicit degrade rule: a server whose CPU series starts mid-window (added,
+    /// or monitoring resumed) would deflate the denominator and inflate the ratio — omit instead.</summary>
+    [Fact]
+    public void PartialCoverage_OmitsRatio_WithThePercentage()
+    {
+        var result = CpuAttribution.Compute(5000, Start, End, 30, Start.AddMinutes(30), End, 25, 8);
+
+        Assert.Null(result.AttributedCpuRatio);
+        Assert.NotNull(result.Note);
+        Assert.Contains("50%", result.Note, StringComparison.Ordinal);
+        Assert.Contains("partial denominator", result.Note, StringComparison.Ordinal);
+    }
+
+    /// <summary>Samples straddling the window edges clamp to full coverage — a series wider than the
+    /// window is the NORMAL case (the store holds more history than any one read).</summary>
+    [Fact]
+    public void SamplesBeyondTheWindow_ClampToFullCoverage()
+    {
+        var result = CpuAttribution.Compute(
+            5000, Start, End, 120, Start.AddHours(-1), End.AddHours(1), 25, 8);
+
+        Assert.Equal(0.694, result.AttributedCpuRatio);
+    }
+
+    /// <summary>An idle box measures zero CPU-seconds; a ratio against zero is undefined, and the
+    /// measured zero is still reported so the caller sees WHY.</summary>
+    [Fact]
+    public void ZeroMeasuredCpu_OmitsRatio_ReportsTheZero()
+    {
+        var result = CpuAttribution.Compute(5000, Start, End, 60, Start, End, avgSqlCpuPercent: 0, cpuCount: 8);
+
+        Assert.Equal(0, result.SqlCpuSecondsInWindow);
+        Assert.Null(result.AttributedCpuRatio);
+        Assert.Contains("zero", result.Note, StringComparison.Ordinal);
+    }
+
+    [Fact]
+    public void EmptyWindow_OmitsRatio()
+    {
+        var result = CpuAttribution.Compute(5000, Start, Start, 60, Start, End, 25, 8);
+
+        Assert.Null(result.AttributedCpuRatio);
+        Assert.Contains("window is empty", result.Note, StringComparison.Ordinal);
+    }
+
+    /// <summary>The numerator is rounded for emission but the ratio divides the RAW value — rounding
+    /// before dividing would move the third decimal on big windows.</summary>
+    [Fact]
+    public void RankedSecondsRoundToOneDecimal_RatioToThree()
+    {
+        var result = CpuAttribution.Compute(1234.5678, Start, End, 60, Start, End, 25, 8);
+
+        Assert.Equal(1234.6, result.RankedCpuSeconds);
+        Assert.Equal(0.171, result.AttributedCpuRatio);
+    }
+}

Darling/Darling.Tests/DarlingMcpDataToolsTests.cs

@@ -191,6 +191,21 @@ public void CpuSql_ReadsBaseTable_DeSkewsSampleTime_WindowsOnCollectionTime()
         Assert.Contains("collection_time >= $2", sql, StringComparison.Ordinal);        /* window on the reliable clock */
     }
 
+    /// <summary>#2320: the attribution denominator windows on collection_time — the SAME bounds the
+    /// rankings use, so numerator and denominator share collection gaps — and aggregates rather than
+    /// pulling sample rows.</summary>
+    [Fact]
+    public void CpuWindowAggregateSql_WindowsOnCollectionTime_BothEdges()
+    {
+        var sql = DarlingDataReader.CpuWindowAggregateSql;
+        Assert.Contains("FROM cpu_utilization_stats", sql, StringComparison.Ordinal);
+        Assert.Contains("AVG(sqlserver_cpu_utilization)", sql, StringComparison.Ordinal);
+        Assert.Contains("MIN(collection_time)", sql, StringComparison.Ordinal);
+        Assert.Contains("MAX(collection_time)", sql, StringComparison.Ordinal);
+        Assert.Contains("collection_time >= $2", sql, StringComparison.Ordinal);
+        Assert.Contains("collection_time <= $3", sql, StringComparison.Ordinal);
+    }
+
     [Fact]
     public void WaitStatsSql_AggregatesDeltas_HeaviestFirst()
     {
@@ -377,6 +392,7 @@ exactly like the viewer's UTC-offset read. */
     [InlineData(nameof(DarlingDataReader.ServerListSql))]
     [InlineData(nameof(DarlingDataReader.CollectionHealthSql))]
     [InlineData(nameof(DarlingDataReader.LatestServerPropertiesSql))]
+    [InlineData(nameof(DarlingDataReader.CpuWindowAggregateSql))]
     public void Reads_ArePostgresDialect_NoTsqlIsms(string sqlName)
     {
         var sql = SqlByName(sqlName);
@@ -405,6 +421,7 @@ public void Reads_ArePostgresDialect_NoTsqlIsms(string sqlName)
         nameof(DarlingDataReader.QueryStoreTopSql) => DarlingDataReader.QueryStoreTopSql,
         nameof(DarlingDataReader.ServerListSql) => DarlingDataReader.ServerListSql,
         nameof(DarlingDataReader.CollectionHealthSql) => DarlingDataReader.CollectionHealthSql,
+        nameof(DarlingDataReader.CpuWindowAggregateSql) => DarlingDataReader.CpuWindowAggregateSql,
         _ => DarlingDataReader.LatestServerPropertiesSql,
     };
 

Darling/PerformanceMonitor.Darling.Service/Mcp/DarlingDataReader.cs

@@ -174,6 +174,46 @@ public static async Task<List<CpuSample>> GetCpuUtilizationAsync(
         return samples;
     }
 
+    public sealed record CpuWindowAggregate(int SampleCount, DateTime? FirstSample, DateTime? LastSample, double? AvgSqlCpuPercent);
+
+    /// <summary>
+    /// The attributed-CPU denominator's pieces (#2320): sample count, coverage bounds, and average SQL
+    /// CPU% over the window. Windowed on collection_time — the SAME bounds the top-queries/procedures
+    /// rankings use — so numerator and denominator share collection gaps; sample_time skew is irrelevant
+    /// to an average. $1 server_id, $2/$3 window (naive UTC).
+    /// </summary>
+    public const string CpuWindowAggregateSql = """
+        SELECT
+            COUNT(*),
+            MIN(collection_time),
+            MAX(collection_time),
+            AVG(sqlserver_cpu_utilization)::double precision
+        FROM cpu_utilization_stats
+        WHERE server_id = $1
+        AND   collection_time >= $2
+        AND   collection_time <= $3
+        """;
+
+    public static async Task<CpuWindowAggregate> GetCpuWindowAggregateAsync(
+        NpgsqlDataSource postgres, int serverId, DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken = default)
+    {
+        await using var command = postgres.CreateCommand(CpuWindowAggregateSql);
+        AddInt(command, serverId);
+        AddTimestamp(command, startUtc);
+        AddTimestamp(command, endUtc);
+        await using var reader = await command.ExecuteReaderAsync(cancellationToken);
+        if (!await reader.ReadAsync(cancellationToken))
+        {
+            return new CpuWindowAggregate(0, null, null, null);
+        }
+
+        return new CpuWindowAggregate(
+            reader.IsDBNull(0) ? 0 : Convert.ToInt32(reader.GetValue(0), System.Globalization.CultureInfo.InvariantCulture),
+            reader.IsDBNull(1) ? null : reader.GetDateTime(1),
+            reader.IsDBNull(2) ? null : reader.GetDateTime(2),
+            reader.IsDBNull(3) ? null : reader.GetDouble(3));
+    }
+
     /* ─────────────────────────── wait stats ─────────────────────────── */
 
     /// <summary>