Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion specs/done_022-default-telemetry-dashboard/OPEN-DECISIONS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Open Decisions

**Status**: 0 open; 5 ruled (OD-2, OD-3 and OD-4 on 2026-10-06, OD-1 and OD-5 on 2026-10-07).
**Status**: 0 open; 6 ruled (OD-2, OD-3 and OD-4 on 2026-10-06, OD-1 and OD-5 on 2026-10-07, OD-6 on 2026-10-08).

Per CLAUDE.md rule 10, an open value MUST NOT be committed as a settled contract in code, chart defaults or docs until it is ruled here.

Expand Down Expand Up @@ -102,3 +102,11 @@ Per CLAUDE.md rule 10, an open value MUST NOT be committed as a settled contract
| Challenge lifetime | 15 minutes | Longer than a Raspberry Pi 4 needs at the 26-bit cap. |
| Challenge requests per source | 10 per minute, burst 5 | One source can't raise the difficulty for everyone. |
| Used-challenge memory | 1,000,000 entries | Filling it needs that many solved challenges within 15 minutes. |

### OD-6: Dashboard as-of day

**Status**: RULED (2026-10-08, user)

**Question**: The receiver dashboard's ranges ended on the latest complete UTC day (yesterday), so a provider saw "No reports yet" for up to a day after its first accepted report, and today's reports never showed until the day closed.

**Ruling**: the dashboard shows all data. Every range (7, 30, 90 or 365 days) ends on today, the current UTC day, which is still in progress and labelled so. The empty state appears only when the range holds no report at all. Today's lapsed-installs value stays `0` until the lifecycle job finalises the day. The public `GET /v1/summary` is unchanged and keeps the latest complete day.
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ An E2E check (R17) compares the unauthenticated response against the login page
}
```

- `asOf` is today, the current UTC day, which is still in progress (OD-6). Every range ends on it, its figures change until the day closes, and its lapsed-installs value stays `0` until the lifecycle job finalises the day. `/v1/summary` keeps the latest complete day.
- When the range has no data: `empty: true`, with `basic` and `extended` set to `null`. The HTML renders the empty state (FR-027).
- When there is basic data but no extended reports: `extended: null`, and the HTML explains that no reports in the range included extended data.

Expand Down
6 changes: 3 additions & 3 deletions specs/done_022-default-telemetry-dashboard/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,19 +224,19 @@ Every `value` is a `telemetryschema` enumeration member or `other`, so the row c

### Derived views (computed at read time, [contracts/receiver-http.md](contracts/receiver-http.md))

The range R is 7, 30, 90 or 365 days and ends on the latest complete day.
The range R is 7, 30, 90 or 365 days and ends on today, the current UTC day, which is still in progress (OD-6). Today's figures change until the day closes, and its lapsed-installs value stays 0 until the lifecycle job finalises the day.

| View | Computation |
|---|---|
| Reports per day | `daily_basic.reports` series over R |
| Version adoption (reports) | `daily_version` summed over R. Top 10 by count, then `Other` (the rest) and `Invalid`. |
| Fleet distribution and median | `daily_fleet` summed over R. Bands `0,1,2,5,10,25,50,100,250,250+` are applied at read. The exact median comes from the cumulative counts. |
| Latest-day totals | `daily_basic.servers_sum`, `templates_sum` for the latest complete day |
| Latest-day totals | `daily_basic.servers_sum`, `templates_sum` for today (the as-of day, in progress) |
| Unique active installs | `count(activity where last_seen ≥ asOf − {0, 6, 29})` |
| New and lapsed installs | `daily_ext.new_installs` and `lapsed_installs` series over R |
| Version adoption (installs) | `activity.last_version` for records with `last_seen` inside `min(R, ACTIVITY_EXPIRY_DAYS)`. The window used is shown. |
| Environment, games, feature shares | Σ `installs` ÷ Σ `daily_ext.ext_reports` over R (install-day weighted, research R4) |
| Extended coverage (FR-026) | Σ `ext_reports` ÷ Σ `reports` over R |
| Public summary (FR-029) | `reports` for asOf; Σ `reports` over the 30 days ending asOf; `meta.reports_total`; `count(activity where last_seen ≥ asOf − 29)`; `asOf` |

`asOf` is the latest complete UTC day, which is yesterday. When no `daily_basic` row exists in R, every view returns its empty state instead of zeros (FR-027).
For the dashboard, `asOf` is today (the current UTC day, in progress); the public summary keeps the latest complete day. When no `daily_basic` row exists in R, every view returns its empty state instead of zeros (FR-027).
6 changes: 3 additions & 3 deletions specs/done_022-default-telemetry-dashboard/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ These answer questions that basic counts can't, such as "how many installs are s
**Acceptance Scenarios**:

1. **Given** extended reports over a range, **When** an authorized viewer looks at installs, **Then** the dashboard shows:
- unique active installs for the latest day and the trailing 7 and 30 days
- unique active installs for today (in progress) and the trailing 7 and 30 days
- new installs (first seen) per period
- lapsed installs (not seen for 30 days) per period
2. **Given** extended reports, **When** an authorized viewer looks at environment, **Then** it shows each install's share by Kubernetes minor version, distribution, CPU architecture, and node-count band.
Expand Down Expand Up @@ -330,10 +330,10 @@ Someone learns an install's ID, for example from a screenshot of Admin Settings,
- reports per day as a trend
- version adoption, with the top 10 versions plus "Other" and "Invalid"
- fleet-size distributions and medians for servers and templates
- total servers and templates on the latest day
- total servers and templates today (in progress)
- a "data as of" timestamp
- **FR-025**: The dashboard MUST show these **extended** views:
- unique active installs (latest day, trailing 7 and 30 days)
- unique active installs (today in progress, trailing 7 and 30 days)
- new and lapsed installs per period
- version adoption by install
- environment breakdowns
Expand Down
4 changes: 2 additions & 2 deletions specs/done_022-default-telemetry-dashboard/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ description: "Task list for feature 022: default telemetry destination, extended
- fleet bands and the exact median
- `latestDay` totals
- `empty`
- `asOf` = yesterday (UTC)
- `asOf` = yesterday (UTC). **Superseded 2026-10-08 (OD-6):** the dashboard ends on today (UTC, in progress); `/v1/summary` keeps yesterday.

Invalid ranges fall back to 30.
- [X] T066 [US4] Load the `dataviz` skill, then design the receiver pages in `telemetry-receiver/telemetry-dashboard.pen` (seeded with a copy of the HeroUI design, spec Q10) through Pencil MCP: the login page (also used for the refusal, with "Invalid credentials"), the overview's basic section with the range selector (7, 30, 90, 365), the empty state, and the "approximate, self-reported" labelling (FR-026). Ask the user to open that file, then save it, run `design-export` into `telemetry-receiver/design-export/`, and commit.
Expand Down Expand Up @@ -397,7 +397,7 @@ description: "Task list for feature 022: default telemetry destination, extended
- `GET /metrics` returns `404` on the public listener, `401` on the dashboard listener without the Bearer token, and `200` with it
- [X] T071 [US4] Add these subtests to `test/e2e/telemetry_e2e_test.go`:
- `dashboard_refuses_unauthenticated`, where `/` returns 303 to a login page that contains no digits from the data and `/api/v1/views` returns 401
- `dashboard_shows_reports`, which signs in with the Secret's token through `Env.PortForward` on 8081 (setting the `Cookie` header by hand, since the session cookie is `Secure` and the port-forward is plain HTTP). Views end at yesterday (UTC), so same-day reports from earlier subtests are not in them: it asserts that `/api/v1/views` returns `200` with `asOf` = yesterday and a well-formed body, and that `/metrics` (Bearer) counts the earlier reports. The exact figures are covered by `views_test.go`.
- `dashboard_shows_reports`, which signs in with the Secret's token through `Env.PortForward` on 8081 (setting the `Cookie` header by hand, since the session cookie is `Secure` and the port-forward is plain HTTP). Views end at yesterday (UTC), so same-day reports from earlier subtests are not in them: it asserts that `/api/v1/views` returns `200` with `asOf` = yesterday (**superseded 2026-10-08, OD-6:** views end on today and the check asserts `asOf` = today) and a well-formed body, and that `/metrics` (Bearer) counts the earlier reports. The exact figures are covered by `views_test.go`.

**Checkpoint**: the provider keeps history and shows basic views privately.

Expand Down
3 changes: 2 additions & 1 deletion telemetry-receiver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@ public provider turns it on. See
### Dashboard

Server-rendered HTML with inline SVG charts, no JavaScript, for the last 7,
30, 90 or 365 days (`?range=`). It needs the dashboard token: a browser
30, 90 or 365 days (`?range=`), ending on today (UTC), which is still in
progress. It needs the dashboard token: a browser
logs in at `/login` and gets a 12-hour session cookie; scripts can send
`Authorization: Bearer <token>` to `/api/v1/views`. `/metrics` accepts only
the Bearer token. An unauthenticated request never sees a figure: the HTML
Expand Down
2 changes: 1 addition & 1 deletion telemetry-receiver/charts_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ func TestDayLabelEvery(t *testing.T) {
}

// seedExtendedDash stores one basic report and two extended reports on the
// latest complete day (2026-10-06) of a dashboard test server.
// day before today (2026-10-05) of a dashboard test server, inside every range.
func seedExtendedDash(t *testing.T, s *server) {
t.Helper()
s.now = func() time.Time { return dashNow.Add(-24 * time.Hour) }
Expand Down
6 changes: 3 additions & 3 deletions telemetry-receiver/dashboard_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ const (
dashOrigin = "http://example.com" // httptest.NewRequest uses Host example.com
)

var dashNow = time.Date(2026, 10, 7, 12, 0, 0, 0, time.UTC)
var dashNow = time.Date(2026, 10, 6, 12, 0, 0, 0, time.UTC)

// newDash builds a receiver with a fixed clock and returns it with its
// dashboard handler.
Expand All @@ -31,8 +31,8 @@ func newDash(t *testing.T, cfg config) (*server, http.Handler) {
return s, s.dashboardRoutes()
}

// seedDash stores reports on the latest complete day (2026-10-06) and the
// day before it.
// seedDash stores reports on today (2026-10-06, the as-of day) and the day
// before it.
func seedDash(t *testing.T, s *server) {
t.Helper()
for _, r := range []struct {
Expand Down
10 changes: 10 additions & 0 deletions telemetry-receiver/design-export/MANIFEST.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,13 @@ This is a snapshot of `telemetry-receiver/telemetry-dashboard.pen` for review in
- "Lapsed installs per day" → `i0xK1E`
- "No reports in this range included extended data" → `jy9Rn`
- "DASHBOARD_TOKEN value" → `HBYBE`, `a7wUpP`

## Incremental export (2026-10-08): dashboard ends on today (OD-6)

| Node | Change |
|---|---|
| `i0xK1E` | "Data through 2026-10-06 (UTC, today in progress)"; tiles "Reports, today" and "Active installs, today" with "2026-10-06 (UTC, in progress)"; "Median servers per install" sub "Last 30 days"; captions "…, ending today (in progress)", "Reports in range by reported version…", "Reports by game-server count. Median 3." and the lapsed note "Today's bar is final after the UTC day closes." |
| `Z079vK` | Empty-state text: "Figures appear as soon as this receiver accepts a report; today's figures are in progress and update as reports arrive. …" |
| `y49Ie` | `sDDfo` Stat tile sample: "Reports, today", "2026-10-06 (UTC, in progress)" |

Method: 15 `Update` calls in one `execute`. The JSON files were produced by applying the same edits to the previous export, then checked equal to Pencil (`JSON.stringify(Get(id, {depth: 40, includePathGeometry: true}))` length and FNV-1a hash per frame: `i0xK1E` 69071, `Z079vK` 4684, `y49Ie` 1854). PNGs: `Export(ids, "png", …, {scale: 2})`, same sizes as before. Content check: "today in progress" → `i0xK1E`; "accepts a report" → `Z079vK`.
2 changes: 1 addition & 1 deletion telemetry-receiver/design-export/json/Z079vK.json
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,7 @@
"fill": "$foreground/muted",
"textGrowth": "fixed-width",
"width": 520,
"content": "Figures appear once this receiver has accepted reports for a full UTC day. Installs send to POST /ingest on the public port; nothing is shown until there is data, so no chart is drawn from zeros.",
"content": "Figures appear as soon as this receiver accepts a report; today's figures are in progress and update as reports arrive. Installs send to POST /ingest on the public port; nothing is shown until there is data, so no chart is drawn from zeros.",
"lineHeight": 1.5,
"textAlign": "center",
"fontFamily": "$typography/font-sans",
Expand Down
24 changes: 12 additions & 12 deletions telemetry-receiver/design-export/json/i0xK1E.json
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@
"id": "W416z",
"name": "As of",
"fill": "$foreground/muted",
"content": "Data through 2026-10-06 (UTC)",
"content": "Data through 2026-10-06 (UTC, today in progress)",
"fontFamily": "$typography/font-sans",
"fontSize": 13,
"fontWeight": "normal"
Expand Down Expand Up @@ -289,17 +289,17 @@
"id": "WySx8",
"type": "ref",
"ref": "sDDfo",
"name": "Reports, latest day",
"name": "Reports, today",
"width": "fill_container",
"descendants": {
"yia1E": {
"content": "Reports, latest day"
"content": "Reports, today"
},
"tZRdh": {
"content": "1,284"
},
"oPQEN": {
"content": "2026-10-06 (UTC)"
"content": "2026-10-06 (UTC, in progress)"
}
}
},
Expand Down Expand Up @@ -353,7 +353,7 @@
"content": "3"
},
"oPQEN": {
"content": "Latest day; mean 4.6"
"content": "Last 30 days"
}
}
}
Expand Down Expand Up @@ -402,7 +402,7 @@
"id": "rZQiH",
"name": "Sub",
"fill": "$foreground/muted",
"content": "Accepted reports per UTC day, last 30 days",
"content": "Accepted reports per UTC day, last 30 days, ending today (in progress)",
"fontFamily": "$typography/font-sans",
"fontSize": 13,
"fontWeight": "normal"
Expand Down Expand Up @@ -1665,7 +1665,7 @@
"fill": "$foreground/muted",
"textGrowth": "fixed-width",
"width": 600,
"content": "Reports on the latest day by reported version. Top 10, then Other and Invalid.",
"content": "Reports in range by reported version. Top 10, then Other and Invalid.",
"fontFamily": "$typography/font-sans",
"fontSize": 13,
"fontWeight": "normal"
Expand Down Expand Up @@ -1893,7 +1893,7 @@
"fill": "$foreground/muted",
"textGrowth": "fixed-width",
"width": 600,
"content": "Installs on the latest day by game-server count. Median 3, mean 4.6.",
"content": "Reports by game-server count. Median 3.",
"fontFamily": "$typography/font-sans",
"fontSize": 13,
"fontWeight": "normal"
Expand Down Expand Up @@ -2592,17 +2592,17 @@
"id": "aRoss",
"type": "ref",
"ref": "sDDfo",
"name": "Active installs, latest day",
"name": "Active installs, today",
"width": "fill_container",
"descendants": {
"yia1E": {
"content": "Active installs, latest day"
"content": "Active installs, today"
},
"tZRdh": {
"content": "912"
},
"oPQEN": {
"content": "2026-10-06 (UTC)"
"content": "2026-10-06 (UTC, in progress)"
}
}
},
Expand Down Expand Up @@ -3696,7 +3696,7 @@
"id": "NMwMD",
"name": "Sub",
"fill": "$foreground/muted",
"content": "Installs whose last report was 30 days earlier",
"content": "Installs whose last report was 30 days earlier. Today's bar is final after the UTC day closes.",
"fontFamily": "$typography/font-sans",
"fontSize": 13,
"fontWeight": "normal"
Expand Down
4 changes: 2 additions & 2 deletions telemetry-receiver/design-export/json/y49Ie.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"id": "yia1E",
"name": "Label",
"fill": "$foreground/secondary",
"content": "Reports, latest day",
"content": "Reports, today",
"fontFamily": "$typography/font-sans",
"fontSize": 13,
"fontWeight": "normal"
Expand All @@ -61,7 +61,7 @@
"id": "oPQEN",
"name": "Note",
"fill": "$foreground/muted",
"content": "2026-10-06 (UTC)",
"content": "2026-10-06 (UTC, in progress)",
"fontFamily": "$typography/font-sans",
"fontSize": 12,
"fontWeight": "normal"
Expand Down
Binary file modified telemetry-receiver/design-export/screenshots/Z079vK.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified telemetry-receiver/design-export/screenshots/i0xK1E.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified telemetry-receiver/design-export/screenshots/y49Ie.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions telemetry-receiver/population_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ func TestPopulationProducesExactNewLapsedAndActiveCounts(t *testing.T) {

// The views at the end of day 59: active counts and the 30-day unique
// count (SC-008 allows 1%; the distinct count is exact).
*clock = popDay(popDays).Add(12 * time.Hour)
*clock = popDay(popDays - 1).Add(12 * time.Hour) // today (asOf) is day 59
v, err := BuildViews(ctx, st, 30, *clock)
if err != nil {
t.Fatalf("BuildViews: %v", err)
Expand Down Expand Up @@ -341,7 +341,7 @@ func TestBuildViewsExtendedSharesGamesAndFeatures(t *testing.T) {
t.Fatalf("status = %d (%s)", w.Code, w.Body)
}
}
*clock = extStart.Add(24 * time.Hour) // asOf is 2026-10-07
*clock = extStart.Add(24 * time.Hour) // asOf is 2026-10-08 (today); the reports are on 2026-10-07
v, err := BuildViews(ctx, s.store, 7, *clock)
if err != nil {
t.Fatal(err)
Expand Down
2 changes: 1 addition & 1 deletion telemetry-receiver/specs.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Optional collector for the usage telemetry that Gameplane installs report daily.
- **Proof-of-work (optional)**: with `INGEST_POW=true`, `/ingest` requires a solved challenge from `/v1/challenge` before the body is read; difficulty adapts to the challenge rate
- **Aggregation and storage**: daily aggregates and expiring per-install activity records in SQLite (`$DATA_DIR/telemetry.db`); an install ID is stored only as `HMAC-SHA256(pepper, installID)`
- **Per-source limits**: accepted reports per source per UTC day (`INGEST_SOURCE_DAILY_LIMIT`), login attempts, summary reads and challenge requests, all held in memory; source addresses come from the TCP peer, or from `X-Forwarded-For` only when the peer is inside `TRUSTED_PROXY_CIDRS`
- **Dashboard**: server-rendered HTML with inline SVG charts and no JavaScript, behind one operator token; the same view model is served as JSON at `/api/v1/views`
- **Dashboard**: server-rendered HTML with inline SVG charts and no JavaScript, behind one operator token; the same view model is served as JSON at `/api/v1/views`. Every range ends on today (UTC), which is still in progress
- **Public summary (optional)**: `GET /v1/summary` returns five counts when `PUBLIC_SUMMARY=true`
- **Version sanitization**: invalid syntax is bucketed as `"invalid"`. Retain at most 128 distinct strings matching `^[A-Za-z0-9][A-Za-z0-9._+-]{0,31}$` per process for the Prometheus label; additional versions share `"other"`. The two bucket names are reserved, for at most 130 series. Retained labels are never evicted; restart resets the budget with the counters.
- **Metrics**: Prometheus series on the dashboard listener only, behind the token
Expand Down
Loading
Loading