Skip to content

Report every app and server error to Exceptionless and fix the top crashes - #125

Merged
ejsmith merged 3 commits into
mainfrom
exceptionless-error-telemetry
Oct 29, 2026
Merged

ejsmith merged 3 commits into
mainfrom
exceptionless-error-telemetry

Conversation

@ejsmith

@ejsmith ejsmith commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Why

Production errors were reaching Exceptionless only partly. Many were caught and shown to players, or logged and dropped, and the server never reported its own failures. Unrelated failures also piled into a few big stacks. This PR closes that loop so every real error a player hits arrives with enough context to fix it, without flooding the project with noise. It also fixes the top crashes found in the data.

Crash and bug fixes

  • "Something went wrong" after returning to a game (still hitting build 49). Realtime channels were reused by name, so returning to a game got back an already-subscribed channel and crashed. Every subscription now gets a unique topic.
  • Cannot read property 'scorecard' of undefined. Opening your own invite before a friend joined mounted a one-player board. That case now shows "Waiting for Friend" instead.
  • iOS keychain errors on background wake. Sessions are now stored readable after first unlock. Existing items are migrated once, and the old session is restored if the migration write fails.
  • Notifications broken for the next account on a device. New claim_push_token / claim_web_push_subscription functions let the signed-in account take over a stale device registration.
  • Moves stuck after a resume. Moves that fail during background recovery stay pending and retry at 5s, 15s and 45s. They're reported only if every retry fails, and concurrent recovery triggers share one pass.

Telemetry

  • Coverage: every on-screen error, swallowed error, error-carrying console.warn/console.error, realtime channel error and render crash is reported. Render crashes are marked critical.
  • Policy (src/monitoring/errorPolicy.ts, unit tested):
    • Expected player mistakes and game rule rejections are skipped.
    • Network failures are sent at most once per operation every 10 minutes.
    • Library and coded errors group by operation, code and normalized message. App errors keep stack-trace grouping.
  • Server: game-action reports unexpected faults even when it responds 4xx, with GameId and RequestId. Game rules reject with a plain Error, and InternalServerError marks server invariants. The reporter never throws and times out its submissions. Unregistered Expo push tokens are removed, and other ticket failures are reported.
  • Privacy: messages, stacks and details are redacted on both sides (shared/redaction.ts): emails, push tokens and endpoints, JWTs and database row values. Credential fields are excluded on the client.
  • Context: game screen events carry GameId. Native versions include the over-the-air update ID, and web versions include the commit (EXPO_PUBLIC_RELEASE_ID, set by the web build).
  • Tooling: npm run errors lists stacks and events for an AI agent or developer. The rules are documented in docs/release-quality.md.

Deploy order (required)

The app no longer reports 4xx game-action errors itself; it relies on the server to report real faults. Before the web deploy or any over-the-air update that carries this client:

  1. Apply migration 20261023130000_claim_push_destinations.sql. The new client calls these functions, so push registration fails without it.
  2. Set the Supabase secret EXCEPTIONLESS_API_KEY.
  3. Deploy game-action and delete-account.

Validation

  • npm run typecheck, npm run typecheck:edge, npm run lint
  • npm run test: 228 pass, including the new errorPolicy tests
  • npm run test:edge: 49 pass, including the new server classification and reporter tests
  • Web e2e against local Supabase: all behavioral tests pass, including the new invite and push-takeover tests. remote-turn-reveal was updated for the per-subscription topic names. Screenshot baselines are Linux-rendered, so this PR's CI is what verifies them.

Known follow-ups

  • Any signed-in user who knows a device's push token or endpoint can claim it. Those are long random strings other users can't read, so the risk is low.
  • The Waiting for Friend screen has no share or cancel button yet.
  • Adopting the unreleased Exceptionless client (breadcrumbs, error causes, source-mappable native frames) and native source map uploads.

🤖 Generated with Claude Code

ejsmith and others added 2 commits October 23, 2026 21:00
… top crashes

Telemetry
- Report on-screen, swallowed, and console-logged errors with Supabase
  details, network tags, de-duplication, and stable grouping.
- Report edge function 5xx failures from game-action and delete-account.
- Add `npm run errors` to read Exceptionless stacks and events.

Fixes
- Give every realtime subscription a unique channel so returning to a game
  no longer reuses a subscribed channel and crashes.
- Show Waiting for Friend instead of a one-player board for an open invite.
- Store sessions readable after first unlock so locked background wakes
  stop failing with keychain errors.
- Let the signed-in account claim a device's push destination from a stale
  owner.
- Retry background move recovery with backoff instead of reporting
  transient network failures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From an independent review of the Exceptionless work.

Server
- Report unexpected game-action faults even when they respond 4xx, with
  GameId and RequestId; game rules still reject with a plain Error.
- Never throw from the reporter, time out submissions, keep wrapped causes,
  and redact messages, stacks and details.
- Remove unregistered Expo push tokens and report other ticket failures
  without retrying, and keep Expo response bodies out of error messages.

App
- One unit-tested policy: skip expected player mistakes and game rule
  rejections, send network failures at most once per operation every 10
  minutes, and stack library and coded errors by operation and code.
- Redact emails, push tokens and endpoints, JWTs and row values, and exclude
  credential fields.
- Retry pending moves that fail during background recovery with backoff and
  report only when retries run out; share one recovery pass at a time.
- Restore the saved session if the keychain migration write fails.
- Attach GameId to game screen events, mark render crashes critical, report
  realtime channel errors, stop double-reporting sign-in failures, and cover
  two silent catches.
- Identify releases by build plus over-the-air update, or web commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T02:42:14.506523Z f2a2a13 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ejsmith
ejsmith merged commit 381959e into main Oct 29, 2026
12 checks passed
@ejsmith
ejsmith deleted the exceptionless-error-telemetry branch October 29, 2026 03:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant