Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# PR gate: the deployable bundle must type-check and build. Keeps main deployable.
name: ci

on:
pull_request:
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: app/package-lock.json
- name: Install app deps
run: npm --prefix app ci
- name: Build (tsc -b && vite build)
run: npm run build
48 changes: 48 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Build the Fathom cinema (app/) and deploy the static replay demo to GitHub Pages.
# The public demo runs the labeled client-side replay (?source=real) — no server needed.
# One-time repo setup: Settings → Pages → Source = "GitHub Actions".
name: deploy

on:
push:
branches: [main]
workflow_dispatch:

permissions:
contents: read
pages: write
id-token: write

# Only one deploy at a time; don't cancel an in-flight release.
concurrency:
group: pages
cancel-in-progress: false

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: app/package-lock.json
- uses: actions/configure-pages@v5
- name: Install app deps
run: npm --prefix app ci
- name: Build (tsc -b && vite build -> app/dist)
run: npm run build
- uses: actions/upload-pages-artifact@v3
with:
path: app/dist

deploy:
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ app/m3-flame-real.png
*.webm
*.mp4
*.gif
# ...but keep the M5 demo hero gif tracked (README autoplay). webm/mp4 stay ignored
# (regenerable via `npm run app:record`; the mp4 is uploaded directly at launch).
!app/fathom-demo.gif

# dev-only sample render (the synthetic sample; keep the *real* hero pngs tracked)
cinema-sample.png
Expand Down
21 changes: 21 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,27 @@ The Fable aesthetic milestone, with the perf numbers kept honest (`PROOF.md` §7
and query-pairs compute / scene / bloom every 4th frame; `window.__fathom.perf()` returns medians/p95 +
rAF pacing. `app/perf.mjs` drives it on the real GPU (bloom on/off) and prints the PROOF §7 table.

## Deploy (M5 — hosted demo)
The public demo is the built `app/dist` served **statically** — no server. `?source=real` (the default)
fetches the bundled `traces.json` and runs the **labeled client-side replay**, so a static host is enough;
only `?source=live` needs the Node OTLP+SSE server (deferred).

- **GitHub Pages via CI.** `.github/workflows/deploy.yml` builds on every push to `main` (`npm --prefix app ci`
→ `npm run build`), uploads `app/dist` as a Pages artifact, and deploys it. `ci.yml` runs the same build on
PRs as a gate. One-time repo setup: **Settings → Pages → Source = "GitHub Actions"**. URL:
`https://manvendra-github.github.io/Fathom/`.
- **Relative base.** Pages serves under a subpath (`/Fathom/`), so `vite.config.ts` sets `base:'./'`. All asset
URLs and the `import.meta.env.BASE_URL`-relative `traces.json` fetch resolve under any mount point — the same
bundle also works when the recorder serves `dist` at root. Fathom routes only via query params, so relative
base has no routing downside.
- **Demo clip.** `app/record.mjs` records the running `app/dist` on the real GPU (serve dist → `channel:'chrome'`
+ `--force_high_performance_gpu` → wait `[fathom]` ready → `?source=real` → `recordVideo`), then ffmpeg emits
`.mp4` + a two-pass-palette `.gif`. The GIF (`app/fathom-demo.gif`, ~30fps for size) is the README hero and is
git-tracked; `webm`/`mp4` are gitignored (regenerable; the mp4 is uploaded directly at launch). Honesty: the
clip is a real capture, replayed — the HUD's "requests replayed" / "replayed as a live stream" says so.
- **Deferred (live mode).** Hosting the Node server for public `?source=live` needs a live process (Fly.io/Render;
Cloudflare Pages is static-only, a Worker+Durable Object is the advanced alt). Not required for the launch demo.

## Decisions & gotchas
- **Closed-form vs compute for the cinema.** M0–M3 stayed closed-form in the vertex shader for simplicity +
seamless looping. M4 moved motion into a compute pass for the curl-noise flow — but kept it **stateless**
Expand Down
22 changes: 19 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ v1 build (M0–M4 done; see `SPEC.md`):
| `spike/` | perf spike + benchmark (`index.html`, `main.js`, `bench.mjs`, `river-1M.png`) |
| `ingest.mjs` | sentinel `TraceRecord[]` → normalized schema (the mapper to port into `server/`) |
| `synth-traces.mjs` | schema-faithful *synthetic* sample (dev only; clearly labeled in-app) |
| `record.mjs` | Playwright capture → hero `.png` + `.webm` (+ `.mp4`/`.gif` if ffmpeg) |
| `record.mjs` | Playwright capture → hero `.png` + `.webm` (+ `.mp4`/`.gif` if ffmpeg) — the **legacy** `fathom.html` |
| `app/record.mjs` | M5 demo-clip recorder for the **new** `app/dist` (serves dist, `[fathom]`-ready, `?source=real`, `recordVideo`+ffmpeg) → `app/fathom-demo.{gif,mp4,webm,png}` (gif tracked; webm/mp4 gitignored) |
| `.github/workflows/deploy.yml` · `ci.yml` | M5 CI: `deploy.yml` builds `app/dist` → **GitHub Pages** on `main`; `ci.yml` = PR build gate |
| `tools/sentinel-dump.ts` | reference capture script (copy into `<sentinel>/load/`, run, delete) |
| `data/` | captured raw trace records (gitignored) |
| `fathom.html` · `fathom.js` | **legacy** standalone cinema (superseded by `app/`; kept for `record.mjs`) |
Expand All @@ -46,6 +48,11 @@ npm run dev # http://localhost:5173 (Vite dev; ?source=live|r
npm run build # tsc -b + vite build -> app/dist
node app/shot.mjs # build first; screenshots the running app on the real GPU
node app/perf.mjs # build first; M4 per-pass GPU times (compute/scene/bloom, timestamp-query) -> app/m4-richness.png
npm run app:record # build first; M5 demo clip (webm+mp4+gif) on the real GPU -> app/fathom-demo.* (ffmpeg for gif/mp4; FFMPEG=<path> if not on PATH)

# hosted demo (M5) — the deploy is CI-driven; push to main triggers .github/workflows/deploy.yml -> GitHub Pages
# one-time: repo Settings -> Pages -> Source = "GitHub Actions"; demo runs ?source=real (labeled replay, no server)
# URL: https://manvendra-github.github.io/Fathom/

# live server (M1) — point any OTel gateway's OTEL_EXPORTER_OTLP_ENDPOINT at http://localhost:4319/v1/traces
npm run server:install # once
Expand Down Expand Up @@ -126,6 +133,15 @@ GPU/frame with bloom on** — 157× under the 16.7 ms budget on the 4070 (`PROOF
`app/m4-richness.png`, also the README hero). Curl samples global time → replay loops drift subtly within
±0.02 NDC (documented caveat); the curl amplitude budget (jitter 0.03 + turb 0.035 + curl 0.015 = 0.080
< 0.09 lane gap) is load-bearing — don't raise it without redoing the arithmetic (pick-check enforces).
Still missing (v1): M5 hosted demo.
**The v1 build plan is in [`SPEC.md`](./SPEC.md)** (milestones M0–M5). **Next: M5 (hosted demo + launch).**
**M5 in progress (hosted demo — Opus half done)**: the static replay demo is deployed to **GitHub Pages**
via CI — `.github/workflows/deploy.yml` builds `app/dist` and publishes on every push to `main`
(`ci.yml` gates PRs). The demo runs `?source=real` — the **labeled client-side replay**, no server (live
SSE mode is deferred). Vite `base` is now **`'./'`** (relative) so the bundle works under the Pages subpath
*and* at local root (the recorder serves `dist` at root). The **README hero is now an animated GIF**
(`app/fathom-demo.gif`, tracked) captured by `app/record.mjs` on the 4070 (real capture, replayed — the HUD
says so); `webm`/`mp4` are gitignored (regenerable; mp4 uploaded at launch). GIF is ~30fps for size; the
demo *runs* at 60fps. One-time manual step: repo **Settings → Pages → Source = GitHub Actions**. URL:
`https://manvendra-github.github.io/Fathom/`. Still missing (M5, Fable half): README hero copy + public-demo
visual polish + launch (X/HN). Deferred: hosting the Node OTLP+SSE server for public `?source=live`.
**The v1 build plan is in [`SPEC.md`](./SPEC.md)** (milestones M0–M5). **Next: finish M5 (Fable polish + launch).**
Decision context: vault note `next-flagship-project-research.md`.
39 changes: 39 additions & 0 deletions PROOF.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,43 @@ curl flow, model sub-streams, `est. $ saved (cache) —` honest unpriced state).

---

## 8. M5 — hosted demo build + capture (Opus half)
The public demo is the static `app/dist` on **GitHub Pages** (CI-built), running `?source=real` — the
labeled client-side replay, no server. Deploy config: `.github/workflows/deploy.yml` (build → Pages on
`main`) + `ci.yml` (PR build gate). URL: `https://manvendra-github.github.io/Fathom/`.

**(a) Build is green and the bundle is subpath-safe** (`base:'./'` → relative asset URLs, so it works
under `/Fathom/` on Pages and at root for the recorder):
```bash
npm run build
# tsc -b && vite build — ✓ 51 modules transformed, built in 2.53s
# dist/index.html 0.42 kB dist/assets/index-*.css 8.41 kB dist/assets/index-*.js 253.39 kB (gzip 81.74)
# dist/index.html references ./assets/index-*.js ./assets/index-*.css (relative — base:'./' took)
# dist/ also carries traces.json (88,309 B) + traces.sample.json (client-side replay, no server)
```

**(b) Demo clip captured on the real GPU** (`app/record.mjs` → serve dist → Chrome
`--force_high_performance_gpu` → `[fathom]` ready → `?source=real` → `recordVideo` → ffmpeg):
```bash
npm run app:record
# [record] serving app/dist at http://localhost:8976 source=real
# [page] [fathom] replay mode · 24,982 particles · nvidia · lovelace
# [record] hero still -> fathom-demo.png
# [record] video -> fathom-demo.webm
# [record] ffmpeg -> fathom-demo.mp4, fathom-demo.gif
# artifacts: fathom-demo.gif 6.69 MB (760×428, 128 frames, ~30fps) · fathom-demo.mp4 5.32 MB ·
# fathom-demo.webm 3.52 MB · fathom-demo.png 0.42 MB
```
`fathom-demo.gif` is the README hero (git-tracked); `webm`/`mp4` are gitignored (regenerable; the mp4 is
uploaded directly at launch). The clip is a **real capture, replayed** — the HUD reads "requests replayed"
and the panel says "replayed as a live stream" (honest, no manipulation beyond replay-order interleaving).

**M5 Opus exit criterion met:** the static replay demo builds and deploys via CI to a public URL, and the
README autoplay GIF is captured from the shipping bundle on the 4070. GIF is ~30fps for size; the demo
*runs* at 60fps (§7). Remaining (Fable half): README hero copy + public-demo polish + launch.

---

## Claim → evidence
| Claim | Evidence |
|---|---|
Expand All @@ -295,6 +332,8 @@ curl flow, model sub-streams, `est. $ saved (cache) —` honest unpriced state).
| Ingestion is source-agnostic | renderer consumes only the normalized schema (`ARCHITECTURE.md`) |
| Richness pass holds 60fps | `node app/perf.mjs` → 0.106 ms median GPU with bloom on (157× headroom, §7) |
| Picks survive curl-noise motion | pick-check 21/21 (curl bounds + divergence) + pick-e2e 8/8 on the real GPU |
| Hosted demo builds + deploys | `npm run build` green → `app/dist` (subpath-safe, `base:'./'`) → GitHub Pages via CI (§8) |
| Demo clip is real, captured on GPU | `npm run app:record` → `fathom-demo.gif` on nvidia·lovelace, `?source=real` (real capture, replayed — §8) |

## Honest caveats
- Benchmark FPS is vsync-capped (240 Hz) — the **GPU ms** column is the real headroom signal, not FPS.
Expand Down
17 changes: 14 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@ Every particle is one real LLM-gateway span. Requests flow as glowing comets int
real outcome: **cache hits** stream off as a cyan tributary, **429s** retry and fail over in amber,
**PII** is caught and blocked in-path as red flares, and plain **spans/misses** run as the blue river.

![Fathom cinema — real sentinel spans: curl-noise flow + bloom (M4)](./app/m4-richness.png)
[![Fathom cinema — real sentinel spans: curl-noise flow + bloom, replayed at 60fps](./app/fathom-demo.gif)](https://manvendra-github.github.io/Fathom/)

> **▶ Live demo — https://manvendra-github.github.io/Fathom/** · real sentinel spans, replayed
> client-side at 60fps on the GPU (needs a [WebGPU](https://caniuse.com/webgpu) browser). The clip
> above is a real capture, replayed — the HUD says so.

> Perf and the real-data look are proven (see [`PROOF.md`](./PROOF.md)); the v1 build is underway per
> [`SPEC.md`](./SPEC.md). **Done: M0** (the app) **+ M1** (a generic OTLP live server — real sentinel spans
Expand All @@ -16,7 +20,8 @@ real outcome: **cache hits** stream off as a cyan tributary, **429s** retry and
> **+ M4** (richness pass — curl-noise flow in a stateless compute pass, a real HDR **bloom** chain with a
> toggle, model-shaded **sub-streams** within each lane, an honest `est. $ saved (cache)` HUD counter — the
> whole pipeline measured at **0.106 ms GPU/frame** on the 4070, 157× under the 60fps budget).
> **Next:** hosted demo (M5).
> **M5 (underway):** the static replay demo is deployed to **GitHub Pages** via CI (link above);
> README hero GIF + launch polish next.

---

Expand All @@ -35,8 +40,13 @@ npm run dev # http://localhost:5173 (?source=live|real|sample)
npm run build # tsc -b + vite build -> app/dist
node app/shot.mjs # build first; screenshots the running app on the real GPU
node app/perf.mjs # build first; per-pass GPU times (compute/scene/bloom) via timestamp-query
npm run app:record # build first; records the demo clip (webm+mp4+gif) on the real GPU -> app/fathom-demo.*
```

The deployed demo (`.github/workflows/deploy.yml` → GitHub Pages) is the built `app/dist` served
statically; `?source=real` runs the labeled replay fully client-side (no server). ffmpeg is needed
for the `.gif`/`.mp4` from `app:record` (webm-only without it; set `FFMPEG=<path>` if not on PATH).

### 2. Live server — `server/` (M1/M2)
Node + TypeScript: a **generic OTLP/HTTP receiver** (`POST /v1/traces`) → normalized mapper → ring buffer →
**SSE `/stream`**, plus **`GET /traces/:id`** for drill-down (a span's full detail incl. raw attributes; the
Expand Down Expand Up @@ -91,7 +101,8 @@ develop the renderer.
| `spike/` | perf spike + benchmark (`index.html`, `main.js`, `bench.mjs`) |
| `ingest.mjs` | sentinel `TraceRecord[]` → normalized schema (the ingestion contract) |
| `synth-traces.mjs` | synthetic sample generator (dev only) |
| `record.mjs` | Playwright capture → `.png` + `.webm` (+ `.mp4`/`.gif` with ffmpeg) |
| `record.mjs` · `app/record.mjs` | Playwright capture → `.png`+`.webm`(+`.mp4`/`.gif`): `record.mjs` = legacy `fathom.html`; `app/record.mjs` = the **new** `app/` demo clip (M5 hero) |
| `.github/workflows/` | `deploy.yml` (build `app/dist` → **GitHub Pages** on `main`) · `ci.yml` (PR build gate) |
| `tools/sentinel-dump.ts` | reference capture script (copy into `<sentinel>/load/`, run, delete) |
| `fathom.html` · `fathom.js` | legacy standalone cinema (superseded by `app/`) |
| `traces.json` · `data/` | current normalized trace · captured raw records |
Expand Down
13 changes: 8 additions & 5 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,13 @@ receiver and the poller are just two mappers to it.
back the pixel under the cursor). Ship the spatial hash; document the ID-buffer path for scale.
3. **3D cost flame graph** — a second mode aggregating token/$ cost by `provider → model → outcome` as a rotating
3D flame/treemap (WebGPU instanced boxes), reading the same ring buffer. Answers "where does the money go."
4. **Hosted public demo** — deploy the static client + the Node server (SSE needs a live process):
**Fly.io or Render** for server+SSE (Cloudflare Pages is static-only; a Worker+Durable Object is the advanced
alt). Demo runs **replay** of a captured trace (clearly labeled) or a sandboxed demo gateway. A clickable live
link is the retention hook the research calls out.
4. **Hosted public demo** — **shipped (Opus half):** the static client is deployed to **GitHub Pages** via
CI (`.github/workflows/deploy.yml` builds `app/dist` on push to `main`; `ci.yml` gates PRs). The demo runs
`?source=real` — the **labeled client-side replay**, no server needed (Vite `base:'./'` so it works under
the Pages subpath). URL: `https://manvendra-github.github.io/Fathom/`. **Deferred:** hosting the Node
server for public `?source=live` (SSE needs a live process — **Fly.io/Render**; Cloudflare Pages is
static-only, a Worker+Durable Object is the advanced alt). A clickable live link is the retention hook the
research calls out.
5. **Richness pass** — bloom (downsample→blur→add post pass), curl-noise flow (compute pass — reuse the spike's
proven compute path), model-colored sub-streams within a lane, and live counters ($ saved, cache-hit %,
PII caught) driven by the ring buffer.
Expand All @@ -121,7 +124,7 @@ Restructuring is **M0 work**, not done in this spec turn.
| **M2 ✅ done** | Drill-down | ✅ clicking a comet opens the correct span's real attributes, reconciled against `GET /traces/:id` on the real GPU (see `PROOF.md` §5); pick is CPU math mirroring the shader (`app/src/gpu/motion.ts`) | **Opus** (pick math, `/traces/:id`, wiring) ✅; **Fable** (attribute-panel UI — outcome-lit callout card: reticle + leader line, card parked opposite the comet) ✅ |
| **M3 ✅ done** | 3D cost flame graph | ✅ toggle + cost-by-model/provider from the live buffer, reconciles with the HUD on screen; rendered as a **3D WGSL scene** (orbiting monoliths + embers, 4.5 ms/frame — `PROOF.md` §6) | **Opus** (cost aggregation + toggle + reconciliation) ✅; **Fable** (3D WGSL flame: monoliths, embers, orbit camera, ray pick, tracked DOM labels) ✅ |
| **M4 ✅ done** | Richness pass | ✅ bloom (HDR post chain + toggle) + curl-noise (stateless compute pass, pick mirror intact) + model sub-streams + `est. $ saved` counter — measured at **0.106 ms GPU/frame** with bloom on (157× headroom, `timestamp-query`; see `PROOF.md` §7) | **Fable** (curl field, bloom look, sub-streams) ✅; **Opus** (perf measurement, $-saved honesty, pick-mirror integrity) ✅ |
| **M5** | Hosted demo + launch | Public URL runs labeled replay at 60fps; README autoplay GIF + honest headline number; ready for HN/X | **Fable** for the README hero + public-demo polish; **Opus** for deploy/CI config + GIF capture |
| **M5 🚧 in progress** | Hosted demo + launch | **Opus half ✅:** static replay demo deployed to **GitHub Pages** via CI (`deploy.yml` builds `app/dist` on `main`, `ci.yml` gates PRs; `?source=real`, `base:'./'`), README autoplay GIF captured on the 4070 (`app/fathom-demo.gif`, real-capture-replayed, `PROOF.md` §8). **Fable half:** README hero copy + public-demo polish; **launch** (X/HN) pending | **Opus** (deploy/CI config + GIF capture) ✅; **Fable** (README hero + public-demo polish) — pending |

**Model policy** (Opus + Fable only): **Fable** for anything people *look at* (shaders, viz, UI polish, launch surface); **Opus** for anything that must be *provably correct* (pick math, cost reconciliation, OTLP mapping) and all plumbing/tests/doc-sync. Highest-value Fable milestone: **M4**. Don't drop below Opus on **M3's aggregation** — a wrong cost number undercuts the "money deserves to be seen" pitch.

Expand Down
Binary file added app/fathom-demo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added app/fathom-demo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading