Persist closed candles in PostgreSQL and read them back

[coder058]

Why

Candle snapshots only lived in one process's memory. A restart lost them, and
when the upstream endpoint was unreachable the workspace had nothing to show.
There was no SQL database in the project at all.

What this adds

• ingest/ — a Python ingester reading the same documented Hyperliquid
  candleSnapshot endpoint the browser uses, applying the same boundary rules
  (ingest/candles.py mirrors app/publicMarket.ts): a still-forming candle, a
  candle that does not span its interval, a non-numeric price or inconsistent
  OHLC bounds are refused before any write. Rows sharing an opening timestamp
  with different values are a conflict, not a duplicate, and reject the response.
• ingest/schema.sql — (symbol, interval, open_time) is the primary key. The
  upsert suppresses no-op updates, so each pass reports inserted, updated and
  unchanged counts, and a corrected candle updates in place. ingest_runs
  records every attempt, including failures, with the wall time measured around
  its fetch and its write.
• GET /api/stored/:symbol — reads those rows back for the existing chart,
  replay cursor and timeframe comparison. Selector group Stored candles.
  An empty store is 404 with an instruction; an unreachable database is 503,
  never a silent fall back to the live endpoint.
• docker-compose.yml — database, ingester and app against a named volume.
• A persistence CI job against a real PostgreSQL service container.

Evidence from CI

Run 34031460448,
both jobs green. From the persistence job log:

inserted 48, updated 0, unchanged 0   first pass
inserted  0, updated 0, unchanged 48   same payload again
inserted  0, updated 1, unchanged 47   one corrected candle
fetch_ms null, write_ms null, error CandleError   a failed attempt, still recorded

scripts/check_persistence.mjs then read the rows back in a separate process
that never saw the ingester, and again after docker restart of the PostgreSQL
container. Both times: rows came from the database, the stored frames passed the
chart's own validateBars unchanged, and a replay prefix aggregated to 4h
without pulling in a later candle.

Test counts: 44 Node tests (12 new), 24 Python tests (6 of them PostgreSQL
integration, skipped unless DATABASE_URL is set).

Limits

Closed candles only, so this is a persisted snapshot history and not a tick
feed. One ingester process; no retention, partitioning or backfill policy.
Reads are bounded to 300 bars per interval. The timings are wall-clock
measurements between explicit start and end points in one process — not
throughput, database server time, network time or exchange-to-screen latency,
and no synchronized clocks are involved. The counts describe one statement's
view, not concurrent ingesters. Vercel does not serve stored candles.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b457304d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Select an available timeframe for stored sources

When a user selects any new Stored candles entry, MarketWorkspace.selectMarket initializes every non-recording source to 5m, while this branch restricts stored sources to 1h, 4h, and 1d; the post-load timeframe correction also runs only for public sources. Consequently, a successful stored response initially renders an empty chart with no selected timeframe or replay controls until the user manually clicks an available interval. Initialize stored sources to 1h or apply the loaded-frame fallback to them as well.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Mount the PostgreSQL 18 data directory

With the selected postgres:18-alpine image, PostgreSQL stores data beneath /var/lib/postgresql/18/docker, and the official image guidance for 18+ therefore requires mounting /var/lib/postgresql, not the legacy /var/lib/postgresql/data path (official image documentation). This named volume is mounted at an unused child path, so the actual cluster resides in an anonymous image volume and can be lost when the database container is recreated, defeating the persistence guarantee this compose setup introduces.

Useful? React with 👍 / 👎.