Skip to content

Algorithm SSE stream sends one frame then goes silent, and that frame drops the client poll from 2s to 30s with no way back #1837

Description

@jacobo-ortiz

Summary

GET /api/algorithm/stream opens, sends one frame, and then goes silent. It never pushes again, even when work.json changes.

On its own that would be a degraded-but-harmless realtime channel. What makes it worse is the client: the first frame is treated as proof the stream works, and the poll interval drops from 2s to 30s on the strength of it. Since a silent-but-open EventSource never fires onerror, the client never returns to the fast poll. The dashboard ends up refreshing 15× slower precisely because the realtime channel connected successfully.

The visible symptom is "the Algorithm board isn't updating", which is what sent me looking.

Measurement

T+0s    listener opened:  curl -sN --max-time 90 http://localhost:31337/api/algorithm/stream
T+14s   work.json rewritten by ISASync (ISA edit)   <-- inside the listener window
T+79s   captured log: 3 lines total (one `event:`, one `data:`, one blank)

No push, no keepalive, in ~80s. A first attempt was discarded on method: that curl could have expired at the same moment the change landed, so it was repeated with a longer window before concluding anything.

The client half

LIFEOS/PULSE/Observability/src/hooks/useAlgorithmState.ts:

  • :19-20 — FAST_POLL_INTERVAL = 2000 (SSE unavailable) and STALE_POLL_INTERVAL = 30_000 (SSE connected).
  • :100-104 — the first frame sets sseConnectedRef = true and calls restartPolling(STALE_POLL_INTERVAL).
  • :111 — es.onerror only fires on connection errors. An open, mute stream produces none, so the fallback never re-engages.

Server-side: three candidates, none confirmed

The route lives in LIFEOS/PULSE/Observability/observability.ts (handleAlgorithmStreamApi, poller + fs.watch on the STATE directory). I did not attribute a root cause, because I could not discriminate between:

  1. The poller or watcher not starting for the subscriber.
  2. The debounce at :804, which compares now (wall clock) against lastBroadcastMtimeMs (a file mtime). Those are different magnitudes; the intent reads like "time since last broadcast" but the value is "age of the file version".
  3. MEMORY_DIR resolving to a path where statSync throws, making the tick return silently on every pass.

Discriminating probe: log one line per tick inside ensureAlgorithmStreamPoller and broadcastAlgorithmState and see which branch dies.

Suggested fix, two independent halves

  • Server: make the push work (depends on which of the three it is).
  • Client: require more than the opening frame before declaring SSE healthy, and degrade back to the fast poll after N seconds of silence. Worth doing even if the server is fixed: today a server-side failure turns into a silently slow dashboard with no error anywhere.

Both files are stock v7.28.3 here, so this should reproduce on any install with the Algorithm board open.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions