Skip to content

The edge cannot front a streaming workload, and nothing says so #110

Description

@The-Running-Dev

What changes, and for whom. A product whose whole surface is a long-lived stream — a console
watching an agent work, a live log, anything where the first byte matters and the last one arrives
minutes later — cannot sit behind the edge today. It can be pointed at it, and it will appear to
work in a smoke test, but a real session will hang for the length of the forward budget and then
answer with a timeout. Nothing in the edge's own surface says this; you find out by trying it. For
whoever owns that product, the change is that "can Platform front this?" becomes a question with a
written answer instead of an experiment.

Why now. The question arrived from outside: SkyNet HR is a self-hosted browser console that
drives a coding agent CLI, and its entire downstream is Server-Sent Events. It asked whether
Platform could host it. The answer today is no, and the reason turned out to be structural rather
than incidental — the edge forwards by reading the whole response body into memory before it
answers, under a single budget that covers connecting, headers and body together. That is exactly
right for what it currently fronts, where every response is a few kilobytes of complete JSON. It is
not a defect. It is a scope boundary that nobody has written down, and the cost of leaving it
unwritten is that the next person asks the same question and re-derives the same answer by reading
the forwarder.

The part that makes this a real decision rather than a patch. The edge's honest-answer property
and streaming are mutually exclusive, and choosing between them is the whole of this issue.

Today, a workload that dies part-way through replying is caught while the body is still being read,
and the caller gets a clean, attributable answer naming which hop failed. That is a genuinely good
property — it was settled deliberately while closing S7, and #106 is currently widening the
contract to describe it properly.

It only works because the body is buffered. The moment the edge flushes the first byte to its own
caller, the status line has already been sent and cannot be revised. A streaming edge physically
cannot answer a mid-stream workload death with a status code. It would need a different mechanism
entirely — an in-band error the client is expected to understand — and every client would need to
understand it. So this is not "add streaming support alongside what exists". It is a decision about
whether the edge has one behaviour or two, and if two, what makes them different and who chooses.

Deliberately not proposing a design here. The forwarder, the response type, the timeout model
and the error contract are all implicated, and picking an approach in an issue would frame the
question before it has been asked properly.

Done when

  • A decision is recorded on whether the edge fronts streaming responses at all, or whether
    request/response is a stated and documented boundary
  • If it is a boundary: design/20-contract.md says so plainly, so a reader learns it from the
    contract rather than from the forwarder's implementation
  • If streaming is adopted: the mid-stream failure question above is answered explicitly —
    what a caller receives when a workload dies after the first byte, and how that differs from
    the status-code answer available today
  • If streaming is adopted: the timeout model distinguishes time-to-first-byte from total
    duration, so a long-lived stream is not cancelled on the same budget that guards a hung
    connect
  • Whichever way it goes, the outcome is reconciled with The edge's unreachable error is described more narrowly than it behaves #106, which is currently amending the
    contract to describe post-headers failure as an answerable status
  • A test proves the chosen behaviour with a workload that emits incrementally and then dies
    mid-response — not one that returns a complete body

Agent instructions

Provenance. Read-only survey from outside this repository, 2026-08-11, at main = c880e56.
Nothing here was executed and no test was run. Each claim below names the file it came from; the
"not read" list at the end is as load-bearing as the rest.

What was read, and what it says

  • workloads/game-edge/SubZeroDev.Platform.GameEdge/Forwarding.cs
    • GameWorkloadForwarder.ForwardAsync sends with HttpCompletionOption.ResponseHeadersRead,
      then calls response.Content.ReadAsByteArrayAsync(budget.Token). The body is fully
      materialised in memory before the method returns. Its own comment states the intent plainly:
      ResponseHeadersRead is chosen so that the timeout budget covers the body read rather than
      only connect-and-headers — i.e. the buffering is deliberate, not an oversight.
    • budget is a linked CancellationTokenSource with CancelAfter(options.ForwardTimeout),
      covering connect, headers and body under one deadline. A stream that stays open longer than
      ForwardTimeout by design is cancelled and surfaces as EdgeError.WorkloadTimeout().
    • Exactly one attempt, no retry, per its own comment. Keep this. Any streaming design must
      preserve it — a retrying edge replays a resumable stream's position header and breaks
      reconnect-and-replay for any consumer that has one.
    • ForwardedResponse is constructed as (int status, byte[] body, string? contentType). The
      response type itself has no streaming shape, so this is a type change, not a call-site change.
    • The catch for HttpRequestException or IOException around the body read is the mechanism
      behind the honest-answer property, and its comment explains the alternative it rejects
      (escaping as an unattributed 500). This is the exact code path streaming would remove.
  • workloads/game-edge/SubZeroDev.Platform.GameEdge/Options.csGameEdgeOptions carries
    WorkloadBaseAddress, ForwardTimeout, LivenessTimeout. One forward timeout, no separate
    first-byte budget.
  • workloads/game-edge/SubZeroDev.Platform.GameEdge/Readiness.cs — not implicated, but read for
    context. Its liveness-must-not-touch-a-dependency rule is unaffected either way.
  • workloads/game-service/src/lifecycle.ts — the workload behind the edge answers every route
    with a complete JSON body over a uniform POST /v<n>/<operation> wire. The edge's buffering is
    correctly matched to its only current consumer.
    This issue is not a complaint about that.

Not read, and each needs checking before any of the above is treated as complete

  • Endpoints.cs — how ForwardedResponse reaches the wire. ASP.NET response buffering, and
    whether anything there would also need to change to flush per chunk, is unverified.
  • Errors.cs — the full EdgeError shape beyond the two factory calls named above.
  • design/20-contract.md — referenced by a comment in Forwarding.cs as "The edge — EdgeError".
    Whether the contract already says anything about response size, shape or streaming was not
    checked, and it is the first thing to confirm: if it already excludes streaming, most of this
    issue collapses into a documentation pointer.
  • SubZeroDev.Platform.GameEdge.Tests/ForwardingTests.cs, EdgeHostTests.cs,
    Support/FakeWorkload.cs — the existing fake workload almost certainly returns complete bodies,
    which is why nothing today exercises the incremental case.

Relationship to #106. #106 widens the contract's WorkloadUnreachable description to cover a
forward that fails after the response headers arrive. That amendment is correct for the edge as
it stands, and it should not be blocked on this. But it documents a guarantee that a streaming edge
cannot keep — so whichever way this issue is decided, #106's wording is the thing to revisit, and
the two should not be reconciled independently by different sessions.

Tier. Deep reasoning. This is an architectural and contract question about a public interface,
not an implementation task, and AGENTS.md § Model, effort, and review budget routes it
accordingly. Do not open it at implementation tier and start editing the forwarder.

Stop if the investigation turns into writing the streaming implementation. The deliverable here
is a decision and, if it goes that way, a contract amendment through /contract — the same
boundary #106 states for itself.

External context, for the reader who wants the concrete case. SkyNet HR
(The-Running-Dev/SubZeroDev.SkyNetHR) had already established this failure independently against
a different proxy: its decision D10 records that Open WebUI's HTTP proxy buffers any content type
outside its streaming allow-list and therefore breaks Server-Sent Events entirely. The edge's
ReadAsByteArrayAsync reproduces that behaviour in C#. That repository is not a Platform consumer
and nothing here depends on it; it is named only because it is where the question came from and it
holds a worked description of what the failure looks like from the client side.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:edgeGame edge and public roadmapenhancementNew feature or requestneeds-decisionRequires an architectural or contract decisionspikeFindings from a read-only investigation; not a slice

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions