Skip to content

Relay: forward a later upstream's Object Properties on a merged Subgroup - #103

Merged
floatdrop merged 3 commits into
draft-20from
fix/merged-subgroup-properties
Sep 26, 2026
Merged

floatdrop merged 3 commits into
draft-20from
fix/merged-subgroup-properties

Conversation

@floatdrop

Copy link
Copy Markdown
Owner

Summary

When several upstreams feed one Subgroup (§9.3), each subscriber's downstream stream copies the first contributor's SUBGROUP_HEADER. If that header had PROPERTIES clear (§11.4.2), a later contributor's Object Properties were silently left out on write. That breaks §2.5: a relay "MUST forward the Property to downstream subscribers".

The fix is reopen on demand (the design you chose). The writer tracks whether its streams set PROPERTIES. When an Object with Properties meets a stream without the bit:

  • Stream not opened yet: it opens with the bit set, and nothing is reset.
  • Stream already open: it is reset and reopened with the bit set.

Later streams keep the bit. Streams whose Objects carry no Properties are unchanged.

FIRST_OBJECT fix (review follow-ups). A contributor's FIRST_OBJECT claim (§11.4.2) is honoured only if no lower Object ID was already forwarded in the Subgroup, whichever contributor forwarded it and whether or not this subscriber received it. Objects are published in ascending order (§2.2), so a lower ID proves the claim wrong, and a higher one doesn't. This is decided once per Subgroup, under its lock. Before this, a properties reopen, a gap reopen or a lazy open could set the bit wrongly when two publishers each open with FIRST_OBJECT set.

API

  • New: relay.ResetCauseProperties (metric label "properties"). It is added at the end of the enum, so existing values keep their numbers.

Trade-off (documented in the constant's doc and STATUS.md)

Like a §11.4.3 gap reopen, the reset keeps the Objects already written only where RESET_STREAM_AT is in use (MarkReliable). That's quic-go with the extension negotiated. Over WebTransport, or without the extension, unacknowledged Objects on the old stream can be lost. Always setting PROPERTIES would avoid resets entirely, at one byte per Object; you chose against that.

Not in this PR

  • Other header fields still come from the first contributor. None of them loses data covered by a MUST, so I left them:
    • END_OF_GROUP: if only a later upstream sets it, the "largest Object is here" hint is lost.
    • Publisher priority: the downstream stream uses the first upstream's priority, including its default.
  • FIRST_OBJECT state is lost with the Subgroup set. It lives only while some contributor feeds the Subgroup. If every contributor leaves and a new one arrives, its FIRST_OBJECT claim is not checked against Objects forwarded earlier. Fixing that means keeping per-Subgroup state on the track entry, next to the dedup ledger.
  • Expiry vs. reopen ordering: an expired Object that carries Properties can still trigger the reopen before it's dropped. That costs at most one extra reset. Expiry is deliberately checked after the open, because the open can block.

Tests

  • Red first: TestFanout_MultiPublisher_ForwardsEveryContributorsProperties failed on the unpatched relay in its "first without, second with" case. It also checks the reopen count and that a new stream isn't marked FIRST_OBJECT; that assertion failed before the follow-up fix.

  • Red first: TestFanout_MultiPublisher_FirstObjectOnlyForSubgroupsFirst has two cases:

    • Lower Object forwarded first, filtered out for this subscriber: FIRST_OBJECT must be clear. A second, unfiltered subscriber confirms the Object reached the relay first.
    • A replay starting at Object 5 first, then the true Object 0: FIRST_OBJECT must be set.

    Each case was verified red against the version before its fix. Removing the lowest-ID check fails both FIRST_OBJECT tests.

  • Guards: the other two cases pass before and after.

  • Metrics: TestResetCauseString covers the new label, and the metrics test checks that a clean Subgroup reports no properties resets.

  • Checks: lint, the full suite, -race and make bench-quick all pass, with no allocs/op increase.

🤖 Generated with Claude Code

floatdrop and others added 3 commits September 26, 2026 11:23
…Subgroup

With several upstreams feeding one Subgroup (§9.3), each subscriber's
downstream stream copies the first contributor's SUBGROUP_HEADER. When
that header had PROPERTIES clear (§11.4.2), a later contributor's Object
Properties were silently left out on write, though a relay MUST forward
them (§2.5).

The writer now tracks whether its streams set PROPERTIES. An Object with
Properties on a stream without the bit turns it on: a stream not yet
opened just opens with it; an open one is reset and reopened with it
(reopen on demand, the design chosen with the user). Later streams keep
the bit. New metrics cause relay.ResetCauseProperties ("properties"),
appended to the enum.

TestFanout_MultiPublisher_ForwardsEveryContributorsProperties was
verified red on the unpatched relay ("first without, second with"); the
other two cases guard the paths that already worked.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…sent

Review follow-up. A properties reopen (or a gap reopen) for an Object
that was its contributor's first could open the new stream with
FIRST_OBJECT set (§11.4.2), though another contributor's Object had gone
out before it. reopen now clears it once the writer sent any Object.

Also: the ResetCauseProperties doc no longer claims no Object is lost;
written Objects survive the reset only where RESET_STREAM_AT is in use,
noted in STATUS.md with the always-set-PROPERTIES alternative.

The FIRST_OBJECT assertion added to
TestFanout_MultiPublisher_ForwardsEveryContributorsProperties was
verified red before the fix.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…orwarded

Review follow-up. The writer-local sentAny counted Objects sent to one
subscriber, but FIRST_OBJECT (§11.4.2) is about the first Object ever
published in the Subgroup (§2.2): a subscriber whose filter hid Object 0
still got a FIRST_OBJECT stream beginning at another contributor's
Object 1. And any earlier Object is not a contradiction, only a lower
one: Objects are published in ascending order (§2.2), so a replay
starting at Object 5 must not suppress the true Object 0's bit.

The Subgroup's writer set now records the lowest Object ID forwarded,
under sg.Mu (claimFirst), and fwdObject.first honours a contributor's
claim only below it; sentAny is gone. Limitation, documented on the
field: the state lives as long as the set, so after every contributor
left a new one's claim is not checked against earlier Objects.

TestFanout_MultiPublisher_FirstObjectOnlyForSubgroupsFirst was verified
red for each case against the code before its fix; removing the check
fails it and the properties test's FIRST_OBJECT assertion.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@floatdrop
floatdrop merged commit 028f4cf into draft-20 Sep 26, 2026
11 checks passed
@floatdrop
floatdrop deleted the fix/merged-subgroup-properties branch September 26, 2026 06:39
floatdrop added a commit that referenced this pull request Sep 26, 2026
…ributors

#103 honours a contributor's FIRST_OBJECT claim (§11.4.2) only if no
lower Object ID was forwarded in the Subgroup (§2.2), but kept that in
the Subgroup's writer set, which is torn down when its last contributor
leaves: a later contributor claiming FIRST_OBJECT above an Object
already forwarded was then honoured.

The dedup ledger now records the lowest Object ID forwarded per
Subgroup (status Objects included, datagrams not), and a new writer set
starts from it (TrackEntry.LowestForwarded), within the ledger's
32-Group window. It reads the ledger before taking sg.Mu and keeps the
minimum, since a contributor that joined meanwhile may already have
forwarded a lower Object; that race is not reproducible in a test and
was found in review.

TestFanout_MultiPublisher_FirstObjectAfterTeardown and
TestTrackEntry_LowestForwarded were verified red first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant