Relay: use §11.4.3's same-upstream and Prior Object ID Gap next-Object rules - #102
Merged
Merged
Conversation
…Object rules The relay kept an Object on an existing downstream subgroup stream only when its ID was one greater than the previous Object's, so a publisher skipping Object IDs cost a reset and a reopen per gap. §11.4.3 lists three ways to tell "the next Object"; the relay now also uses: - the Object was read next from the same upstream subgroup stream as the previously sent one, with only Objects that failed the subscriber's filters between (Forward State is one, §5.1.5); - its Prior Object ID Gap (§12.9) says the IDs between do not exist. Knowing from the cache or the Range Filters that the missing IDs are in other Subgroups or filtered out is not used (scope chosen with the user). runFanout numbers every Object read from an inbound stream, dedup losers included, since each is an Object between its neighbours. admit carries the run across filtered Objects; a queue overflow or a §12.3 expiry drop breaks it, since the dropped Object was not sent. New: message.PriorObjectIDGap (zero-alloc, shares the CheckObjectProperties walker). Tests: TestFanout_NextObject_GapOnOneUpstreamStaysOnStream, _FilteredObjectsBetween, _AcrossUpstreams (gap case) and TestRelay_ForwardStateOmissionKeepsStream were verified red on the unpatched relay; the other-upstream, duplicate-between and relay-drop cases guard against over-applying the rule and pass before and after. Each part of the change was removed in turn and a test failed each time. Tests that relied on a single-upstream gap forcing a reopen now use a second publisher (PUBLISH_DONE) or moved into the white-box drop test (FIRST_OBJECT on a reopened stream). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ious Object Review follow-up. A gap reaching past the previous Object covers an Object the relay already received, which §12.9 makes a malformed track, so it cannot show the Object is the next one (§11.4.3). The Object now goes on a new stream. Ending the track for it (§2.4.2) is left out: like the other §12.9 conditions that need earlier Objects, it is not detected. Also asserts message.PriorObjectIDGap does not allocate. The "gap covering the previous Object" case was verified red before the fix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The relay kept an Object on an existing downstream subgroup stream only when its ID was one greater than the previous Object's. A publisher that skips Object IDs therefore cost one reset and reopen per gap. §11.4.3 gives three ways to tell "the next Object", and the relay now uses two more:
The run breaks, and the relay resets and reopens as before, when something that should have been sent in between wasn't:
Whether a stream ends with a FIN or a reset is unchanged. A publisher skipping IDs (0, 2, 5) now gets one stream that ends with a FIN. Filtered or paused runs still end with a reset.
Decisions
TestFanout_NextObject_DuplicateBetweenBreaksRuncovers it.API
message.PriorObjectIDGap(raw []byte) (uint64, bool). It is zero-alloc (a test checks this) and reuses theCheckObjectPropertieswalker.Tests
TestFanout_NextObject_GapOnOneUpstreamStaysOnStream_FilteredObjectsBetween_AcrossUpstreams(the gap cases, including the review follow-up's covering gap)TestRelay_ForwardStateOmissionKeepsStreamTestPriorObjectIDGapTestSubgroupWriter_RelayDropBreaksRun: overflow and expiry) keep the rule from being over-applied. They pass before and after.TestPublishDone_AfterGapReopennow uses a second publisher.TestFanout_GapInForwardedObjectIDsOpensNewStreamwas replaced by the new tests.-raceandmake bench-quick(no allocs/op increase) all pass.Known gaps, not in this PR
Assumption, marked per CLAUDE.md: a gap property covering a received Object doesn't end the track. The relay doesn't trust such a gap and reopens the stream, but it doesn't end the track either. §12.9 makes that a malformed track, and §2.4.2 then requires PUBLISH_DONE MALFORMED_TRACK and no caching.
CheckObjectPropertiesalready skips every §12.9 condition that needs earlier Objects. Detecting them properly needs state for the Objects received in each Group, so it belongs on its own branch rather than as a special case in the fanout.endMalformedTrack, and thegap covering the previous Objecttest should expect PUBLISH_DONE MALFORMED_TRACK instead of a reopen.Object Properties may be lost when two publishers feed one Subgroup. When two publishers feed one Subgroup, the merged downstream stream reuses the first contributor's SUBGROUP_HEADER. If that header has the Properties bit clear, a later contributor's Object Properties are probably lost on the way out. I suspect this from reading the code and haven't confirmed it with a test. It needs its own branch.
🤖 Generated with Claude Code