Skip to content

Relay: a duplicate that differs from the cached copy makes the track malformed (§9.1) - #105

Merged
floatdrop merged 2 commits into
draft-20from
fix/duplicate-consistency
Sep 26, 2026
Merged

floatdrop merged 2 commits into
draft-20from
fix/duplicate-consistency

Conversation

@floatdrop

Copy link
Copy Markdown
Owner

Summary

§9.1: "An endpoint that receives a duplicate Object with a different Forwarding Preference, Subgroup ID, Priority or Payload MUST treat the track as Malformed" (also §2.4.2 items 6 and 7). Until now, when a second publisher's copy of an Object lost the dedup claim (§9.3), the relay dropped it without comparing it to anything.

The relay now compares that copy with the first copy in its cache, on both the subgroup and datagram paths. It ends the track (PUBLISH_DONE MALFORMED_TRACK, §2.4.2) when the copies differ in:

  • Forwarding Preference, Subgroup ID or Priority: always compared. Priority is the resolved value, inline or the alias's DEFAULT_PUBLISHER_PRIORITY. Subgroup ID is compared only when both copies came on subgroup streams.
  • Payload or Immutable Properties: compared only when both copies are Normal, because only a Normal Object has a Payload or Properties (§11.2.1.1, §11.2.1.2). Immutable Properties are compared byte for byte, and one copy having them while the other doesn't counts as a difference (§12.7).

Not treated as malformed:

  • Mutable Properties differ: allowed by §9.1 item 2.
  • Normal becomes End of Group or End of Track: the existing-to-not-existing change §9.1 item 1 allows.
  • The reverse order: a late Object, which §2.1 says is not malformed.

Decisions (made with you)

  • No cached copy, no check: a duplicate is compared only against a cached first copy. If that copy was evicted, expired (§12.3), or not yet cached by another publisher's stream, nothing is compared. This is a documented limitation; it keeps per-Object hashing off the hot path.
  • Immutable Properties included: they're compared on top of §9.1's four fields.
  • Status differences allowed: as described above.

Interpretation

A copy that an announced gap says doesn't exist is still compared against a cached first copy. §2.1 excuses its late arrival, not different content.

API

  • New: message.ImmutableProperties(raw) ([]byte, bool), which returns the Immutable Properties value exactly as serialized, without allocating.

Test changes

  • Existing tests corrected: three tests sent "duplicates" with a different Payload or Subgroup, which §9.1 now makes malformed. Two multi-publisher tests now send identical copies. TestFanout_UpdatesTrackEntryLargestObject no longer re-sends Object {4, 1} with different content.
  • Refactor: runFanout sets terminalSeen once instead of in two branches. It behaves the same and keeps the function under the gocyclo limit.

Tests

  • Malformed cases: the seven cases in TestRelay_DuplicateConsistency were verified red with the check disabled. They include a subgroup copy duplicated by a datagram, and two datagrams.
  • Guard cases:
    • Six cases must not end the track, and publisher B's own next Object has to arrive afterwards.
    • Comparing all Properties, or Payloads regardless of Status, makes a guard fail.
    • The two Status-plus-Immutable-Properties guards were verified red before the review fix.
  • Message: TestImmutableProperties, including a zero-allocation check.
  • Checks: lint, the full suite, -race and make bench-quick all pass, with no allocs/op increase.

Not in this PR

The rest of the §2.4.2 list is still undetected:

  • Priority changing within a Subgroup.
  • Objects past a Subgroup's, Group's or Track's final Object, beyond the existing same-stream END_OF_GROUP check.

🤖 Generated with Claude Code

floatdrop and others added 2 commits September 26, 2026 13:21
…rack malformed (§9.1)

§9.1: "An endpoint that receives a duplicate Object with a different
Forwarding Preference, Subgroup ID, Priority or Payload MUST treat the
track as Malformed" (also §2.4.2 items 6 and 7). The relay dropped a
copy that lost the dedup claim without looking at it.

It now compares the copy with the first one in the cache, on the
subgroup and datagram paths, and ends the track (PUBLISH_DONE
MALFORMED_TRACK) on a difference in those fields or in Immutable
Properties (§12.7, byte for byte; one copy having them and the other
not counts). Mutable Properties may differ (§9.1). Payloads are compared
only when both copies are Normal: Normal may become End of Group or
Track (§9.1), and the reverse is a late Object (§2.1). Decisions made
with the user; the limitation is documented: a copy is not compared
once the first one left the cache.

New: message.ImmutableProperties (zero-alloc). runFanout sets
terminalSeen once, which is equivalent and keeps it under gocyclo.

Tests whose "duplicates" differed in Payload or Subgroup are corrected:
§9.1 now makes them malformed. TestRelay_DuplicateConsistency's seven
malformed cases were verified red with the check disabled; its guards
fail when all Properties, or Payloads regardless of Status, are compared.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… copies are Normal

Review follow-up. Only a Normal Object has a Payload or Properties
(§11.2.1.1, §11.2.1.2), but Immutable Properties were compared whatever
the Status, so a legal duplicate ended the track: a Normal copy with
Immutable Properties becoming End of Group (§9.1), or the reverse, a
late Object (§2.1). Forwarding Preference, Subgroup ID and Priority are
still compared across a Status change.

Also documents that a copy an announced gap says does not exist is still
compared with a cached first copy: §2.1 excuses its arrival, not a
different content.

Both new guard cases of TestRelay_DuplicateConsistency were verified red
before the fix.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@floatdrop
floatdrop merged commit 31020b9 into draft-20 Sep 26, 2026
11 checks passed
@floatdrop
floatdrop deleted the fix/duplicate-consistency branch September 26, 2026 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant