fix(relay): force monotonic created_at on the NIP-IA archive snapshot - #5117
Open
AI-OWEN wants to merge 1 commit into
Open
fix(relay): force monotonic created_at on the NIP-IA archive snapshot#5117AI-OWEN wants to merge 1 commit into
AI-OWEN wants to merge 1 commit into
Conversation
Rapid archive/unarchive mutations publish several kind:13535 snapshots inside one wall-clock second. created_at is second-resolution, so NIP-16 breaks the tie on the lowest event id — a content hash, and therefore an arbitrary winner. An intermediate snapshot can stay authoritative while every kind:8002 delta and every archived_identities row is accepted, and nothing surfaces the mismatch: no error, no warning, no drift check. Observed in the field twice. Six archive requests ~330ms apart grew the table 7 -> 12 while the published snapshot stayed at 7 and remained there for six days across relay restarts. Because Desktop filters mention autocomplete, the DM recipient picker, search and the add-member dialog on this snapshot, the five lost identities stayed fully selectable, and the archive looked like it had silently failed. Force the snapshot's created_at strictly past the previous snapshot's, the same guard emit_addressable_discovery_event and publish_dm_visibility_snapshot already apply. The shared arithmetic is extracted into monotonic_snapshot_created_at so it can be unit-tested without a database; the two existing call sites are left inline and unchanged to keep this diff to the reported bug. Regression coverage simulates six snapshot updates in one wall-clock second and asserts strictly increasing timestamps from T through T+5, plus first-publish and clock-overtakes-previous cases. Refs: block#3848 Signed-off-by: AI-OWEN <AI-OWEN@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3848.
The bug
A burst of NIP-IA archive/unarchive mutations publishes several
kind:13535snapshots inside one wall-clock second.created_atis second-resolution, so NIP-16 resolves the tie on the lowest event id (buzz-db/src/lib.rs— "On same-second tie, lowest event id (lexicographic) wins"). That id is a content hash, so which snapshot survives is effectively arbitrary — it is not the first or the last, and an intermediate list can remain authoritative.Every
kind:8002delta and everyarchived_identitiesrow is accepted. Nothing anywhere reports the mismatch.Why it is worse than a stale count
useIsArchivedPredicate(desktop/src/features/identity-archive/hooks.ts) filters mention autocomplete, the new-message recipient picker, search results and the add-member dialog on this snapshot —useMentions.ts:246drops archived pubkeys from candidates entirely.So a lost snapshot update leaves those identities fully selectable indefinitely, and from the operator's side the archive looks like it silently failed: the receipt says
ok, the table is right, the UI keeps offering the identity.Seen twice in the field. Most recently, six archive requests ~330 ms apart grew
archived_identities7 → 12 while the published snapshot stayed at 7 — and stayed there for six days across relay restarts, because there is no error, no warning and no drift check in the path. It was only caught by comparingselect count(*) from archived_identitiesagainstbuzz agents archivedby hand.The fix
Force the snapshot's
created_atstrictly past the previous snapshot's, exactly asemit_addressable_discovery_eventandpublish_dm_visibility_snapshotalready do for their replaceable snapshots.The arithmetic is extracted into a small pure helper,
monotonic_snapshot_created_at(now, previous), so the burst behaviour is unit-testable without a database.Deliberately left alone: the two existing call sites keep their inline copies. Their semantics are identical to the helper, but migrating them is unrelated to this bug and I would rather keep the diff to what #3848 reports. Happy to fold them in if you would prefer one shared guard.
Tests
crates/buzz-relay/src/handlers/side_effects.rs:snapshots_within_one_second_get_strictly_increasing_timestamps— the case from the issue: six updates at a fixednow, assertingTthroughT+5and strict monotonicity.first_snapshot_uses_wall_clock— no previous snapshot.snapshot_follows_wall_clock_once_it_overtakes_the_previous— a burst can pushcreated_atahead of the clock; once real time passes it, the snapshot returns to wall-clock rather than drifting further ahead.The full
cargo test -p buzz-relay --librun is 852 passed / 9 failed in my environment. All nine failures areapi::adminandapi::mediatests that need Postgres (Sqlx(PoolTimedOut)); I confirmed the identical nine fail on unmodifiedorigin/mainhere, so they are environmental and not from this change. Worth a second look on CI.Related
#4617 is the other half of this.
handlers/identity_archive.rsearly-returns onif !changedbefore reachingpublish_nipia_archival_list, so re-archiving an already-archived identity is a no-op that never republishes — which is why an operator stuck in this state cannot repair it by simply re-archiving the missing entries. The two fixes are complementary: monotoniccreated_atprevents the loss, unconditional republish makes it recoverable.For anyone currently stuck, the workaround is one state-changing cycle, since
publish_nipia_archival_listrebuilds the full list fromlist_archived()rather than applying a delta: