lync durability: awaited-union cursor + loss-free trial (dee-s6dc, dee-i1wc) - #7
Closed
deepfates wants to merge 6 commits into
Closed
lync durability: awaited-union cursor + loss-free trial (dee-s6dc, dee-i1wc)#7deepfates wants to merge 6 commits into
deepfates wants to merge 6 commits into
Conversation
…eration that issued it (dee-u6tq)
recoverRoom mints a random generation id (never persisted: every restart is
a new generation), carried additively on ev/live frames. The CLI sync cursor
file stores {seq, generation}; a mismatch resets to 0 and resubscribes —
union makes the re-download duplicate no-ops. Stale lives from superseded
subs are counted, not trusted. Regression test is the bug's exact shape:
persisted events, a broadcast whose disk write failed consuming a seq, a
restart, a stale cursor — the client ends with every persisted event
(verified failing with the reset neutered).
…(dee-s6dc) Frames apply strictly in arrival order (serialized chain). A store-write failure freezes the root's cursor — live frames cannot leapfrog it — and screams through the new additive SyncStatus.failures channel; the line is re-fetched on the next resubscribe. Conflicts and garbage are surfaced too. Also carries the generation reset (dee-u6tq) on the synced-store side, with stale-live counting so a superseded sub cannot re-poison a reset cursor. Neuter-verified: freezing disabled makes the regression tests fail.
…(dee-i1wc) The synced store swallowed all relay errs except same-id-conflict — a persist-failed durability failure fanned out but never reached the client's status channel, so a store consumer could not see the relay lose a write. Route every non-conflict relay err (persist-failed, conflict-persist-failed, recovered-damaged-tail, line-without-id, server-error, ...) into the failures channel. Nothing fails invisibly. Unit test asserts persist-failed surfaces and is not miscategorised as a conflict.
A real relay plus real synced stores over the global WebSocket, run through the full gauntlet against one shared root: (a) a client's socket drops mid-stream and auto-reconnects; (b) the relay's .lync file goes read-only, so a write is fanned out but refused by disk and surfaced as persist-failed; (c) the server restarts into a new log generation. Final invariant: every event a client successfully appended — including the one the dead disk refused — ends up in every other client's store AND on the relay's on-disk .lync file, and every failure (persist-failed, generation-changed) was surfaced on every client's status channel. Neuter-verified: dropping the err surfacing fails leg (b); neutering the generation reset fails leg (c).
A relay whose disk write failed kept the line in memory and broadcast it, but a same-generation re-push was a byId duplicate no-op that never retried the write. The durable log stayed silently incomplete until a restart rebuilt byId from disk -- undercutting the format's thesis that the saved log is the truth. Now each room tracks its unpersisted lines (ordered by id = append order). persistPending() drains them, in order, as one serialized unit before the next append to the room, and a same-line re-push retries the write instead of no-oping. A successful flush clears the line; a still-dead disk stops the drain (later lines stay pending, never reordered) and re-surfaces persist-failed. The on-disk log heals on its own, no restart required. Proof: test/cli/loss-free-trial.heal-without-restart.integration.test.ts -- disk fails for X (X reaches all clients, off disk, persist-failed surfaces), disk heals, another client appends Y to the same root with NO restart, and both X and Y land on disk in append order and in every client's store. The retry is load-bearing: neuter the pending flush and the leg fails.
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finishes the durability wave on top of
consolidate-one-package. Stacked commits: the relay generation-id fix (dee-u6tq), the awaited-union cursor fix (dee-s6dc), the relay-error surfacing fix, and the loss-free trial (dee-i1wc).dee-s6dc — await + inspect every union before the cursor advances
The synced store did fire-and-forget
void inner.union(...)and advanced the receive cursor unconditionally. Now every frame's union is awaited in a serialized chain and inspected: the cursor advances only on real success (added / duplicate / buffered) or on unusable garbage; a store-write failure freezes the root's cursor (the line is re-fetched on the next resubscribe, never skipped) and screams through the newSyncStatus.failureschannel. A slow union cannot let a later frame leapfrog it. Regression test injects a store-write failure on frame k, asserts the cursor stays at k-1, the failure surfaces, and the event applies after the store heals and the client resubscribes.Relay-error surfacing (prerequisite for the trial)
The synced store swallowed every relay err except
same-id-conflict. Apersist-faileddurability failure was fanned out but never reached the client's status. Now every non-conflict relay err lands infailures. Nothing fails invisibly.dee-i1wc — the loss-free trial (milestone-6 proof)
test/cli/loss-free-trial.integration.test.ts: a real relay plus real synced stores over the global WebSocket, run through the full gauntlet against one shared root — (a) a client's socket drops mid-stream and auto-reconnects; (b) the relay's.lyncfile goes read-only, so a write is fanned out but refused by disk and surfaced aspersist-failed; (c) the server restarts into a new log generation. Final invariant: every event any client successfully appended — including the one the dead disk refused — ends up in every other client's store AND on the relay's on-disk.lyncfile, and every failure was surfaced on every client's status channel. Neuter-verified both ways (drop err surfacing → leg b fails; neuter the generation reset → leg c fails).Evidence
pnpm verifygreen from the worktree root: 108 tests, typecheck, readme examples under@deepfates/lync.bash scripts/fresh-clone-smoke.shgreen.pnpm packproducesdeepfates-lync-0.3.0.tgz.Do not merge/close — coordinator holds the key.