@deepfates/lync 0.3.0: one package, durable sync, ready to publish

[deepfates]

What this is

lync is a file format for keeping interaction history. Conversations, stories, and the notes people make about them. A .lync file is plain text, one JSON object per line, and each line is one event that never changes. You correct or retract something by adding a new event that points back at the old one, never by editing in place. Merging two copies means keeping every event that appears in either copy. There is no merge logic to get wrong, because two copies of the same event are the same event, and two different events that claim the same id are both kept and flagged out loud.

@deepfates/lync is the reference implementation, and after this change it is one package with no dependencies. It holds the parser, the event stores, the computed views, the loom and turn API, live sync, the lync command, and the sync relay. A program that only wants to read a .lync file installs nothing extra. The relay needs one library called ws, and it asks for that only if you actually run a relay.

Merging this says: this is @deepfates/lync 0.3.0, one name, one story, ready to publish.

What is in the box

• The format spec, in FORMAT.md. It stands on its own. You do not need any of this code to implement lync in another language.
• Conformance vectors, in test/vectors/v0. These are the test cases that let someone check a lync implementation written in any language. We treat them as a product, not as test plumbing.
• The library. Parse, merge, compute views, run looms, keep an index, resolve reference links.
• The lync command. init, append, verify, merge, view, serve, and sync.
• The relay. A small server that passes events between people and stores each room as a plain .lync file you can open with any lync tool. You can also ask a running relay for its status: for each room, how many people are connected and how many writes are still waiting to reach disk.

Durable sync, and how we proved it

The point of sync is that what one person sees saved is really saved. This change makes that true and proves it. Three fixes and one proof.

The relay used to treat sending an event and saving an event as the same thing. If a disk write failed, the relay could still tell a reconnecting client it was up to date, and the client would skip the event forever. The relay now tracks which log it is on, and a client that reconnects after a restart starts over from a safe point instead of trusting a stale marker.

The receiving side used to record how far it had gotten before it had actually stored the event. Now it stores the event first, checks that the store succeeded, and only then records progress. A failed store freezes progress and reports the failure out loud instead of losing the event.

The relay used to keep a line only in memory if its disk write failed and the disk later healed without a restart. That line would stay off disk until the next restart. Now the relay retries the write on the next activity, so its saved log catches up on its own.

The proof is a test called the loss-free trial. It runs a real relay and three real clients, then disconnects a client and reconnects it, fails the relay's disk and heals it with no restart, and restarts the whole server. After each step it checks that every event anyone successfully wrote is in every client and on the relay's disk, and that every failure was reported out loud. It runs clean five times in a row with no flake. A reviewer confirmed each fix is load-bearing by removing it and watching the trial fail.

How it was checked

111 tests, green on every commit. Every code example in the README is run by CI, so a broken example fails the build. A fresh-clone script proves the install and first use. The whole change went through an adversarial panel on the package shape, a steel-on-steel debate, and repeated hostile review of the sync code, which is the code most able to lose data quietly. Each real bug it found is now a test.

The edges we looked at, and where they landed

Both loose ends from the durability work are now settled, not left hanging. The relay had no way to show its own state, so we added the read-only status method above. And we looked at one more shutdown edge, where a graceful shutdown during a disk outage leaves a line to reach disk on the next restart, and chose not to touch it. Nothing is lost in that case because every client still holds the line, and the fix would have added risk to the shutdown path that the loss-free proof depends on. It is recorded as a deliberate decision, not an oversight.

After merge

You log in to npm, then the one package publishes. After that, splice and textile move from the vendored copy to the published package.

[deepfates]

...should this be one package no?