Skip to content

Pearl th-e979ac: unified backoff+jitter retry on the dolt read-only lock-flap#267

Merged
brentrager merged 2 commits into
mainfrom
th-e979ac-lock-flap-retry
Jul 25, 2026
Merged

Pearl th-e979ac: unified backoff+jitter retry on the dolt read-only lock-flap#267
brentrager merged 2 commits into
mainfrom
th-e979ac-lock-flap-retry

Conversation

@brentrager

Copy link
Copy Markdown
Contributor

Problem

The transient Error 1105: cannot update manifest: database is read only — a concurrent writer (usually another agent's push) briefly holding the single-writer Dolt lock — failed fast: CLI mode ran auto_doctor, found no orphan, and returned the error immediately; server mode force-respawned once and gave up. Agents (and the th-mail flow) hand-retried it ad-hoc.

Fix — one funnel, all commands (answers "does this apply to all dolt commands?")

Consolidates the two scattered one-shot self-heals (run_cli doctor-retry + run_with_self_heal respawn-retry) into a single retry_on_lock_flap(op, recover) helper that every dolt write funnels through:

  1. Recover once on the first flap — the mode's cheap local self-heal (CLI: reap an orphaned smooth-dolt serve; server: force-respawn a wedged child) — clears a genuinely stuck local holder.
  2. Backoff + jitter, retry until success or budget — waits out a live peer's push (SMOOTH_DOLT_LOCK_RETRY_BUDGET_SECS, default 30s).

Non-lock errors (syntax, transport, corruption) propagate immediately; budget exhaustion returns the read-only error with a th pearls doctor --reap hint rather than hanging.

Now covers th pearls create/update/close, th msg send/reply, th pearls push, and memory writes alike — the flap is waited out transparently everywhere. The th-mail skill's hand-rolled retry guidance is dropped since the store self-heals. No new dependency (jitter from SystemTime).

Dedup (answers "fewer hooks / dedup of efforts")

Before: two independent inline retries + a skill-level manual retry, none with real backoff. After: one helper, one backoff/jitter/budget policy, called from both modes. The transport-error and sync-timeout paths stay separate (distinct failure modes).

Tests

5 new (lock_flap_tests): retries-then-succeeds + recover-exactly-once, non-lock propagates without recover, success-first never recovers, jitter within [0.5,1.5)×, constants sane. Full crate suite: 179 passed; clippy/fmt clean.

🤖 Generated with Claude Code

…ock-flap

Consolidates the two scattered one-shot self-heals (run_cli doctor-retry,
run_with_self_heal respawn-retry) into one retry_on_lock_flap(op, recover)
helper: recover once (clear local orphan / force-respawn), then exponential
backoff + jitter until success or a bounded budget (default 30s, env
override). Every dolt write funnels through it — pearls create/update/close,
th msg send/reply, push, memory — so a concurrent writer's transient
"database is read only" is waited out transparently everywhere instead of
failing fast in CLI mode. Non-lock errors propagate immediately. Drops the
th-mail skill's hand-rolled retry. No new dep (jitter from SystemTime).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TwUg6nyMqXHzFDsibzTKmp
@changeset-bot

changeset-bot Bot commented Jul 25, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 62ce8cd

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@smooai/smooth Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@brentrager
brentrager enabled auto-merge (squash) July 25, 2026 19:54
@brentrager
brentrager merged commit da9afac into main Jul 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant