Skip to content

Fix repair retry accounting and replay recovery - #301

Open
NeerajGodiyal wants to merge 1 commit into
Overclock-Validator:alpenglow-devfrom
NeerajGodiyal:koro/repair-replay-recovery
Open

NeerajGodiyal wants to merge 1 commit into
Overclock-Validator:alpenglow-devfrom
NeerajGodiyal:koro/repair-replay-recovery

Conversation

@NeerajGodiyal

Copy link
Copy Markdown
Collaborator

Summary

Fix several failure paths in repair, replay shutdown and saved-state recovery:

  • Correct repair reservation accounting across retries, cancellation and failed writes, and prevent local attempt IDs from wrapping after 256 attempts.
  • Return replay failures through the CLI. A fetch stall returns exit code 75 only after successful cleanup; persistence or cleanup failures prevent retry eligibility.
  • Wait for receiver and background work before closing their resources, propagate cleanup errors, and stop replay workers and queued rate-limit waits safely.
  • Accept valid saved progress before the first durable fold without a false corruption error, and retry state persistence after a failed post-fold save.

This does not change restart policies or include the separate logger lifecycle changes in #267.

Testing

  • All ten affected packages, repeated regression tests, race checks and full-tree go vet passed.
  • Native Linux A/B tests used the actual CLI with copied snapshot/accounts data:
    • Fetch stall: upstream exited 0; the fix exited 75 after cleanup.
    • State-save failure: upstream exited 0; the fix exited 1.
    • Valid saved-state reopening: upstream rejected it; the fix resumed.
    • Graceful shutdown completed with exit 0.
  • All four stall/save-failure runs executed the same 3,023-transaction block and produced the same bank hash.

The full repository suite still has three failures reproduced on unchanged upstream: missing conformance vectors, a genesis archive decoding error, and a network-dependent RPC test under network isolation.

These checks cover the failure and recovery paths above, not a long-running voting soak.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant