Skip to content

Shared process lifecycle: ordered start, staged stop, failure fan-in, readiness and drain #1

Closed

Description

@patdowney

Problem

Each Dioad server process starts a set of services (HTTP servers, workers, publishers, a store), fans their failures into one channel, reports readiness, drains, and then stops them in order within a deadline. Four separate implementations exist today:

Where Type What only it has
dioad/connect server internal/cmd/server/service.Manager rolls back already-started services on a start failure; skips the drain when a failure caused the stop; Service{Start,Stop,Health,Name} plus Failer and Liveness
dioad/connect agent internal/cmd/agent.ServiceRegistry stop budget per service; StopStarted
dioad/connect-control internal/lifecycle.Orchestrator (ShutdownStages) + internal/server.Readiness staged shutdown; readiness with named, cached, degraded checks and a start gate; Drain(delay)
dioad/connect-control services hand-written Run per service the serverErr channel and shutdown copied three times, and already drifting

connect's two Failer interfaces are identical apart from their doc comments. An operator sees different /health/ready and drain semantics depending on which server type they run, and every new server picks one of these implementations or writes a fifth.

Proposal

Add a lifecycle package next to NewOperationalServer, for example dioad/net/lifecycle or dioad/net/http/app, that combines the best parts:

  • Interface: Service{Start(ctx) error; Stop(ctx) error}, with optional Name, Health, Live and Failed() <-chan error. These line up with http: add Start/Failed/Live lifecycle to Server so consumers stop re-implementing the serve wrapper net#342's proposed dch.Server methods, so a *dch.Server is a Service with no adapter.
  • Starting: in registration order, rolling back on failure.
  • Stopping: in stages (a stage is a set of services stopped concurrently), each stage with an optional budget, all within one overall deadline. Errors are collected with errors.Join.
  • Failures: fanned into one channel; a failure-triggered stop skips the drain.
  • Readiness: a gate with named checks, an optional degraded state (the process stays ready but reports the check), result caching with a TTL, and a start gate. It registers as a health source on the operational server, so /health/ready means the same thing everywhere.
  • Drain: a configurable delay that reports not-ready before listeners close.
  • Constructor: New(mgmt OperationalConfig, primary *dch.Server, ...), which also builds the management listener.

Check functions stay in callers. For example, connect-control's nosqlite store check stays in connect-control, so dioad/net gains no storage dependency.

Related

Source: dioad/connect-control doc/claude-review-dioad-server-platform.md, findings P4 and P5.

Acceptance criteria: documentation by example

  • func Example tests:
    • ExampleApp: ordered start, a failing service, rollback, and the stop order printed.
    • ExampleReadiness: a degraded check and drain.
  • examples/ gains a non-HTTP worker daemon that uses the lifecycle with no listener, plus an API server with an operational listener that shows readiness and drain.
  • examples/basic-http-server and the other net examples move off hand-written signal and serve loops onto the lifecycle (see http: add Start/Failed/Live lifecycle to Server so consumers stop re-implementing the serve wrapper net#342).
  • Examples build in CI. Any examples/ program that runs a long-lived process is started in CI, checked for readiness, sent SIGTERM, and must exit 0.
  • Once the cross-repo reference server exists, it uses this capability in place of its own copy.

Activity

  1. transferred this issue fromdioad/neton Nov 4, 2026
  2. patdowney commented on Nov 14, 2026

    @patdowney
    ContributorAuthor

    The package from this issue is implemented and released as v0.1.0 (PR #2, merged).

    Covered

    • App: ordered start with rollback; staged stop with per-stage budgets inside one deadline, errors joined with errors.Join; failure fan-in into one channel; a failure-triggered stop skips the drain; a configurable drain delay.
    • Readiness: start gate, named checks with a cache TTL, a degraded state that keeps the process ready, and a drain. It has the Ready method that dioad/net's AddHealthSource looks for, so it is a health source without this module importing net.
    • func Example tests: ExampleApp, ExampleApp_Stop and ExampleReadiness.

    Differences from the proposal

    • The stop method is Shutdown, not Stop, so that dioad/net's *http.Server is a Service with no adapter. Failed, Live and Ready are optional. A service's name is given at Add.
    • There is no New(mgmt OperationalConfig, primary *dch.Server, ...) constructor: it would make this module import dioad/net while net's examples import this module. A convenience constructor can live in net if wanted.

    Acceptance criteria in dioad/net

    The "once the cross-repo reference server exists, it uses this capability" item is not done here.

    This can be closed once dioad/net#362 is merged.

  3. patdowney commented on Nov 15, 2026

    @patdowney
    ContributorAuthor

    Delivered in dioad/net#362 (released in v0.72.0), which runs the net examples and Server on dioad/lifecycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions