Skip to content

fix(rs-dapi,rs-sdk): generalize retryable post-broadcast result-wait errors #4137

Description

@Claudius-Maginificent

Expected Behavior

After broadcast() succeeds, failures from the separate
wait_for_response() phase must preserve that phase information.

For infrastructure or observation failures such as UNAVAILABLE or
DEADLINE_EXCEEDED:

  1. Retry only wait_for_response() against another DAPI node.
  2. Never rebroadcast the state transition.
  3. If wait retries are exhausted, return a typed accepted-but-unconfirmed error
    containing the state transition ID.
  4. Reserve rejection errors for structured Platform consensus verdicts.

Conceptually:

broadcast succeeded
    ↓
wait failed with retryable infrastructure error
    ↓
retry wait only
    ↓
confirmed result | consensus rejection | accepted but unconfirmed

Current Behavior

The Rust SDK already separates the phases internally:

self.broadcast(sdk, settings).await?;
self.wait_for_response_with_metadata(sdk, settings).await

However, rs-dapi converts infrastructure failures from
wait_for_state_transition_result_impl into a successful gRPC response
containing StateTransitionBroadcastError.

For example, when the DAPI node's Tenderdash WebSocket is disconnected:

DapiError::Unavailable("Tenderdash is not available")

is converted into:

StateTransitionBroadcastError {
    code: 14,
    message: "Tenderdash is not available",
    data: [],
}

The SDK decodes the empty consensus data as:

StateTransitionBroadcastError {
    code: 14,
    message: "Tenderdash is not available",
    cause: None,
}

This has two consequences:

  • StateTransitionBroadcastError is not retryable, so the existing wait retry
    loop stops instead of trying another DAPI node.
  • Generic SDK consumers cannot distinguish an actual Platform rejection from a
    successful broadcast whose result could not be observed.

Production Observation

A valid identity credit withdrawal produced this sequence:

  1. broadcast: request succeeded
  2. broadcast: completed successfully
  3. broadcast_and_wait: step 2 - waiting for response
  4. Wait returned code 14, "Tenderdash is not available", cause: None
  5. The SDK returned StateTransitionBroadcastError
  6. The withdrawal subsequently executed and delivered the Core funds

A client interpreting this as rejection can invite an unsafe retry of an action
that may already have completed.

Prior Art

The accepted-but-unconfirmed model already exists in Platform, but only in
specific consumers:

This issue proposes moving the safety model already proven in
platform-wallet into the generic rs-dapi/rs-sdk contract.

Possible Solution

rs-dapi

Infrastructure and observation failures should use the gRPC status channel:

match self.wait_for_state_transition_result_impl(request).await {
    Ok(response) => Ok(response),
    Err(error) => Err(error.to_status()),
}

Actual transaction execution errors can continue to be returned as
WaitForStateTransitionResultResponse.result.error.

This preserves the distinction already present inside
wait_for_state_transition_result_impl:

  • Transaction execution result → Ok(response)
  • Tenderdash/WebSocket/timeout/service failure → Err(DapiError)

rs-sdk

Introduce a phase-specific retryable wait error, for example:

Error::StateTransitionResultUnavailable {
    source: DapiClientError,
}

The existing wait-only retry loop should retry this error against another DAPI
node.

If retries are exhausted after broadcast succeeded, broadcast_and_wait
should return a semantic wrapper such as:

Error::WaitForStateTransitionResultFailedAfterBroadcast {
    state_transition_id: [u8; 32],
    source: Box<Error>,
}

or:

Error::BroadcastAcceptedResultUnconfirmed {
    state_transition_id: [u8; 32],
    source: Box<Error>,
}

This final combined-operation error should not imply that rebroadcasting or
reconstructing the state transition is safe.

Compatibility with older DAPI nodes

When an older node returns a response-embedded
StateTransitionBroadcastError:

  • cause: Some(consensus_error) → definitive Platform rejection
  • cause: None → ambiguous wait failure; retry the wait when appropriate and
    otherwise return accepted-but-unconfirmed

This matches the safety classification already implemented by #3863.

Steps to Reproduce

  1. Submit a valid state transition through broadcast_and_wait.
  2. Allow broadcastStateTransition to succeed.
  3. Make the selected DAPI node's Tenderdash WebSocket unavailable before
    waitForStateTransitionResult.
  4. Observe a response-embedded error with code 14 and empty consensus data.
  5. Observe that the SDK does not retry the result wait.
  6. Observe that the combined call returns StateTransitionBroadcastError even
    if the transition later executes.

Suggested Tests

  • An rs-dapi WebSocket-unavailable failure returns gRPC UNAVAILABLE, not a
    response-embedded state-transition error.
  • UNAVAILABLE retries only wait_for_response() against another node.
  • The broadcast request is called exactly once during wait retries.
  • A structured cause: Some(consensus_error) remains a non-retryable
    rejection.
  • A cause-less response from an older DAPI node is treated as ambiguous.
  • Exhausted wait retries preserve the state transition ID and return an
    accepted-but-unconfirmed error.
  • Convenience APIs propagate the phase-specific error without constructing a
    new state transition or consuming another nonce.

Context

This affects withdrawals, transfers, document operations, token operations,
and other paid or state-changing calls using the generic
broadcast_and_wait path.

The shielded wallet already handles this safely by splitting the phases.
Generic SDK consumers should receive the same guarantees without independently
reimplementing the classification.

Your Environment

  • Observed with Platform commit
    d18020f526e2a8eb1d1e868b436a7a9735795abb
  • Current rs-dapi and rs-sdk sources retain the relevant behavior
  • Mainnet identity credit withdrawal
  • Client: Dash Evo Tool 1.0.0-weekly.20260715

🤖 Co-authored by Claudius the Magnificent AI Agent

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions