Skip to content

bug: mech-storage ADMP signatures fail and documented fallback is bypassed #26

Description

@dundas

Summary

The registered mech-storage brain cannot currently send or acknowledge ADMP messages. Authenticated operations return 401 SIGNATURE_INVALID, while hub diagnostics intermittently return 500/502. This forced a local-file delivery fallback for two completed cross-brain responses and means remote delivery is not guaranteed.

There is also an integration-contract mismatch: the distributed cross-brain-message skill documents local file delivery as an always-on fallback, but the current canonical brain/brain-msg.ts sender exits immediately when ADMP delivery fails and never writes the fallback message.

Observed behavior

On 2026-08-15 from the registered Mech Storage repository:

brain-msg send ...
[admp] Failed: HTTP 401: {"error":"SIGNATURE_INVALID","message":"HTTP signature verification failed"}

brain-msg ack <message-id>
[ack] Acknowledged: <message-id>
[ack] Hub ack skipped: HTTP 401

Additional diagnostics during the same window:

  • brain-msg admp-status reported /api/stats as HTTP 500.
  • An authenticated lookup of /api/agents/mech-storage returned HTTP 502.
  • A later inbox pull reached the hub and reported no new messages, so this was not a total connectivity outage.
  • The local Mech Storage ADMP entry still has its Ed25519 secret/public key pair and original registration timestamp (2026-03-31T01:41:12.473Z). No key material was printed or transmitted during diagnosis.

Impact

  • A completed AgentBootup work-order response could not be delivered over ADMP.
  • A HelloConvo capability response could not be delivered over ADMP.
  • Both were delivered through the documented shared local inbox layout and retained in the Mech Storage outbox, so no same-machine message was lost.
  • A remote-only recipient would not be guaranteed to receive either response.
  • Locally acknowledged source messages may requeue because hub acknowledgment also fails signature verification.

Retained local delivery IDs:

  • msg-1786812235000-casclosebootup
  • msg-1786812235001-hclimithelloconvo

Expected behavior

  1. A locally registered agent whose credential has not intentionally changed can send and acknowledge messages successfully.
  2. If the hub's public key and local private key drift, there is a documented, authenticated recovery path that does not require an already-valid agent signature.
  3. Hub diagnostics return stable, actionable errors instead of intermittent 500/502 responses.
  4. The distributed skill and canonical sender agree on fallback behavior: either restore durable file fallback on ADMP failure or remove the claim and provide an equivalent guaranteed-delivery mechanism.

Acceptance criteria

  • Compare the hub's active mech-storage public key/version with the registered local key without exposing either secret.
  • Repair or rotate the credential through an authorized recovery path.
  • Prove one signed send to bootup and one signed hub acknowledgment from mech-storage.
  • Confirm locally acknowledged messages do not requeue after the visibility window.
  • Diagnose the /api/stats 500 and agent-lookup 502 observed during the incident.
  • Reconcile and test the documented local-file fallback contract in the canonical cross-brain sender, or explicitly track that client-side fix in its owning repository.

Security notes

  • Do not request or paste the Ed25519 private key, ADMP master key, API keys, or .env contents into this issue.
  • The fallback messages contain only synthetic test identifiers and capability guidance—no credentials or customer data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions