Skip to content

feat(ops): E2E testnet integration scaffold (COW-1064)#44

Open
brunota20 wants to merge 1 commit into
feat/multi-chain-isolation-cow-1073from
feat/e2e-testnet-cow-1064
Open

feat(ops): E2E testnet integration scaffold (COW-1064)#44
brunota20 wants to merge 1 commit into
feat/multi-chain-isolation-cow-1073from
feat/e2e-testnet-cow-1064

Conversation

@brunota20

Copy link
Copy Markdown
Collaborator

Summary

Wires the engine config, runbook, and report template for the
4-6 h E2E run on Sepolia with all 5 modules dispatched
simultaneously. This PR is scaffold-only — the run itself is
operator-driven (real Sepolia RPC keys, funded test EOA,
on-chain order submissions, 4-6 h wall clock) and cannot be
exercised from CI.

Stacks on #43 (COW-1073 multi-chain isolation). Linear: COW-1064.

What ships

File What
engine.e2e.toml Unified Sepolia config: 5 modules + [engine.metrics] enabled = true on 127.0.0.1:9100 + separate state_dir = ./data/e2e.
justfile build-e2e (reuses build-m2 + build-m3) and run-e2e targets. JSON logs (no --pretty-logs) for jq mining.
docs/operations/e2e-testnet-runbook.md Operator runbook: RPC selection, on-chain prep, boot expectations, the 3 on-chain triggers that satisfy the per-module terminal-markers acceptance, metrics capture, red flags, report-filing flow.
docs/operations/e2e-reports/e2e-report.template.md Empty report skeleton with sections for metadata, chain coverage, submitted on-chain actions, per-module terminal markers, error-count deltas from /metrics, anomalies, acceptance checklist, sign-off (the gating decision for COW-1031).

What this PR does NOT do

The 4-6 h run itself is operator-driven and requires:

  1. Real Sepolia RPC (Alchemy / Infura WS) — public node
    throttles eth_subscribe and eth_call under sustained
    load (≥ 4 calls/block from price-alert + balance-tracker ×2 +
    stop-loss + 1/registered-order from twap-monitor).
  2. Funded test EOA + ComposableCoW Safe access on Sepolia —
    twap-monitor's only path to a submitted: marker.
  3. EthFlow swap from that EOA on Sepolia —
    ethflow-watcher's only path to a submitted: marker.
  4. setPreSignature + sell-token allowance from the stop-loss
    owner EOA — stop-loss's only path to a submitted:
    marker that is not a typed TransferSimulationFailed warn.
  5. 4-6 h wall clock + metrics-start.txt / metrics-end.txt
    capture + the report's section 8 sign-off, which gates
    COW-1031 (7-day soak).

The runbook is explicit about each step; the report template's
acceptance checklist makes the closure conditions for COW-1064
unambiguous.

Smoke-validation done before commit

Booted engine.e2e.toml end-to-end against live Sepolia for
60+ s. Observed:

INFO supervisor ready modules=5 chains=1
INFO log subscription open module=twap-monitor chain_id=11155111
INFO block subscription open chain_id=11155111
INFO log subscription open module=ethflow-watcher chain_id=11155111
DEBUG dispatch ok module=twap-monitor    block_number=11088259 latency_ms=1
WARN  price-alert: TRIGGERED answer=168110190000 threshold=250000000000 (Below)
DEBUG dispatch ok module=price-alert     block_number=11088259 latency_ms=130
DEBUG dispatch ok module=balance-tracker block_number=11088259 latency_ms=271
WARN  stop-loss retry on next block (0): orderbook error
      (TransferSimulationFailed): sell token cannot be transferred
DEBUG dispatch ok module=stop-loss       block_number=11088259 latency_ms=1802

5/5 modules init successfully, both log subscriptions + the
block subscription open, dispatch loop ticks against real
Sepolia blocks, every block-subscribing module dispatches every
block, and the real RPC + Chainlink decode + cow-api submit
path is exercised inside seconds. Acceptance markers for
ethflow-watcher + the on-chain submitted: paths require the
operator's on-chain actions.

Workspace impact

  • No production-code changes (pure ops scaffolding).
  • cargo fmt --all --check clean.
  • All 5 module artefacts build cleanly to the names
    engine.e2e.toml references (twap_monitor.wasm,
    ethflow_watcher.wasm, price_alert.wasm,
    balance_tracker.wasm, stop_loss.wasm).

AI assistance disclosure

Claude (Opus 4.7, 1M-context) drafted the runbook, report
template, and engine.e2e.toml from the existing M2 + M3
runbook shapes. I reviewed each file, ran the 60 s smoke boot
against live Sepolia (output above), and confirmed the
acceptance bar's "scaffold half" is complete. The "run half" I
will execute manually before pushing the COW-1064 report PR.

Wires the engine config, runbook, and report template for the
4-6 h E2E run on Sepolia with all 5 modules dispatched
simultaneously. This is the integration step between unit-test
coverage (MockHost, per-module strategy tests) and the COW-1031
7-day soak; the soak validates stability, this validates
correctness in a live dispatch context.

## What this PR ships (scaffold, not run)

- `engine.e2e.toml` — unified Sepolia config loading all 5
  modules (twap-monitor + ethflow-watcher + price-alert +
  balance-tracker + stop-loss), separate `state_dir = ./data/e2e`,
  Prometheus `/metrics` enabled on 127.0.0.1:9100. Operator
  swaps in their Alchemy/Infura WS URL before launching the run.
- `justfile` targets `build-e2e` + `run-e2e`. `build-e2e`
  reuses `build-m2 + build-m3` so the 5 wasm artefacts are
  produced in one go; `run-e2e` boots the engine pointed at
  `engine.e2e.toml` (no `--pretty-logs` so production-shape
  JSON logs are emitted, ready for jq mining).
- `docs/operations/e2e-testnet-runbook.md` — full operator
  runbook mirroring the M2 + M3 shape. Sections cover RPC
  selection, on-chain prep (test EOA + Safe + stop-loss
  pre-sign), boot sequence + expected log shape, the three
  on-chain triggers that satisfy the per-module terminal-state
  markers, metrics capture, red flags to watch, and report
  filing. Acceptance bar from COW-1064 reproduced verbatim.
- `docs/operations/e2e-reports/e2e-report.template.md` — empty
  report skeleton operator copies to
  `e2e-report-YYYY-MM-DD.md` at the start of each run and
  fills in as the run progresses. Sections: run metadata,
  chain coverage, on-chain actions submitted, per-module
  terminal-state markers, error-count deltas from Prometheus,
  anomalies, acceptance checklist, sign-off.

## What this PR explicitly does NOT do

The 4-6 h run itself is operator-driven and cannot be
exercised from CI:

1. Real Sepolia RPC keys (rate-limited public node will not
   survive a multi-hour run with 4+ eth_call per block).
2. Funded test EOA + ComposableCoW Safe access to submit a
   real conditional order (twap-monitor's only path to a
   `submitted:` marker).
3. EthFlow swap from a real EOA on Sepolia
   (ethflow-watcher's only path to a `submitted:` marker).
4. `setPreSignature` + sell-token allowance from the stop-loss
   `owner` EOA (stop-loss's only path to a `submitted:`
   marker that is not a typed `TransferSimulationFailed` warn).
5. 4-6 h wall clock + metrics-start.txt / metrics-end.txt
   capture.

The runbook is unambiguous about what each step requires; the
report template's section 8 is the gating sign-off for
COW-1031 (7-day soak).

## Smoke-validation done before commit

Booted `engine.e2e.toml` end-to-end against live Sepolia for
60+ s (kill -INT-style early shutdown):

```
INFO supervisor ready modules=5 chains=1
INFO log subscription open module=twap-monitor chain_id=11155111
INFO block subscription open chain_id=11155111
INFO log subscription open module=ethflow-watcher chain_id=11155111
DEBUG dispatch ok module=twap-monitor    block_number=11088259 latency_ms=1
WARN  price-alert: TRIGGERED answer=168110190000 threshold=250000000000 (Below)
DEBUG dispatch ok module=balance-tracker block_number=11088259 latency_ms=271
WARN  stop-loss retry on next block (0): orderbook error
      (TransferSimulationFailed): sell token cannot be transferred
DEBUG dispatch ok module=stop-loss       block_number=11088259 latency_ms=1802
```

This proves: 5/5 modules init successfully, both log
subscriptions + the block subscription open, the dispatch loop
ticks against real Sepolia blocks, every module that has a
block subscription dispatches on every block, and the real
RPC + Chainlink decode + cow-api submit path is exercised
inside seconds. The remaining acceptance bar (terminal
markers on twap-monitor + ethflow-watcher, 1500-block run,
0-ERROR supervisor log) only the operator can produce.

## Workspace impact

- No production-code changes (this is pure ops scaffolding).
- `cargo fmt --all --check` clean.
- `cargo build --target wasm32-wasip2 --release` produces all
  5 module artefacts named exactly as `engine.e2e.toml`
  references them (twap_monitor.wasm, ethflow_watcher.wasm,
  price_alert.wasm, balance_tracker.wasm, stop_loss.wasm).

Linear: COW-1064. Tenth M4 issue landed; stacks on #43 (COW-1073).
@linear-code

linear-code Bot commented Jun 18, 2026

Copy link
Copy Markdown

COW-1064

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant