Skip to content

Add deterministic live supervisor#39

Merged
phroi merged 11 commits into
masterfrom
review/live-supervisor-pr3
May 20, 2026
Merged

Add deterministic live supervisor#39
phroi merged 11 commits into
masterfrom
review/live-supervisor-pr3

Conversation

@phroi
Copy link
Copy Markdown
Member

@phroi phroi commented May 20, 2026

Why

Live testnet runs need a deterministic supervisor that can exercise bounded bot/tester cycles, classify known outcomes, and leave secret-safe artifacts for operator review without embedding repair or LLM behavior in the funded actor path.

Changes

  • Add @ickb/supervisor with bounded scenario planning, outcome classification, coverage ledgers, incident bundles, and dry-run fixtures.
  • Add an external summary-only supervisor loop for repeated bounded invocations.
  • Wire pnpm live:supervisor and pnpm live:supervisor:loop, include supervisor tests in Vitest wiring, ignore logs/live-supervisor/, and document the deterministic external-repair boundary.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a deterministic live testnet supervisor app and a supporting loop script to manage bounded bot and tester stress cycles. The supervisor classifies outcomes, records artifacts, and generates incident reports for unknown or unsafe results. Feedback focuses on improving the robustness of integer parsing in both the supervisor and the loop script by using 'BigInt' to prevent precision loss during safe integer validation. Additionally, it is recommended to preserve error context in the loop script by using the 'cause' property when re-throwing JSON parsing errors.

Comment thread apps/supervisor/src/index.ts Outdated
Comment thread scripts/ickb-supervisor-loop.mjs Outdated
Comment thread scripts/ickb-supervisor-loop.mjs Outdated
Comment thread scripts/ickb-supervisor-loop.mjs Outdated
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new deterministic live testnet supervisor app and a corresponding loop script to manage bounded bot and tester stress cycles. The implementation includes logic for scenario planning, outcome classification, and artifact generation. The reviewer feedback suggests enhancing the error reporting for process spawning by including spawnError when the exit status is unavailable. Further improvements are recommended to avoid precision loss during integer validation by utilizing BigInt and to increase the reliability of workspace artifact checks by using canonical paths instead of relying on hoisted node_modules layouts.

Comment thread apps/supervisor/src/index.ts
Comment thread apps/supervisor/src/index.ts
Comment thread apps/supervisor/src/index.ts
Comment thread apps/supervisor/src/index.ts Outdated
Comment thread apps/supervisor/src/index.ts
Comment thread scripts/ickb-supervisor-loop.mjs Outdated
Comment thread scripts/ickb-supervisor-loop.mjs Outdated
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the apps/supervisor package, a deterministic live testnet supervisor designed for bounded stress cycles of iCKB bots and testers. The implementation includes a core supervisor app, a companion loop script for automated repeated runs, and extensive unit tests. Feedback was provided regarding the summarySignature function in the loop script, where sorting the skipReasons array is recommended to ensure a deterministic signature for detecting progress.

Comment thread scripts/ickb-supervisor-loop.mjs Outdated
@phroi
Copy link
Copy Markdown
Member Author

phroi commented May 20, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the @ickb/supervisor application, a deterministic live testnet supervisor designed for bounded stress cycles and incident classification. The changes include the core supervisor logic, test suites, and integration with the repository's build and loop scripts. I have identified two issues in the documentation where specific tester scenarios were either duplicated or incorrectly named, which should be corrected for clarity.

Comment thread apps/supervisor/README.md Outdated
Comment thread apps/supervisor/README.md Outdated
@phroi
Copy link
Copy Markdown
Member Author

phroi commented May 20, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new deterministic live testnet supervisor app and a corresponding loop script to automate stress testing and incident reporting. The supervisor classifies bot and tester outcomes, while the loop script manages repeated runs until stability or specific transaction outcomes are achieved. Additionally, the tester app was updated to apply CKB reserve checks to all transactions. Review feedback suggests removing duplicate entries in the documentation and centralizing shared constants by importing them from workspace build locations to improve maintainability.

Comment thread apps/supervisor/README.md Outdated
Comment thread scripts/ickb-supervisor-loop.mjs Outdated
@phroi
Copy link
Copy Markdown
Member Author

phroi commented May 20, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new supervisor application for deterministic testnet validation, featuring scenario planning, process supervision, and automated artifact classification. It also includes a watcher script for running supervisor loops. Key feedback includes making the CKB reserve check in the tester conditional to prevent deadlocks during account replenishment, adding null checks for child process streams to improve robustness, and utilizing the in operator for more idiomatic TypeScript type narrowing in error-handling utilities.

Comment thread apps/tester/src/index.ts Outdated
Comment thread apps/supervisor/src/index.ts
Comment thread apps/supervisor/src/index.ts
Comment thread apps/supervisor/src/index.ts
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new apps/supervisor application, which provides a deterministic live testnet supervisor for managing bot and tester stress cycles. The changes include the implementation of the supervisor logic, a corresponding CLI, and a watcher script for automated loops. The review identified a duplicate entry for all-ickb-limit-order in the TesterScenario type definition, which should be removed for consistency.

Comment thread apps/supervisor/src/index.ts Outdated
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new deterministic testnet supervisor app and a corresponding loop script to manage bounded stress cycles for bot and tester processes. The supervisor handles outcome classification, artifact logging, and incident reporting. Key feedback focuses on a regression in the tester's asset reserve logic, where an unconditional check could prevent account replenishment, and a recommendation to cap timeout values in the supervisor to avoid Node.js setTimeout overflow issues.

Comment thread apps/tester/src/index.ts Outdated
Comment thread apps/supervisor/src/index.ts Outdated
@phroi
Copy link
Copy Markdown
Member Author

phroi commented May 20, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the iCKB Live Supervisor, a deterministic operator application designed for funded testnet validation. The changes include the core supervisor implementation, a watcher script for automated loops, extensive unit tests, and detailed documentation. The supervisor manages bounded execution of bot and tester processes, classifies outcomes based on evidence, and generates incident reports for unknown or unsafe states. Review feedback correctly identifies redundant listings of the all-ickb-limit-order scenario within the documentation, type definitions, and CLI validation logic, which should be consolidated for better clarity and consistency.

Comment thread apps/supervisor/README.md Outdated
Comment thread apps/supervisor/src/index.ts Outdated
@phroi
Copy link
Copy Markdown
Member Author

phroi commented May 20, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new apps/supervisor package, a deterministic operator app designed for funded testnet validation. The supervisor manages bounded bot and tester processes, classifies their outcomes, and generates incident bundles for unknown or unsafe results. The PR also includes a new ickb-supervisor-loop.mjs script to automate repeated supervisor runs until specific coverage goals or transactions are observed. Documentation, extensive tests, and repository-level scripts have been updated to support this new infrastructure. I have no feedback to provide as there were no review comments to assess.

@phroi
Copy link
Copy Markdown
Member Author

phroi commented May 20, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new apps/supervisor package, a deterministic operator application designed for funded testnet validation through bounded bot and tester process execution. The implementation includes logic for outcome classification, incident bundling, and coverage-based scenario planning, supported by a new supervisor loop script for automated testing cycles. Related updates include the removal of legacy aliases in the tester app and the deletion of the archived pool snapshot codec in the SDK. I have no feedback to provide.

@phroi
Copy link
Copy Markdown
Member Author

phroi commented May 20, 2026

LGTM

Phroi %261

@phroi phroi merged commit 43ee8d8 into master May 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant