Skip to content

feat(deployer): add compact.toml patterns and per-deploy options #193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
0xisk merged 9 commits into from
Sep 24, 2026
Merged

feat(deployer): add compact.toml patterns and per-deploy options #193

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
276841a
build(deployer): move ledger-v9 to 1.0.0-rc.5
0xisk Sep 24, 2026
48c2313
docs(deployer): split out the 0.2.0 changelog
0xisk Sep 24, 2026
3b187f5
feat(deployer): match [contracts] keys by pattern
0xisk Sep 24, 2026
8bc24eb
feat(deployer): take per-deploy values from code
0xisk Sep 24, 2026
13e5080
feat(deployer): remember the accepted fragment size
0xisk Sep 24, 2026
6e07bbc
test(integration): deploy Fragmented through a pattern
0xisk Sep 24, 2026
33ef64b
docs(deployer): document patterns and new options
0xisk Sep 24, 2026
a472fc0
docs(deployer): date the changelog for 0.3.0
0xisk Sep 24, 2026
07487d8
docs(deployer): add the npm ledger pin
0xisk Sep 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
"vitest": "^4.1.9"
},
"resolutions": {
"@midnightntwrk/ledger-v9": "1.0.0-rc.3",
"@midnightntwrk/ledger-v9": "1.0.0-rc.5",
"glob": "^11.0.0",
"undici": "^6.24.0",
"uuid": "^13.0.0"
Expand Down
27 changes: 21 additions & 6 deletions packages/deployer/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,37 @@ file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased
## 0.3.0 (2026-09-24)

### Added

- Fragmented deploy for contracts too large for one block, driven by `--circuits-per-tx` / `[contracts.X].circuits_per_tx`, with resume from chain state and a strict verify before `confirmed`. See the README's "Large contracts" section
- `DeployResult` gains `fragments` and `circuits`; both appear in `--json`, alongside a failed fragmented deploy's `address`, `circuitsOnChain`, `circuitsPending`, and `txId`
- Exit codes `7` (`BlockLimitError`: the deploy tx was refused as too large at the configured or minimum fragment size) and `8` (`FragmentDeployError`: fragmented deploy incomplete and resumable)
- `[contracts]` pattern keys, so one entry covers many contracts: a name pattern (`"Mock*"`) or a directory pattern (`"**/test/mocks/*"`) matched under the new `[profile].src_dir`. `{name}` expands to the contract name and `artifact` defaults to it. See the README's "Patterns" section (#193)
- `initialPrivateState` on `DeployerOptions` and `RunDeployOptions` sets the initial private state in code and overrides `[contracts.X].init_private_state`. A `private_state_id` with no initial private state from either source fails with exit 2, and so does an in-code state with no `private_state_id` (#193)
- `witnesses` on `DeployerOptions` and `RunDeployOptions` sets the witness implementations in code and overrides `[contracts.X].witnesses` (#193)
- `record` on `DeployerOptions` and `RunDeployOptions`, default `true`. With `false` a deploy reads and writes no deployments ledger, so it never resumes and `deploymentsFile` is `''` (#193)
- With no fragment budget, a later deploy of the same artifact on the same network starts at the size an earlier halving in the process settled on, instead of every circuit (#193)

### Changed

- **Breaking:** the deploy stack moves to Ledger v9. Compile artifacts with `compact compile +0.34.0`; an older one fails at submit with `Version mismatch`. Versions in the README's "Supported stack" (#192)
- **Breaking:** signing and verifying keys are tagged `{ tag, value }`, not hex. `SigningKey.ledgerKey` gives that form and `ChainSnapshot.committee` is `SignatureVerifyingKey[]`. `signing_key_file` on disk is unchanged (#192)
- yarn and pnpm need one resolution, `@midnightntwrk/ledger-v9`, down from six (#192)
- `proof_server = "auto"` boots `midnightntwrk/proof-server:9.0.0-rc.6` (#192)
- `DeploymentRecord` gains a `partial` member. A single-tx deploy still writes only `pending` then `confirmed`, so an exhaustive `switch` on `status` needs a new arm only if it reads records from a split deploy
- A deploy that fits one transaction is unchanged: same transaction, same `pending` then `confirmed` records, no chain read and no maintenance transaction
- `@midnightntwrk/ledger-v9` moves from 1.0.0-rc.3 to 1.0.0-rc.5. npm now needs the pin too, as `overrides`. Update the yarn / pnpm resolution to match (#193)
- **Breaking:** `CompactConfigData.contracts` holds partial entries, which `CompactConfig.contract(name)` merges and validates. `listContracts()` returns exact keys only; `listPatterns()` returns the pattern keys (#193)

## 0.2.0 (2026-09-15)

### Added

- Fragmented deploy for contracts too large for one block, driven by `--circuits-per-tx` / `[contracts.X].circuits_per_tx`, with resume from chain state and a strict verify before `confirmed`. See the README's "Large contracts" section (#174)
- `DeployResult` gains `fragments` and `circuits`; both appear in `--json`, alongside a failed fragmented deploy's `address`, `circuitsOnChain`, `circuitsPending`, and `txId` (#174)
- Exit codes `7` (`BlockLimitError`: the deploy tx was refused as too large at the configured or minimum fragment size) and `8` (`FragmentDeployError`: fragmented deploy incomplete and resumable) (#174)

### Changed

- `DeploymentRecord` gains a `partial` member. A single-tx deploy still writes only `pending` then `confirmed`, so an exhaustive `switch` on `status` needs a new arm only if it reads records from a split deploy (#174)
- A deploy that fits one transaction is unchanged: same transaction, same `pending` then `confirmed` records, no chain read and no maintenance transaction (#174)

## 0.1.0 (2026-09-09)

Expand Down
56 changes: 50 additions & 6 deletions packages/deployer/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The deployer pins one Midnight stack, and artifacts have to be compiled against
| Component | Version |
|---|---|
| `@midnight-ntwrk/compact-runtime` | 0.19.0 |
| `@midnightntwrk/ledger-v9` | 1.0.0-rc.3 |
| `@midnightntwrk/ledger-v9` | 1.0.0-rc.5 |
| `@midnight-ntwrk/compact-js` | 2.5.5-rc.8 |
| `@midnight-ntwrk/midnight-js-*` | 5.0.0-beta.7 |
| `@midnight-ntwrk/testkit-js` | 5.0.0-beta.7 |
Expand All @@ -51,15 +51,15 @@ Compile with `compact compile +0.34.0`. An older artifact fails at submit with `

The ledger and the wallet SDK sit under the `@midnightntwrk` scope, no hyphen. Everything else keeps `@midnight-ntwrk`.

yarn and pnpm need one pin, because `compact-js` declares the ledger as a range and a second ledger copy breaks every deploy:
Every package manager needs one pin, because the stack's packages ask for different ledger versions and a second ledger copy breaks every deploy. With npm:

```json
"resolutions": {
"@midnightntwrk/ledger-v9": "1.0.0-rc.3"
"overrides": {
"@midnightntwrk/ledger-v9": "1.0.0-rc.5"
}
```

(`pnpm.overrides` for pnpm.) Nothing else in the stack floats.
yarn takes the same entry under `resolutions`, and pnpm under `pnpm.overrides`. Nothing else in the stack floats.

## CLI

Expand Down Expand Up @@ -96,6 +96,7 @@ The node rejects a deploy tx above the per-block extrinsic limit with `1010: Inv

- Left unset, the deployer submits the largest batch it can and halves on a refusal, down to a single circuit. There is no pre-flight weight check.
- Set at or above the circuit count, it pins a single-tx deploy and disables halving: a refusal is exit 7, never a split.
- A size found by halving is remembered in memory: later deploys of the same artifact on the same network, in the same process, start there. A set budget and a resume ignore it.
- Fragments are ordered by sorted circuit name, so a rerun rebuilds the same plan.
- One `deploy()` call does the deploy, every insert, and a byte-for-byte check of every on-chain key against the artifact. Only that check writes `confirmed`.
- Between fragment 0 landing and the last insert the contract is live with a subset of its circuits, ordered by name rather than by dependency. Keep every circuit safe to call alone, or hold traffic until the deploy confirms.
Expand Down Expand Up @@ -228,6 +229,28 @@ circuits_per_tx = 8

`"auto"` needs Docker and boots the `proof-server.yml` shipped in this package, pinning `midnightntwrk/proof-server:9.0.0-rc.6`. To boot a different image, put your own `proof-server.yml` in the directory you run `compact-deploy` from; it wins over the packaged one.

### Patterns

One `[contracts]` entry can cover many contracts. A key is one of:

- `Token`: exact, that contract only.
- `"Mock*"`: a name pattern (holds `*`, `?`, `[` or `{`), matched against the contract name.
- `"**/test/mocks/*"`: a directory pattern (holds `/`), matched against the contract's source path under `[profile].src_dir`, `.compact` included. The source of `<name>` is the unique `<src_dir>/**/<name>.compact`, skipping `node_modules` and dot-directories. A second match is an error.

Every matching pattern applies in file order, a later one overriding an earlier one field by field. The exact entry applies last. `{name}` in any string value expands to the contract name, and `artifact` defaults to it. `runDeploy(Contract)` searches exact keys only.

```toml
[profile]
src_dir = "contracts/src"

[contracts."**/test/mocks/*"]
signing_key_file = "./deploy/{name}.signingkey"

# Inherits signing_key_file from the pattern above.
[contracts.MockToken]
args = ["MyToken", "MTK", 18]
```

## Keystore format

An Ethereum V3 JSON keystore (scrypt + AES-128-CTR) tagged `version: "midnight-1"`, so other tooling does not mis-read it as an Ethereum key. The encrypted secret is a 32-byte Midnight wallet seed (hex).
Expand All @@ -246,7 +269,7 @@ An Ethereum V3 JSON keystore (scrypt + AES-128-CTR) tagged `version: "midnight-1

6. **The root `@midnightntwrk/ledger-v9` resolution is load-bearing.** `compact-js` declares it as a range, so without the pin yarn nests a second ledger copy and deploys fail. Do not drop it on a bump.

7. **`@midnight-ntwrk/compact-runtime` resolves to two copies**, `0.19.0-rc.0` under `compact-js` and `midnight-js-protocol` against `0.19.0` everywhere else. The integration suite deploys on that tree, including the pruned constructor path through `compact-js`. Forcing one copy is untested.
7. **`@midnight-ntwrk/compact-runtime` resolves to two copies**, `0.19.0-rc.0` under `compact-js` and `midnight-js-protocol` against `0.19.0` everywhere else. The integration suite deploys on that tree, including the pruned constructor path through `compact-js`. Forcing one `0.19.0` copy with a resolution also deploys, the split path included.

## Programmatic API

Expand Down Expand Up @@ -293,3 +316,24 @@ console.log(result.address);
```

Every option has a `process.argv` default (`--network`, `--config`, `--dry-run`, …), so the same script works with flags at the call site. On failure `runDeploy` sets `process.exitCode` and rethrows; it never calls `process.exit`, so catch it if you want that code to reach the shell.

Three options suit a caller that deploys many contracts from code, such as a test harness. `Deployer.prepare` and `runDeploy` both take them, with no argv flag:

- `initialPrivateState` and `witnesses` override `[contracts.X].init_private_state` and `[contracts.X].witnesses`. A `private_state_id` and an initial private state come together: `prepare` throws `ConfigError` when one lacks the other.
- `record: false` skips the deployments ledger. Nothing is read or written, so every deploy is fresh and `deploymentsFile` is `''`.

```ts
import { Deployer } from "@openzeppelin/compact-deployer/deployer";

await using deployer = await Deployer.prepare({
contract: "MockToken",
network: "local",
logger,
walletProvider, // one wallet shared across deploys
privateStateProvider, // e.g. testkit-js inMemoryPrivateStateProvider()
initialPrivateState: { secretKey },
witnesses,
record: false,
});
const { address } = await deployer.deploy();
```
2 changes: 1 addition & 1 deletion packages/deployer/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@
"@midnight-ntwrk/midnight-js-types": "5.0.0-beta.7",
"@midnight-ntwrk/midnight-js-utils": "5.0.0-beta.7",
"@midnight-ntwrk/testkit-js": "5.0.0-beta.7",
"@midnightntwrk/ledger-v9": "1.0.0-rc.3",
"@midnightntwrk/ledger-v9": "1.0.0-rc.5",

@pepebndc pepebndc Sep 24, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker for 0.3.0 (claude, codex): npm consumers get two ledger copies.

This line pins @midnightntwrk/ledger-v9 to exactly 1.0.0-rc.5. The deployer's own dependencies pin exactly 1.0.0-rc.3: for example @midnightntwrk/wallet-sdk-facade@5.0.0-beta.2, the other wallet-sdk-* packages, and midnight-js-protocol. In this repo the root resolutions forces rc.5 on all of them, so CI passes. npm does not read resolutions. An npm resolution of the published stack gives one top-level rc.5 copy plus eight nested rc.3 copies. With 0.2.0 it gave one copy.

The deployer passes its own ledger objects into the other copy, for example ZswapSecretKeys.fromSeed and DustSecretKey.fromSeed in wallet/handler.ts:226-227, and the deploy txs that go through midnight-js. The README says it at line 54: "a second ledger copy breaks every deploy". The README gives the pin for yarn and pnpm, but not for npm, although the install line at README.md:30 uses npm i.

Suggested fix, one of:

  1. Add the npm form to the README next to the yarn and pnpm pins, "overrides": { "@midnightntwrk/ledger-v9": "1.0.0-rc.5" }, and say that npm and bun users need it too.
  2. Add a fail-fast check in Deployer.prepare: resolve @midnightntwrk/ledger-v9/package.json from the deployer and from @midnightntwrk/wallet-sdk-facade, and throw a ConfigError that names both versions and the override when the paths differ.
  3. Keep rc.3 until the midnight-js and wallet-sdk releases pin rc.5, and let compact-contracts override it locally.

Options 1 and 2 together are the safest.

@0xisk 0xisk Sep 24, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree — fixed in 07487d8.

"@midnightntwrk/wallet-sdk-address-format": "4.0.0-beta.2",
"@midnightntwrk/wallet-sdk-capabilities": "4.0.0-beta.2",
"@midnightntwrk/wallet-sdk-dust-wallet": "5.0.0-beta.2",
Expand Down
Loading
Loading