cardano-testnet: Add `--nodes` flag for per-node binary configuration by palas · Pull Request #6559 · IntersectMBO/cardano-node

palas · 2026-05-06T01:59:06Z

Description

Adds a --nodes CLI flag to cardano-testnet that allows specifying the role (SPO or relay) and optionally a custom cardano-node binary for each node in the testnet. This enables running testnets with mixed node versions, which is needed for testing compatibility across node releases.

The new flag coexists with the existing --num-pool-nodes via <|> in the parser, so either can be used:

--nodes spo,spo:node-bin=/path/to/custom-node,relay,relay
--num-pool-nodes 3   # unchanged, still works

At creation time, custom binaries are validated by running --version and the result is recorded in a YAML env file (node-data/nodeN/env). At runtime, the env file is read to resolve the binary, falling back to the default resolution when no env file exists.

Base PR: #6563 (enforces SPOs come first and splits node list into SPO and relay)

Checklist

Commit sequence broadly makes sense and commits have useful messages
New tests are added if needed and existing tests are updated. These may include:
- golden tests
- property tests
- roundtrip tests
- integration tests
  See Running tests for more details
Any changes are noted in the CHANGELOG.md for affected package
The version bounds in .cabal files are updated
CI passes. See note on CI. The following CI checks are required:
- Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
- Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
- Code builds on Linux, MacOS and Windows for ghc-9.6 and ghc-9.12
Self-reviewed the diff

carbolymer

Exception handling is most important here

…ndently

Co-authored-by: Mateusz Galazyn <228866+carbolymer@users.noreply.github.com>

Jimbo4350 · 2026-05-08T19:55:12Z

  Create a sandbox for Cardano testnet

 Available options:
+  --nodes SPEC[,SPEC...]   Comma-separated node specifications. SPO nodes must


My question would be is do we expect more options to be specified beyond what is added today? If so then we may be better off specifying a file.

It is hard to tell, I imagine there must be things you could potentially want to specify per node, but then there is also the possibility of creating the env and modifying before executing. The functionality that I was thinking about that seems would be convenient would be to have multipliers, like 3x:spo,4x:relay:node-bin=<...>.

Copilot

Pull request overview

Adds per-node configuration support to cardano-testnet via a new --nodes CLI flag, allowing each node to be designated as SPO/relay and optionally pinned to a specific cardano-node binary (recorded in the generated env and reused when starting from --node-env).

Changes:

Introduces --nodes SPEC[,SPEC...] parsing (with role ordering validation and optional node-bin= path) and updates CLI help golden files.
Extends testnet node configuration types to carry an optional per-node binary path, and persists that choice into node-data/nodeN/env.
Updates node startup to optionally use a custom binary via a new procCustom helper and a startNode signature change.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/Node/Shutdown.hs	Updates test to new node options type (`NodeWithOptions`).
cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/Gov/ProposeNewConstitutionSPO.hs	Updates test to new node options type (`NodeWithOptions`).
cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/Cli/LeadershipSchedule.hs	Updates test node creation + `startNode` call to pass optional custom binary.
cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/Cli/KesPeriodInfo.hs	Updates `startNode` call to pass optional custom binary.
cardano-testnet/test/cardano-testnet-golden/files/golden/help/create-env.cli	Updates golden help output to document `--nodes`.
cardano-testnet/test/cardano-testnet-golden/files/golden/help/cardano.cli	Updates golden help output to document `--nodes`.
cardano-testnet/test/cardano-testnet-golden/files/golden/help.cli	Updates top-level golden help usage to show `--nodes` alternative.
cardano-testnet/src/Testnet/Start/Types.hs	Replaces exported node option types with `NodeWithOptions`/`TestnetNodesWithOptions` and adds `nodeBin`.
cardano-testnet/src/Testnet/Start/Cardano.hs	Writes/reads per-node env YAML for custom binaries; threads `nodeBin` into node startup.
cardano-testnet/src/Testnet/Runtime.hs	Extends `startNode` to accept an optional custom node binary.
cardano-testnet/src/Testnet/Process/RunIO.hs	Adds `procCustom` helper for running an explicitly provided binary path.
cardano-testnet/src/Parsers/Run.hs	Uses new `readNodesWithOptionsFromEnv` reader.
cardano-testnet/src/Parsers/Cardano.hs	Adds `--nodes` flag and Parsec-based `parseNodeSpecs` implementation.
cardano-testnet/src/Cardano/Testnet.hs	Updates re-exports to the renamed node options API.
cardano-testnet/changelog.d/20260506_035740_palas_testnet_specify_node_bin_per_node.md	Adds changelog entry for `--nodes`.
cardano-testnet/cardano-testnet.cabal	Adds `parsec` dependency for `--nodes` parsing.
cardano-node/src/Cardano/Node/Testnet/Paths.hs	Adds `defaultNodeEnvFile` path helper.
cardano-node-chairman/test/Spec/Chairman/Cardano.hs	Updates chairman test to use the new default node options value.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -517,12 +530,48 @@ readNodeOptionsFromEnv envDir = do
  when (null spoFlags) $
    throwString "No SPO node directories found in environment"


+readNodeBinFromEnvFile :: (HasCallStack, MonadIO m) => FilePath -> m (Maybe FilePath)
+readNodeBinFromEnvFile envFile = runMaybeT $ do
+  guard =<< liftIOAnnotated (IO.doesFileExist envFile)
+  NodeEnv{nodeBinary} <- either failParse pure =<< liftIOAnnotated (Yaml.decodeFileEither envFile)


  , UpdateTimestamps(..)
  , TestnetOnChainParams(..)
  , mainnetParamsRequest
-  , TestnetNodeOptions(..)
-  , NodeOptions(..)
-  , cardanoDefaultTestnetNodeOptions
+  , TestnetNodesWithOptions(..)
+  , NodeWithOptions(..)
+  , cardanoDefaultTestnetNodesWithOptions


+-- | Parse a @--nodes@ argument string into 'TestnetNodesWithOptions'.
+parseNodeSpecs :: String -> Either Parsec.ParseError TestnetNodesWithOptions
+parseNodeSpecs = parse (nodeSpecsParser <* eof) "Error parsing node specifications"
+  where
+    nodeSpecsParser :: Parsec.Parsec String () TestnetNodesWithOptions
+    nodeSpecsParser = do
+      specs <- nodeSpec `sepBy1` char ','
+      let (spos, relays) = span (\(role, _) -> role == Spo) specs
+      unless (all (\(role, _) -> role == Relay) relays) $
+        fail "SPO nodes must come before relay nodes. Example: --nodes spo,spo,relay,relay"
+      case map snd spos of
+        [] -> fail "Need at least one SPO node to produce blocks."
+        (s:ss) -> pure $ TestnetNodesWithOptions
+          { optSpoNodes = s :| ss
+          , optRelayNodes = map snd relays
+          }
+
+    nodeSpec :: Parsec.Parsec String () (NodeRole, NodeWithOptions)
+    nodeSpec = do
+      role <- nodeRole
+      bin <- optional $ char ':' *> nodeBinKV
+      pure (role, NodeWithOptions bin [])
+
+    nodeRole :: Parsec.Parsec String () NodeRole
+    nodeRole =
+          Spo <$ try (string "spo" <* notFollowedBy (noneOf ",:\"\\"))
+      <|> Relay <$ try (string "relay" <* notFollowedBy (noneOf ",:\"\\"))
+      <?> "node role (\"spo\" or \"relay\")"
+
+    nodeBinKV :: Parsec.Parsec String () FilePath
+    nodeBinKV = string "node-bin=" *> (quotedPath <|> unquotedPath) <?> "\"node-bin=<path>\", where <path> is the path to the node binary, optionally quoted if it contains special characters"
+
+    quotedPath :: Parsec.Parsec String () FilePath
+    quotedPath = char '"' *> Parsec.many quotedChar <* char '"'
+      where
+        quotedChar = try (char '\\' *> (char '"' <|> char '\\')) <|> noneOf "\""
+
+    unquotedPath :: Parsec.Parsec String () FilePath
+    unquotedPath = many1 (noneOf ",:\"\\")
+
+data NodeRole = Spo | Relay deriving Eq


Jimbo4350

LGTM; suggested some minor changes. Will have another look after changes/discussion.

Jimbo4350 · 2026-05-11T13:47:23Z

+    pNodes = OA.option readNodeSpecs
+      (   OA.long "nodes"
+      <>  OA.help "Comma-separated node specifications. SPO nodes must come before relay nodes. \
+                   \Each spec is a role (spo or relay) optionally followed by :node-bin=<path>. \


This needs to exist in documentation somewhere it can't only be in the help text.

Jimbo4350 · 2026-05-11T14:09:17Z

+      specs <- nodeSpec `sepBy1` char ','
+      let (spos, relays) = span (\(role, _) -> role == Spo) specs
+      unless (all (\(role, _) -> role == Relay) relays) $
+        fail "SPO nodes must come before relay nodes. Example: --nodes spo,spo,relay,relay"


Why? Does the order actually matter or is it a matter of the presence of SPOs? Can you also separate the parsing and validation logic here?

Jimbo4350 · 2026-05-11T14:10:33Z

+parseNodeSpecs :: String -> Either Parsec.ParseError TestnetNodesWithOptions
+parseNodeSpecs = parse (nodeSpecsParser <* eof) "Error parsing node specifications"
+  where
+    nodeSpecsParser :: Parsec.Parsec String () TestnetNodesWithOptions


Can you write a basic property test for this that confirms we get the expected number of SPOs and relays?

Jimbo4350 · 2026-05-11T14:16:19Z

+      bin <- readNodeBinFromEnvFile (envDir </> defaultNodeEnvFile i)
+      pure $ NodeWithOptions bin []
+
+data NodeEnv = NodeEnv


Missing haddocks

Jimbo4350 · 2026-05-11T14:17:41Z

-newtype NodeOptions = NodeOptions
-  { nodeExtraCliArgs :: [String] -- ^ Extra CLI arguments passed to @cardano-node run@
+data NodeWithOptions = NodeWithOptions
+  { nodeBin :: Maybe FilePath -- ^ Path to the @cardano-node@ binary to use for running this node. 'Nothing' uses the default resolution mechanism.


Can you expand on this a little. What's a potential purpose of specifying the node binary and roughly what is the default resolution mechanism?

palas requested a review from a team as a code owner May 6, 2026 01:59

palas self-assigned this May 6, 2026

palas force-pushed the testnet-specify-node-bin-per-node branch 2 times, most recently from f0bcf8d to 88a49cf Compare May 6, 2026 23:54

palas mentioned this pull request May 6, 2026

cardano-testnet: Enforce SPOs come first and split node list into SPO and relay #6563

Merged

9 tasks

palas changed the base branch from master to split-nodes-list-into-spo-and-relay May 6, 2026 23:56

palas changed the title ~~cardano-testnet: Add --nodes flag for per-node binary configuration~~ cardano-testnet: Add --nodes flag for per-node binary configuration May 6, 2026

palas force-pushed the split-nodes-list-into-spo-and-relay branch from 9ca8af4 to 77fa7d8 Compare May 7, 2026 00:06

palas force-pushed the testnet-specify-node-bin-per-node branch from 88a49cf to 450f24b Compare May 7, 2026 00:06