Allow rules to execute ordered command lists (#550)

.gitignore

@@ -4,6 +4,8 @@ target/
 *.swp
 .crush/
 .claude/
+.vtcode/
+vtcode.toml
 .memdb/
 .grepai/
 build.ninja
@@ -17,5 +19,4 @@ __pycache__/
 .pytest_cache/
 .typos-oxendict-base.json
 .typos-oxendict-base.toml
-.vtcode/
 *.swo

CHANGELOG.md

@@ -22,6 +22,13 @@
   reproduces the existing behaviour, so `run_ninja` and `run_ninja_tool` keep
   their signatures and no embedder needs to change
   ([#490](https://github.com/leynos/netsuke/issues/490))
+- Accept a non-empty ordered list of commands for a rule or target `command`
+  recipe, executed as a single fail-fast `&&` shell chain, so the build stops
+  at the first non-zero exit; an empty command list is rejected at parse time,
+  and entries with multiple background jobs or unsupported `exec` structures
+  are rejected during Ninja generation as `MultipleBackgroundJobs` or
+  `UnsupportedCommandListExec`
+  ([#550](https://github.com/leynos/netsuke/issues/550))
 
 ### Changed
 

Cargo.toml

@@ -117,6 +117,7 @@ glob = "0.3.3"
 hashbrown = "0.17.1"
 walkdir = "2.5"
 metrics = "0.24.6"
+monotony = "0.1.0"
 mockable = "3.0"
 tracing = "0.1"
 tracing-subscriber = { version = "0.3", features = ["fmt"] }
@@ -153,6 +154,7 @@ predicates = "3"
 # global recorder; constrained to the family that pairs with metrics 0.24.
 metrics-util = { version = "0.20", features = ["debugging"] }
 mockable = { version = "3.0", features = ["mock"] }
+monotony = { version = "0.1.0", features = ["test-util"] }
 serial_test = "3"
 mockall = "0.11"
 camino = "1.2.0"

docs/developers-guide.md

@@ -196,6 +196,84 @@ they are per-invocation arguments tagged `#[serde(skip)]` on
 would silently change the artefact destination — a footgun the design avoids by
 construction.
 
+## Command and recipe lowering
+
+Command recipes use the `StringOrList` AST type. A scalar command remains one
+shell-text value; a YAML sequence is an ordered list of entries. The same
+recipe path handles commands declared on reusable rules, direct targets, and
+actions. Manifest deserialization rejects an empty command list. Code that
+constructs the IR directly must also reject both `StringOrList::Empty` and an
+empty `StringOrList::List(Vec::new())` during Ninja generation rather than
+emitting an unusable rule.
+
+The lowering stages have deliberately separate responsibilities:
+
+- `src/manifest/render.rs` renders a scalar or each list entry independently.
+  Every entry sees the same cloned recipe context, including target variables
+  and delayed `ins`/`outs` markers. A rendering error for a list includes its
+  one-based entry position.
+- `src/ir/from_manifest_support.rs` prepares one shell-quoted input/output
+  binding set for the recipe, then interpolates every scalar or list entry with
+  that set. `{{ ins }}` and `{{ outs }}` markers and standalone `$in` and
+  `$out` tokens are resolved per entry; tokens inside backticks are preserved.
+  The resulting action contains ordinary command text and no Ninja
+  placeholders.
+- `src/ninja_gen.rs` emits a scalar command unchanged. For a list, it puts
+  each entry in a brace group and joins the groups with `&&`. Each group uses
+  `eval` with a shell-quoted entry payload. This keeps an inline comment or a
+  trailing control operator such as `&` inside the entry from consuming the
+  generated group terminator. Braces run in the current shell, not a
+  subshell, so directory changes, environment assignments, and shell
+  variables can carry from one entry to the next. The `&&` chain remains
+  fail-fast. Each entry may start at most one background job; the generated
+  wrapper waits for that job before it evaluates a later entry. Ninja
+  generation rejects entries that start more than one background job. It also
+  rejects entries whose nested `eval` payload makes the background-job count
+  dynamic, because the wrapper cannot safely determine which jobs to wait for.
+  A direct simple `exec`, optionally prefixed by shell assignments, is
+  evaluated in a retaining subshell so its success or failure remains visible
+  to the wrapper; a successful `exec` ends the remaining chain. Structured or
+  nested `exec` forms are rejected during Ninja generation because the wrapper
+  cannot supervise them without changing their shell semantics.
+- `src/runner/process` forwards the command's output and recognizes the
+  bounded `netsuke command-list failure: action HASH, entry M` marker. A failed
+  list therefore retains the original exit status while adding the fixed-width
+  hashed action fingerprint and one-based entry index to the Ninja failure
+  error.
+
+Failure attribution is private to Ninja process execution:
+`FailureAttributionWriter` parses only Ninja's stderr. Because Ninja relays a
+failed subcommand's stderr on its own stdout, build runs retain only a fixed
+512-byte stdout tail and use its parsed marker only after a non-zero exit.
+Ordinary child stdout streams forward directly and must not use this tail.
+
+The lowest-layer POSIX shell-word quoting used for input/output paths during IR
+lowering is `shell_quote::QuoteRefExt::quoted(Sh)`. It performs minimal,
+fragmented shell quoting, which is appropriate for a literal shell word but not
+for the command-list `eval` payload. That renderer requires a canonical
+single-quoted payload so existing generated Ninja list text remains
+byte-for-byte stable, and the delimiter/boundary tests continue to hold. Keep
+that quoting in the deliberately local `shell_single_quote` function; it is
+not a general-purpose helper. Neither quoting path is the platform-specific
+`src/stdlib/command/quote.rs` implementation behind the `command.quote`
+template wrapper, which must retain its `cmd.exe` quoting behaviour on Windows.
+
+Attributed list failures emit the bounded tracing fields
+`command_list_action` (a fixed-width action fingerprint) and
+`command_list_entry` (the one-based entry index), plus the matching
+`command_list_failure` marker. The process boundary records
+`netsuke_ninja_command_list_failures_total` and
+`netsuke_ninja_command_list_failure_duration_seconds`, with an `outcome`
+label of `failure`. Elapsed failure duration is measured through the injected
+`monotony::MonotonicClock`; production uses `StdMonotonicClock`, while tests use
+deterministic test clocks. These diagnostics and metrics contain no command
+text.
+
+Changes to this pipeline must preserve the scalar/list distinction, per-entry
+rendering, current-shell state sharing, and failure attribution. The focused
+rendering, lowering, Ninja-generation, and real-Ninja integration tests are
+the behavioural contract for these boundaries.
+
 ## Package and target naming
 
 The crates.io package is `netsuke-build`; the library target, the binary

docs/netsuke-design.md

@@ -223,7 +223,7 @@ erDiagram
         bool always
     }
     RECIPE {
-        string command
+        StringOrList command
         string script
         StringOrList rule
     }
@@ -245,18 +245,28 @@ Each entry in the `rules` list is a mapping that defines a reusable action.
 
 - `name`: A unique string identifier for the rule.
 
-- `command`: A single command string to be executed. It may include the
-  placeholders `{{ ins }}` and `{{ outs }}` to represent input and output
-  files. Netsuke expands these placeholders to space-separated lists of file
-  paths quoted for POSIX `/bin/sh` using the
+- `command`: A command string, or a non-empty ordered list of command strings,
+  to be executed. `StringOrList` is also used for direct target and action
+  commands, so the rule and target forms have the same scalar/list semantics.
+  Each entry may include the placeholders `{{ ins }}` and `{{ outs }}`. Jinja
+  renders a scalar or each list entry separately with the same recipe context;
+  the placeholders are delayed until IR lowering, then replaced in every entry
+  with space-separated, POSIX-shell-quoted input and output paths using the
   [`shell-quote`](https://docs.rs/shell-quote/latest/shell_quote/) crate (Sh
-  mode) before hashing the action. The IR stores the fully expanded command;
-  Ninja executes this text verbatim. After interpolation, the command must be
-  parsable by [shlex](https://docs.rs/shlex/latest/shlex/) (POSIX mode).
-  Automatic shell escaping applies only where the schema has enough structure
-  to identify argument boundaries. Plain command strings remain shell text;
-  authors should use structured recipes or explicit quoting helpers for
-  arbitrary variables.
+  mode) before hashing the action. Standalone `$in` and `$out` tokens are
+  resolved at the same boundary, while tokens inside backticks are preserved.
+  A scalar command is emitted unchanged. A list is lowered to brace groups
+  that evaluate each entry through a shell-quoted `eval` payload and are joined
+  by `&&`. The groups run in declaration order in one shell process and stop
+  at the first non-zero exit, so working directory, environment, and shell
+  variables carry forward. The `eval` boundary keeps an entry's inline
+  comments or trailing control operators from consuming the generated group
+  terminator. A failed entry emits a bounded action/entry marker for the
+  runner to include in the failure diagnostic. The resulting command must be
+  parsable by [shlex](https://docs.rs/shlex/latest/shlex/) (POSIX mode). An
+  empty command list is rejected during manifest deserialization. Plain command
+  strings remain shell text; authors should use structured recipes or explicit
+  quoting helpers for arbitrary variables.
 
 - `script`: A multi-line script declared with the YAML `|` block style. The
   entire block is passed to an interpreter. If the first line begins with `#!`
@@ -326,7 +336,10 @@ rule:
   - clean-up
 ```
 
-- `command`: A single command string to run directly for this target.
+- `command`: A command string or non-empty ordered list of command strings to
+  run directly for this target. Direct target lists follow the same per-entry
+  Jinja rendering, delayed `ins`/`outs` interpolation, and shell lowering as
+  rule lists.
 
 - `script`: A multi-line script passed to the interpreter. When present, it is
   defined using the YAML `|` block style.
@@ -711,7 +724,7 @@ pub struct Rule {
 /// A union of execution styles for both rules and targets.
 #[serde(untagged)]
 pub enum Recipe {
-    Command { command: String },
+    Command { command: StringOrList },
     Script { script: String },
     Rule { rule: StringOrList },
     // FUTURE: planned Recipe::Exec extension; not present in src/ast.rs yet.
@@ -787,9 +800,11 @@ pub enum StringOrList {
 }
 ```
 
-*Note: The* `StringOrList` *enum with* `#[serde(untagged)]` *provides the
-flexibility for users to specify single sources, dependencies, and rule names
-as a simple string and multiple as a list, enhancing user-friendliness.*
+*Note: The* `StringOrList` *enum with* `#[serde(untagged)]` *preserves whether
+the manifest supplied one string or an ordered list. The same type represents
+command recipes, sources, dependencies, order-only dependencies, and rule
+selectors; command lists are executed in order, while path-like fields are
+interpreted only at the manifest-to-IR boundary.*
 
 `StringOrList` owns the conversions that only need to know its own shape:
 `map_each` applies a function to every contained string, and `to_string_vec`
@@ -1956,17 +1971,19 @@ This transformation involves several steps:
    Current behaviour:
 
    For each expanded target, resolve the referenced rule template, merge
-   rule-level and target-level execution metadata, interpolate its command with
-   the target's input and output paths, and register the resulting `ir::Action`
-   in the `actions` map. Actions are hashed on the fully resolved recipe and
-   file set, so identical rule templates yield distinct actions when their
-   paths differ. Create a corresponding `ir::BuildEdge` linking the target to
-   the action identifier and transfer the `phony` and `always` flags. `sources`
-   are lowered into the edge's explicit input list so recipe interpolation and
-   Ninja `$in` see only material inputs. `deps` are lowered into a separate
-   `implicit_deps` list, which maps to Ninja's implicit dependency syntax (`|`)
-   so Ninja orders and rebuilds them without exposing them as recipe arguments;
-   `order_only_deps` remains separate and maps to Ninja's `||` class.
+   rule-level and target-level execution metadata, and interpolate every
+   command entry with the target's input and output paths. Direct target and
+   action commands use the same path. Register the resulting scalar or ordered
+   `StringOrList` recipe in the `ir::Action` map. Actions are hashed on the
+   fully resolved recipe and file set, so identical rule templates yield
+   distinct actions when their paths differ. Create a corresponding
+   `ir::BuildEdge` linking the target to the action identifier and transfer the
+   `phony` and `always` flags. `sources` are lowered into the edge's explicit
+   input list so recipe interpolation and Ninja `$in` see only material inputs.
+   `deps` are lowered into a separate `implicit_deps` list, which maps to Ninja's
+   implicit dependency syntax (`|`) so Ninja orders and rebuilds them without
+   exposing them as recipe arguments; `order_only_deps` remains separate and
+   maps to Ninja's `||` class.
 
    FUTURE:
 
@@ -2010,9 +2027,12 @@ structures to the Ninja file syntax.
    be written at the top of the file (e.g., `msvc_deps_prefix` for Windows
 
 2. **Write Rules:** Iterate through the `graph.actions` map. For each
-   `ir::Action`, write a corresponding Ninja `rule` statement. The input and
-   output lists stored in the action replace the `ins` and `outs` placeholders.
-   These lists are then rewritten as Ninja's `$in` and `$out`.
+   `ir::Action`, write a corresponding Ninja `rule` statement. The IR already
+   contains ordinary command text: its input and output paths have replaced
+   Netsuke's `ins`/`outs` and `$in`/`$out` placeholders during lowering. Scalar
+   commands are emitted as-is. List commands are emitted as the brace-group,
+   `eval`, and `&&` chain described in §2.3, including the bounded failure
+   marker for each one-based entry.
 
    When an action's `recipe` is a script, the generated rule wraps the script
    in an invocation of `/bin/sh -e -c` so that multi-line scripts execute
@@ -2184,33 +2204,21 @@ catastrophic consequences.
 For this critical task, the recommended crate is `shell-quote`.
 
 While other crates like `shlex` exist, `shell-quote` offers a more robust and
-flexible API specifically designed for this purpose.[^22] It supports quoting
-for multiple shell flavours (e.g., Bash, sh, Fish), which is vital for a
-cross-platform build tool. It also correctly handles a wide variety of input
-types, including byte strings and OS-native strings, which is essential for
-dealing with non-UTF8 file paths. The
-
-`QuoteExt` trait provided by the crate offers an ergonomic and safe method for
-building command strings by pushing quoted components into a buffer:
-`script.push_quoted(Bash, "foo bar")`.
+flexible API specifically designed for this purpose.[^22] The current lowering
+path uses its `QuoteRefExt::quoted` method with `Sh` mode, producing
+POSIX-compatible quoted path arguments before the command is hashed. `shlex`
+remains a validation parser; it does not perform the quoting.
 
 ### 6.3 Implementation Strategy
 
-The command generation logic within the `ninja_gen.rs` module must not use
-simple string formatting (like `format!`) to construct the final command
-strings. Instead, parse the Netsuke command template (e.g.,
-`{{ cc }} -c {{ ins }} -o` `{{ outs }}`) and build the final command string
-step by step. The placeholders `{{ ins }}` and `{{ outs }}` are expanded to
-space-separated lists of file paths within Netsuke itself, each path being
-shell-escaped using the `shell-quote` API. Netsuke uses the `Sh` quoting mode
-to emit POSIX-compliant single-quoted strings and scans the template for
-standalone `$in` and `$out` tokens to avoid rewriting unrelated variables.
-Substitution happens during IR generation and the fully expanded command is
-emitted to `build.ninja` unchanged. After substitution, the command is
-validated with \[`shlex`\](<https://docs.rs/shlex/latest/shlex/>) to ensure it
-parses correctly. This approach guarantees that every dynamic part of the
-command is securely quoted, albeit at the cost of deduplicating only actions
-with identical file sets.
+The command interpolation logic in `src/ir/cmd_interpolate.rs` prepares one
+quoted input/output binding set per recipe and applies it to each scalar or
+list entry. It replaces the delayed `{{ ins }}`/`{{ outs }}` markers and
+standalone `$in`/`$out` tokens outside backticks, preserving longer identifiers
+and backtick-delimited text. Unbalanced backticks or text that `shlex` cannot
+parse produce an IR error before an action is hashed. Ninja generation then
+receives fully expanded command text and is responsible only for preserving the
+scalar form or constructing the list-entry shell boundaries.
 
 ### 6.4 Automatic Security as a "Friendliness" Feature
 
@@ -2220,10 +2228,11 @@ user to trivial security vulnerabilities is fundamentally unfriendly. In many
 build systems, the burden of correct shell quoting falls on the user, an
 error-prone task that requires specialized knowledge.
 
-Netsuke's design elevates security to a core feature by making it automatic and
-transparent. The user writes a simple, unquoted command template, and Netsuke
-performs the complex and critical task of making it secure behind the scenes.
-By integrating `shell-quote` directly into the Ninja file synthesis stage,
+Netsuke's design makes identified path substitution safe by default. Netsuke
+quotes the `ins`/`outs` path values before action hashing and Ninja synthesis;
+arbitrary Jinja values and handwritten shell fragments remain the manifest
+author's responsibility. By integrating `shell-quote` into IR command
+lowering, before action hashing and Ninja file synthesis,
 Netsuke protects users from a common and dangerous class of errors by default.
 This approach embodies a deeper form of user-friendliness: one that anticipates
 and mitigates risks on the user's behalf.