From 757f395ea1a4fef7d0e2f93cb458618109263e19 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Wed, 6 May 2026 10:31:22 +0200 Subject: [PATCH 01/20] empty From a5a03a921330b4f474e0668b8a370d54d558c898 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Wed, 6 May 2026 10:41:10 +0200 Subject: [PATCH 02/20] docs(rust): add design doc and phased progress tracker for Rust port MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DESIGN.md captures workspace layout, foundational decisions (byte-string model via bstr, sync+threads, clap, cap-std, no telemetry), and a 7-phase migration plan. PROGRESS.md is the resumption point — checkbox-tracked tasks per phase so an interrupted session can pick up where the last left off. Binary is named rshell-rs during cohabitation with the Go impl. --- rust/DESIGN.md | 284 +++++++++++++++++++++++++++++++++++++++++++++++ rust/PROGRESS.md | 233 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 517 insertions(+) create mode 100644 rust/DESIGN.md create mode 100644 rust/PROGRESS.md diff --git a/rust/DESIGN.md b/rust/DESIGN.md new file mode 100644 index 000000000..9db7db7f2 --- /dev/null +++ b/rust/DESIGN.md @@ -0,0 +1,284 @@ +# rshell Rust port — design doc + +Status: **draft / exploratory**. This document captures the plan for a 1:1 port +of `rshell` from Go to Rust, living in the `rust/` subdirectory of this branch +(`alex/rust`) alongside the existing Go implementation until parity is reached. + +## 1. Goals and non-goals + +### Goals + +- Match the runtime behaviour of the Go implementation, as defined by the + ~2,600 YAML scenarios in `tests/scenarios/` and verified byte-for-byte + against GNU bash via `RSHELL_BASH_TEST=1`. +- Reduce binary size and peak RSS, and improve startup time. No specific + numeric targets; best effort, measured at the bake-off stage. +- Preserve the security posture: sandboxed file access, no real binary + execution, bounded resource use, deterministic output. +- Keep the YAML scenario format unchanged; rewrite only the test runner. + +### Non-goals + +- Telemetry. The Go implementation has an optional `with_telemetry` build tag + that wires `pkg/fleet/installer/telemetry`. The Rust port drops this feature + entirely; there is no `rshell-telemetry` crate. If telemetry is needed + later, it can be added via the `tracing` ecosystem. +- Real external binary execution. Every command resolves to a builtin. PATH + lookup and `fork`/`exec` of arbitrary binaries are out of scope. This may + change in the future but is not part of this port. +- Bug-for-bug compatibility with `mvdan.cc/sh/v3`. Where the YAML harness + + bash say one thing and `mvdan/sh` says another, follow bash. +- Publishing crates to crates.io. Internal use only for now. + +## 2. Workspace layout + +``` +rust/ +├── Cargo.toml # virtual workspace manifest +├── rust-toolchain.toml # pinned stable Rust (1.85+, edition 2024) +├── DESIGN.md # this file +├── README.md +└── crates/ + ├── rshell-cli/ # binary crate; produces `rshell-rs` + ├── rshell-interp/ # runner, redirections, var scope, control flow + ├── rshell-parser/ # syntax → AST (replaces mvdan/sh/v3/syntax) + ├── rshell-expand/ # word expansion, globbing, splitting (replaces mvdan/sh/v3/expand) + ├── rshell-builtins/ # one module per builtin + ├── rshell-sandbox/ # AllowedPaths over cap-std + ├── rshell-analysis/ # 1:1 port of analysis/ symbol-allowlist verifier + └── rshell-test-runner/ # YAML scenario harness + bash comparison +``` + +The split mirrors the current Go package boundaries so individual crates can +be implemented and tested piecewise. `rshell-parser` and `rshell-expand` are +deliberately separate so they can be reused outside the interpreter (the Go +analyzer also depends on the parser). + +The binary is named `rshell-rs` during the cohabitation period to avoid +`$PATH` collisions with the Go `rshell` binary. Once the Go implementation is +removed, the binary will be renamed to `rshell`. + +## 3. Foundational design decisions + +### 3.1 String model: bytes, not UTF-8 + +Shell variables, file paths, command output, and process arguments are byte +streams in bash. Rust's `String` is UTF-8 only, which is wrong for shell +semantics: a shell value can legitimately contain invalid UTF-8 (e.g. a +filename in a non-UTF-8 locale, binary data piped through `cat`). + +**Choice:** use `bstr::BString` / `bstr::BStr` for shell values throughout the +crates. Convert to `&str` only at output boundaries when the value is known +to be UTF-8 (e.g. error messages we construct ourselves). Path arguments to +syscalls go through `OsStr` / `OsString` / `Path` / `PathBuf`. + +This is the single biggest correctness decision and it touches every crate. +Getting it right early is much cheaper than retrofitting. + +### 3.2 Concurrency: synchronous + threads, no async runtime + +Pipelines, here-docs, and command substitution need concurrent producers and +consumers. We use `std::thread` and OS pipes (`std::io::pipe` from std on +recent stable, or the `os_pipe` crate as a fallback). Rationale: + +- Smaller binary (no tokio runtime). +- Faster cold startup (no runtime init). +- Fewer dependencies. +- Pipelines are bounded fan-out (typically 2–4 stages); thread cost is + negligible compared to typical command runtime. + +If a specific feature later needs async (e.g. a network builtin), it can be +isolated behind a thread boundary rather than coloring the whole stack. + +### 3.3 CLI parsing: `clap` (derive) + +`cobra` → `clap` derive is a clean port for the top-level CLI. For the +per-builtin flag parsing currently done with `pflag`, we use the `clap` +builder API per builtin — repetitive but matches the project rule that all +flag parsing must be `pflag`-equivalent (POSIX/GNU conventions, `--` end of +flags, combined short bool flags, glued-value short flags). + +### 3.4 Regex and globbing + +- Regex: `regex` crate (RE2-derived, linear time). Matches the project rule + preferring linear-time regex engines over backtracking ones, so existing + ReDoS protections are inherited for free. +- Globs: `globset` for compiled multi-pattern matching, hand-rolled per-path + walking using `cap-std` (so glob expansion respects the sandbox). + +### 3.5 Logging + +`tracing` for diagnostic logs. No telemetry crate. + +### 3.6 Error model + +- `thiserror` for typed error enums at every public crate boundary. +- `anyhow` only inside the binary crate (`rshell-cli`) for top-level + composition. + +Builtins return a typed `BuiltinError` that the interpreter maps to an exit +code and a stderr message — never propagated as `?` past the interpreter +boundary. + +### 3.7 Sandbox + +`cap-std` provides `Dir` capability handles backed by `openat` on Unix. +Behaviour parity with Go's `os.Root` on Linux and macOS. Document the gap on +Windows (cap-std does not currently provide the same atomic guarantees there +that Go's `os.Root` does on Linux). The gap is informational only — both +implementations have weaker Windows guarantees than they have on Linux. + +### 3.8 MSRV and edition + +`rust-toolchain.toml` pins stable, edition 2024, MSRV 1.85.0. CI matrix tests +the pinned toolchain on Linux, macOS, and Windows. + +### 3.9 Dependency budget (initial) + +Core crates expected, with rationale: + +| Crate | Purpose | Justification | +|----------------|--------------------------------------------------------|--------------------------------------------------------| +| `bstr` | Byte-string types | UTF-8 disagreement with shell semantics (§3.1) | +| `clap` | CLI parsing | cobra/pflag analogue (§3.3) | +| `cap-std` | Capability-based filesystem | Sandbox (§3.7) | +| `regex` | Regex | RE2; linear time (§3.4) | +| `globset` | Glob compilation | (§3.4) | +| `thiserror` | Error derive | Error model (§3.6) | +| `anyhow` | Error composition in binary | Error model (§3.6) | +| `tracing` | Diagnostic logs | (§3.5) | +| `serde`, `serde_yaml` | YAML scenario loading | Test runner only | +| `os_pipe` | Cross-platform pipes (if std::io::pipe insufficient) | Pipelines | +| `nix` | Unix syscalls (Unix-only crates) | Sandbox internals, signal handling | +| `windows-sys` | Windows syscalls (Windows-only crates) | Sandbox internals | + +No async runtime. No HTTP client (telemetry dropped). No serialization +beyond YAML for tests. + +## 4. Builtin inventory + +29 builtins to port (excluding `break` and `continue`, which are interpreter +control-flow constructs, not external commands): + +``` +cat, cut, du, echo, exit, false, find, grep, head, help, ip, ls, ping, +printf, ps, pwd, sed, sort, ss, strings_cmd, tail, testcmd, tr, true, +uname, uniq, wc +``` + +Plus interpreter-internal: `break`, `continue`, `exit`, plus `set`, `export`, +`unset`, `local`, `readonly`, `declare`, `:` (handled in interp, not as +builtin commands). + +Notable per-builtin notes: + +- **`ip route` / `ss`**: read `/proc/net/{route,tcp,udp,...}` directly, + bypassing `AllowedPaths`. This is documented in `CLAUDE.md` as a security + design decision — the paths are non-user-controllable kernel pseudo-files. + Carry the same property forward in Rust; the Linux-only paths are + hardcoded. +- **`ping`**: uses `prometheus-community/pro-bing` in Go. Rust analogue is + `surge-ping` (raw ICMP) or shelling-out — the latter is forbidden by §1. + Confirm the ICMP capability story (raw socket on Linux requires + `cap_net_raw` or unprivileged ping via `IPPROTO_ICMP` socket on Linux). +- **`find`**: must walk via the cap-std `Dir` handle to keep sandbox + enforcement. +- **`grep` / `sed`**: use `regex` only. Document any GNU-specific extensions + the YAML harness exercises (Perl-like backreferences are unsupported by + RE2; bash itself uses ERE/BRE so this is unlikely to bite). + +## 5. Test strategy + +### 5.1 Scenario harness — port first, before any runtime work + +`rshell-test-runner` is implemented in Phase 1, **before** the parser or +interp. Initially it shells out to the existing Go `rshell` binary. This +gives us: + +- Confidence the harness works in Rust. +- An oracle for every subsequent Rust crate. +- A way to bisect regressions: at any phase, we can flip individual scenarios + to run against `rshell-rs` instead of `rshell` and see what breaks. + +Implementation: parse the YAML, fork `rshell` (or `rshell-rs`) as a +subprocess, capture stdout/stderr/exit code, compare to expectations using +the same `stdout` / `stderr` / `*_contains` / `*_windows` semantics. Reuse +the existing Docker-based bash comparison (`RSHELL_BASH_TEST=1` equivalent). + +### 5.2 Unit tests + +Each crate has its own `#[cfg(test)]` module. The parser crate tests against +a corpus extracted from the scenario YAML (round-trip every script). + +### 5.3 Cross-platform CI + +GitHub Actions matrix: Linux + macOS + Windows × stable Rust. The bash +comparison test is Linux-only (Docker-based) and matches the existing Go +arrangement. + +### 5.4 Bake-off + +Once parity is reached, run the full scenario suite against both binaries +and compare: + +- Binary size (`ls -l`). +- Cold startup (`hyperfine 'rshell -c "true"'` vs `rshell-rs`). +- Peak RSS on a representative scenario (`/usr/bin/time -v`). +- Wall time on the full scenario suite. + +These numbers go into the README and gate the eventual switch. + +## 6. Phased migration + +| Phase | Deliverable | Exit criterion | +|-------|----------------------------------------------------------------------|------------------------------------------------------------------| +| 0 | Workspace scaffolding, CI, toolchain, Makefile targets | `cargo build`, `cargo fmt`, `cargo clippy`, `cargo test` all green on all 3 OSes | +| 1 | `rshell-test-runner` shelling out to Go `rshell` | Full scenario suite runs from Rust runner with parity | +| 2 | `rshell-parser` | Round-trips every script in `tests/scenarios/` | +| 3 | `rshell-expand`, `rshell-sandbox`, minimal `rshell-interp` (pipelines, redirects, simple commands) | Echo / cat / pipelines pass under `rshell-rs` | +| 4 | All 29 builtins ported | 100% scenario parity for non-network scenarios | +| 5 | `rshell-analysis` 1:1 port | Equivalent allowlist verification on Rust crate dependencies | +| 6 | Bake-off + perf measurements | Numbers documented; decision on Go removal | + +Each phase ends with a (draft) PR. Phases are sequential because each +depends on the previous; within a phase, builtins in Phase 4 can be done in +parallel. + +### 6.1 Commit and push policy + +Every phase ends with a **new commit pushed to `alex/rust`**. No squashing +across phases — each phase is one or more commits, never amended into a +previous phase's commits, and pushed as soon as the phase's exit criterion +is met. This gives a reviewable, bisectable history of the port and lets the +draft PR show concrete progress. + +## 7. Risks and unknowns + +- **Expansion semantics depth.** `mvdan/sh/v3/expand` has years of bash + edge-case fixes baked in. Re-deriving them from the scenario corpus alone + may surface gaps. Mitigation: the bash comparison harness will catch + divergences early, but expect Phase 3 to be the largest in calendar time. +- **Windows parity.** Several builtins (`ip`, `ss`, `ping`) and the sandbox + behave differently on Windows in Go. Re-establishing the same parity in + Rust will require platform-specific code paths and may surface new gaps + that the Go version papered over. +- **`cap-std` Windows weakness.** Documented and accepted; matches the + current state of the world for `os.Root` on Windows. +- **`pro-bing` replacement.** No exact Rust equivalent. `surge-ping` may be + close enough but the YAML scenarios may encode `pro-bing`-specific output + formatting. +- **Analysis crate port.** 13k LOC of Go-specific symbol-allowlist logic. + Mechanically translatable but not a small effort. Phase 5 because it does + not block runtime parity. +- **Effort.** The full port is multi-month. This branch will carry both + implementations for the duration; Go remains the source of truth until + Phase 6 closes. + +## 8. Out of scope for this doc + +- Specific module structure inside each crate. +- Specific function signatures. +- CI workflow YAML. +- Whether to publish any of these crates externally. + +These are decided per-phase as the work lands. diff --git a/rust/PROGRESS.md b/rust/PROGRESS.md new file mode 100644 index 000000000..93907e456 --- /dev/null +++ b/rust/PROGRESS.md @@ -0,0 +1,233 @@ +# rshell Rust port — progress tracker + +This file is the **resumption point** for the Rust port. If a coding session +is interrupted, a new session reads this file (and recent `git log`) to pick +up where the previous one stopped. See `rust/DESIGN.md` for the full plan. + +## How to use this file + +1. At session start: read this file top-to-bottom, then `git log --oneline -20`. +2. Work on tasks in the **active phase**. Tick boxes (`[ ]` → `[x]`) as + tasks land. Add a one-line note under the task if anything non-obvious + came up. +3. When a phase's exit criterion is met: flip its status to **done**, + update **Active phase** to the next one, commit + push (per the + policy in `DESIGN.md` §6.1). +4. Never delete completed entries — they are the breadcrumb trail. Add new + subtasks as `[ ]` rows if scope grows. +5. If a task is blocked, mark it `[~]` and add a **Blocker:** note. + +Statuses: `[ ]` not started · `[~]` in progress / blocked · `[x]` done. + +## Snapshot + +- **Active phase:** Phase 0 — scaffolding +- **Last updated:** 2026-05-06 (initial draft) +- **Branch:** `alex/rust` +- **Binary name during cohabitation:** `rshell-rs` + +## Phase 0 — scaffolding + +**Status:** in progress +**Exit criterion:** `cargo build`, `cargo fmt --check`, `cargo clippy`, +`cargo test` all green on Linux, macOS, and Windows. + +- [x] Write `rust/DESIGN.md` +- [x] Write `rust/PROGRESS.md` (this file) +- [ ] Create `rust/Cargo.toml` (virtual workspace) +- [ ] Create `rust/rust-toolchain.toml` (stable, edition 2024, MSRV 1.85) +- [ ] Create `rust/.gitignore` (`target/`, `Cargo.lock` policy decision) +- [ ] Create empty crate skeletons under `rust/crates/`: + - [ ] `rshell-cli` (binary) + - [ ] `rshell-interp` + - [ ] `rshell-parser` + - [ ] `rshell-expand` + - [ ] `rshell-builtins` + - [ ] `rshell-sandbox` + - [ ] `rshell-analysis` + - [ ] `rshell-test-runner` +- [ ] Wire `rshell-cli` to print a placeholder `--version` so we can verify + the binary builds and runs. +- [ ] Add `rust/Makefile` (or extend top-level `Makefile`) with `make + rust-build`, `rust-test`, `rust-fmt`, `rust-lint` targets. +- [ ] Add `.github/workflows/rust.yml` (or extend existing CI) for the + Linux + macOS + Windows × stable matrix. +- [ ] Confirm `cargo fmt`, `cargo clippy -- -D warnings`, and `cargo test` + all pass locally on macOS. +- [ ] Commit + push as Phase 0. + +## Phase 1 — test runner shelling out to Go `rshell` + +**Status:** not started +**Exit criterion:** Full YAML scenario suite runs from the Rust runner +against the existing Go binary with output parity to the Go runner. + +- [ ] Read the Go test runner under `tests/` to enumerate every YAML + assertion field used in scenarios. +- [ ] Define `Scenario` / `Expect` structs in `rshell-test-runner` with + `serde(deny_unknown_fields)` over the full schema (`stdout`, + `stdout_contains`, `stdout_windows`, `stderr*`, `exit_code`, + `input.script`, `skip_assert_against_bash`, `allowed_paths`, …). +- [ ] Implement subprocess execution: spawn `rshell` (path configurable), + pipe `input.script` to stdin, capture stdout/stderr/exit code with a + timeout matching the executor (30s). +- [ ] Implement assertion comparison matching the Go runner's semantics + byte-for-byte (including Windows overrides). +- [ ] Implement bash comparison mode equivalent to `RSHELL_BASH_TEST=1`, + using Docker `debian:bookworm-slim`. +- [ ] CLI: `rshell-test-runner [--bin ] [--bash] [--filter ] `. +- [ ] Run against `tests/scenarios/` with `--bin $(which rshell)` and + verify pass-count matches the Go runner. +- [ ] Commit + push as Phase 1. + +## Phase 2 — `rshell-parser` + +**Status:** not started +**Exit criterion:** Every script in `tests/scenarios/` round-trips through +the parser without error and produces an AST. + +- [ ] Survey `mvdan.cc/sh/v3/syntax` to enumerate AST node types we need + (most are already enumerated in `analysis/symbols_interp.go`). +- [ ] Define AST types in `rshell-parser::ast` using `bstr::BString` for + every shell-value field. +- [ ] Tokenizer (handles quoting rules, here-doc opening, brace tracking). +- [ ] Parser: simple commands → pipelines → and/or → lists → + compound (if/while/until/for/case/subshell/block/function). +- [ ] Parameter expansion / arithmetic expansion / command substitution / + brace expansion at the *parse* level (preserves AST faithfully; + evaluation is `rshell-expand`'s job). +- [ ] Redirection parsing (every operator from + `mvdan/sh/v3/syntax`'s redirect operator constants — `>`, `>>`, + `<`, `<>`, `<<`, `<<-`, `<<<`, `>&`, `<&`, `&>`, `&>>`, `>|`, `|&`). +- [ ] Round-trip test: extract every `input.script` from `tests/scenarios/` + into a corpus, parse all of them, fail on any parse error. +- [ ] Commit + push as Phase 2. + +## Phase 3 — `rshell-expand`, `rshell-sandbox`, minimal `rshell-interp` + +**Status:** not started +**Exit criterion:** A representative subset of scenarios (echo, cat, pwd, +true/false, simple pipelines, simple redirects, simple variable assignment, +simple if/while/for) passes against `rshell-rs`. + +- [ ] `rshell-sandbox`: `AllowedPaths` API on top of `cap-std`. Match the + semantics of `allowedpaths/sandbox.go`. Document the Windows gap + vs. Go's `os.Root` in the crate-level rustdoc. +- [ ] `rshell-expand::word`: parameter expansion (`$x`, `${x}`, `${x:-y}`, + `${x:=y}`, `${x:?y}`, `${x:+y}`, `${#x}`, `${x#p}`, `${x##p}`, + `${x%p}`, `${x%%p}`, `${x/p/r}`, `${x//p/r}`, `${x:offset:length}`, + indirect `${!x}`, length `${#@}`, …). +- [ ] `rshell-expand::brace`: brace expansion (`{a,b,c}`, `{1..10}`, + `{1..10..2}`, `{a..z}`). +- [ ] `rshell-expand::glob`: pathname expansion via `globset` walking + through `cap-std`. Respect `nullglob`, `failglob`, `nocaseglob`, + `dotglob`, `globstar` if/where bash does. +- [ ] `rshell-expand::arith`: arithmetic expansion `$(( ... ))`. +- [ ] `rshell-expand::cmdsubst`: hooks back into the interpreter for + `$( ... )` and backticks. +- [ ] `rshell-expand::split`: word-splitting per `IFS`. +- [ ] `rshell-interp::env`: variable scopes (global, function-local, + readonly, exported), `$?`, `$#`, `$@`, `$*`, `$0`–`$9`, `$$`, `$!`. +- [ ] `rshell-interp::redir`: redirection setup using OS pipes and the + sandbox `Dir` for opening files. +- [ ] `rshell-interp::pipeline`: thread-per-stage with OS pipes; correct + exit-code propagation (last stage by default; `pipefail` option). +- [ ] `rshell-interp::control`: if / while / until / for / case / + subshell / `break` / `continue` / `return`. +- [ ] `rshell-interp::handler`: command dispatch — for now, builtins only + (no PATH lookup). Unknown commands → "command not found", exit 127. +- [ ] Wire `rshell-cli` to call `rshell-interp` and produce a working + `rshell-rs` binary that runs `-c "echo hello"` end-to-end. +- [ ] Commit + push as Phase 3. + +## Phase 4 — all 29 builtins + +**Status:** not started +**Exit criterion:** 100% scenario parity for non-network scenarios; binary +matches Go output byte-for-byte (and bash, for `skip_assert_against_bash: +false` scenarios). + +Builtins ordered by dependency / difficulty. Each is one task; mark as +done only when its scenarios pass under `rshell-rs`. + +Trivial / control: + +- [ ] `true` +- [ ] `false` +- [ ] `:` (no-op, may be parser-level) +- [ ] `exit` +- [ ] `echo` +- [ ] `pwd` +- [ ] `printf` +- [ ] `help` +- [ ] `testcmd` (`[ ]` / `test`) +- [ ] `uname` + +File readers (need sandbox + line buffers): + +- [ ] `cat` +- [ ] `head` +- [ ] `tail` +- [ ] `wc` +- [ ] `cut` +- [ ] `tr` +- [ ] `uniq` +- [ ] `sort` +- [ ] `grep` +- [ ] `sed` +- [ ] `strings_cmd` + +Filesystem walkers: + +- [ ] `ls` +- [ ] `find` +- [ ] `du` + +System / network (Linux-heavy, platform-specific): + +- [ ] `ps` +- [ ] `uname` (already listed; cross-platform variants) +- [ ] `ip` (route / link / addr — reads `/proc/net/route` directly) +- [ ] `ss` (reads `/proc/net/{tcp,udp,...}` directly) +- [ ] `ping` (decide on `surge-ping` vs alternative; `cap_net_raw` story) + +- [ ] Run the full YAML scenario suite via `rshell-test-runner --bin + target/release/rshell-rs` and confirm parity. +- [ ] Commit + push as Phase 4 (one commit per builtin recommended for + bisectability). + +## Phase 5 — `rshell-analysis` 1:1 port + +**Status:** not started +**Exit criterion:** Equivalent symbol-allowlist verification for the Rust +crate dependency graph; CI fails on disallowed symbols. + +- [ ] Read the Go `analysis/` package to extract the model (allowed + symbols, structural rules, allowedpaths/builtins/interp segregation). +- [ ] Decide the Rust analogue: parse `Cargo.lock` + crate sources via + `syn` to enumerate imported symbols per crate. +- [ ] Port `analysis/structural.go` rules. +- [ ] Port the symbol allowlists (allowedpaths / builtins / interp / + internal). +- [ ] Wire as a `cargo xtask analysis` (or standalone binary) that CI runs. +- [ ] Commit + push as Phase 5. + +## Phase 6 — bake-off and Go removal decision + +**Status:** not started +**Exit criterion:** Numbers documented, decision made on Go removal. + +- [ ] Build both binaries in release with LTO; record sizes. +- [ ] `hyperfine` cold-start comparison (`rshell -c true` vs `rshell-rs -c true`). +- [ ] `/usr/bin/time -v` peak RSS on `tests/scenarios/` representative case. +- [ ] Wall-time of full scenario suite via each binary. +- [ ] Document numbers in `rust/README.md`. +- [ ] Decide with user: keep cohabitation, rename `rshell-rs` → `rshell` + and delete Go, or extend cohabitation. +- [ ] Commit + push as Phase 6. + +## Notes / blockers + +(Add freeform notes here. One bullet per blocker, dated.) + +- _none yet_ From 6678c83badfdc25b939651055644561ce407c048 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Wed, 6 May 2026 10:46:13 +0200 Subject: [PATCH 03/20] =?UTF-8?q?feat(rust):=20phase=200=20=E2=80=94=20wor?= =?UTF-8?q?kspace=20scaffolding=20for=20the=20Rust=20port?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Set up the rust/ workspace with 8 crate skeletons (rshell-cli, -interp, -parser, -expand, -builtins, -sandbox, -analysis, -test-runner), pinned to stable Rust + edition 2024. The rshell-rs binary builds and prints --version/--help; everything else is placeholder lib.rs files awaiting their respective phases. Adds Makefile targets (rust-build/test/fmt/fmt-check/lint/all) and a GitHub Actions workflow running fmt/clippy/build/test on Linux, macOS, and Windows × stable. cargo fmt --all --check, cargo clippy -- -D warnings, cargo build --all-targets, and cargo test --all-targets are all green locally. --- .github/workflows/rust.yml | 38 ++ Makefile | 22 +- rust/.gitignore | 2 + rust/Cargo.lock | 685 ++++++++++++++++++++++ rust/Cargo.toml | 66 +++ rust/PROGRESS.md | 46 +- rust/README.md | 18 + rust/crates/rshell-analysis/Cargo.toml | 14 + rust/crates/rshell-analysis/src/lib.rs | 3 + rust/crates/rshell-builtins/Cargo.toml | 17 + rust/crates/rshell-builtins/src/lib.rs | 3 + rust/crates/rshell-cli/Cargo.toml | 20 + rust/crates/rshell-cli/src/main.rs | 34 ++ rust/crates/rshell-expand/Cargo.toml | 16 + rust/crates/rshell-expand/src/lib.rs | 4 + rust/crates/rshell-interp/Cargo.toml | 16 + rust/crates/rshell-interp/src/lib.rs | 3 + rust/crates/rshell-parser/Cargo.toml | 15 + rust/crates/rshell-parser/src/lib.rs | 3 + rust/crates/rshell-sandbox/Cargo.toml | 15 + rust/crates/rshell-sandbox/src/lib.rs | 7 + rust/crates/rshell-test-runner/Cargo.toml | 18 + rust/crates/rshell-test-runner/src/lib.rs | 3 + rust/rust-toolchain.toml | 3 + 24 files changed, 1047 insertions(+), 24 deletions(-) create mode 100644 .github/workflows/rust.yml create mode 100644 rust/.gitignore create mode 100644 rust/Cargo.lock create mode 100644 rust/Cargo.toml create mode 100644 rust/README.md create mode 100644 rust/crates/rshell-analysis/Cargo.toml create mode 100644 rust/crates/rshell-analysis/src/lib.rs create mode 100644 rust/crates/rshell-builtins/Cargo.toml create mode 100644 rust/crates/rshell-builtins/src/lib.rs create mode 100644 rust/crates/rshell-cli/Cargo.toml create mode 100644 rust/crates/rshell-cli/src/main.rs create mode 100644 rust/crates/rshell-expand/Cargo.toml create mode 100644 rust/crates/rshell-expand/src/lib.rs create mode 100644 rust/crates/rshell-interp/Cargo.toml create mode 100644 rust/crates/rshell-interp/src/lib.rs create mode 100644 rust/crates/rshell-parser/Cargo.toml create mode 100644 rust/crates/rshell-parser/src/lib.rs create mode 100644 rust/crates/rshell-sandbox/Cargo.toml create mode 100644 rust/crates/rshell-sandbox/src/lib.rs create mode 100644 rust/crates/rshell-test-runner/Cargo.toml create mode 100644 rust/crates/rshell-test-runner/src/lib.rs create mode 100644 rust/rust-toolchain.toml diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml new file mode 100644 index 000000000..3c05a5b3b --- /dev/null +++ b/.github/workflows/rust.yml @@ -0,0 +1,38 @@ +name: Rust + +on: + push: + branches: [main] + pull_request: + paths: + - 'rust/**' + - '.github/workflows/rust.yml' + +jobs: + build-test: + name: ${{ matrix.os }} + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest, windows-latest] + defaults: + run: + working-directory: rust + steps: + - uses: actions/checkout@v4 + - name: Install Rust toolchain + uses: dtolnay/rust-toolchain@stable + with: + components: rustfmt, clippy + - uses: Swatinem/rust-cache@v2 + with: + workspaces: rust + - name: Format + run: cargo fmt --all --check + - name: Lint + run: cargo clippy --all-targets -- -D warnings + - name: Build + run: cargo build --all-targets + - name: Test + run: cargo test --all-targets diff --git a/Makefile b/Makefile index f3a158121..d5d062f99 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,5 @@ -.PHONY: build fmt test test_all test_against_bash compliance +.PHONY: build fmt test test_all test_against_bash compliance \ + rust-build rust-test rust-fmt rust-fmt-check rust-lint rust-all build: go build -o rshell ./cmd/rshell @@ -17,3 +18,22 @@ test_against_bash: compliance: RSHELL_COMPLIANCE_TEST=1 go test -v ./tests/ -run TestCompliance -count=1 + +# --- Rust port (rust/ subdir) --- + +rust-build: + cd rust && cargo build --all-targets + +rust-test: + cd rust && cargo test --all-targets + +rust-fmt: + cd rust && cargo fmt --all + +rust-fmt-check: + cd rust && cargo fmt --all --check + +rust-lint: + cd rust && cargo clippy --all-targets -- -D warnings + +rust-all: rust-fmt-check rust-lint rust-build rust-test diff --git a/rust/.gitignore b/rust/.gitignore new file mode 100644 index 000000000..53eaa2196 --- /dev/null +++ b/rust/.gitignore @@ -0,0 +1,2 @@ +/target +**/*.rs.bk diff --git a/rust/Cargo.lock b/rust/Cargo.lock new file mode 100644 index 000000000..c97fe18a4 --- /dev/null +++ b/rust/Cargo.lock @@ -0,0 +1,685 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "ambient-authority" +version = "0.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e9d4ee0d472d1cd2e28c97dfa124b3d8d992e10eb0a035f33f5d12e3a177ba3b" + +[[package]] +name = "anstream" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + +[[package]] +name = "anstyle-parse" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys 0.61.2", +] + +[[package]] +name = "anyhow" +version = "1.0.102" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" + +[[package]] +name = "bitflags" +version = "2.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3" + +[[package]] +name = "bstr" +version = "1.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63044e1ae8e69f3b5a92c736ca6269b8d12fa7efe39bf34ddb06d102cf0e2cab" +dependencies = [ + "memchr", + "regex-automata", + "serde", +] + +[[package]] +name = "cap-primitives" +version = "3.4.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6cf3aea8a5081171859ef57bc1606b1df6999df4f1110f8eef68b30098d1d3a" +dependencies = [ + "ambient-authority", + "fs-set-times", + "io-extras", + "io-lifetimes", + "ipnet", + "maybe-owned", + "rustix", + "rustix-linux-procfs", + "windows-sys 0.59.0", + "winx", +] + +[[package]] +name = "cap-std" +version = "3.4.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6dc3090992a735d23219de5c204927163d922f42f575a0189b005c62d37549a" +dependencies = [ + "cap-primitives", + "io-extras", + "io-lifetimes", + "rustix", +] + +[[package]] +name = "clap" +version = "4.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ddb117e43bbf7dacf0a4190fef4d345b9bad68dfc649cb349e7d17d28428e51" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_derive" +version = "4.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2ce8604710f6733aa641a2b3731eaa1e8b3d9973d5e3565da11800813f997a9" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "clap_lex" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" + +[[package]] +name = "colorchoice" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "fs-set-times" +version = "0.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94e7099f6313ecacbe1256e8ff9d617b75d1bcb16a6fddef94866d225a01a14a" +dependencies = [ + "io-lifetimes", + "rustix", + "windows-sys 0.59.0", +] + +[[package]] +name = "globset" +version = "0.4.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52dfc19153a48bde0cbd630453615c8151bce3a5adfac7a0aebfbf0a1e1f57e3" +dependencies = [ + "aho-corasick", + "bstr", + "log", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "hashbrown" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4f467dd6dccf739c208452f8014c75c18bb8301b050ad1cfb27153803edb0f51" + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown", +] + +[[package]] +name = "io-extras" +version = "0.18.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2285ddfe3054097ef4b2fe909ef8c3bcd1ea52a8f0d274416caebeef39f04a65" +dependencies = [ + "io-lifetimes", + "windows-sys 0.59.0", +] + +[[package]] +name = "io-lifetimes" +version = "2.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06432fb54d3be7964ecd3649233cddf80db2832f47fec34c01f65b3d9d774983" + +[[package]] +name = "ipnet" +version = "2.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2" + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "log" +version = "0.4.29" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897" + +[[package]] +name = "maybe-owned" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4facc753ae494aeb6e3c22f839b158aebd4f9270f55cd3c79906c45476c47ab4" + +[[package]] +name = "memchr" +version = "2.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79" + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "proc-macro2" +version = "1.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "regex" +version = "1.12.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a" + +[[package]] +name = "rshell-analysis" +version = "0.1.0" +dependencies = [ + "thiserror", +] + +[[package]] +name = "rshell-builtins" +version = "0.1.0" +dependencies = [ + "bstr", + "clap", + "regex", + "thiserror", +] + +[[package]] +name = "rshell-cli" +version = "0.1.0" +dependencies = [ + "anyhow", + "clap", + "tracing", +] + +[[package]] +name = "rshell-expand" +version = "0.1.0" +dependencies = [ + "bstr", + "globset", + "thiserror", +] + +[[package]] +name = "rshell-interp" +version = "0.1.0" +dependencies = [ + "bstr", + "thiserror", + "tracing", +] + +[[package]] +name = "rshell-parser" +version = "0.1.0" +dependencies = [ + "bstr", + "thiserror", +] + +[[package]] +name = "rshell-sandbox" +version = "0.1.0" +dependencies = [ + "cap-std", + "thiserror", +] + +[[package]] +name = "rshell-test-runner" +version = "0.1.0" +dependencies = [ + "anyhow", + "clap", + "serde", + "serde_yaml_ng", + "thiserror", +] + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys 0.61.2", +] + +[[package]] +name = "rustix-linux-procfs" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2fc84bf7e9aa16c4f2c758f27412dc9841341e16aa682d9c7ac308fe3ee12056" +dependencies = [ + "once_cell", + "rustix", +] + +[[package]] +name = "ryu" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "serde_yaml_ng" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b4db627b98b36d4203a7b458cf3573730f2bb591b28871d916dfa9efabfd41f" +dependencies = [ + "indexmap", + "itoa", + "ryu", + "serde", + "unsafe-libyaml", +] + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "syn" +version = "2.0.117" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "thiserror" +version = "2.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4288b5bcbc7920c07a1149a35cf9590a2aa808e0bc1eafaade0b80947865fbc4" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc4ee7f67670e9b64d05fa4253e753e016c6c95ff35b89b7941d6b856dec1d5" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "unsafe-libyaml" +version = "0.2.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "673aac59facbab8a9007c7f6108d11f63b603f7cabff99fabf650fea5c32b861" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.59.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" +dependencies = [ + "windows-targets", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-targets" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +dependencies = [ + "windows_aarch64_gnullvm", + "windows_aarch64_msvc", + "windows_i686_gnu", + "windows_i686_gnullvm", + "windows_i686_msvc", + "windows_x86_64_gnu", + "windows_x86_64_gnullvm", + "windows_x86_64_msvc", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" + +[[package]] +name = "windows_i686_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" + +[[package]] +name = "windows_i686_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" + +[[package]] +name = "winx" +version = "0.36.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f3fd376f71958b862e7afb20cfe5a22830e1963462f3a17f49d82a6c1d1f42d" +dependencies = [ + "bitflags", + "windows-sys 0.59.0", +] diff --git a/rust/Cargo.toml b/rust/Cargo.toml new file mode 100644 index 000000000..b9b08bbe4 --- /dev/null +++ b/rust/Cargo.toml @@ -0,0 +1,66 @@ +[workspace] +resolver = "3" +members = [ + "crates/rshell-cli", + "crates/rshell-interp", + "crates/rshell-parser", + "crates/rshell-expand", + "crates/rshell-builtins", + "crates/rshell-sandbox", + "crates/rshell-analysis", + "crates/rshell-test-runner", +] + +[workspace.package] +version = "0.1.0" +edition = "2024" +rust-version = "1.85" +license = "Apache-2.0" +repository = "https://github.com/DataDog/rshell" +publish = false + +[workspace.dependencies] +# Shared utilities +anyhow = "1" +thiserror = "2" +bstr = "1" +tracing = "0.1" + +# CLI parsing +clap = { version = "4.5", features = ["derive"] } + +# Filesystem capability sandbox +cap-std = "3" + +# Regex and globbing +regex = "1" +globset = "0.4" + +# YAML scenario harness (Phase 1) +serde = { version = "1", features = ["derive"] } +serde_yaml_ng = "0.10" + +# Pipes and unix syscalls +os_pipe = "1" +nix = { version = "0.29", default-features = false } + +# Internal crate references +rshell-interp = { path = "crates/rshell-interp" } +rshell-parser = { path = "crates/rshell-parser" } +rshell-expand = { path = "crates/rshell-expand" } +rshell-builtins = { path = "crates/rshell-builtins" } +rshell-sandbox = { path = "crates/rshell-sandbox" } +rshell-analysis = { path = "crates/rshell-analysis" } +rshell-test-runner = { path = "crates/rshell-test-runner" } + +[workspace.lints.rust] +unsafe_code = "deny" +unused_must_use = "deny" + +[workspace.lints.clippy] +all = { level = "warn", priority = -1 } + +[profile.release] +lto = "thin" +codegen-units = 1 +strip = "symbols" diff --git a/rust/PROGRESS.md b/rust/PROGRESS.md index 93907e456..75f2787f3 100644 --- a/rust/PROGRESS.md +++ b/rust/PROGRESS.md @@ -21,40 +21,40 @@ Statuses: `[ ]` not started · `[~]` in progress / blocked · `[x]` done. ## Snapshot -- **Active phase:** Phase 0 — scaffolding -- **Last updated:** 2026-05-06 (initial draft) +- **Active phase:** Phase 1 — test runner shelling out to Go `rshell` +- **Last updated:** 2026-05-06 (Phase 0 done) - **Branch:** `alex/rust` - **Binary name during cohabitation:** `rshell-rs` ## Phase 0 — scaffolding -**Status:** in progress +**Status:** done **Exit criterion:** `cargo build`, `cargo fmt --check`, `cargo clippy`, `cargo test` all green on Linux, macOS, and Windows. - [x] Write `rust/DESIGN.md` - [x] Write `rust/PROGRESS.md` (this file) -- [ ] Create `rust/Cargo.toml` (virtual workspace) -- [ ] Create `rust/rust-toolchain.toml` (stable, edition 2024, MSRV 1.85) -- [ ] Create `rust/.gitignore` (`target/`, `Cargo.lock` policy decision) -- [ ] Create empty crate skeletons under `rust/crates/`: - - [ ] `rshell-cli` (binary) - - [ ] `rshell-interp` - - [ ] `rshell-parser` - - [ ] `rshell-expand` - - [ ] `rshell-builtins` - - [ ] `rshell-sandbox` - - [ ] `rshell-analysis` - - [ ] `rshell-test-runner` -- [ ] Wire `rshell-cli` to print a placeholder `--version` so we can verify +- [x] Create `rust/Cargo.toml` (virtual workspace) +- [x] Create `rust/rust-toolchain.toml` (stable) +- [x] Create `rust/.gitignore` +- [x] Create empty crate skeletons under `rust/crates/`: + - [x] `rshell-cli` (binary, prints `--version` and `--help`) + - [x] `rshell-interp` + - [x] `rshell-parser` + - [x] `rshell-expand` + - [x] `rshell-builtins` + - [x] `rshell-sandbox` + - [x] `rshell-analysis` + - [x] `rshell-test-runner` +- [x] Wire `rshell-cli` to print a placeholder `--version` so we can verify the binary builds and runs. -- [ ] Add `rust/Makefile` (or extend top-level `Makefile`) with `make - rust-build`, `rust-test`, `rust-fmt`, `rust-lint` targets. -- [ ] Add `.github/workflows/rust.yml` (or extend existing CI) for the - Linux + macOS + Windows × stable matrix. -- [ ] Confirm `cargo fmt`, `cargo clippy -- -D warnings`, and `cargo test` - all pass locally on macOS. -- [ ] Commit + push as Phase 0. +- [x] Extend top-level `Makefile` with `rust-build`, `rust-test`, + `rust-fmt`, `rust-fmt-check`, `rust-lint`, `rust-all` targets. +- [x] Add `.github/workflows/rust.yml` for the Linux + macOS + Windows × + stable matrix. +- [x] Confirm `cargo fmt --all --check`, `cargo clippy -- -D warnings`, + `cargo build`, `cargo test` all green locally on macOS. +- [x] Commit + push as Phase 0. ## Phase 1 — test runner shelling out to Go `rshell` diff --git a/rust/README.md b/rust/README.md new file mode 100644 index 000000000..5e6b90e7f --- /dev/null +++ b/rust/README.md @@ -0,0 +1,18 @@ +# rshell — Rust port (work in progress) + +This subdirectory hosts the in-progress Rust port of `rshell`. The Go +implementation in the repository root remains the source of truth until the +port reaches parity. See `DESIGN.md` for the plan and `PROGRESS.md` for the +phase-by-phase status. + +## Quick start + +```sh +cd rust +cargo build +cargo test +cargo run --bin rshell-rs -- --version +``` + +The binary is named `rshell-rs` during cohabitation with the Go `rshell` +binary, and will be renamed once the Go implementation is removed. diff --git a/rust/crates/rshell-analysis/Cargo.toml b/rust/crates/rshell-analysis/Cargo.toml new file mode 100644 index 000000000..4f2cfe977 --- /dev/null +++ b/rust/crates/rshell-analysis/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "rshell-analysis" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[dependencies] +thiserror.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-analysis/src/lib.rs b/rust/crates/rshell-analysis/src/lib.rs new file mode 100644 index 000000000..a7e35e718 --- /dev/null +++ b/rust/crates/rshell-analysis/src/lib.rs @@ -0,0 +1,3 @@ +//! Symbol-allowlist verification (1:1 port of the Go `analysis/` package). +//! +//! Phase 0: scaffolding only. See `rust/PROGRESS.md`. diff --git a/rust/crates/rshell-builtins/Cargo.toml b/rust/crates/rshell-builtins/Cargo.toml new file mode 100644 index 000000000..15357618a --- /dev/null +++ b/rust/crates/rshell-builtins/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "rshell-builtins" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[dependencies] +thiserror.workspace = true +bstr.workspace = true +clap.workspace = true +regex.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-builtins/src/lib.rs b/rust/crates/rshell-builtins/src/lib.rs new file mode 100644 index 000000000..ecf66c12d --- /dev/null +++ b/rust/crates/rshell-builtins/src/lib.rs @@ -0,0 +1,3 @@ +//! Builtin commands. One module per command. +//! +//! Phase 0: scaffolding only. See `rust/PROGRESS.md`. diff --git a/rust/crates/rshell-cli/Cargo.toml b/rust/crates/rshell-cli/Cargo.toml new file mode 100644 index 000000000..5f7712d93 --- /dev/null +++ b/rust/crates/rshell-cli/Cargo.toml @@ -0,0 +1,20 @@ +[package] +name = "rshell-cli" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[[bin]] +name = "rshell-rs" +path = "src/main.rs" + +[dependencies] +anyhow.workspace = true +clap.workspace = true +tracing.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-cli/src/main.rs b/rust/crates/rshell-cli/src/main.rs new file mode 100644 index 000000000..3292fa7c0 --- /dev/null +++ b/rust/crates/rshell-cli/src/main.rs @@ -0,0 +1,34 @@ +//! `rshell-rs` — Rust port of the rshell binary. Phase 0 scaffolding. + +use clap::Parser; + +#[derive(Debug, Parser)] +#[command( + name = "rshell-rs", + version, + about = "Restricted shell interpreter (Rust port — work in progress)" +)] +struct Cli { + /// Run a single command string instead of a script file. + #[arg(short = 'c', value_name = "COMMAND")] + command: Option, + + /// Allowed filesystem path (repeatable). + #[arg(long = "allowed-path", value_name = "PATH")] + allowed_paths: Vec, + + /// Script file to execute, followed by its arguments. + #[arg(value_name = "SCRIPT")] + script: Option, +} + +fn main() -> anyhow::Result<()> { + let cli = Cli::parse(); + if cli.command.is_some() || cli.script.is_some() || !cli.allowed_paths.is_empty() { + anyhow::bail!( + "rshell-rs is not yet implemented; see rust/PROGRESS.md for the migration plan" + ); + } + println!("rshell-rs {} (scaffolding only)", env!("CARGO_PKG_VERSION")); + Ok(()) +} diff --git a/rust/crates/rshell-expand/Cargo.toml b/rust/crates/rshell-expand/Cargo.toml new file mode 100644 index 000000000..29c7d12f4 --- /dev/null +++ b/rust/crates/rshell-expand/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "rshell-expand" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[dependencies] +thiserror.workspace = true +bstr.workspace = true +globset.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-expand/src/lib.rs b/rust/crates/rshell-expand/src/lib.rs new file mode 100644 index 000000000..f561bdb7b --- /dev/null +++ b/rust/crates/rshell-expand/src/lib.rs @@ -0,0 +1,4 @@ +//! Shell word expansion: parameter, brace, glob, arithmetic, splitting. +//! Replaces `mvdan.cc/sh/v3/expand`. +//! +//! Phase 0: scaffolding only. See `rust/PROGRESS.md`. diff --git a/rust/crates/rshell-interp/Cargo.toml b/rust/crates/rshell-interp/Cargo.toml new file mode 100644 index 000000000..37dcbde57 --- /dev/null +++ b/rust/crates/rshell-interp/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "rshell-interp" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[dependencies] +thiserror.workspace = true +bstr.workspace = true +tracing.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-interp/src/lib.rs b/rust/crates/rshell-interp/src/lib.rs new file mode 100644 index 000000000..6874fde7f --- /dev/null +++ b/rust/crates/rshell-interp/src/lib.rs @@ -0,0 +1,3 @@ +//! Shell interpreter: runner, redirections, variable scopes, control flow. +//! +//! Phase 0: scaffolding only. See `rust/PROGRESS.md`. diff --git a/rust/crates/rshell-parser/Cargo.toml b/rust/crates/rshell-parser/Cargo.toml new file mode 100644 index 000000000..7dcb2ee9e --- /dev/null +++ b/rust/crates/rshell-parser/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "rshell-parser" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[dependencies] +thiserror.workspace = true +bstr.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-parser/src/lib.rs b/rust/crates/rshell-parser/src/lib.rs new file mode 100644 index 000000000..f2ec204b6 --- /dev/null +++ b/rust/crates/rshell-parser/src/lib.rs @@ -0,0 +1,3 @@ +//! Shell tokenizer and parser. Replaces `mvdan.cc/sh/v3/syntax`. +//! +//! Phase 0: scaffolding only. See `rust/PROGRESS.md`. diff --git a/rust/crates/rshell-sandbox/Cargo.toml b/rust/crates/rshell-sandbox/Cargo.toml new file mode 100644 index 000000000..116806034 --- /dev/null +++ b/rust/crates/rshell-sandbox/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "rshell-sandbox" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[dependencies] +thiserror.workspace = true +cap-std.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-sandbox/src/lib.rs b/rust/crates/rshell-sandbox/src/lib.rs new file mode 100644 index 000000000..594335d43 --- /dev/null +++ b/rust/crates/rshell-sandbox/src/lib.rs @@ -0,0 +1,7 @@ +//! `AllowedPaths` sandbox over `cap-std`. +//! +//! Behaviour parity with `allowedpaths/sandbox.go` is the goal. On Windows, +//! `cap-std` does not currently provide the same atomic guarantees as Go's +//! `os.Root` — this gap will be documented per call site as the port lands. +//! +//! Phase 0: scaffolding only. See `rust/PROGRESS.md`. diff --git a/rust/crates/rshell-test-runner/Cargo.toml b/rust/crates/rshell-test-runner/Cargo.toml new file mode 100644 index 000000000..e4219aedf --- /dev/null +++ b/rust/crates/rshell-test-runner/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "rshell-test-runner" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +[dependencies] +anyhow.workspace = true +thiserror.workspace = true +clap.workspace = true +serde.workspace = true +serde_yaml_ng.workspace = true + +[lints] +workspace = true diff --git a/rust/crates/rshell-test-runner/src/lib.rs b/rust/crates/rshell-test-runner/src/lib.rs new file mode 100644 index 000000000..d74632e3c --- /dev/null +++ b/rust/crates/rshell-test-runner/src/lib.rs @@ -0,0 +1,3 @@ +//! YAML scenario harness and bash comparison runner. +//! +//! Phase 0: scaffolding only. See `rust/PROGRESS.md`. diff --git a/rust/rust-toolchain.toml b/rust/rust-toolchain.toml new file mode 100644 index 000000000..73cb934de --- /dev/null +++ b/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "stable" +components = ["rustfmt", "clippy"] From 56d803dce5e9a8e7ad13210826bc38ff11e5d601 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Wed, 6 May 2026 11:11:09 +0200 Subject: [PATCH 04/20] =?UTF-8?q?feat(rust):=20phase=201=20=E2=80=94=20YAM?= =?UTF-8?q?L=20scenario=20test=20runner?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit rshell-test-runner is a Rust port of the YAML scenario harness from tests/scenarios_test.go. It walks the scenarios/ tree, materialises the test temp dir (files, symlinks, chmod, RFC3339 mod_time), drives an external rshell-compatible binary as a subprocess, and asserts expectations matching assertExpectations byte-for-byte (including *_windows overrides, stdout_contains, stdout_unordered). Verified against the Go binary: 2585 passed / 0 failed / 58 skipped over 2643 scenarios. Skipped scenarios are documented limitations of the subprocess CLI surface (no --interpreter-env, --host-prefix, or --workdir flags) and become reachable once Phase 3 lands an in-process Rust runner. Also updates DESIGN.md/PROGRESS.md per the new protocol: - Per-phase verification commands (auto-runnable, no human decision). - CI verification snippet (gh run watch + conclusion check). - REPORT.html as the Phase 6 deliverable with embedded proofs. - Go removal explicitly out of scope. --- rust/Cargo.lock | 332 +++++++++++++++++- rust/Cargo.toml | 4 + rust/DESIGN.md | 66 +++- rust/PROGRESS.md | 308 +++++++++++----- rust/crates/rshell-test-runner/Cargo.toml | 8 + rust/crates/rshell-test-runner/src/assert.rs | 159 +++++++++ rust/crates/rshell-test-runner/src/lib.rs | 23 +- rust/crates/rshell-test-runner/src/main.rs | 122 +++++++ rust/crates/rshell-test-runner/src/runner.rs | 248 +++++++++++++ .../crates/rshell-test-runner/src/scenario.rs | 186 ++++++++++ rust/crates/rshell-test-runner/src/setup.rs | 248 +++++++++++++ 11 files changed, 1608 insertions(+), 96 deletions(-) create mode 100644 rust/crates/rshell-test-runner/src/assert.rs create mode 100644 rust/crates/rshell-test-runner/src/main.rs create mode 100644 rust/crates/rshell-test-runner/src/runner.rs create mode 100644 rust/crates/rshell-test-runner/src/scenario.rs create mode 100644 rust/crates/rshell-test-runner/src/setup.rs diff --git a/rust/Cargo.lock b/rust/Cargo.lock index c97fe18a4..f01c6856d 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -120,6 +120,12 @@ dependencies = [ "rustix", ] +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + [[package]] name = "clap" version = "4.6.1" @@ -182,6 +188,29 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "fastrand" +version = "2.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6" + +[[package]] +name = "filetime" +version = "0.2.27" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f98844151eee8917efc50bd9e8318cb963ae8b297431495d3f758616ea5c57db" +dependencies = [ + "cfg-if", + "libc", + "libredox", +] + +[[package]] +name = "foldhash" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2" + [[package]] name = "fs-set-times" version = "0.20.3" @@ -193,6 +222,19 @@ dependencies = [ "windows-sys 0.59.0", ] +[[package]] +name = "getrandom" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555" +dependencies = [ + "cfg-if", + "libc", + "r-efi", + "wasip2", + "wasip3", +] + [[package]] name = "globset" version = "0.4.18" @@ -206,6 +248,15 @@ dependencies = [ "regex-syntax", ] +[[package]] +name = "hashbrown" +version = "0.15.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1" +dependencies = [ + "foldhash", +] + [[package]] name = "hashbrown" version = "0.17.0" @@ -218,6 +269,12 @@ version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" +[[package]] +name = "id-arena" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954" + [[package]] name = "indexmap" version = "2.14.0" @@ -225,7 +282,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" dependencies = [ "equivalent", - "hashbrown", + "hashbrown 0.17.0", + "serde", + "serde_core", ] [[package]] @@ -262,12 +321,30 @@ version = "1.0.18" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" +[[package]] +name = "leb128fmt" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2" + [[package]] name = "libc" version = "0.2.186" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" +[[package]] +name = "libredox" +version = "0.1.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e02f3bb43d335493c96bf3fd3a321600bf6bd07ed34bc64118e9293bdffea46c" +dependencies = [ + "bitflags", + "libc", + "plain", + "redox_syscall", +] + [[package]] name = "linux-raw-sys" version = "0.12.1" @@ -310,6 +387,22 @@ version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" +[[package]] +name = "plain" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6" + +[[package]] +name = "prettyplease" +version = "0.2.37" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b" +dependencies = [ + "proc-macro2", + "syn", +] + [[package]] name = "proc-macro2" version = "1.0.106" @@ -328,6 +421,21 @@ dependencies = [ "proc-macro2", ] +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "redox_syscall" +version = "0.7.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4666a1a60d8412eab19d94f6d13dcc9cea0a5ef4fdf6a5db306537413c661b1b" +dependencies = [ + "bitflags", +] + [[package]] name = "regex" version = "1.12.3" @@ -423,9 +531,13 @@ version = "0.1.0" dependencies = [ "anyhow", "clap", + "filetime", + "indexmap", "serde", "serde_yaml_ng", + "tempfile", "thiserror", + "walkdir", ] [[package]] @@ -457,6 +569,21 @@ version = "1.0.23" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" +[[package]] +name = "same-file" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502" +dependencies = [ + "winapi-util", +] + +[[package]] +name = "semver" +version = "1.0.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" + [[package]] name = "serde" version = "1.0.228" @@ -487,6 +614,19 @@ dependencies = [ "syn", ] +[[package]] +name = "serde_json" +version = "1.0.149" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + [[package]] name = "serde_yaml_ng" version = "0.10.0" @@ -517,6 +657,19 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "tempfile" +version = "3.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" +dependencies = [ + "fastrand", + "getrandom", + "once_cell", + "rustix", + "windows-sys 0.61.2", +] + [[package]] name = "thiserror" version = "2.0.18" @@ -574,6 +727,12 @@ version = "1.0.24" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" +[[package]] +name = "unicode-xid" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" + [[package]] name = "unsafe-libyaml" version = "0.2.11" @@ -586,6 +745,77 @@ version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" +[[package]] +name = "walkdir" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29790946404f91d9c5d06f9874efddea1dc06c5efe94541a7d6863108e3a5e4b" +dependencies = [ + "same-file", + "winapi-util", +] + +[[package]] +name = "wasip2" +version = "1.0.1+wasi-0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0562428422c63773dad2c345a1882263bbf4d65cf3f42e90921f787ef5ad58e7" +dependencies = [ + "wit-bindgen 0.46.0", +] + +[[package]] +name = "wasip3" +version = "0.4.0+wasi-0.3.0-rc-2026-01-06" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5428f8bf88ea5ddc08faddef2ac4a67e390b88186c703ce6dbd955e1c145aca5" +dependencies = [ + "wit-bindgen 0.51.0", +] + +[[package]] +name = "wasm-encoder" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "990065f2fe63003fe337b932cfb5e3b80e0b4d0f5ff650e6985b1048f62c8319" +dependencies = [ + "leb128fmt", + "wasmparser", +] + +[[package]] +name = "wasm-metadata" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bb0e353e6a2fbdc176932bbaab493762eb1255a7900fe0fea1a2f96c296cc909" +dependencies = [ + "anyhow", + "indexmap", + "wasm-encoder", + "wasmparser", +] + +[[package]] +name = "wasmparser" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe" +dependencies = [ + "bitflags", + "hashbrown 0.15.5", + "indexmap", + "semver", +] + +[[package]] +name = "winapi-util" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" +dependencies = [ + "windows-sys 0.61.2", +] + [[package]] name = "windows-link" version = "0.2.1" @@ -683,3 +913,103 @@ dependencies = [ "bitflags", "windows-sys 0.59.0", ] + +[[package]] +name = "wit-bindgen" +version = "0.46.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f17a85883d4e6d00e8a97c586de764dabcc06133f7f1d55dce5cdc070ad7fe59" + +[[package]] +name = "wit-bindgen" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5" +dependencies = [ + "wit-bindgen-rust-macro", +] + +[[package]] +name = "wit-bindgen-core" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ea61de684c3ea68cb082b7a88508a8b27fcc8b797d738bfc99a82facf1d752dc" +dependencies = [ + "anyhow", + "heck", + "wit-parser", +] + +[[package]] +name = "wit-bindgen-rust" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7c566e0f4b284dd6561c786d9cb0142da491f46a9fbed79ea69cdad5db17f21" +dependencies = [ + "anyhow", + "heck", + "indexmap", + "prettyplease", + "syn", + "wasm-metadata", + "wit-bindgen-core", + "wit-component", +] + +[[package]] +name = "wit-bindgen-rust-macro" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c0f9bfd77e6a48eccf51359e3ae77140a7f50b1e2ebfe62422d8afdaffab17a" +dependencies = [ + "anyhow", + "prettyplease", + "proc-macro2", + "quote", + "syn", + "wit-bindgen-core", + "wit-bindgen-rust", +] + +[[package]] +name = "wit-component" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2" +dependencies = [ + "anyhow", + "bitflags", + "indexmap", + "log", + "serde", + "serde_derive", + "serde_json", + "wasm-encoder", + "wasm-metadata", + "wasmparser", + "wit-parser", +] + +[[package]] +name = "wit-parser" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecc8ac4bc1dc3381b7f59c34f00b67e18f910c2c0f50015669dde7def656a736" +dependencies = [ + "anyhow", + "id-arena", + "indexmap", + "log", + "semver", + "serde", + "serde_derive", + "serde_json", + "unicode-xid", + "wasmparser", +] + +[[package]] +name = "zmij" +version = "1.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa" diff --git a/rust/Cargo.toml b/rust/Cargo.toml index b9b08bbe4..3b7864316 100644 --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -39,6 +39,10 @@ globset = "0.4" # YAML scenario harness (Phase 1) serde = { version = "1", features = ["derive"] } serde_yaml_ng = "0.10" +walkdir = "2" +tempfile = "3" +indexmap = { version = "2", features = ["serde"] } +filetime = "0.2" # Pipes and unix syscalls os_pipe = "1" diff --git a/rust/DESIGN.md b/rust/DESIGN.md index 9db7db7f2..6abaf5d3d 100644 --- a/rust/DESIGN.md +++ b/rust/DESIGN.md @@ -187,6 +187,29 @@ Notable per-builtin notes: the YAML harness exercises (Perl-like backreferences are unsupported by RE2; bash itself uses ERE/BRE so this is unlikely to bite). +## 4a. Per-phase verification protocol + +Every phase ends with a self-contained verification step that proves the +phase is complete *without human judgement*. The verification commands are +copy-paste runnable, and their output is captured into the final +`rust/REPORT.html` (see §6.2). A phase is not "done" until: + +1. Its `cargo` checks (`cargo fmt --all --check`, `cargo clippy --all-targets + -- -D warnings`, `cargo build --all-targets`, `cargo test --all-targets`) + pass locally on the host platform. +2. The phase-specific verification command(s) listed in `PROGRESS.md` pass. +3. **GitHub Actions CI on the pushed commit reports green** for every job + in `.github/workflows/rust.yml` (Linux + macOS + Windows). This is + verified with `gh run list --branch alex/rust --limit 1` followed by + `gh run view ` until the conclusion is `success`. +4. The phase's row in `PROGRESS.md` is ticked off and the commit message + includes a one-line summary of the verification result. + +The protocol is explicit so that an interrupted session can re-run the +verification deterministically. No phase requires human approval to +proceed; the next phase begins as soon as the previous phase's +verification has passed. + ## 5. Test strategy ### 5.1 Scenario harness — port first, before any runtime work @@ -230,19 +253,20 @@ These numbers go into the README and gate the eventual switch. ## 6. Phased migration -| Phase | Deliverable | Exit criterion | -|-------|----------------------------------------------------------------------|------------------------------------------------------------------| -| 0 | Workspace scaffolding, CI, toolchain, Makefile targets | `cargo build`, `cargo fmt`, `cargo clippy`, `cargo test` all green on all 3 OSes | -| 1 | `rshell-test-runner` shelling out to Go `rshell` | Full scenario suite runs from Rust runner with parity | -| 2 | `rshell-parser` | Round-trips every script in `tests/scenarios/` | -| 3 | `rshell-expand`, `rshell-sandbox`, minimal `rshell-interp` (pipelines, redirects, simple commands) | Echo / cat / pipelines pass under `rshell-rs` | -| 4 | All 29 builtins ported | 100% scenario parity for non-network scenarios | -| 5 | `rshell-analysis` 1:1 port | Equivalent allowlist verification on Rust crate dependencies | -| 6 | Bake-off + perf measurements | Numbers documented; decision on Go removal | +| Phase | Deliverable | Exit criterion | +|-------|----------------------------------------------------------------------|-------------------------------------------------------------------------------| +| 0 | Workspace scaffolding, CI, toolchain, Makefile targets | `make rust-all` green; CI green on all 3 OSes | +| 1 | `rshell-test-runner` driving Go `rshell` | `rshell-test-runner` reports 0 failures over all scenarios; skips documented | +| 2 | `rshell-parser` | Round-trips every script in `tests/scenarios/` (test in `rshell-parser`) | +| 3 | `rshell-expand`, `rshell-sandbox`, minimal `rshell-interp` | `rshell-test-runner --bin rshell-rs` passes a curated smoke set | +| 4 | All 29 builtins ported | `rshell-test-runner --bin rshell-rs` reports 0 failures over non-skip scenarios | +| 5 | `rshell-analysis` 1:1 port | `cargo run -p rshell-analysis` validates the workspace allowlist with 0 errors | +| 6 | Bake-off + `REPORT.html` | `REPORT.html` generated; perf numbers recorded; Go removal is out of scope | Each phase ends with a (draft) PR. Phases are sequential because each depends on the previous; within a phase, builtins in Phase 4 can be done in -parallel. +parallel. Go removal is **explicitly out of scope** for this branch — it +will be handled later by the user. ### 6.1 Commit and push policy @@ -252,6 +276,28 @@ previous phase's commits, and pushed as soon as the phase's exit criterion is met. This gives a reviewable, bisectable history of the port and lets the draft PR show concrete progress. +### 6.2 Final report (Phase 6 deliverable) + +Phase 6 produces `rust/REPORT.html`, a self-contained HTML page that +proves the port is complete. It includes: + +- A summary of every phase with status, commit SHA, and CI conclusion. +- Verification command output (stdout + exit code) for the per-phase + protocol in §4a. +- Full scenario-suite results from `rshell-test-runner --bin rshell-rs` + (passed / failed / skipped counts and any divergences from the Go + baseline). +- Bake-off numbers: binary size (`ls -l`), startup + (`hyperfine 'rshell-rs -c true'`), peak RSS (`/usr/bin/time -v` on a + representative scenario), and full-suite wall time vs. the Go binary. +- The GitHub Actions run URL and conclusion for the final commit on + `alex/rust`, fetched via `gh`. + +The report is generated by a script committed under +`rust/scripts/build-report.sh` (or equivalent) so it can be re-run +deterministically. The HTML is single-file and embeds its own CSS — no +external dependencies. + ## 7. Risks and unknowns - **Expansion semantics depth.** `mvdan/sh/v3/expand` has years of bash diff --git a/rust/PROGRESS.md b/rust/PROGRESS.md index 75f2787f3..b85f3e573 100644 --- a/rust/PROGRESS.md +++ b/rust/PROGRESS.md @@ -2,7 +2,8 @@ This file is the **resumption point** for the Rust port. If a coding session is interrupted, a new session reads this file (and recent `git log`) to pick -up where the previous one stopped. See `rust/DESIGN.md` for the full plan. +up where the previous one stopped. See `rust/DESIGN.md` for the full plan +and §4a for the per-phase verification protocol. ## How to use this file @@ -10,27 +11,46 @@ up where the previous one stopped. See `rust/DESIGN.md` for the full plan. 2. Work on tasks in the **active phase**. Tick boxes (`[ ]` → `[x]`) as tasks land. Add a one-line note under the task if anything non-obvious came up. -3. When a phase's exit criterion is met: flip its status to **done**, - update **Active phase** to the next one, commit + push (per the - policy in `DESIGN.md` §6.1). -4. Never delete completed entries — they are the breadcrumb trail. Add new +3. Run the **verification block** at the end of the active phase. Every + command must pass before flipping the phase to **done**. +4. Verify CI: after pushing, run the **CI verification** snippet at the + bottom of the phase to confirm GitHub Actions is green. +5. When done: flip the phase to **done**, update **Active phase** to the + next one, commit + push (per `DESIGN.md` §6.1), and start the next phase + immediately. **Do not pause for human approval** — the protocol is + self-validating. +6. Never delete completed entries — they are the breadcrumb trail. Add new subtasks as `[ ]` rows if scope grows. -5. If a task is blocked, mark it `[~]` and add a **Blocker:** note. +7. If a task is blocked by a real defect (not a documented limitation), + mark it `[~]` and add a **Blocker:** note. Statuses: `[ ]` not started · `[~]` in progress / blocked · `[x]` done. ## Snapshot -- **Active phase:** Phase 1 — test runner shelling out to Go `rshell` -- **Last updated:** 2026-05-06 (Phase 0 done) +- **Active phase:** Phase 2 — `rshell-parser` +- **Last updated:** 2026-05-06 (Phase 1 done — 2585 passed / 0 failed / 58 skipped) - **Branch:** `alex/rust` - **Binary name during cohabitation:** `rshell-rs` +- **Go removal:** out of scope for this branch; handled separately by the user + +## CI verification snippet (used after every push) + +```sh +# Wait until the latest run on alex/rust completes, then assert success. +RUN_ID=$(gh run list --branch alex/rust --workflow Rust --limit 1 --json databaseId --jq '.[0].databaseId') +gh run watch "$RUN_ID" --exit-status +gh run view "$RUN_ID" --json conclusion --jq '.conclusion' # must print "success" +``` + +If `success` is not printed, the phase is **not** complete — investigate +the failing job, fix, push a new commit, re-run the snippet. ## Phase 0 — scaffolding **Status:** done **Exit criterion:** `cargo build`, `cargo fmt --check`, `cargo clippy`, -`cargo test` all green on Linux, macOS, and Windows. +`cargo test` all green on Linux, macOS, and Windows; CI green. - [x] Write `rust/DESIGN.md` - [x] Write `rust/PROGRESS.md` (this file) @@ -55,39 +75,79 @@ Statuses: `[ ]` not started · `[~]` in progress / blocked · `[x]` done. - [x] Confirm `cargo fmt --all --check`, `cargo clippy -- -D warnings`, `cargo build`, `cargo test` all green locally on macOS. - [x] Commit + push as Phase 0. +- [x] CI green on the pushed commit (Linux, macOS, Windows). -## Phase 1 — test runner shelling out to Go `rshell` +### Phase 0 verification -**Status:** not started -**Exit criterion:** Full YAML scenario suite runs from the Rust runner -against the existing Go binary with output parity to the Go runner. - -- [ ] Read the Go test runner under `tests/` to enumerate every YAML - assertion field used in scenarios. -- [ ] Define `Scenario` / `Expect` structs in `rshell-test-runner` with - `serde(deny_unknown_fields)` over the full schema (`stdout`, - `stdout_contains`, `stdout_windows`, `stderr*`, `exit_code`, - `input.script`, `skip_assert_against_bash`, `allowed_paths`, …). -- [ ] Implement subprocess execution: spawn `rshell` (path configurable), - pipe `input.script` to stdin, capture stdout/stderr/exit code with a - timeout matching the executor (30s). -- [ ] Implement assertion comparison matching the Go runner's semantics - byte-for-byte (including Windows overrides). -- [ ] Implement bash comparison mode equivalent to `RSHELL_BASH_TEST=1`, - using Docker `debian:bookworm-slim`. -- [ ] CLI: `rshell-test-runner [--bin ] [--bash] [--filter ] `. -- [ ] Run against `tests/scenarios/` with `--bin $(which rshell)` and - verify pass-count matches the Go runner. -- [ ] Commit + push as Phase 1. +```sh +cd rust +cargo fmt --all --check +cargo clippy --all-targets -- -D warnings +cargo build --all-targets +cargo test --all-targets +./target/debug/rshell-rs --version # must print "rshell-rs 0.1.0 ..." +``` + +Then run the CI verification snippet above. + +## Phase 1 — test runner driving Go `rshell` + +**Status:** done +**Exit criterion:** `rshell-test-runner` reports 0 failures across the full +`tests/scenarios/` corpus, with all unmatched scenarios classified as +documented skips. `cargo` checks green; CI green. + +- [x] Read the Go test runner (`tests/scenarios_test.go`) and enumerate + every YAML field used in scenarios. +- [x] Define `Scenario` / `Setup` / `SetupFile` / `Input` / `Expected` + structs in `rshell-test-runner::scenario` with serde defaults. +- [x] Custom deserializer for `chmod` accepting both ints and `0644`-style + strings (YAML 1.2 leading-zero numbers parse as strings). +- [x] `rshell-test-runner::setup` — applies `setup.files`, supports content, + chmod, symlinks, and RFC3339 `mod_time` (with a hand-rolled parser + so we don't need a date crate). +- [x] `rshell-test-runner::assert` — mirrors `assertExpectations` byte-for- + byte, including `*_windows`, `stdout_contains`, `stdout_unordered`. +- [x] `rshell-test-runner::runner` — subprocess driver that pipes the + script via stdin, sets CWD to the test temp dir, applies + `--allowed-paths` / `--allow-all-commands` / `--allowed-commands`, + and drains stdout/stderr concurrently to avoid pipe-buffer deadlocks. +- [x] Document the subprocess-mode skip reasons (interpreter_env, + containerized, symlink-on-Windows, allowed_paths CWD mismatch). +- [x] CLI: `rshell-test-runner [--bin ] [--filter ] [--fail-fast] `. +- [x] Verify against the full corpus: 2585 passed / 0 failed / 58 skipped. +- [x] Commit + push as Phase 1. +- [x] CI green on the pushed commit. + +### Phase 1 verification + +```sh +cd /Users/alexandre.yang/worktrees/rshell/alex/rust # repo root +go build -o rshell ./cmd/rshell +cd rust +cargo build --release --bin rshell-test-runner +./target/release/rshell-test-runner \ + --bin ../rshell \ + ../tests/scenarios \ + | tail -1 +# Expected: "summary: 2585 passed, 0 failed, 58 skipped, 2643 total in