fix(bundle): carry --manifests files as written, without redaction by exo-nikita · Pull Request #182 · ExodusOSS/stasis

exo-nikita · 2026-09-29T08:47:03Z

Rule

A bundle includes the files it packages exactly as they are on disk: stasis never modifies them. It may choose which files to carry, and it may refuse to carry one, but it never edits one.

Problem

stasis bundle --manifests (Solidity) broke that rule. A carried foundry.toml lost its [rpc_endpoints] and [etherscan] tables and every key named like a secret. Every carried file also lost the user info of its URLs (redactFoundryToml and scrubUrlCredentials in loaders/foundry.js). So the bundle carried a foundry.toml or .gitmodules that wasn't the file the build used, with no error.

The redaction also couldn't be complete. A deny-list of key names and a text regex over URLs miss credentials in query strings, nested inline tables, TOML escapes, and a second URL in one value. --package-json has always carried package.json byte for byte.

Change

Carried files are now exactly what's on disk.

stasis/src/loaders/foundry.js: redactFoundryToml, scrubUrlCredentials and their key and table lists are removed.
stasis/src/cmd/bundle.js: solidityManifests carries each file's text unchanged.
Unchanged: which files may be carried.
- only files inside the root, checked by real path;
- never a dependency config whose extends leaves the dependency;
- never .env or hardhat.config.*.
Docs: doc/file-formats.md and the usage text now say the carried files may hold credentials. They tell users to keep secrets in the environment (${VAR} in foundry.toml) or not pass --manifests. There is no runtime warning, which matches --package-json.

After this, nothing on main edits a file it packages. --package-json already carries package.json exactly as read.

Tests

tests/bundle-cmd.test.js: the credentials test now asserts that foundry.toml, .gitmodules and the dependency's foundry.toml are carried byte for byte. It still asserts that .env, hardhat.config.ts and the dependency's outside extends are never read as config or carried.
tests/solidity-loader.test.js: the redactFoundryToml unit test is removed. Its one assertion not about redaction, that foundryTomlRemappings names the line of text that isn't TOML, is kept as its own test.
Results:
- solidity-loader 60/60, toml 4/4, rust-loader 71/71 and rust-features 20/20 pass.
- Every Solidity and --manifests test in bundle-cmd passes.
- bundle-cmd otherwise fails the same 90 JS/TS tests as main in this environment, which has no npm install and so no oxc-parser.
Lint: oxlint 1.52 is clean.

Follow-ups for the open PRs

Both still edit packaged files, and should carry them as they are when they rebase onto this.

fix(bundle): Rust loader follow-ups: lexer offsets, dead items, paths through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests #175 (abd3f50):
- stasis/src/cmd/bundle.js:294 passes Solidity manifests through redactFoundryToml / stripUrlCredentials.
- stasis/src/cmd/bundle.js:487 passes carried Cargo files through filterCargoConfig (.cargo/config* loses tables and keys) and stripUrlCredentials (Cargo.toml / Cargo.lock lose URL user info).
- If a .cargo/config.toml shouldn't be carried, leave it out whole rather than filtering it.
fix(bundle): Solidity loader — decide file ownership by real path; invalid configs are errors #178 (3a52659): stasis/src/cmd/bundle.js:259 passes Solidity manifests through redactManifest, which drops comments, keys, scripts and homepage and strips URL user info.

🤖 Generated with Claude Code

https://claude.ai/code/session_016XK2nLruNc3U23JzQ3fxNm

--manifests redacted what it carried: a carried foundry.toml lost its [rpc_endpoints] and [etherscan] tables and any key named like a secret, and every carried file lost the user info of its URLs. That can't be complete (a key in a query string, a nested inline table, a TOML escape all get past a text or key-name rule), and it makes the bundle carry a file that isn't the one the build used, which is the silent kind of wrong the bundle must not be. --package-json has never edited what it carries either. The files are now carried byte for byte. What decides which files may be carried stays as it was: nothing outside the root, no dependency config whose `extends` leaves the dependency, never `.env` or hardhat.config.*. The docs and the usage text say that the carried files may hold credentials, and to keep them in the environment (`${VAR}` in foundry.toml) or not pass --manifests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016XK2nLruNc3U23JzQ3fxNm

… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests (#175) * fix(bundle): Rust loader follow-ups: lexer offsets, dead items, paths through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): Rust loader review 7: compile units, honest ranking, template mods - Refuse a Rust source or include_str! file that isn't UTF-8 (and a non-UTF-8 manifest); carry an include_bytes! file as base64. - Compile units: code built for the host (build scripts, build-dependencies, proc-macro crates) is scanned against the host's cfgs and, under resolver 2, the host's own feature context; a crate built both ways is followed as both. Features are on, maybe (only an undecided target table enables them) or off. - --cargo-manifests carries a vendored crate's Cargo.toml and .cargo-checksum.json, and a project package's configs up to the bundle root; the vendored directory follows the replace-with chain whatever its name. - Custom cfgs a build script prints or rustflags set are not presumed off; cfg_x! gates take the cfg their macro_rules! definition writes. - Ambiguous path edges record a cfg-keyed map of their candidates instead of the first written; candidates the build rules out come last; a child module under a custom or dead cfg gives way to an import of its name. - Macro scope: a template's bare calls are made where it is invoked; a mod a template declares belongs to each in-package invoking module and is found beside it (libc's prelude!() -> src/types.rs), else beside the definition. - `extern crate self as x;` is in the extern prelude; a declared dependency is reported missing whatever a glob into a missing crate may provide. - Vendored versions come from the lock, else the one that fits, else a warning and no guess; out-of-root path/[patch] deps never bind to vendored copies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): decide a cfg an unknown leaf repeats in when it can't vary evalCfg is three-valued per leaf, so an item's cfg joined with its enclosing blocks' -- zerocopy's `#[cfg(any(test, kani))] mod tests { #[cfg(not(kani))] mod compatibility { use rand::…; } }`, i.e. all(any(test, kani), not(kani)) -- stayed undecided and its `use rand` was reported as a missing crate. When an unknown leaf occurs more than once (up to six of them), the predicate is evaluated under each assignment and decided when every one agrees. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): rank a module's own items with its imports, and imports that lead out A path's segment was looked up in the module's imports first and its own definitions only when no import answered, so an import leading out of the bundle (tokio's `pub(crate) use std::sync::atomic::AtomicU64;` in one `imp` variant) hid the mutex-based `AtomicU64` the other variant defines, and a dead import (`cfg_taskdump!`'s) beat the live fn under `cfg_not_taskdump!`. The module's items are now candidates right after its named imports, ahead of globs, and an import leading out of the bundle is an answer under its cfgs rather than a last resort: `imp::AtomicU64` maps both variant files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * perf(bundle): cache leaf verdicts, alternative keys and package lookups in the tree pass The cfg-keyed alternatives and the build-deadness check added in review 7 rebuilt the same cfg texts and re-parsed the same leaf predicates for every candidate: libc's tree pass went from 227ms to 351ms. A leaf's verdict is kept per interned build, an alternative's cfg text per asker set, and a file's package per tree pass (asked for every import and item). libc 282ms, tokio 492ms -> 464ms (589afce: 444ms); output unchanged on the corpus. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): Rust loader review 8: refused roots, missing crates, per-table deps, resolver regressions - buildRustBundle: a crate root the tree pass wants but the walk refused is asked for once, so the walk/tree loop ends; every crate the bundle lacks is reported (refused roots included) whether or not a vendor dir exists, the `cargo vendor` hint only without one. - Cargo: each dependency table is its own dependency (version, path, package, workspace per table), and a file follows the one its target uses (build script, test target, normal); a Cargo.lock pin is taken only when the requirement allows it, else the lock is reported out of date; a [patch] is used only when its version fits, and [patch] in .cargo/config.toml (nearest first, ahead of the manifest's) is read; a path dependency or patch outside the bundle root, or naming no package, is reported. - Resolver: a local module under a custom cfg gives way only to the module's own `use` or item, never to a glob; a macro call is looked up among macros (a `fn m`, `mod m` or an import of a module/crate is no `m!`); a template's calls and includes stay with it past a nested `macro_rules!`; cfg values keep their spacing inside quotes, so `"a b"` and `"ab"` are not merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * feat(bundle): read Cargo through @preventive/lockfile, and resolve with cargo's resolver where it can - Cargo.toml, Cargo.lock and the cargo config's [patch] are read by @preventive/lockfile 1.0.0-alpha.3 (parseCargoManifest, parseCargoLock, parseCargoConfig): what cargo refuses, or what the reader can't tell how cargo reads, stops the build naming the file; a Cargo.lock older than version 3 is refused. Version requirements are matched by its rust-semver.js. The loader's own manifest and lock readers and semver matcher are gone. - With a lockfile, --cargo-target and every locked package in-tree (path packages inside the bundle root, the rest vendored with checksums), features come from cargo's resolver: linkCargo over the manifests, then resolveCargoFeatures for `cargo build -p <the entries' packages>`; a lockfile out of date or a vendored copy whose checksum isn't the lockfile's stops the build. An unknown host is resolved both ways (its target tables maybe). - Otherwise the manifests are replayed as before, with fixes: a git dependency takes a git checkout's vendored copy and a registry one a registry's (by .cargo-checksum.json and the lockfile's sources); `dep/feat` turns on the package's feature of that name, written or implicit; a proc-macro entry is resolved and scanned for the host; prereleases follow the semver crate; the lockfile is the entries' workspace's, below the bundle root too. - cfg(true), cfg(false) and raw identifiers in cfgs. - Tests: cargo's recorded builds of @preventive/lockfile's workspace fixture (PreventiveMeasures/libraries#56, MIT) through createCargoContext, the refusals, the fallback, and each fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * refactor(bundle): Rust loader simplification pass - scanRustItems: one cfg verdict per predicate per scan, and the open scopes' conjunction kept as they open and close, rather than joined and evaluated again at every token (a 220 KB file inside one #[cfg] block: 588 ms -> 36 ms; with two compile units, 1.3 s -> 28 ms). - cfg leaves: leafFalse evaluates a leaf by key and value (evalCfgKey) instead of rebuilding its text for evalCfg; leavesOfCfgs uses cfgLeaves; joinCfgs where it was inlined; one cache helper for the per-asker verdicts. - One walk driver (collectRustBundle) for buildRustBundle and loadRust: build scripts, compile units and the wantedRoots loop live in rust.js; the link-out-of-package check (withinRealDir), boundaryOf and addUnits are shared rather than copied into bundle.js. - cargo.js: each path dependency's directory resolved once, against the workspace root its inheritance was read from (depSpec gone); requests carry kind and target (no `kind@target` parsing); resolver 1's single context folded into nodeKey; one [patch] source list; the git/registry flag stored with the checksum; tableOf's TOML reused for the manifest; build script and build files memoized per package; one error-naming helper shared with toml.js; internals no longer exported. - rust.js: the macro-template helpers shared by the walk and the tree pass (keyed by the package's own template set, not its identity); providedFrom's per-module file set computed once; resolveModDecl's unused cfg branch removed. - The Rust-only flag checks in stasis.js and bundle.js table-driven. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): Rust loader review 10: platform tables, build-script crates, proc-macro roots, out-of-root workspaces Crate resolution: - A dependency name is resolved from the tables the asking code links: per crate root the file is compiled in (a build script's modules and a file it shares with the lib through #[path] take the [build-dependencies]; tests, benches and examples the dev ones too), and only tables for the platforms the code is compiled for. Tables that may each apply (no target to decide them, or two roles) give cfg-keyed alternatives, each carried. Both on the replay and on cargo's resolver. - A crate's lib-name fallback resolves only dependencies that may have that lib name: `u8::MAX` no longer resolves every declared dependency and warns about versions. - A declared dependency the bundle lacks is reported when only an expression names it (`serde_json::to_string(…)`). - Macros of other crates get edges: `use dep::mac;` (an alias, a prelude glob too), `dep::mac!(…)` and `#[macro_use] extern crate`. Cargo: - The replay seeds a proc-macro root in the target context too, as cargo activates a member it is asked to build (fixes a regression from f0287e0). - A member bundled from its own directory reads its workspace root above the bundle root (never bundled) for what it inherits and the resolver, when that root's members take it. - [patch] tables apply to their own source only: a git URL's not to the crates.io dependency. - The vendor directory comes from the cargo config nearest the entries (a nested workspace's own). - `stasis bundle` says when the features come from the replay and why; a vendored copy the lockfile doesn't list no longer forces the replay. EXODUS_STASIS_DEBUG labels the mode and prints both contexts. - A vendored file whose bytes aren't those .cargo-checksum.json lists stops the bundle, as cargo refuses to build it. cfgs and resolution: - `r#true` / `r#false` are custom cfgs, not the literals, in every reader. - A child module under a custom cfg gives way to a glob of its name, as any doubtful candidate does. - An include in a nested macro_rules! is that macro's, resolved where it is invoked. - A refused or missing crate root is warned about once. - Dense cycles of cfg-gated globs no longer blow up: leaf keys are cached, absorbed alternatives dropped, and alternatives capped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * test(bundle): hold the Rust loader's new fixture projects in one tests/fixtures/rust-bundle.json.br `includes` and `cargo-recorded` (30 files, 44 KB) become one brotli-compressed JSON file (10 KB): project name -> project-relative path -> the file's text, or `{ base64 }` for one that isn't UTF-8. tests/rust-fixtures.helper.js writes a project out once per test process, to a temporary directory removed when the process exits, and `rustFixture(name)` gives its path; `node tests/rust-fixtures.helper.js unpack|pack <dir>` edits them. The fixtures main already holds stay as they are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL --------- Co-authored-by: Claude <noreply@anthropic.com>

ChALkeR merged commit d45276a into main Sep 29, 2026
5 checks passed

Navigation Menu

fix(bundle): carry --manifests files as written, without redaction - #182

fix(bundle): carry --manifests files as written, without redaction#182
ChALkeR merged 1 commit into
mainfrom
claude/wizardly-keller-mesfi0

exo-nikita commented Sep 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

exo-nikita commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rule

Problem

Change

Tests

Follow-ups for the open PRs

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

exo-nikita commented Sep 29, 2026 •

edited

Loading