fix(bundle): a strict TOML reader in loaders/toml.js - #179
Merged
Merged
Conversation
The TOML reading that Cargo.toml, Cargo.lock, .cargo/config and foundry.toml share -- logical lines, dotted keys, values, the entry stream, and the bracket helpers the Rust scanner uses too -- moves out of cargo.js into toml.js, as is: no new cases. cargo.js keeps the Cargo semantics on top of it and the path helper; rust.js and foundry.js import what they used from the new module. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
force-pushed
the
claude/toml-reader-02j8b2
branch
from
September 28, 2026 08:11
a88c17a to
c406cdb
Compare
…bers as numbers
The TOML reader in loaders/toml.js took whatever it was handed and guessed: a lone
surrogate or an invalid escape passed through, an unterminated string or inline table
was read up to wherever it happened to stop, text after a value was dropped, a missing
comma lost an item, a bare word was a string, a duplicate key or table silently won,
and every number came back as its text.
It now reads TOML 1.0 whole -- keys, the four kinds of string with their escapes
checked, numbers (integer, float, radix, underscores, inf/nan; an integer past 2^53 as
a BigInt), booleans, dates and times (checked, kept as text), arrays, inline tables,
table and array-table headers, and the rules on defining a key or table twice -- and
throws a TomlError naming the file and line on anything else. The API is one function,
tomlEntries(text, { file }): the headers and pairs in order, each with the table path it
lands at and its physical line span.
Cargo.toml and Cargo.lock are read by table path, so `[dependencies.foo] features`,
`[dependencies] foo.features` and `foo = { features }` are one thing, and a
`[workspace.*]` table alone marks a workspace root, as cargo sees it. Readers pass the
file name for the messages: the manifest's project-relative path, `Cargo.lock`, a
foundry.toml's path. A carried `--manifests` config that can't be read is refused
rather than carried unredacted.
Checked against the toml-test 1.0.0 suite (703 of 709 right; the six left are invalid
UTF-8 byte cases a decoded string never holds) and, differentially against Python's
tomllib, the 132 Cargo.toml/Cargo.lock/*.toml files of a vendored corpus (all agree).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Sep 29, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, a filtered cargo config and build scripts, less the credentials in their URLs; - vendored crates are held to their package through symlinks and #[path]. Rebased onto the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling), and the config filter keeps each carried entry as written less its comments. A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Sep 29, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
ChALkeR
pushed a commit
that referenced
this pull request
Sep 29, 2026
Adds @preventive/lockfile 1.0.0-alpha.1 and reads Cargo.toml, Cargo.lock and foundry.toml with its strict TOML parser (parseToml) in place of the reader loaders/toml.js carried since #179. The readers walk the parsed table tree: the dotted, inline and header spellings of a table are one table by construction. loaders/toml.js keeps readToml (the parser, with the file named in its errors), isTomlTable and the bracket helpers the Rust scanner shares. The parser refuses what those files are never written in, on top of what isn't TOML: local dates and times, a byte order mark, U+FFFD where a lenient decoder replaced bytes, a dotted key through a table a header made (Cargo's toml crate refuses it too). A foundry.toml profile's sub-tables are its values whichever way they are written, so a `no-collision` extends sees a `[profile.default.fuzz]` table as forge does. Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL Co-authored-by: Claude <noreply@anthropic.com>
exo-nikita
pushed a commit
that referenced
this pull request
Sep 29, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Oct 1, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Oct 1, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Oct 1, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita
pushed a commit
that referenced
this pull request
Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
ChALkeR
pushed a commit
that referenced
this pull request
Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests (#175) * fix(bundle): Rust loader follow-ups: lexer offsets, dead items, paths through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests Follow-ups to the Rust loader from #173, squashed from the review rounds of this PR: - the lexer's masked view stays aligned after astral characters; - a dead item is skipped whole whatever commas its generics hold; - paths resolve through imports, re-exports, globs and exported macros, with a memoized per-crate import table, textual macro scope and the extern prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off); - include!, include_str!/include_bytes! and #[doc = include_str!] are carried; #[path] on an inline module is honoured; - --cargo-target decides target cfgs from rustc (run from the home dir, $RUSTC honoured, no auto-install); - --cargo-manifests carries manifests, the lockfile, the cargo config and build scripts, each as written (as #182 carries --manifests files): a file that isn't UTF-8 text is refused, not altered; - vendored crates are held to their package through symlinks and #[path]. On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo config are read by table path (`[package] build`, `edition.workspace`, target-specific dependency tables, every `[patch]` spelling). A Cargo or Foundry file that exists but isn't TOML stops the build, naming the file and line; a dependency's foundry.toml the loader can't read is no longer skipped with a warning but stops the build too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): Rust loader review 7: compile units, honest ranking, template mods - Refuse a Rust source or include_str! file that isn't UTF-8 (and a non-UTF-8 manifest); carry an include_bytes! file as base64. - Compile units: code built for the host (build scripts, build-dependencies, proc-macro crates) is scanned against the host's cfgs and, under resolver 2, the host's own feature context; a crate built both ways is followed as both. Features are on, maybe (only an undecided target table enables them) or off. - --cargo-manifests carries a vendored crate's Cargo.toml and .cargo-checksum.json, and a project package's configs up to the bundle root; the vendored directory follows the replace-with chain whatever its name. - Custom cfgs a build script prints or rustflags set are not presumed off; cfg_x! gates take the cfg their macro_rules! definition writes. - Ambiguous path edges record a cfg-keyed map of their candidates instead of the first written; candidates the build rules out come last; a child module under a custom or dead cfg gives way to an import of its name. - Macro scope: a template's bare calls are made where it is invoked; a mod a template declares belongs to each in-package invoking module and is found beside it (libc's prelude!() -> src/types.rs), else beside the definition. - `extern crate self as x;` is in the extern prelude; a declared dependency is reported missing whatever a glob into a missing crate may provide. - Vendored versions come from the lock, else the one that fits, else a warning and no guess; out-of-root path/[patch] deps never bind to vendored copies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): decide a cfg an unknown leaf repeats in when it can't vary evalCfg is three-valued per leaf, so an item's cfg joined with its enclosing blocks' -- zerocopy's `#[cfg(any(test, kani))] mod tests { #[cfg(not(kani))] mod compatibility { use rand::…; } }`, i.e. all(any(test, kani), not(kani)) -- stayed undecided and its `use rand` was reported as a missing crate. When an unknown leaf occurs more than once (up to six of them), the predicate is evaluated under each assignment and decided when every one agrees. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): rank a module's own items with its imports, and imports that lead out A path's segment was looked up in the module's imports first and its own definitions only when no import answered, so an import leading out of the bundle (tokio's `pub(crate) use std::sync::atomic::AtomicU64;` in one `imp` variant) hid the mutex-based `AtomicU64` the other variant defines, and a dead import (`cfg_taskdump!`'s) beat the live fn under `cfg_not_taskdump!`. The module's items are now candidates right after its named imports, ahead of globs, and an import leading out of the bundle is an answer under its cfgs rather than a last resort: `imp::AtomicU64` maps both variant files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * perf(bundle): cache leaf verdicts, alternative keys and package lookups in the tree pass The cfg-keyed alternatives and the build-deadness check added in review 7 rebuilt the same cfg texts and re-parsed the same leaf predicates for every candidate: libc's tree pass went from 227ms to 351ms. A leaf's verdict is kept per interned build, an alternative's cfg text per asker set, and a file's package per tree pass (asked for every import and item). libc 282ms, tokio 492ms -> 464ms (589afce: 444ms); output unchanged on the corpus. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): Rust loader review 8: refused roots, missing crates, per-table deps, resolver regressions - buildRustBundle: a crate root the tree pass wants but the walk refused is asked for once, so the walk/tree loop ends; every crate the bundle lacks is reported (refused roots included) whether or not a vendor dir exists, the `cargo vendor` hint only without one. - Cargo: each dependency table is its own dependency (version, path, package, workspace per table), and a file follows the one its target uses (build script, test target, normal); a Cargo.lock pin is taken only when the requirement allows it, else the lock is reported out of date; a [patch] is used only when its version fits, and [patch] in .cargo/config.toml (nearest first, ahead of the manifest's) is read; a path dependency or patch outside the bundle root, or naming no package, is reported. - Resolver: a local module under a custom cfg gives way only to the module's own `use` or item, never to a glob; a macro call is looked up among macros (a `fn m`, `mod m` or an import of a module/crate is no `m!`); a template's calls and includes stay with it past a nested `macro_rules!`; cfg values keep their spacing inside quotes, so `"a b"` and `"ab"` are not merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * feat(bundle): read Cargo through @preventive/lockfile, and resolve with cargo's resolver where it can - Cargo.toml, Cargo.lock and the cargo config's [patch] are read by @preventive/lockfile 1.0.0-alpha.3 (parseCargoManifest, parseCargoLock, parseCargoConfig): what cargo refuses, or what the reader can't tell how cargo reads, stops the build naming the file; a Cargo.lock older than version 3 is refused. Version requirements are matched by its rust-semver.js. The loader's own manifest and lock readers and semver matcher are gone. - With a lockfile, --cargo-target and every locked package in-tree (path packages inside the bundle root, the rest vendored with checksums), features come from cargo's resolver: linkCargo over the manifests, then resolveCargoFeatures for `cargo build -p <the entries' packages>`; a lockfile out of date or a vendored copy whose checksum isn't the lockfile's stops the build. An unknown host is resolved both ways (its target tables maybe). - Otherwise the manifests are replayed as before, with fixes: a git dependency takes a git checkout's vendored copy and a registry one a registry's (by .cargo-checksum.json and the lockfile's sources); `dep/feat` turns on the package's feature of that name, written or implicit; a proc-macro entry is resolved and scanned for the host; prereleases follow the semver crate; the lockfile is the entries' workspace's, below the bundle root too. - cfg(true), cfg(false) and raw identifiers in cfgs. - Tests: cargo's recorded builds of @preventive/lockfile's workspace fixture (PreventiveMeasures/libraries#56, MIT) through createCargoContext, the refusals, the fallback, and each fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * refactor(bundle): Rust loader simplification pass - scanRustItems: one cfg verdict per predicate per scan, and the open scopes' conjunction kept as they open and close, rather than joined and evaluated again at every token (a 220 KB file inside one #[cfg] block: 588 ms -> 36 ms; with two compile units, 1.3 s -> 28 ms). - cfg leaves: leafFalse evaluates a leaf by key and value (evalCfgKey) instead of rebuilding its text for evalCfg; leavesOfCfgs uses cfgLeaves; joinCfgs where it was inlined; one cache helper for the per-asker verdicts. - One walk driver (collectRustBundle) for buildRustBundle and loadRust: build scripts, compile units and the wantedRoots loop live in rust.js; the link-out-of-package check (withinRealDir), boundaryOf and addUnits are shared rather than copied into bundle.js. - cargo.js: each path dependency's directory resolved once, against the workspace root its inheritance was read from (depSpec gone); requests carry kind and target (no `kind@target` parsing); resolver 1's single context folded into nodeKey; one [patch] source list; the git/registry flag stored with the checksum; tableOf's TOML reused for the manifest; build script and build files memoized per package; one error-naming helper shared with toml.js; internals no longer exported. - rust.js: the macro-template helpers shared by the walk and the tree pass (keyed by the package's own template set, not its identity); providedFrom's per-module file set computed once; resolveModDecl's unused cfg branch removed. - The Rust-only flag checks in stasis.js and bundle.js table-driven. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * fix(bundle): Rust loader review 10: platform tables, build-script crates, proc-macro roots, out-of-root workspaces Crate resolution: - A dependency name is resolved from the tables the asking code links: per crate root the file is compiled in (a build script's modules and a file it shares with the lib through #[path] take the [build-dependencies]; tests, benches and examples the dev ones too), and only tables for the platforms the code is compiled for. Tables that may each apply (no target to decide them, or two roles) give cfg-keyed alternatives, each carried. Both on the replay and on cargo's resolver. - A crate's lib-name fallback resolves only dependencies that may have that lib name: `u8::MAX` no longer resolves every declared dependency and warns about versions. - A declared dependency the bundle lacks is reported when only an expression names it (`serde_json::to_string(…)`). - Macros of other crates get edges: `use dep::mac;` (an alias, a prelude glob too), `dep::mac!(…)` and `#[macro_use] extern crate`. Cargo: - The replay seeds a proc-macro root in the target context too, as cargo activates a member it is asked to build (fixes a regression from f0287e0). - A member bundled from its own directory reads its workspace root above the bundle root (never bundled) for what it inherits and the resolver, when that root's members take it. - [patch] tables apply to their own source only: a git URL's not to the crates.io dependency. - The vendor directory comes from the cargo config nearest the entries (a nested workspace's own). - `stasis bundle` says when the features come from the replay and why; a vendored copy the lockfile doesn't list no longer forces the replay. EXODUS_STASIS_DEBUG labels the mode and prints both contexts. - A vendored file whose bytes aren't those .cargo-checksum.json lists stops the bundle, as cargo refuses to build it. cfgs and resolution: - `r#true` / `r#false` are custom cfgs, not the literals, in every reader. - A child module under a custom cfg gives way to a glob of its name, as any doubtful candidate does. - An include in a nested macro_rules! is that macro's, resolved where it is invoked. - A refused or missing crate root is warned about once. - Dense cycles of cfg-gated globs no longer blow up: leaf keys are cached, absorbed alternatives dropped, and alternatives capped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL * test(bundle): hold the Rust loader's new fixture projects in one tests/fixtures/rust-bundle.json.br `includes` and `cargo-recorded` (30 files, 44 KB) become one brotli-compressed JSON file (10 KB): project name -> project-relative path -> the file's text, or `{ base64 }` for one that isn't UTF-8. tests/rust-fixtures.helper.js writes a project out once per test process, to a temporary directory removed when the process exits, and `rustFixture(name)` gives its path; `node tests/rust-fixtures.helper.js unpack|pack <dir>` edits them. The fixtures main already holds stay as they are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits, split out of #175 at review's request.
1. The move (
c406cdb). The TOML reading that Cargo.toml, Cargo.lock and foundry.toml share moves out ofcargo.jsintostasis/src/loaders/toml.js, with the two bracket helpers (matchClose,splitTopLevel) the Rust scanner uses for cfg predicates and macro bodies. A pure move; the new file is in the package'sfiles.2. Strictness (review: "the reader silently misreads invalid TOML"). The reader took whatever it was handed and guessed. It now reads TOML 1.0 whole and throws a
TomlErrornaming the file and line on anything else. The review's table:"\uD800""\uD800" is not a Unicode scalar value"a\qb","a\eb"aqb,aebinvalid escape "\q"/"\e""abc"abcas textunterminated string"a" junkaunexpected text after the value["a" "b"]["a"]expected a comma or "]" after the array item["a", b]["a", "b"]invalid value "b"{ a = 1, a = 2 }duplicate key "t.a"{ a = "x"{ a: '"x' }unterminated inline table[a]twiceduplicate table [a](at the second)"abcat the end of the fileunterminated string1_000,0x1f,+inf,[1, "a"],version = 31000,31,Infinity,[1, 'a'],3Also: leading zeros,
1__0,1.,.5,-0x1,True, impossible dates, control characters in strings and comments, a lone CR, a trailing comma in an inline table, a table header after dotted keys or an inline table defined it,[[x]]over a table and[x]over an array of tables, text after a header.Values are JavaScript ones: numbers (an integer past 2^53 is a BigInt, so a foundry
gas_limit = 9223372036854775807stays exact), booleans, strings (all four kinds), arrays, inline tables (null-prototype objects, so a key is only a key), dates and times checked and kept as their text. A leading byte-order mark is skipped, as cargo and forge do.The API is one function:
tomlEntries(text, { file })→ the document's[header]s andkey = valuepairs in order, each with the table path it lands at and its physical line span. The line-based helpers (logicalLines,TABLE_HEADER_RE,KEY_VALUE_RE,parseTomlValue,splitTomlKey) are gone.parseCargoManifestandparseCargoLockread by table path, so[dependencies.foo] features = […],[dependencies] foo.features = […]andfoo = { features = […] }are one thing, and a[workspace.*]table alone marks a workspace root (as cargo sees it). Readers pass the file name: the manifest's project-relative path (crates/app/Cargo.toml:3: unterminated stringcomes out ofcreateCargoContext),Cargo.lock, a foundry.toml's path; a--manifestsconfig that can't be read is refused rather than carried unredacted.Verification:
tomllib: the 132 Cargo.toml / Cargo.lock / *.toml files of a vendored corpus (tokio, mio, serde, regex, libc, …) all agree; of 158 hand-written edge cases all agree but the BOM (accepted here) and CRLF inside a multi-line string (kept as written; tomllib normalizes).oxlintclean; full suite 2030 pass, 0 fail, 3 skipped (the cargo/rustc-dependent ones). Newtests/toml.test.jscovers every row above plus numbers, strings, dates, spans and the redefinition rules; the cargo and foundry tests cover the file:line messages end to end. One fixture changed: theredactFoundryTomltest definedrpc_endpointsboth inline and as a header, which is not TOML.#175 is independent (the same move taken back out); whichever merges second needs a mechanical rebase: the import lines, and #175's
.cargo/configreaders ontotomlEntries.🤖 Generated with Claude Code
https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
Generated by Claude Code