Skip to content

fix(bundle): a strict TOML reader in loaders/toml.js - #179

Merged
ChALkeR merged 2 commits into
mainfrom
claude/toml-reader-02j8b2
Sep 29, 2026
Merged

ChALkeR merged 2 commits into
mainfrom
claude/toml-reader-02j8b2

Conversation

@exo-nikita

@exo-nikita exo-nikita commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Two commits, split out of #175 at review's request.

1. The move (c406cdb). The TOML reading that Cargo.toml, Cargo.lock and foundry.toml share moves out of cargo.js into stasis/src/loaders/toml.js, with the two bracket helpers (matchClose, splitTopLevel) the Rust scanner uses for cfg predicates and macro bodies. A pure move; the new file is in the package's files.

2. Strictness (review: "the reader silently misreads invalid TOML"). The reader took whatever it was handed and guessed. It now reads TOML 1.0 whole and throws a TomlError naming the file and line on anything else. The review's table:

input before now
"\uD800" a lone surrogate "\uD800" is not a Unicode scalar value
"a\qb", "a\eb" aqb, aeb invalid escape "\q" / "\e"
"abc "abc as text unterminated string
"a" junk a unexpected text after the value
["a" "b"] ["a"] expected a comma or "]" after the array item
["a", b] ["a", "b"] invalid value "b"
{ a = 1, a = 2 } last wins duplicate key "t.a"
{ a = "x" { a: '"x' } unterminated inline table
[a] twice merged duplicate table [a] (at the second)
"abc at the end of the file accepted unterminated string
1_000, 0x1f, +inf, [1, "a"], version = 3 their text 1000, 31, Infinity, [1, 'a'], 3

Also: leading zeros, 1__0, 1., .5, -0x1, True, impossible dates, control characters in strings and comments, a lone CR, a trailing comma in an inline table, a table header after dotted keys or an inline table defined it, [[x]] over a table and [x] over an array of tables, text after a header.

Values are JavaScript ones: numbers (an integer past 2^53 is a BigInt, so a foundry gas_limit = 9223372036854775807 stays exact), booleans, strings (all four kinds), arrays, inline tables (null-prototype objects, so a key is only a key), dates and times checked and kept as their text. A leading byte-order mark is skipped, as cargo and forge do.

The API is one function: tomlEntries(text, { file }) → the document's [header]s and key = value pairs in order, each with the table path it lands at and its physical line span. The line-based helpers (logicalLines, TABLE_HEADER_RE, KEY_VALUE_RE, parseTomlValue, splitTomlKey) are gone. parseCargoManifest and parseCargoLock read by table path, so [dependencies.foo] features = […], [dependencies] foo.features = […] and foo = { features = […] } are one thing, and a [workspace.*] table alone marks a workspace root (as cargo sees it). Readers pass the file name: the manifest's project-relative path (crates/app/Cargo.toml:3: unterminated string comes out of createCargoContext), Cargo.lock, a foundry.toml's path; a --manifests config that can't be read is refused rather than carried unredacted.

Verification:

  • toml-test 1.0.0 file list: 703 of 709 right; the six left are the invalid-UTF-8 byte cases, which a decoded string never holds (the bundle checks UTF-8 before reading).
  • Differential against Python 3.11's tomllib: the 132 Cargo.toml / Cargo.lock / *.toml files of a vendored corpus (tokio, mio, serde, regex, libc, …) all agree; of 158 hand-written edge cases all agree but the BOM (accepted here) and CRLF inside a multi-line string (kept as written; tomllib normalizes).
  • 781 KiB of corpus manifests and locks read in 40 ms; a 4.5 MiB synthetic Cargo.lock in 0.3 s.
  • oxlint clean; full suite 2030 pass, 0 fail, 3 skipped (the cargo/rustc-dependent ones). New tests/toml.test.js covers every row above plus numbers, strings, dates, spans and the redefinition rules; the cargo and foundry tests cover the file:line messages end to end. One fixture changed: the redactFoundryToml test defined rpc_endpoints both inline and as a header, which is not TOML.

#175 is independent (the same move taken back out); whichever merges second needs a mechanical rebase: the import lines, and #175's .cargo/config readers onto tomlEntries.

🤖 Generated with Claude Code

https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL


Generated by Claude Code

The TOML reading that Cargo.toml, Cargo.lock, .cargo/config and foundry.toml
share -- logical lines, dotted keys, values, the entry stream, and the
bracket helpers the Rust scanner uses too -- moves out of cargo.js into
toml.js, as is: no new cases. cargo.js keeps the Cargo semantics on top of
it and the path helper; rust.js and foundry.js import what they used from
the new module.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
@exo-nikita
exo-nikita force-pushed the claude/toml-reader-02j8b2 branch from a88c17a to c406cdb Compare September 28, 2026 08:11
…bers as numbers

The TOML reader in loaders/toml.js took whatever it was handed and guessed: a lone
surrogate or an invalid escape passed through, an unterminated string or inline table
was read up to wherever it happened to stop, text after a value was dropped, a missing
comma lost an item, a bare word was a string, a duplicate key or table silently won,
and every number came back as its text.

It now reads TOML 1.0 whole -- keys, the four kinds of string with their escapes
checked, numbers (integer, float, radix, underscores, inf/nan; an integer past 2^53 as
a BigInt), booleans, dates and times (checked, kept as text), arrays, inline tables,
table and array-table headers, and the rules on defining a key or table twice -- and
throws a TomlError naming the file and line on anything else. The API is one function,
tomlEntries(text, { file }): the headers and pairs in order, each with the table path it
lands at and its physical line span.

Cargo.toml and Cargo.lock are read by table path, so `[dependencies.foo] features`,
`[dependencies] foo.features` and `foo = { features }` are one thing, and a
`[workspace.*]` table alone marks a workspace root, as cargo sees it. Readers pass the
file name for the messages: the manifest's project-relative path, `Cargo.lock`, a
foundry.toml's path. A carried `--manifests` config that can't be read is refused
rather than carried unredacted.

Checked against the toml-test 1.0.0 suite (703 of 709 right; the six left are invalid
UTF-8 byte cases a decoded string never holds) and, differentially against Python's
tomllib, the 132 Cargo.toml/Cargo.lock/*.toml files of a vendored corpus (all agree).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
@exo-nikita exo-nikita changed the title refactor(bundle): move the TOML subset reader into loaders/toml.js fix(bundle): a strict TOML reader in loaders/toml.js Sep 28, 2026
@ChALkeR
ChALkeR merged commit abd3b3e into main Sep 29, 2026
5 checks passed
exo-nikita pushed a commit that referenced this pull request Sep 29, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, a filtered cargo config
  and build scripts, less the credentials in their URLs;
- vendored crates are held to their package through symlinks and #[path].

Rebased onto the strict TOML reader of #179: Cargo.toml, Cargo.lock and the
cargo config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling), and the config
filter keeps each carried entry as written less its comments. A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita pushed a commit that referenced this pull request Sep 29, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
ChALkeR pushed a commit that referenced this pull request Sep 29, 2026
Adds @preventive/lockfile 1.0.0-alpha.1 and reads Cargo.toml, Cargo.lock and
foundry.toml with its strict TOML parser (parseToml) in place of the reader
loaders/toml.js carried since #179. The readers walk the parsed table tree:
the dotted, inline and header spellings of a table are one table by
construction. loaders/toml.js keeps readToml (the parser, with the file named
in its errors), isTomlTable and the bracket helpers the Rust scanner shares.

The parser refuses what those files are never written in, on top of what
isn't TOML: local dates and times, a byte order mark, U+FFFD where a lenient
decoder replaced bytes, a dotted key through a table a header made (Cargo's
toml crate refuses it too). A foundry.toml profile's sub-tables are its values
whichever way they are written, so a `no-collision` extends sees a
`[profile.default.fuzz]` table as forge does.


Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

Co-authored-by: Claude <noreply@anthropic.com>
exo-nikita pushed a commit that referenced this pull request Sep 29, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita pushed a commit that referenced this pull request Oct 1, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita pushed a commit that referenced this pull request Oct 1, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita pushed a commit that referenced this pull request Oct 1, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita pushed a commit that referenced this pull request Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita pushed a commit that referenced this pull request Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
exo-nikita pushed a commit that referenced this pull request Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL
ChALkeR pushed a commit that referenced this pull request Oct 2, 2026
… through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests (#175)

* fix(bundle): Rust loader follow-ups: lexer offsets, dead items, paths through re-exports and macros, include!, review fixes, --cargo-target, --cargo-manifests

Follow-ups to the Rust loader from #173, squashed from the review rounds of
this PR:

- the lexer's masked view stays aligned after astral characters;
- a dead item is skipped whole whatever commas its generics hold;
- paths resolve through imports, re-exports, globs and exported macros, with
  a memoized per-crate import table, textual macro scope and the extern
  prelude, cfg-aware (`any(…)` leaves, gate macros, custom cfgs presumed off);
- include!, include_str!/include_bytes! and #[doc = include_str!] are carried;
  #[path] on an inline module is honoured;
- --cargo-target decides target cfgs from rustc (run from the home dir,
  $RUSTC honoured, no auto-install);
- --cargo-manifests carries manifests, the lockfile, the cargo config and
  build scripts, each as written (as #182 carries --manifests files): a file
  that isn't UTF-8 text is refused, not altered;
- vendored crates are held to their package through symlinks and #[path].

On the strict TOML reader of #179: Cargo.toml, Cargo.lock and the cargo
config are read by table path (`[package] build`, `edition.workspace`,
target-specific dependency tables, every `[patch]` spelling). A Cargo or
Foundry file that exists but isn't TOML stops the build, naming the file and
line; a dependency's foundry.toml the loader can't read is no longer skipped
with a warning but stops the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* fix(bundle): Rust loader review 7: compile units, honest ranking, template mods

- Refuse a Rust source or include_str! file that isn't UTF-8 (and a non-UTF-8
  manifest); carry an include_bytes! file as base64.
- Compile units: code built for the host (build scripts, build-dependencies,
  proc-macro crates) is scanned against the host's cfgs and, under resolver 2,
  the host's own feature context; a crate built both ways is followed as both.
  Features are on, maybe (only an undecided target table enables them) or off.
- --cargo-manifests carries a vendored crate's Cargo.toml and
  .cargo-checksum.json, and a project package's configs up to the bundle root;
  the vendored directory follows the replace-with chain whatever its name.
- Custom cfgs a build script prints or rustflags set are not presumed off;
  cfg_x! gates take the cfg their macro_rules! definition writes.
- Ambiguous path edges record a cfg-keyed map of their candidates instead of
  the first written; candidates the build rules out come last; a child module
  under a custom or dead cfg gives way to an import of its name.
- Macro scope: a template's bare calls are made where it is invoked; a mod a
  template declares belongs to each in-package invoking module and is found
  beside it (libc's prelude!() -> src/types.rs), else beside the definition.
- `extern crate self as x;` is in the extern prelude; a declared dependency is
  reported missing whatever a glob into a missing crate may provide.
- Vendored versions come from the lock, else the one that fits, else a warning
  and no guess; out-of-root path/[patch] deps never bind to vendored copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* fix(bundle): decide a cfg an unknown leaf repeats in when it can't vary

evalCfg is three-valued per leaf, so an item's cfg joined with its enclosing
blocks' -- zerocopy's `#[cfg(any(test, kani))] mod tests { #[cfg(not(kani))]
mod compatibility { use rand::…; } }`, i.e. all(any(test, kani), not(kani)) --
stayed undecided and its `use rand` was reported as a missing crate. When an
unknown leaf occurs more than once (up to six of them), the predicate is
evaluated under each assignment and decided when every one agrees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* fix(bundle): rank a module's own items with its imports, and imports that lead out

A path's segment was looked up in the module's imports first and its own
definitions only when no import answered, so an import leading out of the
bundle (tokio's `pub(crate) use std::sync::atomic::AtomicU64;` in one `imp`
variant) hid the mutex-based `AtomicU64` the other variant defines, and a
dead import (`cfg_taskdump!`'s) beat the live fn under `cfg_not_taskdump!`.
The module's items are now candidates right after its named imports, ahead of
globs, and an import leading out of the bundle is an answer under its cfgs
rather than a last resort: `imp::AtomicU64` maps both variant files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* perf(bundle): cache leaf verdicts, alternative keys and package lookups in the tree pass

The cfg-keyed alternatives and the build-deadness check added in review 7
rebuilt the same cfg texts and re-parsed the same leaf predicates for every
candidate: libc's tree pass went from 227ms to 351ms. A leaf's verdict is kept
per interned build, an alternative's cfg text per asker set, and a file's
package per tree pass (asked for every import and item). libc 282ms, tokio
492ms -> 464ms (589afce: 444ms); output unchanged on the corpus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* fix(bundle): Rust loader review 8: refused roots, missing crates, per-table deps, resolver regressions

- buildRustBundle: a crate root the tree pass wants but the walk refused is
  asked for once, so the walk/tree loop ends; every crate the bundle lacks is
  reported (refused roots included) whether or not a vendor dir exists, the
  `cargo vendor` hint only without one.
- Cargo: each dependency table is its own dependency (version, path, package,
  workspace per table), and a file follows the one its target uses (build
  script, test target, normal); a Cargo.lock pin is taken only when the
  requirement allows it, else the lock is reported out of date; a [patch] is
  used only when its version fits, and [patch] in .cargo/config.toml (nearest
  first, ahead of the manifest's) is read; a path dependency or patch outside
  the bundle root, or naming no package, is reported.
- Resolver: a local module under a custom cfg gives way only to the module's
  own `use` or item, never to a glob; a macro call is looked up among macros
  (a `fn m`, `mod m` or an import of a module/crate is no `m!`); a template's
  calls and includes stay with it past a nested `macro_rules!`; cfg values
  keep their spacing inside quotes, so `"a b"` and `"ab"` are not merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* feat(bundle): read Cargo through @preventive/lockfile, and resolve with cargo's resolver where it can

- Cargo.toml, Cargo.lock and the cargo config's [patch] are read by
  @preventive/lockfile 1.0.0-alpha.3 (parseCargoManifest, parseCargoLock,
  parseCargoConfig): what cargo refuses, or what the reader can't tell how
  cargo reads, stops the build naming the file; a Cargo.lock older than
  version 3 is refused. Version requirements are matched by its rust-semver.js.
  The loader's own manifest and lock readers and semver matcher are gone.
- With a lockfile, --cargo-target and every locked package in-tree (path
  packages inside the bundle root, the rest vendored with checksums), features
  come from cargo's resolver: linkCargo over the manifests, then
  resolveCargoFeatures for `cargo build -p <the entries' packages>`; a lockfile
  out of date or a vendored copy whose checksum isn't the lockfile's stops the
  build. An unknown host is resolved both ways (its target tables maybe).
- Otherwise the manifests are replayed as before, with fixes: a git
  dependency takes a git checkout's vendored copy and a registry one a
  registry's (by .cargo-checksum.json and the lockfile's sources); `dep/feat`
  turns on the package's feature of that name, written or implicit; a
  proc-macro entry is resolved and scanned for the host; prereleases follow
  the semver crate; the lockfile is the entries' workspace's, below the bundle
  root too.
- cfg(true), cfg(false) and raw identifiers in cfgs.
- Tests: cargo's recorded builds of @preventive/lockfile's workspace fixture
  (PreventiveMeasures/libraries#56, MIT) through createCargoContext, the
  refusals, the fallback, and each fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* refactor(bundle): Rust loader simplification pass

- scanRustItems: one cfg verdict per predicate per scan, and the open
  scopes' conjunction kept as they open and close, rather than joined and
  evaluated again at every token (a 220 KB file inside one #[cfg] block:
  588 ms -> 36 ms; with two compile units, 1.3 s -> 28 ms).
- cfg leaves: leafFalse evaluates a leaf by key and value (evalCfgKey)
  instead of rebuilding its text for evalCfg; leavesOfCfgs uses cfgLeaves;
  joinCfgs where it was inlined; one cache helper for the per-asker
  verdicts.
- One walk driver (collectRustBundle) for buildRustBundle and loadRust:
  build scripts, compile units and the wantedRoots loop live in rust.js;
  the link-out-of-package check (withinRealDir), boundaryOf and addUnits
  are shared rather than copied into bundle.js.
- cargo.js: each path dependency's directory resolved once, against the
  workspace root its inheritance was read from (depSpec gone); requests
  carry kind and target (no `kind@target` parsing); resolver 1's single
  context folded into nodeKey; one [patch] source list; the git/registry
  flag stored with the checksum; tableOf's TOML reused for the manifest;
  build script and build files memoized per package; one error-naming
  helper shared with toml.js; internals no longer exported.
- rust.js: the macro-template helpers shared by the walk and the tree
  pass (keyed by the package's own template set, not its identity);
  providedFrom's per-module file set computed once; resolveModDecl's
  unused cfg branch removed.
- The Rust-only flag checks in stasis.js and bundle.js table-driven.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* fix(bundle): Rust loader review 10: platform tables, build-script crates, proc-macro roots, out-of-root workspaces

Crate resolution:
- A dependency name is resolved from the tables the asking code links:
  per crate root the file is compiled in (a build script's modules and a
  file it shares with the lib through #[path] take the
  [build-dependencies]; tests, benches and examples the dev ones too),
  and only tables for the platforms the code is compiled for. Tables
  that may each apply (no target to decide them, or two roles) give
  cfg-keyed alternatives, each carried. Both on the replay and on
  cargo's resolver.
- A crate's lib-name fallback resolves only dependencies that may have
  that lib name: `u8::MAX` no longer resolves every declared dependency
  and warns about versions.
- A declared dependency the bundle lacks is reported when only an
  expression names it (`serde_json::to_string(…)`).
- Macros of other crates get edges: `use dep::mac;` (an alias, a
  prelude glob too), `dep::mac!(…)` and `#[macro_use] extern crate`.

Cargo:
- The replay seeds a proc-macro root in the target context too, as
  cargo activates a member it is asked to build (fixes a regression
  from f0287e0).
- A member bundled from its own directory reads its workspace root
  above the bundle root (never bundled) for what it inherits and the
  resolver, when that root's members take it.
- [patch] tables apply to their own source only: a git URL's not to
  the crates.io dependency.
- The vendor directory comes from the cargo config nearest the
  entries (a nested workspace's own).
- `stasis bundle` says when the features come from the replay and
  why; a vendored copy the lockfile doesn't list no longer forces the
  replay. EXODUS_STASIS_DEBUG labels the mode and prints both contexts.
- A vendored file whose bytes aren't those .cargo-checksum.json lists
  stops the bundle, as cargo refuses to build it.

cfgs and resolution:
- `r#true` / `r#false` are custom cfgs, not the literals, in every reader.
- A child module under a custom cfg gives way to a glob of its name,
  as any doubtful candidate does.
- An include in a nested macro_rules! is that macro's, resolved where
  it is invoked.
- A refused or missing crate root is warned about once.
- Dense cycles of cfg-gated globs no longer blow up: leaf keys are
  cached, absorbed alternatives dropped, and alternatives capped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

* test(bundle): hold the Rust loader's new fixture projects in one tests/fixtures/rust-bundle.json.br

`includes` and `cargo-recorded` (30 files, 44 KB) become one brotli-compressed
JSON file (10 KB): project name -> project-relative path -> the file's text, or
`{ base64 }` for one that isn't UTF-8. tests/rust-fixtures.helper.js writes a
project out once per test process, to a temporary directory removed when the
process exits, and `rustFixture(name)` gives its path; `node
tests/rust-fixtures.helper.js unpack|pack <dir>` edits them. The fixtures
main already holds stay as they are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nbp2vC4jLPbGN8cmorxAL

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants