Skip to content

Preserve symlinks when staging, and link node_modules into web builds - #15

Merged
arcaputo3 merged 3 commits into
agent/idiomatic-scalajs-mill-workflowsfrom
agent/bun-symlink-staging
Aug 26, 2026
Merged

arcaputo3 merged 3 commits into
agent/idiomatic-scalajs-mill-workflowsfrom
agent/bun-symlink-staging

Conversation

@arcaputo3

Copy link
Copy Markdown
Contributor

Stack 2/N. Base is agent/bun-140-toolchain (#14), not main.

The bug

BunScalaJSWebModule could not resolve any npm dependency.

Every tree copy in the plugin walks with os.walk (followLinks = false) but branches on os.isDir / os.copy (followLinks = true). A node_modules symlink — which ensureLinkedWorkspace plants in the link report — is therefore misclassified as a directory:

  • BunWebSupport.copyContents called os.makeDir.all, producing an empty node_modules in the staged web build. bun build index.html then had nothing to resolve @JSImported packages against.
  • BunToolchainModule.copyWorkspace called os.copy.over, which follows the link and deep-copies the entire resolved tree — O(node_modules) on every compileExecutable, bundle, and test workspace — and throws outright on any broken link, which is routine for the .bin shims of skipped optional dependencies.

Both verified with standalone os-lib repros before fixing.

The fix

BunToolchainModule.copyTree recreates symlinks instead of resolving them, and dispatches on os.isLink before os.isDir(followLinks = false). copyWorkspace, copyPathRefs, and both BunWebSupport copiers route through it.

BunScalaJSWebModule now stages node_modules explicitly — mirroring BunTypeScriptWebModule.prepareWebStage, which already did this and was correct — rather than hoping the link survives a copy.

Staging tasks now declare the install they link into. Mill's filesystem checker rejects reading a dest the task doesn't depend on, so recreating the symlink surfaced a dependency that was always real but never declared. The old deep-copy slipped past the checker only because os.copy checks the source path (inside the link report, a legal read) rather than the resolved target. Affected: BunScalaJSModule.compileExecutable/compileExecutables, BunTypeScriptModule.bundle/compileExecutable/compileExecutables, BunWorkersModule.bundleWorkers.

Also in scope, same files:

  • htmlEntries is split into a pure resolver and materializeHtmlEntries. dev() and bundle called the writing version against a staging task's already-cached dest.
  • BunTypeScriptWebModule.webDevelopmentStage and webProductionStage were byte-for-byte identical; collapsed into one webStage, so a production build no longer runs the same staging work twice.

Proof the test catches it

scalajs-web had no npm dependencies — which is precisely why CI was blind. It now imports lodash through @JSImport, so the linked output carries a real npm import.

Verified by stashing the source fix and running the fixture against the parent commit:

X BunScalaJSIntegrationTests.web bundle includes HTML CSS and JavaScript
X BunScalaJSIntegrationTests.web stage resolves npm dependencies
  os.SubprocessException: Result of .../bun...: 1

With the fix, both pass, stage/node_modules is a link, and lodash is inlined into the bundle rather than left as a bare import.

Verification

  • Unit: 67 passing (was 57). New CopyTreeTests covers symlinked directories, broken links, exclusion, symlinked files, and empty directories; BunWebSupportTests covers htmlEntries purity and single-materialization.
  • Integration: 37 passing.

🤖 Generated with Claude Code

arcaputo3 and others added 2 commits August 26, 2026 14:44
`BunScalaJSWebModule` could not resolve any npm dependency. Every tree copy
in the plugin walked with `os.walk` (followLinks=false) but branched on
`os.isDir`/`os.copy` (followLinks=true), so the `node_modules` symlink that
`ensureLinkedWorkspace` plants in the link report was misclassified:

- `BunWebSupport.copyContents` turned it into an empty directory, so
  `bun build index.html` had nothing to resolve imports against.
- `BunToolchainModule.copyWorkspace` deep-copied the entire resolved tree --
  O(node_modules) on every compileExecutable, bundle, and test workspace --
  and threw outright on any broken link, routine for the .bin shims of
  skipped optional dependencies.

Add `BunToolchainModule.copyTree`, which recreates links instead of
resolving them, and route copyWorkspace, copyPathRefs, and both
BunWebSupport copiers through it.

`BunScalaJSWebModule` now stages `node_modules` explicitly, mirroring
`BunTypeScriptWebModule.prepareWebStage`, rather than hoping the link
survives the copy.

Staging tasks that consume the linked tree now declare the install they
symlink into. Mill's filesystem checker rejects reading a dest the task does
not depend on, and the previous deep-copy only slipped past it because
`os.copy` checks the source path rather than the resolved target -- the
dependency was always real, just undeclared.

Split `htmlEntries` into a pure resolver and `materializeHtmlEntries`, so
`dev()` and `bundle` stop writing `index.html` into a staging task's
already-cached output. Collapse `BunTypeScriptWebModule`'s two byte-identical
staging tasks into one `webStage`.

The scalajs-web fixture had no npm dependencies, which is exactly why CI was
blind to this. It now imports lodash through @jsimport; on the parent commit
that fixture fails with a bun build error, and both web tests fail.

Unit 67 passing (was 57), integration 37 passing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`BunTypeScriptTests.npmInstall` builds a package.json that is a strict
superset of the outer module's, but `bunLockfile` is not a member of Mill's
`TypeScriptTests`, so the unqualified reference resolved through Scala's
outer scope to the *enclosing* module's bun.lock. Installing a superset
package.json against that lock under --frozen-lockfile fails with:

    error: lockfile had changes, but lockfile is frozen

and there was no `test.bunLock` to generate a matching one. Any TypeScript
test module declaring a dependency the outer module lacks was broken under
the shipped default.

- `bunTestPackageJson` is now one task shared by `npmInstall` and `bunLock`,
  so the install and the lockfile cannot describe different dependency sets.
- The test trait declares its own `bunLockfile` at `<test module>/bun.lock`
  and its own `bunLock` command. Defined directly rather than by mixing in
  BunToolchainModule, which would fork bunVersion/bunExecutable into separate
  task instances and re-download managed Bun per test module.
- A test module that adds nothing now delegates to the outer install instead
  of demanding a second lockfile, so bare test modules are unaffected.
- Every Bun member in the test trait is qualified `outer.` or `this.`. The
  outer-vs-inherited split is what caused this, and only `bunLockfile` was
  ever wrong -- `transitiveUnmanagedDeps` and friends are declared by Mill's
  TypeScriptTests and correctly bind to the test module.
- `requireBunLockfile` takes the expected path, so the error names the test
  module's lockfile rather than the outer one.

The typescript-test-deps fixture (outer is-even, test is-odd) now pins
bunRequireLockfile itself, following the typescript-lock precedent, so the
strict path is exercised regardless of the suite's env. On the parent commit
both new tests fail.

Unit 67 passing, integration 39 passing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@arcaputo3
arcaputo3 deleted the branch agent/idiomatic-scalajs-mill-workflows August 26, 2026 22:11
@arcaputo3 arcaputo3 closed this Aug 26, 2026
Give TypeScript test modules their own lockfile identity
@arcaputo3 arcaputo3 reopened this Aug 26, 2026
@arcaputo3
arcaputo3 changed the base branch from agent/bun-140-toolchain to agent/idiomatic-scalajs-mill-workflows August 26, 2026 22:15
@arcaputo3
arcaputo3 merged commit 0cb7770 into agent/idiomatic-scalajs-mill-workflows Aug 26, 2026
@arcaputo3
arcaputo3 deleted the agent/bun-symlink-staging branch August 26, 2026 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant