Skip to content

perf(manifest): collect Webpack stats from the compilation graph - #5080

Open
ScriptedAlchemy wants to merge 4 commits into
mainfrom
codex/manifest-graph-foundation
Open

ScriptedAlchemy wants to merge 4 commits into
mainfrom
codex/manifest-graph-foundation

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 15, 2026 •

Copy link
Copy Markdown
Member

Description

Collect Webpack federation manifest data directly from the compilation graph instead of serializing Webpack stats and parsing module identifiers.

This is the base PR for layer support, which is stacked on this branch. This PR adds no layer API, layer metadata, runtime protocol, preview dependency, or lockfile change.

What changes

  • Read exposes, shared modules, remotes, and assets from module/chunk/code-generation data, reusing the existing normalized managers.

  • Use this path by default when the required Webpack metadata is available. Keep the previous reader when it is unavailable.

  • Add an explicit rollback option:

    new ModuleFederationPlugin({
      name: 'host',
      manifest: { useLegacyStats: true },
    });

    Omitting the option, or setting it to false, uses graph collection where supported. Native Rspack stats and their JavaScript post-processing are unchanged. The option is also documented in the English and Chinese manifest configuration guides.

  • Keep the rollback reader’s pre-PR handling of unnamed chunk groups. A real Webpack fixture with import("./other.js"), other.js importing an exposed component, and split chunks confirms that rollback excludes the unrelated other_js.js from expose assets. The graph collector explicitly retains its wider group collection.

  • Correct direct expose-to-shared relationships: an expose importing a shared module now records that share in requires, and the share records the expose in usedIn. These existing fields were empty in the old reader for the tested enhanced-sharing case. This is an intentional output correction, not a claim of byte-for-byte equivalence.

Review order

  1. packages/manifest/src/collectGraph.ts: structured collection, actual compiler-selected remote aliases, and asset association.
  2. StatsManager.ts: select the collector or old reader, then use common finalization. Removes synchronous Promise wrappers; preserves asset emission and additionalData order.
  3. Graph.spec.ts: real Webpack builds compare both emitted JSON artifacts with default, explicit graph, and rollback modes; includes filesystem-cache restoration, shared prefixes, and overlapping/unused remotes.
  4. SDK manifest option and enhanced schema. The validator changes are generated by pnpm run generate:schema.

Related Issue

Related performance reports: #3713 and #4286. This does not claim to reproduce or fix those applications' complete build times. The layers follow-up is #5039.

Validation

  • Enhanced dependency build: 15 tasks passed.
  • Manifest: 12 tests; SDK: 69; managers: 23; enhanced sharing/schema: 316; focused config cases: 5 passed. One existing SDK test is skipped; unrelated config cases were filtered.
  • Schema generation, manifest lint, root formatting, changeset validation, and normal commit hooks passed.
  • Full application/Metro E2Es and native Rspack execution were not rerun for this Webpack-only collector slice. CI results are separate from these local checks.

Latest follow-up: the foundation manifest suite passes 13/13, including the regression above against both emitted JSON files; manifest build, package lint, changed-file formatting and normal commit hooks pass. The exact pre-PR asset helper at ee32a0d1 returns only exposed.js for this fixture. Full local root formatting is blocked by an unrelated missing Next-app @tailwindcss/typography dependency; the exact-head hosted package, Node 20, and broad E2E jobs pass. No full application/native compiler rerun is claimed for this foundation check.

Benchmark

Successful isolated CI run and raw artifacts: 100 exposes, 20 common imports per expose, one shared singleton, and synchronous/asynchronous JS/CSS. Webpack 5.104.1, Node 24.18.0, GitHub-hosted Ubuntu 24.04. Both modes use commit 07527d3 and the same built packages, changing only manifest.useLegacyStats.

Five alternating independent process pairs, one discarded warmup pair, and three real single-source HMR edits per process. Values are median [min–max] in milliseconds unless noted. Rebuild statistics summarize each process's median first.

Measurement Previous reader Graph collector
Cold build 1,016.14 [999.51–1,017.13] 906.12 [893.81–958.14]
Cold stats generation 90.20 [86.03–101.79] 7.59 [7.57–9.72]
Incremental build 226.90 [207.24–230.49] 155.00 [151.27–160.66]
Incremental stats generation 70.67 [69.24–73.46] 5.54 [4.72–6.74]
Edit to completion, including watcher latency 248.42 [229.68–253.11] 176.68 [173.04–182.32]
Whole-process peak RSS, MiB 342.72 [342.09–381.51] 313.02 [309.73–321.91]

Manifests and asset lists matched. Stats differed only by the documented requires/usedIn correction; no unexpected differences or warnings. Every measured edit rebuilt exactly one source module and emitted HMR updates. Each compilation called Stats.toJson once with the previous reader and zero times with graph collection.

Stats generation measures generateStats, not just the inner graph function. Build time excludes process startup and watcher debounce; RSS includes setup and rebuilds. OS caches were not cleared. Process sampling found no competing compiler/test, but cannot exclude underlying hosted-machine contention. These are synthetic-fixture results, not a universal speedup or the original report's full application.

Additional local evidence, measured separately on Node 22.23.2: a 2,000-module fixture with one remote and no exposes/shares improved median cold build from 1,581 to 1,448 ms and rebuilds from 289 to 201 ms, with exact JSON equality. Earlier local expose runs overlapped unrelated builds and remain excluded.

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • I have updated the documentation.

Focused checks passed as listed above; the complete workspace suite was not run.

@changeset-bot

changeset-bot Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b99a1c3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 48 packages
Name Type
@module-federation/manifest Patch
@module-federation/sdk Patch
@module-federation/enhanced Patch
@module-federation/modern-js-v3 Patch
@module-federation/modern-js Patch
@module-federation/rspack Patch
@module-federation/devtools Patch
@module-federation/cli Patch
@module-federation/dts-plugin Patch
@module-federation/esbuild Patch
@module-federation/managers Patch
@module-federation/metro Patch
@module-federation/nextjs-mf Patch
@module-federation/node Patch
@module-federation/observability-plugin Patch
@module-federation/retry-plugin Patch
@module-federation/rsbuild-plugin Patch
@module-federation/rspress-plugin Patch
@module-federation/rstest Patch
@module-federation/runtime-core Patch
@module-federation/runtime Patch
@module-federation/storybook-addon Patch
@module-federation/utilities Patch
@module-federation/webpack-bundler-runtime Patch
@module-federation/bridge-react-webpack-plugin Patch
@module-federation/bridge-react Patch
@module-federation/bridge-vue3 Patch
shared-tree-shaking-no-server-host Patch
shared-tree-shaking-no-server-provider Patch
@module-federation/metro-plugin-rnc-cli Patch
@module-federation/metro-plugin-rnef Patch
@module-federation/metro-plugin-rock Patch
shared-tree-shaking-with-server-host Patch
shared-tree-shaking-with-server-provider Patch
node-dynamic-remote-new-version Patch
node-dynamic-remote Patch
@module-federation/playground Patch
remote5 Patch
remote6 Patch
website-new Patch
@module-federation/runtime-tools Patch
@module-federation/inject-external-runtime-core-plugin Patch
create-module-federation Patch
@module-federation/error-codes Patch
@module-federation/third-party-dts-extractor Patch
@module-federation/treeshake-frontend Patch
@module-federation/treeshake-server Patch
@module-federation/bridge-shared Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-15T01:07:11.225171Z 07527d3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@pkg-pr-new

pkg-pr-new Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@module-federation/devtools

pnpm add https://pkg.pr.new/@module-federation/devtools@d2903b2

@module-federation/cli

pnpm add https://pkg.pr.new/@module-federation/cli@d2903b2

create-module-federation

pnpm add https://pkg.pr.new/create-module-federation@d2903b2

@module-federation/dts-plugin

pnpm add https://pkg.pr.new/@module-federation/dts-plugin@d2903b2

@module-federation/enhanced

pnpm add https://pkg.pr.new/@module-federation/enhanced@d2903b2

@module-federation/error-codes

pnpm add https://pkg.pr.new/@module-federation/error-codes@d2903b2

@module-federation/esbuild

pnpm add https://pkg.pr.new/@module-federation/esbuild@d2903b2

@module-federation/managers

pnpm add https://pkg.pr.new/@module-federation/managers@d2903b2

@module-federation/manifest

pnpm add https://pkg.pr.new/@module-federation/manifest@d2903b2

@module-federation/metro

pnpm add https://pkg.pr.new/@module-federation/metro@d2903b2

@module-federation/metro-plugin-rnc-cli

pnpm add https://pkg.pr.new/@module-federation/metro-plugin-rnc-cli@d2903b2

@module-federation/metro-plugin-rnef

pnpm add https://pkg.pr.new/@module-federation/metro-plugin-rnef@d2903b2

@module-federation/metro-plugin-rock

pnpm add https://pkg.pr.new/@module-federation/metro-plugin-rock@d2903b2

@module-federation/modern-js

pnpm add https://pkg.pr.new/@module-federation/modern-js@d2903b2

@module-federation/modern-js-v3

pnpm add https://pkg.pr.new/@module-federation/modern-js-v3@d2903b2

@module-federation/native-federation-tests

pnpm add https://pkg.pr.new/@module-federation/native-federation-tests@d2903b2

@module-federation/native-federation-typescript

pnpm add https://pkg.pr.new/@module-federation/native-federation-typescript@d2903b2

@module-federation/nextjs-mf

pnpm add https://pkg.pr.new/@module-federation/nextjs-mf@d2903b2

@module-federation/node

pnpm add https://pkg.pr.new/@module-federation/node@d2903b2

@module-federation/observability-plugin

pnpm add https://pkg.pr.new/@module-federation/observability-plugin@d2903b2

@module-federation/playground

pnpm add https://pkg.pr.new/@module-federation/playground@d2903b2

@module-federation/retry-plugin

pnpm add https://pkg.pr.new/@module-federation/retry-plugin@d2903b2

@module-federation/rsbuild-plugin

pnpm add https://pkg.pr.new/@module-federation/rsbuild-plugin@d2903b2

@module-federation/rspack

pnpm add https://pkg.pr.new/@module-federation/rspack@d2903b2

@module-federation/rspress-plugin

pnpm add https://pkg.pr.new/@module-federation/rspress-plugin@d2903b2

@module-federation/rstest

pnpm add https://pkg.pr.new/@module-federation/rstest@d2903b2

@module-federation/runtime

pnpm add https://pkg.pr.new/@module-federation/runtime@d2903b2

@module-federation/runtime-core

pnpm add https://pkg.pr.new/@module-federation/runtime-core@d2903b2

@module-federation/runtime-tools

pnpm add https://pkg.pr.new/@module-federation/runtime-tools@d2903b2

@module-federation/sdk

pnpm add https://pkg.pr.new/@module-federation/sdk@d2903b2

@module-federation/storybook-addon

pnpm add https://pkg.pr.new/@module-federation/storybook-addon@d2903b2

@module-federation/third-party-dts-extractor

pnpm add https://pkg.pr.new/@module-federation/third-party-dts-extractor@d2903b2

@module-federation/treeshake-frontend

pnpm add https://pkg.pr.new/@module-federation/treeshake-frontend@d2903b2

@module-federation/treeshake-server

pnpm add https://pkg.pr.new/@module-federation/treeshake-server@d2903b2

@module-federation/typescript

pnpm add https://pkg.pr.new/@module-federation/typescript@d2903b2

@module-federation/utilities

pnpm add https://pkg.pr.new/@module-federation/utilities@d2903b2

@module-federation/webpack-bundler-runtime

pnpm add https://pkg.pr.new/@module-federation/webpack-bundler-runtime@d2903b2

@module-federation/bridge-react

pnpm add https://pkg.pr.new/@module-federation/bridge-react@d2903b2

@module-federation/bridge-react-webpack-plugin

pnpm add https://pkg.pr.new/@module-federation/bridge-react-webpack-plugin@d2903b2

@module-federation/bridge-shared

pnpm add https://pkg.pr.new/@module-federation/bridge-shared@d2903b2

@module-federation/bridge-vue3

pnpm add https://pkg.pr.new/@module-federation/bridge-vue3@d2903b2

@module-federation/inject-external-runtime-core-plugin

pnpm add https://pkg.pr.new/@module-federation/inject-external-runtime-core-plugin@d2903b2

commit: d2903b2

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Bundle Size Report

11 package(s) changed, 32 unchanged.

Package dist + ESM entry

Package Total dist (raw) Delta ESM gzip Delta
@module-federation/enhanced 820.5 kB +1.0 kB (+0.1%) 672 B no change
@module-federation/inject-external-runtime-core-plugin 5.2 kB +26 B (+0.5%) 518 B no change
@module-federation/manifest 145.6 kB +13.7 kB (+10.3%) 106 B no change
@module-federation/playground 16.08 MB -704 B (-0.0%) 45.8 kB no change
@module-federation/runtime 20.4 kB +13 B (+0.1%) 724 B no change
@module-federation/sdk 129.1 kB +144 B (+0.1%) 785 B no change

Bundle targets

Package Web bundle (gzip) Delta Node bundle (gzip) Delta
@module-federation/cli 2.3 kB -4 B (-0.2%) 2.4 kB -32 B (-1.3%)
@module-federation/core 1.0 kB -1 B (-0.1%) 1.0 kB -33 B (-3.0%)
@module-federation/devtools 30.3 kB -8 B (-0.0%) 30.3 kB -32 B (-0.1%)
@module-federation/enhanced 2.7 kB +10 B (+0.4%) 2.8 kB -43 B (-1.5%)
@module-federation/manifest 6.5 kB +954 B (+16.8%) 6.5 kB +954 B (+16.8%)
@module-federation/metro-plugin-rnc-cli 416 B +1 B (+0.2%) 435 B -26 B (-5.6%)
@module-federation/node 9.1 kB no change 9.1 kB -28 B (-0.3%)

Consumer scenarios

Scenario Web output (gzip) Delta Node output (gzip) Delta Gap (node-web) Delta
Enhanced remoteEntry 22.2 kB no change 23.7 kB +2 B (+0.0%) +1.6 kB +2 B

Total dist (raw): 23.42 MB (+14.1 kB (+0.1%))
Total ESM gzip: 111.7 kB (no change)
Total web bundle (gzip): 254.2 kB (+952 B (+0.4%))
Total node bundle (gzip): 256.5 kB (+760 B (+0.3%))
Tracked ./bundler entry gzip: 563 B (no change)
Tracked ./bundler web bundle (gzip): 4.9 kB (no change)
Tracked ./bundler node bundle (gzip): 4.9 kB (no change)

Bundle sizes are generated with rslib (Rspack). Package-root metrics preserve the historical report. Tracked subpath exports such as ./bundler are measured separately so ENV_TARGET-driven tree-shaking is visible. Bare imports are externalized to keep package-level sizes consistent, and assets are emitted as resources.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07527d3fd3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +118 to +122
const version =
'version' in data
? JSON.parse(data.version)
: (normalized?.version ?? sharedMap[key]?.version);
if (!name || !version) continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve consume-only shares with exact versions

When a shared dependency is configured with import: false, is absent from the local installation, and has an exact requiredVersion, normalized.version is undefined and consume code-generation data contains no version, so this branch skips the dependency entirely. The legacy reader recovered the exact version from the consume identifier; because graph collection is now the default, the generated stats and manifest unexpectedly lose this valid shared entry unless useLegacyStats is enabled.

Useful? React with 👍 / 👎.

@ScriptedAlchemy

Copy link
Copy Markdown
Member Author

One thing from a read of this diff, not reproduced. getAssetsByChunk in packages/manifest/src/utils.ts changed its !chunkGroup.name || condition, and the same helper is still called from _getModuleAssets in StatsManager.ts. Under manifest: { useLegacyStats: true } that means unnamed chunk groups (an import() without webpackChunkName that also reaches an expose chunk) now contribute to the expose's sync assets, which the previous reader skipped, so the rollback option no longer reproduces pre-PR mf-stats.json. Graph.spec.ts cannot catch it because it compares the graph collector against the new legacy path in the same build. If the graph path needs the wider condition, passing it as a flag from collectGraph only would keep the legacy reader byte-identical. Also, useLegacyStats is in the README but not in apps/website-new/docs/{en,zh}/configure/manifest.mdx, where users look for manifest options.

@github-actions

Copy link
Copy Markdown
Contributor

Deploying with ☁️ Zephyr Cloud

The latest preview deployments for this pull request.

Status Name Latest Commit Version URL Updated (UTC)
✅ Deployment successful!
View deployment
website-new d2903b2 Version ↗ 30 Sept 2026, 21:12

Affected deployment targets

Application Type Target
website-new Environment latest ↗
website-new Tag latest ↗
website-new Tag web_codex-manifest-graph-foundation_nestor_lopez ↗

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant