ci: shard Node integration tests and increase browser Playwright workers - #24676
Draft
nicohrubec wants to merge 10 commits into
Draft
nicohrubec wants to merge 10 commits into
nicohrubec wants to merge 10 commits into
Conversation
Co-Authored-By: GPT-6 <codex@openai.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| name: node-integration-timings-node-${{ matrix.node }}-ts-${{ matrix.typescript || 'default' }} | ||
| path: dev-packages/node-integration-tests/vitest.junit.xml | ||
| retention-days: 7 | ||
| if-no-files-found: warn |
There was a problem hiding this comment.
Artifact upload fails on job rerun
Medium Severity
The timing upload omits overwrite, so a job rerun conflicts after the prior attempt already stored the same artifact name. That fails the rerun even when tests pass, because this step also runs after a failed first attempt.
Reviewed by Cursor Bugbot for commit c27d965. Configure here.
Contributor
size-limit report 📦
|
Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
Co-Authored-By: GPT-6 <codex@openai.com>
Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
Restore serial browser bundling while retaining the Node integration test shards. Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
Generate unminified and minified debug bundles from one Rollup build, retaining a separate production build and the existing output plugin order. Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
Increase browser Playwright CI workers to six. Remove Node timing reports and restore the original bundle build, retaining three shards for every Node integration configuration. Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
Restore three Rollup processes on the existing bundle runner to measure their effect alongside the current test parallelism. Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
Restore serial browser bundling and retain three Node integration shards. Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit cda6116. Configure here.
| return { | ||
| ...config, | ||
| ...overwriteConfig, | ||
| workers: process.env.CI ? 6 : (overwriteConfig?.workers ?? 1), |
There was a problem hiding this comment.
Parallel workers race E2E events
Medium Severity
Forcing workers to 6 on CI runs multiple E2E files against one app and event proxy. Helpers such as waitForError and waitForStreamedSpan accept the first matching event, so concurrent tests can claim each other's telemetry and fail intermittently. The helper also ignores any overwriteConfig.workers value on CI.
Additional Locations (2)
Triggered by project rule: PR Review Guidelines for Cursor Bot
Reviewed by Cursor Bugbot for commit cda6116. Configure here.
| // Use 3 workers on CI, else use defaults (based on available CPU cores) | ||
| // Note that 3 is a random number selected to work well with our CI setup | ||
| workers: process.env.CI ? 3 : undefined, | ||
| workers: process.env.CI ? 6 : undefined, |
There was a problem hiding this comment.
Loader tests inherit extra workers
Low Severity
playwright.loader.config.ts spreads this shared config, so loader jobs now use 6 CI workers. The worker bump was meant for browser suite tests on large runners; loader jobs stay on standard runners and were supposed to keep their existing worker count.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit cda6116. Configure here.
Restore the previously passing configuration with three Node integration shards and original bundling. Co-Authored-By: GPT-6 <codex@openai.com>
nicohrubec
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reduce CI wall time with three shards for every Node integration configuration and six CI workers for browser Playwright. Loader and framework E2E worker settings remain unchanged. Bundle sharding and timing-report additions are removed.
CI experiment comparisons
Compare CI wall time from workflow start until the required-check gate finishes, including queueing and setup. The Node column is the longest individual Node integration job, including setup; the bundle column is the actual
Build bundlesstep, excluding runner setup and artifact handling.Workloads differ: the parallel-bundle run included 171 framework E2E jobs, the restored-serial run included none, and the shared-helper change triggered 256. These total durations are not controlled comparisons. The two-shard rerun also had a six-minute gap before the last Playwright job started, despite unchanged 4m01s test execution.
The shared-debug-build and parallel-bundle experiments are reverted. The final configuration keeps three Node shards and six workers only for browser Playwright, matching the previously passing 36006150096 run. Timing-report additions remain removed.
Six workers reduced heavy Chromium test execution from 4m20s to 2m14s (tracing/replay) and 4m31s to 2m20s (tracing/replay/logs); WebKit fell from 3m42s to 2m05s. Total CI wall time was 11m45s versus 11m40s for the comparable three-worker run: WebKit started 69s after bundles were ready and its Playwright installation took 2m13s versus 41s previously. The last Node shard finished at 10m23s; WebKit determined the final finish time. This single run passed without test failures.
In the second parallel-bundle run, bundles were ready at 5m00s (previously 5m29s), all browser tests finished at 9m44s (11m39s), and all Node shards finished at 11m49s (10m23s). Excluding the Nuxt sourcemaps job, the last required job finished at 11m49s; about 11m55s including the gate is an estimate, not a measured workflow duration. The last Node/TS5 shard spent 4m07s installing TypeScript versus 3m07s previously. The bundle step saved 45s, but Node setup determined the finish time when excluding Nuxt. The run also includes 171 framework E2E jobs absent from the preceding run. Nuxt eventually passed after 11m01s installing Playwright; the measured total was 16m43s.
The all-Playwright experiment caused widespread framework E2E failures, including mismatched trace IDs when concurrent test files consumed events from the same app. That expansion is reverted; loader and framework E2E configs retain their original worker settings.