Skip to content

feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init - #1647

Open
kacperkapusciak wants to merge 9 commits into
mainfrom
kacper/eng-1084-add-cicd-for-e2e-init
Open

kacperkapusciak wants to merge 9 commits into
mainfrom
kacper/eng-1084-add-cicd-for-e2e-init

Conversation

@kacperkapusciak

@kacperkapusciak kacperkapusciak commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

e2e init gets a CI step. In a project with the web or mobile engine it asks which CI service to set up, then writes one job for each platform the config's targets run on: web, iOS, and Android.

◆  Add a CI pipeline that runs the tests?
│  ○ GitHub Actions
│  ● GitLab CI/CD (in every merge request and on the default branch)
│  ○ None
Targets in the config Jobs
web only web, running every target
iOS and/or Android only ios and/or android
web and mobile all of them, each with --target <its targets>
  • --ci github-actions|gitlab|none answers the question. --yes alone writes none, since a pipeline runs on the next push. The wizard preselects the service the repository points to: its remote, a .github/ folder, or a .gitlab-ci.yml.
  • Targets come from the new config init writes, or from loading an existing one the way e2e run does (loadConfigModule, and resolvePlatform/targetName from config/targets.ts). The load happens after the user confirms and after the install, so the config never runs on a cancelled wizard and an example's engine is installed first. A config that still doesn't load falls back to package.json: web engine only, one web job runs every target; mobile engine, no pipeline.
  • Web job: the project's package manager (npm, pnpm, Yarn 1, Yarn 2+, Bun), Chromium with a cache, e2e run --reporter list,junit. It runs every target that isn't iOS or Android, HTTP tests included.
  • Mobile jobs on GitHub follow e2e-inc/e2e-react-native's workflow: macos-26 with a Release simulator build, the image's own iPhone 17 Pro on a pinned iOS runtime, and E2E_DEVICE set to its UDID; ubuntu-latest with KVM, an x86_64 release APK, and an API 36 emulator, with a system-image install that retries a download that drops or comes back empty.
  • The app build: Expo (expo run:ios, expo prebuild + Gradle, no prebuild when android/ is committed), bare React Native (xcodebuild on ios/*.xcworkspace, through Bundler with a Gemfile; android/gradlew), or a step that fails until replaced, which init names.
  • GitLab's hosted runners can't run a simulator or an emulator, so its mobile jobs target self-managed shell runners with tags to replace, as on docs/ci/gitlab.mdx; init says so, and asks for device: process.env.E2E_DEVICE when the config lacks it.
  • Node comes from .nvmrc or .node-version, else 24. A project in a subfolder or a workspace member gets the file at the repository root, the install at the workspace root, and the jobs in the project folder.
  • Model key: GitHub passes the repository secret in the run step's env; GitLab gets CI/CD variables on its own. init reads the gateway from the config's import and prints the gh secret set or glab variable set command.
  • init never overwrites a pipeline file. Beside an existing .gitlab-ci.yml it writes .gitlab/e2e.gitlab-ci.yml and prints the include: to add, renaming the jobs with -run when the pipeline has one of their names and saying when its stages: leave out test. A workflow or pipeline that already runs e2e run gets nothing. Nothing is written through a symlinked .github or .gitlab.
  • Warnings for what would fail on the first push: no app.command (web job only), a subscription gateway or a loopback model endpoint, a build step to replace, GitLab mobile runners.
  • The pipelines are YAML templates in packages/e2e/templates/ci/, each a complete pipeline for npm at the repository root and an Expo app, which actionlint and glab ci lint accept as is. init edits a copy with the yaml package's Document API (new runtime dependency, pure JS), keeping comments and formatting; a platform with no targets loses its job. For the template's own case the output equals the template byte for byte.
  • GitHub action pins live only in the templates and match the docs and the e2e skill; tests/unit/init-ci.test.ts fails when they drift.
  • Telemetry: e2e_init_completed gets a ci property.
flowchart TD
  A[web or mobile engine in package.json?] -- no --> N[no question]
  A -- yes --> B{e2e run already in a workflow or .gitlab-ci.yml?}
  B -- yes --> N
  B -- no --> C[ask, preselect from remote / .github / .gitlab-ci.yml]
  C --> D[confirm changes, install]
  D --> E[targets: new config, or load the existing one]
  E --> F[web job for non-phone targets, ios and android jobs for phone targets]
  F --> G[.github/workflows/e2e.yml, .gitlab-ci.yml, or .gitlab/e2e.gitlab-ci.yml to include]
Loading

Verified

Ran it locally: yes. The built CLI wrote the pipelines into real repos, and each ran unedited on hosted runners:

Case Where Result
GitHub, web + iOS + Android, Expo, existing config with three targets e2e-react-native#5, its own workflow replaced by the generated one RN_RESULT
GitHub Actions, npm, repo root e2e-github-actions#2, merged 5/5, agent tests included
GitHub Actions, pnpm in site/, .nvmrc at the root e2e-github-actions#3 5/5
GitHub Actions, pnpm version from site/package.json e2e-github-actions#5 passed
GitHub, web job from the YAML template e2e-github-actions#6 5/5, cache replayed
GitLab, npm, new .gitlab-ci.yml e2e-gitlab-ci!1, merged 5/5 in the MR test report
GitLab, beside an existing .gitlab-ci.yml via include: e2e-gitlab-ci!2 5/5
GitLab, pnpm via corepack on node:26 e2e-gitlab-ci!5 5/5
GitLab, web job from the YAML template e2e-gitlab-ci!6 5/5
  • actionlint and glab ci lint pass on every combination rendered: web only, mobile only, both, each build kind, npm/pnpm/Yarn/Bun, a subfolder, and a workspace member. The GitLab file for e2e-react-native passes glab ci lint too.
  • Packed tarball in a scratch project, and the interactive wizard in a pseudo-terminal: the question above, with the service preselected from the remote.
  • Not run on a runner: the bare React Native build, and GitLab's mobile jobs (gitlab.com's hosted runners can't run them).

Two fresh-context reviews found issues, all fixed: the first nine in the web-only version, then eight in the mobile one (the config loaded before the install, so an example got no pipeline; the GitLab header lost with the web job; and smaller ones). cubic's findings are fixed or answered in their threads, and kdy1's suggestion to keep the pipelines as YAML templates is in.

Tracked in ENG-1084.

@github-actions github-actions Bot added the size:XL Up to 999 changed lines, not counting tests, fixtures, or the lockfile label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web: 91 passed

All 91 tests in 41 files
Test Time
🟢 tests/aria-hidden-flow.e2e.ts · 2 passed 656ms
🟢 aria hidden flow › picks the Pro plan by its visible text and confirms it 470ms
🟢 aria hidden flow › another plan shows the error and keeps the picker open 186ms
🟢 tests/async-states.e2e.ts · 2 passed 7.1s
🟢 async states › reads the code off the transient toast and verifies it once the button enables 2.3s
🟢 async states › a wrong code is rejected after the toast has gone 4.8s
🟢 tests/canvas-only.e2e.ts · 2 passed 293ms
🟢 canvas only › taps square, circle, then diamond at their pixel coordinates 160ms
🟢 canvas only › a shape out of order paints the error and resets progress 133ms
🟢 tests/canvas-whiteboard.e2e.ts · 2 passed 390ms
🟢 canvas whiteboard › drags both shapes into their slots and saves 240ms
🟢 canvas whiteboard › bounces a shape off the wrong slot and refuses to save early 150ms
🟢 tests/cart-totals.e2e.ts · 2 passed 342ms
🟢 cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug) 176ms
🟢 cart totals › a quantity never drops below one 166ms
🟢 tests/checkout-review.e2e.ts · 1 passed 178ms
🟢 shows a raw greeting token and an undefined delivery date (planted bug), yet places the order 178ms
🟢 tests/context-menu-trap.e2e.ts · 2 passed 528ms
🟢 context menu trap › right-click opens nothing; the row menu renames the file 281ms
🟢 context menu trap › the decoy menu items refuse and keep the file 247ms
🟢 tests/control-inventory.e2e.ts · 8 passed 3.7s
🟢 control inventory › check sets the checkbox and the radio without flipping a set one 265ms
🟢 control inventory › check picks a radio the pick replaces with its summary 159ms
🟢 control inventory › a long press and a double click are told apart from a tap 1.0s
🟢 control inventory › the file menu opens on right-click only and Rename renames the file 297ms
🟢 control inventory › setInputFiles attaches files named from the project root 160ms
🟢 control inventory › the details view is left through the browser history 210ms
🟢 control inventory › scrollIntoView brings the footnote into view 260ms
🟢 control inventory › every exercise done shows the success message 1.4s
🟢 tests/css-content-ui.e2e.ts · 2 passed 397ms
🟢 css content ui › the tile labelled Ship it advances and confirming ships 243ms
🟢 css content ui › a wrong tile shows the CSS-rendered error and keeps the picker 154ms
🟢 tests/date-picker.e2e.ts · 2 passed 676ms
🟢 date picker › navigates three months ahead and books the required date 393ms
🟢 date picker › confirming without the exact date shows the error 283ms
🟢 tests/debounced-search.e2e.ts · 2 passed 2.8s
🟢 debounced search › a decoy one character off is refused 1.4s
🟢 debounced search › picking the exact item adds it to the cart 1.4s
🟢 tests/deferred-dialog.e2e.ts · 1 passed 1.1s
🟢 one submit opens one confirm, even when double-clicked 1.1s
🟢 tests/div-soup.e2e.ts · 2 passed 663ms
🟢 div soup › a wrong code is rejected 282ms
🟢 div soup › the right code grants access after the delete tile fixes a typo 381ms
🟢 tests/drag-and-drop.e2e.ts · 2 passed 633ms
🟢 drag and drop › an out-of-order item resets the dropzone 332ms
🟢 drag and drop › dropping Banana then Cherry passes 301ms
🟢 tests/filter-deep-link.e2e.ts · 2 passed 247ms
🟢 filter deep link › filtering writes the query to the URL 137ms
🟢 filter deep link › opening a shared link restores the filter 110ms
🟢 tests/gift-card-purchase.e2e.ts · 3 passed 550ms
🟢 gift card purchase › refuses to buy without an amount 153ms
🟢 gift card purchase › refuses a recipient without an at sign 197ms
🟢 gift card purchase › sends the gift card 200ms
🟢 tests/hover-menu.e2e.ts · 2 passed 448ms
🟢 hover menu › a decoy leaf is refused 196ms
🟢 hover menu › hovering Account, then Billing, reaches Redeem voucher 252ms
🟢 tests/iframe-form.e2e.ts · 2 passed 558ms
🟢 iframe form › a wrong code is refused by the host page 279ms
🟢 iframe form › the code read from one frame applies through the other 279ms
🟢 tests/image-only-ui.e2e.ts · 3 passed 519ms
🟢 image only ui › the icons have no names, so the bare svg locator is ambiguous 115ms
🟢 image only ui › a wrong icon shows the error image and keeps the icons 174ms
🟢 image only ui › tapping the star, then the heart, verifies the icons 230ms
🟢 tests/infinite-scroll.e2e.ts · 1 passed 2.8s
🟢 scrolls the feed until the Golden Ticket loads, then claims it 2.8s
🟢 tests/kanban-board.e2e.ts · 3 passed 751ms
🟢 kanban board › pointer drags move both cards into the goal layout 384ms
🟢 kanban board › a swipe along a path drags a card as well 149ms
🟢 kanban board › a plain tap moves nothing and the untouched board is rejected 218ms
🟢 tests/login-form.e2e.ts · 4 passed 745ms
🟢 login form › rejects a malformed email 181ms
🟢 login form › rejects a short password 182ms
🟢 login form › rejects an unknown account 183ms
🟢 login form › signs in and logs out 199ms
🟢 tests/lying-labels.e2e.ts · 4 passed 631ms
🟢 lying labels › the button whose accessible name says Pay now is the wrong one 133ms
🟢 lying labels › a text match for Pay now catches the decoy phrase too 102ms
🟢 lying labels › the field whose accessible name says Amount is the wrong one 213ms
🟢 lying labels › visible text drives the payment through 183ms
🟢 tests/native-dialogs.e2e.ts · 3 passed 467ms
🟢 native dialogs › answers the confirm, prompt, and alert chain 154ms
🟢 native dialogs › dismissing the confirm cancels the order 145ms
🟢 native dialogs › a wrong gift code is rejected 168ms
🟢 tests/newsletter-signup.e2e.ts · 1 passed 2.2s
🟢 subscribe renders no feedback and logs a TypeError (planted bug) 2.2s
🟢 tests/onboarding-wizard.e2e.ts · 5 passed 4.4s
🟢 onboarding wizard › back keeps what was typed and the brief comes back with it 262ms
🟢 onboarding wizard › the address step takes the brief through its bare-div dropdown 313ms
🟢 onboarding wizard › the profile step steers the calendar three decades back 1.2s
🟢 onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off 1.3s
🟢 onboarding wizard › the review step repeats the brief and finishes with the PIN 1.4s
🟢 tests/order-history.e2e.ts · 2 passed 1.5s
🟢 order history › overview and settings render their panels 193ms
🟢 order history › the orders tab never leaves its loading state (planted bug) 1.3s
🟢 tests/otp-auto-advance.e2e.ts · 3 passed 1.4s
🟢 otp auto-advance › focus advances with each digit and the full code verifies 481ms
🟢 otp auto-advance › backspace on an empty box clears and refocuses the previous one 152ms
🟢 otp auto-advance › a wrong code clears every box and starts over 791ms
🟢 tests/overlay-trap.e2e.ts · 1 passed 736ms
🟢 continues only after the banner and the overlay are gone 736ms
🟢 tests/playbook-cleanup.e2e.ts · 2 passed 579ms
🟢 playbook cleanup › deleting the draft removes it from an exact-name search 279ms
🟢 playbook cleanup › cancelling the dialog keeps the draft searchable 300ms
🟢 tests/price-sorting.e2e.ts · 3 passed 426ms
🟢 price sorting › opens in the featured order with sale prices struck through 111ms
🟢 price sorting › low to high sorts by the original price, not the paid one (planted bug) 149ms
🟢 price sorting › high to low is the same wrong order reversed (planted bug) 166ms
🟢 tests/product-reviews.e2e.ts · 1 passed 1.1s
🟢 the running-shoe page lists coffee-grinder reviews (planted bug) 1.1s
🟢 tests/promo-storefront.e2e.ts · 1 passed 1.4s
🟢 buys a deal once the promo chrome is out of the way 1.4s
🟢 tests/rich-text-editor.e2e.ts · 2 passed 502ms
🟢 rich text editor › bolding exactly the required word passes the check 275ms
🟢 rich text editor › bold that spills into the previous word fails the check 227ms
🟢 tests/saved-addresses.e2e.ts · 3 passed 585ms
🟢 saved addresses › adds an address and clears the form 190ms
🟢 saved addresses › ignores an add with an empty field 180ms
🟢 saved addresses › delete shows its toast but keeps the address, on retry too (planted bug) 215ms
🟢 tests/shadow-dom-form.e2e.ts · 2 passed 551ms
🟢 shadow DOM form › rejects a wrong access code 263ms
🟢 shadow DOM form › grants access with the code from the hint inside the root 288ms
🟢 tests/smoke.e2e.ts · 1 passed 445ms
🟢 home lists the scenarios and opens one 445ms
🟢 tests/stale-dom.e2e.ts · 1 passed 2.2s
🟢 taps the moving target three times across rebuilds of the list 2.2s
🟢 tests/sticky-chrome.e2e.ts · 1 passed 186ms
🟢 accepts the terms once the button clears the sticky footer 186ms
🟢 tests/virtualized-table.e2e.ts · 1 passed 14.7s
🟢 scrolls the windowed table to the Golden Row and claims it 14.7s
🟢 tests/wishlist.e2e.ts · 2 passed 1.6s
🟢 wishlist › saving toggles the button and the tab badge 265ms
🟢 wishlist › the wishlist tab renders the empty state despite the badge count (planted bug) 1.3s

Tested by e2e 0.23.0 · 1m 2s · web · run artifacts

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web agent: 33 passed

34 agent steps · 26 replayed from cache · 20 model calls · 88.4k tokens (94% cached) · $0.0036

All 33 tests in 5 files
Test Agent Time
🟢 tests-agent/control-inventory.e2e.ts · 7 passed 7 steps 7.8s
🟢 control inventory › check sets the checkbox and the radio 1 step 823ms
🟢 control inventory › check picks a radio the pick replaces with its summary 1 step 332ms
🟢 control inventory › a long press and a double click are chosen over a plain tap 1 step 948ms
🟢 control inventory › a right-click opens the file menu and Rename renames the file 1 step 2.6s
🟢 control inventory › upload attaches the named project files 1 step 2.4s
🟢 control inventory › back returns from the details view to the inventory 1 step 305ms
🟢 control inventory › scroll_to brings the footnote into view 1 step 392ms
🟢 tests-agent/login-form.e2e.ts · 1 passed 1 step 397ms
🟢 act signs in with the benchmark account 1 step 397ms
🟢 tests-agent/planted-bugs.e2e.ts · 3 passed 4 steps · 8 calls 10.6s
🟢 cart totals: a judgment catches the stale order summary 2 steps · 4 calls 5.6s
🟢 checkout review: a judgment catches the broken copy 1 step · 1 call 1.6s
🟢 wishlist: a judgment catches the lost saved items 1 step · 3 calls 3.4s
🟢 tests-agent/scenarios.e2e.ts · 21 passed 21 steps · 12 calls 1m 5s
🟢 act completes shadow-dom-form 1 step · 3 calls 4.1s
🟢 act completes lying-labels 1 step · 3 calls 4.2s
🟢 act completes iframe-form 1 step · 3 calls 3.4s
🟢 act completes infinite-scroll 1 step 3.3s
🟢 act completes virtualized-table 1 step 17.4s
🟢 act completes overlay-trap 1 step 875ms
🟢 act completes sticky-chrome 1 step 926ms
🟢 act completes async-states 1 step · 3 calls 6.5s
🟢 act completes debounced-search 1 step 2.3s
🟢 act completes hover-menu 1 step 532ms
🟢 act completes drag-and-drop 1 step 467ms
🟢 act completes kanban-board 1 step 611ms
🟢 act completes context-menu-trap 1 step 2.7s
🟢 act completes stale-dom 1 step 496ms
🟢 act completes otp-auto-advance 1 step 722ms
🟢 act completes date-picker 1 step 857ms
🟢 act completes rich-text-editor 1 step 10.1s
🟢 act completes playbook-cleanup 1 step 568ms
🟢 act completes promo-storefront 1 step 2.3s
🟢 act completes gift-card-purchase 1 step 2.5s
🟢 act filters the list and the URL carries the filter 1 step 297ms
🟢 tests-agent/unique-cache.e2e.ts · 1 passed 1 step 292ms
🟢 act adds an address with a run-unique label 1 step 292ms

Tested by e2e 0.23.0 · 1m 25s · web · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 17 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread packages/e2e/src/cli/init/ci.ts Resolved
Comment thread packages/e2e/src/cli/init/ci.ts Resolved
Comment thread skills/e2e/references/setup.md Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/telemetry/events.ts Resolved
Comment thread docs/reference/cli.mdx Resolved
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios agent: 21 passed, 1 skipped

25 agent steps · 11 replayed from cache · 68 model calls · 443.0k tokens (82% cached) · $0.0306

All 22 tests in 1 file
Test Agent Time
⏭️ tests-agent/scenarios.e2e.ts · 21 passed, 1 skipped 25 steps · 68 calls 9m 24s
🟢 act completes Login Form 1 step · 5 calls 1m 1s
🟢 act completes Infinite Scroll List 1 step 39.3s
🟢 act completes Modal Flow 1 step 10.2s
🟢 act completes Bottom Tabs 1 step 7.3s
🟢 act completes Text Input Variations 1 step 13.4s
🟢 act completes Broken Accessibility 2 steps · 8 calls 42.9s
🟢 act completes Vision Only 2 steps · 6 calls 27.3s
🟢 act completes Gestures 1 step · 8 calls 38.5s
🟢 act completes Async States 1 step · 12 calls 40.3s
🟢 act completes Debounced Search 1 step · 1 call 29.0s
🟢 act completes Flattened Registration Form 2 steps · 1 call 30.1s
🟢 act completes Flattened Login 2 steps · 11 calls 54.4s
🟢 act completes Sticky Chrome Target 1 step 14.2s
🟢 act completes WebView Accessibility 1 step 13.3s
🟢 act completes Permission Prompt 1 step 12.1s
🟢 act completes Photo Picker 1 step · 2 calls 30.5s
🟢 act completes Product Catalog 1 step 10.8s
🟢 act completes Error Recovery 1 step 10.4s
🟢 act completes Choice Controls 1 step · 5 calls 35.3s
🟢 act completes Stripe PaymentSheet 1 step · 9 calls 32.8s
🟢 act completes Apple Pay 1 step 11.1s
⏭️ act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

Tested by e2e 0.23.0 · 9m 25s · ios-simulator · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 9 files (changes from recent commits).

View guided diff | Turn on auto-fix | Re-trigger cubic

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android agent: 20 passed

24 agent steps · 15 replayed from cache · 44 model calls · 258.9k tokens (86% cached) · $0.0154

All 20 tests in 1 file
Test Agent Time
🟢 tests-agent/scenarios.e2e.ts · 20 passed 24 steps · 44 calls 6m 12s
🟢 act completes Login Form 1 step 26.6s
🟢 act completes Infinite Scroll List 1 step 22.3s
🟢 act completes Modal Flow 1 step 10.2s
🟢 act completes Bottom Tabs 1 step 7.0s
🟢 act completes Text Input Variations 1 step 6.0s
🟢 act completes Broken Accessibility 2 steps · 5 calls 13.6s
🟢 act completes Vision Only 2 steps · 1 call 5.9s
🟢 act completes Gestures 1 step · 14 calls 33.9s
🟢 act completes Async States 1 step · 13 calls 27.1s
🟢 act completes Debounced Search 1 step 7.2s
🟢 act completes Huge Virtualized List 1 step 1m 55s
🟢 act completes Flattened Registration Form 2 steps · 1 call 8.8s
🟢 act completes Flattened Login 2 steps · 6 calls 14.8s
🟢 act completes Sticky Chrome Target 1 step 9.1s
🟢 act completes WebView Accessibility 1 step · 4 calls 21.9s
🟢 act completes Permission Prompt 1 step 6.7s
🟢 act completes Product Catalog 1 step 8.4s
🟢 act completes Error Recovery 1 step 9.0s
🟢 act completes Choice Controls 1 step 8.5s
🟢 act completes Sequential Onboarding 1 step 9.8s

Tested by e2e 0.23.0 · 6m 13s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios: 55 passed, 1 skipped

All 56 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 23 passed, 1 skipped 3m 1s
🟢 control inventory › a label query resolves a labeled text field 10.2s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 15.0s
🟢 control inventory › a placeholder query and attribute reads find the field 4.8s
🟢 control inventory › a secure field takes a fill and denies value reads 8.5s
🟢 control inventory › press sends a character and Enter through the soft keyboard 9.8s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 11.8s
🟢 control inventory › toBeFocused follows the focus 9.5s
🟢 control inventory › a switch checks, unchecks, and reports its state 11.7s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 9.5s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 12.8s
🟢 control inventory › a radio group reports the checked option 9.2s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 8.0s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 7.3s
🟢 control inventory › a header answers its accessible name 4.4s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 6.2s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 6.7s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 8.7s
🟢 control inventory › pointer verbs land at the point asked for 9.5s
🟢 control inventory › an ambiguous locator fails at once 5.4s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 10.2s
🟢 refusals › verbs a device never declares are refused before the engine 2ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 569ms
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 11 passed 1m 43s
🟢 device fixture › setAppearance flips the color scheme the app reads 12.7s
🟢 device fixture › setOrientation rotates the window the app measures 8.2s
🟢 device fixture › the clipboard reads back what was written 6.3s
🟢 device fixture › home leaves the app, openApp brings it back where it was 6.1s
🟢 device fixture › closeApp then openApp relaunches on the home list 7.9s
🟢 device fixture › closeApp leaves the next commands on the same device 11.7s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 8.7s
🟢 device fixture › device.back pops the scenario 10.1s
🟢 device fixture › unstable_raw reaches the test session through the raw driver 7.5s
🟢 device fixture › setLocation and clearLocation change what the app reads 13.2s
🟢 device fixture › enrollBiometrics and setBiometrics answer a biometric prompt 10.9s
🟢 tests/device.e2e.ts · 3 passed 1m 7s
🟢 sequential onboarding echoes the exact values 41.8s
🟢 onboarding rejects a malformed email before advancing 14.1s
🟢 dismissing the keyboard over choice controls leaves them as they were 11.1s
🟢 tests/flows-lists.e2e.ts · 7 passed 4m 5s
🟢 infinite scroll reaches the target item 32.9s
🟢 product catalog adds exactly two of the right variant 10.3s
🟢 gestures: long-press, then swipe left 9.2s
🟢 sticky chrome: accept below the fold, then continue 12.5s
🟢 async states: load, pull to refresh, claim through the toast 11.9s
🟢 huge virtualized list reaches row 512 2m 37s
🟢 web view coupon form applies the code shown on the page 11.9s
🟢 tests/flows.e2e.ts · 7 passed 1m 8s
🟢 text inputs unlock the submit button 12.8s
🟢 filled text inputs report their values 10.0s
🟢 modal flow confirms through the native alert 10.0s
🟢 bottom tabs act inside the last tab 7.7s
🟢 error recovery retries, then confirms the delete 9.2s
🟢 choice controls place the exact order 7.3s
🟢 debounced search selects the target once results arrive 11.3s
🟢 tests/login-form.e2e.ts · 3 passed 47.0s
🟢 login form › rejects a malformed email 13.5s
🟢 login form › rejects an unknown account 13.6s
🟢 login form › signs in and logs out 19.9s
🟢 tests/smoke.e2e.ts · 1 passed 7.3s
🟢 home lists the scenarios and opens one 7.3s

Tested by e2e 0.23.0 · 6m 56s · ios-simulator · run artifacts

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android: 57 passed, 1 skipped

All 58 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 24 passed, 1 skipped 2m 31s
🟢 control inventory › a label query resolves a labeled text field 15.7s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 11.4s
🟢 control inventory › a placeholder query and attribute reads find the field 4.8s
🟢 control inventory › a secure field takes a fill and denies value reads 5.4s
🟢 control inventory › press sends a character and Enter through the soft keyboard 7.8s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 8.0s
🟢 control inventory › toBeFocused follows the focus 7.6s
🟢 control inventory › a switch checks, unchecks, and reports its state 7.6s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 6.5s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 10.1s
🟢 control inventory › a tab role answers getByRole 3.8s
🟢 control inventory › a radio group reports the checked option 7.5s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 6.6s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.4s
🟢 control inventory › a header answers its accessible name 3.7s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 5.3s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 5.4s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 7.0s
🟢 control inventory › a double tap lands inside the 300 ms window 6.2s
🟢 control inventory › an ambiguous locator fails at once 4.6s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 6.3s
🟢 refusals › verbs a device never declares are refused before the engine 2ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 1.7s
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 10 passed 1m 10s
🟢 device fixture › setOrientation rotates the window the app measures 4.8s
🟢 device fixture › home leaves the app, openApp brings it back where it was 4.0s
🟢 device fixture › foregroundApp sees the home screen after home() 3.8s
🟢 device fixture › closeApp then openApp relaunches on the home list 5.8s
🟢 device fixture › closeApp leaves the next commands on the same device 5.9s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 3.9s
🟢 device fixture › device.back pops the scenario 4.1s
🟢 device fixture › unstable_raw reaches the test session through the raw driver 3.3s
🟢 device fixture › setNetwork and setAirplaneMode change what the app sees 14.9s
🟢 device fixture › setLocation and clearLocation change what the app reads 19.2s
🟢 tests/device.e2e.ts · 5 passed 37.9s
🟢 permission prompt › allowing the system dialog enables the microphone 5.0s
🟢 permission prompt › a denied prompt recovers once the permission is granted outside 10.9s
🟢 sequential onboarding echoes the exact values 9.8s
🟢 onboarding rejects a malformed email before advancing 5.5s
🟢 dismissing the keyboard over choice controls leaves them as they were 6.7s
🟢 tests/flows-lists.e2e.ts · 7 passed 3m 16s
🟢 infinite scroll reaches the target item 29.8s
🟢 product catalog adds exactly two of the right variant 8.8s
🟢 gestures: long-press, then swipe left 8.2s
🟢 gestures: double-tap completes the flow 9.6s
🟢 sticky chrome: accept below the fold, then continue 11.3s
🟢 async states: load, pull to refresh, claim through the toast 13.4s
🟢 huge virtualized list reaches row 512 1m 55s
🟢 tests/flows.e2e.ts · 7 passed 46.5s
🟢 text inputs unlock the submit button 5.5s
🟢 filled text inputs report their values 5.1s
🟢 modal flow confirms through the native alert 8.1s
🟢 bottom tabs act inside the last tab 6.0s
🟢 error recovery retries, then confirms the delete 8.0s
🟢 choice controls place the exact order 5.7s
🟢 debounced search selects the target once results arrive 8.1s
🟢 tests/login-form.e2e.ts · 3 passed 25.4s
🟢 login form › rejects a malformed email 6.7s
🟢 login form › rejects an unknown account 6.8s
🟢 login form › signs in and logs out 11.8s
🟢 tests/smoke.e2e.ts · 1 passed 5.0s
🟢 home lists the scenarios and opens one 5.0s

Tested by e2e 0.23.0 · 8m 51s · android-emulator · run artifacts

Comment thread packages/e2e/src/cli/init/ci.ts Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

View guided diff | Turn on auto-fix | Re-trigger cubic

e2e init asks for a CI pipeline in a project with the web engine (--ci github-actions|gitlab|none). It writes one job for the project's package manager and Node version that installs Chromium, runs e2e run, and keeps the report, and names the secret or CI/CD variable for the model key. It never overwrites a pipeline file; beside an existing .gitlab-ci.yml it writes a file to include.
Exact GitLab stage names, a distinct job name beside an existing e2e job, loopback baseURL in an existing config, no writes through a symlinked .github or .gitlab, --ci none in the skill, gitlab in the telemetry comment, and the GitLab Node image's major version in the docs.
The GitHub Actions workflow and the GitLab job live in packages/e2e/templates/ci/ as complete pipelines for npm at the repository root, linted as they are. init loads them with the yaml package's Document API and edits a copy for the project's package manager, folders, Node version, secrets, and target, keeping the comments. The pinned actions now live only in the templates.
@kacperkapusciak
kacperkapusciak force-pushed the kacper/eng-1084-add-cicd-for-e2e-init branch from b3f72d7 to de6bff7 Compare November 9, 2026 13:50

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 4 files (changes from recent commits).

View guided diff | Turn on auto-fix | Re-trigger cubic

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • We need to figure out if you use mobile, then we should output GitHub or GitLab Mobile CI/CD. And if you select web, then appropriate. We should also handle the case where you select two of those.

init writes one job per platform the config's targets run on: web targets in the web job, iOS and Android targets in their own jobs, which build the app, boot a simulator or an emulator, and install the app. Targets come from the new config, or from loading an existing one after the confirmation and the install. Expo and bare React Native apps get their build; any other app gets a step to replace. GitHub's mobile jobs follow e2e-inc/e2e-react-native's workflow; GitLab's run on self-managed runners.
@github-actions github-actions Bot added size:XXL 1,000+ changed lines, not counting tests, fixtures, or the lockfile and removed Ready for Human Review size:XL Up to 999 changed lines, not counting tests, fixtures, or the lockfile labels Nov 12, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 26 files

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread packages/e2e/src/config/targets.ts Resolved
Comment thread packages/e2e/templates/ci/gitlab-ci.yml Resolved
Comment thread skills/e2e/references/running.md Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/cli/init/ci.ts Resolved
Upload the emulator log from the GitLab Android job, count peer and optional engine dependencies, ask nothing for a new config with no engine, key caches on Bun's bun.lockb when that is the lockfile, and run the symlink test with a Windows junction.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 6 files (changes from recent commits).

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Turn on auto-fix | Re-trigger cubic

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines, not counting tests, fixtures, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants