Skip to content

feat(e2e): run the suite against vue-starter-template, and fix what it exposed - #2677

Open
Maciek Kucmus (mkucmus) wants to merge 34 commits into
mainfrom
ci/e2e-suite-against-starter
Open

Maciek Kucmus (mkucmus) wants to merge 34 commits into
mainfrom
ci/e2e-suite-against-starter

Conversation

@mkucmus

@mkucmus Maciek Kucmus (mkucmus) commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a workflow that builds vue-starter-template, serves it with nuxt preview, and runs test:e2e against it. Runs nightly and on manual dispatch.

Retags the suite from @vue-demo-store to @frontends, so the tag names the behaviour rather than one template.

Along the way it fixes three defects the suite exposed. Two are in @shopware/cms-base-layer, a published package, despite the commit subjects reading as test work:

  • Listing filters wrote the URL only after the listing request resolved, so a slow or failed request left the filter out of the address bar and lost it on refresh or share. Reordered to push first, matching changePage, changeLimit and sorting. Applied to both the sidebar and horizontal filter components.
  • Checkout always sent guest: true, so a customer who chose to create an account got a guest account and could never sign in.

closes #2649
closes #2651

Type of change

Bugfix and new feature (non-breaking)

ToDo's

  • Documentation added/updated
  • E2E-Tests added/updated
  • Changeset file provided - patch for @shopware/cms-base-layer

Result

21 passed, 3 flaky, 2 skipped, 0 failed on CI in 7.4m.

It does not run on pull requests. The path filter it started with matched 19 of the last 25 merged PRs, at a median of 12 minutes a run, which is the same trade #2706 made for the example builds.

Because nothing is gated on it any more, a failure now notifies Slack through failed-job-check.yml, and the test step no longer uses continue-on-error (with it, the run concludes success and the notifier never fires).

Also fixes the branch link in that Slack message, which pointed at github.com/<repo>/<branch> and 404'd. It affects all seven workflows listed in failed-job-check.yml, not just this one.

Every flaky scenario traces to one cause, now filed as #2682: Store API calls that fail without a response. Nitro logs none of it, so this branch adds a collector (apps/e2e-tests/fixtures.ts) that records every failed call, how long it ran before dying, and any error page the browser was shown. A summary lands in the job page and the raw JSONL in the artifact.

That instrumentation corrected two assumptions worth stating. Requests are not hanging: on CI they die at a median of 171ms and none exceeded 8s, so a request timeout would never fire. And about a quarter of what looked like backend failures are net::ERR_ABORTED, which is the client cancelling in flight, mostly the country combobox firing per keystroke.

The classifier itself had two bugs, fixed in 13a1d4ea: it logged a success even on a 4xx or 5xx, and counted 429s, 5xx and error pages as no-response failures. They pulled the rate in opposite directions and the corrected figure is 4.8% rather than 9.5%. The latency numbers above never went through that path and are unaffected. Every CI run on this branch predates the fix, so treat its reported rate as approximate until the first nightly run.

Built and served locally rather than pointed at a Vercel preview, so a run tests the branch it runs on. Sets NUXT_PUBLIC_SHOPWARE_DEV_STOREFRONT_URL because the template default is the demo store domain, which the starter's sales channel rejects (#2585).

Review follow-up

All four findings addressed.

  1. Account creation. createAccount is now a model on CustomerBaseInfo, bound from the checkout page. Guest validates email and billing address, account validates the whole base-info form, and guest is derived rather than hardcoded. One spec per mode; the account one signs in over the Store API, which is what proves the customer is not a guest, since a guest cannot sign in at all.
  2. Path filter. All seven @shopware/* dependencies are symlinked into packages/, so a change in one rebuilds the starter without touching the template. Added the six runtime packages and root dependency metadata. api-gen is left out, it is codegen only.
  3. Listing synchronisation. The wait is armed before the click and resolves only on a successful listing response whose body carries the filter, then waits for the render. Bounded at 30s with specific failure messages. Applied to manufacturer and property filters on both listings, and to sorting.
  4. Variant identity. data-product-id on the add-to-cart button and the cart line item; the test asserts exactly one line item carries the id the button submitted, rather than checking an image is visible.

A second round of nine findings is answered in this comment. All are fixed on the branch, including one real regression it caught in SwProductListingFiltersHorizontal.vue, where property-group filters stopped rendering as active.

What this closes

What is still queued, and why

Worth a reviewer's attention

Nothing in CI runs e2e against vue-demo-store, and retagging removed the last demo-store-specific suite. Three published-package files change here (SwProductListingFilters.vue, SwProductListingFiltersHorizontal.vue, SwProductAddToCart.vue) with no automated coverage of the app most affected by them. The reordering matches the convention already used elsewhere in the same package, and no template overrides those components, but that is analysis rather than a test.

Also fixes a -- in the e2e README that silently drops arguments under pnpm 11.

@vercel

vercel Bot commented Aug 31, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
frontends-starter-template-extended Ready Ready Preview Sep 14, 2026 8:21am UTC
frontends-vue-starter-template Ready Ready Preview Sep 14, 2026 8:21am UTC

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds non-gating CI coverage for vue-starter-template and adapts the E2E suite to run independently of demo-store content.

Changes:

  • Adds a scheduled and PR-triggered starter-template E2E workflow.
  • Retags and updates Playwright tests, page objects, selectors, and synchronization.
  • Fixes checkout guest validation and listing-filter URL ordering.

Reviewed changes

Copilot reviewed 46 out of 46 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
templates/vue-starter-template/app/pages/wishlist.vue Adds empty-wishlist selector.
templates/vue-starter-template/app/pages/checkout/index.vue Fixes guest validation and adds selectors.
templates/vue-starter-template/app/components/wishlist/ProductTile.vue Adds wishlist product selectors.
templates/vue-starter-template/app/components/LoginForm.vue Adds login selectors.
templates/vue-starter-template/app/components/layout/MiniCart.vue Adds mini-cart selectors.
templates/vue-starter-template/app/components/layout/header/Search.vue Adds search selector.
templates/vue-starter-template/app/components/layout/Header.vue Adds account, wishlist, and cart selectors.
templates/vue-starter-template/app/components/layout/AccountMenu.vue Adds account-link selector.
templates/vue-starter-template/app/components/checkout/ShippingMethods.vue Adds shipping-method selector.
templates/vue-starter-template/app/components/checkout/ProductTile.vue Adds cart product selectors.
templates/vue-starter-template/app/components/checkout/CustomerBaseInfo.vue Adds checkout email selector.
templates/vue-starter-template/app/components/checkout/CustomerAddress.vue Adds address-field selectors.
templates/vue-starter-template/app/components/account/personal/DataForm.vue Adds profile-form selectors.
templates/vue-starter-template/app/components/account/LoginForm.vue Adds registration-navigation selector.
packages/cms-base-layer/app/components/SwProductListingFilters.vue Writes filter URLs before fetching.
apps/e2e-tests/utils/listing.ts Adds shared listing helpers.
apps/e2e-tests/utils/form.ts Adds cross-template form helpers.
apps/e2e-tests/utils/data-helpers.ts Adds unique test emails.
apps/e2e-tests/tests/search.spec.ts Makes search tests content-independent.
apps/e2e-tests/tests/registrationUser.spec.ts Updates registration assertions and data.
apps/e2e-tests/tests/myAccountTests.spec.ts Retags account tests and adjusts timeout.
apps/e2e-tests/tests/login.spec.ts Retags login tests.
apps/e2e-tests/tests/createOrder.spec.ts Adapts order scenarios to starter checkout.
apps/e2e-tests/tests/checkSearch.spec.ts Enables generic search-listing tests.
apps/e2e-tests/tests/checkProductVariants.spec.ts Retags variant coverage.
apps/e2e-tests/tests/checkForSeoUrl.spec.ts Uses generic navigation.
apps/e2e-tests/tests/checkCategory.spec.ts Retags category tests.
apps/e2e-tests/tests/addToWishlist.spec.ts Uses isolated wishlist accounts.
apps/e2e-tests/tests/addToCart.spec.ts Adapts wishlist-to-cart coverage.
apps/e2e-tests/tests/addProductReview.spec.ts Documents the unsupported review scenario.
apps/e2e-tests/README.md Documents starter-template E2E execution.
apps/e2e-tests/playwright.config.ts Adjusts Playwright timeouts.
apps/e2e-tests/page-objects/WishlistPage.ts Adds wishlist synchronization and cart action.
apps/e2e-tests/page-objects/SearchResultPage.ts Uses shared listing helpers.
apps/e2e-tests/page-objects/RegisterPage.ts Supports inline starter registration.
apps/e2e-tests/page-objects/ProductPage.ts Adapts variant-cart handling.
apps/e2e-tests/page-objects/MyAccountPage.ts Supports alternate profile navigation.
apps/e2e-tests/page-objects/LoginPage.ts Improves login field synchronization.
apps/e2e-tests/page-objects/HomePage.ts Adds content-independent navigation helpers.
apps/e2e-tests/page-objects/CheckoutPage.ts Supports starter guest checkout.
apps/e2e-tests/page-objects/CategoryPage.ts Uses shared listing helpers.
apps/e2e-tests/page-objects/CartPage.ts Waits correctly for the mini-cart.
apps/e2e-tests/package.json Changes the suite tag to @frontends.
AGENTS.md Updates E2E contributor guidance.
.github/workflows/e2e-starter-template.yml Adds the starter-template CI workflow.
.changeset/olive-hounds-repeat.md Records the listing-filter fix.
Suppressed comments (1)

apps/e2e-tests/utils/listing.ts:81

  • The existing product image is already visible when the URL is pushed, before changeCurrentPage starts, so this wait can return with page 1 still rendered. Wait until pagination marks page 2 as current (or otherwise observe the listing response) before continuing.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/e2e-starter-template.yml Outdated
Comment thread apps/e2e-tests/utils/listing.ts Outdated
Comment thread apps/e2e-tests/page-objects/ProductPage.ts Outdated
Comment thread apps/e2e-tests/tests/myAccountTests.spec.ts Outdated
Comment thread .github/workflows/e2e-starter-template.yml Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security review of 4b7c997 (“synchronize”): no medium, high, or critical issues.

Checked the new e2e workflow (contents: read, pull_request not pull_request_target, no secrets), SSR/network diagnostics (URLs and status only; artifacts are demo-store traffic), listing-from-URL (query values stay strings; Store API remains the trust boundary), and checkout guest: !createAccount (password rules apply when creating an account). The earlier CodeQL weak-randomness finding is already fixed with uniquePassword().

No prior automation security threads. No inline findings.

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security review of 9215075 (“synchronize”): no medium, high, or critical issues.

New since the last pass is only the build step (pnpm exec turbo run build --filter=vue-starter-template). That is a static command: no untrusted interpolation, still pull_request (not pull_request_target), contents: read, and no secrets.

Re-checked the rest of the PR: SSR/network diagnostics still log URLs and status only; listing filters keep query values as strings with the Store API as the trust boundary; checkout still derives guest: !createAccount and validates the password path when creating an account; uniquePassword() remains CSPRNG-backed. Prior automation review had no finding threads to re-open.

No inline findings.

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

@patzick Patryk Tomczyk (patzick) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — changes requested

CI is fully green (13/13, e2e included). One real regression in a published package, the rest are minor/nit.

Blocking

  1. SwProductListingFiltersHorizontal.vue — the rewritten hasActiveFilter never reports property-group filters as active. Property filters carry the group's own code, not "properties", so the new branch is dead and the changed fallback (return false) drops the behaviour the old fallback provided. Inline comment has the detail.

Non-blocking (inline)
2. CmsElementProductListing.vue — the local limit ref no longer resyncs from the URL, so Back/Forward updates the products but not the "Per Page" select or the skeleton count.
3. CmsElementProductListing.vue — the watcher is now the only fetch path and passes route.query verbatim, so min-price/rating/shipping-free reach the API as strings where they used to be typed; the post-filter branch in useListingFilters became unreachable as a result.
4. changePage/changeLimit scroll while the element they target is being unmounted for the skeleton.
5. ListingPage.checkFirstOptionIn picks a checkbox by page-wide index delta.
6. ListingPage.NON_PROPERTY_FILTERS mixes rendered labels with API codes.
7. checkForSeoUrl.spec.ts — the last assertion lost its wait and is now vacuous.
8. WishlistPage.countEntries() is unused.
9. instrument-ssr.mjs writes synchronously on every successful SSR call, inflating the latency it measures.

Open thread, your call: Maciej D (@mdanilowicz)'s failed-job-check.yml point is still unresolved. With continue-on-error: true and no gate anywhere, a total e2e failure leaves the job green and notifies nobody — the reporting-only trade-off is reasonable, but it should be an explicit team decision rather than an unresolved thread.

Also noting the gap you flagged yourself: three published cms-base-layer components change with no automated coverage of the app most affected. Finding 1 is exactly what that gap lets through.

Comment thread packages/cms-base-layer/app/components/SwProductListingFiltersHorizontal.vue Outdated
Comment thread apps/e2e-tests/page-objects/ListingPage.ts Outdated
Comment thread apps/e2e-tests/page-objects/ListingPage.ts Outdated
Comment thread apps/e2e-tests/tests/checkForSeoUrl.spec.ts
Comment thread apps/e2e-tests/page-objects/WishlistPage.ts Outdated
Comment thread apps/e2e-tests/instrument-ssr.mjs

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security review of da17a8a (“synchronize”): no medium, high, or critical issues.

Delta since the last pass is review follow-up: the e2e job is now schedule/workflow_dispatch only (contents: read, no secrets, static commands), buildCriteria allowlists listing query keys instead of forwarding route.query verbatim, and SSR diagnostics still log URLs/status only. Checkout still derives guest: !createAccount with password validation on the account path; uniquePassword() remains CSPRNG-backed.

No prior automation security finding threads to re-open. No inline findings.

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow misses its required PR trigger, listing failures can become unhandled rejections, and network statistics are currently inaccurate.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

apps/e2e-tests/scripts/summarise-network.mjs:74

  • entries also contains storefront-error-page and storefront-5xx document records, so treating its full length as Store API attempts adds browser error pages as extra API calls; the per-side denominator below has the same problem. Count only store-api-* records when deriving API totals.
const attempted = completed + entries.length;
  • Files reviewed: 62/62 changed files
  • Comments generated: 5
  • Review effort level: Balanced

Comment thread .github/workflows/e2e-starter-template.yml
Comment thread .github/workflows/failed-job-check.yml
Comment thread apps/e2e-tests/fixtures.ts
Comment thread apps/e2e-tests/scripts/summarise-network.mjs

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security review of 13a1d4ea (“synchronize”): no medium, high, or critical issues.

Delta since the last pass is diagnostics and error-handling only: successful Store API responses are the only attempts counted, transport failures are restricted to store-api-no-response, and listing refetches catch Store API rejections instead of leaving them unhandled. No new sinks.

Re-checked the rest of the PR: the e2e job remains schedule/workflow_dispatch only (contents: read, no secrets, static commands); buildCriteria still allowlists listing query keys; checkout still derives guest: !createAccount with password validation on the account path; diagnostics still log URLs/status/IPs/trace ids of demo-store traffic, not bodies or credentials; uniquePassword() remains CSPRNG-backed.

No prior automation security finding threads to re-open. No inline findings.

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

@mkucmus

Copy link
Copy Markdown
Contributor Author

Thanks, this was a useful review. All nine are fixed and on the branch, plus the open question at the end.

1 (blocking). Right, and it was my regression. The fallback is back as you wrote it, comment included.

2. limit.value now resyncs from the query in the same watcher.

3. Added a buildCriteria() helper that converts the query strings to numbers and booleans, so the API gets typed values again. Deleted the unreachable post-filter branch, its import, and the searchCriteriaForRequest export. The computed stays as an internal for executeSearch; search.vue builds its own post-filter independently.

4. scrollToListing() awaits until(loading).toBe(false) before scrolling.

5. checkFirstOptionIn scopes to the panel's own container instead of a page-wide index delta. Went with your second suggestion and added data-testid="listing-filter-{code}" in SwProductListingFilter.vue.

6. Now matches on filter.code from that same testid, so the elimination list is codes only and survives localisation.

7. Added a wait for the destination listing to render.

8. Dropped.

9. Buffered, flushed on a 2s interval and on exit.

While fixing 9 I found two bugs in my own measurement pointing opposite ways: the attempts log counted 4xx and 5xx twice, and transport counted 429s, 5xx and error pages as no-response failures. The number moved from 9.5% to 4.8%. Every percentage in the description before this should be read as approximate.

On the open thread. Settled by removing the pull_request trigger entirely: the suite is nightly plus manual dispatch. continue-on-error is gone, so a failure fails the run, and failed-job-check.yml notifies Slack the same way the audit check does. Nothing is gated because nothing runs on PRs any more.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security review of c994ac92 (“synchronize”): no medium, high, or critical issues.

Delta since the last pass is a one-line diagnostics refactor: summarise-network.mjs reuses apiCalls() for the attempted-count denominator. Counting-only; no new sinks.

Re-checked the rest of the PR: the e2e job remains schedule/workflow_dispatch only (contents: read, no secrets, static commands); buildCriteria / applyQueryToFilters still allowlist listing query keys; checkout still derives guest: !createAccount with password validation on the account path; diagnostics still log URLs/status/IPs/trace ids of demo-store traffic, not bodies or credentials; uniquePassword() remains CSPRNG-backed.

No prior automation security finding threads to re-open. No inline findings.

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security review of 507e32fc (“synchronize”): no medium, high, or critical issues.

Delta since the last pass is a Slack mrkdwn path fix in failed-job-check.yml: the failure link now uses /tree/${head_branch} instead of interpolating the branch as a repo-root path. Same GitHub-controlled values as before (server_url, repository, workflow_run.head_branch); the notifier still only fires for workflow_run completions on main/prod. Not a new sink.

Re-checked the rest of the PR: the e2e job remains schedule/workflow_dispatch only (contents: read, no secrets, static commands); buildCriteria / applyQueryToFilters still allowlist listing query keys; checkout still derives guest: !createAccount with password validation on the account path; diagnostics still log URLs/status/IPs/trace ids of demo-store traffic, not bodies or credentials; uniquePassword() remains CSPRNG-backed.

No prior automation security finding threads to re-open. No inline findings.

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

@patzick Patryk Tomczyk (patzick) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — all 9 previous findings addressed

Every item from my last review is fixed, and correctly:

  • The blocking onehasActiveFilter fallback restored to properties.size > 0 with a comment explaining why it must stay a fallback. ✅
  • limit now resyncs from the URL inside the watcher. ✅
  • buildCriteria() normalises query strings to numbers/booleans, drops unknown params, and the unreachable post-filter branch plus the unused searchCriteriaForRequest export are gone. Rejections are now caught rather than floating. ✅
  • scrollToListing() waits on until(loading).toBe(false). ✅ (Vue's pre-flush scheduler queues the route watcher before router.push resolves, so loading is already true by then — the wait is real, not a no-op.)
  • ListingPage scopes the option to the panel via the new listing-filter-${filter.code} testid, and matches filter codes instead of localised button text. ✅ Nice generalisation.
  • checkForSeoUrl waits for the destination to render. ✅
  • countEntries() removed; SSR logging buffered with a timer + exit flush. ✅

And Maciej D (@mdanilowicz)'s failed-job-check.yml thread is resolved properly — continue-on-error dropped, workflow added to the notify list.

Two things from the latest push

  1. The docs now contradict the workflow (blocking, two lines). Dropping the pull_request trigger and continue-on-error left apps/e2e-tests/README.md:27,29 and AGENTS.md:131 describing behaviour that no longer exists. Inline.

  2. The harness fixes are unverified. With the PR trigger gone, E2E Starter Template no longer runs here — the last run was on 9215075, before the ListingPage rework, the checkForSeoUrl wait and the new listing-filter-* testid. Those are exactly the changes most likely to break the suite, and nothing has exercised them. A workflow_dispatch run on ci/e2e-suite-against-starter before merge would close it.

The scheduled-only trade-off itself reads as the right call given the Slack notification — just worth confirming the team is happy that the PR titled "run the e2e suite against vue-starter-template" runs nightly rather than per-PR, since #2649 closes with it.

Comment thread apps/e2e-tests/README.md Outdated
Comment thread AGENTS.md Outdated
Comment thread .github/workflows/e2e-starter-template.yml
@mkucmus

Copy link
Copy Markdown
Contributor Author

Both fixed, and thanks for catching the docs drift. That is exactly the kind of thing the change should not have left behind.

1. Docs. apps/e2e-tests/README.md no longer claims a pull request trigger, and both it and AGENTS.md now say the job runs nightly and on dispatch, that nothing is gated on it, and that a failure notifies Slack. I swept the rest of the repo for the same two claims: the only remaining mention of continue-on-error is the comment in the workflow explaining why it is absent, which is intentional.

While in there I found one more the review did not name. The README's "To reproduce the CI run locally" block built with pnpm --filter vue-starter-template build, while the workflow builds through turbo run build --filter=vue-starter-template. It works locally because postinstall builds the packages, but it is not what the workflow does, and it silently picks up stale linked packages if they are not built. Aligned. The second occurrence, under the accessibility section, is left alone since it makes no claim about reproducing CI.

2. Verification. Fair point, and I had the same worry. Dispatched a run on this branch: https://github.com/shopware/frontends/actions/runs/34515297522

I will report the result here either way rather than only if it is green.

On the trade-off. Agreed it should be an explicit team call rather than something that slips through with the PR. To be clear about what changed and why: the path filter matched 19 of the last 25 merged PRs at a median of 12 minutes a run, which is the same trade #2706 made for the example builds. #2649 asked for the suite to run in CI, and it does; nightly rather than per-PR, with a Slack notification on failure. If the team would rather have it per-PR, the trigger is three lines and I am happy to put it back.

@mkucmus

Copy link
Copy Markdown
Contributor Author

Dispatch run finished green: https://github.com/shopware/frontends/actions/runs/34515297522

18 passed, 6 flaky, 2 skipped, 0 failed, 13.5 minutes, on 507e32fc. Every step succeeded including Run tests, which is the one that counts now: with continue-on-error gone, a real failure fails the job.

That covers what you flagged as unexercised. 507e32fc is downstream of da17a8a7, so this run did exercise the ListingPage rework, the checkForSeoUrl wait and the new listing-filter-* testid.

Two caveats so the number is not read as more than it is. The 6 flaky are the retry-absorbed store-api drops tracked in #2682, not new; the count moves run to run. And the run predates the docs commit above, which is fine since docs cannot affect the suite, but it does mean the green tick is on the code as reviewed rather than on the branch tip.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security review of 288de64e (“synchronize”): no medium, high, or critical issues.

Delta since the last pass is documentation only (AGENTS.md, apps/e2e-tests/README.md): the workflow is described as nightly/workflow_dispatch, and the local repro uses the same turbo build as CI. No new sinks.

Re-checked the rest of the PR: the e2e job remains schedule/workflow_dispatch only (contents: read, no secrets, static commands). buildCriteria / applyQueryToFilters still allowlist listing query keys and coerce numbers/booleans. Checkout guest: !createAccount is the intended guest-vs-account split. Diagnostics log URLs, status, IPs, and trace ids only. Slack notifier still interpolates GitHub workflow_run context on main/prod only.

No prior automation security-finding threads to re-open. CodeQL insecure-randomness on createOrder.spec.ts remains resolved via uniquePassword().

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review of 273fbf71 (“synchronize”): no medium, high, or critical issues.

Delta since the last pass is a merge of main (#2698 CountrySearchSelect, #2700 checkout flow, #2719 changesets v3). Those changes introduce no new sinks here. Checkout still derives guest: !createAccount and applies password rules only on the account path; uniquePassword() remains CSPRNG-backed.

Re-checked the rest of the PR: the e2e job remains schedule/workflow_dispatch only (contents: read, no secrets, static commands). buildCriteria / applyQueryToFilters still allowlist listing query keys and coerce numbers/booleans. Diagnostics log URLs, status, IPs, and trace ids of demo-store traffic, not bodies or credentials. Slack notifier still interpolates GitHub workflow_run context on main/prod only.

No prior automation security-finding threads to re-open. CodeQL insecure-randomness on createOrder.spec.ts remains resolved via uniquePassword().

Open in Web View Automation 

Sent by Cursor Automation: Review pull requests for exploitable security issues and flag only validated findings before merge

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to run its full agentic suite in this review.

Pull request overview

Copilot reviewed 62 out of 62 changed files in this pull request and generated 3 comments.

Suppressed comments (3)

packages/cms-base-layer/app/composables/useListingFilters.ts:1

  • These checks drop valid falsy values (e.g., 0 for min/max price or rating, and false for boolean flags) because they use truthiness. This can cause the URL to lose an explicitly-set filter value and the listing to refetch with different criteria. Use explicit !== undefined / != null checks (and for booleans, check === true or presence semantics) when deciding whether to write each field into the query.
    packages/cms-base-layer/app/composables/useListingFilters.ts:1
  • The shipping-free filter is handled as tri-state elsewhere in this PR (e.g., chips treat it as active when the value is not undefined), but here it is stored via Boolean(value) and the reset button uses !!selectedFilters[\"shipping-free\"]. If value can be the string 'false', Boolean('false') becomes true (wrong), and if it becomes false, the chip logic may still consider it active while the reset button does not. Normalize shipping-free consistently (e.g., store only true | undefined, parse string values explicitly, and align showResetFiltersButton to the same semantics).
    packages/cms-base-layer/app/composables/useListingFilters.ts:1
  • The shipping-free filter is handled as tri-state elsewhere in this PR (e.g., chips treat it as active when the value is not undefined), but here it is stored via Boolean(value) and the reset button uses !!selectedFilters[\"shipping-free\"]. If value can be the string 'false', Boolean('false') becomes true (wrong), and if it becomes false, the chip logic may still consider it active while the reset button does not. Normalize shipping-free consistently (e.g., store only true | undefined, parse string values explicitly, and align showResetFiltersButton to the same semantics).

Comment on lines +7 to +9
export const NETWORK_LOG = "diagnostics/network-failures.jsonl";
export const ATTEMPT_LOG = "diagnostics/store-api-attempts.log";

Comment on lines +22 to +36
const record = (entry: Entry) => {
try {
mkdirSync(dirname(NETWORK_LOG), { recursive: true });
appendFileSync(
NETWORK_LOG,
`${JSON.stringify({
at: new Date().toISOString(),
test: testInfo.titlePath.join(" > "),
...entry,
})}\n`,
);
} catch {
// Diagnostics must never fail a test.
}
};
Comment on lines +113 to +114
mkdirSync(dirname(ATTEMPT_LOG), { recursive: true });
appendFileSync(ATTEMPT_LOG, `${peer?.ipAddress ?? "unknown"}\n`);

@patzick Patryk Tomczyk (patzick) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

Both items from the last round are closed, and my three threads are resolved:

  • 288de64e corrects README.md and AGENTS.md to "nightly and on manual dispatch, not on pull requests" and "nothing is gated, a failure notifies Slack". The local-repro snippet also switched to turbo run build, which matches the workflow. ✅
  • The workflow_dispatch run on 507e32f succeeded, so the ListingPage panel-scoping rework, the checkForSeoUrl wait and the listing-filter-${filter.code} testid have all actually executed. ✅

The merge from main shrank this PR usefully: #2700 landed the createAccount / guest: !createAccount rework independently, so checkout/index.vue is down to two data-testid lines here and CustomerBaseInfo.vue to three. The version in main is better than what I reviewed — baseInfoRules makes the password rule conditional on createAccount, so handleRegister can validate the whole form in one check instead of special-casing email. hasActiveFilter's property-group fallback survived the merge intact.

Two follow-ups, neither blocking

  1. Worth one more workflow_dispatch on the merged head before merging. The last e2e run predates the merge, and the merge is exactly what the checkout and address specs drive: #2700 restructured handleSaveAddress into handleRegister with an isRegistering state and put the whole form behind v-if="!isUserSession", while #2698 reworked CountrySearchSelect — which utils/form.ts:selectCountry probes by tag name. The testids all line up on inspection, so I expect it to pass; it is just cheap certainty on the one thing nothing has exercised.

  2. Copilot's fixtures.ts point is fair. NETWORK_LOG and ATTEMPT_LOG are appended at fixed relative paths and never truncated, so locally the second run's summarise-network.mjs output mixes in the first run's calls and the failure rate stops meaning anything. Harmless on a fresh runner. Truncating once per run (or keying the filename on the run) would fix it — fine as a follow-up.

Still open and yours to close: Maciej D (@mdanilowicz)'s two threads on playwright.config.ts:34 and the @frontends tag name, both already answered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CF][Templates] Rebuild login and register page objects for the starter [CF][Templates] Run the e2e suite in CI against vue-starter-template

5 participants