Skip to content

test(mosaic): add screenshot testing POC - #10451

Draft
austincalvelage wants to merge 8 commits into
mainfrom
austin/snapshot-test-poc
Draft

austincalvelage wants to merge 8 commits into
mainfrom
austin/snapshot-test-poc

Conversation

@austincalvelage

@austincalvelage austincalvelage commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Description

Token changes can alter text, borders, and layout without failing behavior tests. Add six Mosaic screenshot tests covering both open switchers, the Account overlay in light, dark, and narrow layouts, and an edit-name form with banner and field errors.

Local commands and CI use the same digest-pinned Playwright 1.62.1 Docker image on Linux amd64. The suite bundles Inter, waits for its fonts, disables CSS motion, hides carets, and checks the narrow profile content for horizontal overflow. Only Linux baselines are kept.

Run pnpm --filter @clerk/mosaic test:visual to compare. Run pnpm --filter @clerk/mosaic test:visual:update to replace baselines, then review and commit the images. Both commands require Docker and isolate Linux dependencies from the host checkout.

The CI comparison is advisory during rollout. Image-build failures still fail the job. Screenshot failures upload reference, actual, and diff images for seven days. The first container comparison caught all six font changes and uploaded its diffs while the job remained green. Review repeated runs at the same commit before making comparisons required. All six comparisons pass in the latest Linux comparison check. The initial rollout also passed twice at the same commit.

Container baselines and the original token-change check

Linux dark Account baseline

Linux narrow Account baseline

The original POC also caught a temporary secondary-text token change. That change was reverted.

Detected secondary-text token change

Checklist

  • pnpm test runs as expected for Mosaic.
  • pnpm build runs as expected for Mosaic and its dependencies.
  • All six container screenshot comparisons pass against latest main.
  • Scoped lint and formatting pass.
  • Failed comparisons upload diff images without failing the advisory visual job.
  • (If applicable) JSDoc comments have been added or updated for any package exports
  • (If applicable) Documentation has been updated

Type of change

  • 🐛 Bug fix
  • 🌟 New feature
  • 🔨 Breaking change
  • 📖 Refactoring / dependency upgrade / documentation
  • other: test infrastructure

@changeset-bot

changeset-bot Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 29d69f1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
clerk-js-sandbox Ready Ready Preview Oct 27, 2026 9:55pm UTC
swingset Ready Ready Preview Oct 27, 2026 9:55pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@clerk/astro

npm i https://pkg.pr.new/@clerk/astro@10451

@clerk/backend

npm i https://pkg.pr.new/@clerk/backend@10451

@clerk/chrome-extension

npm i https://pkg.pr.new/@clerk/chrome-extension@10451

@clerk/clerk-js

npm i https://pkg.pr.new/@clerk/clerk-js@10451

@clerk/electron

npm i https://pkg.pr.new/@clerk/electron@10451

@clerk/electron-passkeys

npm i https://pkg.pr.new/@clerk/electron-passkeys@10451

@clerk/eslint-plugin

npm i https://pkg.pr.new/@clerk/eslint-plugin@10451

@clerk/expo

npm i https://pkg.pr.new/@clerk/expo@10451

@clerk/expo-biometrics

npm i https://pkg.pr.new/@clerk/expo-biometrics@10451

@clerk/expo-google-signin

npm i https://pkg.pr.new/@clerk/expo-google-signin@10451

@clerk/expo-native-components

npm i https://pkg.pr.new/@clerk/expo-native-components@10451

@clerk/expo-passkeys

npm i https://pkg.pr.new/@clerk/expo-passkeys@10451

@clerk/express

npm i https://pkg.pr.new/@clerk/express@10451

@clerk/fastify

npm i https://pkg.pr.new/@clerk/fastify@10451

@clerk/hono

npm i https://pkg.pr.new/@clerk/hono@10451

@clerk/localizations

npm i https://pkg.pr.new/@clerk/localizations@10451

@clerk/mosaic

npm i https://pkg.pr.new/@clerk/mosaic@10451

@clerk/nextjs

npm i https://pkg.pr.new/@clerk/nextjs@10451

@clerk/nuxt

npm i https://pkg.pr.new/@clerk/nuxt@10451

@clerk/react

npm i https://pkg.pr.new/@clerk/react@10451

@clerk/react-router

npm i https://pkg.pr.new/@clerk/react-router@10451

@clerk/shared

npm i https://pkg.pr.new/@clerk/shared@10451

@clerk/tanstack-react-start

npm i https://pkg.pr.new/@clerk/tanstack-react-start@10451

@clerk/testing

npm i https://pkg.pr.new/@clerk/testing@10451

@clerk/ui

npm i https://pkg.pr.new/@clerk/ui@10451

@clerk/upgrade

npm i https://pkg.pr.new/@clerk/upgrade@10451

@clerk/vue

npm i https://pkg.pr.new/@clerk/vue@10451

commit: 65a2009

@alexcarpenter

alexcarpenter commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

This is awesome, always found visual tests to be a bit flaky, mostly comes from the rendering environment, so maybe we:

  1. Pin the environment. Baselines are *-darwin.png, so Linux CI won't match. Generate and compare only in the pinned Playwright Docker image, locally and in CI.
  2. Self-host a font in test/public instead of Arial, which is likely missing on Linux (not verified).
  3. Kill motion globally. reducedMotion only helps where CSS respects it. Add a setup stylesheet that disables animations/transitions and hides the caret.
  4. Roll out non-blocking. Upload diffs as artifacts and rerun the same commit to measure flake rate before making it required.
  5. Prefer layout assertions for layout bugs, e.g. scrollWidth === clientWidth overflow checks.

I haven't run the suite; these come from reading the code.

This branch was successfully deployed

2 active deployments
Preview – swingset — 29d69f1d Deployed Oct 27, 2026 by vercel[bot]
Preview – clerk-js-sandbox — 29d69f1d Deployed Oct 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants