Visual regression testing for GitHub Actions — baselines live in build artifacts, not your git history.
- Pushes to your default branch publish a
vrt-baselineartifact. No screenshots are committed to git. - Pull requests download the latest baseline from the base branch, diff it against the PR's screenshots, and report the result — a sticky PR comment, a step summary, and an HTML report that opens straight in your browser.
- Merging the PR is the approval. There's no "update baselines" step: once the PR lands, the default branch rebuilds and publishes fresh baselines automatically.
If the baseline artifact is missing or expired (GitHub Actions artifacts expire after at most 90 days), every screenshot reports as "new," the job passes with a note, and the next default-branch build self-heals by publishing a new baseline.
The baseline used for comparison is the newest baseline artifact on the branch, even if that run later failed — the action does not check the producing run's conclusion. Keep your baseline-upload step after your capture step succeeds so a broken run doesn't publish a bad baseline.
Screenshot capture is out of scope — bring your own tool (Playwright, Storybook, Cypress, ...) and point this action at the directory it writes PNGs to. See examples/playwright.yml and examples/storybook.yml.
name: Visual tests
on:
push:
branches: [main]
pull_request:
workflow_dispatch: # manual "Run workflow" republishes the baseline
permissions:
contents: read
actions: read # download baseline artifacts from other runs
pull-requests: write # sticky results comment
jobs:
vrt:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
# ... capture screenshots into ./screenshots with your tool of choice ...
- uses: fiestaboard/visual-regression-action@v2
with:
screenshots-dir: ./screenshotsOn a push to main this publishes a baseline. On a pull request it downloads the base branch's baseline, diffs, and reports — no other configuration required.
| Input | Default | Description |
|---|---|---|
screenshots-dir |
(required) | Directory containing the captured PNG screenshots |
github-token |
${{ github.token }} |
Token used to look up/download baseline artifacts and post the PR comment |
mode |
auto |
auto | baseline | compare. auto = pull requests compare, pushes to the default branch publish baselines |
key |
'' |
Optional suffix for artifact names — use for matrix builds or multiple apps in one repo |
threshold |
0.1 |
Per-pixel color sensitivity passed to pixelmatch (0-1, smaller = more sensitive) |
diff-ratio |
0 |
Fraction of differing pixels (0-1) above which an image counts as changed |
fail-on-diff |
true |
Fail the job when screenshots changed or were removed |
comment |
true |
Post/update a sticky PR comment with the results |
baseline-branch |
'' |
Branch whose baseline artifact to compare against. Defaults to the PR base branch (or the default branch) |
retention-days |
'' |
Artifact retention override in days (empty = repo default) |
| Output | Description |
|---|---|
changed |
Number of changed screenshots |
added |
Number of added screenshots |
removed |
Number of removed screenshots |
unchanged |
Number of unchanged screenshots |
has-changes |
true when changed + removed > 0 |
report-path |
Local path of the generated HTML report |
Baseline mode (a push to the default branch) sets no outputs — there's nothing to compare yet.
Results surface in three places:
- Sticky PR comment — a pass/fail headline, an approval checklist with one box per changed or removed screenshot plus Approve all, a table of added screenshots, and a link that opens the report. It updates in place on every push rather than adding a new comment each time.
$GITHUB_STEP_SUMMARY— the same summary rendered on the workflow run page.vrt-report[-<key>].htmlartifact: one self-contained HTML file, uploaded un-zipped so the link in the PR comment opens it directly in the browser. There's nothing to download or unzip. The reviewer is built with FiestaUI, the FiestaBoard design system. Each changed screenshot gets a large comparison stage with three views: Swipe (drag a divider between baseline and current), Overlay (the current screenshot with detected changes highlighted in pink), and Side-by-side (baseline / current / diff thumbnails). A fullscreen reviewer steps through every change:←/→to move between screenshots,S/O/Dto switch views, scroll to zoom, drag to pan,Escto close. It makes no external requests.
Large reports stay quick to open:
- The report streams. The reviewer shows up as soon as the first few hundred kilobytes arrive. Screenshots fill in while the rest downloads, changed ones first, and each image is decoded only when its card scrolls into view.
- No diff images are shipped. The overlay and diff views are computed in your browser from baseline and current, using the same
thresholdas the compare. - Local changes are cropped. When a change covers only part of a tall screenshot, the report carries just that band (plus 200 px above and below) and says which rows it shows. A page whose size changed always ships whole.
Screenshots are embedded as base64, so a run with many changed or added full-page captures still produces a large file. Viewport-sized captures keep it small.
An intentional visual change leaves the check red — and if the check is required by branch protection, you need a way to accept it without weakening the gate.
The checkbox path: the report comment on the PR has a checkbox for each changed or removed screenshot, plus Approve all. Tick the screenshots you accept. Once every one is ticked, or Approve all is, the failed check reruns and passes automatically. While only some are ticked, an approval-status comment shows the count and nothing reruns. Only users with write access can tick boxes (GitHub restricts editing the bot's comment), and every box pins the current commit, so pushing new commits resets them. Suites with more than 50 changes get only the Approve all box.
The review path uses /vrt approve commands:
- Open the report and hit Review changes. Approve (
A) or reject (R) each screenshot as you step through. - The report assembles a command as you go, e.g.
/vrt approve home.png@1a2b3c4d5e6f old.png@9f8e7d6c5b4a— each entry pins a hash of that screenshot's exact pixels. Copy it. - Post it as a comment on the PR. If you've added the approvals workflow, the failed check reruns automatically (the comment gets a 👀 when the rerun kicks off); otherwise hit "Re-run failed jobs" yourself.
- On the rerun, the action reads the PR's comments, honors approvals whose hashes still match, and passes when every changed/removed screenshot is approved. Approved items stay visible in the report and comment, marked ✅.
Details worth knowing:
- Only comments from users with an
OWNER,MEMBER, orCOLLABORATORassociation count. Approvals are ordinary PR comments, so there's a permanent audit trail of who accepted what. - The hash pin means an approval covers those exact pixels. Push another change to the same screenshot and the stale approval silently stops matching — the check goes red again.
/vrt approve allbulk-approves every changed/removed screenshot — but only while the head it was posted against is still the PR head; any newer push invalidates it./vrt approve all@<commit-sha>(7+ hex chars) is the explicit, commit-pinned form.- Approvals only greenlight the check; baselines still update the normal way, when the merge lands and your default branch republishes them.
- Screenshot names containing whitespace can't be approved individually — use
all@<sha>.
To enable auto-rerun, add this second workflow (examples/vrt-approvals.yml):
name: VRT approvals
on:
issue_comment:
types: [created, edited]
permissions:
contents: read
jobs:
approve:
if: ${{ github.event.issue.pull_request && (startsWith(github.event.comment.body, '/vrt approve') || contains(github.event.comment.body, 'vrt:approve-all')) }}
runs-on: ubuntu-latest
permissions:
contents: read
actions: write # rerun the failed visual check
issues: write # rocket reaction on the approval comment
pull-requests: write # (reactions on PR comments use this scope too)
steps:
- uses: fiestaboard/visual-regression-action@v2
with:
mode: approveMultiple workflows in one repo — if more than one workflow file runs this action (not just multiple steps in the same workflow), give each workflow its own key too; otherwise their baseline artifacts share a name and can cross-contaminate each other's comparisons.
Matrix builds / multiple apps in one repo — give each invocation a distinct key so artifact names don't collide (vrt-baseline-<key>, vrt-report-<key>.html):
- uses: fiestaboard/visual-regression-action@v2
with:
screenshots-dir: ./screenshots
key: ${{ matrix.browser }}Report-only mode — never fail the check, but still branch on whether anything changed:
- uses: fiestaboard/visual-regression-action@v2
id: vrt
with:
screenshots-dir: ./screenshots
fail-on-diff: 'false'
- if: steps.vrt.outputs.has-changes == 'true'
run: echo "Visuals changed — see the PR comment"Comparing against a non-default branch — override which branch's baseline to diff against:
- uses: fiestaboard/visual-regression-action@v2
with:
screenshots-dir: ./screenshots
baseline-branch: releaseMonorepos — run the action twice with different key and screenshots-dir values, one per app:
- uses: fiestaboard/visual-regression-action@v2
with:
screenshots-dir: ./apps/web/screenshots
key: web
- uses: fiestaboard/visual-regression-action@v2
with:
screenshots-dir: ./apps/admin/screenshots
key: adminPull requests from forks run with a read-only GITHUB_TOKEN and can't be granted pull-requests: write. Diffing and reporting still work — the base branch's baseline is public and downloadable, and the HTML report still uploads as an artifact and appears in the step summary. Only the sticky PR comment is skipped, since posting it requires write access this token doesn't have.
The permissions block in the quick start is the full requirement:
permissions:
contents: read
actions: read # download baseline artifacts from other runs
pull-requests: write # sticky results commentWhy is my check red after an intentional visual change?
That's the design: a changed or removed screenshot fails the job by default (fail-on-diff: true) so the change gets a deliberate look before merge. To accept it, use the approval flow: tick the boxes in the PR comment (or post a /vrt approve command) and the check goes green. Once merged, the default branch rebuilds and publishes the new baseline automatically. If you'd rather never fail the check, run report-only mode and branch on has-changes instead.
What if the baseline expired or got deleted?
Nothing breaks — compares report everything as "new" and pass with a note until a baseline exists again. To republish one on demand, keep workflow_dispatch: in your workflow's triggers and hit "Run workflow" on your default branch (manual runs and schedule runs publish baselines in auto mode). Quiet repo? Add a monthly cron so the baseline never ages out of artifact retention:
on:
schedule:
- cron: '0 6 1 * *' # monthly baseline keep-aliveWhat happens on the very first run? There's no baseline artifact yet, so every screenshot reports as "new." The job passes with a note. The first push to your default branch after that publishes the initial baseline, and PRs after that compare normally.
Do artifacts cost money?
Baseline and report artifacts count toward your repository's Actions storage quota like any other artifact. Use retention-days to shorten how long they're kept if storage is a concern; GitHub also expires artifacts automatically after at most 90 days regardless of setting, at which point the next default-branch build self-heals the baseline.
Which run's baseline gets used?
The newest non-expired vrt-baseline[-<key>] artifact whose producing run is on the base branch — regardless of whether that run ultimately succeeded or failed. This is a deliberate v1 simplification: the action does not walk run conclusions, it just picks the newest matching artifact. If a run can fail after uploading the baseline, move the baseline-upload step after your capture step succeeds.
v2 changes nothing in your workflow inputs. Bump @v1 to @v2, and check one thing:
- The report artifact is now
vrt-report[-<key>].html(it wasvrt-report[-<key>]), uploaded un-zipped so it opens in the browser. Thereport-pathoutput's file name changes to match. Update any step that downloads the report artifact by name or readsreport-pathexpectingindex.html.
Approval workflows keep working as they are: the existing if: condition already matches the new per-screenshot checkboxes.
npm ci
npm run dev:ui # report UI (report-ui/, React + FiestaUI) with sample data
npm run build:ui # bundle the UI into src/generated/report-template.ts
npm test
npm run build # bundle the action into dist/The report UI is a React app built on @fiestaboard/ui. build:ui bundles it into a single HTML template. At runtime the action injects that run's screenshots as JSON (src/report-data.ts defines the shape). Commit src/generated/ and dist/ after rebuilding; CI fails if either is stale.