PER-8985 feat: Playwright drop-in baseline seeding — build attributes + exec orchestration#2335
Open
Shivanshu-07 wants to merge 13 commits into
Open
PER-8985 feat: Playwright drop-in baseline seeding — build attributes + exec orchestration#2335Shivanshu-07 wants to merge 13 commits into
Shivanshu-07 wants to merge 13 commits into
Conversation
…orchestration - @percy/client createBuild: allow-listed drop-in build sources via PERCY_BUILD_SOURCE, plus dropin-baseline-candidate / dropin-baseline-setup attributes (env or explicit options) the API keys baseline auto-approval on. - @percy/cli-exec: generic baseline-provider discovery (any installed SDK package can declare "@percy/cli".baselineProvider). On an empty project, percy exec uploads the committed baseline screenshots as an auto-approved build #1 before the user's command runs; established projects get a notice pointing at the SDK's explicit setup command. Seeding helpers exported for SDK-contributed commands. - @percy/core: expose the server-decided build source on percy.build so SDKs can key on it through the healthcheck. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verified E2E against a local percy-api: a web project's snapshot create requires a root resource, so raw-PNG comparison tiles are rejected. Seed uploads now use client.sendSnapshot with the same generated root-DOM + image-resource shape 'percy upload' uses for web projects — Percy renders the baseline in the project's own browsers, so it pairs with the head run's DOM snapshots by (name, browser, width). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Seeding dispatches by project type: web projects seed rendered snapshots (root DOM + image resource); app projects seed raw comparison uploads (tag + tile), exactly how App Percy ingests screenshots. Exec seeding now engages for app tokens too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
The same env var the SDK override honors — one switch turns off both the toHaveScreenshot override and exec's baseline provider discovery. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- core: only set percy.build.source when the API returned one — an own 'source: undefined' key survives in-process but drops out of JSON responses, breaking the API-server shape specs. - cli-exec: cover the exec seeding block (real provider-package fixture) and the remaining baseline.js branches for the 100% coverage gate. - cli-exec: sanitize dirent names / paths at the fs boundary in the provider walk (also clears semgrep's path-join-resolve-traversal findings on the diff). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Direct sanitizer unit tests, walk fixtures for every discovery skip branch (empty dir, ESM named-export provider without discoverBaselines), and the non-web/app token exec branch; istanbul-ignore the two defense-in-depth dirent guards unreachable via readdir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The seed build keeps rendering and only auto-approves once it reaches finished; head snapshots select their baseline as they process, so starting the head immediately raced the approval and a lost race made the whole first run report as new instead of diffing. Poll the seed build up to the pipeline latency budget (10m) with a 30s heartbeat, degrade to a warning instead of blocking on timeout or status errors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pin-baseline # Conflicts: # packages/client/src/client.js # packages/client/test/client.test.js
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Master-inherited coverage gap from the metadata-SSRF change — the catch path for a colon-containing host that fails URL parsing was never exercised, holding @percy/core under the 100% gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every caller (matchMetadataHost) null-guards before calling, so the branch cannot be exercised by tests; it held @percy/core under the 100% gate on master too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
CLI-side support for the
@percy/playwrighttoHaveScreenshot drop-in's baseline flow:createBuildaccepts allow-listed drop-in build sources viaPERCY_BUILD_SOURCE(playwright-dropin/playwright-dropin-baseline) plusdropin-baseline-candidate/dropin-baseline-setupattributes (env or explicit options). The API keys baseline auto-approval on these."@percy/cli": { "baselineProvider": "..." }(no framework code enters the CLI, same scaling model as command discovery). On an EMPTY project,percy execuploads the committed baseline screenshots as an auto-approved build ⬆️ Bump @babel/register from 7.10.4 to 7.10.5 #1 before the user's command runs (the suite never executes for the baseline), then starts the head build. Established projects get a notice pointing at the SDK's explicitplaywright:setup-baselinecommand — the API's no-build-persisted sentinel guarantees no stray builds. Seeding dispatches by project type: web → rendered web snapshots (root DOM + image resource, thepercy uploadshape); app → comparison ingest (tag + tile, no render).sourceis kept onpercy.buildso SDKs can key on it through the healthcheck.Depends on
Testing
Client + exec suites green incl. 11 new seeding specs; E2E verified against a local percy-api and on staging (empty web/app seed → head diff flow, established-project sentinel, explicit setup command).
🤖 Generated with Claude Code