Skip to content

upstream: playwright-cli c2d8d19 conflicts with our test port #18

Closed

Description

@github-actions

microsoft/playwright-cli b85c7a7 -> c2d8d19

Verdict: blocked

  • tests/integration/core.spec.ts: 3 conflict(s) between our changes and upstream changes

Commits

Mapped files (three-way merge)

file result
tests/integration/core.spec.ts conflict (3)

Files we rewrote (upstream patch, for reading)

.github/workflows/ci.yml (modified)

@@ -16,7 +16,7 @@ jobs:
     steps:
     - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
     - name: Use Node.js 20
-      uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+      uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
       with:
         node-version: '20'
         cache: 'npm'

CLAUDE.md (modified)

@@ -1,3 +1,22 @@
+## Build and test
+
+The CLI source lives in `microsoft/playwright`, in `packages/playwright-core/src/tools/`. This repo
+ships `@playwright/cli`: `playwright-cli.js` runs the CLI of the `playwright-core` alpha that
+`package.json` pins, and adds the update check, with `skillCheck.js`. `skills/playwright-cli/` is a
+copy of the skill that `playwright-cli install --skills` writes, and `node scripts/update.js`
+refreshes it.
+
+```bash
+npm ci
+npm test                                 # tests/integration.spec.ts
+npx playwright test -g "open data URL"
+```
+
+There is no build step and no lint. Tests are in `tests/integration.spec.ts`. They run
+`playwright-cli.js` with `runCli()` in the output directory of the test, and the `npmRegistry`
+fixture serves a fake registry for the update check. Add a case to the test whose setup is closest
+instead of a new test.
+
 ## Commit Convention
 
 Semantic commit messages: `label(scope): description`
@@ -28,3 +47,19 @@ EOF
 
 Never add Co-Authored-By agents in commit message.
 Branch naming for issue fixes: `fix-<issue-number>`
+
+## Rolling Playwright and preparing a release
+
+See [.claude/skills/dev/roll.md](.claude/skills/dev/roll.md) and [.claude/skills/dev/release.md](.claude/skills/dev/release.md).
+
+## Bots
+
+Labels start GitHub Actions workflows that run the Copilot CLI and open pull requests as the
+`microsoft-playwright-automation` app:
+
+- `needs-triage`, `needs-triage-mac`, `needs-triage-win` on an issue: [triage.yml](.github/workflows/triage.yml) reproduces and fixes it, with the skill in [.claude/skills/playwright-triage/](.claude/skills/playwright-triage/SKILL.md). A fix in `microsoft/playwright` opens as a draft PR there.
+- `needs-revision`, `needs-revision-mac`, `needs-revision-win` on a PR that triage opened here: [revision.yml](.github/workflows/revision.yml) addresses the maintainers' review comments.
+- `needs-roll` on an issue: [roll.yml](.github/workflows/roll.yml) rolls Playwright to `playwright@next`, as the roll skill says.
+- `needs-release` on an issue: [release.yml](.github/workflows/release.yml) opens the `chore: mark v<version>` PR with the release notes as its description, as the release skill says. Copy the notes into the release draft.
+
+Apply a label again to start another run. [bots.md](https://github.com/microsoft/playwright/blob/main/.github/workflows/bots.md) in `microsoft/playwright` describes triage and revision.

README.md (modified)

@@ -185,9 +185,11 @@ playwright-cli keyup <key>              # press a key up on the keyboard
 playwright-cli mousemove <x> <y>        # move mouse to a given position
 playwright-cli mousedown [button]       # press mouse down
 playwright-cli mouseup [button]         # press mouse up
-playwright-cli mousewheel <dx> <dy>     # scroll mouse wheel
+playwright-cli mousewheel <dx> <dy>     # scroll by pixels; negative dy scrolls up

+The element under the mouse pointer scrolls, so use mousemove to move over it first.
+

Save as

@@ -203,6 +205,7 @@ playwright-cli pdf --filename=page.pdf  # save pdf with specific filename
```bash
playwright-cli tab-list                 # list all tabs
playwright-cli tab-new [url]            # create a new tab
+playwright-cli tab-new https://example.com --isolated-context=user-b # use an isolated context
playwright-cli tab-close [index]        # close a browser tab
playwright-cli tab-select <index>       # select a browser tab

@@ -218,6 +221,7 @@ playwright-cli cookie-list [--domain] # list cookies
playwright-cli cookie-get # get a cookie
playwright-cli cookie-set # set a cookie
playwright-cli cookie-delete # delete a cookie
+playwright-cli cookie-delete --domain=example.com # delete only on this domain
playwright-cli cookie-clear # clear all cookies

LocalStorage

@@ -286,6 +290,16 @@ playwright-cli highlight --hide # hide highlight on a specific element
playwright-cli highlight --hide # hide all page highlights


+Tracing requires an open browser. Open it before starting the trace, then use `goto` to
+record navigation:
+
+```bash
+playwright-cli open
+playwright-cli tracing-start
+playwright-cli goto https://example.com
+playwright-cli tracing-stop
+```
+
### WebMCP

Some pages register their own tools for agents through the experimental WebMCP API. When available,
@@ -357,6 +371,16 @@ playwright-cli find --regex "\\$[0-9]+\\.[0-9]{2}"
playwright-cli find --regex "/sign (in|up)/i"

+### Raw output
+
+The global --raw option strips page status and generated code from the output. It keeps
+the result value, errors, and the page snapshot (or the link to the snapshot file).
+Commands that don't produce output return nothing.
+
+bash +playwright-cli --raw eval "JSON.stringify(performance.timing)" | jq '.loadEventEnd - .navigationStart' +
+

Targeting elements

By default, use refs from the snapshot to interact with page elements.


### package.json (modified)

```diff
@@ -1,6 +1,6 @@
 {
   "name": "@playwright/cli",
-  "version": "0.1.22",
+  "version": "0.1.23",
   "description": "Playwright CLI",
   "repository": {
     "type": "git",
@@ -18,12 +18,12 @@
     "test": "playwright test"
   },
   "devDependencies": {
-    "@playwright/test": "1.64.0-alpha-1790635538000",
+    "@playwright/test": "1.65.0-alpha-2026-10-30",
     "@types/node": "^25.2.1"
   },
   "dependencies": {
-    "playwright": "1.64.0-alpha-1790635538000",
-    "playwright-core": "1.64.0-alpha-1790635538000"
+    "playwright": "1.65.0-alpha-2026-10-30",
+    "playwright-core": "1.65.0-alpha-2026-10-30"
   },
   "bin": {
     "playwright-cli": "playwright-cli.js"

playwright-cli.js (modified)

@@ -19,6 +19,7 @@
 
 const fs = require('fs');
 const path = require('path');
+const { spawn } = require('child_process');
 
 const { program } = require('playwright-core/lib/tools/cli-client/program');
 const coreBundle = require('playwright-core/lib/coreBundle');
@@ -28,6 +29,7 @@ const { checkInstalledSkills, frame } = require('./skillCheck');
 const packageJson = require('./package.json');
 
 const ONE_DAY_MS = 24 * 60 * 60 * 1000;
+const UPDATE_CHECK_COMMANDS = new Set(['open', 'attach', 'list', 'install', 'install-browser']);
 
 main();
 
@@ -40,13 +42,17 @@ async function checkForUpdates() {
   if (process.env.NO_UPDATE_NOTIFIER || process.env.CI)
     return;
 
+  // The check spawns npm, so keep its cost to the commands that start a workflow.
+  const command = process.argv.slice(2).find(arg => !arg.startsWith('-'));
+  if (!command || !UPDATE_CHECK_COMMANDS.has(command))
+    return;
+
   const cache = readCache();
   const stale = !cache || (Date.now() - cache.lastCheck) > ONE_DAY_MS;
   if (!stale)
     return;
   writeCache({ lastCheck: Date.now() });
 
-  const command = process.argv.slice(2).find(arg => !arg.startsWith('-'));
   if (command !== 'install')
     checkInstalledSkills();
 
@@ -55,22 +61,38 @@ async function checkForUpdates() {
     printNotice(packageJson.version, latest);
 }
 
-async function fetchLatestVersion() {
-  try {
-    const controller = new AbortController();
-    const timeout = setTimeout(() => controller.abort(), 1500);
-    try {
-      const res = await fetch(`https://registry.npmjs.org/${packageJson.name}/latest`, { signal: controller.signal });
-      if (!res.ok)
-        return undefined;
-      const json = await res.json();
-      return typeof json.version === 'string' ? json.version : undefined;
-    } finally {
+// Asks npm, so that the user's registry, proxy and auth configuration is respected.
+/** @returns {Promise<string | undefined>} */
+function fetchLatestVersion() {
+  return new Promise(resolve => {
+    const child = spawn(`npm view ${packageJson.name} version --json --no-update-notifier`, {
+      shell: true,
+      stdio: ['ignore', 'pipe', 'ignore'],
+    });
+    let stdout = '';
+    child.stdout.on('data', data => stdout += data);
+    // On Windows, killing the shell leaves npm running: stop listening to it
+    // so that the CLI does not wait for npm before exiting.
+    const timeout = setTimeout(() => {
+      child.kill();
+      child.unref();
+      child.stdout.destroy();
+      resolve(undefined);
+    }, 1500);
+    child.on('error', () => {
       clearTimeout(timeout);
-    }
-  } catch {
-    return undefined;
-  }
+      resolve(undefined);
+    });
+    child.on('close', code => {
+      clearTimeout(timeout);
+      try {
+        const version = JSON.parse(stdout);

Other upstream files

  • .claude/skills/playwright-triage/SKILL.md (added)
  • .github/actions/setup-triage-runner/action.yml (added)
  • .github/workflows/publish.yml (modified)
  • .github/workflows/release.yml (added)
  • .github/workflows/revision.yml (added)
  • .github/workflows/roll.yml (added)
  • .github/workflows/triage.yml (added)
  • package-lock.json (modified)
  • skills/playwright-cli/SKILL.md (modified)
  • skills/playwright-cli/references/storage-state.md (modified)
  • skills/playwright-cli/references/test-generation.md (modified)
  • skills/playwright-cli/references/tracing.md (modified)

core.spec.ts.merged

/**
 * Copyright (c) Microsoft Corporation.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

import fs from 'fs';
import http from 'http';
import path from 'path';
<<<<<<< tests/integration/core.spec.ts
import { test, expect } from 'patchright/test';

import { runCli } from '../fixtures';
=======
import { spawn } from 'child_process';
import { test as baseTest, expect } from '@playwright/test';
import type { AddressInfo } from 'net';

type CliResult = {
  output: string;
  error: string;
  exitCode: number | null;
};

type NpmRegistry = {
  url: string;
  status: number;
  hang: boolean;
  requests: string[];
};

const test = baseTest.extend<{ npmRegistry: NpmRegistry }>({
  npmRegistry: async ({}, use) => {
    const registry: NpmRegistry = { url: '', status: 200, hang: false, requests: [] };
    const server = http.createServer((req, res) => {
      registry.requests.push(req.url!);
      if (registry.hang)
        return;
      res.statusCode = registry.status;
      res.setHeader('content-type', 'application/json');
      res.end(JSON.stringify({
        'name': '@playwright/cli',
        'dist-tags': { latest: '99.0.0' },
        'versions': { '99.0.0': { name: '@playwright/cli', version: '99.0.0' } },
      }));
    });
    await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
    registry.url = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
    await use(registry);
    server.closeAllConnections();
    await new Promise(resolve => server.close(resolve));
  },
});

// Enables the update check and points npm at the test registry, isolated from
// the npm config of the machine running the tests.
function updateCheckEnv(npmRegistry: NpmRegistry): Record<string, string> {
  return {
    'CI': '',
    'NO_UPDATE_NOTIFIER': '',
    'npm_config_registry': npmRegistry.url,
    'npm_config_@playwright:registry': '',
    'npm_config_userconfig': test.info().outputPath('user-npmrc'),
    'npm_config_globalconfig': test.info().outputPath('global-npmrc'),
    'npm_config_cache': test.info().outputPath('npm-cache'),
  };
}

async function runCli(args: string[], env: Record<string, string> = {}): Promise<CliResult> {
  const cliPath = path.join(__dirname, '../playwright-cli.js');

  return new Promise<CliResult>((resolve, reject) => {
    let stdout = '';
    let stderr = '';

    const childProcess = spawn(process.execPath, [cliPath, ...args], {
      env: {
        ...process.env,
        PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST: test.info().outputPath(),
        ...env,
      },
      cwd: test.info().outputPath(),
    });

    childProcess.stdout?.on('data', (data) => {
      stdout += data.toString();
    });

    childProcess.stderr?.on('data', (data) => {
      stderr += data.toString();
    });

    childProcess.on('close', (code) => {
      resolve({
        output: stdout.trim(),
        error: stderr.trim(),
        exitCode: code,
      });
    });

    childProcess.on('error', reject);
  });
}
>>>>>>> upstream tests/integration.spec.ts

test('open data URL', async ({}) => {
  expect(await runCli(['open', 'data:text/html,hello', '--persistent'])).toEqual(expect.objectContaining({
    output: expect.stringContaining('hello'),
    exitCode: 0,
  }));

  expect(await runCli(['delete-data'])).toEqual(expect.objectContaining({
    output: expect.stringContaining('Deleted user data for'),
    exitCode: 0,
  }));
});

test('warns when installed skill is out of date', async ({ npmRegistry }) => {
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'patchright-cli', 'SKILL.md');
  fs.appendFileSync(skillFile, 'x');

  const env = updateCheckEnv(npmRegistry);
  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({
    error: expect.stringContaining('does not match the tool version'),
  }));

  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({
    error: expect.not.stringContaining('does not match the tool version'),
  }));
});

<<<<<<< tests/integration/core.spec.ts
test('warns when an installed reference file drifts', async ({}) => {
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const reference = path.join(test.info().outputPath(), '.claude', 'skills', 'patchright-cli', 'references', 'stealth.md');
  fs.appendFileSync(reference, 'x');

  expect(await runCli(['--help'], { CI: '', NO_UPDATE_NOTIFIER: '' })).toEqual(expect.objectContaining({
    error: expect.stringContaining('does not match the tool version'),
  }));
});

test('does not warn when installed skill only differs in line endings', async ({}) => {
=======
test('does not warn when installed skill only differs in line endings', async ({ npmRegistry }) => {
>>>>>>> upstream tests/integration.spec.ts
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'patchright-cli', 'SKILL.md');
  fs.writeFileSync(skillFile, fs.readFileSync(skillFile, 'utf8').replace(/\n/g, '\r\n'));

  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: expect.not.stringContaining('does not match the tool version'),
  }));
});

test('caches the update check in the default registry directory', async ({ npmRegistry }) => {
  // Redirect the home/cache directories so the real user cache is untouched, and
  // leave PATCHRIGHT_CLI_HOME empty so the default path is used.
  const home = test.info().outputPath('home');
  fs.mkdirSync(home, { recursive: true });
  const env = {
<<<<<<< tests/integration/core.spec.ts
    CI: '',
    NO_UPDATE_NOTIFIER: '',
    PATCHRIGHT_CLI_HOME: '',
=======
    ...updateCheckEnv(npmRegistry),
    PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST: '',
>>>>>>> upstream tests/integration.spec.ts
    HOME: home,
    USERPROFILE: home,
    XDG_CACHE_HOME: path.join(home, '.cache'),
    LOCALAPPDATA: path.join(home, 'AppData', 'Local'),
  };

  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({ exitCode: 0 }));

  const found: string[] = [];
  const walk = (dir: string) => {
    for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
      const full = path.join(dir, entry.name);
      if (entry.isDirectory())
        walk(full);
      else if (entry.name === 'cli-update-check.json')
        found.push(full);
    }
  };
  walk(home);

  expect(found).toHaveLength(1);
  expect(JSON.parse(fs.readFileSync(found[0], 'utf8')).lastCheck).toEqual(expect.any(Number));
});

test('update check asks npm for the latest version on setup commands only', async ({ npmRegistry }) => {
  const env = updateCheckEnv(npmRegistry);
  expect(await runCli(['--version'], env)).toEqual(expect.objectContaining({
    error: '',
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual([]);

  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({
    error: expect.stringContaining('Update available for @playwright/cli'),
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual(['/@playwright%2fcli']);
});

test('update check on install does not warn about the skills being installed', async ({ npmRegistry }) => {
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'playwright-cli', 'SKILL.md');
  fs.appendFileSync(skillFile, 'x');

  const result = await runCli(['install', '--skills'], updateCheckEnv(npmRegistry));
  expect(result.exitCode).toBe(0);
  expect(result.error).toContain('Update available for @playwright/cli');
  expect(result.error).not.toContain('does not match the tool version');
  expect(npmRegistry.requests).toEqual(['/@playwright%2fcli']);
});

test('update check uses the scoped registry from project .npmrc', async ({ npmRegistry }) => {
  fs.writeFileSync(test.info().outputPath('package.json'), '{}');
  fs.writeFileSync(test.info().outputPath('.npmrc'), `@playwright:registry=${npmRegistry.url}/scoped/`);

  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: expect.stringContaining('Update available for @playwright/cli'),
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual(['/scoped/@playwright%2fcli']);
});

test('update check failure is silent', async ({ npmRegistry }) => {
  npmRegistry.status = 401;
  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: '',
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual(['/@playwright%2fcli']);
});

test('update check does not wait for a hanging registry', async ({ npmRegistry }) => {
  npmRegistry.hang = true;
  const start = Date.now();
  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: '',
    exitCode: 0,
  }));
  expect(Date.now() - start).toBeLessThan(10000);
});

Activity

  1. added a commit that references this issue on Nov 8, 2026
    2221361
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    upstream-blockedupstream roll needs a manual port

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions