diff --git a/.claude/skills/playwright-triage/SKILL.md b/.claude/skills/playwright-triage/SKILL.md new file mode 100644 index 000000000..a4fe33cab --- /dev/null +++ b/.claude/skills/playwright-triage/SKILL.md @@ -0,0 +1,85 @@ +--- +name: playwright-triage +description: Triage and fix one microsoft/playwright-dotnet issue, or do a maintainer's task, unattended inside the Issue triage GitHub Actions workflow (.github/workflows/triage.yml), which a needs-triage, needs-triage-mac or needs-triage-win label starts, or a maintainer starts by hand with an issue, a task or both. Reproduces the issue on the runner, then leaves a verified fix proposal for the workflow to open as a draft PR, an investigation note for it to post on the issue, or an upstream bug report for a maintainer to file in microsoft/playwright. Only for that workflow. +--- + +# Playwright .NET Issue Triage + +You run unattended in the [Issue triage](../../../.github/workflows/triage.yml) workflow of +`microsoft/playwright-dotnet`. First read the shared guide, `$AGENT_GUIDES_DIR/triage.md`, a copy of +[microsoft/playwright/.github/actions/run-agent/guides/triage.md](https://github.com/microsoft/playwright/blob/main/.github/actions/run-agent/guides/triage.md), +and then [CLAUDE.md](../../../CLAUDE.md), both in full. The guide says how to triage an issue in any +Playwright repo. CLAUDE.md says how to build, test and format this one, and where a regression test +goes. This file covers what neither has: the checkouts, how to run a published version, and the +outcome that this repo adds, an upstream bug report. + +## The checkouts + +`$REPO_DIR` is the checkout of `microsoft/playwright-dotnet` that the guide describes. The driver is +downloaded (`./build.sh --download-driver`), the solution is built (`dotnet build ./src`), and +Chromium, Firefox and WebKit are installed +(`pwsh src/Playwright/bin/Debug/netstandard2.0/playwright.ps1 install --with-deps`). It is the +working directory, and the checkout that the guides refer to. The clone is shallow: run +`git fetch --unshallow --tags origin` in it before you bisect or use `git tag --contains`. .NET SDK +8 and 10 are installed, as in CI. Tests run with `-f net8.0`. + +`$PLAYWRIGHT_DIR` is a shallow clone of `microsoft/playwright` at `main`. The .NET client is a port +of the JavaScript client in `packages/playwright-core/src/client/`. The API docs are in +`docs/src/api/`, the API generator is `utils/doclint/generateDotnetApi.js`, and the upstream tests +are in `tests/page/` and `tests/library/`. The clone is not at the commit of the driver: +`` in `src/Common/Version.props` names that version, `main` is close to it, and a +release has the tag `v`. The clone is not built. Before you run an upstream test in +it, run `npm ci`, `npm run build` and `npx playwright install` there. Then `npm run ctest `, +`npm run ftest ` and `npm run wtest ` run one spec file on Chromium, Firefox and WebKit. + +## Published versions + +To run a published version, for example to bisect, create +`$SCRATCH_DIR/playwright-dotnet-//` and make a console project there: + +```bash +dotnet new console -n repro -f net8.0 -o . && dotnet add package Microsoft.Playwright --version +dotnet build && pwsh bin/Debug/net8.0/playwright.ps1 install chromium +dotnet run +``` + +Put the repro in `Program.cs` as top-level statements, as `src/Playwright.Examples/Program.cs` +does. `playwright.ps1 install` fetches the browsers of that version. Match the change to commits in +`$REPO_DIR` with `git log v..v -- src/Playwright/`, and to the JavaScript client in +`$PLAYWRIGHT_DIR` with the same tags. + +## Upstream bugs + +The .NET client talks to the Node driver that `microsoft/playwright` publishes, and many bugs that +are reported here are in the driver. That fix lands in `microsoft/playwright`. The family is one +team, so this is routing, not a request to the reporter. Find the owner with the same calls in both +clients: write the repro as a .NET test in `$REPO_DIR` first, then write the same calls as an +upstream test in `$PLAYWRIGHT_DIR` and run it there. When the Node test fails the same way, the root +cause is upstream. When only the .NET test fails, the bug is in the port: compare the method in +`src/Playwright/Core/` with the same method in the JavaScript client, and fix it here. + +The interfaces in `src/Playwright/API/Generated/` and the types in +`src/Playwright/Transport/Protocol/Generated/` are generated by `utils/doclint/generateDotnetApi.js` +and `utils/generate_dotnet_channels.js` from the upstream docs and protocol. Never edit them by +hand. A missing method, option or type is an upstream change, in `docs/src/api/` (the `langs` +annotations decide what .NET gets) or in the generator. .NET-only overloads are hand-written in +`src/Playwright/API/Supplements/`, and the implementations are in `src/Playwright/Core/`. + +For an upstream root cause, create `$OUTPUT_DIR/upstream/` with: + +- `report.md`: the report for a maintainer to file in `microsoft/playwright`, for an engineer who + did not read this issue: what fails, expected versus actual, the driver version and the browsers, + and the condensed Node test from `$PLAYWRIGHT_DIR` as the repro. Say that .NET shows the same. +- `report.json`: `{"title": "..."}`, at most 256 characters. + +The comment job appends the report to your comment, for a maintainer to file. The bot never files +it. Still write the comment as the note, without the report in it. Write `report.json` last, when +`report.md` and the comment are done. Like `proposal.json`, it marks the report as ready. + +## PRs from third parties + +Do not run a PR that changes a `.csproj`, `.props` or `.sln` file, `nuget.config`, `build.sh`, +`src/tools/Playwright.Tooling/` (the driver download), `src/Playwright/build/` (MSBuild targets that +run on every build), `.github/`, `.azure-pipelines/`, `utils/`, `package.json`, `package-lock.json`, +or the generated files under `src/Playwright/API/Generated/` and +`src/Playwright/Transport/Protocol/Generated/`. diff --git a/.github/actions/setup-triage-runner/action.yml b/.github/actions/setup-triage-runner/action.yml new file mode 100644 index 000000000..e508edc68 --- /dev/null +++ b/.github/actions/setup-triage-runner/action.yml @@ -0,0 +1,36 @@ +name: 'Set up the triage runner' +description: > + Prepares the environment that the triage and revision agents work in: the checkout built as CI + builds it, with the browsers and the Copilot CLI installed, and a checkout of microsoft/playwright + to read the client that this repo ports. Exports PLAYWRIGHT_DIR, the path of that checkout, and + CI_ENV_EXEC, the wrapper from microsoft/playwright that runs a command under xvfb on Linux and at + normal I/O priority on Windows. +runs: + using: composite + steps: + - uses: microsoft/playwright/.github/actions/ci-env@main + - uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + dotnet-version: | + 8.0.x + 10.0.x + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - name: Download the driver and build + shell: bash + run: | + ./build.sh --download-driver + dotnet build ./src + - name: Install the browsers + shell: bash + run: pwsh src/Playwright/bin/Debug/netstandard2.0/playwright.ps1 install --with-deps + - name: Check out microsoft/playwright + shell: bash + run: | + dir="${RUNNER_TEMP//\\//}/playwright" + git clone --quiet --depth 1 https://github.com/microsoft/playwright "$dir" + echo "PLAYWRIGHT_DIR=$dir" >> "$GITHUB_ENV" + - name: Install the Copilot CLI + shell: bash + run: npm install -g @github/copilot diff --git a/.github/workflows/revision.yml b/.github/workflows/revision.yml new file mode 100644 index 000000000..fa3af88c1 --- /dev/null +++ b/.github/workflows/revision.yml @@ -0,0 +1,132 @@ +# Copilot addresses the maintainers' review comments on a draft PR that the triage workflow opened, +# without write credentials. A fresh job pushes the new commit to the PR's branch using the app, so +# CI runs on it. The steps that are the same for every Playwright repo are actions in +# microsoft/playwright, and the shared guide they carry says how to revise; the prompt below has +# what is specific to this repo. +name: PR revision + +on: + # Apply needs-revision, needs-revision-mac or needs-revision-win; re-apply it to trigger another pass. + # pull_request_target always runs this file from main; the jobs only accept the triage app's own PRs. + pull_request_target: + types: [labeled] + +permissions: {} + +jobs: + prepare: + name: Collect the maintainers' comments + if: >- + github.repository == 'microsoft/playwright-dotnet' && + contains(fromJSON('["needs-revision","needs-revision-mac","needs-revision-win"]'), github.event.label.name) && + github.event.pull_request.state == 'open' && + github.event.pull_request.user.login == 'microsoft-playwright-automation[bot]' && + github.event.pull_request.head.repo.full_name == github.repository + runs-on: ubuntu-latest + permissions: + pull-requests: write + env: + PLATFORM: ${{ fromJSON('{"needs-revision":"linux","needs-revision-mac":"mac","needs-revision-win":"windows"}')[github.event.label.name] }} + outputs: + head: ${{ steps.collect.outputs.head }} + base: ${{ steps.collect.outputs.base }} + branch: ${{ steps.collect.outputs.branch }} + platform: ${{ env.PLATFORM }} + steps: + - name: Collect the PR and the maintainers' comments + id: collect + uses: microsoft/playwright/.github/actions/collect-review@main + with: + pull-request: ${{ github.event.pull_request.number }} + - name: Post the run link + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows"}')[env.PLATFORM] }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + body=$(printf "Hi, I'm the Playwright bot. I started to address the review comments on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$RUN_URL") + gh pr comment "$PR" --body "$body" || echo "::warning::Could not post the run link" + + revise: + name: Revise PR ${{ github.event.pull_request.number }} (${{ needs.prepare.outputs.platform }}) + needs: prepare + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[needs.prepare.outputs.platform] }} + timeout-minutes: 240 + concurrency: + group: revision-${{ github.event.pull_request.number }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ needs.prepare.outputs.head }} + BASE_SHA: ${{ needs.prepare.outputs.base }} + TRIAGE_PLATFORM: ${{ needs.prepare.outputs.platform }} + permissions: + copilot-requests: write + outputs: + has_revision: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + persist-credentials: false + - name: Fetch the base and record the directories + run: | + git fetch --no-tags --depth=1 origin "$BASE_SHA" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/revision" >> "$GITHUB_ENV" + echo "REVIEW_DIR=${RUNNER_TEMP//\\//}/review" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/revision" "${RUNNER_TEMP//\\//}/scratch" + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: review + path: ${{ runner.temp }}/review + - name: Set up the runner + uses: ./.github/actions/setup-triage-runner + - name: Revise with Copilot CLI + id: agent + # No GitHub tools and no URL access: the agent reads the PR only from the review the prepare job filtered. + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: | + Follow the shared guide revision.md and this repo's CLAUDE.md. Read both in full before starting. $PLAYWRIGHT_DIR is a shallow clone of microsoft/playwright at main, for reading the JavaScript client that this repo ports. + + Pull request: https://github.com/microsoft/playwright-dotnet/pull/${{ github.event.pull_request.number }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: commit/commit.json + artifact-name: revision + exec: ${{ env.CI_ENV_EXEC }} + + push: + name: Push the revision + needs: [prepare, revise] + # commit.json is written last, so a finished revision is pushed even when the job failed later. + if: ${{ !cancelled() && needs.revise.outputs.has_revision == 'true' }} + runs-on: ubuntu-latest + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: revision + path: ${{ runner.temp }}/revision + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate and push + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/revision/commit + patch: revision.patch + metadata: commit.json + branch: ${{ needs.prepare.outputs.branch }} diff --git a/.github/workflows/triage.yml b/.github/workflows/triage.yml new file mode 100644 index 000000000..daafbdcd5 --- /dev/null +++ b/.github/workflows/triage.yml @@ -0,0 +1,227 @@ +# Copilot triages and fixes an issue, or works on a task given by hand, without write credentials. +# Fresh jobs post the run link on the issue, open the completed fix as a draft PR using the app, so +# CI runs on it, and comment on the issue. The steps that are the same for every Playwright repo are +# actions in microsoft/playwright. The skill in .claude/skills/playwright-triage/ has what is +# specific to this repo. +name: Issue triage + +on: + # Apply needs-triage, needs-triage-mac or needs-triage-win; re-apply it to trigger another pass. + issues: + types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue number to triage; leave empty to work on the task alone" + type: string + required: false + default: "" + platform: + description: "Platform to triage on" + type: choice + required: true + default: linux + options: + - linux + - mac + - windows + task: + description: "Task to direct the triage, such as what to focus on; the whole job when there is no issue" + type: string + required: false + default: "" + +permissions: {} + +jobs: + start: + name: Post the run link on the issue + # The triage job's condition, for runs that have an issue. + if: >- + github.repository == 'microsoft/playwright-dotnet' && + ((github.event_name == 'workflow_dispatch' && inputs.issue != '') || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + runs-on: ubuntu-latest + permissions: + issues: write + outputs: + comment: ${{ steps.post.outputs.id }} + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + steps: + - name: Post the run link + id: post + run: | + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + body=$(printf "Hi, I'm the Playwright bot. I started to triage this issue on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$WORKFLOW_URL") + id=$(gh api "repos/$GH_REPO/issues/$ISSUE/comments" -f body="$body" --jq .node_id) + echo "id=$id" >> "$GITHUB_OUTPUT" + + triage: + name: Triage ${{ (github.event.issue.number || inputs.issue) && format('issue {0}', github.event.issue.number || inputs.issue) || 'task' }} (${{ inputs.platform || github.event.label.name }}) + if: >- + github.repository == 'microsoft/playwright-dotnet' && + (github.event_name == 'workflow_dispatch' || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + # Falls back to a runner name for unrelated labels, where the job is skipped anyway. + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name]] || 'ubuntu-latest' }} + timeout-minutes: 240 + concurrency: + group: triage-${{ github.event.issue.number || inputs.issue || github.run_id }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + TRIAGE_PLATFORM: ${{ inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name] }} + TASK: ${{ inputs.task }} + permissions: + copilot-requests: write + outputs: + base: ${{ steps.base.outputs.sha }} + setup: ${{ steps.setup.outcome }} + agent: ${{ steps.agent.outputs.outcome }} + has_fix: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + persist-credentials: false + - name: Record the base and the directories + id: base + run: | + if [[ -n "$ISSUE" ]]; then + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + elif [[ -z "${TASK//[[:space:]]/}" ]]; then + echo "give an issue, a task or both" >&2; exit 1 + fi + echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/triage" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/triage" "${RUNNER_TEMP//\\//}/scratch" + - name: Set up the runner + id: setup + uses: ./.github/actions/setup-triage-runner + - name: Write the prompt + id: prompt + run: | + delimiter="PROMPT_$RANDOM$RANDOM" + { + echo "prompt<<$delimiter" + echo "Follow .claude/skills/playwright-triage/SKILL.md. Read it, the guide and the CLAUDE.md" + echo "it points to in full before starting." + if [[ -n "$ISSUE" ]]; then + printf '\nIssue: https://github.com/%s/issues/%s\n' "$GITHUB_REPOSITORY" "$ISSUE" + fi + if [[ -n "$TASK" ]]; then + printf '\nTask: %s\n' "$TASK" + fi + echo "$delimiter" + } >> "$GITHUB_OUTPUT" + - name: Triage with Copilot CLI + id: agent + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: ${{ steps.prompt.outputs.prompt }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: fix/proposal.json + artifact-name: triage + github-tools: true + exec: ${{ env.CI_ENV_EXEC }} + - name: Check the upstream bug report + run: | + if [[ -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(node -p 'String(JSON.parse(require("fs").readFileSync(process.argv[1], "utf8")).title ?? "")' "$OUTPUT_DIR/upstream/report.json" || true) + if [[ -z "$title" ]]; then + echo "::warning::upstream/report.json has no title, so the report is not appended to the comment" + else + echo "Upstream bug report: $title" >> "$GITHUB_STEP_SUMMARY" + fi + fi + + publish: + name: Open draft pull request + needs: triage + # proposal.json is written last, so a finished proposal is published even when the job failed later. + if: ${{ !cancelled() && needs.triage.outputs.has_fix == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.publish.outputs.pull-request }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.triage.outputs.base }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: triage + path: ${{ runner.temp }}/triage + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate, push and open draft + id: publish + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/triage/fix + patch: fix.patch + metadata: proposal.json + body-file: pr-body.md + branch: triage/${{ github.event.issue.number || inputs.issue || 'task' }}/${{ github.run_id }}-${{ github.run_attempt }} + commit-body: "${{ (github.event.issue.number || inputs.issue) && format('Fixes: https://github.com/{0}/issues/{1}', github.repository, github.event.issue.number || inputs.issue) || '' }}" + pull-request: true + + comment: + name: Comment on the issue + needs: [start, triage, publish] + # Runs after every triage run that has an issue, including failed ones, and whatever publish did. + if: ${{ !cancelled() && needs.triage.result != 'skipped' && (github.event_name == 'issues' || inputs.issue != '') }} + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + # A failed triage run may not have uploaded anything. + continue-on-error: true + with: + name: triage + path: ${{ runner.temp }}/triage + # The bot never files upstream; the comment hands the report to a maintainer. + - name: Describe the upstream bug report + id: upstream + env: + OUTPUT_DIR: ${{ runner.temp }}/triage + run: | + extra="" + if [[ -s "$OUTPUT_DIR/upstream/report.md" && -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(jq -r '.title // empty' "$OUTPUT_DIR/upstream/report.json") + if [[ -n "$title" ]]; then + extra=$(printf 'Upstream bug report, for a maintainer to file in microsoft/playwright:\n\n**Title:** %s\n\n
\nBody\n\n````markdown\n%s\n````\n\n
' "$title" "$(cat "$OUTPUT_DIR/upstream/report.md")") + fi + fi + delimiter="EXTRA_$RANDOM$RANDOM" + { echo "extra<<$delimiter"; echo "$extra"; echo "$delimiter"; } >> "$GITHUB_OUTPUT" + - name: Post the triage comment + uses: microsoft/playwright/.github/actions/triage-comment@main + with: + issue: ${{ github.event.issue.number || inputs.issue }} + dir: ${{ runner.temp }}/triage + footer: Triaged on ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} by the Playwright bot + job-result: ${{ needs.triage.result }} + setup-outcome: ${{ needs.triage.outputs.setup }} + agent-outcome: ${{ needs.triage.outputs.agent }} + setup-failure-reason: setting up the runner, building the project or downloading the browsers failed + pull-request: ${{ needs.publish.outputs.pr }} + publish-result: ${{ needs.publish.result }} + extra: ${{ steps.upstream.outputs.extra }} + start-comment: ${{ needs.start.outputs.comment }} diff --git a/CLAUDE.md b/CLAUDE.md index edd4f5c7c..2021c9acd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,6 +28,20 @@ grep "^ Failed" /tmp/test-results.txt # list failures tail -5 /tmp/test-results.txt # summary ``` +- `--filter Playwright.Tests.PageRouteTests` runs one class, and `--filter "FullyQualifiedName=Microsoft.Playwright.Tests.PageRouteTests.ShouldIntercept"` runs one test. `BROWSER` is `chromium`, `firefox` or `webkit`; `HEADED=1` runs headed. +- Each test has a timeout of 30 seconds; `[PlaywrightTest(TestConstants.SlowTestTimeout)]` gives 150. When `CI` is set, a failed test runs up to 3 times; use `env -u CI` to see one run. +- `dotnet test` rebuilds what it needs. Run `dotnet build ./src` before you commit: it builds every project, also the ones that the tests do not reference, and `src/Playwright` treats analyzer warnings as errors. +- The NUnit, MSTest and xUnit packages next to `src/Playwright.NUnit/` are tested from `src/Playwright.TestingHarnessTest/`, a Node suite that writes a small .NET test project per test: `npm ci` there once, then `npx playwright test tests/nunit.spec.ts`. + +## Tests + +The tests are NUnit tests in `src/Playwright.Tests/`, one class per upstream spec file: `PageRouteTests.cs` ports `tests/page/page-route.spec.ts`. Locator and assertion classes are in `Locator/` and `Assertions/`. A regression test is one added case in the class that owns the feature, also when only the port has the bug. + +- `[PlaywrightTest("page-route.spec.ts", "should intercept")]` names the upstream spec file and test title that the test ports. Upstream titles carry an ID prefix, so grep for the title. `[PlaywrightTest]` without arguments marks a test that only the port has. +- Tag a regression test with `// See ` on the line above the attribute. +- The base classes are in `BaseTests/`. `PageTestEx` gives `Page`, `Context`, `Browser` and `BrowserName`; `ContextTestEx` and `BrowserTestEx` stop at `Context` and `Browser` and add `NewContext(options)`. All give `Server`, a `SimpleServer` (`TestServer/SimpleServer.cs`) that serves `assets/`, with `Server.Prefix`, `Server.EmptyPage`, `Server.SetRoute(path, context => ...)` on an ASP.NET Core `HttpContext`, `Server.SetRedirect(from, to)` and `Server.WaitForRequest(path)`. Drive the page with `Page.SetContentAsync(...)` or `Page.GotoAsync(Server.Prefix + "/...")`. Do not start a server or a browser of your own. Override `ContextOptions()` when the class needs other context options, as `TapTests.cs` does. +- Good models: `UrlShouldWorkWithBothDomcontentloadedAndLoad` in `PageWaitForUrlTests.cs`, a ported test with a route that holds its response and the issue comment, and `ShouldNotReturnDisposedJsonElement` in `PageEvaluateTests.cs`, a .NET-only test. + ## Architecture ### Generated vs hand-written code