From 945f21f7ad368770422bbbc60651a49d3a8e10fa Mon Sep 17 00:00:00 2001 From: Pavel Feldman Date: Wed, 28 Oct 2026 20:38:09 -0700 Subject: [PATCH] devops(triage): add the issue triage and revision bots The needs-triage and needs-revision labels start the workflows. They call the shared actions from microsoft/playwright; setup-triage-runner builds this repo and installs the browsers. The triage skill links to the shared guide and keeps only what is specific to this repo. The agent reads CLAUDE.md for build and test. --- .claude/skills/playwright-triage/SKILL.md | 85 +++++++ .../actions/setup-triage-runner/action.yml | 36 +++ .github/workflows/revision.yml | 132 ++++++++++ .github/workflows/triage.yml | 227 ++++++++++++++++++ CLAUDE.md | 14 ++ 5 files changed, 494 insertions(+) create mode 100644 .claude/skills/playwright-triage/SKILL.md create mode 100644 .github/actions/setup-triage-runner/action.yml create mode 100644 .github/workflows/revision.yml create mode 100644 .github/workflows/triage.yml diff --git a/.claude/skills/playwright-triage/SKILL.md b/.claude/skills/playwright-triage/SKILL.md new file mode 100644 index 000000000..a4fe33cab --- /dev/null +++ b/.claude/skills/playwright-triage/SKILL.md @@ -0,0 +1,85 @@ +--- +name: playwright-triage +description: Triage and fix one microsoft/playwright-dotnet issue, or do a maintainer's task, unattended inside the Issue triage GitHub Actions workflow (.github/workflows/triage.yml), which a needs-triage, needs-triage-mac or needs-triage-win label starts, or a maintainer starts by hand with an issue, a task or both. Reproduces the issue on the runner, then leaves a verified fix proposal for the workflow to open as a draft PR, an investigation note for it to post on the issue, or an upstream bug report for a maintainer to file in microsoft/playwright. Only for that workflow. +--- + +# Playwright .NET Issue Triage + +You run unattended in the [Issue triage](../../../.github/workflows/triage.yml) workflow of +`microsoft/playwright-dotnet`. First read the shared guide, `$AGENT_GUIDES_DIR/triage.md`, a copy of +[microsoft/playwright/.github/actions/run-agent/guides/triage.md](https://github.com/microsoft/playwright/blob/main/.github/actions/run-agent/guides/triage.md), +and then [CLAUDE.md](../../../CLAUDE.md), both in full. The guide says how to triage an issue in any +Playwright repo. CLAUDE.md says how to build, test and format this one, and where a regression test +goes. This file covers what neither has: the checkouts, how to run a published version, and the +outcome that this repo adds, an upstream bug report. + +## The checkouts + +`$REPO_DIR` is the checkout of `microsoft/playwright-dotnet` that the guide describes. The driver is +downloaded (`./build.sh --download-driver`), the solution is built (`dotnet build ./src`), and +Chromium, Firefox and WebKit are installed +(`pwsh src/Playwright/bin/Debug/netstandard2.0/playwright.ps1 install --with-deps`). It is the +working directory, and the checkout that the guides refer to. The clone is shallow: run +`git fetch --unshallow --tags origin` in it before you bisect or use `git tag --contains`. .NET SDK +8 and 10 are installed, as in CI. Tests run with `-f net8.0`. + +`$PLAYWRIGHT_DIR` is a shallow clone of `microsoft/playwright` at `main`. The .NET client is a port +of the JavaScript client in `packages/playwright-core/src/client/`. The API docs are in +`docs/src/api/`, the API generator is `utils/doclint/generateDotnetApi.js`, and the upstream tests +are in `tests/page/` and `tests/library/`. The clone is not at the commit of the driver: +`` in `src/Common/Version.props` names that version, `main` is close to it, and a +release has the tag `v`. The clone is not built. Before you run an upstream test in +it, run `npm ci`, `npm run build` and `npx playwright install` there. Then `npm run ctest `, +`npm run ftest ` and `npm run wtest ` run one spec file on Chromium, Firefox and WebKit. + +## Published versions + +To run a published version, for example to bisect, create +`$SCRATCH_DIR/playwright-dotnet-//` and make a console project there: + +```bash +dotnet new console -n repro -f net8.0 -o . && dotnet add package Microsoft.Playwright --version +dotnet build && pwsh bin/Debug/net8.0/playwright.ps1 install chromium +dotnet run +``` + +Put the repro in `Program.cs` as top-level statements, as `src/Playwright.Examples/Program.cs` +does. `playwright.ps1 install` fetches the browsers of that version. Match the change to commits in +`$REPO_DIR` with `git log v..v -- src/Playwright/`, and to the JavaScript client in +`$PLAYWRIGHT_DIR` with the same tags. + +## Upstream bugs + +The .NET client talks to the Node driver that `microsoft/playwright` publishes, and many bugs that +are reported here are in the driver. That fix lands in `microsoft/playwright`. The family is one +team, so this is routing, not a request to the reporter. Find the owner with the same calls in both +clients: write the repro as a .NET test in `$REPO_DIR` first, then write the same calls as an +upstream test in `$PLAYWRIGHT_DIR` and run it there. When the Node test fails the same way, the root +cause is upstream. When only the .NET test fails, the bug is in the port: compare the method in +`src/Playwright/Core/` with the same method in the JavaScript client, and fix it here. + +The interfaces in `src/Playwright/API/Generated/` and the types in +`src/Playwright/Transport/Protocol/Generated/` are generated by `utils/doclint/generateDotnetApi.js` +and `utils/generate_dotnet_channels.js` from the upstream docs and protocol. Never edit them by +hand. A missing method, option or type is an upstream change, in `docs/src/api/` (the `langs` +annotations decide what .NET gets) or in the generator. .NET-only overloads are hand-written in +`src/Playwright/API/Supplements/`, and the implementations are in `src/Playwright/Core/`. + +For an upstream root cause, create `$OUTPUT_DIR/upstream/` with: + +- `report.md`: the report for a maintainer to file in `microsoft/playwright`, for an engineer who + did not read this issue: what fails, expected versus actual, the driver version and the browsers, + and the condensed Node test from `$PLAYWRIGHT_DIR` as the repro. Say that .NET shows the same. +- `report.json`: `{"title": "..."}`, at most 256 characters. + +The comment job appends the report to your comment, for a maintainer to file. The bot never files +it. Still write the comment as the note, without the report in it. Write `report.json` last, when +`report.md` and the comment are done. Like `proposal.json`, it marks the report as ready. + +## PRs from third parties + +Do not run a PR that changes a `.csproj`, `.props` or `.sln` file, `nuget.config`, `build.sh`, +`src/tools/Playwright.Tooling/` (the driver download), `src/Playwright/build/` (MSBuild targets that +run on every build), `.github/`, `.azure-pipelines/`, `utils/`, `package.json`, `package-lock.json`, +or the generated files under `src/Playwright/API/Generated/` and +`src/Playwright/Transport/Protocol/Generated/`. diff --git a/.github/actions/setup-triage-runner/action.yml b/.github/actions/setup-triage-runner/action.yml new file mode 100644 index 000000000..e508edc68 --- /dev/null +++ b/.github/actions/setup-triage-runner/action.yml @@ -0,0 +1,36 @@ +name: 'Set up the triage runner' +description: > + Prepares the environment that the triage and revision agents work in: the checkout built as CI + builds it, with the browsers and the Copilot CLI installed, and a checkout of microsoft/playwright + to read the client that this repo ports. Exports PLAYWRIGHT_DIR, the path of that checkout, and + CI_ENV_EXEC, the wrapper from microsoft/playwright that runs a command under xvfb on Linux and at + normal I/O priority on Windows. +runs: + using: composite + steps: + - uses: microsoft/playwright/.github/actions/ci-env@main + - uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + dotnet-version: | + 8.0.x + 10.0.x + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - name: Download the driver and build + shell: bash + run: | + ./build.sh --download-driver + dotnet build ./src + - name: Install the browsers + shell: bash + run: pwsh src/Playwright/bin/Debug/netstandard2.0/playwright.ps1 install --with-deps + - name: Check out microsoft/playwright + shell: bash + run: | + dir="${RUNNER_TEMP//\\//}/playwright" + git clone --quiet --depth 1 https://github.com/microsoft/playwright "$dir" + echo "PLAYWRIGHT_DIR=$dir" >> "$GITHUB_ENV" + - name: Install the Copilot CLI + shell: bash + run: npm install -g @github/copilot diff --git a/.github/workflows/revision.yml b/.github/workflows/revision.yml new file mode 100644 index 000000000..fa3af88c1 --- /dev/null +++ b/.github/workflows/revision.yml @@ -0,0 +1,132 @@ +# Copilot addresses the maintainers' review comments on a draft PR that the triage workflow opened, +# without write credentials. A fresh job pushes the new commit to the PR's branch using the app, so +# CI runs on it. The steps that are the same for every Playwright repo are actions in +# microsoft/playwright, and the shared guide they carry says how to revise; the prompt below has +# what is specific to this repo. +name: PR revision + +on: + # Apply needs-revision, needs-revision-mac or needs-revision-win; re-apply it to trigger another pass. + # pull_request_target always runs this file from main; the jobs only accept the triage app's own PRs. + pull_request_target: + types: [labeled] + +permissions: {} + +jobs: + prepare: + name: Collect the maintainers' comments + if: >- + github.repository == 'microsoft/playwright-dotnet' && + contains(fromJSON('["needs-revision","needs-revision-mac","needs-revision-win"]'), github.event.label.name) && + github.event.pull_request.state == 'open' && + github.event.pull_request.user.login == 'microsoft-playwright-automation[bot]' && + github.event.pull_request.head.repo.full_name == github.repository + runs-on: ubuntu-latest + permissions: + pull-requests: write + env: + PLATFORM: ${{ fromJSON('{"needs-revision":"linux","needs-revision-mac":"mac","needs-revision-win":"windows"}')[github.event.label.name] }} + outputs: + head: ${{ steps.collect.outputs.head }} + base: ${{ steps.collect.outputs.base }} + branch: ${{ steps.collect.outputs.branch }} + platform: ${{ env.PLATFORM }} + steps: + - name: Collect the PR and the maintainers' comments + id: collect + uses: microsoft/playwright/.github/actions/collect-review@main + with: + pull-request: ${{ github.event.pull_request.number }} + - name: Post the run link + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows"}')[env.PLATFORM] }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + body=$(printf "Hi, I'm the Playwright bot. I started to address the review comments on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$RUN_URL") + gh pr comment "$PR" --body "$body" || echo "::warning::Could not post the run link" + + revise: + name: Revise PR ${{ github.event.pull_request.number }} (${{ needs.prepare.outputs.platform }}) + needs: prepare + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[needs.prepare.outputs.platform] }} + timeout-minutes: 240 + concurrency: + group: revision-${{ github.event.pull_request.number }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ needs.prepare.outputs.head }} + BASE_SHA: ${{ needs.prepare.outputs.base }} + TRIAGE_PLATFORM: ${{ needs.prepare.outputs.platform }} + permissions: + copilot-requests: write + outputs: + has_revision: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + persist-credentials: false + - name: Fetch the base and record the directories + run: | + git fetch --no-tags --depth=1 origin "$BASE_SHA" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/revision" >> "$GITHUB_ENV" + echo "REVIEW_DIR=${RUNNER_TEMP//\\//}/review" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/revision" "${RUNNER_TEMP//\\//}/scratch" + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: review + path: ${{ runner.temp }}/review + - name: Set up the runner + uses: ./.github/actions/setup-triage-runner + - name: Revise with Copilot CLI + id: agent + # No GitHub tools and no URL access: the agent reads the PR only from the review the prepare job filtered. + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: | + Follow the shared guide revision.md and this repo's CLAUDE.md. Read both in full before starting. $PLAYWRIGHT_DIR is a shallow clone of microsoft/playwright at main, for reading the JavaScript client that this repo ports. + + Pull request: https://github.com/microsoft/playwright-dotnet/pull/${{ github.event.pull_request.number }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: commit/commit.json + artifact-name: revision + exec: ${{ env.CI_ENV_EXEC }} + + push: + name: Push the revision + needs: [prepare, revise] + # commit.json is written last, so a finished revision is pushed even when the job failed later. + if: ${{ !cancelled() && needs.revise.outputs.has_revision == 'true' }} + runs-on: ubuntu-latest + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: revision + path: ${{ runner.temp }}/revision + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate and push + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/revision/commit + patch: revision.patch + metadata: commit.json + branch: ${{ needs.prepare.outputs.branch }} diff --git a/.github/workflows/triage.yml b/.github/workflows/triage.yml new file mode 100644 index 000000000..daafbdcd5 --- /dev/null +++ b/.github/workflows/triage.yml @@ -0,0 +1,227 @@ +# Copilot triages and fixes an issue, or works on a task given by hand, without write credentials. +# Fresh jobs post the run link on the issue, open the completed fix as a draft PR using the app, so +# CI runs on it, and comment on the issue. The steps that are the same for every Playwright repo are +# actions in microsoft/playwright. The skill in .claude/skills/playwright-triage/ has what is +# specific to this repo. +name: Issue triage + +on: + # Apply needs-triage, needs-triage-mac or needs-triage-win; re-apply it to trigger another pass. + issues: + types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue number to triage; leave empty to work on the task alone" + type: string + required: false + default: "" + platform: + description: "Platform to triage on" + type: choice + required: true + default: linux + options: + - linux + - mac + - windows + task: + description: "Task to direct the triage, such as what to focus on; the whole job when there is no issue" + type: string + required: false + default: "" + +permissions: {} + +jobs: + start: + name: Post the run link on the issue + # The triage job's condition, for runs that have an issue. + if: >- + github.repository == 'microsoft/playwright-dotnet' && + ((github.event_name == 'workflow_dispatch' && inputs.issue != '') || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + runs-on: ubuntu-latest + permissions: + issues: write + outputs: + comment: ${{ steps.post.outputs.id }} + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + steps: + - name: Post the run link + id: post + run: | + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + body=$(printf "Hi, I'm the Playwright bot. I started to triage this issue on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$WORKFLOW_URL") + id=$(gh api "repos/$GH_REPO/issues/$ISSUE/comments" -f body="$body" --jq .node_id) + echo "id=$id" >> "$GITHUB_OUTPUT" + + triage: + name: Triage ${{ (github.event.issue.number || inputs.issue) && format('issue {0}', github.event.issue.number || inputs.issue) || 'task' }} (${{ inputs.platform || github.event.label.name }}) + if: >- + github.repository == 'microsoft/playwright-dotnet' && + (github.event_name == 'workflow_dispatch' || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + # Falls back to a runner name for unrelated labels, where the job is skipped anyway. + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name]] || 'ubuntu-latest' }} + timeout-minutes: 240 + concurrency: + group: triage-${{ github.event.issue.number || inputs.issue || github.run_id }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + TRIAGE_PLATFORM: ${{ inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name] }} + TASK: ${{ inputs.task }} + permissions: + copilot-requests: write + outputs: + base: ${{ steps.base.outputs.sha }} + setup: ${{ steps.setup.outcome }} + agent: ${{ steps.agent.outputs.outcome }} + has_fix: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + persist-credentials: false + - name: Record the base and the directories + id: base + run: | + if [[ -n "$ISSUE" ]]; then + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + elif [[ -z "${TASK//[[:space:]]/}" ]]; then + echo "give an issue, a task or both" >&2; exit 1 + fi + echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/triage" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/triage" "${RUNNER_TEMP//\\//}/scratch" + - name: Set up the runner + id: setup + uses: ./.github/actions/setup-triage-runner + - name: Write the prompt + id: prompt + run: | + delimiter="PROMPT_$RANDOM$RANDOM" + { + echo "prompt<<$delimiter" + echo "Follow .claude/skills/playwright-triage/SKILL.md. Read it, the guide and the CLAUDE.md" + echo "it points to in full before starting." + if [[ -n "$ISSUE" ]]; then + printf '\nIssue: https://github.com/%s/issues/%s\n' "$GITHUB_REPOSITORY" "$ISSUE" + fi + if [[ -n "$TASK" ]]; then + printf '\nTask: %s\n' "$TASK" + fi + echo "$delimiter" + } >> "$GITHUB_OUTPUT" + - name: Triage with Copilot CLI + id: agent + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: ${{ steps.prompt.outputs.prompt }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: fix/proposal.json + artifact-name: triage + github-tools: true + exec: ${{ env.CI_ENV_EXEC }} + - name: Check the upstream bug report + run: | + if [[ -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(node -p 'String(JSON.parse(require("fs").readFileSync(process.argv[1], "utf8")).title ?? "")' "$OUTPUT_DIR/upstream/report.json" || true) + if [[ -z "$title" ]]; then + echo "::warning::upstream/report.json has no title, so the report is not appended to the comment" + else + echo "Upstream bug report: $title" >> "$GITHUB_STEP_SUMMARY" + fi + fi + + publish: + name: Open draft pull request + needs: triage + # proposal.json is written last, so a finished proposal is published even when the job failed later. + if: ${{ !cancelled() && needs.triage.outputs.has_fix == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.publish.outputs.pull-request }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.triage.outputs.base }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: triage + path: ${{ runner.temp }}/triage + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate, push and open draft + id: publish + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/triage/fix + patch: fix.patch + metadata: proposal.json + body-file: pr-body.md + branch: triage/${{ github.event.issue.number || inputs.issue || 'task' }}/${{ github.run_id }}-${{ github.run_attempt }} + commit-body: "${{ (github.event.issue.number || inputs.issue) && format('Fixes: https://github.com/{0}/issues/{1}', github.repository, github.event.issue.number || inputs.issue) || '' }}" + pull-request: true + + comment: + name: Comment on the issue + needs: [start, triage, publish] + # Runs after every triage run that has an issue, including failed ones, and whatever publish did. + if: ${{ !cancelled() && needs.triage.result != 'skipped' && (github.event_name == 'issues' || inputs.issue != '') }} + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + # A failed triage run may not have uploaded anything. + continue-on-error: true + with: + name: triage + path: ${{ runner.temp }}/triage + # The bot never files upstream; the comment hands the report to a maintainer. + - name: Describe the upstream bug report + id: upstream + env: + OUTPUT_DIR: ${{ runner.temp }}/triage + run: | + extra="" + if [[ -s "$OUTPUT_DIR/upstream/report.md" && -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(jq -r '.title // empty' "$OUTPUT_DIR/upstream/report.json") + if [[ -n "$title" ]]; then + extra=$(printf 'Upstream bug report, for a maintainer to file in microsoft/playwright:\n\n**Title:** %s\n\n
\nBody\n\n````markdown\n%s\n````\n\n
' "$title" "$(cat "$OUTPUT_DIR/upstream/report.md")") + fi + fi + delimiter="EXTRA_$RANDOM$RANDOM" + { echo "extra<<$delimiter"; echo "$extra"; echo "$delimiter"; } >> "$GITHUB_OUTPUT" + - name: Post the triage comment + uses: microsoft/playwright/.github/actions/triage-comment@main + with: + issue: ${{ github.event.issue.number || inputs.issue }} + dir: ${{ runner.temp }}/triage + footer: Triaged on ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} by the Playwright bot + job-result: ${{ needs.triage.result }} + setup-outcome: ${{ needs.triage.outputs.setup }} + agent-outcome: ${{ needs.triage.outputs.agent }} + setup-failure-reason: setting up the runner, building the project or downloading the browsers failed + pull-request: ${{ needs.publish.outputs.pr }} + publish-result: ${{ needs.publish.result }} + extra: ${{ steps.upstream.outputs.extra }} + start-comment: ${{ needs.start.outputs.comment }} diff --git a/CLAUDE.md b/CLAUDE.md index edd4f5c7c..2021c9acd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,6 +28,20 @@ grep "^ Failed" /tmp/test-results.txt # list failures tail -5 /tmp/test-results.txt # summary ``` +- `--filter Playwright.Tests.PageRouteTests` runs one class, and `--filter "FullyQualifiedName=Microsoft.Playwright.Tests.PageRouteTests.ShouldIntercept"` runs one test. `BROWSER` is `chromium`, `firefox` or `webkit`; `HEADED=1` runs headed. +- Each test has a timeout of 30 seconds; `[PlaywrightTest(TestConstants.SlowTestTimeout)]` gives 150. When `CI` is set, a failed test runs up to 3 times; use `env -u CI` to see one run. +- `dotnet test` rebuilds what it needs. Run `dotnet build ./src` before you commit: it builds every project, also the ones that the tests do not reference, and `src/Playwright` treats analyzer warnings as errors. +- The NUnit, MSTest and xUnit packages next to `src/Playwright.NUnit/` are tested from `src/Playwright.TestingHarnessTest/`, a Node suite that writes a small .NET test project per test: `npm ci` there once, then `npx playwright test tests/nunit.spec.ts`. + +## Tests + +The tests are NUnit tests in `src/Playwright.Tests/`, one class per upstream spec file: `PageRouteTests.cs` ports `tests/page/page-route.spec.ts`. Locator and assertion classes are in `Locator/` and `Assertions/`. A regression test is one added case in the class that owns the feature, also when only the port has the bug. + +- `[PlaywrightTest("page-route.spec.ts", "should intercept")]` names the upstream spec file and test title that the test ports. Upstream titles carry an ID prefix, so grep for the title. `[PlaywrightTest]` without arguments marks a test that only the port has. +- Tag a regression test with `// See ` on the line above the attribute. +- The base classes are in `BaseTests/`. `PageTestEx` gives `Page`, `Context`, `Browser` and `BrowserName`; `ContextTestEx` and `BrowserTestEx` stop at `Context` and `Browser` and add `NewContext(options)`. All give `Server`, a `SimpleServer` (`TestServer/SimpleServer.cs`) that serves `assets/`, with `Server.Prefix`, `Server.EmptyPage`, `Server.SetRoute(path, context => ...)` on an ASP.NET Core `HttpContext`, `Server.SetRedirect(from, to)` and `Server.WaitForRequest(path)`. Drive the page with `Page.SetContentAsync(...)` or `Page.GotoAsync(Server.Prefix + "/...")`. Do not start a server or a browser of your own. Override `ContextOptions()` when the class needs other context options, as `TapTests.cs` does. +- Good models: `UrlShouldWorkWithBothDomcontentloadedAndLoad` in `PageWaitForUrlTests.cs`, a ported test with a route that holds its response and the issue comment, and `ShouldNotReturnDisposedJsonElement` in `PageEvaluateTests.cs`, a .NET-only test. + ## Architecture ### Generated vs hand-written code