Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
8c53c7b
test(e2e): configure the e2e host from launch extras and give it one …
mikepitre Oct 7, 2026
ac1e485
Merge branch 'main' into mike/verify-host-launch-extras
mikepitre Oct 8, 2026
4aafad6
Merge branch 'main' into mike/verify-host-launch-extras
mikepitre Oct 9, 2026
7bd316e
chore: add the end-to-end test package, its entry point, and repo wiring
mikepitre Oct 8, 2026
1ff15b4
feat(verify): add the shared core: the CLI, device leases, the Backen…
mikepitre Oct 8, 2026
ada34fd
feat(verify): create, configure, and delete a throwaway Clerk applica…
mikepitre Oct 8, 2026
df49850
feat(verify): add the platform host and device backends
mikepitre Oct 8, 2026
29c6147
feat(verify): add the golden tests, their fixtures, and the feature f…
mikepitre Oct 8, 2026
4136e63
ci: run the end-to-end tests in the release workflow
mikepitre Oct 7, 2026
06e9581
test(verify): add the runner's unit tests and run them in CI
mikepitre Oct 8, 2026
511d496
docs(verify): an agent proves its own change, and the release workflo…
mikepitre Oct 7, 2026
ba2df83
feat(verify): put a run's evidence in the pull request description
mikepitre Oct 8, 2026
34b0c46
docs(verify): document the skill for agents and contributors
mikepitre Oct 8, 2026
dd02618
test(e2e): remove the Trailblaze trails, their home in the e2e host, …
mikepitre Oct 7, 2026
117ef67
feat(verify): borrow a device on a CI runner for a remote session
mikepitre Oct 8, 2026
c2847c1
feat(verify): choose between the local backend and a remote session
mikepitre Oct 8, 2026
2fab79e
test(verify): add the unit tests of the remote session
mikepitre Oct 8, 2026
b9ac2ca
docs(verify): document the remote session for agents and contributors
mikepitre Oct 8, 2026
6ca43d9
fix(verify): probe an https proxy on port 443 when its URL names no port
mikepitre Oct 13, 2026
7e733fe
test(verify): give each session agent in the unit tests ports that ar…
mikepitre Oct 13, 2026
5d12281
fix(verify): ask a replacement session for the commit the branch has now
mikepitre Oct 13, 2026
c70dcd2
docs(verify): fix what a cold run of the skill found
mikepitre Oct 13, 2026
dda90d6
docs(verify): list attach before down in AGENTS.md
mikepitre Oct 13, 2026
f4eaf89
fix(verify): give a slow build its own limit and message, and notice …
mikepitre Oct 13, 2026
e03ea2f
fix(verify): sign in with the email code on the standard settings
mikepitre Oct 13, 2026
dc920ad
Merge remote-tracking branch 'origin/main' into mike/verify-remote-de…
mikepitre Oct 13, 2026
029b1b1
test(verify): close the backup codes in the two setup-MFA golden tests
mikepitre Oct 13, 2026
b7808cf
Merge branch 'mike/verify-remote-device-android' into mike/verify-ci-…
mikepitre Oct 13, 2026
08d5eca
Merge remote-tracking branch 'origin/main' into fix/verify-host-launc…
swolfand Oct 13, 2026
83c2016
fix(e2e): initialise Clerk in the cleanup receiver when the broadcast…
swolfand Oct 13, 2026
3489909
fix(e2e): skip the launch ticket sign-in when restored storage alread…
swolfand Oct 13, 2026
382b072
Merge remote-tracking branch 'origin/mike/verify-host-launch-extras' …
swolfand Oct 13, 2026
719b680
fix(verify): fail fast on an unsupported Node instead of running node…
swolfand Oct 13, 2026
e406cd5
fix(verify): mint a new sign-in ticket and storage scope when retryin…
swolfand Oct 13, 2026
480602a
fix(verify): keep the Clerk and agent keys out of the Gradle build en…
swolfand Oct 13, 2026
0913d96
chore(renovate): keep vulnerability alerts for e2e-tests while its ot…
swolfand Oct 13, 2026
ffee122
Merge remote-tracking branch 'origin/mike/verify-remote-device-androi…
swolfand Oct 13, 2026
e8d70f6
ci(verify): drop the comment that explains our own CLI's throwaway mi…
swolfand Oct 13, 2026
f6c66ff
Merge remote-tracking branch 'origin/mike/verify-ci-device-specs-andr…
swolfand Oct 13, 2026
2f17453
ci(verify): start a remote session only on the runner labels the repo…
swolfand Oct 13, 2026
f8b4524
revert(verify): run under node@24 through npx again on an unsupported…
mikepitre Oct 13, 2026
ce5a58e
Merge branch 'mike/verify-remote-device-android' into mike/verify-ci-…
mikepitre Oct 13, 2026
a81ec73
Merge remote-tracking branch 'origin/main' into mike/verify-remote-de…
mikepitre Oct 13, 2026
ff377b3
Merge branch 'mike/verify-remote-device-android' into mike/verify-ci-…
mikepitre Oct 13, 2026
4cbd454
Merge branch 'mike/verify-ci-device-specs-android' into mike/verify-b…
mikepitre Oct 13, 2026
725b261
fix(verify): start no build command after the build limit, and report…
mikepitre Oct 13, 2026
695c1c2
ci(verify): print both logs when no tunnel comes up on a Linux runner
mikepitre Oct 13, 2026
e97acd6
Merge remote-tracking branch 'origin/main' into mike/verify-remote-de…
mikepitre Oct 13, 2026
767c7f2
feat(verify): shorten the test email addresses to 37 characters
mikepitre Oct 13, 2026
1679c6b
docs(verify): ask for emulator proof of a change, by the skill or by …
mikepitre Oct 13, 2026
ba23a84
Merge branch 'mike/verify-remote-device-android' into mike/verify-ci-…
mikepitre Oct 13, 2026
a36df62
Merge branch 'mike/verify-ci-device-specs-android' into mike/verify-b…
mikepitre Oct 13, 2026
024ee47
fix(verify): draw the random part of a test address from 0-9 and a-z
mikepitre Oct 13, 2026
588220b
Merge branch 'mike/verify-remote-device-android' into mike/verify-ci-…
mikepitre Oct 13, 2026
2e99a9b
Merge branch 'mike/verify-ci-device-specs-android' into mike/verify-b…
mikepitre Oct 13, 2026
34047d7
Merge remote-tracking branch 'origin/main' into mike/verify-ci-device…
mikepitre Oct 13, 2026
9224a80
Merge branch 'mike/verify-ci-device-specs-android' into mike/verify-b…
mikepitre Oct 13, 2026
e7ba6cf
Merge remote-tracking branch 'origin/main' into mike/verify-borrowed-…
mikepitre Oct 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 45 additions & 7 deletions .claude/skills/verify-clerk-android/SKILL.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions .claude/skills/verify-clerk-android/features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Clerk's test mode makes all of this safe to type into the real app.

The names come from `source/ui/src/main/java/com/clerk/ui/ClerkTestTags.kt`. They are internal test hooks and may change.

On the standard instance an email sign-in starts on the email-link screen ("Check your email"), so a spec switches to the email code through `Use another method`. For a phone sign-in, tap `clerk.auth.start.identifierSwitcher`, fill `clerk.auth.start.phoneNumber` with the ten digits of the seeded user's phone, and continue. A user with a password sees the password screen first and reaches the SMS code through `Use another method`. For sign-up, launch with `authMode: 'signUp'`, fill the email from `host.newEmail()` into `clerk.auth.start.identifier`, continue, fill the run password into `clerk.auth.signUp.password`, continue, then type the code into `clerk.auth.signUp.code`.
On the standard instance an email sign-in for a user with no password goes straight to the email code screen ("Check your email"). The email link is off there. For a phone sign-in, tap `clerk.auth.start.identifierSwitcher`, fill `clerk.auth.start.phoneNumber` with the ten digits of the seeded user's phone, and continue. A user with a password sees the password screen first and reaches the SMS code through `Use another method`. For sign-up, launch with `authMode: 'signUp'`, fill the email from `host.newEmail()` into `clerk.auth.start.identifier`, continue, fill the run password into `clerk.auth.signUp.password`, continue, then type the code into `clerk.auth.signUp.code`.

## Feature entry contract

Expand All @@ -46,4 +46,4 @@ Each feature file starts with an H1 title and one paragraph describing the user-
- [Session tasks](./session-tasks.md) covers the setup-MFA and choose-organization tasks after sign-in, the setup-MFA task after sign-up, and finishing the setup-MFA task with an authenticator code.
- [Organizations](./organizations.md) covers creating an organization from OrganizationSwitcher.

Not mapped: a password sign-in past the password screen, completing the setup-MFA task with an SMS code, social providers (no real OAuth on emulators), passkeys, and biometrics.
Not mapped: a password sign-in past the password screen, the email link (off on the standard instance), completing the setup-MFA task with an SMS code, social providers (no real OAuth on emulators), passkeys, and biometrics.
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
# Sign in with an email code

An existing user enters their email, asks for a one-time code instead of the email link, and signs in with the code.
An existing user enters their email, gets a one-time code, and signs in with the code.

## Sub-features

- `complete` reaches the email code screen for an existing test user through `Use another method`, stays on it, types `424242`, and lands signed in.
- `complete` reaches the email code screen for an existing test user straight from Continue, stays on it, types `424242`, and lands signed in.

## How to get to it (user POV)

- Open sign-in, type the email, tap Continue. On the standard instance the email-link screen ("Check your email") opens first. Tap `Use another method`, then `Email code to <email>`.
- Open sign-in, type the email, tap Continue. On the standard instance a user with no password lands on the email code screen ("Check your email"). Type the code.

## Driving it with verify

Preconditions:

- The spec seeds a `+clerk_test` user on the standard instance and launches with `authMode: 'signIn'`, then taps `e2e.auth.signInFullScreen`.

- **Request code.** Run `e2e-tests/bin/control-clerk-android run sign-in-email-code/complete`. It fills `clerk.auth.start.identifier`, taps `clerk.auth.start.continue`, waits for `clerk.auth.signIn.useAnotherMethod`, taps it and `clerk.auth.signIn.alternativeMethod.email_code`, expects `clerk.auth.signIn.code`, and expects it still on screen 3 seconds later, with the email on the code screen. Screenshot `code-screen`.
- **Request code.** Run `e2e-tests/bin/control-clerk-android run sign-in-email-code/complete`. It fills `clerk.auth.start.identifier`, taps `clerk.auth.start.continue`, expects `clerk.auth.signIn.code`, and expects it still on screen 3 seconds later, with the email on the code screen. Screenshot `code-screen`.
- **Complete.** After the code screen it fills `CLERK_TEST_CODE` and waits for the home to show `Signed in as` the seeded user's email with that user's ID. Screenshot `signed-in`.
- **Proof.** The spec passes, and screenshot `signed-in` shows the home with the seeded user's email and ID.

## Gotchas

- The spec checks the code field a second time after 3 seconds. A sign-in that shows the code screen for a moment and then returns to the email-link screen would pass a single visibility check.
- The email-link screen and the email code screen both have the title "Check your email". The code field, `clerk.auth.signIn.code`, tells them apart.
- The spec checks the code field a second time after 3 seconds. A sign-in that shows the code screen for a moment and then leaves it would pass a single visibility check.
- The standard instance has the email link off, because a test address gets no mail, so a run has no link to follow. With the link on, AuthView opens the email-link screen first. It has the same title as the code screen, "Check your email", and no code field, so `clerk.auth.signIn.code` tells them apart. A spec that needs the link declares `auth_email.sign_in_strategies` in a settings file.
9 changes: 5 additions & 4 deletions .claude/skills/verify-clerk-android/references/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
## Verbs and flags

```console
e2e-tests/bin/control-clerk-android doctor [--backend auto|local] [--live]
e2e-tests/bin/control-clerk-android up [--backend auto|local] [--wait <seconds>]
e2e-tests/bin/control-clerk-android run <feature | feature/spec | path.e2e.ts>... | --all [--backend auto|local] [--grep <regex>] [--retries <n>] [--github-report] [--no-video] [--wait <seconds>]
e2e-tests/bin/control-clerk-android doctor [--backend auto|local|remote] [--runner <label>] [--live]
e2e-tests/bin/control-clerk-android up [--backend auto|local|remote] [--runner <label>] [--wait <seconds>]
e2e-tests/bin/control-clerk-android run <feature | feature/spec | path.e2e.ts>... | --all [--backend auto|local|remote] [--runner <label>] [--grep <regex>] [--retries <n>] [--github-report] [--no-video] [--wait <seconds>]
e2e-tests/bin/control-clerk-android screen [--png]
e2e-tests/bin/control-clerk-android attach <run-id> --pr <n> [--screenshot <label>]...
e2e-tests/bin/control-clerk-android down [--stale] [--dry-run]
Expand All @@ -15,14 +15,15 @@ Every verb takes `--json` and then prints one `{ "ok": ... }` object. Exit codes

`--github-report` is for a CI job. After the run, `run` hands the results of every settings group to `@e2e-dev/github` as one report. The reporter writes that report to the job summary. When the event of the job names a pull request and the step has a `GITHUB_TOKEN` that may write pull request comments, it also posts one comment and updates that same comment on later runs. `run` prints one `github` line that says what the reporter did, the reporter never changes the exit code, and nothing is reported for a run whose files hold a secret. Without the flag, `run` reports nothing to GitHub.

`--backend auto` is the default and leaves the choice to the CLI.
`--backend auto` is the default and leaves the choice to the CLI. `--runner` names the runner label of a remote session. With the local backend it is a usage error, and `doctor` takes it only with `--live`.

## `--wait`

- `--wait` bounds the wait for a free lane in the machine-wide pool, and separately the wait for another verb in this worktree that is driving the device. Each wait gets the full budget. The default is 0, so a full pool fails at once.
- It does not bound the wait for an `up` already running in this worktree. `run` waits for that `up` with no limit and prints that it is waiting.
- Waiters get no turn order. When a lane frees, any waiting worktree can take it.
- When the budget runs out, the verb exits 3 with `POOL_FULL` or `DEVICE_BUSY`.
- A remote session has no lane pool, so there `--wait` only bounds the wait for another verb in this worktree.

While it waits, the CLI prints a `wait` line naming each lane and the worktree that holds it, prints it again when that changes, and prints `still waiting after <n>s` every minute otherwise.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## The standard settings

`src/core/instances/base.json` defines the standard instance, with everything that can coexist switched on: email code, email link, phone code, password, username, TOTP, backup codes, organizations, and social buttons. It has two parts.
`src/core/instances/base.json` defines the standard instance, with everything that can coexist switched on except the email link: email code, phone code, password, username, TOTP, backup codes, organizations, and social buttons. The email link is off so that an email sign-in opens on the code screen. It has two parts.

- `config` is the whole body of the Platform API request that puts an instance on the standard settings.
- `environment` lists the 62 leaves of the instance's public `/v1/environment` that a spec cannot run without. One of them, `user_settings.actions.delete_self`, has no `config` key. A new application lets a user delete their own account, and the check fails if Clerk changes that.
Expand Down
Loading
Loading