feat(pr-pipeline): trust a reconciler-recorded re-run of a lost pr-verify run

.github/actions/check-trusted-trigger/README.md

@@ -28,6 +28,7 @@ condition 1 and the actor and author parts of condition 2).
 | `require-pr` | `true` | `false` checks only the actors and reads no PR |
 | `require-kind` | empty | `human` narrows the actor-only checks to actors listed as `kind: human` |
 | `app-label` | empty | With `require-kind: human`, the one label the lanes App may apply to an issue to start the lane |
+| `reconciler-workflow` | empty | The lost-run reconciler's workflow path: a `github-actions[bot]` re-run proceeds only as its recorded re-run; empty stops every such re-run |
 
 ## Rules
 
@@ -91,7 +92,7 @@ condition 1 and the actor and author parts of condition 2).
   a first `completed` seen after the 24th wait refuses at once. Each wait logs
   `dispatcher record: absent, re-read N` with the two run ids.
 
-  A failed read, a missing or unparseable field, or any failed condition stops with
+  A failed read, a missing or unparsable field, or any failed condition stops with
   `untrusted-actor`. The re-runner of a dispatched run is read by id from this run's
   `triggering_actor`: `github-actions[bot]` on attempt 1, otherwise a listed id that is not
   denied, or the lane stops with `untrusted-rerunner`. The PR author check still applies.
@@ -132,6 +133,37 @@ condition 1 and the actor and author parts of condition 2).
   unlisted or denied re-runner stops with `untrusted-rerunner`, which `report-check-run` maps to
   no skip reason, so the check posts failure: a re-run cannot turn an earlier red check on the same
   SHA neutral.
+- **Reconciler re-run (`reconciler-workflow`).** On the `pull_request` and `workflow_run` paths,
+  when `triggering-actor` is `github-actions[bot]` and `reconciler-workflow` is set, the re-runner
+  check is replaced by a check of the lost-run reconciler's record; the event's actors and the PR
+  author are still checked as above. It proceeds only when every one of these holds, and any
+  failed read or condition stops with `untrusted-rerunner`:
+  - `github-actions[bot]` (41898282) is not in `denied-actor-ids`, `GITHUB_RUN_ATTEMPT` is `2`,
+    and the payload names the repository's default branch;
+  - this run's attempt 2, read through `GET /actions/runs/{github.run_id}/attempts/2`, has that
+    `id` and `run_attempt`, `triggering_actor.id` 41898282, this repository, a 40-hex `head_sha`
+    and a `run_started_at`;
+  - `GET /actions/artifacts?name=reconcile-rerun-<run id>-2-<head_sha>&per_page=100`, up to ten
+    pages, lists an
+    artifact with exactly that name, `expired` `false`, and a `created_at` no later than the
+    attempt's `run_started_at`, whose `workflow_run.id` names a run that has that `id`, this
+    repository, `path` equal to `reconciler-workflow`, `head_branch` the default branch, `event`
+    `schedule` or `workflow_dispatch` by an `actor` listed as `kind: human`, any re-run of it by a
+    `triggering_actor` listed as `kind: human`, and a `run_started_at` no later than the attempt's
+    and at most ten minutes before it;
+  - that run's `head_sha` passes the same tag and ancestry checks as a dispatcher run.
+
+  Any run can upload an artifact with that name, so each listed artifact's run is read and
+  judged, and one that passes is enough; one whose run cannot be read does not vouch. At most 20
+  candidate runs are read per check; a 21st candidate stops it with `untrusted-rerunner`. The run id and the attempt bind the record to one run
+  attempt, which starts once, so a record cannot vouch for another run or a later attempt. The
+  reconciler uploads the record before it calls the re-run, so an absent record is read once more
+  after 5 s and then refused.
+
+  Residuals: whether one run can write another run's artifacts is unverified, as for the
+  dispatcher record. A re-run that waits in a queue past the ten-minute window is refused, red;
+  a person re-runs it. More than 1,000 forged artifacts of one name would push the real record
+  past the pages read, which refuses the re-run: a denial, never a pass.
 - **Denied ids.** When `sender` or a `workflow_run` actor has an id in `denied-actor-ids`, the
   lane stops with `bot-actor`, even though that actor is listed. `pr-run-activity-write.yml`
   denies the lanes App bot, so the bot's own pushes and dispatches never chain a write activity.
@@ -151,7 +183,9 @@ any App token exists, so the job token is the only one available, and it needs
 read-only, so no write token reaches the model. A token that cannot read the PR stops the lane
 with `no-pr`. A dispatcher-sent run also reads two run records, a comparison and the dispatcher
 run's artifact list; on this public repository the job token reads them without `actions: read`,
-and a token that cannot stops the lane with `untrusted-actor`.
+and a token that cannot stops the lane with `untrusted-actor`. A reconciler re-run reads its own
+attempt, the artifact list by name, the recording run and a comparison the same way, and a token
+that cannot stops it with `untrusted-rerunner`.
 
 ## Outputs
 

.github/actions/check-trusted-trigger/action.yml

@@ -31,8 +31,8 @@ inputs:
   github-token:
     description: >
       Token that can read the PR, used on workflow_dispatch and workflow_run
-      to fetch it, and on a dispatcher-sent run to read both run records and
-      compare commits.
+      to fetch it, and on a dispatcher-sent run or a reconciler re-run to read
+      the run records and artifacts and compare commits.
     required: false
     default: ${{ github.token }}
   denied-actor-ids:
@@ -98,6 +98,14 @@ inputs:
       every dispatcher-sent run.
     required: false
     default: ""
+  reconciler-workflow:
+    description: >
+      The workflow path of the lost-run reconciler. A re-run github-actions[bot]
+      started proceeds only as that workflow's recorded re-run of this run's
+      first attempt (the README's reconciler section). Empty keeps every
+      github-actions[bot] re-run stopped with untrusted-rerunner.
+    required: false
+    default: ""
   require-kind:
     description: >
       Empty accepts any listed actor. `human` narrows the actor-only checks
@@ -162,6 +170,7 @@ runs:
         EXPECTED_UPSTREAM_HEAD: ${{ inputs.expected-upstream-head }}
         LANES_APP_ID: ${{ inputs.lanes-app-id }}
         DISPATCHED_WORKFLOW: ${{ inputs.dispatched-workflow }}
+        RECONCILER_WORKFLOW: ${{ inputs.reconciler-workflow }}
         REQUIRE_KIND: ${{ inputs.require-kind }}
         APP_LABEL: ${{ inputs.app-label }}
         REQUIRE_PR: ${{ inputs.require-pr }}

.github/actions/check-trusted-trigger/check-trusted-trigger.mjs

@@ -35,7 +35,20 @@ const RUN_RECORD_NAME = /^pr-feedback-dispatched-([1-9][0-9]{0,14})-([1-9][0-9]{
 // The record can land after this run's gate starts: re-read every 5 s while
 // the dispatcher run is still going, for at most 2 min (judgment).
 const RECORD_WAIT_MS = 5000;
+// Pages of same-name artifacts read for a reconciler record (judgment).
+const RECORD_MAX_PAGES = 10;
+// Candidate runs read per check: anyone's run can upload a record of the
+// predictable name, so each forged one costs a run read (judgment).
+const RECORD_MAX_CANDIDATES = 20;
 const RECORD_MAX_SLEEPS = 24;
+// The lost-run reconciler re-runs a lane run once, on its first attempt, so
+// the attempt it starts is always the second.
+const RERUN_ATTEMPT = 2;
+const RECONCILER_EVENTS = new Set(["schedule", "workflow_dispatch"]);
+
+// The artifact the reconciler run uploads before it re-runs a lane run: the
+// run, the attempt the re-run starts, and the run's head SHA.
+export const rerunRecordName = (runId, headSha) => `reconcile-rerun-${runId}-${RERUN_ATTEMPT}-${headSha}`;
 
 const defaultSleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
 
@@ -200,19 +213,42 @@ function checkRerunnerLogin({ ids, logins }, deniedIds, triggeringActor, pool =
 
 const timeOf = (value) => (typeof value === "string" ? Date.parse(value) : Number.NaN);
 
-async function readOrRefuse(github, apiPath) {
+async function readOrRefuse(github, apiPath, reason = "untrusted-actor") {
   let body;
   try {
     body = await github("GET", apiPath);
   } catch {
-    throw new Stop("untrusted-actor");
+    throw new Stop(reason);
   }
   if (!isPlainObject(body)) {
-    throw new Stop("untrusted-actor");
+    throw new Stop(reason);
   }
   return body;
 }
 
+// The run at headSha ran the default branch's own code: no tag shares the
+// branch's name (a run on that tag would report the same head_branch; only a
+// 404 proves it absent), and headSha is the branch tip or behind it.
+async function requireOnDefaultBranch(github, repository, defaultBranch, headSha, reason) {
+  let tagAbsent = false;
+  try {
+    await github("GET", `/repos/${repository}/git/ref/tags/${defaultBranch}`);
+  } catch (error) {
+    tagAbsent = error instanceof GitHubError && error.status === 404;
+  }
+  if (!tagAbsent) {
+    throw new Stop(reason);
+  }
+  const tip = await readOrRefuse(github, `/repos/${repository}/git/ref/heads/${defaultBranch}`, reason);
+  if (tip.ref !== `refs/heads/${defaultBranch}` || !SHA.test(tip.object?.sha ?? "")) {
+    throw new Stop(reason);
+  }
+  const comparison = await readOrRefuse(github, `/repos/${repository}/compare/${headSha}...${tip.object.sha}`, reason);
+  if (comparison.status !== "ahead" && comparison.status !== "identical") {
+    throw new Stop(reason);
+  }
+}
+
 // What a record name says about this run: null when it does not list it,
 // otherwise true, or for a workflow_run dispatcher's record the upstream run id.
 function readRecordName(name, runId, viaRun) {
@@ -359,28 +395,7 @@ async function verifyDispatcher({
   if (!trusted) {
     throw new Stop("untrusted-actor");
   }
-  // A tag with the branch's name would give a run on that tag the same
-  // head_branch, so one must not exist; only a 404 proves that.
-  let tagAbsent = false;
-  try {
-    await github("GET", `/repos/${repository}/git/ref/tags/${defaultBranch}`);
-  } catch (error) {
-    tagAbsent = error instanceof GitHubError && error.status === 404;
-  }
-  if (!tagAbsent) {
-    throw new Stop("untrusted-actor");
-  }
-  const tip = await readOrRefuse(github, `/repos/${repository}/git/ref/heads/${defaultBranch}`);
-  if (tip.ref !== `refs/heads/${defaultBranch}` || !SHA.test(tip.object?.sha ?? "")) {
-    throw new Stop("untrusted-actor");
-  }
-  const comparison = await readOrRefuse(
-    github,
-    `/repos/${repository}/compare/${dispatcher.head_sha}...${tip.object.sha}`,
-  );
-  if (comparison.status !== "ahead" && comparison.status !== "identical") {
-    throw new Stop("untrusted-actor");
-  }
+  await requireOnDefaultBranch(github, repository, defaultBranch, dispatcher.head_sha, "untrusted-actor");
   if (expectedUpstreamHead !== "" && (!viaRun || !SHA.test(expectedUpstreamHead))) {
     throw new Stop("untrusted-actor");
   }
@@ -464,6 +479,114 @@ function checkDispatchedRerunner({ ids }, deniedIds, own) {
   }
 }
 
+// A re-run github-actions[bot] started names no human, so it is trusted only
+// as the lost-run reconciler's re-run of this run: this run's attempt
+// RERUN_ATTEMPT, started by github-actions[bot], and an unexpired
+// rerunRecordName artifact, created no later than that attempt started, of a
+// reconcilerWorkflow run in this repository on the default branch, started by
+// its schedule or a listed human's dispatch, any re-run of it by a listed
+// human, its head SHA the default branch tip or behind it, and this attempt
+// started between that run's start and WINDOW_MS after it. A run id names one
+// run and its second attempt starts once, so a record vouches for one attempt.
+// The reconciler uploads the record before it re-runs; one more read follows
+// RECORD_WAIT_MS for any lag in the artifact list. Every refusal is
+// untrusted-rerunner, as for an unlisted re-runner.
+async function verifyReconcilerRerun({ github, repository, event, list, deniedIds, reconcilerWorkflow, runId, runAttempt, sleep }) {
+  const refuse = "untrusted-rerunner";
+  const defaultBranch = event.repository?.default_branch;
+  if (
+    deniedIds.has(ACTIONS_BOT_ID) ||
+    !RUN_ID.test(runId) ||
+    runAttempt !== String(RERUN_ATTEMPT) ||
+    typeof defaultBranch !== "string" ||
+    !HEAD_REF.test(defaultBranch)
+  ) {
+    throw new Stop(refuse);
+  }
+  const own = await readOrRefuse(github, `/repos/${repository}/actions/runs/${runId}/attempts/${RERUN_ATTEMPT}`, refuse);
+  const ownStarted = timeOf(own.run_started_at);
+  if (
+    String(own.id) !== runId ||
+    own.run_attempt !== RERUN_ATTEMPT ||
+    own.triggering_actor?.id !== ACTIONS_BOT_ID ||
+    own.repository?.full_name !== repository ||
+    !SHA.test(own.head_sha ?? "") ||
+    Number.isNaN(ownStarted)
+  ) {
+    throw new Stop(refuse);
+  }
+  const name = rerunRecordName(runId, own.head_sha);
+  const isListedHuman = (user) =>
+    Number.isInteger(user?.id) && list.humans.has(user.id) && !deniedIds.has(user.id);
+  // Anyone's run can upload an artifact of this name, so each candidate's run
+  // is read and judged, at most RECORD_MAX_CANDIDATES of them; one that cannot
+  // be read does not vouch.
+  let candidates = 0;
+  const vouches = async (artifact) => {
+    if (
+      !isPlainObject(artifact) ||
+      artifact.name !== name ||
+      artifact.expired !== false ||
+      !Number.isSafeInteger(artifact.workflow_run?.id) ||
+      !(timeOf(artifact.created_at) <= ownStarted)
+    ) {
+      return false;
+    }
+    if (candidates >= RECORD_MAX_CANDIDATES) {
+      console.log(`check-trusted-trigger: reconciler record: over ${RECORD_MAX_CANDIDATES} candidate runs, refusing (run ${runId})`);
+      throw new Stop(refuse);
+    }
+    candidates += 1;
+    let run;
+    try {
+      run = await readOrRefuse(github, `/repos/${repository}/actions/runs/${artifact.workflow_run.id}`, refuse);
+    } catch {
+      return false;
+    }
+    const started = timeOf(run.run_started_at);
+    const trusted =
+      run.id === artifact.workflow_run.id &&
+      run.repository?.full_name === repository &&
+      run.path === reconcilerWorkflow &&
+      run.head_branch === defaultBranch &&
+      RECONCILER_EVENTS.has(run.event) &&
+      (run.event === "schedule" || isListedHuman(run.actor)) &&
+      SHA.test(run.head_sha ?? "") &&
+      Number.isInteger(run.run_attempt) &&
+      run.run_attempt >= 1 &&
+      (run.run_attempt === 1 || isListedHuman(run.triggering_actor)) &&
+      started <= ownStarted &&
+      ownStarted <= started + WINDOW_MS;
+    if (!trusted) {
+      return false;
+    }
+    await requireOnDefaultBranch(github, repository, defaultBranch, run.head_sha, refuse);
+    return true;
+  };
+  const listingPath = `/repos/${repository}/actions/artifacts?name=${name}&per_page=100`;
+  for (let read = 0; read < 2; read += 1) {
+    if (read > 0) {
+      console.log(`check-trusted-trigger: reconciler record: absent, re-read (run ${runId})`);
+      await sleep(RECORD_WAIT_MS);
+    }
+    for (let page = 1; page <= RECORD_MAX_PAGES; page += 1) {
+      const body = await readOrRefuse(github, `${listingPath}&page=${page}`, refuse);
+      if (!Array.isArray(body.artifacts)) {
+        throw new Stop(refuse);
+      }
+      for (const artifact of body.artifacts) {
+        if (await vouches(artifact)) {
+          return;
+        }
+      }
+      if (body.artifacts.length < 100) {
+        break;
+      }
+    }
+  }
+  throw new Stop(refuse);
+}
+
 // app-label: the lanes App applying exactly that label to an issue starts the
 // lane; the label alone admits the issue. Any other App event stops. The
 // re-runner is the App itself or a listed human.
@@ -530,6 +653,8 @@ export async function evaluateTrigger({
   requireKind = "",
   appLabel = "",
   runId = "",
+  runAttempt = "",
+  reconcilerWorkflow = "",
   dispatchedWorkflow = "",
   github,
   sleep = defaultSleep,
@@ -611,7 +736,11 @@ export async function evaluateTrigger({
     if (!actors.every((actor) => isListed(ids, actor))) {
       throw new Stop("untrusted-actor");
     }
-    checkRerunnerLogin(list, deniedIds, triggeringActor);
+    if (reconcilerWorkflow !== "" && String(triggeringActor).toLowerCase() === "github-actions[bot]") {
+      await verifyReconcilerRerun({ github, repository, event, list, deniedIds, reconcilerWorkflow, runId, runAttempt, sleep });
+    } else {
+      checkRerunnerLogin(list, deniedIds, triggeringActor);
+    }
   }
   if (!isListed(ids, pull.user)) {
     throw new Stop("untrusted-author");
@@ -673,6 +802,8 @@ export async function main({ env = process.env, github, sleep = defaultSleep } =
       appLabel: env.APP_LABEL ?? "",
       dispatchedWorkflow: env.DISPATCHED_WORKFLOW ?? "",
       runId: env.GITHUB_RUN_ID ?? "",
+      runAttempt: env.GITHUB_RUN_ATTEMPT ?? "",
+      reconcilerWorkflow: env.RECONCILER_WORKFLOW ?? "",
       github: github ?? createGitHub({ token: env.GITHUB_TOKEN, apiUrl: env.GITHUB_API_URL }),
       sleep,
     });