Skip to content

Don't run the benchmarks job for pull requests from forks - #26

Closed
fikovnik wants to merge 1 commit into
mainfrom
ci/guard-self-hosted-against-forks
Closed

fikovnik wants to merge 1 commit into
mainfrom
ci/guard-self-hosted-against-forks

Conversation

@fikovnik

Copy link
Copy Markdown
Member

This repository is public and run-benchmarks uses runs-on: self-hosted, so without a guard a pull request from any fork can execute arbitrary code on our own hardware. GitHub's first-time-contributor approval is the only thing in front of that today, and it is an approval prompt, not a boundary.

if: >-
  github.event_name != 'pull_request' ||
  github.event.pull_request.head.repo.full_name == github.repository

The event test is deliberate rather than defensive noise: push and workflow_dispatch carry no pull_request object, so the usual head.repo.full_name == github.repository on its own evaluates false for them and would quietly stop benchmarking main — a guard that looks correct while disabling the thing it was meant to protect.

Noticed while moving the bench-2 runner from this repository to the PRL-PRG organization, which is what made the exposure explicit: an organization runner group refuses public repositories by default, precisely because of this.

Only run-benchmarks is guarded. build-images and test run on ubuntu-latest, where a fork's code is already contained.

🤖 Generated with Claude Code

This repository is public and `run-benchmarks` uses `runs-on: self-hosted`, so
without a guard a pull request from any fork can execute arbitrary code on our
own hardware. GitHub's first-time-contributor approval is the only thing
standing in front of that today, and it is an approval prompt, not a boundary.

The event test is deliberate rather than defensive noise. `push` and
`workflow_dispatch` carry no `pull_request` object, so the usual
`head.repo.full_name == github.repository` on its own evaluates false for them
and would quietly stop benchmarking main -- a guard that looks correct and
disables the thing it was meant to protect.

Noticed while moving the `bench-2` runner from this repository to the PRL-PRG
organization, which is what made the exposure explicit: an organization runner
group refuses public repositories by default, precisely because of this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fikovnik

Copy link
Copy Markdown
Member Author

Closing: superseded by the org fork-PR approval policy, now `all_external_contributors`.

This guard was never a boundary. For `pull_request` events GitHub runs the workflow from the merge ref, so a fork can delete the `if:` line in the same PR that attacks the runner. With approval required for every external contributor, nothing from a fork runs unapproved, and once approved the fork's code runs regardless of what this file says — so the guard stopped nothing the gate does not already stop, and nothing the gate lets through.

Kept out deliberately rather than forgotten: its only remaining effect would have been to skip the self-hosted benchmark job on approved fork PRs, which buys a separation between "approve CI" and "grant the machine" at the cost of fork contributions never being benchmarked. Uniform benchmark coverage is worth more here.

Runner hardening that did land instead: `bench-2` moved to an org runner group restricted to `rcp` and `r-benchmarks` only (113 repos down to 2).

@fikovnik fikovnik closed this Sep 15, 2026
@fikovnik
fikovnik deleted the ci/guard-self-hosted-against-forks branch September 15, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant