-
Notifications
You must be signed in to change notification settings - Fork 0
ci(codebuild): add linux x64/arm64 oidc status checks #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
8248018
ci(codebuild): add oidc linux x64/arm64 status checks
cursoragent c9064a5
fix(codebuild): isolate PR builds from the shared cache
cursoragent 63cc878
fix(codebuild): stop untrusted PR workflows from assuming oidc
cursoragent 37690a8
fix(codebuild): validate pr tree in unprivileged wiring
cursoragent 0a377ed
test(cli): accept interrupted abort frames under llvm-cov
cursoragent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,172 @@ | ||
| # AWS CodeBuild CI for CortexLM/cli (Linux x64 + arm64). | ||
| # Marker: CLI_CODEBUILD_CI_READY | ||
| # | ||
| # Assumes a dedicated IAM role via GitHub OIDC. No long-lived AWS keys. | ||
| # Role ARN and region come from repository *variables*, not secrets. | ||
| # See deploy/aws/codebuild/README.md for the one-time admin steps. | ||
| # | ||
| # Does not replace ci.yml / release.yml / publish-r2.yml / homebrew.yml / | ||
| # winget.yml / version-bump.yml / test-stability.yml. | ||
|
|
||
| name: CodeBuild CI | ||
|
|
||
| "on": | ||
| push: | ||
| branches: [main] | ||
| # Unprivileged wiring only. StartBuild is skipped for this event because | ||
| # the workflow file would come from the unapproved head. | ||
| pull_request: | ||
| branches: [main] | ||
| # Privileged StartBuild. Workflow YAML and OIDC come from main. | ||
| pull_request_target: | ||
| branches: [main] | ||
| workflow_dispatch: | ||
|
|
||
| concurrency: | ||
| group: codebuild-${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| statuses: write | ||
|
|
||
| jobs: | ||
| wiring: | ||
| name: CodeBuild wiring | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| outputs: | ||
| enabled: ${{ steps.gate.outputs.enabled }} | ||
| same_repo: ${{ steps.gate.outputs.same_repo }} | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| with: | ||
| persist-credentials: false | ||
| # pull_request must validate the PR tree (github.sha). pin the base | ||
| # only for pull_request_target, which must not execute unapproved code. | ||
| ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.base.sha || github.sha }} | ||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.12' | ||
| - run: python -m pip install -r scripts/readiness/requirements.txt | ||
| - name: Validate CodeBuild assets | ||
| run: python -B -m unittest discover -s scripts/readiness -p test_codebuild.py -v | ||
| - name: Decide whether StartBuild is configured | ||
| id: gate | ||
| env: | ||
| ROLE_ARN: ${{ vars.AWS_CODEBUILD_ROLE_ARN }} | ||
| PR_HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name || github.repository }} | ||
| run: | | ||
| same_repo=false | ||
| if [ "$PR_HEAD_REPO" = "${{ github.repository }}" ]; then | ||
| same_repo=true | ||
| fi | ||
| echo "same_repo=$same_repo" >> "$GITHUB_OUTPUT" | ||
| if [ -z "$ROLE_ARN" ]; then | ||
| echo "enabled=false" >> "$GITHUB_OUTPUT" | ||
| echo "AWS_CODEBUILD_ROLE_ARN is unset. Skipping StartBuild. One-time IAM is in deploy/aws/codebuild/README.md" | ||
| exit 0 | ||
| fi | ||
| case "$ROLE_ARN" in | ||
| arn:aws:iam::*:role/*) ;; | ||
| *) | ||
| echo "::error::AWS_CODEBUILD_ROLE_ARN must be an IAM role ARN (set a variable, do not commit it)" | ||
| exit 1 | ||
| ;; | ||
| esac | ||
| echo "enabled=true" >> "$GITHUB_OUTPUT" | ||
|
|
||
| codebuild: | ||
| name: ${{ matrix.context }} | ||
| needs: wiring | ||
| if: needs.wiring.outputs.enabled == 'true' && needs.wiring.outputs.same_repo == 'true' && github.event_name != 'pull_request' | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 120 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - context: cortex-cli-gha-x64 | ||
| project_var: AWS_CODEBUILD_PROJECT_X64 | ||
| project_default: cortex-cli-gha-x64 | ||
| pr_project_var: AWS_CODEBUILD_PROJECT_X64_PR | ||
| pr_project_default: cortex-cli-gha-x64-pr | ||
| - context: cortex-cli-gha-arm64 | ||
| project_var: AWS_CODEBUILD_PROJECT_ARM64 | ||
| project_default: cortex-cli-gha-arm64 | ||
| pr_project_var: AWS_CODEBUILD_PROJECT_ARM64_PR | ||
| pr_project_default: cortex-cli-gha-arm64-pr | ||
| env: | ||
| STATUS_CONTEXT: ${{ matrix.context }} | ||
| QUALITY_BASE: ${{ github.event.pull_request.base.sha || github.event.before }} | ||
| CORTEX_GITHUB_REPOSITORY: ${{ github.repository }} | ||
| steps: | ||
| - name: Checkout trusted buildspec revision | ||
| uses: actions/checkout@v7 | ||
| with: | ||
| persist-credentials: false | ||
| # pull_request_target already runs on the base. Pin the ref so a | ||
| # later step cannot silently check out the unapproved head. | ||
| ref: ${{ github.event.pull_request.base.sha || github.sha }} | ||
| - name: Resolve head SHA | ||
| id: rev | ||
| run: echo "sha=${{ github.event.pull_request.head.sha || github.sha }}" >> "$GITHUB_OUTPUT" | ||
| - name: Post pending status | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| SHA: ${{ steps.rev.outputs.sha }} | ||
| run: | | ||
| gh api "repos/${{ github.repository }}/statuses/${SHA}" \ | ||
| --field state=pending \ | ||
| --field context="${STATUS_CONTEXT}" \ | ||
| --field description="AWS CodeBuild starting" \ | ||
| --field target_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" | ||
| - name: Configure AWS credentials (OIDC) | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| role-to-assume: ${{ vars.AWS_CODEBUILD_ROLE_ARN }} | ||
| aws-region: ${{ vars.AWS_REGION || 'us-east-1' }} | ||
| role-session-name: ${{ matrix.context }} | ||
| - name: Read trusted buildspec | ||
| id: spec | ||
| run: | | ||
| test -f deploy/aws/codebuild/buildspec-ci.yml | ||
| { | ||
| echo "yaml<<ENDOFFILE" | ||
| cat deploy/aws/codebuild/buildspec-ci.yml | ||
| echo "ENDOFFILE" | ||
| } >> "$GITHUB_OUTPUT" | ||
| - name: Run CodeBuild | ||
| uses: aws-actions/aws-codebuild-run-build@v1 | ||
| with: | ||
| # Only push/workflow_dispatch on main use the S3-cached projects. | ||
| # pull_request_target always uses *-pr (logs-only, NO_CACHE). | ||
| project-name: ${{ (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && (vars[matrix.project_var] || matrix.project_default) || (vars[matrix.pr_project_var] || matrix.pr_project_default) }} | ||
| disable-source-override: true | ||
| buildspec-override: ${{ steps.spec.outputs.yaml }} | ||
| env-vars-for-codebuild: | | ||
| QUALITY_BASE, | ||
| CORTEX_SOURCE_SHA, | ||
| CORTEX_GITHUB_REPOSITORY | ||
| env: | ||
| CORTEX_SOURCE_SHA: ${{ steps.rev.outputs.sha }} | ||
| - name: Post final status | ||
| if: always() | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| SHA: ${{ steps.rev.outputs.sha }} | ||
| OUTCOME: ${{ job.status }} | ||
| run: | | ||
| if [ "$OUTCOME" = "success" ]; then | ||
| state=success | ||
| desc="AWS CodeBuild passed" | ||
| else | ||
| state=failure | ||
| desc="AWS CodeBuild failed" | ||
| fi | ||
| gh api "repos/${{ github.repository }}/statuses/${SHA}" \ | ||
| --field state="$state" \ | ||
| --field context="${STATUS_CONTEXT}" \ | ||
| --field description="$desc" \ | ||
| --field target_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,163 @@ | ||
| # AWS CodeBuild CI for CortexLM/cli | ||
|
|
||
| Public-repo CodeBuild integration for Linux **x64** and **arm64**. GitHub | ||
| Actions assumes a dedicated IAM role with **OIDC** (no long-lived AWS keys) | ||
| and starts the projects. Each project posts a commit status comparable to | ||
| the CortexLM/backend checks `cortex-gha-x64` / `cortex-gha-arm64`. | ||
|
|
||
| Marker: `CLI_CODEBUILD_CI_READY` | ||
|
|
||
| Existing workflows stay in place: `ci.yml`, `release.yml`, `publish-r2.yml`, | ||
| `homebrew.yml`, `winget.yml`, `version-bump.yml`, `test-stability.yml`. | ||
| CodeBuild **extends** them. It does not replace R2 publishing, version | ||
| bumps, or macOS/Windows release jobs. Staging/prod app deploy remains | ||
| unchanged (prod HOLD). | ||
|
|
||
| Windows CodeBuild is **out of scope**. Compliance treats Windows CI as | ||
| outside the production gate; keep `windows-latest` on GitHub-hosted runners | ||
| in `ci.yml` / `release.yml` until a separate follow-up. | ||
|
|
||
| Do not commit AWS account IDs, access keys, PATs, or internal hostnames. | ||
|
|
||
| ## Status checks (branch protection) | ||
|
|
||
| After the one-time AWS setup below, add these **required** checks on `main`: | ||
|
|
||
| | Context | Push/`main` project (S3 cache) | Pull-request project (no cache) | Arch | | ||
| |---------|--------------------------------|----------------------------------|------| | ||
| | `cortex-cli-gha-x64` | `cortex-cli-gha-x64` | `cortex-cli-gha-x64-pr` | Linux x86_64 | | ||
| | `cortex-cli-gha-arm64` | `cortex-cli-gha-arm64` | `cortex-cli-gha-arm64-pr` | Linux aarch64 | | ||
|
|
||
| Unapproved same-repository pull requests are handled with | ||
| `pull_request_target` so this workflow file and the OIDC token come from | ||
| `main`, not from the unapproved head. Those runs start the `*-pr` projects | ||
| only (logs-only service role, `NO_CACHE`). They cannot read, write, or | ||
| delete the shared cargo cache. The runner checks out the PR **base** for | ||
| `buildspec-ci.yml` and passes the head SHA into CodeBuild as | ||
| `CORTEX_SOURCE_SHA`. OIDC trust is only | ||
| `repo:CortexLM/cli:ref:refs/heads/main` plus | ||
| `job_workflow_ref` for `.github/workflows/codebuild.yml` on `main`. | ||
| Untrusted `pull_request` workflows cannot assume the role. | ||
|
|
||
| Keep the existing `ci.yml` checks (`Format`, `Clippy`, `Test`, `TUI checks`, | ||
| `Security Audit`, `Source and dependency policy`, `Changed-line coverage`, | ||
| `CLI Version and Distribution`, `CI Success`). Do not remove them in this | ||
| change. After CodeBuild is required and stable, a later PR can slim the | ||
| duplicate GitHub-hosted Linux cargo jobs. | ||
|
|
||
| Same-repo PRs start the `*-pr` projects via `pull_request_target`. Pushes | ||
| to `main` start the cached projects. Fork PRs keep using GitHub-hosted | ||
| `ci.yml` only (StartBuild is skipped when the head repo is not this | ||
| repository). | ||
|
|
||
| ## Prefer existing org projects? | ||
|
|
||
| If this AWS account already hosts backend projects `cortex-gha-x64` / | ||
| `cortex-gha-arm64`, **reuse the GitHub OIDC provider** and the account, not | ||
| the projects. A CodeBuild project has one source/buildspec; do not point | ||
| backend projects at this public CLI repo. Create dedicated | ||
| `cortex-cli-gha-*` projects. Override names only via GitHub **variables** | ||
| if an admin already created equivalent CLI projects. | ||
|
|
||
| ## One-time admin setup | ||
|
|
||
| ### 1. Reuse or create the GitHub OIDC provider | ||
|
|
||
| In the AWS account that already runs CortexLM/backend CodeBuild (or a new | ||
| account dedicated to public CLI CI): | ||
|
|
||
| 1. IAM → Identity providers → `token.actions.githubusercontent.com`. | ||
| 2. If it exists, **do not recreate it**. Continue to the role. | ||
| 3. If it does not exist, create it: | ||
| - Provider URL: `https://token.actions.githubusercontent.com` | ||
| - Audience: `sts.amazonaws.com` | ||
| - Or pass `CreateGithubOidcProvider=true` to the stack below. | ||
|
|
||
| ### 2. Deploy the stack (recommended) | ||
|
|
||
| From a workstation that can assume an admin role (never from this repo's | ||
| CI, and never with keys committed here): | ||
|
|
||
| ```bash | ||
| aws cloudformation deploy \ | ||
| --stack-name cortex-cli-codebuild \ | ||
| --template-file deploy/aws/codebuild/cloudformation.yaml \ | ||
| --capabilities CAPABILITY_NAMED_IAM \ | ||
| --parameter-overrides \ | ||
| GitHubOrgRepo=CortexLM/cli \ | ||
| ProjectNameX64=cortex-cli-gha-x64 \ | ||
| ProjectNameArm64=cortex-cli-gha-arm64 \ | ||
| GhaRoleName=cortex-cli-codebuild-gha \ | ||
| CreateGithubOidcProvider=false | ||
| ``` | ||
|
|
||
| Copy the `GithubActionsRoleArn` output. It contains the account ID; store | ||
| it as a GitHub **variable**, not in git. | ||
|
|
||
| ### 3. Manual IAM if you do not use CloudFormation | ||
|
|
||
| 1. Create role `cortex-cli-codebuild-gha`. | ||
| 2. Trust policy: `iam-trust-policy.json` with `ACCOUNT_ID` replaced at | ||
| deploy time. The subject must be only | ||
| `repo:CortexLM/cli:ref:refs/heads/main`. Also require | ||
| `job_workflow_ref` `CortexLM/cli/.github/workflows/codebuild.yml@refs/heads/main`. | ||
| Do not trust `repo:CortexLM/cli:pull_request` — that would let an | ||
| unapproved `pull_request` workflow assume the role. | ||
| 3. Permissions: `iam-gha-permissions.json` with `ACCOUNT_ID` and `REGION` | ||
| replaced. Actions are only `codebuild:StartBuild`, | ||
| `codebuild:BatchGetBuilds`, and `logs:GetLogEvents` on the two CLI | ||
| projects. | ||
| 4. Create cached projects `cortex-cli-gha-x64` / `cortex-cli-gha-arm64` | ||
| (`BUILD_GENERAL1_LARGE`, source **NO_SOURCE**, S3 cache, 90 minute | ||
| timeout) and matching `*-pr` projects with **NO_CACHE**. | ||
| 5. Cached-project service role: CloudWatch Logs plus read/write on the | ||
| cache bucket. PR-project service role: CloudWatch Logs **only**. No | ||
| deploy, no R2, no production secrets. | ||
|
|
||
| ### 4. GitHub repository variables (not secrets) | ||
|
|
||
| On `CortexLM/cli` → Settings → Secrets and variables → Actions → Variables: | ||
|
|
||
| | Variable | Value | | ||
| |----------|--------| | ||
| | `AWS_CODEBUILD_ROLE_ARN` | `GithubActionsRoleArn` stack output | | ||
| | `AWS_REGION` | Region of the stack (default in the workflow is `us-east-1`) | | ||
| | `AWS_CODEBUILD_PROJECT_X64` | Optional override; default `cortex-cli-gha-x64` | | ||
| | `AWS_CODEBUILD_PROJECT_ARM64` | Optional override; default `cortex-cli-gha-arm64` | | ||
|
|
||
| Do **not** add `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`. Do not put | ||
| staging or production app secrets on these projects. | ||
|
|
||
| Until `AWS_CODEBUILD_ROLE_ARN` is set, `.github/workflows/codebuild.yml` | ||
| validates the in-repo assets and **skips** StartBuild. It does not post a | ||
| green `cortex-cli-gha-*` status for that skip (no mock-success). | ||
|
|
||
| ### 5. Require the checks | ||
|
|
||
| Branch protection / ruleset on `main`: require | ||
| `cortex-cli-gha-x64` and `cortex-cli-gha-arm64` in addition to the | ||
| existing `ci.yml` jobs. Require these only after a successful StartBuild | ||
| has been observed on a test PR. | ||
|
|
||
| ## What CodeBuild runs | ||
|
|
||
| `buildspec-ci.yml` clones the public `CortexLM/cli` commit over HTTPS | ||
| (no PAT) and runs `run-ci.sh`: | ||
|
|
||
| - `cargo fmt --all -- --check` | ||
| - `./scripts/clippy.sh` | ||
| - `./scripts/check-cli-version.sh` | ||
| - `python3 scripts/readiness/tests.py` | ||
| - `cargo test --locked --workspace --doc` | ||
| - `python3 scripts/readiness/schema.py` | ||
| - `cargo build --locked -p cortex-cli -p cortex-app-server` | ||
| - `python3 scripts/readiness/qa.py` | ||
| - headless TUI / snapshot packages (same set as `ci.yml`) | ||
| - changed-line coverage against the real PR base SHA | ||
|
|
||
| Cargo registry, git, rustup, and `target/` are cached in S3. | ||
|
|
||
| ## Follow-up (Windows) | ||
|
|
||
| Not in this change. If Windows CodeBuild is added later, use a separate | ||
| project and a non-required check. Do not block production on it. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.