Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
name: CI

# Linux clippy/test/coverage/TUI also run on AWS CodeBuild when
# vars.AWS_CODEBUILD_ROLE_ARN is set. Status checks: cortex-cli-gha-x64
# and cortex-cli-gha-arm64. See deploy/aws/codebuild/README.md.
# Jobs in this workflow stay required and unchanged.

on:
push:
branches: [main]
Expand Down
172 changes: 172 additions & 0 deletions .github/workflows/codebuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# AWS CodeBuild CI for CortexLM/cli (Linux x64 + arm64).
# Marker: CLI_CODEBUILD_CI_READY
#
# Assumes a dedicated IAM role via GitHub OIDC. No long-lived AWS keys.
# Role ARN and region come from repository *variables*, not secrets.
# See deploy/aws/codebuild/README.md for the one-time admin steps.
#
# Does not replace ci.yml / release.yml / publish-r2.yml / homebrew.yml /
# winget.yml / version-bump.yml / test-stability.yml.

name: CodeBuild CI

"on":
push:
branches: [main]
# Unprivileged wiring only. StartBuild is skipped for this event because
# the workflow file would come from the unapproved head.
pull_request:
branches: [main]
# Privileged StartBuild. Workflow YAML and OIDC come from main.
pull_request_target:
branches: [main]
workflow_dispatch:

concurrency:
group: codebuild-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read
id-token: write
statuses: write

jobs:
wiring:
name: CodeBuild wiring
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
enabled: ${{ steps.gate.outputs.enabled }}
same_repo: ${{ steps.gate.outputs.same_repo }}
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
# pull_request must validate the PR tree (github.sha). pin the base
# only for pull_request_target, which must not execute unapproved code.
ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.base.sha || github.sha }}
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: python -m pip install -r scripts/readiness/requirements.txt
- name: Validate CodeBuild assets
run: python -B -m unittest discover -s scripts/readiness -p test_codebuild.py -v
- name: Decide whether StartBuild is configured
id: gate
env:
ROLE_ARN: ${{ vars.AWS_CODEBUILD_ROLE_ARN }}
PR_HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
run: |
same_repo=false
if [ "$PR_HEAD_REPO" = "${{ github.repository }}" ]; then
same_repo=true
fi
echo "same_repo=$same_repo" >> "$GITHUB_OUTPUT"
if [ -z "$ROLE_ARN" ]; then
echo "enabled=false" >> "$GITHUB_OUTPUT"
echo "AWS_CODEBUILD_ROLE_ARN is unset. Skipping StartBuild. One-time IAM is in deploy/aws/codebuild/README.md"
exit 0
fi
case "$ROLE_ARN" in
arn:aws:iam::*:role/*) ;;
*)
echo "::error::AWS_CODEBUILD_ROLE_ARN must be an IAM role ARN (set a variable, do not commit it)"
exit 1
;;
esac
echo "enabled=true" >> "$GITHUB_OUTPUT"

codebuild:
name: ${{ matrix.context }}
needs: wiring
if: needs.wiring.outputs.enabled == 'true' && needs.wiring.outputs.same_repo == 'true' && github.event_name != 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
include:
- context: cortex-cli-gha-x64
project_var: AWS_CODEBUILD_PROJECT_X64
project_default: cortex-cli-gha-x64
pr_project_var: AWS_CODEBUILD_PROJECT_X64_PR
pr_project_default: cortex-cli-gha-x64-pr
- context: cortex-cli-gha-arm64
project_var: AWS_CODEBUILD_PROJECT_ARM64
project_default: cortex-cli-gha-arm64
pr_project_var: AWS_CODEBUILD_PROJECT_ARM64_PR
pr_project_default: cortex-cli-gha-arm64-pr
env:
STATUS_CONTEXT: ${{ matrix.context }}
QUALITY_BASE: ${{ github.event.pull_request.base.sha || github.event.before }}
CORTEX_GITHUB_REPOSITORY: ${{ github.repository }}
steps:
- name: Checkout trusted buildspec revision
uses: actions/checkout@v7
with:
persist-credentials: false
# pull_request_target already runs on the base. Pin the ref so a
# later step cannot silently check out the unapproved head.
ref: ${{ github.event.pull_request.base.sha || github.sha }}
- name: Resolve head SHA
id: rev
run: echo "sha=${{ github.event.pull_request.head.sha || github.sha }}" >> "$GITHUB_OUTPUT"
- name: Post pending status
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SHA: ${{ steps.rev.outputs.sha }}
run: |
gh api "repos/${{ github.repository }}/statuses/${SHA}" \
--field state=pending \
--field context="${STATUS_CONTEXT}" \
--field description="AWS CodeBuild starting" \
--field target_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.AWS_CODEBUILD_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION || 'us-east-1' }}
role-session-name: ${{ matrix.context }}
- name: Read trusted buildspec
id: spec
run: |
test -f deploy/aws/codebuild/buildspec-ci.yml
{
echo "yaml<<ENDOFFILE"
cat deploy/aws/codebuild/buildspec-ci.yml
echo "ENDOFFILE"
} >> "$GITHUB_OUTPUT"
- name: Run CodeBuild
uses: aws-actions/aws-codebuild-run-build@v1
with:
# Only push/workflow_dispatch on main use the S3-cached projects.
# pull_request_target always uses *-pr (logs-only, NO_CACHE).
project-name: ${{ (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && (vars[matrix.project_var] || matrix.project_default) || (vars[matrix.pr_project_var] || matrix.pr_project_default) }}
disable-source-override: true
buildspec-override: ${{ steps.spec.outputs.yaml }}
Comment thread
echobt marked this conversation as resolved.
env-vars-for-codebuild: |
QUALITY_BASE,
CORTEX_SOURCE_SHA,
CORTEX_GITHUB_REPOSITORY
env:
CORTEX_SOURCE_SHA: ${{ steps.rev.outputs.sha }}
- name: Post final status
if: always()
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SHA: ${{ steps.rev.outputs.sha }}
OUTCOME: ${{ job.status }}
run: |
if [ "$OUTCOME" = "success" ]; then
state=success
desc="AWS CodeBuild passed"
else
state=failure
desc="AWS CodeBuild failed"
fi
gh api "repos/${{ github.repository }}/statuses/${SHA}" \
--field state="$state" \
--field context="${STATUS_CONTEXT}" \
--field description="$desc" \
--field target_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Working branch: **`main`**. Version bumps land on `main` via PR; `.github/workfl
| `docs/` | User and plugin docs |
| `.rules/` | Engineering rules (security, errors, TUI, tests, …) |
| `scripts/` | Version bump / consistency / release helpers |
| `deploy/aws/codebuild/` | Public-safe CodeBuild buildspecs and OIDC IAM ([README](deploy/aws/codebuild/README.md)) |

## Non-negotiables

Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,8 @@ via [`publish-r2.yml`](.github/workflows/publish-r2.yml).

This repository does not invent cloud accounts. The secret *names* CI expects are
listed in [docs/CI_SECRETS.md](./docs/CI_SECRETS.md). Values never go in git.
Linux CI can additionally run on AWS CodeBuild via GitHub OIDC; the one-time
IAM steps are in [deploy/aws/codebuild/README.md](./deploy/aws/codebuild/README.md).

## Contributing

Expand Down
163 changes: 163 additions & 0 deletions deploy/aws/codebuild/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# AWS CodeBuild CI for CortexLM/cli

Public-repo CodeBuild integration for Linux **x64** and **arm64**. GitHub
Actions assumes a dedicated IAM role with **OIDC** (no long-lived AWS keys)
and starts the projects. Each project posts a commit status comparable to
the CortexLM/backend checks `cortex-gha-x64` / `cortex-gha-arm64`.

Marker: `CLI_CODEBUILD_CI_READY`

Existing workflows stay in place: `ci.yml`, `release.yml`, `publish-r2.yml`,
`homebrew.yml`, `winget.yml`, `version-bump.yml`, `test-stability.yml`.
CodeBuild **extends** them. It does not replace R2 publishing, version
bumps, or macOS/Windows release jobs. Staging/prod app deploy remains
unchanged (prod HOLD).

Windows CodeBuild is **out of scope**. Compliance treats Windows CI as
outside the production gate; keep `windows-latest` on GitHub-hosted runners
in `ci.yml` / `release.yml` until a separate follow-up.

Do not commit AWS account IDs, access keys, PATs, or internal hostnames.

## Status checks (branch protection)

After the one-time AWS setup below, add these **required** checks on `main`:

| Context | Push/`main` project (S3 cache) | Pull-request project (no cache) | Arch |
|---------|--------------------------------|----------------------------------|------|
| `cortex-cli-gha-x64` | `cortex-cli-gha-x64` | `cortex-cli-gha-x64-pr` | Linux x86_64 |
| `cortex-cli-gha-arm64` | `cortex-cli-gha-arm64` | `cortex-cli-gha-arm64-pr` | Linux aarch64 |

Unapproved same-repository pull requests are handled with
`pull_request_target` so this workflow file and the OIDC token come from
`main`, not from the unapproved head. Those runs start the `*-pr` projects
only (logs-only service role, `NO_CACHE`). They cannot read, write, or
delete the shared cargo cache. The runner checks out the PR **base** for
`buildspec-ci.yml` and passes the head SHA into CodeBuild as
`CORTEX_SOURCE_SHA`. OIDC trust is only
`repo:CortexLM/cli:ref:refs/heads/main` plus
`job_workflow_ref` for `.github/workflows/codebuild.yml` on `main`.
Untrusted `pull_request` workflows cannot assume the role.

Keep the existing `ci.yml` checks (`Format`, `Clippy`, `Test`, `TUI checks`,
`Security Audit`, `Source and dependency policy`, `Changed-line coverage`,
`CLI Version and Distribution`, `CI Success`). Do not remove them in this
change. After CodeBuild is required and stable, a later PR can slim the
duplicate GitHub-hosted Linux cargo jobs.

Same-repo PRs start the `*-pr` projects via `pull_request_target`. Pushes
to `main` start the cached projects. Fork PRs keep using GitHub-hosted
`ci.yml` only (StartBuild is skipped when the head repo is not this
repository).

## Prefer existing org projects?

If this AWS account already hosts backend projects `cortex-gha-x64` /
`cortex-gha-arm64`, **reuse the GitHub OIDC provider** and the account, not
the projects. A CodeBuild project has one source/buildspec; do not point
backend projects at this public CLI repo. Create dedicated
`cortex-cli-gha-*` projects. Override names only via GitHub **variables**
if an admin already created equivalent CLI projects.

## One-time admin setup

### 1. Reuse or create the GitHub OIDC provider

In the AWS account that already runs CortexLM/backend CodeBuild (or a new
account dedicated to public CLI CI):

1. IAM → Identity providers → `token.actions.githubusercontent.com`.
2. If it exists, **do not recreate it**. Continue to the role.
3. If it does not exist, create it:
- Provider URL: `https://token.actions.githubusercontent.com`
- Audience: `sts.amazonaws.com`
- Or pass `CreateGithubOidcProvider=true` to the stack below.

### 2. Deploy the stack (recommended)

From a workstation that can assume an admin role (never from this repo's
CI, and never with keys committed here):

```bash
aws cloudformation deploy \
--stack-name cortex-cli-codebuild \
--template-file deploy/aws/codebuild/cloudformation.yaml \
--capabilities CAPABILITY_NAMED_IAM \
--parameter-overrides \
GitHubOrgRepo=CortexLM/cli \
ProjectNameX64=cortex-cli-gha-x64 \
ProjectNameArm64=cortex-cli-gha-arm64 \
GhaRoleName=cortex-cli-codebuild-gha \
CreateGithubOidcProvider=false
```

Copy the `GithubActionsRoleArn` output. It contains the account ID; store
it as a GitHub **variable**, not in git.

### 3. Manual IAM if you do not use CloudFormation

1. Create role `cortex-cli-codebuild-gha`.
2. Trust policy: `iam-trust-policy.json` with `ACCOUNT_ID` replaced at
deploy time. The subject must be only
`repo:CortexLM/cli:ref:refs/heads/main`. Also require
`job_workflow_ref` `CortexLM/cli/.github/workflows/codebuild.yml@refs/heads/main`.
Do not trust `repo:CortexLM/cli:pull_request` — that would let an
unapproved `pull_request` workflow assume the role.
3. Permissions: `iam-gha-permissions.json` with `ACCOUNT_ID` and `REGION`
replaced. Actions are only `codebuild:StartBuild`,
`codebuild:BatchGetBuilds`, and `logs:GetLogEvents` on the two CLI
projects.
4. Create cached projects `cortex-cli-gha-x64` / `cortex-cli-gha-arm64`
(`BUILD_GENERAL1_LARGE`, source **NO_SOURCE**, S3 cache, 90 minute
timeout) and matching `*-pr` projects with **NO_CACHE**.
5. Cached-project service role: CloudWatch Logs plus read/write on the
cache bucket. PR-project service role: CloudWatch Logs **only**. No
deploy, no R2, no production secrets.

### 4. GitHub repository variables (not secrets)

On `CortexLM/cli` → Settings → Secrets and variables → Actions → Variables:

| Variable | Value |
|----------|--------|
| `AWS_CODEBUILD_ROLE_ARN` | `GithubActionsRoleArn` stack output |
| `AWS_REGION` | Region of the stack (default in the workflow is `us-east-1`) |
| `AWS_CODEBUILD_PROJECT_X64` | Optional override; default `cortex-cli-gha-x64` |
| `AWS_CODEBUILD_PROJECT_ARM64` | Optional override; default `cortex-cli-gha-arm64` |

Do **not** add `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`. Do not put
staging or production app secrets on these projects.

Until `AWS_CODEBUILD_ROLE_ARN` is set, `.github/workflows/codebuild.yml`
validates the in-repo assets and **skips** StartBuild. It does not post a
green `cortex-cli-gha-*` status for that skip (no mock-success).

### 5. Require the checks

Branch protection / ruleset on `main`: require
`cortex-cli-gha-x64` and `cortex-cli-gha-arm64` in addition to the
existing `ci.yml` jobs. Require these only after a successful StartBuild
has been observed on a test PR.

## What CodeBuild runs

`buildspec-ci.yml` clones the public `CortexLM/cli` commit over HTTPS
(no PAT) and runs `run-ci.sh`:

- `cargo fmt --all -- --check`
- `./scripts/clippy.sh`
- `./scripts/check-cli-version.sh`
- `python3 scripts/readiness/tests.py`
- `cargo test --locked --workspace --doc`
- `python3 scripts/readiness/schema.py`
- `cargo build --locked -p cortex-cli -p cortex-app-server`
- `python3 scripts/readiness/qa.py`
- headless TUI / snapshot packages (same set as `ci.yml`)
- changed-line coverage against the real PR base SHA

Cargo registry, git, rustup, and `target/` are cached in S3.

## Follow-up (Windows)

Not in this change. If Windows CodeBuild is added later, use a separate
project and a non-required check. Do not block production on it.
Loading
Loading