improvement(ci): target acceptance and reuse validated build caches (#9583)

Select acceptance work from the task graph, separate docs site and platform checks, and preserve complete comparison history for push attribution.

Build docs once and run isolated acceptance groups in parallel after bounded server readiness. Reuse validated content and incremental build inputs with bounded retention.

Validate native source/toolchain fingerprints and restored output integrity, seed trusted branch caches, and balance desktop smoke across four whole-file shards. Bound organization-search diagnostics and reclaim closed-PR merge-ref caches.

Validation: full root tests, all workspace types, lint and 62 audits; real cache invalidation and Git attribution controls; complete hosted docs, integration, organization-search, Electron and packaged smoke coverage; final-head bot reviews.

Author: waleedlatif1

.github/CI.md

@@ -4,6 +4,10 @@ The `changes` job runs Turborepo 2.9.14's `affectedTasks` query before installin
 
 A PR compares its merge base with the checked-out merge commit. Main/staging pushes execute all checks; the parent `ci` workflow skips checks on `dev` pushes, while PRs targeting `dev` still run checks. A manual `checks` run executes all checks unless supplied a full base commit SHA to reproduce a scoped plan. Missing history, an empty diff, unknown files outside the workspace graph, root script/CI changes, or a planner error run every check. Missing selection outputs fail the aggregate gate. The gate also rejects an omitted dependency, any failed/cancelled job, and any skip that its explicit selection flag did not authorize.
 
+Dedicated docs verification uses the same planner, comparing pushes with their previous commit and PRs with their merge base. Site acceptance follows `docs#build` and its transitive package dependencies. The app's docs-consumer task depends on server runtime and that docs build: backend-only changes run the platform consumer against the docs app, while site changes also run corpus and browser checks. Independent SDK and unit-only edits skip those production jobs. The required `docs HTTP` gate verifies explicit site/platform selections before accepting any skip.
+
+Docs production builds run once and publish an artifact to isolated HTTP, corpus, Chromium, and WebKit jobs. Source checks keep their own checkout because generator verification mutates inputs. Content caches use separate lint/source/build namespaces; generation validates every restore, repairs missing declarations, and removes obsolete modules. The Next build cache keeps its supported incremental behavior. Content and Next caches have size limits, and Next cache files expire after seven days; successful content eviction happens after all consumers finish.
+
 | Change | Expensive checks selected |
 | --- | --- |
 | Independent SDK | That workspace's tests and type-check; no app build, services, or browsers |
@@ -28,11 +32,17 @@ Database integration uses 4-vCPU Blacksmith runners with sticky dependency disks
 
 GitHub runners install dependencies directly rather than transferring Bun and `node_modules` archives. In a same-source paired comparison, those restores took 26.2 seconds before an 82-millisecond install, while a clean install took 12.42 seconds. This is one paired observation; retain the policy only while repeated CI timings support it. Blacksmith keeps its sticky dependency disks. GitHub Turbo output caches remain enabled for jobs that request them, with a commit-specific key and a task/event/fork-scoped restore prefix, so a rerun can reuse an exact entry without creating another archive. Push jobs never restore pull-request entries.
 
+Closing a same-repository pull request deletes only its GitHub merge-ref caches after verifying that it remains closed or merged. The regular `pull_request` workflow skips forks and Dependabot because their tokens cannot delete caches; those entries retain GitHub's normal cache eviction. Cleanup checks out no code and never selects a branch-wide or default-branch cache. Blacksmith sticky disks retain their separate lifecycle.
+
 Memory-heavy builds and browser jobs retain appropriately sized Blacksmith runners. Electron uses 4-vCPU Blacksmith: both shards passed within 14 seconds of the 8-vCPU control; the free GitHub rest shard was slower and its lease run stalled.
 
 Every HTTP and browser group consumes the same production build in an isolated job/database. Electron also uses that build and skips its development-route warm-up. Local HTTP and Electron development runs retain their dev-server behavior. Build inputs exclude test-only files so a harness edit can reuse an unchanged production build.
 
-Desktop inbox protocol, assistant actions UI, and organization search UI run as three parallel HTTP jobs with isolated databases, apps, and reports. They retain one acceptance task boundary, so changing any of their harnesses selects all three jobs. Desktop inbox needs no browser. The UI jobs and browser extension install Chromium while their app starts, waiting for both before executing assertions; browser-install failures still fail the job and preserve the install log. No acceptance assertions are removed by the split.
+Desktop inbox protocol, assistant actions UI, and organization search UI run as three parallel HTTP jobs with isolated databases, apps, and reports. Each harness owns its acceptance task, so an edit selects its own job. The inbox protocol task depends on server and realtime code; the browser tasks also depend on workspace UI. Desktop inbox needs no browser. The UI jobs and browser extension install Chromium while their app starts, waiting for both before executing assertions; browser-install failures still fail the job and preserve the install log. No acceptance assertions are removed by the split.
+
+Live Electron acceptance tracks its live-app suite, fixtures, and Playwright configuration separately from shell-only smoke tests. Changing a shell-only smoke case runs the dedicated desktop workflow without starting the app's production build and live lease checks.
+
+Desktop smoke weights also stay outside the desktop runtime task. Editing weights reruns desktop partition coverage without selecting live-app acceptance.
 
 Browser extension acceptance runs native interaction and access-revalidation groups in parallel jobs with isolated databases, apps, browser profiles, and reports. Both retain the same task boundary, so an affected extension harness selects both groups. Native streaming and first-open capture checks retain real timers. Access checks install Playwright's clock before the app mounts: advancing production polling timers preserves real HTTP and database authorization checks without waiting through each polling interval. Capture timestamps originate in Chromium's isolated world, so the harness restores wall time after each advance. The local `browser-extension` group still runs the complete suite; live-provider runs require this complete mode and retain native timing throughout.
 
@@ -53,9 +63,11 @@ The long-import and lease-expiration checks remain: they prove distinct survival
 
 Local planner validation is `bun run test:scripts scripts/ci-changes.test.ts scripts/ci-gate.test.ts`. It executes real Turbo queries in small Git fixtures; no application build or database is required. Full builds, services, browser suites, and Electron checks run on hosted CI.
 
-The separate `desktop-e2e` workflow retains its desktop-specific path filters. Its macOS shell smoke suite runs whole files across three isolated runners, with one worker per runner. `scripts/desktop-e2e.ts` asks Playwright to discover the complete suite, then assigns longer files first to the currently lightest shard using `apps/desktop/e2e-durations.json`. Exact escaped file filters preserve serial fixtures. Unmeasured files receive the median measured weight, deleted files cannot select tests, and invalid or empty selections fail. Normal local runs, packaging smoke and the weekly Electron canary keep their existing commands.
+The separate `desktop-e2e` workflow retains its desktop-specific path filters. Its macOS shell smoke suite runs whole files across four isolated runners, with one worker per runner. The matrix owns the partition count; job names and runner arguments use `strategy.job-total`. `scripts/desktop-e2e.ts` asks Playwright to discover the complete suite, then assigns longer files first to the currently lightest shard using `apps/desktop/e2e-durations.json`. Exact escaped file filters preserve serial fixtures. Unmeasured files receive the median measured weight, deleted files cannot select tests, and invalid or empty selections fail. Normal local runs, packaging smoke and the weekly Electron canary keep their existing commands.
+
+The current weights are median test durations in seconds from runs [41496623284](https://github.com/simstudioai/sim/actions/runs/41496623284), [41497346251](https://github.com/simstudioai/sim/actions/runs/41497346251), and [41502340540](https://github.com/simstudioai/sim/actions/runs/41502340540). Refresh them from complete `desktop-smoke-reports-*` artifacts when the workload changes: sum `tests[].results[].duration` by `spec.file` in each run, take the median across runs, and divide milliseconds by 1,000. The weights steer scheduling only; Playwright discovery owns coverage. Before adopting new weights, compare repeated paired runs and the complete case/status multiset, including skipped cases and retries.
 
-The initial weights are median test durations in seconds from runs [39734295953](https://github.com/simstudioai/sim/actions/runs/39734295953), [39833640524](https://github.com/simstudioai/sim/actions/runs/39833640524), and [39836596108](https://github.com/simstudioai/sim/actions/runs/39836596108). Refresh them from complete `desktop-smoke-reports-*` artifacts when the workload changes: sum `tests[].results[].duration` by `spec.file` in each run, take the median across runs, and divide milliseconds by 1,000. The weights steer scheduling only; Playwright discovery owns coverage. Before adopting new weights, compare repeated paired runs and the complete case/status multiset, including skipped cases and retries.
+Desktop native outputs are keyed by source, build recipes, compiler, SDK, OS, architecture, and Node header bytes. Every build still validates restored checksums, executable modes, universal architectures, and the helper signature; invalid or incomplete outputs rebuild. Native-input pushes to main/staging seed trusted Node 22 and 24 caches without repeating the PR smoke suite. PR cache entries never populate trusted branch caches.
 
 Each shard uploads its coverage/timing report, the complete discovery report, and its deterministic shard plan. `bun run test:scripts scripts/duration-shards.test.ts scripts/desktop-e2e.test.ts` checks partition coverage, unknown files, deterministic ordering, and balancing. The CLI regression executes tiny browser-free Playwright fixtures to prove that imported test factories and paths containing regex characters are selected correctly; it launches no Electron or browser.
 

.github/actions/desktop-native-cache/action.yml

@@ -0,0 +1,16 @@
+name: Cache desktop native outputs
+description: Reuse validated native outputs only for matching source and toolchain inputs
+runs:
+  using: composite
+  steps:
+    - name: Identify native build inputs
+      id: inputs
+      shell: bash
+      working-directory: apps/desktop
+      run: echo "key=$(bun run scripts/native-build-cache.ts)" >> "$GITHUB_OUTPUT"
+    - name: Restore native compilation outputs
+      continue-on-error: true
+      uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
+      with:
+        path: apps/desktop/dist/native
+        key: desktop-native-v1-${{ runner.os }}-${{ steps.inputs.outputs.key }}

.github/actions/docs-content-cache/action.yml

@@ -0,0 +1,68 @@
+name: Docs content cache
+description: Reuse validated incremental documentation compilation without sharing concurrent writer snapshots.
+inputs:
+  provider:
+    description: The CI_PROVIDER repo variable.
+    required: false
+    default: ''
+  namespace:
+    description: A distinct name for each concurrently writing job.
+    required: true
+  generate:
+    description: Generate and validate content after restoring it. Set false only when the following build already runs content:generate.
+    required: false
+    default: 'true'
+  next-cache:
+    description: Also restore the incremental Next build cache.
+    required: false
+    default: 'false'
+runs:
+  using: composite
+  steps:
+    - name: Identify the compiler runtime
+      id: runtime
+      shell: bash
+      env:
+        CACHE_NAMESPACE: ${{ inputs.namespace }}
+        GENERATE: ${{ inputs.generate }}
+        NEXT_CACHE: ${{ inputs.next-cache }}
+      run: |
+        [[ "$CACHE_NAMESPACE" =~ ^[a-z0-9][a-z0-9-]*$ ]]
+        [[ "$GENERATE" == true || "$GENERATE" == false ]]
+        [[ "$NEXT_CACHE" == true || "$NEXT_CACHE" == false ]]
+        toolchain=$(node --version)
+        toolchain="$toolchain-$(bun --version)"
+        echo "toolchain=$toolchain" >> "$GITHUB_OUTPUT"
+        compiler=$(git ls-files -s -- bun.lock package.json patches apps/docs/package.json apps/docs/scripts/compile-content.ts apps/docs/lib apps/docs/components/workflow-preview packages | sha256sum | awk '{ print $1 }')
+        echo "compiler=$compiler" >> "$GITHUB_OUTPUT"
+    - name: Restore incremental content
+      continue-on-error: true
+      uses: ./.github/actions/cache
+      with:
+        provider: ${{ inputs.provider }}
+        key: ${{ github.repository }}-docs-content-v1-${{ inputs.namespace }}-${{ runner.os }}-${{ runner.arch }}-${{ steps.runtime.outputs.toolchain }}-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && format('-fork-{0}', github.event.pull_request.number) || '' }}-${{ steps.runtime.outputs.compiler }}
+        path: ./apps/docs/.content
+    - name: Bound restored content
+      shell: bash
+      run: bash .github/actions/docs-content-cache/prune.sh content
+    - name: Restore incremental Next build
+      if: inputs.next-cache == 'true'
+      continue-on-error: true
+      uses: ./.github/actions/cache
+      with:
+        provider: ${{ inputs.provider }}
+        key: ${{ github.repository }}-docs-next-v1-${{ inputs.namespace }}-${{ runner.os }}-${{ runner.arch }}-${{ steps.runtime.outputs.toolchain }}-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && format('-fork-{0}', github.event.pull_request.number) || '' }}-${{ hashFiles('bun.lock') }}
+        path: ./apps/docs/.next/cache
+    - name: Bound the incremental build cache
+      if: inputs.next-cache == 'true'
+      shell: bash
+      run: bash .github/actions/docs-content-cache/prune.sh next
+    - name: Generate and validate documentation modules
+      id: generate
+      if: inputs.generate == 'true'
+      shell: bash
+      run: bun run --cwd apps/docs content:generate --prune
+    - name: Bound content after an unsuccessful generation
+      if: always() && inputs.generate == 'true' && steps.generate.outcome != 'success'
+      shell: bash
+      run: bash .github/actions/docs-content-cache/prune.sh content

.github/actions/docs-content-cache/prune.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+case ${1:-} in
+  content)
+    cache=apps/docs/.content
+    maximum_kilobytes=524288
+    ;;
+  next)
+    cache=apps/docs/.next/cache
+    maximum_kilobytes=2097152
+    ;;
+  *) echo 'Usage: prune.sh <content|next>' >&2; exit 2 ;;
+esac
+if [ ! -d "$cache" ]; then exit 0; fi
+if [ "$1" = next ]; then find "$cache" -type f -mtime +7 -delete; fi
+kilobytes=$(du -sk "$cache" | awk '{ print $1 }')
+if [ "$kilobytes" -gt "$maximum_kilobytes" ]; then
+  find "$cache" -mindepth 1 -delete
+fi

.github/ci-suites.json

@@ -18,11 +18,11 @@
     },
     {
       "group": "assistant-actions",
-      "task": "ci:e2e-desktop-inbox"
+      "task": "ci:e2e-assistant-actions"
     },
     {
       "group": "organization-search",
-      "task": "ci:e2e-desktop-inbox"
+      "task": "ci:e2e-organization-search"
     },
     {
       "group": "browser-extension-native",

.github/scripts/ci-plan.gql

@@ -3,6 +3,8 @@ query($base: String!, $head: String!) {
   affectedTasks(base: $base, head: $head, tasks: [
     "test", "build", "type-check", "ci:integration", "ci:desktop-live",
     "ci:e2e-scim", "ci:e2e-cli", "ci:e2e-stop-after", "ci:e2e-desktop-inbox",
+    "ci:e2e-assistant-actions", "ci:e2e-organization-search",
+    "ci:e2e-docs-platform",
     "ci:e2e-browser-extension", "ci:e2e-project-files", "ci:e2e-mobile", "ci:e2e-seo"
   ]) {
     items {

.github/workflows/cache-cleanup.yml

@@ -0,0 +1,38 @@
+name: cache cleanup
+
+on:
+  pull_request:
+    types: [closed]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: cache-cleanup-${{ github.event.pull_request.number }}
+  cancel-in-progress: false
+
+jobs:
+  cleanup:
+    if: >-
+      github.event.pull_request.head.repo.full_name == github.repository
+      && github.event.pull_request.user.login != 'dependabot[bot]'
+      && github.actor != 'dependabot[bot]'
+    runs-on: ubuntu-24.04
+    timeout-minutes: 5
+    permissions:
+      actions: write
+      pull-requests: read
+    steps:
+      - name: Remove caches scoped to the closed pull request
+        env:
+          GH_TOKEN: ${{ github.token }}
+          GH_REPO: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+        run: |
+          state=$(gh pr view "$PR_NUMBER" --json state --jq .state)
+          case "$state" in
+            OPEN) exit 0 ;;
+            CLOSED|MERGED) ;;
+            *) echo "::error::Unexpected pull request state: $state"; exit 1 ;;
+          esac
+          gh cache delete --all --ref "refs/pull/${PR_NUMBER}/merge" --succeed-on-no-caches

.github/workflows/checks.yml

@@ -685,7 +685,10 @@ jobs:
           "$RUNNER_TEMP/actionlint" -color -shellcheck= -pyflakes=
 
       - name: Generate documentation modules for import audits
-        run: bun run --cwd apps/docs content:generate
+        uses: ./.github/actions/docs-content-cache
+        with:
+          provider: ${{ vars.CI_PROVIDER }}
+          namespace: lint
 
       # Every zero-argument `check:*` script, run concurrently. The list is derived in
       # scripts/run-audits.ts, which also writes the per-audit timing table to the job
@@ -727,6 +730,10 @@ jobs:
           fi
           echo "✅ Schema and migrations are in sync"
 
+      - name: Bound documentation content cache
+        if: always()
+        run: bash .github/actions/docs-content-cache/prune.sh content
+
   # Type errors in apps/sim's unit tests, ratcheted per file and TS code against a committed
   # baseline (scripts/check-test-types.ts). It type-checks the whole app with its ~3.4k test files:
   # lint's uncached apps/sim type-check alone takes ~35 s on this runner, so it gets its own job