improvement(ci): attribute independent acceptance jobs and reclaim closed PR caches

Author: waleedlatif1

.github/CI.md

@@ -28,11 +28,15 @@ Database integration uses 4-vCPU Blacksmith runners with sticky dependency disks
 
 GitHub runners install dependencies directly rather than transferring Bun and `node_modules` archives. In a same-source paired comparison, those restores took 26.2 seconds before an 82-millisecond install, while a clean install took 12.42 seconds. This is one paired observation; retain the policy only while repeated CI timings support it. Blacksmith keeps its sticky dependency disks. GitHub Turbo output caches remain enabled for jobs that request them, with a commit-specific key and a task/event/fork-scoped restore prefix, so a rerun can reuse an exact entry without creating another archive. Push jobs never restore pull-request entries.
 
+Closing a pull request deletes only its GitHub merge-ref caches. The cleanup runs the base branch's workflow without checking out PR code; it never selects a branch-wide or default-branch cache. Blacksmith sticky disks retain their separate lifecycle.
+
 Memory-heavy builds and browser jobs retain appropriately sized Blacksmith runners. Electron uses 4-vCPU Blacksmith: both shards passed within 14 seconds of the 8-vCPU control; the free GitHub rest shard was slower and its lease run stalled.
 
 Every HTTP and browser group consumes the same production build in an isolated job/database. Electron also uses that build and skips its development-route warm-up. Local HTTP and Electron development runs retain their dev-server behavior. Build inputs exclude test-only files so a harness edit can reuse an unchanged production build.
 
-Desktop inbox protocol, assistant actions UI, and organization search UI run as three parallel HTTP jobs with isolated databases, apps, and reports. They retain one acceptance task boundary, so changing any of their harnesses selects all three jobs. Desktop inbox needs no browser. The UI jobs and browser extension install Chromium while their app starts, waiting for both before executing assertions; browser-install failures still fail the job and preserve the install log. No acceptance assertions are removed by the split.
+Desktop inbox protocol, assistant actions UI, and organization search UI run as three parallel HTTP jobs with isolated databases, apps, and reports. Each harness owns its acceptance task, so an edit selects its own job. The inbox protocol task depends on server and realtime code; the browser tasks also depend on workspace UI. Desktop inbox needs no browser. The UI jobs and browser extension install Chromium while their app starts, waiting for both before executing assertions; browser-install failures still fail the job and preserve the install log. No acceptance assertions are removed by the split.
+
+Live Electron acceptance tracks its live-app suite, fixtures, and Playwright configuration separately from shell-only smoke tests. Changing a shell-only smoke case runs the dedicated desktop workflow without starting the app's production build and live lease checks.
 
 Browser extension acceptance runs native interaction and access-revalidation groups in parallel jobs with isolated databases, apps, browser profiles, and reports. Both retain the same task boundary, so an affected extension harness selects both groups. Native streaming and first-open capture checks retain real timers. Access checks install Playwright's clock before the app mounts: advancing production polling timers preserves real HTTP and database authorization checks without waiting through each polling interval. Capture timestamps originate in Chromium's isolated world, so the harness restores wall time after each advance. The local `browser-extension` group still runs the complete suite; live-provider runs require this complete mode and retain native timing throughout.
 

.github/ci-suites.json

@@ -18,11 +18,11 @@
     },
     {
       "group": "assistant-actions",
-      "task": "ci:e2e-desktop-inbox"
+      "task": "ci:e2e-assistant-actions"
     },
     {
       "group": "organization-search",
-      "task": "ci:e2e-desktop-inbox"
+      "task": "ci:e2e-organization-search"
     },
     {
       "group": "browser-extension-native",

.github/scripts/ci-plan.gql

@@ -3,6 +3,7 @@ query($base: String!, $head: String!) {
   affectedTasks(base: $base, head: $head, tasks: [
     "test", "build", "type-check", "ci:integration", "ci:desktop-live",
     "ci:e2e-scim", "ci:e2e-cli", "ci:e2e-stop-after", "ci:e2e-desktop-inbox",
+    "ci:e2e-assistant-actions", "ci:e2e-organization-search",
     "ci:e2e-browser-extension", "ci:e2e-project-files", "ci:e2e-mobile", "ci:e2e-seo"
   ]) {
     items {

.github/workflows/cache-cleanup.yml

@@ -0,0 +1,31 @@
+name: cache cleanup
+
+on:
+  pull_request_target:
+    types: [closed]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: cache-cleanup-${{ github.event.pull_request.number }}
+  cancel-in-progress: false
+
+jobs:
+  cleanup:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 5
+    permissions:
+      actions: write
+      pull-requests: read
+    steps:
+      - name: Remove caches scoped to the closed pull request
+        env:
+          GH_TOKEN: ${{ github.token }}
+          GH_REPO: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+        run: |
+          if [ "$(gh pr view "$PR_NUMBER" --json state --jq .state)" = OPEN ]; then
+            exit 0
+          fi
+          gh cache delete --all --ref "refs/pull/${PR_NUMBER}/merge" --succeed-on-no-caches

apps/sim/package.json

@@ -51,6 +51,8 @@
     "ci:e2e-cli": "bash ../../.github/scripts/http-e2e.sh cli",
     "ci:e2e-stop-after": "bash ../../.github/scripts/http-e2e.sh stop-after",
     "ci:e2e-desktop-inbox": "bash ../../.github/scripts/http-e2e.sh desktop-inbox",
+    "ci:e2e-assistant-actions": "bash ../../.github/scripts/http-e2e.sh assistant-actions",
+    "ci:e2e-organization-search": "bash ../../.github/scripts/http-e2e.sh organization-search",
     "ci:e2e-project-files": "bash ../../.github/scripts/http-e2e.sh project-files",
     "ci:e2e-mobile": "bash ../../.github/scripts/http-e2e.sh mobile",
     "ci:e2e-seo": "bash ../../.github/scripts/http-e2e.sh seo",

apps/sim/turbo.json

@@ -119,12 +119,18 @@
       "cache": false
     },
     "ci:e2e-desktop-inbox": {
+      "dependsOn": ["ci:server", "@sim/realtime#ci:runtime"],
+      "inputs": ["scripts/test-desktop-inbox-e2e.ts"],
+      "cache": false
+    },
+    "ci:e2e-assistant-actions": {
       "dependsOn": ["ci:server", "ci:ui", "@sim/realtime#ci:runtime"],
-      "inputs": [
-        "scripts/test-desktop-inbox-e2e.ts",
-        "scripts/test-assistant-actions-e2e.ts",
-        "scripts/test-organization-search-e2e.ts"
-      ],
+      "inputs": ["scripts/test-assistant-actions-e2e.ts"],
+      "cache": false
+    },
+    "ci:e2e-organization-search": {
+      "dependsOn": ["ci:server", "ci:ui", "@sim/realtime#ci:runtime"],
+      "inputs": ["scripts/test-organization-search-e2e.ts"],
       "cache": false
     },
     "ci:e2e-project-files": {
@@ -145,7 +151,8 @@
     "ci:desktop-live": {
       "dependsOn": ["ci:server", "ci:ui", "@sim/desktop#ci:runtime", "@sim/realtime#ci:runtime"],
       "inputs": [
-        "$TURBO_ROOT$/apps/desktop/e2e/**",
+        "$TURBO_ROOT$/apps/desktop/e2e/desktop-tools-live-sim.spec.ts",
+        "$TURBO_ROOT$/apps/desktop/e2e/fixtures/**",
         "$TURBO_ROOT$/apps/desktop/playwright.config.ts"
       ],
       "cache": false

scripts/ci-changes.test.ts

@@ -149,7 +149,6 @@ describe('CI task attribution through real Turborepo', () => {
       'assistant-actions',
       'browser-extension-access',
       'browser-extension-native',
-      'desktop-inbox',
       'mobile',
       'organization-search',
       'seo',
@@ -222,13 +221,13 @@ describe('CI task attribution through real Turborepo', () => {
     expect(plan.flags.integration).toBe(true)
   })
   it.each([
-    'test-desktop-inbox-e2e.ts',
-    'test-assistant-actions-e2e.ts',
-    'test-organization-search-e2e.ts',
-  ])('selects every isolated organization acceptance job for %s', (file) => {
+    ['test-desktop-inbox-e2e.ts', 'desktop-inbox'],
+    ['test-assistant-actions-e2e.ts', 'assistant-actions'],
+    ['test-organization-search-e2e.ts', 'organization-search'],
+  ])('selects the owning organization acceptance job for %s', (file, group) => {
     const plan = classifyFiles([`apps/sim/scripts/${file}`])
     expect(plan.fallback).toBeUndefined()
-    expect(groups(plan)).toEqual(['assistant-actions', 'desktop-inbox', 'organization-search'])
+    expect(groups(plan)).toEqual([group])
     expect(plan.flags).toMatchObject({
       http: true,
       production: false,
@@ -349,6 +348,26 @@ describe('CI task attribution through real Turborepo', () => {
     })
     expect(plan.testPackages).toEqual(['@sim/desktop'])
   })
+  it('keeps shell-only smoke edits out of the live Sim execution gate', () => {
+    const plan = classifyFiles(['apps/desktop/e2e/browser-tools.spec.ts'])
+    expect(plan.fallback).toBeUndefined()
+    expect(groups(plan)).toEqual([])
+    expect(plan.flags).toMatchObject({
+      desktop_live: false,
+      e2e_build: false,
+      integration: false,
+      build: false,
+    })
+  })
+  it.each([
+    'apps/desktop/e2e/desktop-tools-live-sim.spec.ts',
+    'apps/desktop/e2e/fixtures/live-sim.ts',
+    'apps/desktop/playwright.config.ts',
+  ])('keeps live execution coverage when its harness changes: %s', (file) => {
+    const plan = classifyFiles([file])
+    expect(plan.fallback).toBeUndefined()
+    expect(plan.flags).toMatchObject({ desktop_live: true, e2e_build: true })
+  })
   it('runs everything if the real task query cannot parse configuration', () => {
     const { repo, base } = fixture()
     write(repo, 'turbo.json', '{invalid')