improvement(ci): partition extension acceptance and trial faster integration runners

Author: waleedlatif1

.github/CI.md

@@ -24,7 +24,7 @@ SEO depends on workspace UI and server code: landing pages reuse workspace prici
 
 Affected type-checks follow the package graph too; repository audits and lint run whenever the checks workflow runs. Workspace-only unit jobs use a free GitHub runner and do not start the app test shards.
 
-Database integration and lightweight HTTP suites run on free standard GitHub runners with provider-specific dependency setup. A same-source integration comparison of 16 jobs per provider (32 total) had a slowest shard of 6m 04s on 2-vCPU Blacksmith versus 6m 55s on free GitHub, with identical coverage. GitHub trades the observed 51 seconds for zero runner compute cost. This is one paired sample, not a long-run percentile estimate. These explicitly selected GitHub jobs do not follow `CI_PROVIDER`.
+Database integration trials 4-vCPU Blacksmith runners with sticky dependency disks, following `CI_PROVIDER` and falling back to standard GitHub runners with matching setup. A prior same-source comparison of 16 jobs per provider (32 total) had a slowest shard of 6m 04s on 2-vCPU Blacksmith versus 6m 55s on GitHub, with identical coverage. This is one paired sample, not a long-run percentile estimate; the 4-vCPU trial needs its own complete coverage and CI-gate comparison. Lightweight HTTP suites retain free standard GitHub runners and do not follow `CI_PROVIDER`.
 
 GitHub runners install dependencies directly rather than transferring Bun and `node_modules` archives. In a same-source paired comparison, those restores took 26.2 seconds before an 82-millisecond install, while a clean install took 12.42 seconds. This is one paired observation; retain the policy only while repeated CI timings support it. Blacksmith keeps its sticky dependency disks. GitHub Turbo output caches remain enabled for jobs that request them, with a commit-specific key and a task/event/fork-scoped restore prefix, so a rerun can reuse an exact entry without creating another archive. Push jobs never restore pull-request entries.
 
@@ -34,7 +34,7 @@ Every HTTP and browser group consumes the same production build in an isolated j
 
 Desktop inbox protocol, assistant actions UI, and organization search UI run as three parallel HTTP jobs with isolated databases, apps, and reports. They retain one acceptance task boundary, so changing any of their harnesses selects all three jobs. Desktop inbox needs no browser. The UI jobs and browser extension install Chromium while their app starts, waiting for both before executing assertions; browser-install failures still fail the job and preserve the install log. No acceptance assertions are removed by the split.
 
-Browser extension acceptance keeps native streaming and first-open capture checks on real timers. Its access-revalidation scenarios then reopen the persistent browser profile and install Playwright's clock before the app mounts. Advancing the production polling timers preserves real HTTP and database authorization checks without waiting through each polling interval. Capture timestamps originate in Chromium's isolated world, so the harness restores wall time after each advance. Live-provider runs retain native timing throughout.
+Browser extension acceptance runs native interaction and access-revalidation groups in parallel jobs with isolated databases, apps, browser profiles, and reports. Both retain the same task boundary, so an affected extension harness selects both groups. Native streaming and first-open capture checks retain real timers. Access checks install Playwright's clock before the app mounts: advancing production polling timers preserves real HTTP and database authorization checks without waiting through each polling interval. Capture timestamps originate in Chromium's isolated world, so the harness restores wall time after each advance. The local `browser-extension` group still runs the complete suite; live-provider runs require this complete mode and retain native timing throughout.
 
 Integration shard weights come from successful reports for both push and migration provisioning paths. Refresh them with `scripts/update-integration-durations.ts`, then check the new partition against a separate complete run before adopting it. Weights only change scheduling; discovery still owns coverage. The current refresh uses runs [41348786822](https://github.com/simstudioai/sim/actions/runs/41348786822) and [41377145857](https://github.com/simstudioai/sim/actions/runs/41377145857), with [41377810718](https://github.com/simstudioai/sim/actions/runs/41377810718) held out for validation.
 

.github/ci-suites.json

@@ -25,7 +25,11 @@
       "task": "ci:e2e-desktop-inbox"
     },
     {
-      "group": "browser-extension",
+      "group": "browser-extension-native",
+      "task": "ci:e2e-browser-extension"
+    },
+    {
+      "group": "browser-extension-access",
       "task": "ci:e2e-browser-extension"
     },
     {

.github/scripts/http-e2e.sh

@@ -3,7 +3,7 @@
 # CI groups, which reuse the production build from `e2e-build`.
 # The job extracts it into apps/sim/.next before running these suites.
 #
-# Usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox|assistant-actions|organization-search|browser-extension|project-files|mobile|seo>   (run from apps/sim)
+# Usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox|assistant-actions|organization-search|browser-extension|browser-extension-native|browser-extension-access|project-files|mobile|seo>   (run from apps/sim)
 #
 # The job provides DATABASE_URL, BETTER_AUTH_SECRET and ENCRYPTION_KEY; each group sets the rest of
 # its app's environment here. Reports and server logs land in $RUNNER_TEMP/e2e.
@@ -20,7 +20,7 @@
 # telemetry flush runs detached and still writes .next/dev).
 set -euo pipefail
 
-group=${1:?usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox|assistant-actions|organization-search|browser-extension|project-files|mobile|seo>}
+group=${1:?usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox|assistant-actions|organization-search|browser-extension|browser-extension-native|browser-extension-access|project-files|mobile|seo>}
 report_dir="$RUNNER_TEMP/e2e"
 ready_timeout_seconds=600
 mkdir -p "$report_dir"
@@ -264,13 +264,18 @@ case "$group" in
     esac
     ;;
 
-  browser-extension)
+  browser-extension|browser-extension-native|browser-extension-access)
+    extension_group=all
+    case "$group" in
+      browser-extension-native) extension_group=native ;;
+      browser-extension-access) extension_group=access ;;
+    esac
     export REDIS_URL=redis://127.0.0.1:6379
     export NEXT_PUBLIC_FORCE_HOSTED=false
     export KNOWLEDGE_MEMBER_ACCESS=true CREDENTIAL_GROUPS=true ACCESS_CONTROL_ENABLED=true
     export NEXT_PUBLIC_ACCESS_CONTROL_ENABLED=true
     export API_ENCRYPTION_KEY="$ENCRYPTION_KEY"
-    start_browser_install browser-extension chromium
+    start_browser_install "$group" chromium
     extension_identity=$(bun - <<'JS'
       import { generateKeyPairSync, createHash } from 'node:crypto'
       const { publicKey } = generateKeyPairSync('rsa', { modulusLength: 2048 })
@@ -282,11 +287,12 @@ JS
     )
     export SIM_EXTENSION_KEY="${extension_identity%%$'\n'*}"
     export BROWSER_EXTENSION_IDS="${extension_identity#*$'\n'}"
-    start_app browser-extension 3007 'browser extension' record-http-status
+    start_app "$group" 3007 'browser extension' record-http-status
     wait_for_browsers
     EXTENSION_APP_E2E_BASE_URL="$local_app_url" \
     EXTENSION_APP_E2E_DATABASE_URL="$DATABASE_URL" \
-    EXTENSION_APP_E2E_REPORT_PATH="$report_dir/browser-extension/report.json" \
+    EXTENSION_APP_E2E_GROUP="$extension_group" \
+    EXTENSION_APP_E2E_REPORT_PATH="$report_dir/$group/report.json" \
       bun run test:browser-extension:e2e
     ;;
 

.github/workflows/checks.yml

@@ -76,7 +76,7 @@ jobs:
     name: integration (${{ matrix.provision }}, ${{ matrix.shard == 'db' && 'db' || format('{0}/8', matrix.shard) }})
     needs: changes
     if: needs.changes.outputs.integration == 'true'
-    runs-on: ubuntu-24.04
+    runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-24.04' }}
     timeout-minutes: 20
     strategy:
       fail-fast: false
@@ -135,7 +135,7 @@ jobs:
       - name: Setup workspace
         uses: ./.github/actions/setup
         with:
-          provider: github
+          provider: ${{ vars.CI_PROVIDER }}
 
       - name: Provision a fresh database through the supported command
         working-directory: packages/db
@@ -219,7 +219,7 @@ jobs:
           --health-retries 10
       # Desktop inbox, browser extension, and Project Files use Redis.
       redis:
-        image: ${{ (matrix.group == 'desktop-inbox' || matrix.group == 'assistant-actions' || matrix.group == 'organization-search' || matrix.group == 'browser-extension' || matrix.group == 'project-files') && 'redis:8.2-alpine' || '' }}
+        image: ${{ (matrix.group == 'desktop-inbox' || matrix.group == 'assistant-actions' || matrix.group == 'organization-search' || startsWith(matrix.group, 'browser-extension-') || matrix.group == 'project-files') && 'redis:8.2-alpine' || '' }}
         ports:
           - 6379:6379
         options: >-
@@ -298,7 +298,7 @@ jobs:
           name: e2e-reports-${{ matrix.group }}
           path: |
             ${{ runner.temp }}/e2e/**/*.json
-            ${{ runner.temp }}/e2e/browser-extension/**/*.png
+            ${{ runner.temp }}/e2e/browser-extension*/**/*.png
             ${{ runner.temp }}/e2e/*-ui-report.json.artifacts/*.png
           if-no-files-found: ignore
           retention-days: 7

.github/workflows/ci.yml

@@ -8,8 +8,8 @@ name: ci
 # A repo variable, not a committed value: during a Blacksmith outage there is no
 # working CI to merge a switchover through. Only unset/'blacksmith' selects
 # Blacksmith; anything unrecognized selects GitHub so a typo can't queue jobs
-# against the provider you're escaping. Paid build/browser jobs and their cache actions share
-# this predicate. Task planning, the aggregate gate, HTTP acceptance and integration tests use free GitHub runners.
+# against the provider you're escaping. Integration, build and browser jobs and their cache
+# actions share this predicate. Task planning, the aggregate gate and HTTP acceptance use free GitHub runners.
 #
 # GitHub mode is break-glass, not a peer — cold layers, slower runs. The app image
 # is the one job on a paid larger runner: next build needs ~32 GB and OOM-kills

scripts/ci-changes.test.ts

@@ -147,7 +147,8 @@ describe('CI task attribution through real Turborepo', () => {
     expect(plan.fallback).toBeUndefined()
     expect(groups(plan)).toEqual([
       'assistant-actions',
-      'browser-extension',
+      'browser-extension-access',
+      'browser-extension-native',
       'desktop-inbox',
       'mobile',
       'organization-search',
@@ -237,7 +238,6 @@ describe('CI task attribution through real Turborepo', () => {
   })
   it.each([
     ['test-mobile-e2e.ts', 'mobile'],
-    ['test-browser-extension-e2e.ts', 'browser-extension'],
     ['test-project-file-sharing-e2e.ts', 'project-files'],
     ['test-scim-e2e.ts', 'scim'],
     ['test-landing-seo-e2e.ts', 'seo'],
@@ -254,12 +254,26 @@ describe('CI task attribution through real Turborepo', () => {
       e2e_build: true,
     })
   })
+  it('selects both isolated extension groups for an extension harness change', () => {
+    const plan = classifyFiles(['apps/sim/scripts/test-browser-extension-e2e.ts'])
+    expect(plan.fallback).toBeUndefined()
+    expect(groups(plan)).toEqual(['browser-extension-access', 'browser-extension-native'])
+    expect(plan.flags).toMatchObject({
+      http: true,
+      production: false,
+      integration: false,
+      test: false,
+      build: false,
+      e2e_build: true,
+    })
+  })
   it('propagates a transitive security dependency to every affected boundary', () => {
     const plan = classifyFiles(['packages/security/src/index.ts'])
     expect(plan.fallback).toBeUndefined()
     expect(groups(plan)).toEqual([
       'assistant-actions',
-      'browser-extension',
+      'browser-extension-access',
+      'browser-extension-native',
       'cli',
       'desktop-inbox',
       'mobile',