Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
92345ec
Claim the remaining compiler-owned snapshot cycle
samchon Sep 1, 2026
8c0233d
Keep both sides of a failed build audible
samchon Sep 1, 2026
13eb4d9
Bound failed stderr diagnostics explicitly
samchon Sep 1, 2026
55783c6
Wait as long as the caller actually allows
samchon Sep 1, 2026
d935f4d
Prove the missing deadline stays missing
samchon Sep 1, 2026
809c68e
Cross the old deadline in the retry tests
samchon Sep 1, 2026
c320521
Give the Clang build one cache owner
samchon Sep 1, 2026
c5b6421
Hand the verified Clang build to every consumer
samchon Sep 1, 2026
59c6de3
Preserve the Clang executable across artifact transfer
samchon Sep 1, 2026
f9cc69f
Bind every measurement to the route that served it
samchon Sep 1, 2026
ba6a796
Bind route evidence to the builds that provisioned it
samchon Sep 1, 2026
cd82e6b
test(benchmark): harden producer pin matching
samchon Sep 1, 2026
63844ae
test(benchmark): accept exact embedded build pins
samchon Sep 1, 2026
cd28896
Finish the repository topology plane
samchon Sep 1, 2026
23987c4
Measure every topology orientation phase
samchon Sep 1, 2026
5d4fc7d
Trace the real topology orientation pipeline
samchon Sep 1, 2026
95d4444
Make topology evidence portable and noninvasive
samchon Sep 1, 2026
3e90d03
Measure Rust lifecycle p95 against native analysis
samchon Sep 1, 2026
890a559
Isolate and exercise Rust performance sampling
samchon Sep 1, 2026
aa864a3
Use prime caches as the Rust native baseline
samchon Sep 1, 2026
39f40df
Require Rust performance sampling to restore provenance
samchon Sep 1, 2026
184d6be
Test the exact Rust p95 threshold failure
samchon Sep 1, 2026
1c2b1b0
Isolate the invalid Rust sampling contract
samchon Sep 1, 2026
493b010
Pin the invalid Rust sampling error
samchon Sep 1, 2026
f8309a7
Complete the Rust HIR acceptance surface
samchon Sep 1, 2026
243834f
Complete the Clang compilation-universe acceptance
samchon Sep 2, 2026
354ccc9
Advance #74: prove Java compiler acceptance and native baselines
samchon Sep 2, 2026
76cef95
Advance #74: add pinned resident JDT snapshot agreement
samchon Sep 2, 2026
fd94d38
Advance #74: make JDT acceptance transactional across build systems
samchon Sep 2, 2026
313e6bf
Advance #74: prove JDT retry cursors remain transactional
samchon Sep 2, 2026
1221304
Advance #74: make JDT retry coverage deterministic
samchon Sep 2, 2026
3190d8b
Advance #74: pin the complete JDT source archive
samchon Sep 2, 2026
1e71316
Finish #74: pin verified JDT producer tree
samchon Sep 2, 2026
17ef236
Finish #74: execute producer pin regressions
samchon Sep 2, 2026
61eea55
Finish #74: pin deterministic JDT fixtures
samchon Sep 2, 2026
85748d2
Finish #74: pin binding-resolving JDT producer
samchon Sep 2, 2026
e895fcd
Expose Java compiler universe drift
samchon Sep 2, 2026
40ccd0a
Fence Java universe evidence links
samchon Sep 2, 2026
ad2df46
Name immutable Java universe differences
samchon Sep 2, 2026
a9b236c
Keep long Java universe differences visible
samchon Sep 2, 2026
4b3e580
Preserve Unicode in Java universe diagnostics
samchon Sep 2, 2026
e8d157b
Compare Java compiler evidence by invocation
samchon Sep 2, 2026
bc220f7
Expose failed Rust producer fixtures
samchon Sep 2, 2026
7023f74
Preserve Rust producer termination causes
samchon Sep 2, 2026
4362a59
Prioritize Java invocation evidence
samchon Sep 2, 2026
dace4e7
Close #74: Export Java compiler snapshots
samchon Sep 2, 2026
f4ba357
Stabilize process-ID fixture publication
samchon Sep 2, 2026
00114a4
Require complete PID fixture publication
samchon Sep 3, 2026
7f26566
Pin source installs to Git trees
samchon Sep 3, 2026
a35c16b
Reuse stable C and C++ input digests
samchon Sep 3, 2026
0f4a772
Cover unstable C and C++ input reads
samchon Sep 3, 2026
80aa313
Pin the repaired Rust graph producer
samchon Sep 3, 2026
981e5e0
Accept single-slash file URIs
samchon Sep 3, 2026
2d78d61
Repair file URI routing and Rust verification
samchon Sep 3, 2026
f49dc54
fix: avoid C++ no-op corpus rescans
samchon Sep 3, 2026
19a3bd7
fix: keep C++ input watches lifecycle-safe
samchon Sep 3, 2026
e6dfebd
fix: close C++ watcher event gaps
samchon Sep 3, 2026
0d0b8fe
fix: pin repaired Rust graph producer
samchon Sep 3, 2026
71e5684
fix: detect replaced C++ input directories
samchon Sep 3, 2026
c2e211f
test: force C++ directory identity fallback
samchon Sep 3, 2026
e48a684
fix: bound C++ server diagnostics
samchon Sep 3, 2026
220683f
test: target the active C++ watcher
samchon Sep 3, 2026
12829b9
fix: keep C++ no-ops event-driven on POSIX
samchon Sep 3, 2026
5e19241
fix: preserve portable snapshot refreshes
samchon Sep 3, 2026
f8bdcb8
fix: make compiler-owned no-ops event-driven
samchon Sep 3, 2026
cc413e3
fix: pin repaired Rust decorator ownership
samchon Sep 3, 2026
eea801f
feat: add resident Roslyn graph snapshots
samchon Sep 3, 2026
84c14be
test: align C# benchmark provider expectation
samchon Sep 3, 2026
6a72fb1
feat: add resident Kotlin graph snapshots
samchon Sep 4, 2026
4149c7a
fix: complete Kotlin graph acceptance evidence
samchon Sep 4, 2026
470e2d4
test: mutate Kotlin compilation universe
samchon Sep 4, 2026
863c08c
chore: settle Kotlin graph snapshots
samchon Sep 4, 2026
57369d9
feat: add resident Scala compiler snapshots
samchon Sep 4, 2026
9c3dbaf
feat: freeze Swift index store generations
samchon Sep 4, 2026
cabdf73
fix: provide Linux dispatch headers to Swift sidecar
samchon Sep 4, 2026
1ef8769
Fix Swift native CI and provider ledgers
samchon Sep 4, 2026
2dfe2ef
fix: pin Clang-compatible IndexStoreDB
samchon Sep 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 86 additions & 52 deletions .github/workflows/experiment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,26 +69,82 @@ jobs:
env:
GITHUB_TOKEN: ${{ github.token }}

experiment:
clang_producer:
needs: latest_update
if: ${{ needs.latest_update.outputs.run == 'true' }}
name: pinned Clang producer
runs-on: ubuntu-latest
timeout-minutes: 150
steps:
- name: Skip when no C-family lane was requested
if: ${{ github.event_name == 'workflow_dispatch' && inputs.language != 'all' && inputs.language != 'c' && inputs.language != 'cpp' }}
run: echo "The requested language does not consume the Clang producer."

- name: Checkout
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == 'c' || inputs.language == 'cpp' }}
uses: actions/checkout@v7

- name: Setup Node
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == 'c' || inputs.language == 'cpp' }}
uses: actions/setup-node@v7
with:
node-version: 22.x

- name: Restore the pinned Clang producer
id: clang_producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == 'c' || inputs.language == 'cpp' }}
uses: actions/cache/restore@v6
with:
path: tests/experiment/.work/tools
key: clang-producer-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('packages/graph/src/provider/cpp/CPP_CLANG_PRODUCER_COMMIT.ts', 'tests/experiment/src/clang-producer.mjs') }}

- name: Provision the pinned Clang producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == 'c' || inputs.language == 'cpp' }}
run: node tests/experiment/src/clang-producer.mjs
env:
# An exact hit is immutable. If validation rejects it, rebuilding
# cannot repair that key, so refuse rather than claiming success.
SAMCHON_GRAPH_CLANG_PRODUCER_ALLOW_BUILD: ${{ steps.clang_producer.outputs.cache-hit != 'true' && '1' || '0' }}

# This job owns the build and saves it before either corpus consumer can
# start. A later C or C++ experiment failure cannot discard a successful
# compiler build, and the two consumers cannot race on one cold key.
- name: Save the pinned Clang producer
if: ${{ steps.clang_producer.outputs.cache-hit != 'true' && (github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == 'c' || inputs.language == 'cpp') }}
continue-on-error: true
uses: actions/cache/save@v6
with:
path: tests/experiment/.work/tools
key: ${{ steps.clang_producer.outputs.cache-primary-key }}

# GitHub artifacts restore ordinary files as 0644. Pack first so the
# executable bits on clangd survive the same-run handoff.
- name: Pack the verified Clang producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == 'c' || inputs.language == 'cpp' }}
run: tar -C tests/experiment/.work/tools -cf pinned-clang-producer.tar .

# Cache writes are best-effort and immutable. The artifact is the
# guaranteed same-run handoff, including on a fork PR that cannot save a
# repository cache.
- name: Upload the verified Clang producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == 'c' || inputs.language == 'cpp' }}
uses: actions/upload-artifact@v7
with:
name: pinned-clang-producer
path: pinned-clang-producer.tar
if-no-files-found: error
retention-days: 1
compression-level: 0

experiment:
needs: [latest_update, clang_producer]
if: ${{ always() && needs.latest_update.outputs.run == 'true' }}
name: ${{ matrix.language }} LSP
runs-on: ubuntu-latest
# Fourteen rows install a released producer and finish in minutes; the
# ninety-minute bound is theirs and stays exactly where it is. C and C++
# build a compiler from source, and that is a property of the row rather
# than a defect inside it. Measured: three minutes of checkout, install and
# package build, then 56 minutes to a linked `clangd` on one runner and 107
# on another in the same workflow, then the real-corpus lifecycle run.
# Ninety did not fit. 150 covers the fast runner comfortably and the slow
# one barely, and it is still a bound, so a lane that hangs is caught. Only
# the two rows that build a compiler get it.
#
# The restore below is what should keep an ordinary push away from that
# cost, and it is an expectation rather than a property: Actions caches are
# branch-scoped and evicted, so a first push on a fresh branch still pays
# the whole build, on whichever runner it draws.
timeout-minutes: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && 150 || 90 }}
# Compiler construction has its own 150-minute predecessor above. Every
# matrix row now spends this budget on setup and the real-corpus lifecycle
# only; C and C++ cannot consume it rebuilding LLVM on a cold cache miss.
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -152,50 +208,27 @@ jobs:
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == matrix.language }}
run: pnpm build

# C and C++ are the only rows whose producer is built rather than
# downloaded, and it is a pinned commit: the same bytes, reproduced from
# scratch, on every push. Restoring them instead is not a shortcut around
# the build but a removal of work that had no reason to happen twice.
#
# The key hashes every file the built bytes depend on. `catalog.mjs` is
# the one `setup` actually reads its commit from, so it has to be here —
# keying on the adapter's constant alone would leave the two bound only
# by a text assertion in a different workflow, and a divergence would hit
# the key, fail the `--version` check, rebuild, and then not re-save,
# because an exact hit has nothing to write. That is silent, permanent,
# full-cost rebuilding, so the binding is made structural instead.
#
# Restore and save are split so the save runs even when a later step
# fails. A campaign iterating on these lanes is exactly the case where
# the producer builds and the experiment does not, and `cache`'s combined
# form would discard an hour or more of correct build because a corpus
# assertion afterwards went red.
- name: Restore the pinned Clang producer
id: clang_producer
# The producer job uploads the exact tree it verified, independently of
# whether cross-run cache saving was permitted.
- name: Download the verified Clang producer
if: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && (github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == matrix.language) }}
uses: actions/cache/restore@v6
uses: actions/download-artifact@v8
with:
path: tests/experiment/.work/tools
key: clang-producer-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('packages/graph/src/provider/cpp/CPP_CLANG_PRODUCER_COMMIT.ts', 'tests/experiment/src/catalog.mjs', 'tests/experiment/src/setup-language.mjs') }}
name: pinned-clang-producer
path: tests/experiment/.work/clang-producer-artifact

- name: Unpack the verified Clang producer
if: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && (github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == matrix.language) }}
run: |
mkdir -p tests/experiment/.work/tools
tar -C tests/experiment/.work/tools -xf tests/experiment/.work/clang-producer-artifact/pinned-clang-producer.tar

- name: Install language server
if: ${{ github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == matrix.language }}
run: pnpm --filter @samchon/graph-experiment run setup -- --language ${{ matrix.language }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

# Saved before the experiment runs, not after it, and only on a miss.
# The producer is proved by this point — `setup` refuses to finish unless
# the installed binary reports the pinned commit — and what follows is a
# corpus run whose failure says nothing about the compiler that was
# built. Waiting until the end would tie a correct build's survival to an
# unrelated assertion.
- name: Save the pinned Clang producer
if: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && steps.clang_producer.outputs.cache-hit != 'true' && (github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == matrix.language) }}
uses: actions/cache/save@v6
with:
path: tests/experiment/.work/tools
key: ${{ steps.clang_producer.outputs.cache-primary-key }}
SAMCHON_GRAPH_CLANG_PRODUCER_ALLOW_BUILD: "0"

# Node sizes its old space from a heuristic, not from the runner, and on
# these 16 GiB hosts it settles near 4 GiB, which a real C project's
Expand Down Expand Up @@ -257,6 +290,7 @@ jobs:
# producer has exactly one witness, and it writes to stderr.
SAMCHON_GRAPH_LSP_SERVER_LOG: "1"
SAMCHON_GRAPH_RUST_ANALYZER_HIR: ${{ github.workspace }}/tests/experiment/.work/tools/bin/samchon-rust-analyzer
SAMCHON_GRAPH_ROSLYN_WORKSPACE: ${{ github.workspace }}/tests/experiment/.work/tools/bin/samchon-roslyn

- name: Upload result
if: ${{ always() && (github.event_name != 'workflow_dispatch' || inputs.language == 'all' || inputs.language == matrix.language) }}
Expand Down
121 changes: 84 additions & 37 deletions .github/workflows/index-time.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,69 @@ jobs:
env:
GITHUB_TOKEN: ${{ github.token }}

measure:
clang_producer:
needs: latest_update
if: ${{ needs.latest_update.outputs.run == 'true' }}
name: pinned Clang producer
runs-on: ubuntu-latest
timeout-minutes: 150
steps:
- name: Skip when no C-family project was requested
if: ${{ github.event_name == 'workflow_dispatch' && inputs.project != 'all' && inputs.project != 'redis' && inputs.project != 'leveldb' }}
run: echo "The requested project does not consume the Clang producer."

- name: Checkout
if: ${{ github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == 'redis' || inputs.project == 'leveldb' }}
uses: actions/checkout@v7

- name: Setup Node
if: ${{ github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == 'redis' || inputs.project == 'leveldb' }}
uses: actions/setup-node@v7
with:
node-version: 22.x

- name: Restore the pinned Clang producer
id: clang_producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == 'redis' || inputs.project == 'leveldb' }}
uses: actions/cache/restore@v6
with:
path: tests/experiment/.work/tools
key: clang-producer-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('packages/graph/src/provider/cpp/CPP_CLANG_PRODUCER_COMMIT.ts', 'tests/experiment/src/clang-producer.mjs') }}

- name: Provision the pinned Clang producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == 'redis' || inputs.project == 'leveldb' }}
run: node tests/experiment/src/clang-producer.mjs
env:
SAMCHON_GRAPH_CLANG_PRODUCER_ALLOW_BUILD: ${{ steps.clang_producer.outputs.cache-hit != 'true' && '1' || '0' }}

# The compiler is durable before either measured consumer starts. A
# failed or timed-out cell cannot discard it, and redis/leveldb cannot
# both build the same cold key in this workflow.
- name: Save the pinned Clang producer
if: ${{ steps.clang_producer.outputs.cache-hit != 'true' && (github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == 'redis' || inputs.project == 'leveldb') }}
continue-on-error: true
uses: actions/cache/save@v6
with:
path: tests/experiment/.work/tools
key: ${{ steps.clang_producer.outputs.cache-primary-key }}

- name: Pack the verified Clang producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == 'redis' || inputs.project == 'leveldb' }}
run: tar -C tests/experiment/.work/tools -cf pinned-clang-producer.tar .

- name: Upload the verified Clang producer
if: ${{ github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == 'redis' || inputs.project == 'leveldb' }}
uses: actions/upload-artifact@v7
with:
name: pinned-clang-producer
path: pinned-clang-producer.tar
if-no-files-found: error
retention-days: 1
compression-level: 0

measure:
needs: [latest_update, clang_producer]
if: ${{ always() && needs.latest_update.outputs.run == 'true' }}
# A skipped lane says so in its own name. Dispatching one project leaves the
# other twelve jobs running nothing but an echo, with every later step gated
# off, and they finish green — indistinguishable in the job list from lanes
Expand All @@ -116,17 +176,10 @@ jobs:
# cleanup, and artifact upload. The process cap below is the measurement
# verdict; this job cap is only the outer lifecycle backstop.
#
# C and C++ provision a compiler built from source, which is a property of
# those two rows rather than a defect inside them, and the restore above is
# an expectation rather than a property: a cold key — a fresh branch, an
# eviction, or the first run after the producer pin moves — pays the whole
# build. Measured at 56 minutes to a linked `clangd` on one runner and 107
# on another in the same workflow. Under the old flat cap those two rows
# spent an hour and three quarters compiling and were killed with the
# measurement still running and nothing recorded. 210 covers the slow
# runner plus both measurement columns, and is still a bound, so a lane
# that hangs is caught rather than left to the six-hour platform limit.
timeout-minutes: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && 210 || 120 }}
# The 56-to-107-minute LLVM build now belongs to the 150-minute predecessor
# above. Every measured row gets the same two 30-minute process caps plus an
# hour for setup, quiet-host waits, cleanup, diagnosis, and artifact upload.
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -183,26 +236,20 @@ jobs:
if: ${{ github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == matrix.project }}
run: pnpm build

# The C and C++ producer is an LLVM build, and this workflow used to make
# every run pay for it: leveldb and redis each spent about an hour and
# three quarters compiling clangd inside a two-hour job, reached the
# measurement with minutes left, and were killed by the job cap with
# nothing measured. The experiment matrix already restores that build by
# a key over the exact files its bytes depend on; measuring is not a
# reason to rebuild it, so the same restore/save pair runs here.
#
# The key is identical to the experiment workflow's, deliberately: the
# two lanes provision the same producer from the same sources, so
# whichever runs first pays and the other reads. Restore and save are
# split so a measurement that fails, times out, or is cancelled still
# leaves the compiler it built behind.
- name: Restore the pinned Clang producer
id: clang_producer
# Same-run artifacts, not best-effort cache visibility, hand the verified
# producer to both measured consumers.
- name: Download the verified Clang producer
if: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && (github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == matrix.project) }}
uses: actions/cache/restore@v6
uses: actions/download-artifact@v8
with:
path: tests/experiment/.work/tools
key: clang-producer-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('packages/graph/src/provider/cpp/CPP_CLANG_PRODUCER_COMMIT.ts', 'tests/experiment/src/catalog.mjs', 'tests/experiment/src/setup-language.mjs') }}
name: pinned-clang-producer
path: tests/experiment/.work/clang-producer-artifact

- name: Unpack the verified Clang producer
if: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && (github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == matrix.project) }}
run: |
mkdir -p tests/experiment/.work/tools
tar -C tests/experiment/.work/tools -xf tests/experiment/.work/clang-producer-artifact/pinned-clang-producer.tar

# The same provisioning the experiment matrix uses. It appends each tool's
# directory to GITHUB_PATH, so the benchmark's own child processes resolve
Expand All @@ -212,13 +259,7 @@ jobs:
run: pnpm --filter @samchon/graph-experiment run setup -- --language ${{ matrix.language }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Save the pinned Clang producer
if: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && steps.clang_producer.outputs.cache-hit != 'true' && (github.event_name != 'workflow_dispatch' || inputs.project == 'all' || inputs.project == matrix.project) }}
uses: actions/cache/save@v6
with:
path: tests/experiment/.work/tools
key: ${{ steps.clang_producer.outputs.cache-primary-key }}
SAMCHON_GRAPH_CLANG_PRODUCER_ALLOW_BUILD: "0"

# What actually got installed, printed where the run can be read.
# `recordTool` has always written this manifest and nothing ever surfaced
Expand Down Expand Up @@ -336,6 +377,12 @@ jobs:
- name: Install renderer dependencies
run: pnpm install --frozen-lockfile

# Route summaries derive the expected owner from the runtime's canonical
# provider registry, so the collect job builds that dependency before it
# validates or prints any folded report.
- name: Build provider registry dependency
run: pnpm build

- name: Download reports
uses: actions/download-artifact@v8
with:
Expand Down
Loading
Loading