[Jake Bailey (jakebailey)]

Comparisons used to measure every new-compiler iteration and then every baseline iteration. On shared Azure VMs the VM's speed can change between those two phases, which shifts every sample on one side and produces false regressions/improvements (up to 7% in A/A runs).

This alternates the two sides in 4 pairs of shorter rounds (new/baseline, then baseline/new, ...) so drift hits both sides about equally, then merges the rounds back into one benchmark file per side.

• Each round runs the configured warmups; the total measured iteration count is unchanged. ts-perf merge gains --iterations to record the total, since merging files with different counts otherwise drops it.
• Scenarios cloned at the compiler commit (the strada scenarios when comparing JS compilers) are set up again when switching sides. clone_strada_scenario marks this with TSPERF_SCENARIO_COMMIT_DEPENDENT_FILE.
• On hosted runners, each round waits for the machine to be quiet, retrying up to 3 times, since 8 checks per job make an occasional 60s timeout likely.
• INTERLEAVE_BENCHMARKS (default on) can turn this off to get the old sequential steps.

Validation on the Azure Linux pool (preset faster), 13 scenarios per run:

A/A runs, abs. baseline/new shift	Interleaved (6 runs)	Sequential (5 runs)
Median	0.42%	0.58%
90th percentile	1.14%	1.53%
Max	2.52%	7.30%
Over 1%	13%	32%

On a real two-commit comparison, the sequential run reported a false +6.1% on vscode; the interleaved run kept every scenario within ±1.8%. Measurement takes about twice as long (warmups per round), but whole jobs take only about 5–10% longer.