Use server GC for the decompiler test suite (workstation GC on the shared Windows CI runner)

[christophwille]

The server-GC part of #3940, redone on its own on top of master (which now has #4012). Nothing else from #3940 is included, and #3940 itself is untouched.

Two commits:

1. <ServerGarbageCollection>true</ServerGarbageCollection> for ICSharpCode.Decompiler.Tests. The suite keeps one NUnit worker per logical CPU busy with allocation-heavy decompiles (~220 GB allocated per run); under workstation GC every gen0/gen1 collection any worker triggers suspends the whole process, and on a 24-thread box that is more than half the wall time.
2. DOTNET_gcServer: 0 on the Windows CI test step. The Windows job runs every test host of the solution concurrently on a 4-core runner, and there server GC starves the neighbours (details below). The env var overrides the runtimeconfig setting, so the suite still gets server GC on machines it has to itself (local runs, the Linux job which runs the projects one at a time).

How this was verified

1. Local: does server GC really remove the GC pause on master?

The commit's numbers were originally measured on #3940, which also had 2x-CPU NUnit workers and all fixtures parallel, so they were re-taken on plain master + commit 1. Same binary, same machine state, Debug, 24-thread Windows 11 box, ILSpy-tests checked out (roundtrips run), machine otherwise idle. The only difference between the two runs is DOTNET_gcServer=0 set for the workstation run. GC counters come from GC.CollectionCount, GC.GetTotalPauseDuration(), GC.GetTotalAllocatedBytes() and Process.TotalProcessorTime read at the end of the run; CPU from \Processor(_Total)\% Processor Time sampled every 4 s during dotnet test.

	workstation GC	server GC
suite wall	553 s	310 s (1.78x)
average CPU	45%	80%
gen0 / gen1 / gen2 collections	27,229 / 6,919 / 197	1,251 / 492 / 125
total GC pause	305 s (55% of wall)	14 s (5%)
test host processor time	45m16	45m58 (same work)
allocated	223 GB	223 GB
peak working set	3.2 GB	2.3 GB
passed / failed / skipped	4254 / 0 / 20	4254 / 0 / 20

Per-test (TRX startTime..endTime):

test	workstation	server
Random_TestCase_1	353 s	133 s
NRefactory_CSharp	337 s	156 s
ExplicitConversions_With_NativeInts	325 s	139 s
ExplicitConversions	319 s	136 s
ExplicitConversions_32	271 s	(below 110 s)
NewtonsoftJson_net45	(below 247 s)	203 s (new longest; MSBuild/NUnit child processes, unaffected by host GC)

Same processor time, half the wall: the suite was waiting on stop-the-world pauses, not on the decompiler.

2. CI: is the Windows workstation-GC valve still needed now that #4012 is in?

#3940's status comment blamed its 3/6 Windows failures on the ILSpy.Tests 15 GB leak (paged-out runner) and suggested that with #4012 the valve might be unnecessary. To test that, a first push of this branch carried commit 1 only, no valve, plus a temporary diagnostics sampler (see the last section) logging memory, pagefile, paging/disk counters, CPU and a timed module walk over every process every 20 s during the test step. Run: https://github.com/icsharpcode/ILSpy/actions/runs/32226154313

	master (post-#4012, run 32177067040)	server GC, no valve (run 32226154313)
Windows Debug job	green, tests 23m22	green, tests 21m35
Windows Release job	green, tests 11m10	failed, tests 13m38
ICSharpCode.Decompiler.Tests host, Debug / Release	1393 s / 664 s	1226 s / 751 s
ILSpy.Tests host, Debug / Release	903 s / 375 s	913 s / 685 s
ILSpy.Tests.Windows host, Debug / Release	45 s / 26 s	20 s / 86 s
failing test	-	NetFrameworkProcessesTests.The_Explorer_Routes_A_Framework_Process_To_The_Module_Scan: OperationCanceledException after 61 s (the explorer's 60 s budget), exactly the #3940 failure

What the sampler recorded in both Windows jobs, the green Debug one included:

sampler	Debug	Release
free memory, minimum	9.6 GB	9.4 GB
pagefile used, maximum	1.8 GB of 2.9 GB	1.6 GB of 2.9 GB
Avg. Disk sec/Read / disk queue length	0 / 0	0 / 0
% Processor Time	87-100%, mostly 98-100%	90-100%, mostly 98-100%
module walk over all ~170 processes	median 29 s, max 168 s	median 38 s, max 113 s
slowest single processes	idle svchost/spoolsv/mqsvc with 7-22 MB working sets at 20-66 s each	same, up to 47 s

So memory is not the constraint any more (#4012 did its job: 9+ GB free, disk idle), but the decompiler host on server GC pins all 4 cores for the whole step and the neighbours' cross-process module enumeration is starved of CPU. Under workstation GC the decompiler host spends half its wall time paused, which is what left the neighbours room on master. The Debug job passed only because its one timed walk happened to land in a quieter moment. The decompiler suite itself gains little on 4 cores (server GC scales with cores; Debug -12%, Release +13%).

Conclusion: the valve stays. Commit 2 was added back with its comment and message rewritten to this evidence, the sampler commit was dropped and the branch force-pushed. Confirmation run of the final branch: https://github.com/icsharpcode/ILSpy/actions/runs/32228410174 - both Windows jobs green (Debug tests 22m09, Release 13m12), Linux green, macOS failed in Right_Clicking_A_Second_Row_Moves_The_Context_Highlight_To_It, the same UI test that failed on master's own post-#4012 run (pre-existing, unrelated).

Not taken from #3940: the concurrent .NET Framework process scan (b592b6164, reverted there already; on a CPU-saturated box AsParallel() cannot help and empirically did not). The fixture-host lifetime hardening (2853a8b08) is unrelated to GC and is its own PR: #4032.

Temporary helpers used for the verification (all backed out, none in this branch)

Local GC counters. An uncommitted [OneTimeTearDown] added to the existing ToolsetSetup [SetUpFixture] in ICSharpCode.Decompiler.Tests/TestTraceListener.cs (namespace ICSharpCode.Decompiler, so it wraps every test in the assembly), appending one line to a file named by GCSTATS_FILE:

[OneTimeTearDown]
public void WriteGcStats()
{
	var proc = Process.GetCurrentProcess();
	var line = $"server={System.Runtime.GCSettings.IsServerGC} gen0={System.GC.CollectionCount(0)} gen1={System.GC.CollectionCount(1)} gen2={System.GC.CollectionCount(2)} pause={System.GC.GetTotalPauseDuration()} allocGB={System.GC.GetTotalAllocatedBytes() / 1e9:F1} cpu={proc.TotalProcessorTime} peakWSMB={proc.PeakWorkingSet64 / 1048576}";
	System.IO.File.AppendAllText(System.Environment.GetEnvironmentVariable("GCSTATS_FILE") ?? "gcstats.txt", line + System.Environment.NewLine);
}

Reverted with git checkout -- before the commit was amended.

Local driver (run-suite.ps1, scratch only). Built once (restore.ps1, then dotnet build ICSharpCode.Decompiler.Tests -c Debug --no-restore), then per mode: set or clear DOTNET_gcServer, start a background Get-Counter '\Processor(_Total)\% Processor Time' -SampleInterval 4 job, Measure-Command { dotnet test --project ICSharpCode.Decompiler.Tests\ICSharpCode.Decompiler.Tests.csproj --no-build --report-trx --results-directory <mode> } (--no-build so no implicit restore prunes packages.lock.json), stop the job, average the samples. Per-test durations were read from the TRX UnitTestResult start/end stamps.

CI diagnostics sampler (commit 4489ff5bb from #3940, cherry-picked and later dropped). BuildTools/ci-diag-sampler.ps1, started hidden by an extra workflow step before "Execute unit tests" and writing into the test-results directory (the upload-artifact path was widened from *.trx to *). Every 20 s it logged Win32_OperatingSystem free memory and Win32_PageFileUsage; \Memory\Pages/sec, \Memory\Pages Input/sec, \Memory\Available MBytes, \PhysicalDisk(_Total)\Avg. Disk sec/Read, \PhysicalDisk(_Total)\Current Disk Queue Length, \Processor(_Total)\% Processor Time, \System\Processes; the six largest working sets; and its own timed Process.Modules.Count walk over every process, naming any process that took more than a second. Present only in run 32226154313; the commit was removed with git reset --hard HEAD~1 and a force-push before commit 2 was added.

CI reader (read-run.ps1, scratch only). gh run download <id> --pattern 'test-results-*', then per TRX the run duration and counters plus the durations of the process-explorer tests, and from diag.log the minimum free memory, maximum pagefile use, module-walk median/max with the slow-process lists, and Pages Input/sec max/average. The master baseline row above comes from running it on run 32177067040.

🤖 Generated with Claude Code