Overlap the sequential waits inside the decompiler test infrastructure - #4034
Open
christophwille wants to merge 7 commits into
Open
Overlap the sequential waits inside the decompiler test infrastructure#4034christophwille wants to merge 7 commits into
christophwille wants to merge 7 commits into
Conversation
Tester.Initialize is about to issue the toolset Fetch calls concurrently; each Fetch ends by registering its install path in a plain Dictionary, which is not safe for concurrent writers. Lookups need no lock: they only happen after Initialize has awaited all registrations. Assisted-by: Claude:claude-fable-5:Claude Code
The setup fixture gates every test in the suite, and on a cold machine it serialized nine NuGet fetches plus two self-contained TestRunner builds. The fetches extract into disjoint directories and the builds depend on no fetched toolset, so everything now runs concurrently and is awaited once. Only the two Windows RID builds stay sequential with each other: they share the TestRunner project's obj/ directory, and their implicit restores would race on project.assets.json. Assisted-by: Claude:claude-fable-5:Claude Code
Every roundtrip test spawned its own vswhere.exe to answer a question that is invariant for the lifetime of the process. Lazy<Task<string>> with ExecutionAndPublication guarantees a single spawn even when the parallel roundtrip fixture hits the lookup from several tests at once. Assisted-by: Claude:claude-fable-5:Claude Code
RunAndCompareOutput awaited the two runs back to back, but they are independent processes with separately buffered output. The new StartRun helper also lets callers begin the original run even earlier and hand the in-flight task to the comparison; it pre-observes the task fault so a run abandoned after an upstream failure cannot surface as an UnobservedTaskException. Plain WhenAll (no error aggregation) keeps NUnit Ignore semantics when both runs raise IgnoreException, and the exit codes are still asserted in the original order, so failure output is unchanged. Assisted-by: Claude:claude-fable-5:Claude Code
The original binary is complete once the first compile (or ilasm) finishes, and the decompile/recompile stages only read it, so its execution now overlaps them instead of waiting at the very end of the pipeline. For mcs configurations the .exe.config write moves ahead of the run start - the runtime reads it at process launch - while the compiler-option mutation stays after the decompile, which must see the original options. Assisted-by: Claude:claude-fable-5:Claude Code
The RunWithTest/RunWithOutput lambdas blocked an NUnit worker thread with GetAwaiter().GetResult() on inherently async work. Passing a Func<string, Task> lets RunInternal await the action, and enables handing an already-running execution into the comparison. Assisted-by: Claude:claude-fable-5:Claude Code
In RunWithOutput roundtrip tests the reference executable from the ILSpy-tests checkout ran only after the whole-project decompile and the MSBuild rebuild had finished, although nothing in that pipeline writes to the input directory. Its execution now starts first and overlaps the multi-minute decompile. The submodule-missing guard moves ahead of the early start so those tests still report Ignored, not a faulted launch. Assisted-by: Claude:claude-fable-5:Claude Code
christophwille
force-pushed
the
decompiler-tests-overlap
branch
from
August 19, 2026 12:34
e67b10b to
43b0d04
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The "Round 2: sequential awaits inside the test infra" commits of #3940, split out on their own. They do not depend on #3940's NUnit worker-count / fixture-parallelism change (which is not being pursued):
RoundtripAssemblyand the matrix runners are alreadyParallelizable(ParallelScope.All)on master, and the overlaps below are per-test or per-process, independent of how many workers NUnit runs. All seven apply to master without conflicts and are cherry-picked verbatim; onlyICSharpCode.Decompiler.Testsis touched (Helpers/Tester.cs,Helpers/RoslynToolset.cs,CorrectnessTestRunner.cs,RoundtripAssembly.cs).lockaround the twoDictionary.AddinRoslynToolset/RefAssembliesToolset.Fetch; lookups stay lock-free (they only happen afterInitializecompleted)Tester.InitializeTask.WhenAll(every failure surfaced, not just the first); only the two Windows RID TestRunner builds stay sequential with each other (sharedobj/, implicit restores would race onproject.assets.json)Initializegates every test in the suite; on a cold machine / CI it serialized all downloads and buildsLazy<Task<string>>(ExecutionAndPublication); the non-WindowsAssert.Ignorestays outside the cache so it is raised per test, not memoized as a faulted taskvswhere.exeper test for a process-invariant answerTester.StartRunreturns the in-flight run (fault pre-observed, so an abandoned run after an upstream failure cannot raiseUnobservedTaskException);RunAndCompareOutputgets an overload taking that task and awaits both with a plainWhenAll(keeps NUnit Ignore semantics); exit codes are still asserted in the original order, so failure output is unchangedRunCS/RunVB/RunILstart the original exe right after the first compile; for mcs configs the.exe.configwrite moves ahead of the run start (the runtime reads it at launch) while the compiler-option mutation stays after the decompile, which must see the original optionstestActionasynchronousFunc<string, Task>instead ofGetAwaiter().GetResult()on NUnit worker threadsRunWithOutputstarts the reference exe from theILSpy-testscheckout before the whole-project decompile + MSBuild rebuild (nothing in that pipeline writes to the input dir); the submodule-missing guard is hoisted intoEnsureTestDirAvailable()and runs before the early start, so those tests still report IgnoredVerification
Local, 24-thread Windows 11 box, Debug,
ILSpy-testschecked out (roundtrips and the mcs matrix run: 140 mcs cases passed, which exercises the.exe.config-before-run reordering), workstation GC (this branch is off master; #4033 is separate):bin/.../{roslyn,vswhere,netfx}deleted, exercises the concurrentInitialize)The 20 skips are the usual environment-gated ones. The wall-time difference is within a few percent and is not the point; the per-test overlap shows up as ~7% less summed test time for the same work. The local "cold"
Initializeis served fromILSpy-tests/nugetand the already-built TestRunner, so it is only a correctness check of the concurrent path here (4 s); CI is where the downloads actually happen.CI
Run https://github.com/icsharpcode/ILSpy/actions/runs/32240841471: all four jobs green (Windows Debug/Release, Linux, macOS).
No measurable CI wall-time effect either way: the Windows numbers sit inside the run-to-run spread of that shared 4-core runner (master's own Release test step has ranged 11m10-15m26 across recent runs), and on Linux the suite is CPU-bound with most of its matrix skipped. The value of these commits is structural (no blocking waits on worker threads, no per-test
vswhere.exe, setup downloads and builds overlapped, original/decompiled runs concurrent), not a headline number.Not taken from #3940: the NUnit
LevelOfParallelism/Parallelizable(Fixtures)/Ordercommit and the Defender doc (dropped), the block-invariant change (reverted there), server GC (#4033), the fixture-host hardening (#4032), the concurrent .NET Framework scan and the diagnostics sampler (reverted there).🤖 Generated with Claude Code