Skip to content

Remove VSTest run/discovery context from PlatformServices (Phase 6d-2)#9621

Open
Evangelink wants to merge 1 commit into
mainfrom
dev/amauryleve/vstest-decoupling-runcontext
Open

Remove VSTest run/discovery context from PlatformServices (Phase 6d-2)#9621
Evangelink wants to merge 1 commit into
mainfrom
dev/amauryleve/vstest-decoupling-runcontext

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6d-2 of the PlatformServices platform-agnostic initiative

Removes the VSTest IRunContext/IDiscoveryContext types entirely from MSTestAdapter.PlatformServices by extracting the only two members ever read off them — .RunSettings.SettingsXml and .TestRunDirectory — at the adapter boundary and passing them in.

What changed

  • DeploymentContext (the { TestRunDirectory, RunSettingsXml } DTO from 6a) is un-guarded and becomes the single execution-inputs carrier.
  • Execution: MSTestExecutor builds new DeploymentContext(runContext?.TestRunDirectory, runContext?.RunSettings?.SettingsXml) and injects it into TestExecutionManager.RunTestsAsync/ExecuteTestsAsync/ExecuteTestsInSourceAsync/Deploy.
  • Discovery: MSTestDiscoverer passes the run-settings XML string into UnitTestDiscoverer.DiscoverTests/DiscoverTestsInSource; MSTestDiscovererHelpers.InitializeDiscovery and MSTestSettings.PopulateSettings take string? settingsXml.

No behavior change

Only .SettingsXml and .TestRunDirectory were ever read, so this is byte-for-byte. The PopulateSettings guard change is provably equivalent (RunSettingsFileHasMSTestSettings(null) returns false). IRunContext/IDiscoveryContext are now absent from PlatformServices code (doc comments only); the remaining ObjectModel.Adapter surface is the IFrameworkHandle-backed deploy/recorder/logger handles.

Verification

Full build all TFMs; PlatformServices.UnitTests 897/935; MSTestAdapter.UnitTests 21; MSTest.IntegrationTests 47 pass/1 skip (incl. TestCaseFilteringTests + deployment); PlatformServices.Desktop.IntegrationTests 15 (AppDomain deployment). Expert-reviewer: no material findings, definitive byte-for-byte verdict.

Base / stacking

Stacked on Phase 6d-1 (#9591). Base = dev/amauryleve/vstest-decoupling-filter; review/merge after #9591.

Remaining

6e (relocate bridges + deep UnitTestElement<->TestCase + EngineConstants; close #9568/#9573), then 7 (drop the ObjectModel/ObjectModel.Utilities PackageReference + guard test).

Base automatically changed from dev/amauryleve/vstest-decoupling-filter to dev/amauryleve/vstest-decoupling-settings July 5, 2026 18:24
Base automatically changed from dev/amauryleve/vstest-decoupling-settings to dev/amauryleve/vstest-decoupling-base July 5, 2026 19:22
@Evangelink Evangelink marked this pull request as ready for review July 5, 2026 19:27
@Evangelink Evangelink force-pushed the dev/amauryleve/vstest-decoupling-runcontext branch from 60d363b to 9bd8614 Compare July 5, 2026 19:55
@Evangelink Evangelink changed the base branch from dev/amauryleve/vstest-decoupling-base to main July 5, 2026 19:55
…ervices

Squashed rebase of the vstest-decoupling PlatformServices stack onto main.
Phases 1, 2 and 5 already landed on main via #9548/#9567/#9550; this commit
carries the remaining net-new work:

- Phase 3  (#9566): abstract the VSTest discovery sink (IUnitTestElementSink).
- Phase 4  (#9572): abstract VSTest execution input.
- Phase 6a (#9576): neutralize deployment input (DeploymentContext).
- Phase 6b (#9579): neutralize test result recording (ITestResultRecorder).
- Phase 6c (#9585): neutralize test message logging.
- Phase 6c2:         neutralize run-settings input in the host layer (settingsXml).
- Phase 6d-1:        move test-case filter parsing to the adapter boundary
                     (ITestElementFilterProvider / TestElementFilterProvider).
- Phase 6d-2:        remove IRunContext/IDiscoveryContext from PlatformServices.
- Phase 6e-1 (#9622): remove IFrameworkHandle from the execution engine.
- Phase 6e-2 (#9623): relocate VSTest logger/sink bridges to the adapter.
- Phase 6e-3a (#9624): neutralize the trait type on UnitTestElement.

Result: MSTestAdapter.PlatformServices no longer references the VSTest
run/discovery context or result object model; those types live only at the
MSTest.TestAdapter boundary.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink Evangelink force-pushed the dev/amauryleve/vstest-decoupling-runcontext branch from 9bd8614 to 9a7d463 Compare July 5, 2026 20:03
Copilot AI review requested due to automatic review settings July 5, 2026 20:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum diff size. Try reducing the number of changed files and lines, and requesting a review from Copilot again.


// Placing this after deployment since we need information post deployment that we pass in as properties.
CacheSessionParameters(runContext, frameworkHandle);
CacheSessionParameters(deploymentContext.RunSettingsXml, messageLogger);
foreach (string warning in warnings)
{
testExecutionRecorder.SendMessage(TestMessageLevel.Warning, warning);
messageLogger.SendMessage(MessageLevel.Warning, warning);
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9621

48 test methods graded across 10 files. Most changes are refactoring-only signature updates; five genuinely new test methods were added in TestCaseDiscoverySinkTests and UnitTestElementTests. Main quality concerns: one hard-coded Windows path, two integration tests with no explicit assertion, and several tests using SequenceEqual.Should().BeTrue() instead of the more diagnostic .Should().Equal(). The 5 new tests are all A-grade.

GradeTestNotes
D (60–69) mod TestExecutionManagerTests.
RunTestsForTestShouldLoadSourceFromDeploymentDirectoryIfDeployed
Hard-coded @"C:\temp" in both mock setup and Verify — fails on Linux; replace with a platform-agnostic temp path.
C (70–79) mod DesktopTestSourceHostTests.
ChildDomainResolutionPathsShouldHaveSearchDirectoriesSpecifiedInRunsettings
No explicit assertion; test only confirms AppDomainUtilities.CreateInstance does not throw — add an assertion on the returned instance.
C (70–79) mod DesktopTestSourceHostTests.
ParentDomainShouldHonorSearchDirectoriesSpecifiedInRunsettings
No explicit assertion; test only verifies Assembly.Load does not throw — add at least one assertion on the loaded assembly.
C (70–79) mod TcmTestPropertiesProviderTests.
GetTcmPropertiesShouldHandleDuplicateTestsProperlyFromTestCase
75-line body with three independent verification cycles — split into separate focused tests per call scenario.
C (70–79) mod TestContextImplementationTests.
WritesFromBackgroundThreadShouldNotThrow
No explicit assertion; test only verifies concurrent writes don't throw — add a buffer-state assertion after t.Join().
C (70–79) mod TestExecutionManagerTests.
RunTestsForTestShouldPassInDeploymentInformationAsPropertiesToTheTest
Only verifies GetProperties was called once with any argument — add an assertion on the actual properties injected into the test context.
B (80–89) mod TcmTestPropertiesProviderTests.
GetTcmPropertiesShouldCopyMultiplePropertiesCorrectlyFromTestCase
50-line body with two verification cycles; thorough assertions — consider splitting into separate tests for each call scenario.
B (80–89) mod TcmTestPropertiesProviderTests.
GetTcmPropertiesShouldReturnEmptyDictionaryIfTestCaseIsNull
Single null-check assertion — complete for this narrow boundary condition.
B (80–89) mod TcmTestPropertiesProviderTests.
GetTcmPropertiesShouldReturnEmptyDictionaryIfTestCaseIdIsZero
Single null assertion after 15-property setup — briefly asserting which property triggered the null return would improve diagnostics.
B (80–89) mod TypeEnumeratorTests.
GetTestsShouldReturnBaseTestMethodsFromAnotherAssemblyByConfiguration
NotBeNull + Contain checks cover the key case — also asserting total count would guard against unexpected extra discovered methods.
B (80–89) mod TypeEnumeratorTests.
GetTestsShouldReturnBaseTestMethodsFromAnotherAssemblyByDefault
Same structure as ByConfiguration — asserting total test count would strengthen coverage.
B (80–89) mod UnitTestDiscovererTests.
DiscoverTestsInSourceShouldSendBackTestCasesDiscovered
Times.AtLeastOnce leaves exact count unchecked — pin to Times.Exactly(N) or verify specific test FQNs.
B (80–89) new UnitTestElementTests.
WithUpdatedSourceShouldReturnSameInstanceWhenSourceUnchanged
Single identity assertion — clear and correct for this same-source edge case.
B (80–89) mod TestExecutionManagerTests.
RunTestsForASingleTestShouldSendSingleResult
SequenceEqual.Should().BeTrue() obscures which element differed on failure — prefer .Should().Equal(...).
B (80–89) mod TestExecutionManagerTests.
RunTestsForIgnoredTestShouldSendResultsMarkingIgnoredTestsAsSkipped
Asserts only index-0 elements — adding count checks would catch spurious extra events.
B (80–89) mod TestExecutionManagerTests.
RunTestsForMultipleSourcesShouldRunEachTestJustOnce
Checks count only — also verifying that no specific test ran twice would make the intent explicit.
B (80–89) mod TestExecutionManagerTests.
RunTestsForMultipleTestShouldSendMultipleResults
Mixes SequenceEqual.Should().BeTrue() with direct Should() calls — standardise on .Should().Equal().
B (80–89) mod TestExecutionManagerTests.
RunTestsForTestShouldPassInTcmPropertiesAsPropertiesToTheTest
Delegates to VerifyTcmProperties loop; thorough, but failure messages point to the helper, not the test scenario.
B (80–89) mod TestExecutionManagerTests.
RunTestsForTestShouldPassInTestRunParametersInformationAsPropertiesToTheTest
Checks only webAppUrl — also assert webAppUserName to cover both configured parameters.
B (80–89) mod TestExecutionManagerTests.
RunTestsForTestWithFilterShouldSendResultsForFilteredTests
SequenceEqual.Should().BeTrue() pattern gives poor failure diagnostics — prefer .Should().Equal(...).
A (90–100) mod TcmTestPropertiesProviderTests.
GetTcmPropertiesShouldGetAllPropertiesFromTestCase
All 15 TCM properties verified via helper loop; clear AAA structure.
A (90–100) new TestCaseDiscoverySinkTests.
SendTestElementShouldAddEachTestElementInOrder
Count + identity at both indices — fully characterises insertion order.
A (90–100) new TestCaseDiscoverySinkTests.
SendTestElementShouldAddTheTestElement
Count + identity + not-null — three distinct assertions on a single add.
A (90–100) new TestCaseDiscoverySinkTests.
TestCaseDiscoverySinkConstructorShouldInitializeTests
Checks not-null and zero count — fully characterises the initialised state.
A (90–100) mod TestContextImplementationTests.
DisplayMessageShouldForwardToIMessageLogger
Three exact mock verifies per message level — exhaustive coverage of all paths.
A (90–100) mod TestPropertyAttributeTests.
GetTestMethodInfoShouldAddPropertiesFromContainingClassAndBaseClassesAndOverriddenMethodsCorrectly_
OverriddenIsNotTestMethod
Exact name-value assertions for all six traits in order; thorough and precise.
A (90–100) mod TestPropertyAttributeTests.
GetTestMethodInfoShouldAddPropertiesFromContainingClassAndBaseClassesAndOverriddenMethodsCorrectly_
OverriddenIsTestMethod
Exact name-value assertions for all six traits in order.
A (90–100) mod TestPropertyAttributeTests.
GetTestMethodInfoShouldAddPropertiesFromContainingClassCorrectly
Exact name-value checks for all three traits at each index; clean AAA.
A (90–100) mod UnitTestDiscovererTests.
DiscoverTestsInSourceShouldNotThrowWhenTreatDiscoveryWarningsAsErrorsIsFalse
No-throw path verified; exact logger call counts for both warning and error levels checked.
A (90–100) mod UnitTestDiscovererTests.
DiscoverTestsInSourceShouldThrowOnFileNotFound
Exception type + message pattern assertion; fully specifies the expected error.
A (90–100) mod UnitTestDiscovererTests.
DiscoverTestsInSourceShouldThrowWhenTreatDiscoveryWarningsAsErrorsIsTrue
Exception assertion + two logger verifies — covers primary outcome and side-effects.
A (90–100) mod UnitTestDiscovererTests.
DiscoverTestsShouldThrowOnFileNotFound
Exception + message pattern; clean and specific.
A (90–100) mod UnitTestDiscovererTests.
SendTestCasesShouldNotSendAnyTestCasesIfFilterError
Both FQNs verified Times.Never — covers the full negative filter-error path.
A (90–100) mod UnitTestDiscovererTests.
SendTestCasesShouldNotSendAnyTestCasesIfThereAreNoTestElements
Concise negative test — verifies no test case was sent for an empty input.
A (90–100) mod UnitTestDiscovererTests.
SendTestCasesShouldSendAllTestCaseData
Each specific FQN verified Times.Once; precise mock-call assertions.
A (90–100) mod UnitTestDiscovererTests.
SendTestCasesShouldSendAllTestCasesIfGetTestCaseFilterNotPresent
Both FQNs sent once when the filter API is absent — correct boundary case.
A (90–100) mod UnitTestDiscovererTests.
SendTestCasesShouldSendAllTestCasesIfNullFilterExpression
Both FQNs sent once for null expression — clear null-filter boundary test.
A (90–100) mod UnitTestDiscovererTests.
SendTestCasesShouldSendFilteredTestCasesIfValidFilterExpression
Matching FQN Times.Once, non-matching Times.Never — clean positive/negative pair.
A (90–100) new UnitTestElementTests.
GetOrCreateHostTestCaseShouldMaterializeAndCacheWhenNoHandle
Null start-state + not-null result + same-instance cache check — verifies the full caching contract.
A (90–100) new UnitTestElementTests.
GetOrCreateHostTestCaseShouldReturnHostHandleWhenPresent
Identity check confirms the pre-set handle is returned unchanged.
A (90–100) new UnitTestElementTests.
WithUpdatedSourceShouldReturnCloneWithNewSourceAndLeaveOriginalUnchanged
Four distinct invariants verified: clone identity, new source, original immutability, separate method object.
A (90–100) mod TestExecutionManagerTests.
RunTestsForCancellationTokenCanceledSetToTrueShouldSendZeroResults
Exception assertion + three empty-list checks — fully characterises the cancelled-run contract.
A (90–100) mod TestExecutionManagerTests.
RunTestsForTestShouldCleanupAfterExecution
Verifies both load and cleanup call ordering by index; meaningful sequence assertion.
A (90–100) mod TestExecutionManagerTests.
RunTestsForTestShouldDeployBeforeExecution
Verifies deploy-before-load ordering; important behavioral constraint.
A (90–100) mod TestExecutionManagerTests.
RunTestsForTestShouldNotCleanupOnTestFailure
Times.Never Moq verify on cleanup when a test fails; precise negative assertion.
A (90–100) mod TestExecutionManagerTests.
RunTestsForTestWithFilterErrorShouldSendZeroResults
All three output lists verified empty on filter error; thorough boundary check.
A (90–100) mod TestExecutionManagerTests.
RunTestsShouldClearSessionParametersAcrossRuns
Verifies updated parameter value after second run — catches stale-session-parameter bugs.
A (90–100) mod TestExecutionManagerTests.
SendTestResults_
WhenUnitTestResultsIsEmpty_
RecordsEndWithoutResult
Equal for end-list + BeEmpty for results; clean dual-assertion boundary test.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Grade Tests on PR (on open / sync) workflow. · 371.7 AIC · ⌖ 9.87 AIC · ⊞ 9.5K · [◷]( · )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid double TestCase construction when filtering during discovery/execution (platform-agnostic refactor follow-up)

2 participants