Skip to content

Fail the accessibility sweep when a window cannot open - #83

Draft
Blackspirits wants to merge 1 commit into
upl/review-base-45f7from
test/accessibility-window-open-failures-45f7
Draft

Blackspirits wants to merge 1 commit into
upl/review-base-45f7from
test/accessibility-window-open-failures-45f7

Conversation

@Blackspirits

@Blackspirits Blackspirits commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Summary

The new accessibility sweep opens every DI-resolvable tool window and checks its visible inputs, but any exception raised while constructing/showing a window is currently added to the generic skipped list and the test continues. The only coverage floor is opened > 50.

That means a real regression that makes one or many previously covered windows fail during setup can silently remove those windows from the accessibility sweep instead of failing it.

This change:

  • keeps intentional structural skips (no one-argument constructor / view model not registered in DI) unchanged;
  • records exceptions raised while constructing, showing or initially draining a candidate window as test failures;
  • reports the window, exception type and message for diagnosis;
  • leaves the existing accessible-name assertions unchanged.

Validation

  • based on upstream 45f7430aa7ec8d32f0079563f12ff7c0bfd1dc8e
  • upstream PR Accessible names for inputs, lists and grid rows in all tool windows SubtitleEdit/subtitleedit#14825 states that 296 of 299 windows are opened by this sweep
  • SkippedWindows is currently empty, so runtime setup failures are not declared expected exclusions
  • diff is limited to tests/UI/Logic/AccessibleNamesTests.cs
  • full solution restore passed
  • full solution build passed
  • full solution test suite passed on the first run
  • UI suite: 5,150 passed, 9 skipped, 0 failed (5,159 total)
  • retry step was not used
  • validation run: #34762328881

Because the stricter test remains green, the old exception-to-skip path was not required by the current baseline.

No production behavior is changed.

AI assistance: ChatGPT was used to adversarially review the accessibility test harness for fail-open coverage gaps.

@Blackspirits Blackspirits left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial re-check: the change is test-only and converts runtime window-open exceptions from silent skips into explicit failures while preserving structural skips. Full CI run #34762328881 passed restore, build and the complete solution suite on the first run; UITests reported 5,150 passed, 9 skipped and 0 failed. This also confirms the stricter setup-failure gate is compatible with the current baseline. No blocker identified. Keep draft; no merge or upstream change performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant