Skip to content

Harden SRV resolver tests for restricted CI runners - #1041

Merged
Tomasz Naumowicz (tnaum-ms) merged 4 commits into
mainfrom
dev/tnaum/harden-srv-resolver-tests
Oct 21, 2026
Merged

Tomasz Naumowicz (tnaum-ms) merged 4 commits into
mainfrom
dev/tnaum/harden-srv-resolver-tests

Conversation

@tnaum-ms

@tnaum-ms Tomasz Naumowicz (tnaum-ms) commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Make the SRV DNS resolver tests reliable on Windows and restricted CI runners without changing production resolver behavior.

Why

The UDP refusal test expected a closed localhost port to promptly produce ECONNREFUSED. Windows or restricted runners can suppress that notification and produce a timeout instead. Other tests also depended on short real-time deadlines, delayed TCP writes, or randomly generated query IDs being different. Increasing timeouts would retain these OS-dependent and probabilistic assumptions, so the tests now control the relevant events directly.

Changes

  • Inject ECONNREFUSED and Windows-style ECONNRESET without sending a datagram, and verify immediate rejection and cleanup.
  • Freeze clocks and resolver timers; advance time explicitly only in timeout scenarios and assert timer cleanup.
  • Split client TCP data events deterministically instead of relying on delayed writes to create separate frames.
  • Remove the random-ID inequality assertion and explicitly synchronize probe cancellation.
  • Guard system DNS discovery and driver lookups against accidental real-network use.
  • Clean up partially started DNS stubs, retry address-in-use conflicts, and add five lifecycle regression tests.
  • Document the test-isolation guarantees and remaining loopback requirement.
  • Keep the forwarded TCP event result explicitly boolean and its saved signature non-overloaded for both root and strict package-test configurations.
  • Align the Case 2 checklists with CI: require strict workspace test typechecks and unconditional localization validation.

Scope

Test code, test helpers, feature documentation, and Case 2 verification instructions only. No production resolver changes, no external DNS access, and no changes to CI permissions. Localhost UDP/TCP sockets must still be permitted.

Validation — Case 2 (2026-10-21)

  • npm run l10n — not applicable: no vscode.l10n.t() strings added, changed, or removed.
  • npm run prettier-fix — passed.
  • npm run l10n:check — passed; localization is up to date.
  • npm run lint — passed after fixing the TCP event mock return type.
  • npx vitest run — passed: 315 files, 5,190 tests passed, 1 skipped.
  • npm run build — passed.
  • npm run typecheck:tests — passed across all workspaces after reproducing and fixing the CI errors.
  • npm run package — passed: VSIX generated successfully. Non-fatal dependency annotation and CSS warnings were emitted.
  • After the final lint correction, formatting, full lint, build, and all 84 resolver tests passed again.
  • Ten shuffled-order package runs also passed during implementation.
  • Validation ran on Linux. Windows and restricted-runner behavior remain to be verified by CI.

Release and readiness

  • Assigned to the existing 1.0.2 milestone.
  • The operator explicitly requested skipping the AI review-agent/pre-review step for this PR. No AI review agent was launched and no review record was fabricated.

Context

Static-check failure follow-up

The initial local Case 2 list omitted npm run typecheck:tests, which CI runs separately. Root build and Vitest runtime success did not cover strict package-test typing: Socket.emit.call forwarding produced TS2554 and TS2556. The saved emitter now has an explicit rest-argument signature. The exact CI workspace typecheck, build, resolver tests, lint, and formatting passed locally. Both contributor and Copilot Case 2 checklists now require that script and npm run l10n:check, and explain why runtime tests are not a substitute.

Avoid OS-dependent UDP refusal signals, wall-clock deadlines, TCP write timing, and random-ID assumptions. Guard DNS isolation and clean up stub startup failures while preserving loopback transport coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ CI passed: standard tier (ready PR)

Commit 7086750, workflow run.

Layer Result Ran or skipped because
Static checks ✅ passed every push
Unit tests ✅ passed every push
npm packages: verify tarballs ✅ passed packages/**, package-lock.json or the CI workflow changed
VSIX: inspect contents ✅ passed ready PR
VSIX: self-test of the checks ⏭️ skipped no change to build/verification/**, build/vite/**, the Vite configs, package-lock.json or the CI workflow
VSIX: activate in VS Code ✅ passed ready PR
VSIX: self-test of the activation check ⏭️ skipped no change to build/verification/**, build/vite/**, the Vite configs, package-lock.json or the CI workflow
Webview host in unbundled extensions ⏭️ skipped no change to packages/vscode-ext-webview/**, package-lock.json or the CI workflow
E2E ⏭️ not available yet arrives with the E2E iteration (full tier)

Full run: add the ci:full-requested label to run every layer once on the current commit.
No full run yet.

Code quality

Check Status How to fix
Localization (l10n) ✅ Passed Nothing to fix
ESLint ✅ Passed Nothing to fix
Prettier formatting ✅ Passed Nothing to fix

Build size

No page, activation, VSIX or other bundle size changed by 5% or 50 KiB or more against main. VSIX: 8.5 MiB.

Artifacts: VSIX | Bundle-reports-39580842714 | L1-manifest-report-39580842714

All graphs (exact bytes)
Graph PR bytes Base bytes Delta bytes Delta %
Collection View page (collectionView) 6,036,286 6,036,286 ✅ 0 0.0%
Document View page (documentView) 4,808,762 4,808,762 ✅ 0 0.0%
Local Quick Start page (localQuickStart) 1,551,408 1,551,408 ✅ 0 0.0%
Atlas Credentials page (atlasCredentials) 1,504,612 1,504,612 ✅ 0 0.0%
Cluster Dashboard page (clusterDashboard) 1,550,863 1,550,863 ✅ 0 0.0%
Activation (mainStartup) 3,447,535 3,447,535 ✅ 0 0.0%
Extension host (main) 9,974,227 9,974,227 ✅ 0 0.0%
Playground worker (playgroundWorker) 5,384,529 5,384,529 ✅ 0 0.0%
VSIX (vsix) 8,940,785 8,940,785 ✅ 0 0.0%
playgroundTsPlugin 6,175 6,175 ✅ 0 0.0%
viewsEntry 1,456,593 1,456,593 ✅ 0 0.0%
viewsAssets 80,340 80,340 ✅ 0 0.0%
editorWorker 259,528 259,528 ✅ 0 0.0%
jsonWorker 389,389 389,389 ✅ 0 0.0%

Packaged bytes. Shared chunks count in every graph that loads them, so rows do not add up.

Updated in place on every run.

@tnaum-ms Tomasz Naumowicz (tnaum-ms) added this to the 1.0.2 milestone Oct 21, 2026
Normalize the forwarded emit result to a boolean because the root non-strict TypeScript context gives Function.call an unchecked return type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tnaum-ms
Tomasz Naumowicz (tnaum-ms) marked this pull request as ready for review October 21, 2026 09:44
CI typechecks package tests with strictBindCallApply, which resolves the overloaded Socket.emit call signature differently from the root build. Give the saved emitter an explicit rest-argument signature so forwarding is valid under both configurations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Require the exact workspace test typecheck instead of inferring it from build or Vitest success. Always run CI's non-mutating localization check, and build workspace outputs before strict test validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tnaum-ms

Copy link
Copy Markdown
Collaborator Author

The static-check failure is fixed and the local verification instructions now match CI.

  • Reproduced TS2554 and TS2556 with the exact npm run typecheck:tests command, then fixed the saved TCP emitter signature for strict package-test typing.
  • Updated both Case 2 checklists to require npm run typecheck:tests and unconditional npm run l10n:check, build before strict test validation, and explain why build/Vitest runtime success is not a substitute.
  • Verified locally: all workspace strict test typechecks, build, resolver tests (84 passed), localization check, formatting, and lint.
  • Latest CI is green, including Static checks, the strict package-test step, and Merge gate: https://github.com/microsoft/vscode-documentdb/actions/runs/39580842714

The earlier local Case 2 result covered the documented commands at the time, but missed the separate strict test-typecheck that CI required. The updated checklist closes that gap.

@tnaum-ms
Tomasz Naumowicz (tnaum-ms) merged commit 509c26e into main Oct 21, 2026
14 checks passed
@tnaum-ms
Tomasz Naumowicz (tnaum-ms) deleted the dev/tnaum/harden-srv-resolver-tests branch October 21, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants