Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions Tests/Integration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,21 @@ Adding the dry-run checks, on a machine you can throw away:
.\Tests\Integration\Invoke-IntegrationTests.ps1 -Ephemeral
```

The full suite runs in Windows Sandbox. Open `Sandbox\Winnow-Tests.wsb`: it
maps the repository read-only, maps `Sandbox\results` writable, installs Pester,
and runs everything inside the sandbox. Edit both `HostFolder` paths in that file
if the repository is not at the default path.
The full suite runs in Windows Sandbox. The simplest way is the launcher, from a
normal (non-elevated) PowerShell at the repository root:

```powershell
.\Tests\Integration\Sandbox\Invoke-SandboxRun.ps1 -CloseWhenDone
```

It writes a sandbox configuration for wherever this clone lives, starts Windows
Sandbox, waits for the suite inside it to finish, prints the result, and exits
`0` when everything passed, `1` when a test failed, and `2` when the run did not
start or finish. The repository is mapped read-only and only `Sandbox\results`
is writable, so the mutating tests can only change the disposable sandbox.

`Sandbox\Winnow-Tests.wsb` does the same by hand: open it to start a run. Its
two `HostFolder` paths are absolute, so edit them to point at your clone first.

**Windows Sandbox has to be enabled first.** It ships with Windows 11 Pro and
Enterprise but is off by default. From an elevated PowerShell:
Expand Down
160 changes: 160 additions & 0 deletions Tests/Integration/Sandbox/Invoke-SandboxRun.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
<#
.SYNOPSIS
Runs the full integration suite, mutating tests included, in Windows Sandbox
and prints the result.

.DESCRIPTION
The committed Winnow-Tests.wsb has to carry absolute HostFolder paths, so it
only works on a machine where the repository sits at exactly that path. This
script writes an equivalent .wsb for wherever this clone lives, starts
Windows Sandbox with it, waits for the harness inside the sandbox
(Start-SandboxRun.ps1) to finish, and prints integration-summary.json.

The repository is mapped read-only and only Tests\Integration\Sandbox\results
is writable, exactly as in Winnow-Tests.wsb. Everything the mutating tests
change happens inside the disposable sandbox.

Only one sandbox can run at a time, and a sandbox started while the previous
one's VM is still shutting down can boot without its mapped folders, in which
case the harness never runs. So this refuses to start while a sandbox window
is open, and waits for a closing VM to exit before launching.

.PARAMETER TimeoutMinutes
How long to wait for the suite to finish once the harness has started.

.PARAMETER CloseWhenDone
Close the sandbox once the results are in. By default it is left open so its
console can still be read; close it before running this again.

.OUTPUTS
Exit code 0 when every test passed, 1 when any test failed, 2 when the
harness did not start or did not finish in time.

.EXAMPLE
.\Tests\Integration\Sandbox\Invoke-SandboxRun.ps1 -CloseWhenDone
#>
[CmdletBinding()]
param(
[int]$TimeoutMinutes = 30,
[switch]$CloseWhenDone
)

$ErrorActionPreference = 'Stop'

$sandboxExe = Join-Path $env:WINDIR 'System32\WindowsSandbox.exe'
if (-not (Test-Path -LiteralPath $sandboxExe)) {
throw 'Windows Sandbox is not enabled. Enable the Containers-DisposableClientVM optional feature and restart; see Tests\Integration\README.md.'
}

$sandboxDir = $PSScriptRoot
$repoRoot = (Resolve-Path (Join-Path $sandboxDir '..\..\..')).Path
$resultsDir = Join-Path $sandboxDir 'results'
if (-not (Test-Path -LiteralPath $resultsDir)) {
New-Item -ItemType Directory -Path $resultsDir -Force | Out-Null
}

$windowProcesses = @('WindowsSandboxRemoteSession', 'WindowsSandboxServer')
if (@(Get-Process -Name $windowProcesses -ErrorAction SilentlyContinue).Count -gt 0) {
throw 'A Windows Sandbox is already running. Close it, then run this again.'
}

# A VM that is still shutting down has no window left, only its memory process.
$teardownDeadline = (Get-Date).AddMinutes(10)
if (@(Get-Process -Name 'vmmemWindowsSandbox' -ErrorAction SilentlyContinue).Count -gt 0) {
Write-Host 'Waiting for the previous sandbox to finish shutting down...'
while ((Get-Date) -lt $teardownDeadline -and @(Get-Process -Name 'vmmemWindowsSandbox' -ErrorAction SilentlyContinue).Count -gt 0) {
Start-Sleep -Seconds 5
}
Start-Sleep -Seconds 30
}

$escape = { param($text) [System.Security.SecurityElement]::Escape($text) }
$wsb = @"
<Configuration>
<MappedFolders>
<MappedFolder>
<HostFolder>$(& $escape $repoRoot)</HostFolder>
<SandboxFolder>C:\Winnow</SandboxFolder>
<ReadOnly>true</ReadOnly>
</MappedFolder>
<MappedFolder>
<HostFolder>$(& $escape $resultsDir)</HostFolder>
<SandboxFolder>C:\Winnow-Results</SandboxFolder>
<ReadOnly>false</ReadOnly>
</MappedFolder>
</MappedFolders>
<LogonCommand>
<Command>powershell.exe -NoProfile -ExecutionPolicy Bypass -File C:\Winnow\Tests\Integration\Sandbox\Start-SandboxRun.ps1</Command>
</LogonCommand>
<MemoryInMB>4096</MemoryInMB>
<Networking>Enable</Networking>
</Configuration>
"@
$wsbPath = Join-Path $env:TEMP ('Winnow-Tests-{0}.wsb' -f [guid]::NewGuid().ToString('N'))
Set-Content -LiteralPath $wsbPath -Value $wsb -Encoding UTF8

$summaryPath = Join-Path $resultsDir 'integration-summary.json'
$transcriptPath = Join-Path $resultsDir 'sandbox-transcript.log'
$bootstrapPath = Join-Path $resultsDir 'bootstrap-error.json'

function Test-WrittenSince {
param([string]$Path, [datetime]$Since)
(Test-Path -LiteralPath $Path) -and ((Get-Item -LiteralPath $Path).LastWriteTime -gt $Since)
}

try {
$launch = Get-Date
Write-Host "Starting Windows Sandbox for $repoRoot"
Start-Process -FilePath $sandboxExe -ArgumentList ('"{0}"' -f $wsbPath)

# The harness writes its transcript within seconds of the sandbox logging on.
$startDeadline = (Get-Date).AddMinutes(6)
while ((Get-Date) -lt $startDeadline -and -not (Test-WrittenSince -Path $transcriptPath -Since $launch)) {
Start-Sleep -Seconds 10
}
if (-not (Test-WrittenSince -Path $transcriptPath -Since $launch)) {
Write-Host 'The test harness did not start inside the sandbox. Check the sandbox window.' -ForegroundColor Red
exit 2
}
Write-Host 'Harness started. Running the suite...'

$finishDeadline = (Get-Date).AddMinutes($TimeoutMinutes)
while ((Get-Date) -lt $finishDeadline -and -not (Test-WrittenSince -Path $summaryPath -Since $launch)) {
Start-Sleep -Seconds 15
}
if (-not (Test-WrittenSince -Path $summaryPath -Since $launch)) {
Write-Host "No results after $TimeoutMinutes minutes. The transcript is at $transcriptPath." -ForegroundColor Red
exit 2
}
Start-Sleep -Seconds 2

$summary = Get-Content -LiteralPath $summaryPath -Raw | ConvertFrom-Json
Write-Host ''
if ($summary.PSObject.Properties['BootstrapError']) {
Write-Host "The harness failed before the suite ran: $($summary.BootstrapError)" -ForegroundColor Red
exit 2
}

$color = if ([int]$summary.Failed -eq 0) { 'Green' } else { 'Red' }
Write-Host ("Integration suite: {0} passed, {1} failed, {2} skipped, {3} total." -f
$summary.Passed, $summary.Failed, $summary.Skipped, $summary.Total) -ForegroundColor $color
foreach ($failure in @($summary.Failures)) {
Write-Host " FAILED: $($failure.Name)" -ForegroundColor Red
Write-Host " $(($failure.Message -split "`n")[0])"
}
foreach ($block in @($summary.BlockFailures)) {
Write-Host " SETUP FAILED: $($block.Name): $($block.Message)" -ForegroundColor Red
}
if (Test-WrittenSince -Path $bootstrapPath -Since $launch) {
Write-Host " The harness also reported an error after the suite: see $bootstrapPath" -ForegroundColor Yellow
}
Write-Host "Results: $resultsDir"

if ([int]$summary.Failed -eq 0) { exit 0 } else { exit 1 }
}
finally {
Remove-Item -LiteralPath $wsbPath -Force -ErrorAction SilentlyContinue
if ($CloseWhenDone) {
Get-Process -Name $windowProcesses -ErrorAction SilentlyContinue | Stop-Process -Force
}
}
4 changes: 4 additions & 0 deletions Tests/Integration/Sandbox/Winnow-Tests.wsb
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
survive: everything else inside the sandbox is destroyed when the window
closes, including the console output.

Easier: run Invoke-SandboxRun.ps1 next to this file. It writes this same
configuration with the paths of your clone, starts the sandbox, and prints
the result, so nothing here needs editing.

The two HostFolder paths below are the maintainer's local example and are
almost certainly wrong for your machine. They are absolute and machine
specific, so if you cloned this repository you MUST edit both to point at
Expand Down