[MDA2AV]

Follow-up to #483, from a review of it after merge. Both problems below are reproduced on main 58620b5.

What was wrong

Reload after Dispose brought the assets back, and leaked the old snapshot

• StaticAssets.Dispose ran outside the reload lock and left _cache pointing at the snapshot it dropped, and Reload never checked for disposal.
• A reload signalled during shutdown (the documented SIGHUP wiring, say) therefore swapped in a fresh snapshot, and dropped the old one's live reference a second time.
• Afterwards, Acquire handed out leases again, and the snapshot a request's lease still held was never freed.
• Repro: hold a lease, Dispose, Reload, release the lease. The old descriptor stays open, and Acquire returns a lease.
• Dispose was not idempotent either. Each call subtracted the live reference again, the 32-bit count wrapped on the fourth call, and Acquire then leased the freed snapshot.

The 2^30 cushion ran out

• A lease disposed twice takes one hold off the live count for good, and a double dispose written into a code path repeats on every request.
• After 2^30 of them (about 18 minutes at a million requests a second) the live snapshot reached 0. Every descriptor closed under the leases still reading, which is the file: a lease disposed twice frees the live snapshot, so its descriptors close and Acquire spins until the next Reload #481 hazard, just later, and Acquire threw on assets nobody disposed.
• Repro: keep one lease out, then make 2^30 + 1 double disposes (21 s). The held lease's descriptor is closed, and Acquire throws.
• Leaked leases went the other way: 2^30 of them overflowed the count into negatives, which Acquire refuses. Main before file: a lease disposed twice no longer frees the live snapshot, and Acquire after Dispose throws instead of spinning (#481) #483 took 2^32.

The fix

• Dispose swaps the snapshot out under the reload lock (_cache becomes null) and drops its live reference once.
  • A second Dispose finds nothing to drop.
  • A reload already in progress finishes first, and its snapshot is the one dropped.
  • Reload does nothing once the assets are disposed.
  • Acquire throws ObjectDisposedException. Its null check takes the place of the null check the JIT already did.
• The count is a long, and the live reference counts as 2^48. A double dispose on every request, or leaked leases, would take years at a million a second to reach either end. The Interlocked operations are the same single instructions on x64 and arm64.
• Acquire keeps file: a lease disposed twice no longer frees the live snapshot, and Acquire after Dispose throws instead of spinning (#481) #483's check, and a refusal from the snapshot that is still live still throws rather than spins.
• The docs and XML comments say "with each lease disposed once" where the guarantee needs it.

Behaviour

• Acquire throws as soon as the assets are disposed, even while leases are out. Before, it kept leasing the dropped snapshot until those leases drained. The leases already out keep their snapshot open until they are released.
• Reload after Dispose does nothing, where before it revived the assets.
• Count and RootDir throw ObjectDisposedException after Dispose. Before, they returned the dropped snapshot's values.
• Dispose can be called any number of times.

API

None. Behaviour only, as above.

Tests

New in the File suite (LeaseTests.cs):

• file: after Dispose, Reload does nothing and Acquire throws, and a lease still out keeps its snapshot
• file: Dispose can be called any number of times
  • A lease is held across one to six calls. Releasing it has to free the snapshot, and Acquire has to throw.
  • On main the second call already takes the live reference off again, so the lease's release never frees the snapshot.
• file: Acquire racing Reload always gets a lease on a whole snapshot
  • Three threads lease while the test reloads 5,000 times.
  • It guards the retry file: a lease disposed twice no longer frees the live snapshot, and Acquire after Dispose throws instead of spinning (#481) #483 left untested: with the refusal check made unconditional, 979 Acquire calls threw.
• file: Dispose racing Reload leaves no descriptor open and nothing to lease
  • 300 rounds of a thread reloading while the test disposes. No reload may throw, Acquire has to throw, and no descriptor may stay open on the file.
  • With Dispose outside the reload lock, the reload throws NullReferenceException in the first round.

file: after Dispose, Reload does nothing... also checks that Count and RootDir throw.

Against main's ioxide.file (the tests only):

FAIL  file: after Dispose, Reload does nothing and Acquire throws, and a lease still out keeps its snapshot: expected ObjectDisposedException (StaticAssets), but nothing was thrown
FAIL  file: Dispose can be called any number of times: after 2 Dispose calls the snapshot kept fd 71 open past its last lease
10 passed, 2 failed, 0 skipped

With the fix:

• File: 13 passed.
• E2E: 453 passed.
• Http: 95 passed.

The 2^48 cushion has no test, since it would take 2^48 iterations to reach. The repro above (2^30 + 1 double disposes) shows the old limit.

Bench

None. Acquire, TryAddRef and Release run the same instructions on a long. The only addition is a null check that replaces the JIT's own.