Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 28 additions & 14 deletions docs/learn/dev-file.html
Original file line number Diff line number Diff line change
Expand Up @@ -62,10 +62,11 @@ <h2>AssetCache &middot; <span class="dim">AssetCache.cs</span></h2>
skipped too, and on Linux that is any name starting with a dot, so dot-files and a
<code>.well-known/</code> directory are not served.</li>
<li><b>Refcounting</b> - the snapshot is reference-counted (<code>TryAddRef</code> /
<code>Release</code>). The live reference counts as 2<sup>30</sup> and each lease as 1, so lease
releases alone, even a lease disposed twice, never take the live snapshot to zero. It is actually
freed - descriptors closed - only once a reload or <code>Dispose</code> has dropped the live
reference and the last lease is gone. This is what makes reloads safe (below).</li>
<code>Release</code>). The live reference counts as 2<sup>48</sup> and each lease as 1, so lease
releases alone cannot take the live snapshot to zero: even a lease disposed twice on every request
would take years at a million requests a second. With each lease disposed once, it is freed -
descriptors closed - only once a reload or <code>Dispose</code> has dropped the live reference and
the last lease is gone. This is what makes reloads safe (below).</li>
</ul>
<p class="src"><b>AssetCache.cs</b> · building the snapshot - one descriptor, path and length per file, no bytes read</p>
<pre><code class="language-csharp">public readonly record struct Asset(int Fd, string Path, long Length);
Expand Down Expand Up @@ -105,12 +106,12 @@ <h2>AssetCache &middot; <span class="dim">AssetCache.cs</span></h2>
AttributesToSkip = FileAttributes.ReparsePoint | FileAttributes.Hidden | FileAttributes.System,
};</code></pre>
<p class="src"><b>AssetCache.cs</b> · a lock-free refcount - the snapshot frees only when the last hold drops</p>
<pre><code class="language-csharp">private const int LiveRef = 1 &lt;&lt; 30; // lease releases alone never reach 0
private int _refs = LiveRef; // plus one per lease
<pre><code class="language-csharp">private const long LiveRef = 1L &lt;&lt; 48; // extra releases or leaked leases need years at 1M/s to drain or overflow it
private long _refs = LiveRef; // plus one per lease

internal bool TryAddRef()
{
int r;
long r;
do
{
r = Volatile.Read(ref _refs);
Expand Down Expand Up @@ -139,28 +140,32 @@ <h2>AssetCache &middot; <span class="dim">AssetCache.cs</span></h2>
<h2>StaticAssets &middot; <span class="dim">StaticAssets.cs</span></h2>
<p>A reloadable holder around an <code>AssetCache</code>. <code>Reload</code> builds a fresh snapshot,
swaps it in atomically (<code>Interlocked.Exchange</code>), and drops the live reference on the old
one - which is then freed only once every outstanding lease on it is released. No timer, no grace
window: an in-flight read on the old snapshot can never hit a closed fd.</p>
one. With each lease disposed once, the old snapshot is freed only when the last lease on it is
released, so with no timer and no grace window, an in-flight read on it never hits a closed fd.
<code>Dispose</code> drops the live snapshot the same way, under the same lock as a reload, and is
idempotent; after it <code>Acquire</code> throws <code>ObjectDisposedException</code> and
<code>Reload</code> does nothing, so a reload signalled during shutdown cannot bring the assets back.</p>
<ul>
<li><b>Acquire / Lease</b> - <code>Acquire()</code> returns a <code>Lease</code> (a refcounted
hold) on the live snapshot; the handler holds it for the whole request - lookup → ring read →
send - and disposes it when the response is fully sent. <code>TryAddRef</code> fails if the
snapshot is already releasing, so <code>Acquire</code> simply re-reads the live one. A reload
swaps the live snapshot before it releases the old one, so a refusal from the snapshot that is
still live means <code>Dispose</code> freed it, and <code>Acquire</code> throws
<code>ObjectDisposedException</code> instead of retrying.</li>
still live means extra lease releases drained it, and <code>Acquire</code> throws
<code>ObjectDisposedException</code> instead of retrying, as it does once the assets are disposed.</li>
</ul>
<p class="src"><b>StaticAssets.cs</b> · Acquire leases the live snapshot; Reload swaps a fresh one in atomically</p>
<pre><code class="language-csharp">public Lease Acquire()
{
while (true)
{
AssetCache cache = Volatile.Read(ref _cache);
AssetCache? cache = Volatile.Read(ref _cache);
ObjectDisposedException.ThrowIf(cache is null, this); // disposed
if (cache.TryAddRef())
{
return new Lease(cache);
}
// refused: swapped out by a reload (re-read the live one), or freed by Dispose if still live
// refused: swapped out by a reload (re-read the live one), or freed under it if still live
ObjectDisposedException.ThrowIf(cache == Volatile.Read(ref _cache), this);
}
}
Expand All @@ -169,16 +174,25 @@ <h2>StaticAssets &middot; <span class="dim">StaticAssets.cs</span></h2>
{
lock (_reloadLock)
{
if (_cache is null) return; // disposed: nothing to reload
AssetCache fresh;
try { fresh = new AssetCache(_root); }
catch (Exception e)
{
Console.Error.WriteLine($"[ioxide] asset reload failed, keeping current snapshot: {e.Message}");
return;
}
AssetCache old = Interlocked.Exchange(ref _cache, fresh);
AssetCache old = Interlocked.Exchange(ref _cache, fresh)!;
old.Dispose(); // drops the live ref; old frees once every outstanding lease releases
}
}

public void Dispose()
{
lock (_reloadLock)
{
Interlocked.Exchange(ref _cache, null)?.Dispose(); // idempotent; a reload in progress finishes first
}
}</code></pre>

<h2>AssetReader &middot; <span class="dim">AssetReader.cs</span></h2>
Expand Down
3 changes: 2 additions & 1 deletion docs/learn/files.html
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,8 @@ <h2>Atomic reloads</h2>
<p>Snapshots are immutable; deploys swap them whole. <code>StaticAssets.Reload()</code>
builds a fresh <code>AssetCache</code>, swaps it in atomically, and drops its reference to
the old one, whose descriptors close when the last lease on it is disposed - a request always
sees one whole snapshot, and a read in flight never meets a closed descriptor. If the rebuild
sees one whole snapshot, and with each lease disposed once, a read in flight never meets a closed
descriptor. If the rebuild
fails, the current snapshot stays. Wire it to SIGHUP and a deploy is
<code>kill&nbsp;-HUP&nbsp;$(pidof&nbsp;server)</code>:</p>
<pre><code class="language-csharp">PosixSignalRegistration.Create(PosixSignal.SIGHUP, ctx =&gt;
Expand Down
9 changes: 4 additions & 5 deletions src/clients/ioxide.file/AssetCache.cs
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,8 @@ public sealed class AssetCache : IDisposable

private readonly Dictionary<string, Asset> _assets;
private readonly SafeFileHandle[] _handles;
// The live reference counts as 2^30, so lease releases alone, even a double one, never reach 0.
private const int LiveRef = 1 << 30;
private int _refs = LiveRef; // plus one per lease
private const long LiveRef = 1L << 48; // extra releases or leaked leases need years at 1M/s to drain or overflow it
private long _refs = LiveRef; // plus one per lease

/// <summary>The absolute root directory the cache was built over.</summary>
public string RootDir { get; }
Expand Down Expand Up @@ -97,13 +96,13 @@ public bool TryGet(ReadOnlySpan<byte> urlPath, out Asset asset)
/// </summary>
internal bool TryAddRef()
{
int r;
long r;
do
{
r = Volatile.Read(ref _refs);
if (r <= 0)
{
return false; // freed, or below zero after a second Dispose - don't resurrect
return false; // freed, or below zero after extra lease releases - don't resurrect
}
}
while (Interlocked.CompareExchange(ref _refs, r + 1, r) != r);
Expand Down
50 changes: 35 additions & 15 deletions src/clients/ioxide.file/StaticAssets.cs
Original file line number Diff line number Diff line change
Expand Up @@ -7,57 +7,67 @@ namespace ioxide.file;
/// </summary>
/// <remarks>
/// To serve safely across a reload, take a <see cref="Lease"/> with <see cref="Acquire"/> and hold
/// it for the whole request (lookup → ring read → send): the old snapshot's descriptors are freed
/// only once every lease on it is released, so an in-flight read can never hit a closed fd.
/// it for the whole request (lookup → ring read → send), and dispose it once: the old snapshot's
/// descriptors are freed only once every lease on it is released, so an in-flight read never hits a
/// closed fd. After <see cref="Dispose"/>, <see cref="Acquire"/>, <see cref="Count"/> and <see cref="RootDir"/>
/// throw and <see cref="Reload"/> does nothing; leases already out keep the last snapshot open until they are released.
/// </remarks>
public sealed class StaticAssets : IDisposable
{
private readonly string _root;
private readonly object _reloadLock = new();

private AssetCache _cache; // swapped atomically by Reload(), read with Volatile
private AssetCache? _cache; // swapped atomically by Reload(), read with Volatile; null once disposed

public StaticAssets(string rootDir)
{
_root = rootDir;
_cache = new AssetCache(rootDir);
}

public int Count => Volatile.Read(ref _cache).Count;
public int Count => Live.Count;

public string RootDir => Volatile.Read(ref _cache).RootDir;
public string RootDir => Live.RootDir;

private AssetCache Live => Volatile.Read(ref _cache) ?? throw new ObjectDisposedException(nameof(StaticAssets));

/// <summary>
/// Lease the live snapshot for the duration of a request. The snapshot it points at stays alive
/// (descriptors open) until the lease is disposed, even across a concurrent <see cref="Reload"/>.
/// Dispose it when the response has been fully sent.
/// (descriptors open) until the lease is disposed, even across a concurrent <see cref="Reload"/>,
/// as long as every lease is disposed once. Dispose it when the response has been fully sent.
/// </summary>
/// <exception cref="ObjectDisposedException">This was disposed and its snapshot freed.</exception>
/// <exception cref="ObjectDisposedException">This was disposed.</exception>
public Lease Acquire()
{
while (true)
{
AssetCache cache = Volatile.Read(ref _cache);
AssetCache? cache = Volatile.Read(ref _cache);
ObjectDisposedException.ThrowIf(cache is null, this);
if (cache.TryAddRef())
{
return new Lease(cache);
}

// Refused. A Reload swaps the live snapshot before releasing the old one, so a refusal from the
// one still live means Dispose freed it and nothing will replace it; otherwise re-read the live one.
// A Reload swaps before it releases, so only the snapshot still live can refuse for good.
ObjectDisposedException.ThrowIf(cache == Volatile.Read(ref _cache), this);
}
}

/// <summary>
/// Rebuild the snapshot from disk and swap it in atomically, then release the old one (it frees
/// once all leases on it are dropped). If the rebuild fails, the current snapshot is kept.
/// Thread-safe.
/// once all leases on it are dropped). If the rebuild fails, the current snapshot is kept. After
/// <see cref="Dispose"/> it does nothing, so a reload signalled during shutdown cannot bring the
/// assets back. Thread-safe.
/// </summary>
public void Reload()
{
lock (_reloadLock)
{
if (_cache is null)
{
return;
}

AssetCache fresh;
try
{
Expand All @@ -69,12 +79,22 @@ public void Reload()
return;
}

AssetCache old = Interlocked.Exchange(ref _cache, fresh);
AssetCache old = Interlocked.Exchange(ref _cache, fresh)!;
old.Dispose(); // drops the live reference; old frees when outstanding leases release
}
}

public void Dispose() => Volatile.Read(ref _cache).Dispose();
/// <summary>
/// Drop the live snapshot, which frees once the leases on it are released. Idempotent; under the reload lock,
/// so a reload in progress finishes first and its snapshot is the one dropped.
/// </summary>
public void Dispose()
{
lock (_reloadLock)
{
Interlocked.Exchange(ref _cache, null)?.Dispose();
}
}

/// <summary>
/// A reference-counted hold on an <see cref="AssetCache"/> snapshot. Look up assets through it
Expand Down
Loading
Loading