Skip to content

Disk-buffered recording and crash recovery - #175

Draft
ruccho wants to merge 3 commits into
mainfrom
feature/disk-buffered-recording
Draft

Disk-buffered recording and crash recovery#175
ruccho wants to merge 3 commits into
mainfrom
feature/disk-buffered-recording

Conversation

@ruccho

@ruccho ruccho commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Closes #39.

Adds an opt-in disk buffer to RealtimeInstantReplaySession. Encoded frames are written to
segment files instead of being held in memory, which lowers memory pressure and makes the
footage leading up to a crash recoverable for bug investigation.

Disabled by default. Enable with:

var options = RealtimeEncodingOptions.Default;
options.DiskBuffer = DiskBufferOptions.Default;

Recover after a crash:

foreach (var session in DiskEncodedFrameBufferRecovery.FindRecoverable())
{
    var path = await session.ExportAsync(seconds: 30);
    session.Delete();
}

MaxDiskUsageBytes is a hard bound

It bounds the whole session directory — manifest, codec configuration, and every segment — and
it is a bound rather than a target. Space is reserved before each write and the oldest segments
are deleted to make room; when the bound still cannot be met the record is dropped rather than
written. The directory never exceeds the limit at any instant, not even transiently between a
write and a subsequent eviction. Segments rotate at video key frames, so discarding one never
leaves a partial group of pictures behind.

Flush policy

The default, DiskBufferSyncMode.OperatingSystem, hands data to the operating system after
every batch and to the storage device only when a segment is closed. Frames therefore survive a
process crash — a native fault, an OOM kill, or an abort — which is what crash recovery targets,
without a device flush per frame. Power loss or a kernel panic loses at most one segment. The
manifest and codec configuration always reach the device immediately.
DiskBufferSyncMode.EveryRecord trades markedly higher flash wear for durability against power
loss.

Crash recovery works on every platform

Codec configuration is written to a dedicated file that is never evicted, and the reader treats
its absence as normal. Recovery therefore succeeds both on platforms that emit configuration
records (Android MediaCodec, Windows Media Foundation, FFmpeg) and on those that carry the
parameter sets inside every sample and emit none (Apple VideoToolbox, WebCodecs).

Records carry a CRC-32 and are only appended, so a file truncated by a crash recovers up to the
last complete record. The live export path and the recovery path share one reader, one
selection, and one muxing helper, so every export exercises the recovery code.

Verification

The storage layer has no dependency on UnityEngine and is exercised outside the Editor:

cd InstantReplay.Externals/src/InstantReplay.DiskBuffer.Tests && dotnet run

Checks cover the record round trip, recovery from truncated and corrupt files, the hard bound
under sustained eviction, survival of codec configuration across full eviction, key-frame
alignment of segments, the manifest, and selection. The final check drives the real platform
encoder, recovers from the files alone, and muxes; on macOS the result is a valid MP4 that
ffprobe and ffmpeg accept.

Design and rationale: docs/disk-buffered-recording.md.

Not yet verified

Not run inside the Unity Editor, and crash recovery has not been exercised on a device.

ruccho and others added 3 commits August 19, 2026 14:52
The previous storage layer kept frame payloads in two append-only files with
a side index. It had two blocking defects: codec configuration was never
written to disk, so a recovered session could not be muxed on Android,
Windows, or Linux; and the payload files were never truncated, so disk usage
grew for the whole session regardless of MaxDiskUsageBytes.

Frames are now written to segment files that rotate at video key frames.
Space is reserved before every write and the oldest segments are deleted to
make room, so MaxDiskUsageBytes is a hard bound on the session directory at
every instant rather than a target. Codec configuration goes to a separate
file that is never evicted, and the reader treats its absence as normal, so
recovery works both on platforms that emit it and on those that do not.

Also addressed:

- Writes move to a dedicated worker thread with a bounded queue, replacing
  synchronous I/O on the encoder thread. No allocation occurs per frame.
- Each record carries a CRC-32, and scanning stops at the first record that
  cannot be complete, so a file truncated by a crash recovers cleanly.
- The flush policy is explicit and documented: data reaches the operating
  system after every batch, which is what survives a process crash, and the
  storage device only at segment boundaries, which keeps flash wear low.
- Recovery no longer deletes the session directory implicitly; Delete is
  separate, so a failed export can be retried.
- Export failures no longer skip FinishVideoAsync and FinishAudioAsync. The
  live and recovery paths share EncodedFrameMuxer and EncodedFrameSelector,
  so both follow the same protocol and select frames identically.
- The inconsistent conditional compilation is gone; the code compiles under
  every supported API compatibility level without directives.
- The manifest records the platform, and the metadata leak on the early
  return paths of BoundedEncodedFrameBuffer is fixed.

The storage layer has no dependency on UnityEngine and is exercised by
InstantReplay.Externals/src/InstantReplay.DiskBuffer.Tests, which covers the
record round trip, recovery from truncated and corrupt files, the hard bound
under sustained eviction, and an end-to-end run that encodes with the real
platform encoder, recovers from the files alone, and muxes a valid MP4.

Design and rationale: docs/disk-buffered-recording.md
Describes enabling the buffer through RealtimeEncodingOptions.DiskBuffer,
recovering a session left behind by an abnormal termination through
DiskEncodedFrameBufferRecovery, and every DiskBufferOptions property.

Spells out the two points that are easy to get wrong: MaxDiskUsageBytes
is a bound rather than a target, so a value close to MaxSegmentBytes
degrades the recording instead of the retention; and the default sync
mode defends against a process crash, not against power loss.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Disk-buffered recording and crash recovery

1 participant