Skip to content

Make the IDR interval configurable - #172

Draft
ruccho wants to merge 4 commits into
mainfrom
feature/configurable-idr-interval
Draft

Make the IDR interval configurable#172
ruccho wants to merge 4 commits into
mainfrom
feature/configurable-idr-interval

Conversation

@ruccho

@ruccho ruccho commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #166.

The interval between IDR (key) frames was decided entirely on the native side, and
inconsistently so:

Platform Before
Android (MediaCodec) KEY_I_FRAME_INTERVAL hardcoded to 1
FFmpeg -force_key_frames expr:gte(t,n_forced*1)
WebCodecs keyframe requested when 1 s elapsed since the previous one
Apple (VideoToolbox) never set — encoder default
Windows (Media Foundation) never set — encoder default

The forced one-second interval exists so that BoundedEncodedFrameBuffer can cut an exported
segment close to the requested duration. Callers that do not use that buffer —
UnboundedRecordingSession, or UniEnc used directly — pay for it without needing it.

Changes

UniEnc.VideoEncoderOptions gains float? IdrIntervalSeconds, in seconds, validated to be
finite and greater than zero when set.

null means the UniEnc default, VideoEncoderOptions.DefaultIdrIntervalSeconds (one second —
the interval every platform effectively used before this branch). It does not mean "defer to
the platform encoder": platform defaults differ per platform and can be far longer, MediaCodec
documents KEY_I_FRAME_INTERVAL as required for video encoders, and WebCodecs has no
GOP-length knob at all, so a genuine "leave it to the platform" case would have had no
consistent meaning. The native layer therefore always receives a concrete value, and each
platform applies it unconditionally.

This is not a breaking change: existing code that constructs VideoEncoderOptions without
setting the property keeps the same one-second interval it had before.

Each platform applies the value through its own knob:

  • VideoToolboxkVTCompressionPropertyKey_MaxKeyFrameIntervalDuration. Also stored on the
    encoder input so the kVTInvalidSessionErr session-recreation path preserves it.
  • MediaCodecKEY_I_FRAME_INTERVAL, now set as a float (accepted since API 25, and
    min_api is 26) so sub-second intervals are expressible.
  • Media FoundationCODECAPI_AVEncMPVGOPSize, converted from seconds to frames using the
    frame rate hint. Applied through a new post-activation hook on Transform::new, best-effort
    so an encoder that does not implement the property still activates.
  • FFmpeg — the -force_key_frames expression.
  • WebCodecs — the per-frame keyframe request threshold.

On the Rust side VideoEncoderOptions::idr_interval_seconds returns a plain f32 with no
default implementation, so an implementor that omits it fails to compile rather than silently
falling back.

RealtimeEncodingOptions.Default sets one second explicitly, since that is InstantReplay's own
choice rather than something to inherit silently from UniEnc.

Note on BoundedEncodedFrameBuffer

GetFramesForDuration can only start a segment at a keyframe, and returns nothing when the
buffer holds none. Raising the IDR interval beyond the retained duration therefore produces an
empty export. Eviction is also per-frame rather than per-GOP, so frames preceding the oldest
surviving keyframe are retained but unusable, and a longer interval leaves a larger share of the
memory budget unusable. This is documented in <remarks> rather than clamped, since clamping
would be a behavioural change.

Testing

cargo check -p unienc_c passes on aarch64-apple-darwin, aarch64-apple-ios,
aarch64-linux-android, x86_64-pc-windows-msvc, x86_64-unknown-linux-gnu and
wasm32-unknown-unknown. cargo check -p unienc --tests passes, cargo fmt --check is clean,
and dotnet build -c Release succeeds for UniEnc (all three target frameworks) and
UniEnc.Example.

The Media Foundation and FFmpeg paths are verified by compilation and review only — no Windows
or Linux runtime was available.

Before merging

The native binaries under Packages/jp.co.cyberagent.instant-replay/UniEnc/Plugins/ must be
rebuilt via the build-unienc.yml workflow (manual dispatch), since VideoEncoderOptionsNative
grew from 16 to 20 bytes and the managed and native sides must be updated together.

ruccho and others added 4 commits August 19, 2026 14:47
The interval between IDR (key) frames was decided entirely on the native
side, and inconsistently so: Android and FFmpeg forced one second,
WebCodecs requested a keyframe every second from the frame loop, while
VideoToolbox and Media Foundation were left at their platform defaults.
Callers that do not rely on the bounded ring buffer, such as
UnboundedRecordingSession or UniEnc used directly, pay for that forced
interval without needing it.

Expose the interval as VideoEncoderOptions.IdrIntervalSeconds, where null
means "leave it at the platform encoder's default". It reaches the native
options struct as a float whose non-positive values act as the sentinel
for the absent case, since Option<f32> has no stable repr(C) layout.

Each platform applies it through its own knob:

- VideoToolbox: kVTCompressionPropertyKey_MaxKeyFrameIntervalDuration
- MediaCodec: KEY_I_FRAME_INTERVAL, now set as a float so sub-second
  intervals are expressible
- Media Foundation: CODECAPI_AVEncMPVGOPSize, converted to frames using
  the frame rate hint, applied best-effort so an encoder that does not
  implement it still activates
- FFmpeg: the -force_key_frames expression, omitted entirely for the
  default case
- WebCodecs: the per-frame keyframe request threshold

RealtimeEncodingOptions.Default keeps the previous one-second interval,
so existing behaviour is unchanged unless the interval is set explicitly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Modelling the interval as optional pushed a "leave it to the platform"
case down to every encoder, and that case had no good answer: MediaCodec
documents KEY_I_FRAME_INTERVAL as required for video encoders, and
WebCodecs has no GOP-length knob at all, so both had to invent a value
anyway. Requiring the interval removes the branch instead of choosing
arbitrarily on each platform.

VideoEncoderOptions.IdrIntervalSeconds becomes a plain float, validated
alongside Width, Height and Bitrate, and every platform now applies it
unconditionally. On the Rust side the trait method returns f32 with no
default implementation, so a missing value is a compile error rather than
a silent fallback. The native struct keeps its f32 field, now without the
sentinel interpretation, so its layout is unchanged.

InstantReplay keeps supplying 1 second: RealtimeEncodingOptions.Default,
UniEncTranscoder, the PersistentRecorder sample and the UniEnc example all
set it explicitly.

BREAKING CHANGE: code that constructs UniEnc.VideoEncoderOptions directly
must now set IdrIntervalSeconds. The struct's zero default is rejected by
Validate(), so leaving it unset throws ArgumentException. Callers going
through RealtimeEncodingOptions.Default are unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… one

Requiring the interval made the property a breaking addition: the struct's
zero default is not a valid interval, so any existing code constructing
VideoEncoderOptions directly would start throwing from Validate().

Make IdrIntervalSeconds nullable again, but give null a definite meaning.
It selects VideoEncoderOptions.DefaultIdrIntervalSeconds — one second, the
interval every platform used before this branch — rather than deferring to
the platform encoder, whose own default differs per platform and can be far
longer. The native side keeps receiving a concrete f32, so no platform code
changes and the struct layout is untouched.

With the default covering them, UniEncTranscoder, the PersistentRecorder
sample and the UniEnc example no longer need to set the interval, so those
three files revert to their original contents.
RealtimeEncodingOptions.Default keeps setting it explicitly, since one
second is InstantReplay's own choice rather than something to inherit
silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the new option to the settings sample and explains what null means:
the UniEnc default of one second, not the platform encoder's own default.

Also notes the interaction with the export path, since an export can only
begin at a key frame and an interval longer than the retained duration
leaves the buffer with none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make IDR interval configurable

1 participant