Skip to content

fix(video): add an optional frame cap for universal analysis - #225

Draft
vieneny wants to merge 1 commit into
google:mainfrom
vieneny:codex/universal-video-frame-cap
Draft

vieneny wants to merge 1 commit into
google:mainfrom
vieneny:codex/universal-video-frame-cap

Conversation

@vieneny

@vieneny vieneny commented Oct 29, 2026

Copy link
Copy Markdown

Universal video analysis can send more extracted images than a model endpoint accepts in one request. Add an optional VideoAnalyzerConfig.max_total_frames cap after regular and action-adjacent keyframes are merged. The default is None, preserving existing frame selection.

When configured, selection keeps the extracted endpoints, prioritizes complete before/after pairs around recent actions, and fills remaining slots across the timeline. It preserves frame timestamps, image bytes, and the original recording. The model prompt explicitly describes the sampling limit so missing instants are not treated as observed. Native video analysis is unchanged.

Validation on Python 3.12:

  • 38 focused sampling and universal-analyzer tests passed, including concurrent chunks.
  • ruff format --check ., ruff check ., scripts/quality_ratchet.py, and pyright --project pyright-core.json passed.
  • The full deterministic suite, with isolated runtime state and external-network/device guards, reported 2,463 passed, 86 failed, and 9 skipped. The unchanged base 351ca8422f7b5b54e80a9c1ce03a222e02415b6b under the same constraints reported 2,439 passed, 85 failed, and 9 skipped. All 85 baseline failures also occurred on the candidate. The additional test_perception_node_async_offloading failure concerned a missing background-persisted image record; it passed once in isolation on each revision without code changes. The original full-suite failure remains recorded; this is not a claim that the complete suite passes.

The tests use synthetic frame bytes and mocked model calls. No device or live model acceptance is claimed for this public candidate.

@google-cla

google-cla Bot commented Oct 29, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant