feat: add Gemini agentic video adapter - #1608
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. OpenSSF Scorecard
Scanned Files
|
Execution receipt — security workflow diagnosisVerified failing run: Security Scan The container build stops before Trivy executes because: The PR changes only:
Interpretation: the failed security workflow is a repository package/lock synchronization blocker, not a vulnerability finding from this adapter. Trivy was skipped because the image never built. Action: recorded here rather than mixing an unrelated lockfile rewrite into this focused PR. The next safe repair is to reconcile the web workspace lockfile in its canonical dependency-maintenance work, then rerun Security Scan. Other exact-head checks currently verified successful: CI, Coverage, CodeQL, Secret Scan, and Dependency Review. |
Execution receipt — Gemini 3.8 adoption boundary and benchmark scopeNew verified fact (2026-09-02): Google released Gemini 3.8 Flash, and Vercel AI Gateway exposes it as Separate verified fact: Google's current agentic-video guide still documents Sources:
Decision: TEST. Keep this draft's default at Re-scoped value of this PR: retain it as the explicit agentic-processing/receipt adapter and use it to benchmark against the main gateway extractor on the same videos:
The adapter already permits a model override through |
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
There was a problem hiding this comment.
🟡 Changes recommended
The locked SDK version loses response text, and malformed video URIs still reach the provider.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds an opt-in Gemini Interactions API adapter for agentic and static video analysis.
Changes:
- Adds video input and execution receipt models.
- Supports mixed processing modes and asynchronous provider calls.
- Tests request construction, receipts, and basic validation.
File summaries
| File | Description |
|---|---|
src/integration/gemini_agentic_video.py |
Implements the Gemini video adapter. |
tests/unit/test_gemini_agentic_video.py |
Adds adapter unit tests. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| usage = getattr(response, "usage", None) | ||
| total_tokens = getattr(usage, "total_tokens", None) | ||
| return AgenticVideoReceipt( | ||
| output_text=str(getattr(response, "output_text", "")), |
There was a problem hiding this comment.
@copilot Fix the code for all comments in this review comment.
When a review comment includes a suggested change, apply the suggestion exactly.
Do not make changes beyond what is described in the linked review comment.
There was a problem hiding this comment.
Addressed in commit 18c876a: the adapter now extracts text from the locked SDK's outputs response shape, tests use the installed Interaction model, and unsupported/non-YouTube video URIs are rejected before provider calls.
There was a problem hiding this comment.
@copilot Fix the code for all comments in this review comment.
When a review comment includes a suggested change, apply the suggestion exactly.
Do not make changes beyond what is described in the linked review comment.
There was a problem hiding this comment.
Implemented in commit d021557: google-genai now has a 2.21.0 floor, uv.lock resolves 2.21.0, and the test constructs the installed SDK 2.21 Interaction model.
| for video in videos: | ||
| if not video.uri.strip(): | ||
| raise ValueError("Video URI must not be empty") | ||
| item = { | ||
| "type": "video", | ||
| "uri": video.uri, |
There was a problem hiding this comment.
@copilot Fix the code for all comments in this review comment.
When a review comment includes a suggested change, apply the suggestion exactly.
Do not make changes beyond what is described in the linked review comment.
There was a problem hiding this comment.
Addressed in commit 18c876a. URI parsing now rejects malformed or unsupported references before request construction, and regression coverage includes malformed URIs with a nonblank prompt.
There was a problem hiding this comment.
@copilot Fix the code for all comments in this review comment.
When a review comment includes a suggested change, apply the suggestion exactly.
Do not make changes beyond what is described in the linked review comment.
There was a problem hiding this comment.
Implemented in commit d021557: URI validation remains covered alongside the refreshed SDK 2.21.0 adapter test; targeted tests pass.
|
Please add a See governance: #898 |
groupthinking
added
high-priority
|
Warning The Automated checkpoint update (pr-iteration-loop)This PR was blocked on the Verification on the new head:
I also opened a canonical tracking issue, #aw_gemvid1, since the required "Canonical issue and evidence" check needs a Closes #aw_gemvid1 Canonical issueCloses #aw_gemvid1 — tracks the opt-in Gemini agentic video adapter. OutcomeAdds an opt-in adapter for Google's agentic video understanding path via the Gemini Interactions API. Callers can request Risk
Production evidenceNo production deployment or default-path switch is included; this is an additive, opt-in module not yet wired into the live paste-URL flow. No further action from this automation this run — please review and merge/close as appropriate. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "fonts.googleapis.com"See Network Configuration for more information.
|
Co-authored-by: groupthinking <154503486+groupthinking@users.noreply.github.com>
Co-authored-by: groupthinking <154503486+groupthinking@users.noreply.github.com>
What changed
Adds an opt-in adapter for Google's agentic video understanding path via the Interactions API.
agenticandstaticprocessing per video, including mixed requestsgemini-3.7-flashWhy
Google AI Studio announced agentic video understanding on September 1, 2026. The capability actively inspects transcript, frames, and audio with adaptive frame sampling, which maps directly to EventRelay's video-to-event pipeline and the See → Act → Record → Review loop.
Sources:
Verification
python -m compileall src/integration/gemini_agentic_video.py tests/unit/test_gemini_agentic_video.py— passedNo module named pytest); repository CI remains the merge gateNo production deployment or default-path switch is included.