Vired fixes local artifacts in AI-generated (or any) video by using the object's own good frames as the reference, and leaves everything else bit-identical.
The problem it solves:
frame 20–21 object is correct
frame 22–30 object is malformed <- repair only this
frame 31 object is correct again
Select the object once, mark the broken span, pick frame 20 or 31 as the reference, and repair. Background, text, branding, camera move, other characters and existing animation are untouched by construction: outside the tracked mask the output frame is the input frame, byte for byte, and the pipeline verifies that before returning anything.
Under the hood this is a thin V2 layer over CleanPlate
(cloned into cleanplate/), which provides the tracking (SAM 2) and the removal
fill (ProPainter). Vired adds the reference workflow, the repair backends, the
3-step UI and the test suite.
chmod +x setup-mac.sh
./setup-mac.shInstalls Homebrew if needed, Python 3.11, ffmpeg, clones CleanPlate, sets up a venv, downloads the models, and starts the app. Then open:
http://127.0.0.1:7860
chmod +x setup-linux-rtx2060.sh
./setup-linux-rtx2060.shcleanplate/.venv/bin/python vired_app.py1. Select. Upload a clip (or open a demo shot) and click the broken object;
Track selection propagates the selection through the clip. One click on a small
or high-detail object often selects only part of it — add a second point on a
missed part; two or three well-separated clicks describe an object far better than
one. Mark the broken span with the two frame sliders.
2. Reference. Pick a nearby frame where the object looks right — the thumbnails either side of the range, the arrow buttons, or the current frame. The reference frame's object pixels, plus some context around them, are what gets copied over the broken frames.
3. Repair. Choose a mode and preview it on one frame before running the range:
| Mode | What it does |
|---|---|
| Match reference | Warps the reference object onto each broken frame (optical flow + mask geometry, OpenCV only — no learned models) and blends it inside the tracked mask. |
| Remove artifact | CleanPlate's ProPainter fill, scoped to the broken range: the object is removed and the background is reconstructed. |
Then Export repaired video (MP4): source fps, dimensions and audio are preserved
(audio is remuxed from the original file with ffmpeg after the picture is repaired).
final_frame = repaired_region * mask + original_frame * (1 - mask)
vired/video/masks.py enforces this at composite time (the original pixels are
copied back outside the mask, not blended with weight zero), and
vired/pipeline.py re-verifies it on every run: outside the permitted region, the
frames must be bit-identical — the run raises rather than returning something
quietly wrong. For Remove artifact the permitted region is the grown hole
(matte plus the hole dilation), because a pixel-tight hole leaves a rim of the
subject behind; for Match reference it is exactly the tracked mask.
Every run prints what it checked: how many pixels changed, per frame, and where the changes were allowed.
The same pipeline the app uses:
V=cleanplate/.venv/bin/python
$V vired_cli.py ingest --video myclip.mp4 # into the shots store
$V vired_cli.py track --shot myclip --point 214,180 \
--at 21:220,185:+ --at 21:180,150:+
$V vired_cli.py repair --shot myclip --start 22 --end 30 --ref 20 \
--out myclip_repaired.mp4
$V vired_cli.py repair --shot myclip --start 22 --end 30 --ref 20 \
--mode remove --out myclip_removed.mp4V=cleanplate/.venv/bin/python
$V tests/test_units.py # masks, contract, transforms - seconds, no models
$V tests/test_e2e_synth.py # ingest -> track -> repair -> export on a clip
# with ground truth, audio, and a known broken range
$V tests/synth.py --out /tmp/clip # just build the test clip, to poke atThe end-to-end test builds a synthetic clip (textured background, moving object,
corrupted frames 18–26, ground truth for every frame), runs the whole pipeline,
scores the repair against ground truth, and checks the export's fps, size and
audio. tests/track_probe.py reports how click placement changes the tracked
mask — the tool behind "add a second click".
vired/
├── repair/
│ ├── base.py # RepairBackend, the contract, the guards
│ ├── reference_warp.py # Match reference (flow + geometry + blend)
│ └── propainter.py # Remove artifact (CleanPlate's ProPainter, range-scoped)
├── tracking/sam2.py # thin adapter over CleanPlate's SAM 2 predictor
├── video/
│ ├── frames.py # ingest any video; shots, fps, size, audio
│ ├── masks.py # mask geometry + the mask-safe composite
│ └── encode.py # mp4 export with the source audio remuxed back
├── pipeline.py # the flow as plain functions (no Gradio)
└── ui/app.py # the 3-step app
vired_app.py # entry point
vired_cli.py # terminal driver
tests/ # unit + synthetic end-to-end
RepairBackend.repair(frames, masks, reference_frame, reference_mask, start, end)
is the seam. A future VideoDiffusionRepairBackend implements the same method and
the UI, CLI, guards and tests work unchanged. Frames a backend cannot repair are
skipped untouched and reported, never guessed at.
- Placement, not re-posing. The reference warp uses a similarity/affine transform. It handles motion, modest rotation and scale; it cannot re-pose a hand or rebuild a shape that has no good frame anywhere in the clip. Those cases want the planned diffusion backend, not a smarter warp.
- The malformed mask's size is not trusted for scaling the reference (a bloated broken blob would otherwise be "fixed" by inflating the object). Scale is only taken from motion, and only within a modest band.
- Coverage is the main failure mode. If the tracked mask misses part of the object, that part is outside the permitted region and stays broken. Fix the mask, not the repair: add clicks and track again.
- Short clips. Uploads are capped at 480 frames / 1280 px wide; SAM 2 on a 16 GB Mac is the constraint.
Remove artifactis the slow stage (~2 s/frame at 960 px) and the most memory-hungry; it runs ProPainter in a child process under CleanPlate's memory guard, which kills the job rather than let the machine swap itself to death.
Vired's own code: MIT (this repo). The engines it calls are not bundled:
CleanPlate (MIT) is cloned on setup; ProPainter and MatAnyone are fetched by
CleanPlate's download script and are S-Lab License 1.0, non-commercial —
see cleanplate/THIRD_PARTY.md before any commercial use. Demo footage from
CleanPlate is (CC) Blender Foundation | mango.blender.org.