Skip to content

[python] Read LeRobot video datasets for training - #9757

Merged
JingsongLi merged 10 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/lerobot-video-dataset
Sep 13, 2026
Merged

[python] Read LeRobot video datasets for training#9757
JingsongLi merged 10 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/lerobot-video-dataset

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Purpose

Read video-backed LeRobot v3 table groups through PaimonLeRobotDataset.

Changes

  • decode .video frame descriptors with TorchCodec or PyAV
  • reuse bounded decoder caches per DataLoader worker
  • build a lightweight packet timestamp/keyframe index for cold PyAV random reads, then seek directly to the target GOP
  • support existing Episode, delta-window, padding, transform, and return_uint8 behavior

TorchCodec is preferred; PyAV is the fallback.

Tests

83 LeRobot/video tests passed, including real MP4, partial delta windows, B-frame random rereads, cold random-read decode bounds, optional-dependency gates, and spawn workers.

@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review September 12, 2026 14:01
Comment thread paimon-python/pypaimon/multimodal/lerobot/dataset.py
@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit be77b25 into apache:master Sep 13, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants