Skip to content

README.md

Latest commit

 

History

History
28 lines (19 loc) · 1.55 KB

README.md

File metadata and controls

28 lines (19 loc) · 1.55 KB
Raw

Instruction-only object grounding

This utility applies Qwen2.5-0.5B-Instruct once to each task instruction and stores compact phrases for every explicitly manipulated object. RGB frames, poses, masks, and temporal metadata are not model inputs.

Input can be a JSON list or JSONL. Every record must contain an instruction field by default. Output ending in .json is written as a JSON list for direct Wan use; other extensions are written as JSONL. Run input validation without loading the model:

pip install -r offline_data/object_grounding/requirements.txt
bash offline_data/object_grounding/run.sh \
  --input offline_data/object_grounding/examples/input.example.jsonl \
  --output /tmp/objects.jsonl \
  --dry-run

Run on one GPU:

bash offline_data/object_grounding/run.sh \
  --input /path/to/metadata.jsonl \
  --output /path/to/metadata_with_objects.jsonl

Run on eight GPUs:

NUM_GPUS=8 bash offline_data/object_grounding/run.sh \
  --input /path/to/metadata.jsonl \
  --output /path/to/metadata_with_objects.jsonl \
  --batch-size 384

The script retries malformed responses, validates the minimal paper schema, removes duplicates while preserving order, rejects hallucinated phrases and excluded agents or scene elements, and writes unrecoverable samples to an invalid JSONL file next to the output. Use --instruction-key and --id-key when metadata uses different field names.

The complete system and user messages are kept in prompt.py, following Appendix A.3. Model generation is deterministic with sampling disabled.