[ConstBob]

The cookbook content has moved out of this repo.
Please review the migrated changes here instead:
NVIDIA/cosmos-framework#300

Summary

• Add Cosmos3-Edge TensorRT-Edge-LLM cookbooks (reasoner + policy), v0.10.0+. On-device TensorRT C++ path, not TensorRT-LLM.
• Shared setup in cookbooks/cosmos3/README.md. Policy needs -DBUILD_EXPERIMENTAL_MODELS=ON.
• Follows upstream: https://nvidia.github.io/TensorRT-Edge-LLM/user_guide/examples/vla/cosmos3.html

Testing

Tested on an x86 developer GPU (RTX PRO 6000, SM120) with TensorRT-Edge-LLM v0.10.0. Both added notebooks in this PR were run top-to-bottom (run_with_trt_edge_llm.ipynb and run_policy_with_trt_edge_llm.ipynb).

• Reasoner: export --task reasoning → llm_build + visual_build → llm_inference on robot_153.jpg. Pipeline finished.
• Policy: export --task policy → cosmos3_policy_build → cosmos3_policy_inference. [1,16,10], finite=true, ~195 ms (runtime smoke).

On this SM120 box the pipeline succeeded for both notebooks (export, engine build, and C++ inference finished; policy returned [1,16,10] with finite=true). Reasoner captions were not usable (text did not match robot_153.jpg); policy was only checked as a finite action tensor, not DROID quality.

Likely cause: TensorRT engines and ViT/FMHA kernels are SM-specific. The device used for this testing is RTX PRO 6000. Official Edge-LLM targets are Thor / Spark / Orin, etc. I do not have an Official Edge-LLM device (Jetson Thor / DRIVE Thor / DGX Spark / Orin), so caption or policy quality on those SKUs are not checked. x86 is Developer-only; on this SM120 box reasoner captions were not usable and should not be treated as gold. Re-run is needed on Thor, or Spark) before treating quality as validated.

The above issue has been resolved (please refer to the comment threads below).

[nvluxiaoz]

Can we add some accuracy benchmark instruction or some references of whether this is correct?

[ConstBob]

Requesting reviews from @nvluxiaoz @qmiao-hub @rickzw so that we can merge this PR. Thanks!

[nvluxiaoz]

LGTM as long as ever has validated.

[ever-wong]

Hi @ConstBob, could you merge the PR after rebasing and testing it with the latest TRT Edge-LLM release? Thanks!

[ever-wong]

I think we need to add --viewPoint concat_view here to match the concatenated DROID camera input. @nvluxiaoz to confirm.

[ConstBob]

Hi @ever-wong , thank you for the comments.

Rebased onto current main. The notebooks now pin TensorRT-Edge-LLM v0.11.0, and the policy command passes --viewPoint concat_view for the 640×540 DROID concat.

I built the v0.11.0 cosmos3_policy_inference binary. Inference does not load with the existing v0.10.0 engines: the v0.11.0 runtime requires attention_position_ids in und_prefill, and vae_encoder builder_config.content_height / content_width. The v0.11.0 tensorrt-edgellm-export --task policy path (make_vae_encoder_config) still writes only height / width / num_frames, and cosmos3_policy_build copies that config through. The experimental builder in the same tag does write content_height / content_width.

Could you confirm whether this cookbook should stay on the VLA-guide export path until that contract matches, or switch to the experimental builder? I have not re-exported or rebuilt engines on v0.11.0 yet.