[ConstBob]

Summary

Add Cosmos3-Edge to the TensorRT-LLM Reasoner cookbook.

Cosmos3-Nano and Cosmos3-Super stay on the pinned nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc22 image. That image does not include Cosmos3EdgeForConditionalGeneration, so Cosmos3-Edge is documented as a single-GPU server started from a TensorRT-LLM source build that includes NVIDIA/TensorRT-LLM#20307.

Changes

• cookbooks/cosmos3/README.md
  • Separates the Nano/Super release-image path from the Edge source-build path.
  • Adds the single-GPU Cosmos3-Edge trtllm-serve command on port 8001.
• cookbooks/cosmos3/reasoner/README.md
  • Lists Cosmos3-Edge in the TensorRT-LLM quickstart and notebook walkthrough.
• cookbooks/cosmos3/reasoner/run_with_tensorrt_llm.ipynb
  • Keeps the Nano and Super Docker cells on release:1.3.0rc22.
  • Adds an Edge cell that starts trtllm-serve nvidia/Cosmos3-Edge from the current environment and exits with a clear message when trtllm-serve is unavailable.
  • Updates the readiness cell to follow either the Docker container log or the Edge server log, and to ignore a missing Docker daemon when Edge is running outside Docker.

Testing

• Validated the notebook JSON and ran bash -n on every bash cell.
• Exercised the readiness cell locally with a stub /health server:
  • the Docker container path reaches ready;
  • the Edge process path reaches ready;
  • an exited Edge process fails instead of waiting until timeout.
• Ran the Edge server and readiness cells on one L40S with the nvidia/Cosmos3-Edge checkpoint, using a TensorRT-LLM environment that includes the Edge architecture.
  • trtllm-serve nvidia/Cosmos3-Edge --host 0.0.0.0 --port 8001 --max_num_tokens 32768 started successfully.
  • /health became ready.
  • /v1/models returned Cosmos3-Edge.
  • An OpenAI-compatible image request with assets/robot_153.jpg returned a non-empty caption.

The run used max_tokens=128 for the image request; the cookbook example keeps max_tokens=4096. Nano and Super launch commands are unchanged and were not rerun.

[ishovkun]

Please check that this PID still belongs to the Edge server before killing it. A stale PID file can point to an unrelated process after the PID gets reused. Also, wait runs in a new %%bash shell here, so it cannot wait for the server started by a previous cell. We need to wait for that server to exit before reusing port 8001.

[ConstBob]

The launch cell now reads /proc/<pid>/cmdline and kills the recorded PID only when it is still trtllm-serve nvidia/Cosmos3-Edge. A stale PID that belongs to another process is left alone.

wait cannot reap a background process started by an earlier cell, so the new cell polls until that process exits. It also refuses to start while port 8001 still has a listening socket.

[ishovkun]

The shutdown cell at the end only runs docker rm -f cosmos3-reasoner-trtllm, so it leaves this Edge server running and holding GPU memory. Please add an Edge shutdown path that stops the server started by this notebook and waits for it to exit.

[ConstBob]

The shutdown cell now stops the Edge server as well as the Nano/Super container. It uses the same PID check, so it only signals trtllm-serve nvidia/Cosmos3-Edge, and it waits until that process exits. If Docker is unavailable, the cell skips the container removal instead of failing.

[ishovkun]

Please make the image-pull cell Nano/Super-only, or tell Edge users to skip it. It currently runs docker pull unconditionally, so following the Edge setup on a machine without Docker fails even though Edge runs from source.

[ConstBob]

The image-pull cell is Nano/Super-only. If Docker is missing or the daemon is down, it prints a skip message and exits 0, so an Edge-only machine can run the notebook without pulling release:1.3.0rc22.

[ishovkun]

Could we use the same request as the Cosmos Framework Edge example? Framework asks for a one-sentence description, limits generation to 1024 tokens, and disables sampling. Here we ask for a detailed caption, allow 4096 tokens, and leave sampling enabled.

I ran TensorRT-LLM with the Framework settings and got essentially the same final caption. Matching the settings would make the examples easier to compare.

[ConstBob]

The TensorRT-LLM quickstart now uses the same request as the Cosmos Framework Edge example: “Describe what is happening in this image in one sentence.”, max_tokens=1024, and temperature=0. The later notebook examples are unchanged because they cover the other tasks.

[pengcuo]

Adding to @ishovkun's comments (PID reuse and wait in a fresh shell, missing Edge shutdown, unconditional docker pull, request parity), all of which I agree with. Two more:

1. The Edge path depends on an unmerged TensorRT-LLM PR. NVIDIA/TensorRT-LLM#20307 is still open (head ConstBob:cosmos3-edge-reasoner, mergeable state blocked as of today), and no release up to v1.4.0rc1 contains Cosmos3EdgeForConditionalGeneration. The README tells readers to install "from a checkout that includes PR 20307" without naming a branch or commit, so the instructions cannot be followed from upstream main or from any image. Either hold the Edge section until it ships in a release image (then pin that image like Nano/Super), or name the exact fork branch and commit and label the section as a preview.
2. The Edge launch cell runs python -m pip install opencv-python-headless with whatever python is first on the %%bash PATH, which is not necessarily the environment that owns trtllm-serve (the Nano/Super cells install it inside the container, where that is unambiguous). Install into the interpreter that owns the entry point, e.g. "$(dirname "$(command -v trtllm-serve)")/python" -m pip install ..., or move it to the prerequisites.

The rest of the readiness cell reads correctly to me (Docker-absent branch, Edge PID liveness check).

[pengcuo]

NVIDIA/TensorRT-LLM#20307 is open as of 2026-10-21 (head ConstBob:cosmos3-edge-reasoner), so there is no checkout on main or in a release that includes it. Name the branch and commit readers should build, or hold this section until the PR lands in a release image.

[ConstBob]

TensorRT-LLM #20307 is now merged as 39a44fab. The docs tell readers to build upstream main at or after that commit.

Release images through v1.4.0rc1, including release:1.3.0rc22, still do not include Cosmos3EdgeForConditionalGeneration, so Edge stays a source build. Nano and Super stay on the pinned release:1.3.0rc22 image.

[pengcuo]

python here is whatever the notebook's %%bash PATH resolves, not necessarily the interpreter that owns trtllm-serve. Install into that one ("$(dirname "$(command -v trtllm-serve)")/python" -m pip install ...) or drop the line and rely on the prerequisite in the shared README.

[ConstBob]

The Edge cell now installs opencv-python-headless with the Python interpreter next to trtllm-serve ("$(dirname "$(command -v trtllm-serve)")/python" -m pip install ...), instead of whatever python is first on the notebook PATH.