22
33[ 中文文档] ( README_zh.md )
44
5- This directory contains LIBERO rollout and latency helpers for two policy paths:
5+ This directory provides LIBERO simulation eval and latency scripts for two
6+ policy paths:
67
7- * robot.cpp C++ Policy: loads split GGUF checkpoints through ` model-server ` .
8- * LeRobot Policy: runs the original Python policy as a baseline.
8+ * robot.cpp C++ Policy: loads GGUF through ` model-server ` and runs rollout and
9+ latency checks.
10+ * LeRobot Policy: uses the original Python policy as the baseline.
911
10- ## Layout
12+ ## File Layout
1113
1214``` text
1315eval/libero/
1416├── policy/
1517│ └── model_server.py # LIBERO observation -> C++ policy request adapter
1618├── runners/
17- │ ├── run_model_server.py # C++ Policy LIBERO rollout runner
19+ │ ├── run_model_server.py # C++ policy LIBERO rollout runner
1820│ ├── run_lerobot.py # LeRobot Policy rollout wrapper
19- │ └── latency_lerobot.py # LeRobot Policy latency runner
21+ │ └── latency_lerobot.py # LeRobot policy latency runner
2022├── scripts/
2123│ └── run_model_server.sh # launch/eval convenience wrapper
2224├── utils/
@@ -26,28 +28,28 @@ eval/libero/
2628```
2729
2830Shared action chunk buffering lives in ` robot_client/policy/base_policy.py ` .
29- LIBERO-specific C++ Policy launch, shutdown, and timing helpers live in
31+ LIBERO C++ Policy launch, shutdown, and timing helpers live in
3032` eval/libero/policy/model_server.py ` .
3133
3234## Usage
3335
3436### step0: Environment
3537
36- Create the optional conda environment:
38+ Using a dedicated conda environment is recommended :
3739
3840``` bash
3941conda env create -f eval/libero/environment.yaml
4042conda activate robotcpp-libero
4143```
4244
43- Or install the minimum dependencies in an existing environment:
45+ If reusing an existing environment, install at least :
4446
4547``` bash
4648pip install " cmake<4"
4749pip install --no-build-isolation " hf-libero>=0.1.3,<0.2.0"
4850```
4951
50- If EGL is unavailable on the machine, use OSMesa:
52+ If the machine has no available EGL device, switch to OSMesa:
5153
5254``` bash
5355export MUJOCO_GL=osmesa
@@ -56,8 +58,9 @@ export PYOPENGL_PLATFORM=osmesa
5658
5759### step1: Prepare C++ Policy GGUF
5860
59- For Pi0, use the split GGUF checkpoint from
60- [ ` rrobottt/pi-libero-bf16 ` ] ( https://huggingface.co/rrobottt/pi-libero-bf16 ) :
61+ For Pi0, refer to the GGUF checkpoint
62+ [ ` rrobottt/pi-libero-bf16 ` ] ( https://huggingface.co/rrobottt/pi-libero-bf16 ) .
63+ The C++ Policy uses converted split GGUF files:
6164
6265``` bash
6366hf download rrobottt/pi-libero-bf16 \
@@ -75,7 +78,7 @@ MODEL=pi-libero-bf16
7578### step2: Run LIBERO Eval
7679
7780The simplest path is ` run_model_server.sh ` . It checks the GGUF files and an
78- existing ` model-server ` binary, launches the C++ Policy, then runs
81+ existing ` model-server ` binary, then launches the C++ Policy and runs
7982` eval.libero.runners.run_model_server ` :
8083
8184``` bash
@@ -85,41 +88,37 @@ bash eval/libero/scripts/run_model_server.sh
8588
8689Common variables:
8790
88- | Variable | Purpose |
91+ | Variable | Description |
8992| --- | --- |
90- | ` GGUF_DIR ` , ` MODEL ` | Split GGUF directory and filename prefix. |
91- | ` BACKEND ` | C++ Policy server preset, matching ` robot_server/test/test_server_latency.sh ` : ` linux-cuda ` , ` linux-cpu ` , ` mac-metal ` , or ` mac-cpu ` . Defaults to ` linux-cuda ` . |
93+ | ` GGUF_DIR ` , ` MODEL ` | Split GGUF inputs. Defaults to the path and filename prefix from step1 . |
94+ | ` BACKEND ` | C++ Policy server preset, matching ` robot_server/test/test_server_latency.sh ` . Options are ` linux-cuda ` , ` linux-cpu ` , ` mac-metal ` , and ` mac-cpu ` ; default is ` linux-cuda ` . |
9295| ` SERVER_BIN ` | Custom ` model-server ` path. By default it is derived from ` BACKEND ` . |
93- | ` HOST ` , ` PORT ` | Shared client/server endpoint. |
94- | ` SUITE ` , ` TASK_IDS ` , ` N_EPISODES ` , ` SEED ` , ` EPISODE_LENGTH ` | LIBERO rollout selection . |
96+ | ` HOST ` , ` PORT ` | Shared client/server endpoint and must stay in sync . |
97+ | ` SUITE ` , ` TASK_IDS ` , ` N_EPISODES ` , ` SEED ` , ` EPISODE_LENGTH ` | LIBERO rollout configuration . |
9598| ` MUJOCO_GL ` , ` PYOPENGL_PLATFORM ` , ` OUTPUT ` | Rendering backend and result output. |
9699
97- If the default ` model-server ` binary does not exist, build it from the project
98- root README or set ` SERVER_BIN=/path/to/model-server ` . ` BUILD_DIR ` is only the
99- intermediate path used to derive the default ` SERVER_BIN ` , and usually does not
100- need to be set directly .
100+ If the default C++ Policy server does not exist, build it from the project root
101+ README or set ` SERVER_BIN=/path/to/model-server ` directly . ` BUILD_DIR ` is only
102+ the intermediate path used to derive the default ` SERVER_BIN ` , and usually does
103+ not need to be set manually .
101104
102- Extra arguments after ` run_model_server.sh ` are passed to
103- ` python -m eval.libero.runners.run_model_server ` before the generated
104- ` --server-command ` block:
105+ Arguments after ` run_model_server.sh ` are passed to
106+ ` python -m eval.libero.runners.run_model_server ` , before the generated
107+ ` --server-command ` block. For example :
105108
106109``` bash
107110OUTPUT=eval/results/pi0-libero-object.json \
108111bash eval/libero/scripts/run_model_server.sh --episode-length 400
109112```
110113
111- The output JSON includes per-episode ` server_timing_avg_ms ` plus top-level
112- ` timing_ms ` summaries such as ` roundtrip_ms ` , ` server_predict_ms ` ,
113- ` model_total_ms ` , ` prefix_ms ` , and ` denoise_ms ` .
114+ The output JSON contains per-episode ` server_timing_avg_ms ` and a top-level
115+ ` timing_ms ` summary, including ` roundtrip_ms ` , ` server_predict_ms ` ,
116+ ` model_total_ms ` , ` prefix_ms ` , ` denoise_ms ` , and other metrics returned by the
117+ C++ Policy.
114118
115- For a fully manual server command, run:
116-
117- ``` bash
118- python -m eval.libero.runners.run_model_server --help
119- ```
120-
121- Keep ` --server-command ... ` at the end because it consumes the remaining
122- arguments.
119+ If you need to write the full server command by hand, use
120+ ` python -m eval.libero.runners.run_model_server --help ` ; note that
121+ ` --server-command ... ` must be the final part of the command.
123122
124123### step3: LeRobot Policy Latency
125124
@@ -128,10 +127,14 @@ python -m eval.libero.runners.latency_lerobot \
128127 --policy-path lerobot/pi0_libero_finetuned_v044
129128```
130129
131- C++ Policy latency is covered by ` robot_server/test/test_server_latency.sh ` .
130+ C++ Policy independent latency testing is unified under
131+ ` robot_server/test/test_server_latency.sh ` .
132132
133133### step4: LeRobot Policy Baseline
134134
135+ LeRobot baseline is used to compare rollout behavior against the original
136+ Python policy:
137+
135138``` bash
136139python -m eval.libero.runners.run_lerobot \
137140 --policy-path lerobot/pi0_libero_finetuned_v044 \
@@ -145,15 +148,16 @@ LeRobot's `eval_info.json` under `eval/results/lerobot-baseline-*`.
145148
146149## C++ Policy Request Semantics
147150
148- ` LiberoModelServerPolicy ` maps LIBERO observations to the C++ Policy server with
149- these conventions:
151+ ` LiberoModelServerPolicy ` sends LIBERO observations to the C++ Policy server
152+ with these conventions:
150153
151154* LIBERO cameras ` image ` and ` image2 ` are sent as ` observation.images.image `
152155 and ` observation.images.image2 ` .
153156* Images are flipped along height and width to match LeRobot's
154157 ` LiberoProcessorStep ` .
155- * The raw 8D LIBERO state is padded to the configured state dimension. Pi0 v044
156- uses 32 by default.
157- * Server action chunks are queued by the policy and consumed one action per env
158- step.
159- * The LIBERO environment receives the first 7 action dimensions by default.
158+ * The raw 8D LIBERO state is padded to the configured state dimension; pi0 v044
159+ defaults to 32.
160+ * After the server returns an action chunk, the policy queues it and consumes
161+ one action per environment step.
162+ * The action sent to the LIBERO environment uses the first 7 dimensions by
163+ default.
0 commit comments