Skip to content

Commit 51bfc36

Browse files
committed
libero readme sync
1 parent c473ba9 commit 51bfc36

1 file changed

Lines changed: 48 additions & 44 deletions

File tree

‎eval/libero/README.md‎

Lines changed: 48 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,23 @@
22

33
[中文文档](README_zh.md)
44

5-
This directory contains LIBERO rollout and latency helpers for two policy paths:
5+
This directory provides LIBERO simulation eval and latency scripts for two
6+
policy paths:
67

7-
* robot.cpp C++ Policy: loads split GGUF checkpoints through `model-server`.
8-
* LeRobot Policy: runs the original Python policy as a baseline.
8+
* robot.cpp C++ Policy: loads GGUF through `model-server` and runs rollout and
9+
latency checks.
10+
* LeRobot Policy: uses the original Python policy as the baseline.
911

10-
## Layout
12+
## File Layout
1113

1214
```text
1315
eval/libero/
1416
├── policy/
1517
│ └── model_server.py # LIBERO observation -> C++ policy request adapter
1618
├── runners/
17-
│ ├── run_model_server.py # C++ Policy LIBERO rollout runner
19+
│ ├── run_model_server.py # C++ policy LIBERO rollout runner
1820
│ ├── run_lerobot.py # LeRobot Policy rollout wrapper
19-
│ └── latency_lerobot.py # LeRobot Policy latency runner
21+
│ └── latency_lerobot.py # LeRobot policy latency runner
2022
├── scripts/
2123
│ └── run_model_server.sh # launch/eval convenience wrapper
2224
├── utils/
@@ -26,28 +28,28 @@ eval/libero/
2628
```
2729

2830
Shared action chunk buffering lives in `robot_client/policy/base_policy.py`.
29-
LIBERO-specific C++ Policy launch, shutdown, and timing helpers live in
31+
LIBERO C++ Policy launch, shutdown, and timing helpers live in
3032
`eval/libero/policy/model_server.py`.
3133

3234
## Usage
3335

3436
### step0: Environment
3537

36-
Create the optional conda environment:
38+
Using a dedicated conda environment is recommended:
3739

3840
```bash
3941
conda env create -f eval/libero/environment.yaml
4042
conda activate robotcpp-libero
4143
```
4244

43-
Or install the minimum dependencies in an existing environment:
45+
If reusing an existing environment, install at least:
4446

4547
```bash
4648
pip install "cmake<4"
4749
pip install --no-build-isolation "hf-libero>=0.1.3,<0.2.0"
4850
```
4951

50-
If EGL is unavailable on the machine, use OSMesa:
52+
If the machine has no available EGL device, switch to OSMesa:
5153

5254
```bash
5355
export MUJOCO_GL=osmesa
@@ -56,8 +58,9 @@ export PYOPENGL_PLATFORM=osmesa
5658

5759
### step1: Prepare C++ Policy GGUF
5860

59-
For Pi0, use the split GGUF checkpoint from
60-
[`rrobottt/pi-libero-bf16`](https://huggingface.co/rrobottt/pi-libero-bf16):
61+
For Pi0, refer to the GGUF checkpoint
62+
[`rrobottt/pi-libero-bf16`](https://huggingface.co/rrobottt/pi-libero-bf16).
63+
The C++ Policy uses converted split GGUF files:
6164

6265
```bash
6366
hf download rrobottt/pi-libero-bf16 \
@@ -75,7 +78,7 @@ MODEL=pi-libero-bf16
7578
### step2: Run LIBERO Eval
7679

7780
The simplest path is `run_model_server.sh`. It checks the GGUF files and an
78-
existing `model-server` binary, launches the C++ Policy, then runs
81+
existing `model-server` binary, then launches the C++ Policy and runs
7982
`eval.libero.runners.run_model_server`:
8083

8184
```bash
@@ -85,41 +88,37 @@ bash eval/libero/scripts/run_model_server.sh
8588

8689
Common variables:
8790

88-
| Variable | Purpose |
91+
| Variable | Description |
8992
| --- | --- |
90-
| `GGUF_DIR`, `MODEL` | Split GGUF directory and filename prefix. |
91-
| `BACKEND` | C++ Policy server preset, matching `robot_server/test/test_server_latency.sh`: `linux-cuda`, `linux-cpu`, `mac-metal`, or `mac-cpu`. Defaults to `linux-cuda`. |
93+
| `GGUF_DIR`, `MODEL` | Split GGUF inputs. Defaults to the path and filename prefix from step1. |
94+
| `BACKEND` | C++ Policy server preset, matching `robot_server/test/test_server_latency.sh`. Options are `linux-cuda`, `linux-cpu`, `mac-metal`, and `mac-cpu`; default is `linux-cuda`. |
9295
| `SERVER_BIN` | Custom `model-server` path. By default it is derived from `BACKEND`. |
93-
| `HOST`, `PORT` | Shared client/server endpoint. |
94-
| `SUITE`, `TASK_IDS`, `N_EPISODES`, `SEED`, `EPISODE_LENGTH` | LIBERO rollout selection. |
96+
| `HOST`, `PORT` | Shared client/server endpoint and must stay in sync. |
97+
| `SUITE`, `TASK_IDS`, `N_EPISODES`, `SEED`, `EPISODE_LENGTH` | LIBERO rollout configuration. |
9598
| `MUJOCO_GL`, `PYOPENGL_PLATFORM`, `OUTPUT` | Rendering backend and result output. |
9699

97-
If the default `model-server` binary does not exist, build it from the project
98-
root README or set `SERVER_BIN=/path/to/model-server`. `BUILD_DIR` is only the
99-
intermediate path used to derive the default `SERVER_BIN`, and usually does not
100-
need to be set directly.
100+
If the default C++ Policy server does not exist, build it from the project root
101+
README or set `SERVER_BIN=/path/to/model-server` directly. `BUILD_DIR` is only
102+
the intermediate path used to derive the default `SERVER_BIN`, and usually does
103+
not need to be set manually.
101104

102-
Extra arguments after `run_model_server.sh` are passed to
103-
`python -m eval.libero.runners.run_model_server` before the generated
104-
`--server-command` block:
105+
Arguments after `run_model_server.sh` are passed to
106+
`python -m eval.libero.runners.run_model_server`, before the generated
107+
`--server-command` block. For example:
105108

106109
```bash
107110
OUTPUT=eval/results/pi0-libero-object.json \
108111
bash eval/libero/scripts/run_model_server.sh --episode-length 400
109112
```
110113

111-
The output JSON includes per-episode `server_timing_avg_ms` plus top-level
112-
`timing_ms` summaries such as `roundtrip_ms`, `server_predict_ms`,
113-
`model_total_ms`, `prefix_ms`, and `denoise_ms`.
114+
The output JSON contains per-episode `server_timing_avg_ms` and a top-level
115+
`timing_ms` summary, including `roundtrip_ms`, `server_predict_ms`,
116+
`model_total_ms`, `prefix_ms`, `denoise_ms`, and other metrics returned by the
117+
C++ Policy.
114118

115-
For a fully manual server command, run:
116-
117-
```bash
118-
python -m eval.libero.runners.run_model_server --help
119-
```
120-
121-
Keep `--server-command ...` at the end because it consumes the remaining
122-
arguments.
119+
If you need to write the full server command by hand, use
120+
`python -m eval.libero.runners.run_model_server --help`; note that
121+
`--server-command ...` must be the final part of the command.
123122

124123
### step3: LeRobot Policy Latency
125124

@@ -128,10 +127,14 @@ python -m eval.libero.runners.latency_lerobot \
128127
--policy-path lerobot/pi0_libero_finetuned_v044
129128
```
130129

131-
C++ Policy latency is covered by `robot_server/test/test_server_latency.sh`.
130+
C++ Policy independent latency testing is unified under
131+
`robot_server/test/test_server_latency.sh`.
132132

133133
### step4: LeRobot Policy Baseline
134134

135+
LeRobot baseline is used to compare rollout behavior against the original
136+
Python policy:
137+
135138
```bash
136139
python -m eval.libero.runners.run_lerobot \
137140
--policy-path lerobot/pi0_libero_finetuned_v044 \
@@ -145,15 +148,16 @@ LeRobot's `eval_info.json` under `eval/results/lerobot-baseline-*`.
145148

146149
## C++ Policy Request Semantics
147150

148-
`LiberoModelServerPolicy` maps LIBERO observations to the C++ Policy server with
149-
these conventions:
151+
`LiberoModelServerPolicy` sends LIBERO observations to the C++ Policy server
152+
with these conventions:
150153

151154
* LIBERO cameras `image` and `image2` are sent as `observation.images.image`
152155
and `observation.images.image2`.
153156
* Images are flipped along height and width to match LeRobot's
154157
`LiberoProcessorStep`.
155-
* The raw 8D LIBERO state is padded to the configured state dimension. Pi0 v044
156-
uses 32 by default.
157-
* Server action chunks are queued by the policy and consumed one action per env
158-
step.
159-
* The LIBERO environment receives the first 7 action dimensions by default.
158+
* The raw 8D LIBERO state is padded to the configured state dimension; pi0 v044
159+
defaults to 32.
160+
* After the server returns an action chunk, the policy queues it and consumes
161+
one action per environment step.
162+
* The action sent to the LIBERO environment uses the first 7 dimensions by
163+
default.

0 commit comments

Comments
 (0)