feat: add Rust Cargo feedback environment#1485
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c155d4c. Configure here.
| score += 0.20 | ||
| if count_assertions(code) >= 2: | ||
| score += 0.10 | ||
| return score |
There was a problem hiding this comment.
Completion is a message list, not a string
High Severity
All six reward functions treat completion as a str and pass it directly to extract_rust_code() / has_single_rust_block(), which call re.search() on it. However, the framework provides completion as a list[dict] (message objects). This causes a TypeError on every call, silently caught by the Rubric error handler, making every reward always return 0.0. The environment produces no useful training signal. The text needs to be extracted first, e.g. via parser.parse_answer(completion) or completion[-1]["content"].
Additional Locations (2)
Reviewed by Cursor Bugbot for commit c155d4c. Configure here.


Summary
rust-cargoenvironment for the Algora Rust w/ Cargo feedback bounty.cargo build,cargo clippy -- -D warnings, andcargo test.Verification
uv run --no-dev ruff check environments/rust_cargouv pip install -e environments/rust_cargouv run --no-dev python environments/rust_cargo/rust_cargo.pyuv run --no-dev python - <<'PY' ... vf.load_environment('rust-cargo') ... PYCHANGED_ENVS=rust_cargo uv run --no-dev pytest tests/test_envs.py -q --tb=shortwas attempted, but the Windows host cannot execute the test's hard-coded/bin/bashsubprocess path.Note: this Windows host does not have
cargoinstalled, so local smoke verification covered environment loading and static reward helpers. The executable reward functions run Cargo when it is available onPATH.Algora bounty: https://algora.io/PrimeIntellect-ai/bounties/FDaU5Y9iKeE8qfk3
Reference: https://github.com/Oxen-AI/GRPO-With-Cargo-Feedback/blob/main/train.py
Note
Medium Risk
Eval runners execute model-generated Rust via subprocess in ephemeral directories (timeouts, no
cargo→ zero reward); hosts need a trusted toolchain and should treat this like other code-exec envs, not a security boundary.Overview
Adds a new installable
rust-cargosingle-turn environment for Rust code generation, aligned with the Algora Cargo-feedback bounty pattern.The model gets Rust prompts and must answer with one fenced
rustblock (implementation,#[cfg(test)]module, assertions, nomain). A built-in dataset covers five small tasks. Scoring uses avf.Rubricmixing static checks (single block, requiredfn, assertion count) with executable feedback: extracted code is written to a temp crate and run throughcargo build,cargo clippy -D warnings, andcargo test(tests weighted highest).load_environment()wires this intovf.SingleTurnEnvwith a TDD-oriented system prompt. Package metadata, quickstart, and the rootenvironments/README.mdcatalog entry document the env and thecargoon PATH requirement.Reviewed by Cursor Bugbot for commit c155d4c. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add Rust Cargo feedback environment for code-generation scoring
rust-cargosingle-turn environment in rust_cargo.py that scores model-generated Rust code using a weighted rubric of static checks and live Cargo tool execution.rustcode block, presence of the required function, nomainfunction, a#[cfg(test)]module, and assertion count (up to 4).cargo build,cargo clippy(zero warnings), andcargo testin a temporary workspace viarun_cargo_tool.pyproject.tomlfor packaging.Macroscope summarized c155d4c.