[sergiopaniego]

Adds a Sudoku tutorial that follows the Wordle one (#1479): Qwen3-1.7B, an SFT warmup on solver-played games, then GRPO against the TextArena Sudoku-v0 environment, with one place(row, col, number) tool call per cell.

• examples/sudoku/: puzzles, SFT data, SFT, GRPO, evaluation, summary and GIF scripts, plus the 30 fixed evaluation puzzles at 26 and 16 empty cells.
• docs/source/tutorials/sudoku-grpo.md: the whole pipeline with the numbers, hardware and cost of our run (about $42 in total on Hugging Face Jobs).

Results on the same 30 puzzles, greedy:

Model	26 empty cells: solved	cells filled	16 empty cells: solved	cells filled
Qwen3-1.7B	0/30	0.1%	0/30	1.9%
SFT	10/30	55.8%	19/30	79.1%
SFT + GRPO (75 steps)	22/30	83.2%	26/30	87.5%

Models, data and curves: SFT, SFT + GRPO, dataset, trackio.

The "Long runs against a Space" section explains the mid-game reconnects we hit during GRPO, which led to #1536.

The two images point to huggingface/documentation-images (openenv/sudoku-progress.gif and openenv/sudoku-grpo-curve.png).

---

Note

Low Risk
Mostly documentation, example scripts, and puzzle fixtures; no changes to core OpenEnv runtime beyond new tests.

Overview
Adds a Train a Sudoku Agent learning path alongside Wordle: docs navigation (sudoku-grpo in the toctree and tutorials index) and a long-form tutorial with run costs, results, and a section on WebSocket drops during GRPO on a Hugging Face Space (fresh client per game, ties to #1536).

Introduces examples/sudoku/ as an end-to-end recipe on TextArena Sudoku-v0: fixed evaluation boards (puzzles_c55.json / puzzles_c65.json), solver-generated SFT tool-calling data, sft.py and grpo.py (TRL environment_factory, dual rewards for solve + fill progress, memory-safe batching), plus evaluate.py, summarize.py, and make_gif.py.

Adds lightweight tests that summarize.py only counts in-bounds “first move on a given cell” stats and that grpo.py’s per-reset() client close does not leak extra sync client threads.

^{Reviewed by Cursor Bugbot for commit 6554415. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit a2dd4ef. Configure here.}

[burtenshaw]

No remaining blocking findings after the pushed fixes.

Fixed the GRPO client thread leak at examples/sudoku/grpo.py:65 and examples/sudoku/grpo.py:71 with explicit synchronous clients. Four resets previously left five event-loop threads; the regression now leaves one active thread. Fixed the out-of-bounds first-move crash and negative-index miscount at examples/sudoku/summarize.py:22. Both fixes have regression tests that failed before and passed after. Corrected the stale result in docs/source/tutorials/index.md:37 from 13 to 22.

Verified 60 targeted CPU tests, a live local TextArena server, 1,000 solver-generated SFT games, exact regeneration of both committed puzzle files, no generated SFT board overlapping evaluation, and GIF generation. Checked the toctree, relative links, and both image assets. Every linked Hugging Face model, dataset, and Space returned HTTP 200. The HF Jobs helper uploads the puzzle argument with the evaluation script. GRPO already requires --env-url, matching #1537. Framework-specific code stays in examples and the integration list retains its neutral order.

Bugbot's sole thread is resolved. Verified its fix at examples/sudoku/evaluate.py:121: a 26-empty-cell puzzle can finish after 35 moves with nine isolated mistakes, consecutive mistakes terminate at two, and unfinished games stop at 40 moves. No unresolved Bugbot threads or author replies remain.

Non-blocking suggestions:

• examples/sudoku/summarize.py:19 counts a game stopped at 40 moves as "stopped calling". Give the move limit its own count; the 40-move reproduction reports one stopped-calling game despite making a tool call on every turn.
• examples/sudoku/grpo.py:87, examples/sudoku/evaluate.py:45, and examples/sudoku/generate_sft_data.py:47 use Google-style tool docstrings, contrary to CLAUDE.md. Record a narrow exception for Transformers schema generation or supply explicit schemas. Converting these arguments directly to the HF format raises DocstringParsingException.

Changed Python files pass usort, Ruff formatting, and Ruff lint. The repository-wide formatting hook reports failures in unchanged files, including the two known usort failures. GPU training and reported training results were not evaluated.

Reviewed a54055a plus fixes through 14ddf1b. All 12 recorded checks passed on the original head. The final snapshot on the pushed head had five checks passed and seven in progress, with no failures.