Skip to content

Show the default V2.1 evaluation command in the README #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MagMueller merged 1 commit into from
Sep 25, 2026
+16 −0
Merged

Show the default V2.1 evaluation command in the README #50

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,22 @@

**200 web tasks scored against weighted findings rubrics — the default task set.**

### Default evaluation command

After [installing the runner and setting your API keys](#running-bu-bench-v21-default),
run this from the repository root on `main`:

```bash
uv run python run_eval.py
```

[`run_eval.py`](run_eval.py) is the default entry point. It loads all 200 tasks
from [`BU_Bench_V2.enc`](BU_Bench_V2.enc), which contains the current V2.1 tasks
and rubrics, and runs BrowserCode with the findings judge. No task-count or
dataset flag is needed. Defaults are 60 minutes per task and up to 100 concurrent
tasks; [model, browser and override details are below](#running-bu-bench-v21-default).
Results go to `run_data/BU_Bench_V2_bcode_<timestamp>/`.

| File | Purpose |
| --- | --- |
| [BU_Bench_V2.enc](BU_Bench_V2.enc) | All 200 current V2.1 tasks, rubrics and weights |
Expand Down
Loading