Add Parseur extract pipeline and leaderboard row

[slybridges]

Hi there!

I'm Sylvestre, co-founder of Parseur.

Thanks for putting this benchmark together! Some of the documents gave our engine a sweat 😅

Would you consider listing Parseur on the leaderboard? This PR adds a runnable pipeline and our results; happy to adjust anything to fit how you want submissions to look.

Summary

Adds a runnable Parseur pipeline, parseur_extract_v3_vision, and its leaderboard row for the 370-document suite.

Parseur takes no JSON Schema as input: extraction is configured as a "mailbox" with a field tree (this is how we handle schemas). The provider converts each benchmark schema into that tree:

• arrays become table fields
• nested objects become dotted field names
• booleans and enums become choice sets
• descriptions become field instructions, and the schema's top-level description becomes the mailbox AI
  instructions

It keeps one mailbox per converted schema, uploads the PDF, polls the document and maps the result back to the
schema's shape.

New category (OK?)

We added Parseur under the IDP Platforms category. Parseur is an Intelligent Document Processing
platform primarily used by non overly-technical users. Even if we have a public API, most users use the web app and no-code integrations. As such, it is not a developer-first extraction API, so that's why i'm hesitant to list it as a Specialized API.

No worries if you would rather not open a new category, let me know and I'll update to Specialized APIs.

Configuration

Setting	Value
Engine	AI Vision v3, pinned in the pipeline config
Schema	Converted to a mailbox field tree (see above)
Timeouts	Runner defaults
Retries and billing	An upload is never retried after a network or 5xx error. A 429 raises ProviderRateLimitError.
Cost	$0.05/page: Our pricing range is $0.02 to $0.50 depending on volume. $0.05 is the list price of the plan required to process this 4k+ pages benchmark
Citations	None: Parseur returns no per-field evidence, so grounding scores 0

Results

Run on 2026-10-14 against api.parseur.com from this PR's code, with --max_concurrent 8. 370/370 documents processed.

Overall	Short	Medium	Long	Precision S / M / L	Recall S / M / L
88.51	92.76	78.83	82.45	93.52 / 78.85 / 82.32	92.45 / 80.72 / 83.49

¢ / page	Latency s/doc S / M / L	Word / page grounding
5.00	17.3 / 107.1 / 548.7	0 / 0

Latency runs from the upload to the parsed result, polled every 5 s. A mailbox is set up once per schema, as a
customer would, and that setup is not counted.

Reproducibility

• Scorer revision: 0e76479 (current main); dataset revision 51bf3a03eb7feeaef8952be1fc2fc7f924ee72dc (v1.1).
• Scores are unweighted document macros: 252 short, 98 medium, 20 long; Overall pools all 370 documents.
• A run needs PARSEUR_API_KEY from an account with enough credits for 4,869 pages.

Validation

• uv run extract-bench pipelines lists parseur_extract_v3_vision.
• 1100 tests pass with the dev and runners extras (39 for the provider). Ruff lint and format pass.
• leaderboard.csv row added and README.md regenerated with scripts/update_readme.py, plus an "IDP Platforms"
  line in the cost-per-run table.