Skip to content

Add benchmark for documented prompts - #1069

Draft
qianl-nv wants to merge 1 commit into
mainfrom
qianl/feature/agentic-endpoint-benchmark
Draft

Add benchmark for documented prompts#1069
qianl-nv wants to merge 1 commit into
mainfrom
qianl/feature/agentic-endpoint-benchmark

Conversation

@qianl-nv

Copy link
Copy Markdown
Collaborator

Summary

Benchmark documented agentic env-gen prompts

Detailed description

  • Add a repeatable benchmark that scores documented prompts against reviewed YAML specs.
  • Record inference calls, retries, validation errors, runtime, and structured reports.
  • Add tests for inference statistics used by the benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant