Leaderboard v1: Computer / Claude Sonnet 5.5

[c-suvamsa]

Summary

• Agent / Model: DevRev Computer / Claude Sonnet 5.5 (Anthropic).
• Source Harbor job URL: https://hub.harborframework.com/jobs/21e21855-1281-40dc-a6f2-189162002516.
• Run: 2026-10-15; enterprise-bench/tasks/v1 via Daytona; 14 tasks × 10 attempts, with the default scored timeout.
• Results: 135/140 rewards of 1.0 = 96.4% accuracy; 45,530,715 recorded input + output tokens; $99.92 SDK-reported cost.

PR checklist – leaderboard row entry

Provenance and access

• Source job is public and owned by the submitter; entry header and URL match its job ID.
• Entry has all required metadata and metrics; 140 distinct trial IDs match n_trials.
• Accuracy, pass@k, and token breakdown reconcile with the recorded run; cost equals the sum of saved SDK reports (Harbor's native cost field is null).
• Agent, model, and provider appear on the job.
• Every local trial has an ATIF trajectory and a reward/result; no failed trial was removed.
• Reviewer to confirm logged-out replay of every trajectory.

Run conditions and review

• Task content digests for all 14 tasks match enterprise-bench main. The job used an absolute local tasks/v1 path rather than a registry-pinned dataset version.
• Scored timeout multiplier is 1.0 and no CPU/memory/storage override was recorded.
• Reviewer to confirm benchmark-answer access boundaries and independently replay the score.

Validation

• Main-branch leaderboard validator passed locally.
• Repository validate and validate-leaderboard CI checks pass.

Notes for reviewers

The run-owner-confirmed model label was added to Harbor metadata after the run; the saved Computer trace shows its generic gateway alias. This is a DevRev Computer entry, separate from the Claude Code / Claude Sonnet 5.5 submission. Trial rewards were not changed.

[nimit2801]

Thanks for the submission. Nimit’s intake workflow has picked this up as a leaderboard entry for DevRev Computer / Claude Sonnet 5.5. Both CI checks are passing; technical review is still required, not approval.

Verification points:

• Independently reconcile scores, pass@k, tokens and cost against the complete Harbor trial bundle.
• Confirm logged-out trajectory access and model provenance, including the post-run model label.
• Check task digests, default timeouts and benchmark-answer access boundaries.