Skip to content

feat(evals): add interactive collaboration evaluation baseline - #1323

Closed
TheGreatAxios wants to merge 1 commit into
mainfrom
cl-9879-establish-an-interactive-collaboration-evaluation-baseline
Closed

feat(evals): add interactive collaboration evaluation baseline#1323
TheGreatAxios wants to merge 1 commit into
mainfrom
cl-9879-establish-an-interactive-collaboration-evaluation-baseline

Conversation

@TheGreatAxios

@TheGreatAxios TheGreatAxios commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a frozen 9-scenario collaboration eval with paired good/bad transcripts and an external transcript scorer
  • Add a deterministic runner with version-drift, control-mismatch, and live-mode guards
  • Refuse to record baselines from non-uniform fixture versions with a permanent regression test

Verification

  • bun run check passes including the full guarded suite
  • Focused collaboration tests pass including seeded good/bad sensitivity
  • Deterministic runner reports full control sensitivity on the frozen set

Fixes CL-9879

@linear-code

linear-code Bot commented Oct 5, 2026

Copy link
Copy Markdown

CL-9879

@TheGreatAxios

TheGreatAxios commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

Closed per operator triage decision — Linear issue being canceled; content superseded or deprioritized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@TheGreatAxios