Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions bench/cdeb/AUTHORIZATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Source authorization for the CDEB corpus

CDEB reads real decision records out of real repositories and publishes what it
finds. §3.3 requires repository ownership and decision authorship to be
disclosed, and CDEB-10 lists privacy authorization among the things that must
exist before the corpus is sealed. This file is that authorization, and the
`source_authorization_id` in each repository's freeze entry (§6.1) points here.

A public repository is not consent to be studied. Visibility and authorization
are different things, and the study needs the second one named by the person who
holds it — not inferred from the first.

## Granted

The owner of the repositories below authorized their use as CDEB source data,
and authorized publishing the result transparently, including repository
ownership and decision authorship, on 2026-08-11.

| authorization_id | repository | owner |
|---|---|---|
| `auth-owner-2026-08-11` | agent-operator-score | MongLong0214 |
| `auth-owner-2026-08-11` | logic-pro-mcp | MongLong0214 |
| `auth-owner-2026-08-11` | stock-ai-newsletter | MongLong0214 |
| `auth-owner-2026-08-11` | hermes-agent (fork) | MongLong0214 |

## What the authorization does not extend to

**Commits authored by other people.** `hermes-agent` is a fork of an upstream
project, and the substantial majority of its recent history is upstream authors'
work pulled in — 1,052 of the commits in one recent week. The owner can
authorize the use of their own repository and their own decisions; they cannot
authorize on behalf of upstream contributors. Any task drawn from that
repository must rest on a record the owner authored, and §3.2's candidate
registry field `decision_source_refs` is where that is checked rather than
assumed.

**The CommitLore repository itself.** §3.3 excludes it from the primary corpus,
and no authorization changes that. It is excluded because the product's own
decisions are not independent of the product being measured, which is a
methodological exclusion rather than a permission one.

## Consequence for what the result may say

Every authorized repository is operated by the same person who authored this
benchmark and the product it measures. That is §3.3's **Tier B**, and the rule
attached to it is not optional: the result must read *"five author-operated
repositories"* and must not describe itself as independent external validation.

Tier A would require at least two repositories where the decision author or
accepting reviewer differs from the benchmark and product author. Nothing here
reaches that, and no wording may imply otherwise.

## Corpus arithmetic, stated plainly

§3.3 requires five repositories, and the exclusion of the CommitLore repository
leaves four authorized ones. A fifth is needed before the corpus can be sealed,
and §3.2's rule 5 is explicit about what may not be done to close that gap:
when candidates are short, the study stops rather than lowering the bar or
creating records for it.
Loading