diff --git a/bench/cdeb/AUTHORIZATION.md b/bench/cdeb/AUTHORIZATION.md new file mode 100644 index 00000000..e7ebdd47 --- /dev/null +++ b/bench/cdeb/AUTHORIZATION.md @@ -0,0 +1,59 @@ +# Source authorization for the CDEB corpus + +CDEB reads real decision records out of real repositories and publishes what it +finds. §3.3 requires repository ownership and decision authorship to be +disclosed, and CDEB-10 lists privacy authorization among the things that must +exist before the corpus is sealed. This file is that authorization, and the +`source_authorization_id` in each repository's freeze entry (§6.1) points here. + +A public repository is not consent to be studied. Visibility and authorization +are different things, and the study needs the second one named by the person who +holds it — not inferred from the first. + +## Granted + +The owner of the repositories below authorized their use as CDEB source data, +and authorized publishing the result transparently, including repository +ownership and decision authorship, on 2026-08-11. + +| authorization_id | repository | owner | +|---|---|---| +| `auth-owner-2026-08-11` | agent-operator-score | MongLong0214 | +| `auth-owner-2026-08-11` | logic-pro-mcp | MongLong0214 | +| `auth-owner-2026-08-11` | stock-ai-newsletter | MongLong0214 | +| `auth-owner-2026-08-11` | hermes-agent (fork) | MongLong0214 | + +## What the authorization does not extend to + +**Commits authored by other people.** `hermes-agent` is a fork of an upstream +project, and the substantial majority of its recent history is upstream authors' +work pulled in — 1,052 of the commits in one recent week. The owner can +authorize the use of their own repository and their own decisions; they cannot +authorize on behalf of upstream contributors. Any task drawn from that +repository must rest on a record the owner authored, and §3.2's candidate +registry field `decision_source_refs` is where that is checked rather than +assumed. + +**The CommitLore repository itself.** §3.3 excludes it from the primary corpus, +and no authorization changes that. It is excluded because the product's own +decisions are not independent of the product being measured, which is a +methodological exclusion rather than a permission one. + +## Consequence for what the result may say + +Every authorized repository is operated by the same person who authored this +benchmark and the product it measures. That is §3.3's **Tier B**, and the rule +attached to it is not optional: the result must read *"five author-operated +repositories"* and must not describe itself as independent external validation. + +Tier A would require at least two repositories where the decision author or +accepting reviewer differs from the benchmark and product author. Nothing here +reaches that, and no wording may imply otherwise. + +## Corpus arithmetic, stated plainly + +§3.3 requires five repositories, and the exclusion of the CommitLore repository +leaves four authorized ones. A fifth is needed before the corpus can be sealed, +and §3.2's rule 5 is explicit about what may not be done to close that gap: +when candidates are short, the study stops rather than lowering the bar or +creating records for it.