From 5ec7413c38689d37ed0b09ba04cbf9fbb7e46c63 Mon Sep 17 00:00:00 2001 From: MongLong0214 Date: Tue, 11 Aug 2026 17:04:07 +0900 Subject: [PATCH] Name the authorization the corpus rests on, and the gap it leaves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CDEB reads real decisions out of real repositories and publishes what it finds, so §3.3 requires ownership and authorship to be disclosed and CDEB-10 requires a privacy authorization to exist before the corpus is sealed. This is that authorization, and each repository's freeze entry points at it. A public repository is not consent to be studied. Visibility and authorization are different things, and the second one has to be named by the person who holds it rather than inferred from the first. The owner granted both the use of the source data and the transparent publication of the result. Two boundaries are written down rather than left to be discovered. The owner cannot authorize on behalf of upstream contributors, and one repository is a fork whose recent history is mostly other people's commits, so a task drawn from it has to rest on a record the owner authored. And the CommitLore repository stays excluded whatever the authorization says, because that exclusion is methodological rather than a permission. The arithmetic is stated instead of deferred: §3.3 wants five repositories, the exclusion leaves four authorized, and §3.2 rule 5 already says what may not be done about a shortfall. Limit: this records that authorization was granted, not that every candidate drawn under it was authored by the grantor -- the candidate registry checks that per record, and nothing here substitutes for it Ruled-out: treating public visibility as consent to study | a repository being readable says nothing about whether its owner agreed to be measured and named Blast: local Undo: easy Certainty: firm Verified: both bench verifiers pass and the package typecheck is clean Provenance: authored Record-Id: r-cdebauth1 --- bench/cdeb/AUTHORIZATION.md | 59 +++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 bench/cdeb/AUTHORIZATION.md diff --git a/bench/cdeb/AUTHORIZATION.md b/bench/cdeb/AUTHORIZATION.md new file mode 100644 index 00000000..e7ebdd47 --- /dev/null +++ b/bench/cdeb/AUTHORIZATION.md @@ -0,0 +1,59 @@ +# Source authorization for the CDEB corpus + +CDEB reads real decision records out of real repositories and publishes what it +finds. §3.3 requires repository ownership and decision authorship to be +disclosed, and CDEB-10 lists privacy authorization among the things that must +exist before the corpus is sealed. This file is that authorization, and the +`source_authorization_id` in each repository's freeze entry (§6.1) points here. + +A public repository is not consent to be studied. Visibility and authorization +are different things, and the study needs the second one named by the person who +holds it — not inferred from the first. + +## Granted + +The owner of the repositories below authorized their use as CDEB source data, +and authorized publishing the result transparently, including repository +ownership and decision authorship, on 2026-08-11. + +| authorization_id | repository | owner | +|---|---|---| +| `auth-owner-2026-08-11` | agent-operator-score | MongLong0214 | +| `auth-owner-2026-08-11` | logic-pro-mcp | MongLong0214 | +| `auth-owner-2026-08-11` | stock-ai-newsletter | MongLong0214 | +| `auth-owner-2026-08-11` | hermes-agent (fork) | MongLong0214 | + +## What the authorization does not extend to + +**Commits authored by other people.** `hermes-agent` is a fork of an upstream +project, and the substantial majority of its recent history is upstream authors' +work pulled in — 1,052 of the commits in one recent week. The owner can +authorize the use of their own repository and their own decisions; they cannot +authorize on behalf of upstream contributors. Any task drawn from that +repository must rest on a record the owner authored, and §3.2's candidate +registry field `decision_source_refs` is where that is checked rather than +assumed. + +**The CommitLore repository itself.** §3.3 excludes it from the primary corpus, +and no authorization changes that. It is excluded because the product's own +decisions are not independent of the product being measured, which is a +methodological exclusion rather than a permission one. + +## Consequence for what the result may say + +Every authorized repository is operated by the same person who authored this +benchmark and the product it measures. That is §3.3's **Tier B**, and the rule +attached to it is not optional: the result must read *"five author-operated +repositories"* and must not describe itself as independent external validation. + +Tier A would require at least two repositories where the decision author or +accepting reviewer differs from the benchmark and product author. Nothing here +reaches that, and no wording may imply otherwise. + +## Corpus arithmetic, stated plainly + +§3.3 requires five repositories, and the exclusion of the CommitLore repository +leaves four authorized ones. A fifth is needed before the corpus can be sealed, +and §3.2's rule 5 is explicit about what may not be done to close that gap: +when candidates are short, the study stops rather than lowering the bar or +creating records for it.