Skip to content

Enumerate CDEB candidates without selecting them - #536

Merged
MongLong0214 merged 1 commit into
devfrom
cdeb-10-registry
Aug 11, 2026
Merged

Enumerate CDEB candidates without selecting them#536
MongLong0214 merged 1 commit into
devfrom
cdeb-10-registry

Conversation

@MongLong0214

@MongLong0214 MongLong0214 commented Aug 11, 2026

Copy link
Copy Markdown
Owner

The enumeration half of CDEB-10. No corpus selection, no sealing, no prompts, no evaluators — those are the rest of the ticket and they come after this reports something worth selecting from.

Why now, when the corpus is not ready

Measured across the authorized repositories, Ruled-out counts are 49, 6, 6 and 0 against six tasks required per repository. §3.2 rule 5 is explicit about the alternatives: when candidates are short the study stops, rather than lowering the bar or creating records to fill it.

So the useful thing to build is not a corpus — it is the instrument that says how far away one is. This turns waiting into measuring.

It reports why, not just how many

"12 eligible" tells you nothing about whether the bar is holding or quietly drifting. The census names every disqualification:

records examined: 30
eligible: 0            rejected: 30      blocked on human review: 30
disqualified:
  missing_explicit_rejection_reason: 7
  invalid_record_identity: 2
  after_snapshot_cutoff: 0
  commitlore_repository: 0

Rejected candidates are preserved with their reason (§3.2 rule 1), not dropped.

What it refuses to decide

Four §3.2 fields are human calls — whether the wrong path is functionally viable, whether a deterministic oracle is possible, whether the current code already reveals the reason, whether the implementation is bounded. They are marked undecided and never counted as eligible.

No model decides them and no heuristic guesses them. A registry that promotes its own guesses would produce exactly the number we want to see, which is the failure mode worth designing against here. Every candidate today is human_review_required, and that is the honest state.

Current census on the authorized repositories

repository candidates with an explicit rejection reason
agent-operator-score 30 23
logic-pro-mcp 24 5
stock-ai-newsletter 5 4
hermes-agent 16 0

32 against a requirement of 30 — but distributed 23/5/4/0, so three of four repositories cannot meet the per-repository quota. The total is not the constraint; the distribution is.

Side finding

invalid_record_identity is non-zero — 3 in this repository, 2 in agent-operator-score. Malformed Record-Id values exist in real history. Separate from #535 (nothing mints one at all), and worth handling together.

7 cases pass; both typechecks clean; two builds leave dist unchanged; both verifiers pass. Reads records through the shipping query engine rather than a reimplementation.

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

CommitLore — record lint

Trailers: clean — 1 commit in origin/dev..48bd5a897fde019ca8910fa968fac2b2776d23a1
Active constraints: 25 limits · 64 ruled-out · 17 warnings — from 29 records over 4 changed paths

Active constraints for the paths this PR touches

Limits (25)

  • r-cdeb10reg 48bd5a8 — wrong-path viability, deterministic oracle feasibility, code disclosure, bounded implementation, and unproven ordinary or benchmark authorship cannot be decided from history and remain undecided for human review
  • r-cdeb08an 60db89f — the paired bootstrap describes resampling stability within these five frozen repositories and thirty frozen tasks, and says nothing about any other repository, task or agent population
  • r-rel071 af8e0ab — 0.7.0 stays published with its notes amended to name the defect at the top; retracting a tag people may already have installed trades a known-bad version for an unknown one
  • r-rel070 d4a4d8b — the README's behaviour claim now rests on M5 while the generated numbers block beneath it still publishes M4, which is The README's generated numbers block still publishes M4; M5 measured the thing the README leads with #480 rather than a release-time edit
  • r-cdebver01 ce7b278 — the schemas freeze protocol 1.2.0 constants -- thresholds, matrix size, category names -- so a protocol change is a schema change and CI notices
  • r-m5analysis 3450656 — the script enforces the row count, not the identity of the rows; a run that produced 1,160 rows under a changed harness would satisfy it, which is what harness_commit and dist_digest on each row are for
  • r-benchscope 67f4375 — nothing checks the shape of the eight metric-row files. This gate names them and steps over them, and bench/deterministic/types.ts is the only definition that family has -- there is no JSON schema for it, so drift on that side is still invisible
  • r-benchscope 67f4375 — the pre-provenance exemption reads started_at, which is data on the row rather than a fact about the file. A row that misreported it would be held to the shorter list of requirements; that is a deliberate falsification rather than the omission this fixes, and nothing here detects it
  • r-3c9d52 dc9e769 — the sweep is two git log calls per path and the delivery phase runs git log --follow on every tracked path, so a full run over the four externals is hours rather than minutes on one machine
  • r-rel060 e999b9d — the install one-liner in all four READMEs now points at a tag that does not exist until this is tagged, so the window between merging to main and pushing v0.6.0 is one where the documented install is broken
  • r-release051 19810d2 — the hook is written at install time, so no release repairs a repository that already has one; every release touching hook behaviour has to restate what does
  • r-convertreadme e12c816 — a README claim about the default workflow is only true if the shipped skill performs it, and the skill currently requires the user to name CommitLore first
  • r-release050 ad402c7 — the hook is written at install time, so a corrected release never reaches a repository that already has one; every release fixing hook behaviour has to say what repairs an existing install
  • r-rel041notes 71efe1f — 0.4.1 makes the installer honest about a verification it cannot complete rather than fixing the kill, so an upgrading user may still see the unverified message instead of a version
  • r-rel040notes 5d57a72 — the 26.3-point density gap quoted in the notes is measured at this head and will drift with merge volume; it is illustrative of the denominator problem rather than a stable figure
  • r-rel021a a79e350 — v0.2.0 remains on the remote with no release attached
  • r-fix93pkg 9c4a396 — package.json remains a development artifact (build, typecheck, dependency floor) -- it is not read as a distribution manifest by anything in this repository
  • r-relinstall c6e1d04 — never tested against the real GitHub release infrastructure (no release exists yet — that is the owner's action) — verified against a locally built SEA binary, a hand-made SHA256SUMS, and a local HTTP server standing in for GitHub's release-asset redirects, which is everything this repository lets a change verify before a tag exists.
  • r-det058 695cdf6 — the suite must need no model, agent, network or uncommitted benchmark input
  • r-7a3e91 cf859e4 — better-sqlite3 stays external because it is native — the bundle degrades to --no-index without it, which only works because r-6f2a08 made that load lazy first
  • r-4a8e15 49e12c7 — git's grammar requires a subject before a trailer block, so a serialized block is not by itself a parseable message
  • r-7e5f02 e5f5e00 — npm installs through an engine mismatch, so the ecosystem's own signal cannot be relied on to stop anything
  • r-5a8c04 c46a577 — git owns the definition of a trailer block, so any behavior we cannot get from interpret-trailers is behavior we must not invent
  • r-c0f4e2 3d249cd — npm gitlore is held by an active same-domain CLI, so the owner's first-choice name was not available
  • r-b2e7f1 00d348d — Parsing must delegate to git interpret-trailers -- reimplementing the block rules would drift from the rest of the git ecosystem

Ruled out (64)

  • r-cdeb08an 60db89f — discovering row files under the result directory | an unregistered file contaminates the matrix while leaving every stopping rule looking satisfied
  • r-cdeb08an 60db89f — filling or dropping unavailable usage | both change a token aggregate without evidence, one by inventing a number and one by redefining the population
  • r-rel071 af8e0ab — deleting the v0.7.0 tag and re-cutting it | rewriting a published ref breaks every clone that already fetched it, and the release notes can carry the truth without that
  • r-rel071 af8e0ab — reading the flag with a nullish check and documenting the commander default | the documentation would sit in this file while the defect sits in every install, which is the arrangement that produced this
  • r-rel070 d4a4d8b — bumping the manifests first and the README pins after | the readme suite pins them to package.json, so the intermediate commit is one where CI is red and the documented install is wrong
  • r-rel070 d4a4d8b — rebuilding dist for the version change | the version is read at runtime, and a rebuild would move the digest every M5 row records without changing a byte of behaviour
  • r-cdebver01 ce7b278 — default-in discovery over the CDEB tree like the legacy gate | that gate's enemy was the unregistered file, CDEB's is the unregistered input, and bench:m5 reads every jsonl in bench/results, so the registered analysis would run over M1, M4 and files marked non-citable #441 showed a glob passing a stopping rule on contamination
  • r-cdebver01 ce7b278 — validating with the draft-07 Ajv entry and downgrading the schemas | the 2020-12 entry ships in the same package, and weakening six schemas to fit a meta-schema is backwards
  • r-m5analysis 3450656 — Computing the table now and not looking | the file would exist, and a number that exists is a number someone reads
  • r-m5analysis 3450656 — Leaving the analysis to be written after the run | that is the freedom this document was written to give up, and writing it afterwards is indistinguishable from writing it to fit
  • r-m5analysis 3450656 — Asserting the M1 control in a vitest suite instead of --validate | it belongs to the analyser, and a reader checking the verdict should be able to re-derive the control with the same command that produced the verdict
  • r-benchscope 67f4375 — A declared list of result files, in the gate or in a manifest | opt-in under another name, and a file left off it is silently ungated -- which is precisely the failure bench/verify.mjs is not run by anything, so the result schema drifted unnoticed #392 reports
  • r-benchscope 67f4375 — Validating only the three files that pass today | a description of the present rather than a rule, and it goes stale the first time a matrix is added or the schema moves
  • r-benchscope 67f4375 — A naming convention on results filenames | the runner's default output is an ISO stamp plus hex and the metric writers' is a family name plus an ISO stamp, but every run-record file in the tree was named by hand through --out (t702-m2, m5-off-design-20-tasks), so the convention is enforced nowhere and cannot carry a gate
  • r-benchscope 67f4375 — A marker file dropped beside each gated result | still a step somebody has to remember, and a forgotten step is the defect being fixed
  • r-benchscope 67f4375 — Widening result.schema.json to describe both row families | they share no required field, so the union would accept a run record with no task and a metric row with no metric, which is a schema that rejects nothing
  • r-benchscope 67f4375 — Making harness_commit and dist_digest optional in the schema | weakens the gate for every future row in order to admit six historical files, and Make per-turn token usage observable, and say what it still cannot price #390's drift was an additionalProperties failure that a laxer schema catches later rather than sooner
  • r-benchscope 67f4375 — Skipping the six pre-provenance files whole | cheaper to write, but it stops checking types, patterns and the closed property set on 380 rows in order to excuse two absent fields
  • r-3c9d52 dc9e769 — reimplementing the scorer for the external corpus | two scorers that drift by a line produce two numbers that look comparable and are not, which is the failure No-index query at 100k regressed from 7.7s to 90s — cause not yet identified #163 already recorded here
  • r-3c9d52 dc9e769 — replacing the per-path --follow chain with one forward pass over a rename graph | it is much faster and it is a different attachment predicate from the one DECISION-DELIVERY.md §3 registered, so the two corpora would stop being comparable
  • r-3c9d52 dc9e769 — turning the census notes option on by default | every measurement registered before it existed read commit messages only, and a denominator that silently grew would invalidate them without saying so
  • r-rel060 e999b9d — 0.5.2 | three observable behaviour changes and one narrowed acceptance rule are not a patch, and an upgrader reading the number would not go looking
  • r-rel060 e999b9d — Publishing the delivery figure in the README | the measurement is one corpus and one repository, and the README already carries more numbers than a first-time reader can weigh; docs/evidence.md and bench/ hold it
  • r-release051 19810d2 — Waiting for 0.5.2 to carry index ingests any key: value line as a trailer; doctor reports 106 records where git has 0, and context serves commit subjects to the agent #335 | false decision context reaching an agent is the inversion of the product claim, and a stable release should not hold it
  • r-release051 19810d2 — 0.6.0 | no command is added and nothing breaks; calling it minor would hide a correctness fix behind a feature number
  • r-release051 19810d2 — Filtering Verified: because a report listed it as noise | it is vocabulary, and separating a release-note use from a record use needs a guess that can discard real records
  • r-convertreadme e12c816 — Adding "Keep coding normally" now | the commit skill still triggers on explicit requests and teaches harvest; shipping the sentence first repeats the doc-ahead-of-code failure this session fixed twice
  • r-convertreadme e12c816 — Writing testimonials to fill the social-proof section | one real field report exists, and fabricated or padded quotes would cost the claim honesty that is this project's strongest asset
  • r-convertreadme e12c816 — Moving the benchmark block and protocol detail out of the README now | the generated block is byte-compared by check-readme-numbers and the guard figures are asserted inside the limitations slice; the move is worth doing and is worth its own change
  • r-convertreadme e12c816 — Dropping "decision authority" everywhere | it is the right term in the architecture sections; it is only the wrong term for a first screen
  • r-release050 ad402c7 — 0.4.2 | a new command and a platform support claim are not a patch, and calling them one hides both from anyone reading version numbers to decide whether to upgrade
  • r-release050 ad402c7 — 1.0.0 | nothing here breaks an existing install, and the milestone that closed is Gate B rather than a stability commitment
  • r-release050 ad402c7 — Describing Windows as supported without the caveat | the word is the one most likely to be read as more than it is, and the release that earns it is the right place to bound it
  • r-nobin284 ab2f08f — Keeping the binary classification arm for the installer's wrapper | the wrapper is a shell script that execs node, so trusting it by name would re-admit any extensionless executable called commitlore while the bundle it runs is the thing with an interpreter to check
  • r-nobin284 ab2f08f — Relaxing the dogfood Evidence check to allow a missing path | it exists to catch a citation nobody can follow, and dropping the check to accommodate a deletion would retire the guarantee instead of correcting the question it asked
  • r-nobin284 ab2f08f — Leaving src/core/paths.ts alone because ADR-0026's inventory did not name it | it was the only remaining code that read files out of a compiled artifact, so leaving it would have left the invariant false while the test asserting it passed
  • r-nobin284 ab2f08f — Keeping /dist/commitlore in gitignore in case a binary returns | an ignore rule for an artifact nothing produces is a claim that something might, and the ADR says otherwise
  • r-rel041notes 71efe1f — folding this into the next feature release | the documented install is broken for upgraders now, and every day it stays tagged is a day the one-liner in the README fails for them
  • r-rel041notes 71efe1f — moving the v0.4.0 tag to the fixed commit | a tag users may already have fetched must not change under them; the fix gets its own version
  • r-rel040notes 5d57a72 — describing 0.4.0 as making agents safer | no study here measures agent behaviour, and the recovery protocol is registered and unrun; the release delivers a loop, not an outcome
  • r-rel040notes 5d57a72 — omitting guard's figures from the release notes because they are unflattering | the notes are read by people deciding whether to trust the tool, and a quality limit hidden at release time is discovered in use
  • r-rel021a a79e350 — retagging v0.2.0 | the tag is already on the remote and moving a published tag breaks anyone who fetched it
  • r-rel0200a a074754 — bumping ci.yml's "v0.1.0 was published with zero attached assets" comments | those describe a historical fact about the actual v0.1.0 release, not a version this project declares; the check they document (releases/latest/download/SHA256SUMS returning 200) is written to start exercising the real path automatically the day any release ships assets, v0.2.0 included, with no workflow edit
  • r-rel0200a a074754 — touching docs/adr/ADR-0001-scope-v010.md, docs/tickets/release.md, bench/VERDICT-M1.md, HANDOFF.md, bench/ROUTE-GAP.md | planning and historical-record prose that names v0.1.0 as a past decision or measurement subject, not a live version carrier
  • r-rel0200a a074754 — changing test/mcp.test.ts's CommitLore-Version: 0.1.0 fixture trailer | that's protocol-version content inside a synthetic seed commit (what an old commit's trailer looked like), unrelated to and never asserted against package.json's version
  • r-fix93pkg 9c4a396 — repointing bin at dist/commitlore.mjs instead of removing it | that entry exists only to serve a package-manager install ADR-0011 already replaced with a git clone; repointing it keeps maintaining a path to an install mode that will never happen
  • r-fix93pkg 9c4a396 — also removing files now | the issue's own "After B-09 · Single static binary — remove the Node runtime dependency #39" note groups bin/files/dependencies as one cleanup once the static binary lands; files alone is inert without a registry rather than actively broken, so doing it now means doing it twice
  • r-relinstall c6e1d04 — guessing the current version to build the asset URL directly | would need either the GitHub API (rate-limited, needs no-auth headers handled correctly) or trusting a redirect's final Location header parsing. Downloading the fixed-URL SHA256SUMS first and reading the real asset name back out of it needs neither and is what the checksum step has to fetch anyway.
  • r-relinstall c6e1d04local for scoping — not POSIX per se, but supported by dash, bash, and every shell this script is realistically piped into (verified directly, see Verified) | not used in the end; the script has few enough variables that scoping was not needed, only noted here because it was considered.
  • r-det058 695cdf6 — add the 100k history suite to CI | CI budget and deterministic local measurement are separate decisions
  • r-7a3e91 cf859e4 — inlining spec/SPEC.md and the schema into the bundle | SPEC.md would need a codegen step that itself needs a drift guard, and the package-root walk removes the reason to want it
  • r-7a3e91 cf859e4 — replacing the tsc output with the bundle | test/cli.test.ts, test/hooks.test.ts and test/mcp.test.ts import dist internals by path
  • r-0d4b81 8005227 — a longer quickstart that demonstrates context, limits, ruled-out, warnings and stale | an agent calls those itself once the MCP server is registered, so listing them teaches the human a workflow that is not theirs
  • r-4a8e15 49e12c7 — make the parser accept a bare block | it would disagree with git, and delegating to git is the one thing SPEC section 2 requires
  • r-7e5f02 e5f5e00 — hold Node 20 and downgrade both dependencies | paying maintenance cost to keep supporting an unpatched runtime
  • r-7e5f02 e5f5e00 — floor at 24 | excludes Node 22 users today for no benefit, and 22 outlives v0.1.0
  • r-7e5f02 e5f5e00 — drop engines entirely | the failure moves from install time to runtime, where it is someone else's confusing bug
  • r-5a8c04 c46a577 — regex trailer parsing | B3 prose false-positives are the failure mode this protocol exists to prevent
  • r-5a8c04 c46a577 — a custom GitError subclass | the convention here is new Error with structured context, and callers read GitResult
  • r-c0f4e2 3d249cd — GitLore published as git-lore | the binary and search results still collide with the existing gitlore tool
  • r-c0f4e2 3d249cd — keep Annals | the sound problem does not decay, and with code near zero this is the cheapest moment the project will ever have
  • r-c0f4e2 3d249cd — rename code and spec first, documents later | the drift window makes every artifact written in it wrong
  • r-b2e7f1 00d348d — line-matching Key: prefixes | prose containing a colon line parses as a record and feeds agents false context (verified B3)
  • r-b2e7f1 00d348d — content-hash Record-Id | must survive rebase and squash, which a hash of the commit does not

Warnings (17)

  • r-cdeb08an 60db89f (claim) — the orchestrator must freeze 180 opaque analysis_inputs.row_files and write canonical rows only at those paths; a row written elsewhere will be refused rather than found
  • r-rel071 af8e0ab (claim) — any option declared with a non-nullish default has the same shape, and nothing checks for it -- this fix repairs one site, not the class
  • r-rel070 d4a4d8b (claim) — between the promotion merge and the tag push, every README install one-liner points at a tag that does not exist -- tag promptly and confirm the URL rather than assuming
  • r-cdebver01 ce7b278 (claim) — every failure fixture is the one valid row with a single mutation, so keep the valid-row builder in lockstep with result.schema.json -- a builder drift fails all eleven cases at once and reads as a broken verifier
  • r-3c9d52 dc9e769 (claim)budgeted_log_coverage is not recall — it counts commits, not records, and only the calibration row at b3f5692 says how far apart the two are on a repository where both are known
  • r-nobin284 ab2f08f (claim) — an extensionless COMMITLORE_BIN is now ignored rather than honoured -- the hook falls through to the recorded install and to PATH, so a caller who pointed it at the wrapper still gets a working hook, by a different route
  • r-release030 510d70e (claim) — Tagging and publishing remain separate release steps.
  • r-rel0200a a074754 (claim) — scripts/commitlore-bootstrap.sh is orphaned -- no hooks.json entry invokes it, and its npm-install strategy contradicts ADR-0011. It still carries a live version default, now bumped for consistency, but nothing exercises it. Worth a follow-up issue: either wire it up correctly or delete it.
  • r-det058 695cdf6 (claim) — guard precision is against the frozen archived labels, not a new hand-adjudication
  • r-7a3e91 cf859e4 (claim) — hardcoding ../ counts back to the package root is what broke this — new code reads assets through installedPath(), never through import.meta.url
  • r-0d4b81 8005227 (claim)claude mcp add commitlore -- commitlore mcp is Claude Code's syntax — other MCP clients register a stdio server their own way
  • r-4a8e15 49e12c7 (claim) — canonical output is a block, not a message -- prepend a subject before parsing it back, or the round-trip silently returns nothing
  • r-7e5f02 e5f5e00 (claim) — raising the floor again needs an ADR superseding ADR-0010 -- a package.json-only change will pass tests and lie to users
  • r-5a8c04 c46a577 (claim) — do not add --grep anywhere under src/ -- test/source-guards enforces its absence and will fail the build
  • r-c0f4e2 3d249cd (claim) — ADR-0008 and ADR-0009 keep the literal string Annals on purpose -- mechanical substitution there destroys the decision trail
  • r-c0f4e2 3d249cd (claim) — the residual grep for lore_query reports a false positive because commitlore_query contains it as a substring, so check the prefix
  • r-b2e7f1 00d348d (claim) — spec/fixtures and spec/contract-cases are the contract -- prose in SPEC.md explains them, it does not define them

the index is unavailable (the index has no baseline commit); answering with a full scan

git log --follow accepts exactly one pathspec, so renames are not followed for 4 paths; query one path at a time to follow its rename chain

withheld the content of 1 record(s) graded blocked: a Verified trailer matching an injection pattern is reported, never quoted (SPEC §7)

Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR.

The benchmark cannot fill a short corpus by lowering the bar or manufacturing
records. It needs an auditable answer about the records currently available
instead: every record is retained in deterministic YAML, cutoff and provenance
facts are recorded as rejections, and task-qualification calls that history
cannot settle stay explicitly undecided.

The enumerator calls the compiled shipping query engine rather than parsing
history itself. It can therefore report the same records the product sees while
separately applying the frozen snapshot reachability test, the product-repo
exclusion, reconstructed provenance, and an explicit fork-author authorization
boundary. The schema now permits rejected and undecided rows rather than
silently encoding them as passing booleans.

Limit: wrong-path viability, deterministic oracle feasibility, code disclosure, bounded implementation, and unproven ordinary or benchmark authorship cannot be decided from history and remain undecided for human review
Blast: module
Undo: easy
Certainty: firm
Verified: npm ci; package and bench typechecks; seven focused candidate-registry tests; two builds with no dist diff; CDEB and benchmark verifiers; candidate schema validation; local history census
Record-Id: r-cdeb10reg
@MongLong0214
MongLong0214 merged commit 96f3f6f into dev Aug 11, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant