Skip to content

fix(claims): two model families by default, live HAL badge, narrowed zkRepID copy, credits (MEDIUM for the voter default, LOW otherwise) - #1462

Merged
DealAppSeo merged 1 commit into
mainfrom
claude/honesty-credit-2026-10-23-94fb8f
Oct 24, 2026
Merged

DealAppSeo merged 1 commit into
mainfrom
claude/honesty-credit-2026-10-23-94fb8f

Conversation

@DealAppSeo

Copy link
Copy Markdown
Owner

Repo-alignment sprint (approved by Sean, 2026-10-23): make this repo's public claims true, code first, copy narrowed only where code can't fix it in about two days. One PR.

Weight. Item 2, the change to the default pair of /classify voters, is MEDIUM: it changes live behaviour, but only on a deployment with no Cerebras key. Production has the key, so production keeps the same pair (see the proof column). Everything else is LOW: copy that narrows or corrects a claim, docs, tests.

Code change (MEDIUM): the default checker pair is always two model families

src/classify/free-votes.ts. Before: with no CEREBRAS_API_KEY, activeVoters returned gpt-oss-120b + gpt-oss-20b on Groq, one family. S47 already made a one-family pair answer not-checked every time, so that default spent two calls on an answer that could never decide. A malformed CLASSIFY_VOTERS fell back to the same pair.

After: the default is always groq:openai/gpt-oss-120b + cerebras:qwen-3.8-27b, and parseVoters falls back to it too. With no Cerebras key, that slot gives no answer (no_key, nothing sent). THE FALLBACK may then fill it with a canary-ready model of another family, and otherwise the label is not-checked. The same-family DEFAULT_VOTERS constant is deleted, so every test that used it had to be looked at by hand rather than silently meaning something else.

  • Fails before / passes after: tests/classify-two-families-default.test.ts. On untouched main: 8 failed, 2 passed (the two Cerebras-keyed cases already held). On this branch: 10/10.
  • Existing tests that pinned the old one-family default now pin the new rule. The question-flow and assumption tests pin their pair explicitly through CLASSIFY_VOTERS or add the Cerebras key, because they test those flows, not the default.
  • Research behind the wording: Kim, Garg, Peng, Garg, Correlated Errors in Large Language Models, ICML 2025, arXiv 2506.07962. Across 350+ models, when two models are both wrong on one leaderboard dataset, they give the same wrong answer 60% of the time, and larger models stay correlated across different architectures and providers. So two families are "different", not "independent".

Every changed claim

# Where Before After Proof
1 README.md:8 Static badge HAL: live Two shields.io dynamic badges reading GET /api/v1/hal/stats: last_24h_classifications and quorum_health.answering_providers Live at 2026-10-23 18:59Z: last_24h_inferences: 0, 3 of 5 configured providers answering. shields.io rendered 0 and 3 from that endpoint. tests/readme-claims.test.ts fails if a badge reads a field src/routes/hal-stats.ts does not return. No code was needed: the public stats JSON already existed.
2 src/classify/free-votes.ts No Cerebras key → Groq x2, one family Always two families; a missing key fails closed to not-checked unless a canary-ready stand-in of another family answers See above. Production pair unchanged: /api/v1/classify/stats lists groq:openai/gpt-oss-120b and cerebras:qwen-3.8-27b, both canary ok at 2026-10-23 15:29Z.
3 src/routes/telegram-public.ts (bot replies) "two independent checks agree" "two checks from different model families agree" Kim et al. above. Telegram tests compare against replyFor, unchanged.
4 src/hal/fact-check.ts decision_reason / quorum_note (public API text) "N of M independent model families/providers judged…" "N of M different model families" when family-aware, else "N of M model providers" When not family-aware the old text called possibly-same-family providers "independent model providers", which was wrong twice. No consumer parses these strings: searched trustshell, trustrepid, trinity-symphony-shared, hyperdag-landing, hyperdag-protocol, repid, trustrails-dev.
5 README.md:63 "zkRepID makes the score checkable without revealing it" "zkRepID proves the score clears a threshold (a range proof; the score is public)" The deployed statement carries repid_score in the clear (src/zkp/proof-statement-guard.ts); the README's own table marks selective disclosure TARGET. Hiding the score needs a new circuit in the prover repo, so this is copy, not code.
6 scripts/demo/trust-receipt.mjs:150, scripts/demo/trust-harness-e2e.mjs:238 (demo output) "without revealing the score" "the score is public (a range proof)" Same as 5. The harness demo contradicted the README's own description of it.
7 CLAIM_LEDGER.md (a VERIFIED row, citeable externally) "statement RepID ≥ 999 without revealing 2070" "with the score (2070) public in the statement", with the correction dated The row's own evidence printed 2070.
8 docs/AGENT-HARNESS-ADOPTION-PLAN.md:121, docs/ZKREPID.md:4, headers of src/zkrepid/index.ts and src/zkrepid/boundary.ts "proves RepID … without revealing the score" "meant to prove … today's statement carries the score in the clear" Same as 5.
9 README.md Evaluation, docs/ARCHITECTURE.md, docs/HAL_TAMPERING_DETECTION.md, docs/RSI-ADOPTION-PLAN.md, eval/kernel/README.md, eval/candidates/README.md "vendor-independence", "model independence is partial", "independent LLMs", "two INDEPENDENT families" "different model families, not independent ones", with the Kim et al. citation where it explains why tests/readme-claims.test.ts fails if the README calls the checkers independent again.
10 README.md top — The hub blurb from the sprint text: role, does / does not, links to BUILDERS.md and ACKNOWLEDGEMENTS.md, "Test network only (Base Sepolia)" "Does not prove scores itself": src/config/prover.ts (every proof job in 30 days used the HyperDAG-core prover). "Test network only": the last real settlement in /api/v1/passport/trinity-shofet, tx 0x7a713be1…caa056 (2026-10-20), is on Base Sepolia (chain 84532) and returns null on Base mainnet [RPC lookup 2026-10-23]; the base entry in src/config/network.ts has no registry addresses. The "Called by" list is not extended.
11 README.md new Credits section Plonky3, x402, TruthfulQA not credited Credits table plus the MIT note Licences read at source today: Plonky3 MIT or Apache-2.0 (its README, Polygon Zero team); coinbase/x402 Apache-2.0 and now x402-foundation/x402 Apache-2.0 (its README says the repo moved); TruthfulQA Apache-2.0, and our CSV is the same git blob as upstream TruthfulQA.csv (4206d1e); FEVER CC BY-SA 3.0 (fever.ai dataset page); HaluEval MIT. Row counts from eval/rigorous/rigorous-corpus-v1.jsonl: FEVER 100 (verbatim), HaluEval 100, TruthfulQA 90. zkp-vault/Cargo.toml has p3-poseidon2; no x402 package is imported anywhere in src/ or scripts/.
12 README.md Credits — contracts/RepIDStaking.sol is SPDX-License-Identifier: MIT inside an Apache-2.0 repo: noted, not relicensed File header.
13 CLAUDE.md:73, CLAUDE.md:96 "proprietary … The code remains proprietary by licence" Apache-2.0 per LICENSE and package.json; if proprietary was the intent, the licence is what would change, and that is Sean's call LICENSE, package.json "license": "Apache-2.0", README badge.
14 README.md License No copyright holder named anywhere "Copyright 2026 Sean Goodwin" Sprint brief; first commit 2026-04-14 by Sean Goodwin. LICENSE text untouched.
15 README.md footer Duplicate "Part of the HyperDAG Protocol ecosystem" line Removed; the blurb at the top carries it —
16 src/routes/telegram-public.ts comment "Names every host a claim can reach" Says it names the default voters' hosts and that a fallback stand-in is not named yet See follow-ups.

Verification

  • npx tsc --noEmit: clean.
  • npm run check: VERIFIED (check:named-env-vars).
  • Full npm test locally on Windows (2 workers): 12,048 passed, 156 failed in 19 suites. I classified every failing suite by running it on untouched main (fb1e350) in the same checkout:
    • 16 suites fail on main too: ENV. The causes are the Windows-only ERR_UNSUPPORTED_ESM_URL_SCHEME (384 hits), no reachable Supabase (fetch failed, ECONNREFUSED), and the missing babybear-leaf binary.
    • 3 suites were load timeouts. client-ip-limiters, kernel/cc1-kernel-fixes and verify-proof failed under the full run ("Exceeded timeout of 5000 ms") and pass alone on this branch.
    • After rebasing onto fb1e350, readme-edges failed to load because of CRLF line endings: its parser wants straight after the code fence, and a Windows autocrlf checkout gives `

. With LF endings, which is what CI gets, it passes 13/13 along with readme-claims. Same on main`.

  • All suites this PR touches pass: classify (6 files), telegram, ledger, hal/fact-check, quorum-receipt-writer, zkrepid-boundary, readme-claims, repid-staking-access-control. That is 565/565 tests.
  • tests/readme-claims.test.ts against main's README: 4/4 fail; against this branch: 4/4 pass.

Not changed, and why

  • /api/v1/hal/stats reports a failed count query as 0 (classifications.last_24h ?? 0), so on a DB error the new badge would show 0, not "unknown". Fixing it changes a public field that trustrepid and hyperdag-landing read, so it is a follow-up, not a quiet edit here.
  • The Telegram privacy line, and every door's privacy line, names Groq and Cerebras but not a fallback stand-in (Cloudflare Workers AI, OpenRouter), which can receive the text when a voter gives no answer. This was already true before this PR; production behaviour does not change. It is shared privacy wording across doors (BUS N-PRIVACY-LINE), so it needs its own change.
  • FEVER's 100 rows are CC BY-SA 3.0 (share-alike) inside an Apache-2.0 repo. Noted in Credits, not relicensed or moved: Sean's call.
  • Internal comments and identifiers that say "independent family" (independentFamilies, auditFamilyIndependence, HAL_STRICT_FAMILY_INDEPENDENCE, comments under src/hal/, src/providers/) are left as they are: renaming identifiers is a refactor, and those comments are not public claims.
  • The ACKNOWLEDGEMENTS.md link 404s until docs: credit the ERC-8004 team, map the trust stack, say why, narrow claims (LOW + approved vision text) hyperdag-protocol#61 lands. That draft credits x402 to coinbase/x402, whose README now says it moved to x402-foundation/x402.
  • trustshell copy still says "independent model families" (components/compatibility-info.tsx, lib/glossary.ts, app/mission/page.tsx, public/llms.txt). Another repo, so not touched here.

🤖 Generated with Claude Code

…zkRepID copy, credits (MEDIUM for the voter default, LOW otherwise)

- /classify's default voter pair is now always gpt-oss on Groq + qwen on
  Cerebras. With no Cerebras key the old default was gpt-oss x2, one family,
  which S47 already forced to not-checked. A missing key now fails closed or
  hands the slot to a canary-ready stand-in of another family.
  tests/classify-two-families-default.test.ts: 8/10 fail on main, 10/10 here.
- README: the static "HAL: live" badge becomes two shields.io badges reading
  /api/v1/hal/stats; tests/readme-claims.test.ts pins the fields to the route.
- "Independent" checkers become "different model families" in README, docs,
  the Telegram bot replies and HAL's decision_reason text (Kim et al.,
  ICML 2025, arXiv 2506.07962: errors correlate across families too).
- zkRepID: "checkable without revealing it" becomes a range proof with the
  score public, in README, both demos, a VERIFIED CLAIM_LEDGER row and docs.
- Credits for Plonky3, x402, TruthfulQA, FEVER (CC BY-SA 3.0) and HaluEval,
  each licence read at source; MIT note for contracts/RepIDStaking.sol;
  hub blurb; CLAUDE.md no longer calls the Apache-2.0 code proprietary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DealAppSeo
DealAppSeo marked this pull request as ready for review October 23, 2026 21:09
@strix-security

strix-security Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Strix Security Review

✅ No open security findings.

PR summary: This PR is a claims-honesty sweep: it changes the default `/classify` voter pair to always be two model families (Groq gpt-oss + Cerebras qwen) instead of falling back to two Groq gpt-oss models, and rewrites README, docs, comments and user-facing copy to stop calling the checkers "independent", stop claiming the zkRepID proof hides the score, add a live HAL badge, add a credits/licence section, and pin the corrected claims with new tests.

Review summary

Reviewed all 27 changed files in the diff. The only behavioural change is in src/classify/free-votes.ts: activeVoters/parseVoters now always default to the two-family pair (Groq gpt-oss + Cerebras qwen) and the one-family DEFAULT_VOTERS constant is removed. I traced the classification path end to end (including THE FALLBACK and the final oneFamily gate) and the two-family guarantee still holds — with no Cerebras key the empty slot abstains no_key, stand-ins are excluded by family and by already-asked host, and a label of pass/veto still requires two deciders on different hosts of different families. Every other change is comment, documentation, marketing copy, demo output, or test code, and no authentication, validation, cryptographic, or data-handling control is added or removed. No security vulnerabilities found in the changed code.

Review 1 on this PR · 441b781 · requested Oct 23 21:09 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

@strix-security strix-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all 27 changed files in the diff. The only behavioural change is in src/classify/free-votes.ts: activeVoters/parseVoters now always default to the two-family pair (Groq gpt-oss + Cerebras qwen) and the one-family DEFAULT_VOTERS constant is removed. I traced the classification path end to end (including THE FALLBACK and the final oneFamily gate) and the two-family guarantee still holds — with no Cerebras key the empty slot abstains no_key, stand-ins are excluded by family and by already-asked host, and a label of pass/veto still requires two deciders on different hosts of different families. Every other change is comment, documentation, marketing copy, demo output, or test code, and no authentication, validation, cryptographic, or data-handling control is added or removed. No security vulnerabilities found in the changed code.


Reviewed by Strix
Configure security review settings

@DealAppSeo
DealAppSeo merged commit 2195aa3 into main Oct 24, 2026
12 checks passed
@DealAppSeo
DealAppSeo deleted the claude/honesty-credit-2026-10-23-94fb8f branch October 24, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant