Skip to content

feat: Stage 2 Phase 3 — labelled corpus seed + executable cull (offline prove) - #3

Merged
aisona-lab merged 1 commit into
mainfrom
stage2/phase-3-corpus-cull-offline
Sep 29, 2026
Merged

aisona-lab merged 1 commit into
mainfrom
stage2/phase-3-corpus-cull-offline

Conversation

@aisona-lab

Copy link
Copy Markdown
Owner

Summary

Stage 2 (corpus / precision) offline half — not more harness packaging.

  • Labelled corpus/seed.jsonl: 22 clean / 15 defective from merged OSS PRs (requests, httpx, click, urllib3) — meets the pre-registered shape floor (≥20 / ≥15).
  • Executable cull: decide_cull + fail_on_gate encode the 2026-08-26 thresholds from docs/hardening-plan.md.
  • python scripts/corpus_cli.py prove corpus/seed.jsonl → exit 0, artifact docs/eval-runs/corpus-prove.txt.
  • Live corpus_cli.py run is BLOCKED without ANTHROPIC_API_KEY (exit 2). No invented precision numbers.
  • Action fail-on stays never — metrics do not justify flipping it.

Proof

uv run python scripts/corpus_cli.py prove corpus/seed.jsonl
# EXIT 0 — OFFLINE_PROVE: PASS  LIVE_SCORING: BLOCKED  FAIL_ON: never

Test plan

  • pytest -q green locally (114 passed)
  • ruff / black / mypy clean
  • prove command exit 0 + artifact
  • CI test workflow green

Phase 3 advances Stage 2 precision work without more harness packaging:

- corpus/seed.jsonl: 22 clean + 15 defective real hunks (requests, httpx,
  click, urllib3) meeting the pre-registered shape floor
- decide_cull / fail_on_gate encode the 2026-08-26 thresholds so the cull
  is a measurement against fixed numbers, not taste
- corpus_cli.py prove: offline proof (shape + 6/6 cull self-check); live
  run stays blocked without ANTHROPIC_API_KEY
- Action fail-on remains never — metrics do not justify flipping it
@github-actions

Copy link
Copy Markdown

lazycoder review — 💥 ERROR

Downloading pydantic-core (2.0MiB)
Downloading anthropic (1.3MiB)
 Downloaded pydantic-core
 Downloaded anthropic
Installed 16 packages in 32ms
error: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits.'}, 'request_id': 'req_011CfXjB5fhFH42Qj9kMXqZX'}

0 finding(s), 0 abstention(s), 0 rule error(s) — R1..R17 rubric via lazycoder

@aisona-lab
aisona-lab merged commit 692eb05 into main Sep 29, 2026
1 of 2 checks passed
@aisona-lab
aisona-lab deleted the stage2/phase-3-corpus-cull-offline branch September 29, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant