Skip to content

feat(review): add paired quality evaluation and OpenELIS design guidance - #8

Draft
pmanko wants to merge 1 commit into
c/ogc-1560-review-isolationfrom
c/ogc-1560-quality-evaluation
Draft

pmanko wants to merge 1 commit into
c/ogc-1560-review-isolationfrom
c/ogc-1560-quality-evaluation

Conversation

@pmanko

@pmanko pmanko commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Adds paired plain-review/harness evaluations with frozen cases/settings, repeated trials, private retained inputs and independent adjudication/scoring. Partial/error runs remain visible; provisional development labels cannot establish review quality. Also adds OpenELIS mock discovery and preparation guidance using openelis-work and its published gallery. Draft references no longer block comparison, and readable reports retain design observations alongside functional/saved-state outcomes.

Validation: 79 tests and seven skill definitions passed; packaging, 45 local documentation links and whitespace checks passed. Eight lightweight Sol development reviews completed execution (four complete reports, four partial). A retained-evidence OpenELIS L5 reassessment completed after eight source inspections, recording visible draft-mock differences without demanding approval. No runtime re-execution is claimed for that reassessment.

The checklist and evidence record now reconcile the full stack. Independent labels/adjudication and held-out quality evaluation remain open; one provisional valid-case expectation needs clarification. No accuracy or superiority claim is made. Unattended deployment acceptance and Claude image conformance also remain explicit limitations. Nothing is merged.

Current implementation checklist · OGC-1560

Stack created with GitHub Stacks CLI • Give Feedback 💬

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant