Skip to content

feat(#453): Quality & Risk Index — Aggregated Risk Score + Complexity Index (closes epic #450)#465

Merged
richard-devbot merged 3 commits into
mainfrom
feat/453-quality-risk-index
Jul 22, 2026
Merged

feat(#453): Quality & Risk Index — Aggregated Risk Score + Complexity Index (closes epic #450)#465
richard-devbot merged 3 commits into
mainfrom
feat/453-quality-risk-index

Conversation

@richard-devbot

@richard-devbot richard-devbot commented Jul 22, 2026

Copy link
Copy Markdown
Owner

#453 — Quality & Risk Index: Aggregated Risk Score + Complexity Index (closes #453, completes epic #450)

Telemetry was financially instrumented but qualitatively blind — the Hub could say "green, cost $X" but not "green, but complexity spiked and two high-severity risks were accepted." This adds the qualitative counterpart, computed server-side from the run's real artifacts + events, with honest nulls.

What ships

  • Aggregated Risk Score (0-100) — builder-reported risks weighted by severity (critical 25 / high 12 / medium 5 / low 2), discounted when mitigated (an accepted risk is NOT mitigated — it keeps full weight), plus guardrail blocks (accepted overrides weigh less than hard blocks) and validator hard-blocks. One index, not a bare count.
  • Complexity Index (0-100) — from real structural signal: files touched (≤50pts), builder-task breadth (≤25pts), executed-command count (≤25pts, from Transient Sandbox Execution — 'The Scientist' (execute-and-observe, real evidence) #452 execution evidence) — never self-report.
  • Cost-to-Value — cumulative cost vs delivered proof coverage (never divides by zero).
  • Execution posture — container-verified vs unverified counts (Transient Sandbox Execution — 'The Scientist' (execute-and-observe, real evidence) #452 signal), as context for the indices.
  • Dedicated "Quality & Risk" BI card on Overview — band-colored score tiles + severity/mitigation/block chips + execution/cost-to-value line.

Honest by contract (the acceptance criteria)

  • ✅ Computed server-side (state/quality-risk.js) — no second brain in the client.
  • Honest nulls: unknown when a source stage is absent; a genuine 0 (contracts present, nothing flagged) is a real, reportable score, distinct from unknown.
  • BI only — never blocks a gate or changes a verdict.
  • ✅ Surfaced as first-class BI, not just PASS/FAIL green.

Design decisions (your sign-off this session)

  • Fixed transparent v1 weights — one documented, opinionated formula (can become config-tunable later without changing callers).
  • Dedicated Overview card — alongside cost/coverage, not folded into the readiness panel.

Verification

  • 1607/0 full suite; typecheck 0; lint 0 errors; validate 196/196; security OK; git diff --check clean.
  • Browser regression green — Overview renders with zero page errors and no axe serious/critical violations (the new card DOM is clean).
  • New tests/quality-risk-453.test.js: severity/mitigation normalization, weighted score + mitigation discount, guardrail/validator contributions, 100-cap, real-0 vs honest-null, complexity roll-up, execution posture, cost-to-value (incl. no /0), and the projection's focus-run/coverage/cost wiring.

This closes epic #450 (Path to Autonomy)

With #451 (critique loop), #452 (Transient Sandbox trilogy), and now #453, the autonomy program is complete: untrusted code runs confined → the gate uses the real exit code → the Critic learns from real logs → and the Hub now reports real risk & complexity, not just cost.

Closes #453

Summary by CodeRabbit

  • New Features

    • Added a Quality & Risk Index to the Command Center.
    • Displays aggregated risk, complexity, execution verification, severity indicators, and cost-to-value insights.
    • Shows clear unknown placeholders when evaluation data is unavailable.
    • Quality and risk information is now included in dashboard overview data.
  • Bug Fixes

    • Added safeguards for missing data and zero-coverage cost calculations.

richardsongunde and others added 2 commits July 22, 2026 13:47
… Complexity Index

Server-side (state/quality-risk.js), computed from a run's REAL artifacts +
events, the qualitative counterpart to cost/coverage telemetry:
- Aggregated Risk Score (0-100) — builder risks weighted by severity, DISCOUNTED
  when mitigated (accepted != mitigated), plus guardrail blocks (accepted
  overrides weigh less than hard blocks) and validator hard-blocks; rolled into
  ONE index, not a bare count.
- Complexity Index (0-100) — from files touched, builder-task breadth, and
  executed-command count (#452 execution evidence) — never self-report.
- Cost-to-Value — cumulative cost vs proof coverage (never /0).
- Execution posture — container-verified vs unverified (#452 signal).
Transparent documented v1 weights. HONEST NULLS: unknown when a source is
absent, a real 0 when contracts exist but nothing is flagged. BI only — never
blocks a gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- index.js: attach buildQualityRiskProjection after readiness (reads coverage
  from it), carried forward to overview.
- overview.js: embed qualityRisk in the Overview projection output.
- command-center.js: a dedicated 'Quality & Risk' card (Risk Score + Complexity
  Index with band colors, severity/mitigation/block chips, execution posture,
  cost-to-value), honest '—/unknown' when a source is absent; injected once via
  the established ensureCommandWavePanels pattern.
- styles.js: card styles (band-colored score tiles, tone chips).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@richard-devbot, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 44 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 29b2e491-0633-487b-bf5a-723f1186b774

📥 Commits

Reviewing files that changed from the base of the PR and between 87c1d96 and 0c9472b.

📒 Files selected for processing (1)
  • src/observability/dashboard/state/quality-risk.js
📝 Walkthrough

Walkthrough

Adds server-side Quality & Risk scoring for risk, complexity, execution posture, and cost-to-value, wires it into dashboard state, and renders the resulting data in a styled Command Center panel with null-safe placeholders.

Changes

Quality Risk Index

Layer / File(s) Summary
Quality risk computation and validation
src/observability/dashboard/state/quality-risk.js, tests/quality-risk-453.test.js
Adds scoring, normalization, mitigation, complexity, execution, cost-to-value, projection logic, and tests for null-safe behavior and score semantics.
Dashboard state integration
src/observability/dashboard/state/index.js, src/observability/dashboard/state/overview.js
Builds the quality risk projection during full-state construction and exposes it through the overview projection.
Command Center quality risk panel
src/observability/dashboard/ui/pages/command-center.js, src/observability/dashboard/ui/styles.js
Adds the Quality & Risk panel, renders scores and details, and defines its responsive styling and score states.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Dashboard
  participant buildFullState
  participant buildQualityRiskProjection
  participant CommandCenter
  Dashboard->>buildFullState: construct dashboard state
  buildFullState->>buildQualityRiskProjection: compute qualityRisk
  buildFullState-->>CommandCenter: provide overview.qualityRisk
  CommandCenter->>CommandCenter: render Quality & Risk card
Loading

Possibly related PRs

Suggested reviewers: richardsongunde

Poem

I’m a rabbit with risks in my hat,
Scoring each danger, both big and flat.
Complexity hops, costs softly glow,
Unknowns stay honest wherever they go.
The dashboard now shines with a careful new view—
Hop, hop, quality risk, all ready for you!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 72.73% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly names the Quality & Risk Index and its main outputs, matching the PR scope.
Linked Issues check ✅ Passed The PR implements server-side risk, complexity, cost-to-value, and BI surfacing with honest nulls, aligning with #453.
Out of Scope Changes check ✅ Passed The changes are limited to the quality-risk projection, dashboard UI, styles, and tests, with no unrelated additions.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/453-quality-risk-index

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…age)

Brief doc comments on bandFor / clampScore / collectRisks / focusRun to satisfy
the docstring-coverage advisory; no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@richard-devbot
richard-devbot merged commit eff3b2d into main Jul 22, 2026
9 checks passed
@richard-devbot
richard-devbot deleted the feat/453-quality-risk-index branch July 22, 2026 08:37
richard-devbot pushed a commit that referenced this pull request Jul 22, 2026
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Quality & Risk Index: Aggregated Risk Score + Complexity Index from the 15 stages

2 participants