Skip to content

fix: show Copilot quota usage past the allowance (#318) - #319

Merged
HemSoft merged 2 commits into
mainfrom
fix/issue-318-copilot-quota-overage
Nov 7, 2026
Merged

HemSoft merged 2 commits into
mainfrom
fix/issue-318-copilot-quota-overage

Conversation

@HemSoft

@HemSoft HemSoft commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Risk level: 4/10 (codexbar)
Worked on by: claude-opus-5-5 on Laptop; quota-decoder refactor by gpt-6.1-sol on Home through the owned Laptop checkout, for Chief dispatch 358679b9

Summary

GitHub's copilot_internal/user reports remaining below zero once a quota passes its entitlement. quota_metric clamped that to zero, so an account 740 premium requests over a 3,000 allowance read as 3,000, the same as one just at it.

  • quota_used in crates/codexbar-providers/src/copilot.rs works in u64 only. A non-negative remaining is subtracted with saturating_sub, and a negative one adds its unsigned_abs() with checked_add. The entitlement is never narrowed to i64, and no input wraps or panics.
  • Behavior change for malformed input: a quota whose remaining is missing, null, fractional, a string, a boolean or past u64::MAX is now left out. Before, it showed the whole entitlement as used. This follows the rule parse_user already applies to absent quotas ("produce no metric rather than a false 0%") and to its own entitlement check. The sibling quotas still show.
  • Behavior change for the forecast: the over-allowance account's pace now comes from 3,740 used, so its "at reset" share reads 174% instead of the 159% it shared with an account at the limit. The share itself stays at 100% (used_fraction clamps), and the account still reads limit reached.
  • Dashboard::capture_copilot_overage (test and capture builds only) runs two synthetic payloads through the real parse_user: entitlement 3000 with remaining -740, and with remaining 0. CODEXBAR_CAPTURE_COPILOT_OVERAGE=1 saves the Usage view, and a headless UI test drives the same scenario.

Billing-API AI credits (credits_metric) are unchanged.

Acceptance Criteria

  • remaining: -740 with entitlement: 3000 gives a premium quota with used 3740 and limit 3000, and its value column reads 3,740: parse_user_quota_past_its_entitlement_counts_the_overage for the premium, chat and completions quotas; copilot_quota_past_its_allowance_shows_the_overage_and_stays_limit_reached (used-copilot-over-allowance = 3,000 requests: 3,740); the screenshots below.
  • remaining: -1 gives a different used from remaining: 0: the same test, 301 against 300.
  • A remaining at or below the entitlement keeps today's result: 1500/216 gives 1284, 300/300 gives 0, 300/5000 gives 0, and the recorded fixture still gives 1284 of 1500 (parse_user_premium_quota_maps_used_limit_and_reset).
  • A remaining too large to fit, or one that isn't an integer, is not read as a giant or wrapped used and doesn't panic: parse_user_quota_used_stays_exact_at_both_ends_of_the_integers and parse_user_quota_without_a_whole_remaining_is_left_out.
    • i64::MIN adds exactly 2^63.
    • u64::MAX - 740 with -740 gives exactly u64::MAX, and -741 is left out.
    • u64::MAX with -1 or i64::MIN is left out.
    • A remaining of u64::MAX gives 0 used.
    • 18446744073709551616, -1e30, 216.5, 216.0, -740.5, "216", true, null and a missing remaining are all left out.
  • The account still reads as limit reached, and alerts still treat it as used up: parse_user_quota_past_its_entitlement_is_used_up_like_one_at_it (same is_exhausted and severity as remaining 0). In the UI test, both rows read 100% used, … · limit reached with no severity pill.
  • Entitlement handling is unchanged: parse_user_quota_entitlement_limits_are_unchanged covers a missing, 0, -5, 300.5, "300" or null entitlement (still no metric), unlimited past the entitlement (still no metric) and an entitlement of 1. parse_user_unlimited_quota_has_no_metric and parse_user_free_plan_reports_chat_and_completions still pass.

Definition of Done

  • parse_user regression tests cover the negative, -1, zero and under-limit cases, plus both integer ends and malformed input, using synthetic payloads built in the tests.
  • Reverting the fix makes the negative-remaining tests fail. RED/GREEN is below.
  • Current-head local formatting, Clippy, production-binary check, all nine parser regression tests, and capture build/Clippy pass. The full workspace run has the Windows SSH limitation recorded below.
  • All seven required CI checks pass on final head 46223e9, including rust-metrics, rust-mutations and Windows workspace gates: CI run 42334236986. All jobs and executed steps verified successful.

Validation

Merged as 58a0945970ab95f5a9e3c9e770473813f7c32105 at 2026-11-06 23:01:34 EST using the normal squash helper with exact-head refusal guard. Reviewed and merged trees match 4e438cb2e9c414d419a203a718bdeb090e4e2012; exact-head SFL PASS, seven CI jobs/all executed steps PASS, zero unresolved threads, clean gh x status. Issue #318 auto-closed. Source delivery verified on main: CI run 42336587226 passed all seven jobs and every executed step at live commit 58a0945970ab95f5a9e3c9e770473813f7c32105. Mini primary is clean and current. Home source sync preserved its pre-existing fix/300-checked-durations branch (nine commits behind); Laptop main audit preserves pre-existing untracked audit material and other worktrees. The owned Laptop PR worktree is clean and its tree equals merged main; source worker settled. Completed owned build caches were removed on Mini and Laptop; proof files, logs and captures remain retained. No installed-runtime or release delivery is claimed.

Current head: 46223e9fc4483934fb1826e10af3ecfba2946cc2. CI run 42334236986: all seven required jobs and every executed step PASS, including Windows/macOS/Ubuntu format, lint, production-binary and workspace tests, security, mutations and strict metrics. SFL Round 2: exact-head PASS, confidence 8/10, no blocking findings, zero unresolved review threads. Author/editor/edit-history authentication verified; the review is unedited.

  • All nine parser regression tests pass locally on Laptop. Local formatting, all-targets Clippy, production-binary check, capture build and capture Clippy pass. Full local SSH workspace test command exited 101 solely on unchanged credentials::tests::windows_store_round_trip with Windows error 1312 (missing/terminated logon session): app 437 PASS (1 ignored), core 123, metrics 2, providers 364 (5 ignored), reset and contract suites 6 each, command-runner fixture qualified, store 162 PASS/1 failed. The full required Windows CI gate subsequently passed at this exact head; no test was skipped or weakened.
  • The original CI failure at d563530 measured quota_metric CRAP 5 against base 3.002197. Current-head CI independently measures quota_metric complexity/CRAP 2.0 at 100% line coverage, quota_counts 5.0/100%, and quota_used 4.0/100%. Overall production line coverage rises from 88.370829% to 88.378898%; branch coverage from 80.355249% to 80.381760%. gate.json has no failures; exact base/head and artifact digest verified. No policy, scope, baseline, threshold or existing function identity changed.
  • Historical RED/GREEN at original implementation d563530, with the same unchanged parser tests: restoring only the old remaining/used calculation gave 6 PASS/3 FAIL (overage, integer limits, malformed remaining). Restoring the fix gave 9 PASS. Positive controls passed both ways.

Fresh UI evidence: Mini, Linux X11/llvmpipe in isolated Xvfb, default 1440x960 app window, light and dark themes. All 535 archived source files matched the current commit blobs before the fresh offline build. Demo-only data and fresh settings; no provider called. Repeatable commands:

CARGO_TARGET_DIR=<isolated-target> CARGO_BUILD_JOBS=4 cargo build -p codexbar-app --locked --offline --features capture
CODEXBAR_CAPTURE_DIR=<fresh-capture> CODEXBAR_SETTINGS_DIR=<fresh-capture>/settings \
CODEXBAR_CAPTURE_COPILOT_OVERAGE=1 CODEXBAR_CAPTURE_X11=1 LIBGL_ALWAYS_SOFTWARE=1 \
xvfb-run -a -s '-screen 0 1600x1000x24' <isolated-target>/debug/codexbar --demo

Capture exit 0, empty stderr. report.json records both copilot-overage-usage frames. The fixture asserts the real rendered table labels before each save: 3,000 requests: 3,740 / 100% used, 174% at reset · limit reached, and the at-limit control 3,000 requests: 3,000 / 100% used, 159% at reset · limit reached. Both PNGs were downloaded from their GitHub URLs, hash-matched to the retained capture, and visually inspected. The prior Windows SSH capture failed because it could not open a window while headless; these current-head Linux captures provide the fresh screenshot evidence.

Current-head light theme: Copilot over-allowance row reads 3,740 used of 3,000, with 174% at reset and limit reached; control reads 3,000 and 159%

Current-head dark theme: the same 3,740 over-allowance and 3,000 at-limit rows, both limit reached

Light SHA-256: 074a3118e4608c56cc19d9262e6572684ca21a7e6fd5a21428a57412c59b022b. Dark SHA-256: 36b681d767fd3769feab847b4de277d15839c0ef66778eb25117fdc0251bb26f.

No recording: only values in existing table cells change, with no interaction or navigation change.

Risks and deferred work

  • The Copilot shape past the allowance comes from a public report (slkiser/opencode-quota#300), not from a recorded CodexBar response, so the tests use synthetic payloads.
  • A remaining of 0 with a non-zero overage_count is still read as exactly at the limit. That shape isn't confirmed, so it's out of scope here.
  • SFL Round 1 reports a non-blocking Low in the informational forecast: it starts from the clamped 100% share and adds the pace calculated from actual over-allowance usage. The fixture therefore reads 174% rather than a linear projection of about 198%. The parser count, limit-reached state, alerts and severity remain correct. The existing forecast formula is unchanged by this PR; that separate forecast behavior is not claimed fixed.

Closes #318

🤖 Generated with Claude Code

GitHub reports `remaining` below zero once a Copilot quota passes its
entitlement. parse_user clamped it to zero, so 740 requests over a 3,000
allowance read as exactly 3,000. Add the overage with checked u64
arithmetic, and leave out a quota whose `remaining` isn't a whole number
instead of showing it used up.

Add a synthetic capture and a headless UI test that run the real parser
for an account over its allowance and one at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HemSoft HemSoft added bug Something isn't working model:claude-opus-5-5 PR authored by claude-opus-5-5 labels Nov 7, 2026
@HemSoft

HemSoft commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

@sfl review

@hemsoft-sfl-reviewer hemsoft-sfl-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Verdict: PASS • Round 1

Confidence: 8/10

Review done by claude-opus-5-5 (same family: same-family: codex gated: codex Weekly 166% SUS 60 🔴 pause, defer)

The PR adds quota_used so that a negative Copilot remaining counts as usage past the entitlement (3,000 allowance with -740 shows 3,740), and drops a quota whose remaining is not a whole number instead of showing it as fully used. It also adds parser tests, a headless UI test and a demo-only capture. The arithmetic is checked and overflow-safe, and every downstream consumer of quota used either clamps it through used_fraction or only formats it, so the only defect found is a Low forecast inconsistency that does not block approval.

🔨 Full-Spectrum Review Summary

Effort: high
Review time: 1m 59s
Token count: Unavailable

Pass 🔴 Critical 🟠 High 🟡 Medium ⚪ Low
🔒 Security 0 0 0 0
✅ Accuracy 0 0 0 1
🧰 Quality & Maintainability 0 0 0 0
Total 0 0 0 1

Key Findings

  1. ⚪ LOW — Accuracy: Over-allowance 'at reset' forecast mixes a clamped share with an unclamped pace. crates/codexbar-providers/src/copilot.rs:139: Observed: elapsed_pace now gets the unclamped fraction used/entitlement (3740/3000 ≈ 1.247), but Metric::projected_at_reset in codexbar-core/src/metric.rs starts from used_fraction(), which is clamped to 1.0, and adds pace × time left. Trigger: any account past its allowance mid-period, such as the fixture at day 19.5 of 31. Impact: the label reads 174% at reset. That is neither the true projection (about 198%, from 1.247 plus the same pace term) nor the earlier 159%, and the new UI test and capture pin this mixed number as expected. The row still shows limit reached, and alerts and severity use the clamped share, so the effect is limited to the informational forecast text. The PR text acknowledges the behavior. (non-blocking)

Coverage

  • Files reviewed:
    • crates/codexbar-providers/src/copilot.rs
    • crates/codexbar-app/src/dashboard.rs
    • crates/codexbar-app/src/self_capture.rs
    • crates/codexbar-app/src/self_capture/copilot_overage.rs
    • crates/codexbar-app/src/ui_tests.rs
  • Context read:
    • crates/codexbar-providers/src/pace.rs
    • crates/codexbar-providers/tests/fixtures/copilot-user.json
    • crates/codexbar-core/src/metric.rs
    • crates/codexbar-core/src/sus.rs
    • crates/codexbar-core/src/alerts.rs
    • crates/codexbar-core/src/severity.rs
    • crates/codexbar-app/src/account_table.rs
    • crates/codexbar-app/src/widget_feed.rs
    • crates/codexbar-app/src/focus_cards.rs
    • crates/codexbar-app/src/usage_bar.rs
    • crates/codexbar-store/src/snapshots.rs
  • Verified:
    • Compare base_commit.sha is b1327bdaf229e5db2a79ae394fccdf96b4a24ac7 and the last commit is d563530b7aafb0faae20186d4c00fa6b0071a134: ahead 1, behind 0, 5 files changed
    • quota_used: non-negative remaining goes through saturating_sub, negative through checked_add of unsigned_abs; neither path can wrap or panic, including i64::MIN and u64::MAX
    • Floats, strings, booleans, null and missing remaining now make quota_metric return None; previously they showed the whole entitlement as used. Siblings in parse_user are unaffected because each quota is mapped on its own
    • Workspace grep found no limit - used arithmetic on Metric::Quota. used_fraction, is_exhausted, time_to_limit and outlook clamp to 0..=1, and the store saves used as u64
    • Checked the 174% and 159% labels by hand against projected_at_reset: clamped 1.0 plus pace over 468 h elapsed and 276 h left
    • The capture path is limited to --demo, isolated settings and synthetic payloads; it reads no account or credential
    • Read-only check-run listing for the head: rust on Windows, Ubuntu and macOS, build, rust-mutations and security-scan succeeded; rust-metrics was still in progress
  • Not verified:
    • That GitHub really reports a negative remaining past the allowance: the only source is the external report cited in the PR, and the recorded fixture has overage_count and no negative value
    • Whether a real remaining can be fractional (for example from premium-request multipliers), which would now hide the metric: no recorded response shows it
    • Runtime rendering, screenshots and the outcome of the rust-metrics check
  • Tests run: None. Reviewer execution is disabled.

HemSoft full-spectrum review • 3 evidence-based passes • Reviewed head d563530b7aafb0faae20186d4c00fa6b0071a134 • SFL v0

Decode the bounded quota counts before constructing its reset and pace
metric. This preserves checked overage arithmetic and malformed-input
omission while removing the added decoding branches from quota_metric.
The original rust-metrics run measured its CRAP increase from 3.002197 to
5.0; the gate, baseline and measured function identity stay unchanged.

Co-Authored-By: Codex GPT-6.1 Sol <noreply@openai.com>
@HemSoft

HemSoft commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

@sfl review

@hemsoft-sfl-reviewer hemsoft-sfl-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Verdict: PASS • Round 2

Confidence: 8/10

Since round 1

New: 1; still open (repeated): 0; resolved: 1.
Resolved means no matching finding in this review, not independent proof of a fix.

New (1)

  • LOW Over-allowance forecast mixes clamped share with unclamped pace at crates/codexbar-providers/src/copilot.rs:135

Still open (repeated) (0)

  • None.

Resolved (1)

  • LOW Over-allowance 'at reset' forecast mixes a clamped share with an unclamped pace at crates/codexbar-providers/src/copilot.rs:139

Review done by claude-opus-5-5 (same family: same-family: codex gated: codex Weekly 167% SUS 60 🔴 pause, defer)

The PR counts Copilot quota usage past the entitlement when GitHub reports remaining below zero. It also leaves out quotas whose remaining isn't a whole number, and adds a capture fixture and a UI test. Round 2 adds one commit, which moves the count decoding into quota_counts without changing behavior; I found no blocking defect, and the only open finding is a Low in the informational forecast.

🔨 Full-Spectrum Review Summary

Effort: high
Review time: 1m 30s
Token count: Unavailable

Pass 🔴 Critical 🟠 High 🟡 Medium ⚪ Low
🔒 Security 0 0 0 0
✅ Accuracy 0 0 0 1
🧰 Quality & Maintainability 0 0 0 0
Total 0 0 0 1

Key Findings

  1. ⚪ LOW — Accuracy: Over-allowance forecast mixes clamped share with unclamped pace. crates/codexbar-providers/src/copilot.rs:135: Trigger: a quota past its entitlement, for example 3000 entitled with -740 remaining. elapsed_pace gets the unclamped fraction 3740/3000. projected_at_reset in codexbar-core/src/metric.rs starts from forecast_usage, which uses the clamped used_fraction of 1.0, and then adds that pace for the time left. The fixture therefore reads 174% at reset, not the clamped-model 159% and not the roughly 198% that a linear projection of actual use gives. Impact: only the informational 'at reset' percentage. The used count, limit-reached state, severity and alerts are correct. This is the same Low as Round 1, and the PR body discloses it. (non-blocking)

Coverage

  • Files reviewed:
    • crates/codexbar-providers/src/copilot.rs
    • crates/codexbar-app/src/dashboard.rs
    • crates/codexbar-app/src/self_capture.rs
    • crates/codexbar-app/src/self_capture/copilot_overage.rs
    • crates/codexbar-app/src/ui_tests.rs
  • Context read:
    • AGENTS.md
    • docs/RUST_TEST_METRICS.md
    • crates/codexbar-core/src/metric.rs
    • PR #319 body
    • compare d563530b7aafb0faae20186d4c00fa6b0071a134...46223e9fc4483934fb1826e10af3ecfba2946cc2
  • Verified:
    • Compare base_commit.sha is b1327bdaf229e5db2a79ae394fccdf96b4a24ac7 and the last commit is 46223e9fc4483934fb1826e10af3ecfba2946cc2, status ahead by 2 and behind by 0
    • The Round 2 delta (d563530..46223e9) touches only copilot.rs. It moves the entitlement, unlimited and remaining decoding into quota_counts unchanged, and keeps the same order of checks and the same early None returns
    • quota_used uses only saturating_sub and checked_add on u64, with i64::unsigned_abs. No input wraps or panics, including i64::MIN and u64::MAX
    • Metric::Quota consumers in metric.rs never subtract used from limit. used_fraction clamps to 0..=1, and time-to-limit uses (1. - used).max(0.)
    • The quota test helper picks a sibling quota different from the key under test, and asserts that sibling stays (10, 50)
    • The capture helpers exist at this SHA: set_accounts, placeholders, sus::capture_time, save and settle. The new capture env var is inside the isolated-settings assertion
    • Under the documented non-regression rule, the new functions are far below the CRAP-30 cap for new functions, and quota_metric's complexity drops below base (author's local figures: 2 against base 3)
    • The fixture's 174% and 159% match the formula: 1 + (3740/3000)/19.5×11.5 and 1 + (1/19.5)×11.5
  • Not verified:
    • Compilation, test results and CRAP/coverage numbers: execution was forbidden, and the author's local figures were not reproduced
    • Required CI on this head: rust (all OSes), rust-metrics, rust-mutations and security-scan were still queued or in progress, and only build had passed
    • Current-head screenshots: the PR says they are still pending, and the attached images are from d563530
    • GitHub's real response shape past the allowance: the PR cites only a third-party report, and the tests use synthetic payloads
  • Tests run: None. Reviewer execution is disabled.

HemSoft full-spectrum review • 3 evidence-based passes • Reviewed head 46223e9fc4483934fb1826e10af3ecfba2946cc2 • SFL v0

@HemSoft
HemSoft merged commit 58a0945 into main Nov 7, 2026
7 checks passed
@HemSoft
HemSoft deleted the fix/issue-318-copilot-quota-overage branch November 7, 2026 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working model:claude-opus-5-5 PR authored by claude-opus-5-5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show Copilot quota usage past the allowance instead of capping it at the entitlement

1 participant