Skip to content

Python: Fix Purview cache size accounting when replacing entries - #8990

Open
dhairya (dhairyajangir) wants to merge 2 commits into
microsoft:mainfrom
dhairyajangir:fix/purview-cache-replacement-accounting
Open

dhairya (dhairyajangir) wants to merge 2 commits into
microsoft:mainfrom
dhairyajangir:fix/purview-cache-replacement-accounting

Conversation

@dhairyajangir

Copy link
Copy Markdown

Motivation & Context

Growing a Purview cache entry at capacity could subtract its old size twice: once in set() and again when eviction encountered the still-live old mapping. A cache limited to 20 bytes could retain 24 estimated bytes while reporting 16, then report a negative size when emptied.

Description & Review Guide

  • What are the major changes? Remove the previous cache mapping before subtracting its size and evicting entries. Add a regression test that grows the earliest-expiring entry while the cache is near capacity.
  • What is the impact of these changes? Eviction skips the replaced entry's stale heap record and evicts another entry when needed. The replacement remains available, and size accounting returns to zero when it is removed.
  • What do you want reviewers to focus on? The ordering of replacement removal and eviction, and the regression's assertions on retained values and tracked bytes.

Implemented and tested with Codex assistance on Linux/Python 3.12.14. The regression fails before the fix and passes afterward. All 200 Purview unit tests pass with 89% package coverage. Package Ruff, changed-file formatting, strict source Pyright, all five configured test type checks, source/wheel builds, and git diff --check pass.

Optional warning-level Pyrefly output contains three existing diagnostics in unchanged middleware tests, so the no-warnings checklist item is left unchecked. The complete repository and live-service integration suites were not run.

Related Issue

Fixes #8988

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Added a test to verify cache behavior when updating entries and evicting others based on size constraints.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@dhairyajangir

Copy link
Copy Markdown
Author

Could a maintainer authorize the waiting fork workflows for this PR and #8992 when convenient? Both fixes are ready for code-owner review; their descriptions include the regression tests, local validation, and any baseline limitations. The CLA checks have passed. I'll address any review feedback.

Codex assisted with the fixes, validation, and this follow-up.

@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Oct 2, 2026
@eavanvalkenburg

Copy link
Copy Markdown
Member

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: No findings
Scope: full PR (2 commit(s)): 11225ab2b015, db8cc2e52c6f
Model: gpt-5.6-sol

Overview

The change removes an existing mapping before capacity eviction, preventing the stale heap record from subtracting the replaced value's size a second time. The operation remains atomic with respect to asyncio scheduling because set() has no suspension point, and the regression test verifies replacement retention, eviction of the other entry, exact byte accounting, and return to zero after removal. No PR-introduced Critical, High, or Medium issue was established.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

@dhairyajangir

Copy link
Copy Markdown
Author

Eduard van Valkenburg (@eavanvalkenburg) thanks for triggering the review. On head db8cc2e, the Python test matrix and code-quality jobs now pass, and the automated review reported no findings.

The remaining failed merge gate reports Failing checks: team_check, command_check (log). Those are cancelled prerequisite jobs from the original DevFlow PR Review run, rather than failing Purview tests (cancelled team check). The gatekeeper treats cancelled checks as failures.

Could you rerun the cancelled DevFlow prerequisites for this PR head and then the merge gate when convenient? The source change is unchanged; code-owner review is still pending.

@eavanvalkenburg

Copy link
Copy Markdown
Member

/review

This branch was previously deployed

1 inactive deployment
github-app-auth — db8cc2e5 Deployed Oct 2, 2026 by dhairyajangir via add_label #24322
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: Purview cache undercounts bytes when growing an entry at capacity

3 participants