Conversation
chihsuan
changed the title
chihsuan
marked this pull request as ready for review
October 2, 2026 15:47
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The corrupt deleted-directory value behavior needs direct regression coverage.
Review effort: Balanced
Findings: 1
What changed in this PR
Speeds up OM quota repair scans after large deletions.
Changes:
- Parallelizes active
deletedTablevalue decoding. - Uses key-only deleted-directory scans.
- Adds quota-repair regression tests.
| File | Description |
|---|---|
QuotaRepairTask.java |
Optimizes deleted-table scans. |
TestQuotaRepairTask.java |
Tests batching, accounting, and decode failures. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
Author
|
cc @chungen0126 🙏 |
rich7420
reviewed
Oct 3, 2026
| // a typed iterator decodes on this thread, so keep values raw and let the workers decode | ||
| try (Table.KeyValueIterator<String, byte[]> keyIter = metadataManager.getStore() | ||
| .getTable(OMDBDefinition.DELETED_TABLE, StringCodec.get(), ByteArrayCodec.get(), CacheType.NO_CACHE) | ||
| .iterator()) { |
Contributor
There was a problem hiding this comment.
nit: please use .iterator(null, VALUE_ONLY) here, with the corresponding static import. The consumer only reads kv.getValue(), so we can avoid copying and decoding unused keys.
Contributor
Author
There was a problem hiding this comment.
Thanks, good point! Switched to VALUE_ONLY in d95fcef.
Contributor
|
overall LGTM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed?
Speed up quota repair after large deletes by:
deletedTablevalues in parallel using the existing batch workers. SnapshotdeletedTablescans remain unchanged.deletedDirectoryTable, avoiding unused value decoding. Corrupt values in this table no longer cause repair to fail.JIRA
https://issues.apache.org/jira/browse/HDDS-15991
How was this patch tested?
Added tests in
TestQuotaRepairTask.Local benchmark with 1M synthetic rows on a MacBook with a warm cache:
deletedTable1.9s → 1.0s.deletedDirectoryTable0.63s → 0.21s.Generated-by: Claude Code (Claude Opus 5.5)