Skip to content

Introduce totalHitsThresholdPerGroup parameter to TopGroupsCollector and TopGroupsCollectorManager - #16548

Open
gaobinlong wants to merge 4 commits into
apache:mainfrom
gaobinlong:disableExactCount
Open

Introduce totalHitsThresholdPerGroup parameter to TopGroupsCollector and TopGroupsCollectorManager#16548
gaobinlong wants to merge 4 commits into
apache:mainfrom
gaobinlong:disableExactCount

Conversation

@gaobinlong

@gaobinlong gaobinlong commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Description

Resolve a TODO in TopGroupsCollector.

Add totalHitsThresholdPerGroup parameter to TopGroupsCollector and TopGroupsCollectorManager. When set to a value less than Integer.MAX_VALUE, per-group hit counts may be reported as TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO once totalHitsThresholdPerGroup hits have been collected, saving the cost of exact counting. Defaults to true (exact counts).

Benchark on wikimedium1m with set totalHitsThresholdPerGroup to maxDocsPerGroup(10):

                        TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                TermGroup100      109.06     (29.0%)      110.12     (31.7%)    1.0% ( -46% -   86%) 0.903
                 TermGroup1M       83.43     (21.7%)       85.24     (21.9%)    2.2% ( -34% -   58%) 0.705
                TermGroup10K      119.36     (11.6%)      123.38     (12.0%)    3.4% ( -18% -   30%) 0.277

…TopGroupsCollectorManager

Signed-off-by: Binlong Gao <gbinlong@amazon.com>
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
@gaobinlong gaobinlong changed the title Introduce exactTotalHitsPerGroup parameter to TopGroupsCollector and TopGroupsCollectorManager Introduce totalHitsThresholdPerGroup parameter to TopGroupsCollector and TopGroupsCollectorManager Aug 28, 2026
@gaobinlong

Copy link
Copy Markdown
Contributor Author

Hi @dweiss @javanna , could you help to review this PR? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant