[guptashivam04]

What problem does this PR solve?

Issue Number: None

Related PR: #69352

Problem Summary:

Backport of #69352 to branch-4.2.

A parallel scan (ParallelScannerBuilder) splits one tablet into several scanners. Each OlapScanner then calls CollectionStatistics::collect() with only the rs_splits of its own split, so the BM25 doc count, avgdl and term doc frequencies come from a subset of the tablet's rowsets. Rows with identical text get different scores depending on which scanner reads them. Because the split depends on the tablet's row count and the core count, a row's score and its rank in ORDER BY score() change when unrelated rows are loaded, and pagination over a score-ordered search returns repeated and missing rows.

Fix (same as #69352):

• TabletCollectionStatistics holds the rowsets of a tablet's entire read source. The first scanner of the tablet to prepare collects the statistics once with collect_full_collection(), and every scanner of that tablet copies them into its own CollectionStatistics. Later scanners wait for the first collection and get its result or error.
• CollectionStatistics::copy_collected_from() copies the collected counters and resets the derived idf/avgdl caches, so no cache is shared between scanner threads.
• ParallelScannerBuilder creates one holder per tablet, only when the scan has a score() runtime. Both the by-rowid and by-segment build paths pass it to their scanners through OlapScanner::Params.
• The non-parallel path is unchanged. Each of its scanners already receives the tablet's full read source.

Differences from #69352: branch-4.2 does not have the changelog, TSO and bucket fields of OlapScanner::Params or the scan_range argument of _build_scanner, so the scanner changes were applied by hand on top of the branch-4.2 code. OlapScanner::Params is initialized positionally on this branch, so the new field is appended at the end. branch-4.2 has no test that calls _build_scanner directly, so the follow-up test fix from #69352 is not needed here.

Statistics remain per tablet. Making them consistent across tablets and BEs is out of scope.

Release note

Fix BM25 score() returning different scores for identical rows of the same tablet when a parallel scan split the tablet into several scanners.

Check List (For Author)

• Test

  • Regression test: inverted_index_p0/test_bm25_score_parallel_scan_statistics (same suite as [fix](inverted index) Share BM25 statistics across parallel scanners of a tablet #69352). Passes on branch-4.2 with this change.

  • Unit Test: CollectionStatisticsDetailedTest.CopyCollectedFromScoresLikeSource, TabletCollectionStatisticsTest.CollectsOnceAndCopiesToEveryCaller (compiled locally; run left to CI)

  • Manual test: ran the reproduction from [fix](inverted index) Share BM25 statistics across parallel scanners of a tablet #69352 against branch-4.2 builds on macOS arm64.

    Build	enable_parallel_scan=true (split forced)	enable_parallel_scan=false
    branch-4.2 a5d2661d, without this change	15 distinct scores (1.34 to 5.81)	1 score (3.647686)
    branch-4.2 a5d2661d with this change	1 score (3.647686), same over 5 repeated runs	1 score (3.647686)

  • No need to test or manual test.

• Behavior changed:

  • Yes. Score queries that use a parallel scan now score every row of a tablet against statistics of the whole tablet. Scores can differ from earlier releases but are now independent of the scanner split.

• Does this need documentation?

  • No.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[yiguolei]

run buildall

[hello-stephen]

BE UT Coverage Report

Increment line coverage 36.07% (22/61) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	64.27% (29899/46523)
Line Coverage	49.12% (315379/642116)
Region Coverage	44.98% (254115/564971)
Branch Coverage	46.61% (119899/257247)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 96.72% (59/61) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.39% (34513/45182)
Line Coverage	61.23% (389846/636737)
Region Coverage	58.15% (329086/565884)
Branch Coverage	59.04% (151717/256955)

[github-actions]

PR approved by anyone and no changes requested.

[github-actions]

PR approved by at least one committer and no changes requested.