Skip to content

branch-4.1: [chore](build) Update lance-c to the latest upstream main revision #69332 - #69336

Merged
yiguolei merged 1 commit into
apache:branch-4.1from
Gabriel39:dev/pick-69332-4.1
Oct 21, 2026
Merged

yiguolei merged 1 commit into
apache:branch-4.1from
Gabriel39:dev/pick-69332-4.1

Conversation

@Gabriel39

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Backport #69332 to branch-4.1 (source commit eeb54b230ddf7a7a5a06cbaab1bdbd6796d8ff22).

Pin lance-c to 4fafcd64525b274b113208390bbcca86bbbc093b and update the archive checksum. This includes the upstream fix for cold-session vector prewarm when an index name matches another schema column, plus the row-address allowlist/blocklist prefilter APIs.

Refresh the existing Foyer patch's base revision and one C++ test context. Its actual added/deleted code is unchanged. The target branch's only cherry-pick conflict was the previous revision's descriptive comment.

Validation

  • Passed bash thirdparty/test/lance-prefilter-patch-test.sh thirdparty/src on this branch: platform definitions, fresh extraction, idempotence, re-extraction, existing markers, stale-cache invalidation, and patch-failure handling.
  • Verified the pinned archive checksum through the downloader.
  • Passed shell syntax checks and git diff --cached --check.
  • Confirmed the patch payload is unchanged and the resulting patch matches [chore](build) Update lance-c to the latest upstream main revision #69332 exactly.
  • Full Doris build and runtime regressions were not run locally; requested through run buildall.

Release note

Fix Lance index prewarm failures when a vector index name matches a different schema column.

Check List (For Author)

  • Test: dependency patch harness and manual checks above.
  • Behavior changed: Yes, includes the upstream prewarm fix and new prefilter APIs.
  • Does this need documentation: No.

Backport apache#69332. Pin lance-c to 4fafcd64525b274b113208390bbcca86bbbc093b
and refresh the archive checksum and Foyer patch context.

Includes the upstream vector index/column-name collision prewarm fix and
row-address prefilter APIs. The Foyer patch payload is unchanged.

Validation: branch-local dependency patch harness, archive checksum,
shell syntax and diff checks passed. Full build/runtime tests remain for CI.

(cherry picked from commit eeb54b2)
@Gabriel39
Gabriel39 requested a review from yiguolei as a code owner October 20, 2026 08:48
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed both changed paths and the exact lance-c revision delta at Doris head b82c046. No actionable PR-introduced defects found. No existing blocking inline findings were present, and no additional user focus was supplied.

The archive includes the additive row-address prefilter APIs (#96) and cold-session prewarm collision fix (#97). The checksum matches the downloaded immutable archive, the complete retained Foyer patch applies with zero fuzz, and its added/deleted implementation payload is unchanged.

Critical checkpoint conclusions:

  1. Goal and evidence: the dependency contains the intended collision resolution and prefilter APIs. Inspected upstream collision, scheduling and C/C++ regression assertions; runtime behavior was not executed locally.
  2. Scope: the two-file pin/checksum and patch-context refresh is focused; upstream Cargo.toml, Cargo.lock and build.rs are unchanged.
  3. Concurrency: each collision segment opens and warms inside try_join_all, retaining concurrency and fail-fast cancellation. The dataset lock is released before I/O; no new BE lock order or worker is introduced.
  4. Lifecycle/static initialization: a single Arc dataset snapshot retains its session across prewarm; scanners own copied row lists and Arc masks. Existing call_once session initialization and teardown remain unchanged; no new cross-translation-unit static dependency.
  5. Configuration: no added or changed configuration; existing process-scoped session settings remain restart-scoped.
  6. Compatibility: new C exports/C++ wrappers are additive, scanner state remains opaque, existing declarations and SDK pin are unchanged, and no persisted-format change is introduced.
  7. Thrift/protobuf: no definitions, IDs, enums or required fields change.
  8. Parallel paths: traced plain/indexed/flat vector and FTS, prepared Match/Phrase, scalar segment, take and batch fallback paths. Optional masks remain absent for current Doris callers; explicit incompatible scalar-mask combinations and unsupported legacy plain scans fail visibly.
  9. Conditional behavior: same-field, noncolliding, missing-index and scalar/FTS paths retain SDK routing; the collision workaround resolves the intended field path and segment UUID. Comments explain the branches and mask restrictions.
  10. Test coverage: source inspection covers cold/shared sessions, snapshots, repeated calls, cancellation/concurrency gates, invalid/null/empty/replaced row lists, filter composition, scores/top-k and C/C++ wrappers. The existing patch harness covers extraction, idempotence, stale markers and patch failure.
  11. Expected results: no Doris golden outputs change; inspected upstream assertions check rows, scores, partition reloads and scheduling behavior.
  12. Observability/error handling: existing FFI panic/error guards, caller-thread error capture, BE status context, session metrics and reader timers remain intact. No new Doris operator needs profile plumbing.
  13. Transactions/persistence: no EditLog, transaction, failover, delete-bitmap or committed-data visibility mechanism changes; reads retain snapshot identity.
  14. Writes/crashes: the upstream delta affects read-side APIs; write atomicity and storage mutation paths are unchanged. The retained Foyer implementation has no new ownership or lock change.
  15. FE-BE variables: no transported state is added, so no sender or receiving path requires modification.
  16. Performance: full segment I/O remains concurrent; row masks are constructed only for explicit use, with SDK indexed/flat handling and batch fallback preserved. No runtime performance result is claimed.
  17. Additional review: download/cache invalidation, platform variables, installed headers/static archive, dependency isolation, Foyer overlap and existing reader/session consumers were traced without finding another defect. Current Doris code does not yet invoke the explicit prewarm or new row-list APIs.

All three assigned reviewers returned NO_NEW_VALUABLE_FINDINGS on the unchanged empty inline set in round 1. Both changed paths and every investigated risk have evidence-backed dispositions; no unresolved candidate or uncovered review scope remains. Review converged in 1 round.

Validation limit: static review and temporary archive/patch inspection only. No Doris or upstream build, test, benchmark or runtime integration was run; submodules were left uninitialized. These limits follow the review contract.

Review converged after 1 round(s).

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 75.06% (32032/42675)
Line Coverage 59.42% (359166/604475)
Region Coverage 56.07% (299185/533626)
Branch Coverage 57.02% (135936/238419)

@yiguolei

Copy link
Copy Markdown
Contributor

skip buildall

@yiguolei
yiguolei merged commit 1a117b8 into apache:branch-4.1 Oct 21, 2026
39 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Oct 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants