Repository navigation
branch-4.1: [chore](build) Update lance-c to the latest upstream main revision #69332 - #69336
Conversation
Backport apache#69332. Pin lance-c to 4fafcd64525b274b113208390bbcca86bbbc093b and refresh the archive checksum and Foyer patch context. Includes the upstream vector index/column-name collision prewarm fix and row-address prefilter APIs. The Foyer patch payload is unchanged. Validation: branch-local dependency patch harness, archive checksum, shell syntax and diff checks passed. Full build/runtime tests remain for CI. (cherry picked from commit eeb54b2)
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
There was a problem hiding this comment.
Reviewed both changed paths and the exact lance-c revision delta at Doris head b82c046. No actionable PR-introduced defects found. No existing blocking inline findings were present, and no additional user focus was supplied.
The archive includes the additive row-address prefilter APIs (#96) and cold-session prewarm collision fix (#97). The checksum matches the downloaded immutable archive, the complete retained Foyer patch applies with zero fuzz, and its added/deleted implementation payload is unchanged.
Critical checkpoint conclusions:
- Goal and evidence: the dependency contains the intended collision resolution and prefilter APIs. Inspected upstream collision, scheduling and C/C++ regression assertions; runtime behavior was not executed locally.
- Scope: the two-file pin/checksum and patch-context refresh is focused; upstream Cargo.toml, Cargo.lock and build.rs are unchanged.
- Concurrency: each collision segment opens and warms inside try_join_all, retaining concurrency and fail-fast cancellation. The dataset lock is released before I/O; no new BE lock order or worker is introduced.
- Lifecycle/static initialization: a single Arc dataset snapshot retains its session across prewarm; scanners own copied row lists and Arc masks. Existing call_once session initialization and teardown remain unchanged; no new cross-translation-unit static dependency.
- Configuration: no added or changed configuration; existing process-scoped session settings remain restart-scoped.
- Compatibility: new C exports/C++ wrappers are additive, scanner state remains opaque, existing declarations and SDK pin are unchanged, and no persisted-format change is introduced.
- Thrift/protobuf: no definitions, IDs, enums or required fields change.
- Parallel paths: traced plain/indexed/flat vector and FTS, prepared Match/Phrase, scalar segment, take and batch fallback paths. Optional masks remain absent for current Doris callers; explicit incompatible scalar-mask combinations and unsupported legacy plain scans fail visibly.
- Conditional behavior: same-field, noncolliding, missing-index and scalar/FTS paths retain SDK routing; the collision workaround resolves the intended field path and segment UUID. Comments explain the branches and mask restrictions.
- Test coverage: source inspection covers cold/shared sessions, snapshots, repeated calls, cancellation/concurrency gates, invalid/null/empty/replaced row lists, filter composition, scores/top-k and C/C++ wrappers. The existing patch harness covers extraction, idempotence, stale markers and patch failure.
- Expected results: no Doris golden outputs change; inspected upstream assertions check rows, scores, partition reloads and scheduling behavior.
- Observability/error handling: existing FFI panic/error guards, caller-thread error capture, BE status context, session metrics and reader timers remain intact. No new Doris operator needs profile plumbing.
- Transactions/persistence: no EditLog, transaction, failover, delete-bitmap or committed-data visibility mechanism changes; reads retain snapshot identity.
- Writes/crashes: the upstream delta affects read-side APIs; write atomicity and storage mutation paths are unchanged. The retained Foyer implementation has no new ownership or lock change.
- FE-BE variables: no transported state is added, so no sender or receiving path requires modification.
- Performance: full segment I/O remains concurrent; row masks are constructed only for explicit use, with SDK indexed/flat handling and batch fallback preserved. No runtime performance result is claimed.
- Additional review: download/cache invalidation, platform variables, installed headers/static archive, dependency isolation, Foyer overlap and existing reader/session consumers were traced without finding another defect. Current Doris code does not yet invoke the explicit prewarm or new row-list APIs.
All three assigned reviewers returned NO_NEW_VALUABLE_FINDINGS on the unchanged empty inline set in round 1. Both changed paths and every investigated risk have evidence-backed dispositions; no unresolved candidate or uncovered review scope remains. Review converged in 1 round.
Validation limit: static review and temporary archive/patch inspection only. No Doris or upstream build, test, benchmark or runtime integration was run; submodules were left uninitialized. These limits follow the review contract.
Review converged after 1 round(s).
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
skip buildall |
|
PR approved by anyone and no changes requested. |
|
PR approved by at least one committer and no changes requested. |
What problem does this PR solve?
Backport #69332 to
branch-4.1(source commiteeb54b230ddf7a7a5a06cbaab1bdbd6796d8ff22).Pin lance-c to
4fafcd64525b274b113208390bbcca86bbbc093band update the archive checksum. This includes the upstream fix for cold-session vector prewarm when an index name matches another schema column, plus the row-address allowlist/blocklist prefilter APIs.Refresh the existing Foyer patch's base revision and one C++ test context. Its actual added/deleted code is unchanged. The target branch's only cherry-pick conflict was the previous revision's descriptive comment.
Validation
bash thirdparty/test/lance-prefilter-patch-test.sh thirdparty/srcon this branch: platform definitions, fresh extraction, idempotence, re-extraction, existing markers, stale-cache invalidation, and patch-failure handling.git diff --cached --check.run buildall.Release note
Fix Lance index prewarm failures when a vector index name matches a different schema column.
Check List (For Author)