[ljyss9]

Purpose

Reuse opened vector File Index readers across reads through the existing user-provided cache. Cache values now support typed objects as well as memory segments, and external index readers are keyed by index URI and column. Embedded indexes and reads without a cache retain their existing behavior.

Keep query filters and search options independent for each call, and give Lumina searches a per-call session memory resource. Migrate existing page, manifest, snapshot and Parquet cache users to the typed value API.

Tests

Built the existing Release configuration with tests and Lumina enabled:

cmake --build /tmp/paimon-bitmap-first-build --target paimon-common-test paimon-core-test paimon-parquet-format-test paimon-lumina-index-test -j 8

Executed the complete test binaries with the repository test fixtures:

• paimon-common-test: 1,797 passed, 2 skipped.
• paimon-core-test: 2,504 passed.
• paimon-parquet-format-test: 233 passed.
• paimon-lumina-index-test: 42 passed.

Coverage includes typed payload lifetime after eviction, memory accounting, cache key separation, reader reuse, disabled/embedded caching, failed loads, oversized objects, externally populated readers, independent concurrent Lumina queries, and eviction during a query.

git diff --check passed. The full pre-commit runner was not available locally.

API and Format

This changes the public cache API and requires downstream migration/recompilation:

• CacheValue becomes an abstract base class; use MemorySegmentCacheValue for byte segments and ObjectCacheValue<T> for objects.
• Replace GetSegment() with CacheValue::Get<const MemorySegment>(value); Get<T>() retains the cache value owner and returns null for a type mismatch.
• Add CacheKind::INDEX_READER and CacheKey::ForIndexReader(uri, column_name).

No storage format or protocol changes. Reader cache identity is the immutable index URI and column; construction options are not part of the key.

Documentation

Update the manifest cache documentation to use the typed segment accessor.

Generative AI tooling

Generated-by: OpenAI Codex (GPT-6)