[Feature](lance) Support additional vector index algorithms #66495

Parent

Tracked by #66340. IVF_PQ query support was introduced by #65730; #66779 subsequently supplied the missing physical IVF_PQ regression index, so the baseline fixture no longer silently exercised only flat search.

Scope

• IVF_FLAT query support.
• IVF_SQ query support.
• IVF_HNSW_FLAT query support.
• IVF_HNSW_SQ query support.
• IVF_HNSW_PQ query support.
• Define compatible vector element types, distance metrics, and ANN tuning parameters for each algorithm.

Completion criteria

• Query execution works through Lance automatic index selection for every listed algorithm.
• Regression fixtures and tests cover every algorithm and compatible metric combination.
• EXPLAIN and error handling make index selection and incompatibilities diagnosable.
• User documentation contains a supported algorithm compatibility matrix.

Completion evidence — 2026-09-30

Completed on branch-4.1, satisfying the parent tracker's branch criterion:

• [test](lance) Build the missing IVF_PQ regression fixture #66779 merged the real IVF_PQ fixture and a discriminator against silent flat fallback.
• [test](lance) Add query coverage for the remaining ANN index types and vector element types #66512 merged indexed Float32/L2 fixtures and query coverage for all six ANN algorithms, including nprobes, refinement, and HNSW ef checks.
• [test](lance) Add indexed coverage for cosine, dot and non-Float32 element types #67039 merged the compatible element-type/metric coverage: 56 buildable combinations from the pinned 96-cell matrix, with index-planning, indexed-versus-flat, and partition-probe assertions. Unsupported combinations are documented rather than treated as supported.
• [doc](lance) Document the vector element type and metric compatibility matrix doris-website#4082 merged the supported compatibility matrix and index/metric-selection limitations. [fix](lance) Select metric-compatible vector index #67553 subsequently fixed selection of a metric-compatible logical index when a column has multiple indexes.

The recorded External Regression statuses for #66512 and #67039 are successful. The implementation/test and documentation PRs are merged; all listed algorithms and the scoped compatibility criteria are covered, so this issue is closed as completed. This is query support, not index creation/lifecycle delivery (#66497), and does not claim universal algorithm/type/metric compatibility or general ANN recall guarantees. No new tests were run during this status audit.