perf(index): use dot distance for cosine in FlatFloatStorage

[clearlove10-c]

Problem

For IVF indexes with cosine distance, vectors are already normalized
by the IVF transform pipeline (NormalizeTransformer), so cosine
distance is mathematically equivalent to dot distance over normalized
vectors. However, cosine still computes the L2 norm of each vector
during distance calculation — redundant work that dot avoids.

Solution

Convert Cosine to Dot in FlatFloatStorage::try_from_batch, consistent
with how PQ and RQ already convert Cosine to L2 for the same reason.

Benchmark

scripts below
my_ann_bench.py

Metric	Origin	Optimized
avg latency	4.1298	2.3223
min latency	3.9978	2.1293
max latency	4.2227	2.4728
QPS	242.1435	430.5993

Tests

• test_try_from_batch_converts_cosine_to_dot — verifies Cosine → Dot
• test_try_from_batch_keeps_non_cosine_distance_types — verifies L2/Dot unchanged

Summary by CodeRabbit

• Bug Fixes
  • Improved vector distance handling by treating cosine distance as dot-product distance for normalized vectors.
  • Preserved existing behavior for L2, dot-product, and Hamming distance types.
  • Updated validation to confirm equivalent cosine and dot-product results on normalized vectors.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ebb4148d-f7ec-41a0-a0aa-ee31ddb2828f

📥 Commits

Reviewing files that changed from the base of the PR and between bc2d137 and 7ee1282.

📒 Files selected for processing (2)

• rust/lance-index/src/vector/flat/storage.rs
• rust/lance/src/index/vector/ivf/partition_serde.rs

---

📝 Walkthrough

Walkthrough

Flat vector storage now normalizes cosine distance to dot distance for IVF-normalized vectors. Tests cover conversion, preservation of other distance types, cosine/dot equivalence, and serialized codec restoration.

Changes

Distance normalization

Layer / File(s)	Summary
Normalize flat storage distance types rust/lance-index/src/vector/flat/storage.rs	FlatFloatStorage::try_from_batch converts cosine distance to dot distance while preserving other distance types.
Validate distance conversion and persistence rust/lance-index/src/vector/flat/storage.rs, rust/lance/src/index/vector/ivf/partition_serde.rs	Tests verify the conversion, non-cosine preservation, normalized-vector equivalence, and updated IVF deserialization expectations.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

• lance-format/lance issue 7747 — Tracks the same cosine-to-dot optimization for normalized IVF vectors.

Suggested reviewers: jackye1995

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and accurately summarizes the main change: converting cosine distance to dot distance in FlatFloatStorage.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}