Skip to content

Commit d24e41e

Browse files
committed
refactor(python): align Python namespaces with C++
- Move rerank from FloatSpace.rerank to deglib.search.rerank - Move quantize_batch from deglib.distances to deglib.optimization - Update bindings, tests, examples, and API docs
1 parent f40b225 commit d24e41e

10 files changed

Lines changed: 108 additions & 281 deletions

File tree

‎cpp/ARCHITECTURE.md‎

Lines changed: 0 additions & 193 deletions
This file was deleted.

‎examples/knng/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Given $N$ high-dimensional vectors, the goal of k-NNG construction (self-join) i
1111

1212
The approach demonstrates **Mode 4 (`evp-rerank`)**, which achieves state-of-the-art trade-offs between construction speed and neighbor recall ($\ge 88\%$):
1313

14-
1. **EVP Quantization**: Feature vectors are converted to compact sparse EVP-bit representations using the C++ `deglib_cpp.quantize_batch` function (`--non-zeros 512`).
14+
1. **EVP Quantization**: Feature vectors are converted to compact sparse EVP-bit representations using `deglib.optimization.quantize_batch` (`--non-zeros 512`).
1515
2. **DEG Construction**: A dynamic exploration graph is constructed using DEG's `GraphBuilder` with the `EVP_InnerProduct` metric for fast quantized distance computation.
1616
3. **Graph Exploration**: Exploration for vertex $i$ walks the DEG graph neighborhood using fast EVP bit-level inner product distances to collect candidates (`evpK = 50`).
1717
4. **FP16 Candidate Reranking**: Exact inner-product distances are computed using `deglib_cpp.floats_to_fp16` and `deglib_cpp.fp16_to_floats` for candidate sets to produce final $k$-nearest neighbor edges.

‎examples/knng/main.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,9 @@
2424
import matplotlib.pyplot as plt
2525

2626
import deglib
27-
from deglib.distances import FloatSpace, Metric, quantize_batch
27+
from deglib.distances import FloatSpace, Metric
28+
from deglib.optimization import quantize_batch
29+
from deglib.search import rerank
2830
from dataset_utils import load_hdf5_dataset, ensure_small_dataset, DEFAULT_CACHE_DIR
2931

3032
DEFAULT_K_TOP = 15
@@ -125,7 +127,8 @@ def construct_knng(
125127
t0 = time.perf_counter()
126128
rerank_space = FloatSpace.create(dims, Metric.FP16_InnerProduct)
127129
print(f"Rerank Space: {rerank_space.metric().name} ({rerank_space.get_instruction().name})")
128-
final_knng_edges = rerank_space.rerank(
130+
final_knng_edges = rerank(
131+
space=rerank_space,
129132
queries=train_vectors,
130133
candidate_indices=indices,
131134
base_vectors=train_vectors,

‎python/API.md‎

Lines changed: 25 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,9 @@ import deglib
1616
|---|---|---|
1717
| [`deglib`](#1-root-package-deglib) | `DynamicExplorationGraph`, `build_from_data`, `load_*` | Main user facade for querying, exploration, and graph lifecycle. |
1818
| [`deglib.builder`](#2-module-deglibbuilder) | `GraphBuilder`, `OptimizationTarget`, `BuilderStatus`, `build_from_data` | Incremental vector addition/deletion, parallel batch construction, and optimization. |
19-
| [`deglib.distances`](#3-module-deglibdistances) | `FloatSpace`, `Metric`, `quantize_batch`, `floats_to_fp16`, `fp16_to_floats` | Vector metrics, SIMD feature spaces, batch distance evaluation, and quantization. |
20-
| [`deglib.search`](#4-module-deglibsearch) | `Filter` | Bitset-based label filtering for ANNS search and exploration. |
21-
| [`deglib.optimization`](#5-module-degliboptimization) | `prune_*`, `presort`, `mips_l2_*` | MRNG graph pruning, FLAS 1D dataset presorting, and MIPS L2 transformations. |
19+
| [`deglib.distances`](#3-module-deglibdistances) | `FloatSpace`, `Metric`, `floats_to_fp16`, `fp16_to_floats` | Vector metrics, SIMD feature spaces, and batch distance evaluation. |
20+
| [`deglib.search`](#4-module-deglibsearch) | `Filter`, `rerank` | Bitset-based label filtering and exact distance candidate reranking for ANNS search. |
21+
| [`deglib.optimization`](#5-module-degliboptimization) | `prune_*`, `presort`, `mips_l2_*`, `quantize_batch` | MRNG graph pruning, FLAS 1D dataset presorting, MIPS L2 transformations, and EVP quantization. |
2222
| [`deglib.analysis`](#6-module-deglibanalysis) | `analyze_graph`, `check_*`, `calc_*` | Graph validation, connectivity verification, and reachability metrics. |
2323
| [`deglib.cpu`](#7-module-deglibcpu) | `InstructionSet`, `has_avx2`, `has_avx512` | Runtime CPU SIMD capability detection and instruction set enum. |
2424

@@ -191,7 +191,7 @@ class GraphBuilder:
191191
Vector metrics, SIMD feature spaces, batch distance evaluation, and quantization.
192192

193193
```python
194-
from deglib.distances import FloatSpace, Metric, quantize_batch, floats_to_fp16, fp16_to_floats
194+
from deglib.distances import FloatSpace, Metric, floats_to_fp16, fp16_to_floats
195195

196196
# Supported distance metrics
197197
class Metric:
@@ -219,19 +219,8 @@ class FloatSpace:
219219
# Compute 1D array of distances between query [D] and a batch of targets [N, D]
220220
compute_distances(query, targets) -> np.ndarray
221221

222-
# Multi-threaded exact distance candidate reranking in C++.
223-
# queries: 2D array [Q, D]
224-
# candidate_indices: 2D uint32 array [Q, K_cand]
225-
# base_vectors: 2D array [N, D] (defaults to queries if None)
226-
# k_top: Number of nearest candidates to return per query (0 = all)
227-
# return_distances: If True, returns (indices, distances) tuple
228-
rerank(queries, candidate_indices, base_vectors=None, k_top=0, num_threads=0, return_distances=False, unsorted=False)
229222

230-
231-
# --- Quantization & Type Conversion ---
232-
233-
# Quantize float32 or float16 vectors to byte-packed EVP format using C++ multithreading
234-
quantize_batch(vectors, non_zeros, num_threads=0) -> np.ndarray
223+
# --- Type Conversion ---
235224

236225
# Convert float32 numpy array to uint16-packed FP16 representation
237226
floats_to_fp16(floats) -> np.ndarray
@@ -244,10 +233,10 @@ fp16_to_floats(fp16_vals) -> np.ndarray
244233

245234
## 4. Module: `deglib.search`
246235

247-
Bitset-based label filtering for search and exploration.
236+
Bitset-based label filtering and exact distance candidate reranking for search and exploration.
248237

249238
```python
250-
from deglib.search import Filter
239+
from deglib.search import Filter, rerank
251240

252241
class Filter:
253242
def __init__(valid_labels, max_value=-1, max_label_count=-1)
@@ -261,13 +250,26 @@ class Filter:
261250

262251
# Static factory supporting None, numpy array, or existing Filter
263252
create_filter(filter_labels, graph_size) -> cpp_filter
253+
254+
255+
# Multi-threaded exact distance candidate reranking in C++.
256+
# space: FloatSpace used to compute distances (metric + SIMD instruction set)
257+
# queries: 2D array [Q, D]
258+
# candidate_indices: 2D uint32 array [Q, K_cand]
259+
# base_vectors: 2D array [N, D] (defaults to queries if None)
260+
# k_top: Number of nearest candidates to return per query (0 = all)
261+
# num_threads: Number of threads (0 = all CPU cores)
262+
# return_distances: If True, returns (indices, distances) tuple
263+
# unsorted: If True, skips sorting the resulting candidates
264+
# Returns 2D uint32 array [Q, k_top] of candidate IDs, or (indices, distances) if return_distances is True.
265+
rerank(space, queries, candidate_indices, base_vectors=None, k_top=0, num_threads=0, return_distances=False, unsorted=False)
264266
```
265267

266268
---
267269

268270
## 5. Module: `deglib.optimization`
269271

270-
Graph topology refinement, FLAS 1D dataset presorting, and MIPS L2 transformations.
272+
Graph topology refinement, FLAS 1D dataset presorting, MIPS L2 transformations, and EVP quantization.
271273

272274
```python
273275
from deglib.optimization import (
@@ -276,8 +278,12 @@ from deglib.optimization import (
276278
presort,
277279
mips_l2_transform,
278280
mips_l2_transform_query,
281+
quantize_batch,
279282
)
280283

284+
# Quantize float32 or float16 vectors to byte-packed EVP format using C++ multithreading
285+
quantize_batch(vectors, non_zeros, num_threads=0) -> np.ndarray
286+
281287
# Remove all edges violating the Monotonic Relative Neighbor Graph (MRNG) rule. Returns removed edge count.
282288
prune_non_mrng_edges(graph, num_threads=0) -> int
283289

0 commit comments

Comments
 (0)