FireDB

FireDB is a small GPU-backed vector database for exact L2 nearest-neighbour search. It keeps vectors in a memory-mapped file, mirrors the active collection in GPU memory, and evaluates batches of queries with CUDA and cuBLAS.

The project is an experimental C++ and CUDA implementation. It is useful for learning how persistent vector storage, write-ahead logs, memory mapping, and brute-force GPU search fit together.

How search works

For database vectors X and query vectors Q, FireDB computes squared L2 distance as:

||x - q||² = ||x||² + ||q||² - 2xᵀq

The dot products are evaluated as one cuBLAS matrix multiplication. CUDA kernels compute vector norms and finish the distance calculation. The current implementation then copies scores to the CPU and uses a partial sort to select the nearest rows.

Features

• Exact L2 nearest-neighbour search
• Batched GPU search with CUDA and cuBLAS
• Persistent vector storage backed by mmap
• Incremental vector insertion
• NumPy .npy import for two-dimensional float32 arrays
• Write-ahead logging for the user ID to row mapping
• Interactive command-line interface

Requirements

• Linux
• An NVIDIA GPU with CUDA support
• CUDA Toolkit 11 or newer
• CMake 3.18 or newer
• A C++17 compiler supported by the installed CUDA Toolkit

Build

git clone https://github.com/ishnbl/fire-db.git
cd fire-db
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

Run the interactive program:

./build/FireDB

FireDB first asks for a database name. Press Enter to use main. A new database uses 128-dimensional vectors and creates two files in the working directory:

• <name>.slab stores the vectors.
• <name>.wal stores the user ID mapping.

Commands

Command	Purpose
help	List commands.
status	Show the vector count and dimension.
import <file.npy>	Import a two-dimensional float32 NumPy array.
gen <count>	Generate and insert random vectors.
add <id>	Insert a random vector with a user ID.
put <id> <values...>	Insert a vector with a user ID.
search	Search with a random query and return five rows.
find <id>	Find neighbours of an existing user ID.
batch <count>	Benchmark random queries and report queries per second.
exit	Close the program.

Example session:

$ ./build/FireDB
FireDB
demo
Creating 'demo' (Dim: 128)
demo> gen 10000
demo> status
Vectors: 10000
Dim:     128
demo> batch 100
demo> exit

Project structure

.
├── CMakeLists.txt
├── main.cpp
└── src/core
    ├── gpu.h    # GPU index and exact L2 search
    └── slab.h   # Memory-mapped vectors and ID log

Current limitations

• New databases use a fixed dimension of 128.
• The GPU index has a fixed capacity of one million vectors.
• Exact search requires the active vectors and score buffers to fit in GPU memory.
• Result selection happens on the CPU after scores are copied from the GPU.
• The NumPy reader supports the narrow format used by this prototype rather than every .npy variant.
• CUDA calls do not yet have complete error handling.

FireDB is a learning project and is not intended for production workloads.