turbovec

turbovec is an open-source in-process vector index written in Rust with Python bindings. It implements Google Research's TurboQuant approach to compress dense vectors without a separate codebook-training phase, then searches the packed codes with architecture-specific SIMD kernels. Its practical appeal is a combination of online ingest, 2-bit or 4-bit storage, persistence, stable external IDs, delete support, allowlist-filtered search and adapters for common RAG frameworks.

It is important to classify it correctly. turbovec is an index library, not a hosted vector database or a complete retrieval service. It does not by itself provide distributed replication, multi-node sharding, backups, authentication, tenant administration, network APIs, hybrid lexical search, observability or a control plane. Teams gain local control and memory efficiency in exchange for owning those surrounding concerns.

How TurboQuant compression works

The underlying insight is that a random orthogonal rotation makes coordinates of high-dimensional unit vectors follow a predictable distribution. turbovec first separates each vector's norm from its direction. It applies one shared random rotation, then quantizes rotated coordinates using precomputed Lloyd–Max buckets. Two bits provide four values per coordinate; four bits provide sixteen. Codes are bit-packed, and a per-vector correction removes systematic inner-product shrinkage introduced by quantization.

The project adds TQ+ calibration: on the first add, it estimates a shift and scale for each coordinate using empirical quantiles, then freezes those values for later ingests. This is not the same as conventional product-quantization training, but the first batch influences calibration. A tiny or unrepresentative first add can therefore be a poor production initialization. Seed the index with a representative sample and test distribution drift.

Memory math without the marketing shortcut

A 1,536-dimensional float32 vector uses 6,144 bytes for raw coordinates. Its coordinate codes require 384 bytes at 2-bit or 768 bytes at 4-bit before metadata. That is a theoretical 16× or 8× reduction for the coordinate payload. The repository's headline says a 10-million-document float32 corpus that takes about 31 GB can fit in about 4 GB; that example reflects a particular dimension and representation and should not be generalized to every corpus.

Capacity planning must add external IDs, per-vector correction/norm values, calibration data, alignment, allocator overhead, deleted slots, application metadata, query buffers and the original document store. Embeddings are rarely the whole RAG memory bill. Measure resident set size after loading the real persisted index and serving concurrent queries.

What the API supports

import numpy as np
from turbovec import IdMapIndex

index = IdMapIndex(dim=1536, bit_width=4)
index.add_with_ids(vectors.astype(np.float32), ids.astype(np.uint64))
scores, result_ids = index.search(query.astype(np.float32), k=10)
index.remove(document_id)
index.write("corpus.tvim")
restored = IdMapIndex.load("corpus.tvim")

The Python API deliberately rejects non-float32 vectors instead of silently converting them. Stable IDs are important because internal slots can change after deletes and maintenance. Persistence makes local restart practical, but applications still need atomic publication, checksums, backup/version policy and compatibility tests across library upgrades.

Filtered retrieval is a meaningful differentiator

For multi-tenant, time-window or permission-aware retrieval, the application can first produce an allowed ID set from SQL, BM25, an ACL service or another system, then ask turbovec to rank only those candidates. Filtering occurs inside the SIMD path at 32-vector block granularity. Empty blocks can be skipped, and disallowed slots are rejected before heap insertion, so selective filters can avoid much of the dense-scoring work.

This is better than retrieving a global top-k and discarding unauthorized results, which can return too few valid documents and can leak ranking information. It is still the caller's responsibility to construct the allowlist correctly, bind it to the authenticated tenant and test empty, tiny, huge and rapidly changing sets. Never use metadata filtering as the only authorization check on the final document fetch.

How to read the published benchmarks

The comparison baseline is FAISS IndexPQ/IndexPQFastScan, not every FAISS index type. Flat exact search, HNSW, IVF-PQ, GPU indexes and managed databases solve different points on the recall, latency, memory and operations curve. Reproduce the repository benchmarks first, then substitute your data and acceptance target one factor at a time.

A useful RAG evaluation protocol

1. Freeze embeddings. Use the exact model, normalization, dimension and distance convention planned for production.
2. Create ground truth. Compute exact top-k neighbors with a trusted flat implementation and separately maintain task relevance judgments.
3. Test both bit widths. Compare 2-bit and 4-bit with float32 or exact search, not only against each other.
4. Stratify queries. Include common, rare, multilingual, short, long, duplicated and out-of-domain cases.
5. Exercise ingestion. Initialize with a representative batch, append later distributions, delete IDs, persist, reload and verify deterministic behavior.
6. Benchmark filters. Measure no filter and allowlists at 0%, 0.1%, 1%, 10%, 50% and 100% coverage, including adversarial block layouts.
7. Load test. Record p50/p95/p99 latency, throughput, CPU utilization, resident memory and tail behavior at real concurrency.
8. Evaluate the answer. Measure retrieval recall, reranker quality, citation correctness and final-answer success. Faster approximate neighbors are valuable only if the application outcome survives.

Decision metrics

Framework integrations

The repository documents adapters for LangChain, LlamaIndex, Haystack and Agno that replace their in-memory reference stores while keeping familiar interfaces. This can make a proof of concept quick, but “drop-in” refers to a public software surface—not identical scoring, filtering, deletion, persistence, threading or failure semantics. Run each framework's retrieval tests and pin compatible versions before deploying.

Alternatives

Limits and production risks

• Approximate compression can change nearest-neighbor order, especially at aggressive bit width or low dimension.
• CPU-specific kernels mean benchmark results should not be transferred between ARM, AVX2 and AVX-512 hardware.
• First-add calibration, changing embedding models and corpus drift need explicit migration tests.
• Local deployment keeps vectors under your control but does not automatically add encryption, access control or secure backups.
• An in-process crash affects the host application unless the service boundary and recovery strategy are designed deliberately.
• The project is evolving; pin releases, inspect changelogs/security guidance and validate persisted-index compatibility.

Frequently asked questions

Is turbovec a vector database?

No. It is a vector index library. Applications must supply document storage, networking, authorization, replication, monitoring and lifecycle operations as needed.

Does it require an offline training step?

It avoids conventional codebook training and supports online adds. TQ+ does calibrate per-coordinate values during the first add, so the initialization batch still deserves care.

Should I choose 2-bit or 4-bit?

Use 2-bit when memory is the binding constraint and your measured task recall remains acceptable. Four-bit usually buys more fidelity at roughly twice the coordinate-code storage. Benchmark both.

Does filtered search enforce tenant security?

It can efficiently restrict ranking to an allowlist. The application must build the correct list and recheck authorization before returning source content.

Are its FAISS speed claims universal?

No. Published results cover named hardware, datasets, dimensions, bit widths and FAISS PQ/FastScan configurations. x86 2-bit results include cases where FAISS is faster.

Can it be air-gapped?

The index runs locally and does not require a managed service. A complete air-gapped RAG stack also needs local embeddings, documents, package/model provenance and controlled updates.

Primary sources

• turbovec official repository and benchmark documentation
• turbovec API reference
• Reproducible benchmark scripts and results
• TurboQuant research paper
• RaBitQ paper cited for length correction
• FAISS FastScan technical reference
• turbovec Python package
• turbovec Rust crate

Last reviewed July 25, 2026. Performance statements are scoped to the repository's published setup unless stated otherwise. Reproduce on your corpus, embedding model, hardware and filter distribution before adoption.