Skip to main content
Back

Chroma Rewrites Its Vector Database in Rust, Hitting 4x Faster AI Search

The open source vector database rewrote its core in Rust, delivering 4x faster reads and writes while keeping the same four-function Python API.

Repo, Full Access
  • Chroma 1.0 ships a full Rust-core rewrite while staying fully API compatible with prior versions.
  • Delivers up to 4x faster reads and writes by eliminating Python GIL bottlenecks on the hot path.
  • Same four-function API: create_collection, add, query, get, with automatic embedding and metadata filtering.
  • Runs embedded like SQLite for vectors, or in client-server mode for horizontal scaling.
  • Chroma Cloud now offers serverless vector, hybrid, and full-text search with $5 in free credits.
  • Best under ~5M vectors on a single node; no distributed sharding, IVF, PQ, or GPU indexing yet.

Vector databases are having a moment, and Chroma is one of the most downloaded ones in the retrieval augmented generation stack. It sits between your embedding model and your LLM, storing dense vectors alongside metadata so agents can pull only the relevant chunks into context instead of stuffing entire documents into a prompt. The project just crossed 29k GitHub stars, and the recent 1.0 release rewrote the internals from Python to Rust.

What the Rust rewrite buys you

The headline number is speed. Moving the core to Rust eliminates Python's Global Interpreter Lock bottleneck, enabling true multithreading and delivering up to 4x throughput on both writes and queries. The GIL had been the ceiling on parallel embedding work in the old implementation, and shifting the hot path to Rust unlocks concurrent indexing without the data corruption risks of hacking around it.

The Python API stays intact, following the same pattern NumPy uses to wrap C and Fortran. Nothing changes in your application code, and 1.0 is API-compatible with previous releases. Same Chroma, significantly faster.

The four-function API

Simplicity has always been the pitch. The entire core API is four functions, which is why Chroma shows up in so many tutorials and prototypes.