Unsloth Brings bge-small-en-v1.5 to GGUF, Beating OpenAI's Ada on CPU
Unsloth packages BAAI's bge-small-en-v1.5 in GGUF format, making a 33M-param embedding model runnable anywhere llama.cpp runs — no GPU required.
Takeaways
- Unsloth released bge-small-en-v1.5 in GGUF format, enabling CPU-only embedding inference via llama.cpp and Ollama.
- The model has 33.4M parameters, produces 384-dim embeddings, and fits in a 67.6 MB F16 GGUF file.
- It scores 62.17 average on MTEB (56 tasks), beating OpenAI's text-embedding-ada-002 at a fraction of the size.
- Runs under 8ms per embedding on a laptop CPU; quantized variants offer up to 30% additional speedup with minimal accuracy loss.
- Compatible with sentence-transformers, LangChain, Ollama, LM Studio, llama-cpp-python, and any RAG framework — no GPU required.
- Free for commercial use under MIT license; Unsloth also supports fine-tuning embedding models at 1.8–3.3x faster speeds.
Embedding models are the quiet backbone of every RAG pipeline, semantic search system, and document retrieval stack. But most of them still ship as PyTorch weights that demand a Python environment and, ideally, a GPU. Unsloth's GGUF release of bge-small-en-v1.5 changes that equation: it repackages one of the most battle-tested English embedding models into the same portable binary format used for local LLM inference, making it trivially deployable on any machine that can run llama.cpp.
The model behind the format
bge-small-en-v1.5 is a lightweight English sentence embedding model developed by the Beijing Academy of Artificial Intelligence (BAAI) as part of the BGE (BAAI General Embedding) series, designed for dense retrieval, semantic search, and similarity tasks. It produces 384-dimensional embeddings for comparing and ranking sentences or passages. With only 33.4M parameters, it provides a strong balance of accuracy and performance for English-only use cases.
The v1.5 update was released to enhance retrieval ability without requiring query instructions and to fix an earlier similarity distribution issue. The BGE v1.5 models are still among the most popular embedding models that balance embedding quality and model size. On the MTEB benchmark , the standard evaluation suite for embedding models, covering 56 tasks across retrieval, classification, clustering, and more , bge-small-en-v1.5 scores 62.17 average across 56 tasks, with 51.68 on retrieval and 81.59 on STS (semantic textual similarity). That puts it ahead of OpenAI's text-embedding-ada-002 (60.99) despite being a fraction of the size.