Bekko Beats 300M-Parameter Rivals at Multilingual Search With Just 7.7M Active Parameters
A 7.7M active-parameter multilingual embedding model that matches or beats models 3-40x its size on retrieval, running on a Raspberry Pi.
Takeaways
- Bekko-embedding-v1-a8m is a 7.7M active-parameter multilingual embedding model covering 100+ languages with 8k context.
- Beats multilingual-e5-large and BGE-M3 on MMTEB Retrieval despite being 40x smaller in active parameters.
- Runs 364 docs/s on a Ryzen 9 7950X with OpenVINO and 33 docs/s on a Raspberry Pi 5.
- Ships with ONNX, OpenVINO, safetensors and GGUF artifacts under MIT license.
- 384-dim Matryoshka embeddings truncate cleanly to 256, 128, or 64 with small quality loss.
- Larger 25M-parameter a25m sibling is available when quality matters more than raw speed.
Multilingual embedding models with broad language coverage are usually large: multilingual-e5-large and BGE-M3 each activate roughly 300M parameters. The Bekko model card describes a smaller option with 7.7M active parameters, 106M total parameters, and retrieval scores competitive with encoders that activate 3 to 40 times more weights.
The MIT-licensed model runs on hardware ranging from browsers to a Raspberry Pi 5. Its release includes ONNX, OpenVINO, and Safetensors artifacts.. A 24.9M-active-parameter sibling, bekko-a25m, offers a higher-quality tier with the same deployment focus.
Why 106M stored behaves like 8M active
The release defines active parameters as the attention and feed-forward weights applied to every token. Most of Bekko’s 106M total parameters reside in its multilingual token embedding table. The encoder retrieves selected rows from that table, then sends each token through a transformer stack containing about 7.7M active weights. This keeps the expensive per-token matrix multiplications close to those of an 8M-parameter encoder.
The full 106M parameters still determine download size and memory use. Bekko’s default OpenVINO and ONNX exports quantize the static embedding table row by row to int8, reducing the artifact from about 404 MiB in fp32 to 124 MiB. The release reports cosine similarity of 0.9994 between the quantized and PyTorch outputs.
- Backbone: four-layer mmBERT encoder pruned from mmBERT-small
- Hidden size: 384
- Output: 384-dimensional embeddings
- Context: up to 8,192 tokens
- Smaller vectors: 256, 128, or 64 dimensions through Matryoshka representation learning
Matryoshka training concentrates useful information in the leading dimensions. A stored 384-dimensional vector can therefore be truncated and renormalized without encoding the source text again. Queries and indexed documents must use the same dimension and normalization.