OM AI Lab's VLX-Seek 1.5-10B Drops Python and Runs 2.18x Faster
A community port brings VLX-Seek's open-vocabulary detection VLM to pure C++ with GGUF quantization, hitting 2.2x decode speedup and running in 12GB VRAM.
Takeaways
- GGUF quantized build of VLX-Seek 1.5-10B enables open-vocabulary object detection in ~12GB VRAM.
- Pure C++17/GGML inference runtime via VLX-Seek-GGML, no Python required at inference time.
- CUDA q8_0 hits 93.3 tok/s decode versus 42.9 for the official Python backend, a 2.2x speedup.
- Greedy generation is token-identical to the reference on CPU, CUDA, and Vulkan at f16 and q8_0.
- Pipeline chains WeDetect-Uni proposals, Qwen3.5-VL, C-RADIOv4, SimpleFPN, HFRE, and a Qwen3.5 LLM head.
- Apache 2.0 license, stable C ABI for embedding, based on omlab/VLX-Seek-1.5-10B.
A new set of GGUF files brings VLX-Seek 1.5-10B to a native C++17 and GGML runtime. The project targets open-vocabulary object detection on a single consumer GPU, with the quantized model artifacts occupying about 11.3 GB and no Python required during inference.
The release converts OM AI Lab's source model for VLX-Seek-GGML, which reimplements the detection pipeline as native tensor operations. The model and runtime list the Apache-2.0 license.
What shipped
- Model: A 10-billion-parameter vision-language detector with open-vocabulary prompts.
- Runtime: C++17 with GGML and a C ABI for embedding.
- Backends: CPU, CUDA and Vulkan.
- Formats: Mostly f16 or q8_0, with the proposal detector fixed at f32.
- Hardware target: Approximately 12 GB of VRAM for q8_0 and 24 GB for the mostly f16 package.
From words to regions
VLX-Seek accepts an image and a semicolon-separated prompt such as "orange; apple", then returns boxes for the requested objects. The label set comes from the prompt. Conventional fixed-vocabulary detectors choose from classes defined during training, while captioning models generally produce prose without precise object coordinates.
Its inference graph combines several models and feature-processing stages:
- WeDetect-Uni proposes candidate regions that may contain objects.
- Qwen3.5-VL and C-RADIOv4 encode the image, while SimpleFPN builds feature maps at several scales.
- HFRE applies RoIAlign, which samples each proposed region into a fixed-size feature representation.
- Qwen3.5 combines the prompt with those region features, selects matching candidates and emits box coordinates.
This design gives the language model access to localized image features instead of asking it to infer coordinates from a global image representation alone.