NeuML's Open-Source txtai Hits 13K Stars Merging Search, RAG, and Agents
The all-in-one Python framework unifies vector search, agents, RAG, and multi-model workflows behind a single embeddings database that scales from laptop to cluster.
Takeaways
- txtai is a Python framework unifying semantic search, RAG, agents, and model pipelines.
- Core is an embeddings database combining sparse and dense vectors, graphs, and SQL.
- Agents run on smolagents with support for Hugging Face, llama.cpp, OpenAI, Claude, Bedrock via LiteLLM.
- FastAPI service plus MCP support, with JavaScript, Java, Rust and Go client bindings.
- Apache 2.0, Python 3.10+, installable via
pip install txtai, local-first by design. - Over 70 example notebooks cover RAG, GraphRAG, speech-to-speech, knowledge graphs and workflows.
NeuML’s open-source txtai project has passed 13,000 GitHub stars. The framework combines semantic search, retrieval-augmented generation (RAG), model pipelines, workflows, and agents behind Python APIs and a configurable web service. For teams operating a vector store, retrieval layer, and agent framework separately, txtai offers a smaller set of interfaces and deployment boundaries.
The milestone reflects developer interest in that consolidated design. Production suitability still depends on retrieval quality, latency, model choices, and operational scale.
One database, four retrieval modes
txtai calls its foundation an embeddings database, an abstraction that combines sparse and dense vector indexes, graph networks, and relational data. Sparse indexes support lexical matching, dense vectors capture semantic similarity, graphs represent relationships, and relational fields enable structured filters and SQL queries. Object storage can retain the source content associated with each record.
Developers can compose the main capabilities through Python or YAML configuration:
| Capability | What txtai provides |
|---|---|
| Retrieval | Vector search, hybrid search, SQL filtering, graph analysis, topic modeling, and object storage |
| Multimodal indexing | Embeddings for text, documents, audio, images, and video |
| Model pipelines | Prompt execution, question answering, labeling, transcription, translation, and summarization |
| Workflows | Ordered pipelines with routing, aggregation, and application logic |
| Agents | Tool-using loops that can call embeddings, pipelines, workflows, and other agents |
| Serving | FastAPI and Model Context Protocol endpoints, plus JavaScript, Java, Rust, and Go bindings |
A small core with optional weight
txtai supports Python 3.10 and later, uses libraries including Hugging Face Transformers and Sentence Transformers, and ships under the Apache 2.0 license. Its core installation stays compact, while model pipelines, media processing, and other specialized features add optional packages and runtimes.
A minimal text-search example creates an index and runs a nearest-neighbor query in a few lines. The first run may download model weights, and result scores can vary by model and version.
from txtai import Embeddings
embeddings = Embeddings()
embeddings.index(["Correct", "Not what we hoped"])
results = embeddings.search("positive", 1)
print(results)
Agents reuse the retrieval layer
txtai’s agent implementation builds on the smolagents framework for tool calling and execution. It can use Hugging Face models, local GGUF models through llama.cpp, and hosted providers such as OpenAI, Anthropic, and AWS Bedrock through LiteLLM. Agent prompts can also follow the agents.md and skill.md conventions.
That shared interface lets an application change model backends without replacing its retrieval and workflow APIs. Provider-specific authentication, context limits, pricing, and tool-call behavior still require separate testing.
During a RAG request, txtai retrieves relevant records and inserts them into the model’s context. This grounding can reduce unsupported answers and attach responses to source material, although it cannot guarantee factual output. The examples cover GraphRAG over Wikipedia, web and SQL retrieval, and speech-to-speech pipelines. More than 70 example notebooks document the broader API.
Consolidation comes with tradeoffs
txtai overlaps several product categories that teams often deploy separately. A managed vector database concentrates on hosted storage and retrieval, while orchestration projects such as LangChain and LlamaIndex emphasize connectors and application composition. txtai packages retrieval, model execution, and serving into one framework.
| Option | Typical role | Operational consideration |
|---|---|---|
| txtai | Integrated retrieval and model workflows, including local deployments | The application team owns capacity planning, service operation, and framework-specific extensions |
| Managed vector database | Hosted vector storage, indexing, and query infrastructure | Model pipelines and orchestration usually remain separate services |
| LangChain or LlamaIndex | Broad orchestration and connector ecosystems | Retrieval storage and some execution services remain separate architectural choices |
Teams building on-premises, air-gapped, or self-hosted systems can run embeddings, indexes, workflows, and local models within their own environment. Data remains local only when every selected model and storage backend is local. Calls to OpenAI, Anthropic, or Bedrock follow those providers’ data paths, and air-gapped installations must mirror model weights and dependencies in advance.
Teams already standardized on a managed vector database or a large orchestration platform may gain less from migration. Custom retrieval designs outside txtai’s sparse, dense, graph, and relational abstractions can also require framework extensions.
A production evaluation should measure retrieval recall, ranking quality, indexing time, memory use, query latency, and concurrent throughput on representative data. Internet-facing deployments also need authentication, authorization, rate limits, observability, backups, and a scaling plan around the FastAPI service.
Test one complete workflow
The repository provides focused entry points for evaluating the framework:
- RAG quickstart for retrieval and grounded generation
- Agent quickstart for tool-driven model execution
- rag app for a complete RAG application
- paperai for scientific literature analysis
- ncoder for an open-source coding agent
A useful proof of concept should use one representative corpus and one end-to-end workflow, then compare service count, retrieval quality, latency, and operating effort with the current stack. That test reveals whether pip install txtai produces meaningful consolidation for the application.