Labeling Labeling allows Pinecone to efficiently search and label the most relevant data for model fine-tuning. For example, our customer Gong, a revenue intelligence platform, uses Pinecone to develop an innovative “active learning system” called Smart Trackers that detect and track complex concepts in conversations.
This is an on-demand use case where queries are infrequent and aren’t sensitive to latency — several seconds is good enough since it is a human labeling activity. The critical challenge is to run cost-efficient searches over billions of vectors.
In the classic vector database search engine architecture, scatter-gather, data is sharded across many smaller individual indexes, and queries are sent to all shards. In this architecture, we don’t know what parts to page into memory until we touch all shards.
Such an architecture doesn’t make sense for on-demand queries over large datasets where only a portion of your corpus is relevant for any query. To improve efficiency, we need a new design that can effectively page portions of the index as needed from persistent, low-cost storage. That is, we need true decoupling of storage from compute for vector search.
Multitenant search
Notion, the productivity tool that allows users to create, update, and search for recommended content, represents another popular use case. The usage - data sizes and update frequencies of individual users - varies dramatically but still demands sub-100ms latencies.
Pinecone’s solution to this multi-tenancy within an index is namespaces. Customers like Notion leverage thousands of namespaces within a single index for cost-effectiveness at scale while ensuring isolation.
To improve cost efficiencies for this workflow, we need to effectively collocate and cache namespaces so that the frequently-used-namespaces can be queried with low latency, while the rest can be efficiently paged from persistent low-cost storage on demand.
Retrieval Augmented Generation (RAG)RAG combines LLMs alongside a vector database like Pinecone to augment the LLM with up-to-date and domain-specific knowledge to reduce hallucinations.
Our research on RAG shows that you can dramatically improve the performance of LLMs on many tasks, by leveraging a vector database that stores the embeddings generated from their billion-scale training corpus. |