What it is
A vector database is a place to store embeddings and find the ones closest to a question, fast. An embedding is a long list of numbers that stands for the meaning of a piece of text, an image, or a song. Similar meanings get similar lists. A vector database keeps millions of those lists and, when you hand it a new one, tells you which stored items sit nearest to it. "Vector" is simply the mathematician's word for a list of numbers like this, which is where the name comes from.
The easiest way to picture it is a library shelved by meaning instead of by the alphabet. In an ordinary library, a book about sourdough sits wherever its author's surname puts it, and a book about bread-making machines might be three aisles away. In a library shelved by meaning, every book about baking bread sits together, cookbooks on cakes are one shelf over, and books on growing wheat are just round the corner. You walk to the spot that matches what you're thinking about, look at what's nearby, and you've found your books, even if none of them has the word you had in mind in its title.
A normal database, the kind behind a shop's stock list or a spreadsheet, is built to answer exact questions. Which customers live in Leeds? Which orders are over £100? Which product has the code BX-204? It either matches or it doesn't. A vector database answers a fuzzier question: which things are most like this one? It returns a ranked list, closest first, with a score for how close each item is.
That job has a name, nearest-neighbour search. You have a point on the meaning map (your question, turned into an embedding), and you want the handful of stored points nearest to it. The "neighbours" might be passages from a company handbook, product descriptions, support tickets, photos, or songs.
Here's where it gets interesting. The simplest way to find the nearest neighbours is to measure the distance from your question to every single stored item, then sort. That's called exact search, or brute force. It is perfectly accurate, and for a few thousand items a computer does it in the blink of an eye. For a hundred million items, each with a thousand numbers, it becomes slow and expensive, because every search means a hundred billion small sums.
So vector databases cheat a little, on purpose. They use approximate nearest-neighbour search, which builds a kind of map or index in advance so each search only has to look in the most promising places. Picture finding a house by first driving to the right town, then the right street, then the right door, instead of knocking on every door in the country. The catch is that the shortcut occasionally misses one of the true nearest neighbours. You trade a little accuracy, usually a small fraction of results, for searches that are many times faster. Engineers call the share of true neighbours found "recall", and tuning a vector database is largely a matter of choosing how much recall to give up for speed.
Two more features turn a clever index into a useful database:
- Metadata filters. Alongside each embedding you store ordinary facts: the date, the author, the product category, which customer it belongs to, whether it's still current. Then you can ask "find the passages most like this question, but only from documents updated this year and only ones this customer is allowed to see." Without filters, meaning search happily returns an outdated policy or someone else's file.
- Keeping it up to date. Documents change. A real database lets you add, edit and delete items without rebuilding everything, which is harder than it sounds for a carefully built index.
It also helps to know the three kinds of search you'll hear about:
- Keyword search matches the words you typed. It's excellent for names, codes, and exact phrases, and useless when the page says "budget airfares" and you typed "cheap flights". The classic scoring method has the unmemorable name BM25.
- Meaning search, also called semantic or vector search, compares embeddings. It finds "budget airfares" from "cheap flights", but can be vague about exact names and numbers.
- Hybrid search runs both and blends the results. Many serious systems use hybrid search, for reasons the examples below make clear.
Finally, a vector database is usually one part of a bigger system rather than something you use directly. Its best-known job is inside RAG, retrieval-augmented generation, where a chatbot first fetches the most relevant passages from your documents and then writes its answer from them.
Why it matters
It's how chatbots read your documents. When a tool lets you "chat with your files", it almost always cuts the files into passages, turns each passage into an embedding, and stores them in a vector database. Your question is turned into an embedding too, the nearest passages come back, and a language model writes the reply from them. If the retrieval step brings back the wrong passages, the chatbot answers fluently from the wrong material. The quality of the vector search sets the ceiling for the quality of the answer, which is why this element sits at the heart of grounding.
It makes search forgiving. Customers, readers and staff rarely use the same words as the people who wrote the documents. Meaning search closes that gap. A help centre can find "Managing your subscription" when someone types "how do I stop my box".
It works for more than text. Anything that can be turned into an embedding can be searched this way: photos ("find pictures like this one"), music ("songs that feel like this"), products, and even faces, which is where privacy questions start. Recommendation systems, duplicate detection and fraud spotting all use nearest-neighbour search under the hood.
It gives AI a longer memory. A language model can only read so much at once; that limit is its context window. Storing past conversations or notes as embeddings, and pulling back only the relevant ones, is one of the main ways assistants seem to remember. Our Memory article covers that side.
Its limits become your AI's limits. Approximate search can miss things. Meaning search can confuse two versions of the same policy. Filters that are set up wrongly can leak one customer's data into another's answer. None of these are chatbot problems, strictly speaking, but they show up as chatbot mistakes, so anyone building or buying an AI tool over their own documents benefits from knowing where they come from.
Real-world examples
Facebook releases FAISS, and fast similarity search goes open source (2017)
On 29 March 2017, Facebook's AI research team, in a post by Hervé Jégou, Matthijs Douze and Jeff Johnson, released FAISS (Facebook AI Similarity Search) as free, open-source software. Its purpose was exactly the problem described above: given a very large collection of vectors, find the most similar ones quickly. The post pointed out that ordinary databases weren't built for this kind of question about AI-generated vectors.
The numbers in the announcement show why approximate search matters. The team reported building what it called the first k-nearest-neighbour graph (a map linking every item to its closest neighbours) on 1 billion high-dimensional vectors, and said its approach was 8.5 times faster than the previously reported state of the art for nearest-neighbour search at that scale. The example they gave was easy to grasp: upload a photo of a building and find similar-looking buildings, because the image embeddings of similar pictures sit near each other.
FAISS is a library, a toolbox programmers build into their own software, rather than a full database with users, filters and backups. But much of what came later, including several commercial vector databases, built on its ideas or its code. It wasn't the first such tool. Four years earlier, in 2013, Erik Bernhardsson at Spotify had written Annoy, a small approximate nearest-neighbour library used to power Spotify's music recommendations, and released it as open source. FAISS brought the same idea to the scale of billions.
A year before FAISS, in March 2016, two researchers, Yury Malkov and Dmitry Yashunin, had posted a paper describing a method called HNSW, short for Hierarchical Navigable Small World graphs. It organises items in layers like a set of maps at different zoom levels: a search starts on a sparse top layer to get to roughly the right area, then drops down layer by layer into more detail, like the town, street and door in the analogy above. HNSW became one of the most widely used approaches in vector databases, and FAISS and pgvector (below) both offer it.
pgvector: add vectors to the database you already have (2021)
On 20 April 2021, a developer named Andrew Kane published the first release of pgvector, a free add-on for PostgreSQL, one of the world's most popular open-source databases. The idea was modest and turned out to be very influential: instead of running a separate, specialised vector database next to your normal one, add a vector column to the tables you already have.
That matters for ordinary businesses. If your products, customers and orders already live in PostgreSQL, you can store each product description's embedding right beside its price and stock level. A single query can then say "the ten products most similar to this one, but only in stock and under £50", combining meaning search with ordinary filters, with no second system to keep in sync.
pgvector's own documentation spells out the trade-off this article keeps returning to. By default it does exact nearest-neighbour search, which it describes as giving perfect recall. You can then add an approximate index, which, in the README's words, trades "some recall for speed", and it warns that results may differ once you do. The project added an HNSW index in version 0.5.0, released on 28 August 2023, as demand for AI search took off. Today many hosted databases offer pgvector as a switch you can turn on, which is a large part of why "do I need a dedicated vector database at all?" became a real question.
Pinecone and the vector database boom (2021 to 2025)
Pinecone is the best-known company built around a dedicated vector database. It was founded in 2019 by Edo Liberty, who had previously led Amazon AI Labs and, before that, a Yahoo research lab in New York. On 27 January 2021 it came out of stealth (the startup term for going public about what you're building) with a $10 million seed investment led by Wing Venture Capital, pitching a managed vector database for machine learning applications such as semantic search and recommendations.
Then ChatGPT arrived at the end of 2022, and developers everywhere wanted to connect chatbots to their own documents. In late April 2023, Pinecone announced a $100 million Series B round, led by Andreessen Horowitz, at a $750 million valuation, describing its product as "long-term memory for AI". It named Shopify, Gong, HubSpot and Zapier among its customers. Around the same time, many other vector databases launched or raised money, and "vector database" became one of the hottest phrases in AI infrastructure.
The story since then is a useful reminder that tools change quickly. As pgvector and similar features spread into existing databases and search engines, the case for a separate system got weaker for many projects. In August 2025, the Israeli business site Calcalist, citing a report in The Information, said Pinecone was exploring a sale amid rising competition. Edo Liberty's own site now says he led the company as chief executive for six years before becoming chief scientist. The lesson for a reader choosing tools isn't about any single company. It's that the job, finding the nearest meanings quickly, is here to stay, while the question of which product does it keeps moving.
Anthropic measures what hybrid search adds (2024)
On 19 September 2024, Anthropic published a detailed write-up of a technique it called Contextual Retrieval, aimed at the most common failure in RAG systems: the right passage exists but the search doesn't bring it back. It's one of the clearest public measurements of why pure meaning search isn't enough.
Anthropic tested several set-ups on collections from different fields, including code, fiction and scientific papers, measuring how often a relevant passage was missing from the top 20 results. Its improved embeddings alone cut those failures by 35%, from 5.7% to 3.7%. Adding keyword search alongside the embeddings (the hybrid approach) cut failures by 49%, to 2.9%. Adding a final reranking step, where a separate model re-sorts the shortlist, brought the reduction to 67%, with failures down to 1.9%.
The write-up explains why keywords help in terms anyone can follow. Its example was a search for "Error code TS-999" in technical documentation. Meaning search tends to return passages about error codes in general, while keyword search looks for that exact string. Real questions are full of exact strings: product codes, surnames, order numbers, clause numbers.
The same article offers the advice that surprises people most. If your whole collection is smaller than about 200,000 tokens, which it puts at about 500 pages, you can often skip retrieval entirely and give the model the whole thing in its prompt. At the time of writing that is still true of several large models, but context sizes and prices change, so check current limits for the tool you use.
An everyday scenario: the course creator and the "private tutor"
Imagine Priya, who sells an online course on starting a small bakery. She has 40 video lessons with transcripts, a 90-page workbook, and three years of answers to student questions in a forum. She wants a "private tutor" chatbot that answers students from her own material.
A developer friend sets it up with a vector database. Each transcript, workbook section and forum answer is split into passages of a few paragraphs, turned into embeddings and stored, with metadata saying which module it came from and when it was written. It works well for questions like "how do I price my loaves?", which pulls back passages about costing even though students rarely use Priya's word, "margin".
Two problems show up in the first month. A student asks about "Lesson 12's worksheet" and gets a vague answer, because meaning search doesn't care much about the number 12; switching on hybrid search fixes it. Another gets advice about food-safety rules from a 2023 forum post that Priya later corrected; adding a date filter, and deleting the outdated answer, fixes that. Neither was the chatbot's fault. Both were retrieval.
Try it yourself
You can see how meaning search differs from keyword search with nothing but a pen and a chatbot. Allow about 30 minutes.
- Write 12 sentences. Use these, or write your own about a topic you know. Keep the numbers.
1. How do I get my money back?
2. Refunds are processed within 14 days.
3. My parcel never arrived.
4. Where is my order?
5. Can I change my delivery address?
6. The courier left the box at the wrong house.
7. I'd like to cancel my subscription.
8. How do I stop next month's box?
9. What is your returns policy?
10. Do you ship to Canada?
11. Order number 48213 is missing an item.
12. I was charged twice this month.
- Sort them by meaning, on paper. Draw a big square and place each sentence's number so that similar meanings sit close together. You'll probably end up with a money-back cluster (1, 2, 9, 12), a delivery cluster (3, 4, 5, 6, 10) and a subscription cluster (7, 8), with 11 somewhere between delivery and missing items. Congratulations: you've just built a tiny two-dimensional meaning map, which is what a vector database stores, only with hundreds of dimensions.
- Run a keyword search by hand. Take the question "I want a refund". Circle every sentence that contains the word "refund". Only sentence 2 ("Refunds") comes close, and a strict keyword match might miss even that. Now look at your map: the question lands in the money-back cluster, and its nearest neighbours are 1, 9, 2 and 12. That's the gap meaning search closes.
- Find where keywords win. Take the question "What's happening with 48213?" Keyword search finds sentence 11 instantly. On your meaning map, the number has no meaning at all, so a pure meaning search might just return general delivery questions. That's why hybrid search exists.
- Have a chatbot act as the similarity engine. Paste this into any chatbot, with your 12 sentences:
Act as a vector database doing nearest-neighbour search. Here are 12 stored sentences, numbered:
[paste the 12 sentences]
For each query below, list the 4 stored sentences closest in MEANING, closest first, with a similarity score from 0 to 1. Then, separately, list which sentences a strict KEYWORD search would return (sentences sharing at least one important word with the query, ignoring words like "I", "my", "the").
Queries:
A. I want a refund
B. Nothing showed up at my door
C. What's happening with 48213?
D. Please end my plan
Finally, say in plain language which queries meaning search handled better, which keyword search handled better, and how a hybrid search would combine them.
The chatbot is only imitating a vector database, and its scores are made up for illustration rather than calculated, but the rankings usually match what a real system does. Compare them with your paper map.
- Try a metadata filter. Add this follow-up:
Now pretend each sentence has a date. Sentences 1 to 6 are from 2024 and sentences 7 to 12 are from 2026. Repeat query A, but only return sentences from 2026. Explain what changed and why a real system would store dates alongside embeddings.
- Decide whether you'd need one. If you're thinking of a "chat with my documents" project, paste this:
I want a chatbot to answer questions from my own material. Here's what I have: [describe it: what kind, how many pages or items, how often it changes, who should see what].
In plain language for a non-technical person, tell me:
1. Roughly how many pages or tokens this is, and whether it would fit in a large model's prompt without any retrieval at all.
2. Whether the built-in file or project features of common chat tools would be enough.
3. If I do need retrieval, whether adding vector search to a database or tool I already use would do, or whether I need a dedicated vector database.
4. Whether I need keyword or hybrid search, given the kinds of questions people will ask.
Do not recommend specific paid products unless I ask, and remind me that limits and prices change.
You'll often find the answer is "you don't need one yet", which is worth knowing before anyone sells you one.
Common mistakes and misunderstandings
Thinking you need a vector database to use AI with your documents. Many projects don't. A few dozen pages can go straight into a chatbot's prompt. Many chat tools offer file uploads or project folders that handle retrieval for you behind the scenes. A small catalogue in a spreadsheet can be searched by meaning with an add-on, or simply by pasting it in. If you already run a database or search engine, adding vector search to it (as pgvector does for PostgreSQL) is often simpler than adding a new system. A dedicated vector database earns its place when you have very large collections, heavy traffic, or strict speed needs.
Treating "closest" as "correct". A vector database returns what is most similar, not what is true or current. Two versions of the same policy sit side by side on the meaning map. Keep outdated material out, or use date filters, or the chatbot will answer from last year's rules.
Relying on meaning search alone. Exact names, codes, numbers and rare words are where meaning search is weakest. Anthropic's 2024 measurements showed that adding keyword search cut retrieval failures substantially. If your users search for order numbers, model numbers or people's names, use hybrid search.
Forgetting that approximate means approximate. The fast indexes occasionally miss a true nearest neighbour. For a recommendation feed that's harmless. For legal or medical lookups, test how often the right passage is missed, and tune for higher recall even if searches get a little slower.
Skipping metadata and permissions. If every customer's documents live in one collection, a missing filter can show one customer another's data. Store who may see each item, and filter on it every time, before the results ever reach the chatbot.
Mixing embeddings from different models. Each embedding model makes its own map. If you switch models, or the provider updates one, every stored item has to be converted again. Vectors from two different models can't be compared, and the database won't warn you.
Blaming the language model for a retrieval failure. When a "chat with your files" tool gives a wrong answer, look at which passages it retrieved before rewriting the prompt. Often the right passage was never fetched. Many tools let you see the sources used; check them, and build a small set of test questions, as described in our Evals article.
Chunking documents badly. Passages that are too long blur several topics into one point on the map; passages that are too short lose their context ("it costs £12", but what does?). A few paragraphs each is a common starting point, and adding a short line saying what the passage is about, the idea behind Anthropic's Contextual Retrieval, helps a lot.
Believing a product's features will stay fixed. Vector search features, limits and prices change often, and companies in this area have merged, pivoted and changed leaders. At the time of writing the options are many and shifting, so check current terms and prefer set-ups that let you export your data.
Related elements
- Embeddings: the lists of numbers a vector database stores and compares.
- RAG: the technique that uses a vector database to fetch the right passages before a chatbot answers.
- Grounding: keeping answers tied to real sources, which depends on retrieving the right ones.
- Context Engineering: deciding what goes into the model's prompt, where retrieved passages are one of the main ingredients.
- Memory: how assistants recall past conversations, often by storing them as embeddings.
- Context Window: the reading limit that makes retrieval necessary for large collections, and unnecessary for small ones.
- Web Search: the other main way to bring outside information into an answer, using the open web instead of your own files.
- Hallucination: what happens when retrieval brings back the wrong passages and the model answers anyway.
- Evals: a small test set of questions is the best way to check whether retrieval is finding the right passages.
- Unsupervised Learning: clustering similar items without labels, a close cousin of nearest-neighbour search.