A vector database stores embeddings — numeric vectors that represent the meaning of a piece of text, image, or row — and lets you search by meaning instead of by keyword. A founder building an AI MVP runs into the term in week three of every proposal: the vendor wants to spin up Pinecone, or Weaviate, or Qdrant, or “just pgvector on Postgres”. The founder cannot tell which is right because the proposals quietly assume the answer is yes. This is the founder’s version of the answer — what a vector database is, the three ingredients you are paying for, when you need one, when you do not, and how to grade it.
This piece builds on the eval-first build playbook, which itself sits within the idea-to-product manifesto. By the end you can push back on the spin-up-a-vector-DB default and either confirm it or replace it with something cheaper that ships faster.
What a vector database is, in plain English
A regular database stores rows. A vector database stores rows where one column is a long list of numbers — usually between 384 and 4,096 — that represents the meaning of whatever the row is about. The list is an embedding. A search no longer asks “find rows where the title contains ‘refund’”; it asks “find rows whose meaning is closest to this question’s meaning”. Closeness is measured by an arithmetic distance — usually cosine similarity. Two pieces of text that mean roughly the same thing land near each other in this high-dimensional space, even if they share no words.
The vector database is the infrastructure that does this at scale. Brute-forcing a thousand vectors is trivial; storing fifty million and answering in 50 ms requires an approximate-nearest-neighbour (ANN) index. Two common families: HNSW (Malkov and Yashunin, arXiv:1603.09320) and DiskANN (Subramanya et al., arXiv:1907.07799). “Vector database” is shorthand for (storage + ANN index + similarity-search API).
The architecture that sits on top of it is usually retrieval-augmented generation — where a model looks something up before answering. A vector database is the storage engine for RAG; RAG is the application pattern. Confusing the two is the most common founder error in vendor calls.
The three ingredients you are paying for
When a proposal says “we will set up a vector database”, three things show up on the invoice.
1. The embedding model
Turns text (or images, audio) into the vector. A separate API call to a separate vendor — OpenAI, Cohere, Voyage, or an open-source model on your own hardware. Two facts:
- Dimension matters. Common dimensions in 2026 are 768 (many open-source models), 1024 (Cohere Embed v4), 1536 (OpenAI
text-embedding-3-small), and 3072 (text-embedding-3-large). Higher = finer distinctions, more storage cost. - Embedding choice locks in everything else. You cannot mix two models in one index — they live in incompatible geometries. Switching later means re-embedding the entire corpus. A vendor who picks an embedding model without explaining the lock-in has not thought about migration.
2. The index
The ANN data structure that makes search fast. The choice is RAM (HNSW — fast, expensive at scale) or disk with an in-memory shortlist (DiskANN — slower per query, cheaper above tens of millions of vectors). pgvector exposes both; managed services hide the choice behind a tier. The index is also where pricing differs: serverless = per query; provisioned = per pod.
3. The similarity search
The query API embeds the question, asks the index for the top-k nearest neighbours, returns the rows. Three knobs:
- Distance metric — cosine for most text embeddings; some families want dot-product. Fixed at index creation.
k— neighbours returned. Five to twenty is typical for RAG.- Filtering — restrict to rows whose metadata matches (e.g., this tenant only). Poor filtering forces the app layer to over-fetch and re-filter, destroying latency and recall.
A proposal that names embedding model, index family, and filtering strategy is doing it right. A proposal that says “we will use Pinecone” without those three is selling you a logo.
Why your MVP might need one — four signals
Signal 1: You are building RAG
If the product answers questions from a corpus the model did not train on — internal policies, customer transcripts, research papers, support articles — you are building RAG, and RAG needs a vector index. Below a few hundred thousand chunks, pgvector works; above that, a dedicated vector database earns its place.
Signal 2: You need semantic search, not keyword search
Semantic search matches the meaning of the query. A user typing “how do I get my money back” should find “Refund policy” even though no word overlaps. Keyword-only search (Postgres full-text, Elasticsearch BM25) misses unless you maintain a synonym dictionary by hand. Embedding-based search makes the match for free.
Signal 3: You are doing recommendation or clustering on unstructured items
If the product recommends similar documents, customers, products, or media items by content — “find articles like this one” — embeddings plus nearest-neighbour search are the modern default. Embeddings win when items are unstructured and behaviour data is sparse (cold-start MVP).
Signal 4: Your corpus is large or growing fast
If you have more than a few hundred thousand vector rows today, or ingest will cross that in twelve months, a dedicated index — pgvectorscale, DiskANN, or a managed vector database — pays back in latency and operational simplicity.
Why your MVP might NOT — four signals
Vector databases are the most over-prescribed piece of AI infrastructure in 2026. Each signal below is a reason to push back.
Anti-signal 1: Your corpus is a static FAQ
If the product answers forty fixed questions, the right architecture is a CASE statement, a routed prompt, or a small retrieval over a curated FAQ. Indexing forty embeddings to answer questions a LIKE query could route is engineering theatre.
Anti-signal 2: Your document count is below 500
For a corpus under a few hundred documents, frontier-model context windows now reach 200K tokens (Claude Opus 4.8) and beyond (Gemini 2.5 Pro). You can stuff the entire corpus into the prompt on every query. At MVP launch sizes, prompt-stuffing often beats RAG on cost and recall — because retrieval can miss the right passage and prompt-stuffing cannot miss what is already in the prompt.
Anti-signal 3: Your search is structured, not semantic
If users search by SKU, date range, customer ID, status, or any column with discrete values, a SQL WHERE clause is the right answer. Vector search is for meaning; SQL is for facts. The failure mode is forcing fact-shaped queries through vector search because the proposal labelled it AI.
Anti-signal 4: A managed search service already handles it
Algolia, Typesense, Meilisearch, and Postgres full-text search now ship vector modes that piggyback on the operational footprint you already have. Adding a separate vector database on top doubles the operational surface for marginal gains. “The search I already have, with vectors switched on” is almost always the right starting point for an MVP.
The four vendor options, named factually
Four options show up in MVP proposals most often. The framing is what each one is, not which is best.
Postgres + pgvector
A Postgres extension that adds a vector type and HNSW / IVF indexes. Runs anywhere Postgres runs. Cost = the Postgres instance. Right for MVP-scale corpora (up to several million vectors); wrong above that without extensions like pgvectorscale (DiskANN on Postgres). You almost certainly already have Postgres — one fewer moving part is one fewer thing that breaks the demo. Repo: github.com/pgvector/pgvector.
Pinecone
Managed vector database with a serverless tier that decouples storage from query cost. Common default in proposals — friendly SDK, near-zero operational footprint. Trade-offs: separate vendor on the invoice, vendor-specific filtering syntax, pricing that scales with stored vectors and queries. Check the Pinecone pricing page before signing.
Weaviate
Strong hybrid (BM25 + vector) search and metadata filtering, managed or self-hosted. The right pick when the product needs both keyword precision and semantic recall in one query.
Qdrant
Open-source vector database with a managed cloud. Strength: payload-based filtering integrated with the index, so filters do not destroy recall. Same managed-vs-self-hosted trade-off as Weaviate.
Other names — Milvus, Chroma, LanceDB, Vespa, MongoDB Atlas Vector Search, Elasticsearch’s vector mode — all have legitimate use cases. The choice is rarely the binding constraint on whether the MVP ships. The eval shape is.
The cost shape
Concrete prices change month to month. The shape does not. Three things drive what you pay.
- Storage — most managed vector databases charge per stored vector, billed by dimension and replicas. A 1,536-dim embedding takes ~4× the storage of a 384-dim one. Smaller embedding model = direct cost lever.
- Query volume — serverless tiers charge per query; provisioned tiers charge per pod with effectively unlimited queries. Bursty traffic favours serverless; sustained load favours provisioned.
- Embedding API calls — the vector DB itself is rarely the largest line item. Pay-per-token embedding calls (OpenAI, Cohere, Voyage) add up. A proposal that quotes vector-DB cost without naming embedding cost is incomplete.
At MVP scale, most managed vector databases sit in the free tier or low single-digit monthly cost. The decision is not today’s bill — it is the slope at six and twelve months. Push your vendor to quote three points: launch, month six, month twelve.
The eval shape — how to grade a vector database
The vector database is a graded component, not a checkbox. Four metrics show up in any evaluation worth running.
Recall@k
Given a labelled set of (query → correct document) pairs, how often does the correct document appear in the top-k? Recall@10 is the most common report. Above 0.9 is a working retriever; below 0.8 and the application layer is masking retrieval failures with prompt cleverness while the model hallucinates around what the index missed. Recall is load-bearing — and the metric vendors most often inflate by choosing their own labelled set.
Latency p95
P95 (95th-percentile slowest) is the honest number, not the average. For interactive products, p95 above 300 ms on retrieval degrades UX; for batch products, latency rarely matters until it crosses tens of seconds.
Ingest throughput
How many vectors per second can the system index under read load? This becomes the binding constraint when the corpus changes daily — support transcripts, new product launches, fresh research papers.
Cost per million queries
The number you budget against. Combine query-time embedding cost (the API call) with vector-DB query cost (storage + serverless query or provisioned-pod amortisation). Use vendor pricing pages.
A vendor proposal that does not name these four metrics, with methodology and target thresholds, will silently fail the production-ready test when you go to ship.
A worked example: a 500-document research assistant
A solo founder is building a research assistant over 500 long-form papers (median ~30 pages each). Launch traffic ~100 queries per day.
Corpus shape. At ~30 pages × ~400 words/page × ~1.3 tokens/word, each paper is ~15K tokens. Chunked at 500 tokens with overlap = ~30 chunks per paper, or ~15K chunks total. Each chunk gets a 1,536-dim embedding. Storage = 15K vectors × 1,536 dims — inside the free tier of every managed vector database, and trivially inside one Postgres instance.
The vendor-default path. “Set up Pinecone, ingest the 500 documents, serve from a serverless tier.” Reasonable. Also more infrastructure than the corpus needs.
The defensible default. Postgres + pgvector. Reasoning:
- Corpus is well under a million vectors — pgvector is comfortable.
- The product already needs Postgres for users, sessions, and metadata.
- pgvector supports HNSW and IVFFlat indexes with cosine, L2, and dot-product.
- Migration to a dedicated vector database later, if growth justifies it, is a one-week project.
The eval shape is identical either way: recall@10 ≥ 0.9; latency p95 ≤ 200 ms; cost per million queries computed at launch and month twelve. Both architectures hit the targets at this scale. The Postgres path ships with one fewer vendor on the invoice, one fewer SDK, and one fewer dashboard.
The lesson generalises. The vector-database question is not “which vendor?” — it is “is the corpus large enough that a dedicated index pays back the operational tax?” For most MVPs, the answer is not yet. Ship the smallest architecture that passes the eval.
Frequently Asked Questions
What is a vector database in simple terms?
A vector database stores embeddings — long lists of numbers that represent meaning — and lets you search by meaning instead of by keyword. It is the storage engine that makes semantic search, RAG, and content-based recommendation feasible above toy scale.
Do I need a vector database for my AI MVP?
You need a vector index, not necessarily a dedicated vector database. Below a few hundred thousand vectors, pgvector inside an existing Postgres usually works; above that, a dedicated service starts to earn its place.
What is the difference between a vector database and a regular database?
A regular database indexes scalar values and answers exact or range queries. A vector database also indexes high-dimensional vectors and answers similarity queries — “find rows whose vector is closest to this one”. Postgres + pgvector adds the second to the first rather than replacing it.
Is pgvector good enough for production?
For corpora up to a few million vectors with moderate query rates, yes — and pgvectorscale (DiskANN on Postgres) extends the range further. Above tens of millions of vectors with high query rates, dedicated vector databases start to win on latency, recall under load, and operational simplicity.
What is the difference between Pinecone, Weaviate, and Qdrant?
Pinecone is fully managed and serverless-first. Weaviate ships strong hybrid (keyword + vector) search, managed or self-hosted. Qdrant is open-source-first with deep payload-filtering. Compare on recall, latency, filtering, and price against your own workload — not vendor benchmarks.
How big does my corpus need to be before I need a vector database?
Under 500 documents, prompt-stuffing into a long-context model is often cheaper and more accurate. Between 500 and a few hundred thousand chunks, pgvector is comfortable. Above that, dedicated vector databases earn their place.
Will long context windows make vector databases obsolete?
No. Retrieval accuracy degrades inside long context before the window fills, and token cost per query is dramatically higher than retrieving the right few thousand tokens. The two complement each other: long context for cheap prototypes, vector databases for scale.
How do I evaluate a vector database vendor?
Four metrics: recall@k against your own labelled queries, latency p95 under realistic load, ingest throughput under read load, and cost per million queries at launch, month six, and month twelve. A vendor that resists naming these is selling logo.
What does a vector database cost at MVP scale?
At launch traffic, most managed vector databases sit in the free tier or low single-digit monthly cost, and pgvector adds essentially nothing to an existing Postgres bill. The variable cost that matters is the embedding API. Check vendor pricing pages for current rates.
Can I switch vector databases later?
Yes. The data (embeddings + metadata) is portable; query and filtering syntax are not. Migration is typically a one-to-two-week project — bulk-export, bulk-import, swap the client. Real but bounded.
Closing
A vector database stores embeddings and serves similarity search. It is not magic, and it is not a checkbox — it is a graded component with a measurable eval shape (recall, latency, ingest, cost). For most AI MVPs in 2026, the right starting point is Postgres + pgvector, because the corpus is small, the team is small, and one fewer vendor on the invoice is one less thing that breaks at launch. Migrate when your eval — against your own data — proves the binding constraint is the index, not the embedding model, the chunker, or the generator.
For the technique frame one layer up, read What is fine-tuning vs prompting vs RAG?. For the eval-first frame this all turns on, read The eval-first build playbook. For the broader programme, read the Idea-to-Product Manifesto.
Dirk Jan van Veen, PhD
