What a vector database does, and why we still run Postgres

Every tutorial about building an AI assistant starts the same way: choose a vector database. It is presented as the first decision, the foundation, the thing you cannot get wrong.

For most of the people reading those tutorials it is the least consequential choice they will make. We serve retrieval for every WelcomeAI workspace out of PostgreSQL with a vector extension, and the reasons are worth writing down, because they are reasons you can check against your own situation rather than a preference.

What the thing actually does

Strip the marketing and a vector database does four jobs.

It stores a list of numbers next to each chunk of text, the embedding. It indexes those lists so that finding the closest ones does not mean comparing against every row. It searches, returning the nearest neighbours to a query embedding. And it filters, so a search can be limited to one workspace, one knowledge base, one language.

Three of those four are things a relational database has always done. The interesting one is the index, because a naive nearest-neighbour search compares your question to every stored vector, and that becomes slow exactly when your corpus becomes interesting.

Specialised indexes solve this by being approximate on purpose: they look at a fraction of the candidates and accept a small chance of missing the true closest match in exchange for speed. That trade, recall against latency, is the actual product a vector database sells. Postgres extensions implement the same kinds of index; a dedicated system gives you more knobs on them and more tooling around them.

Do you need one?

Notice that two of the three conditions are about scale and the third is about staffing. None of them is about whether you are "doing AI properly".

What you give up by adding one

This is the part the tutorials skip, and it is the reason for our own choice.

A separate vector store means your text lives in one system and its embeddings in another, and nothing enforces that they agree. Delete a document in the database and the vectors stay until some job removes them. That job fails one night, and your assistant answers from a page that no longer exists, confidently, with a citation to a dead link.

In one database that class of bug does not exist. A chunk belongs to a document by a foreign key, deleting the document takes its chunks with it in the same transaction, and a backup is a backup of all of it. For a product where a stale answer is the failure that matters most, that consistency is worth more than a faster index we would not notice.

Two smaller things follow from it. Hybrid retrieval, which runs meaning-based and keyword search together, happens in one query in one place rather than across two systems. And filtering by workspace is a condition in the same query, which is what multi-tenant isolation should look like.

What people think they need it for

Three things come up in nearly every conversation, and none of them is a database problem.

Answers that cite the wrong page are usually a chunking problem or a content problem. Slow answers are usually the model writing, not the search finding. And an assistant that misses things it should know is almost always missing content, not missing an index. Changing databases fixes none of those, which is the expensive way to find out they were never the cause.

Questions people ask

If you are at the start of this, spend the afternoon you would have spent comparing vector databases on your content instead. The database is the easiest thing to change later, and the content is the thing that decides whether any of it answers correctly.