Why we serve citations instead of confidence scores

A percentage next to an answer looks like rigour. "94% confident" suggests someone measured something and the answer passed.

We decided early not to show one. Every answer names the passage it was built from instead, and this post is the reasoning, because the choice is less obvious than it sounds and a lot of products make the other one.

What a confidence number is actually measuring

When an assistant shows a confidence percentage, it is usually reporting one of two things: the model's own estimate of how likely its wording is, or a similarity score from retrieval converted into something that looks like a probability.

Neither is a measure of whether the answer is true. The first describes how fluent the sentence is; a model can be fluent and wrong at the same time. The second describes how close a passage is in meaning to the question, which is useful, but it is not the same as the passage containing the answer.

Similarity is not correctness

Two passages about the same subject can score almost identically, and only one of them answers the question. A score tells you the passage is about refunds. It cannot tell you the passage says fourteen days.

Here is the same question answered both ways. Open each one and look at what the answer was built on.

The badge on the left is the dangerous one, and not because the number is invented. It could be computed honestly. The problem is that it invites the reader to stop, when the only thing that would have caught the error is looking at the source.

What each one lets a reader do

The last row is the one that decided it for us. An assistant that must cite has no way to answer from nothing: if no passage is strong enough, there is nothing to put under the answer, and the honest thing left is to say the material does not cover it.

Where similarity scores do belong

We did not throw the numbers away. Every retrieved passage still carries its similarity score, and that score is the fastest diagnostic a team has when an answer goes wrong.

The difference is where it sits. A score next to a named passage is evidence you can reason about: this passage, this close, for this question. A single percentage stamped on the answer hides the passages and asks you to accept the summary.

Questions people ask

If you are evaluating an assistant that shows a confidence score, ask it one question whose answer you know is not in its material, and watch what the number does.