Documents and how they are indexed
What happens between adding a file and the assistant being able to answer from it.
A document is one piece of content: a file you uploaded, a page we crawled, an article a connector sent, a Q&A pair you typed.
What you can add
PDF, DOCX, PPTX, XLSX, CSV, Markdown, plain text and HTML. Plus anything a connector brings in, and anything you push through the REST API.
A PDF that is a scan of paper has no text in it. We do not read images, so a scanned document arrives empty. Run it through OCR first.
What happens after you add one
- Extraction. We pull the text out and keep the structure -- headings, lists, tables, and which page each passage came from, so a citation can say "page 7".
- Chunking. The text is cut into passages of a few hundred words with a little overlap, so a sentence that straddles a boundary is not lost.
- Embedding. Each chunk is turned into a list of numbers standing for its meaning. This is what makes "can I get my money back" find a paragraph about refunds.
- Indexed. Now it can be answered from.
The whole thing takes seconds for a page and a minute or two for a long PDF. Status tells you where it is, and a failure says why rather than sitting at "processing" forever.
Re-indexing
Changing the chunk size, the overlap or the embedding model changes how every existing document was processed, so those settings offer to re-index the knowledge base. It costs embedding tokens and a few minutes; nothing is unavailable while it runs.
Editing the text of one document re-indexes only that one.
Deleting
Deleting a document removes its chunks, its embeddings and the file we stored. It is immediate and it is not recoverable, which is why the confirmation names the document.
Deleting the source a document came from is a different question, and the product asks it: you can remove a connection and keep its documents, which is usually what you want -- they stop updating and keep answering.
Two things worth knowing
More is not better. A knowledge base with your ten clearest pages answers better than one with two hundred pages of everything, because retrieval has fewer near-misses to choose between. Old versions of a policy are worse than no policy.
Titles matter more than you would think. They are what a citation shows, and a document called "Untitled" or "export-final-2.pdf" makes an answer look untrustworthy even when it is right.