ModelRefs / Document Q&A — AI Glossary

Document Q&A — AI Glossary

An AI application answering questions about specific uploaded documents by combining OCR, chunking, retrieval, and LLM generation.

Overview

Document Q&A (also 'chat with your PDF') uses RAG: extract text from the document (PDFMiner, pdfplumber, OCR for scanned), chunk it, embed and store in a vector DB, then retrieve relevant chunks at query time. Applications: contract analysis, policy Q&A, financial report Q&A. Accuracy depends on OCR quality, chunking strategy, and retrieval precision.

Reference details

Topic	applications
Last reviewed	2026-06-24

Example: Retrieval sees under a tenth of the document

A 40-page contract at roughly 500 words a page is about 20,000 words, or near 27,000 tokens. Chunked at 512 tokens with 50 tokens of overlap the stride is 462, giving about 58 chunks. Retrieve the top 5 and you have put 2,560 tokens in front of the model — under 10% of the document. That works when the answer sits in one place. It fails on questions whose answer is assembled from a definition on page 3 and an exception on page 31, and it fails when a clause is split across a chunk boundary so neither half states the whole rule. Both failures look like the model being wrong, and neither is.

Commonly confused with

Document question answering is retrieval-augmented generation over a bounded, user-supplied set — which makes it different from long-context prompting, where the whole document goes in and no retrieval step can lose anything. If a document fits in the window, putting it in the window is usually more reliable than chunking it; retrieval earns its place when the corpus does not fit or the cost does not justify it.

When to use it

Reach for it when:

Corpora too large for the context window, or too costly to send in full
Where answers must cite a location in the source document
Repeated queries over a stable set, where embedding once amortises

Reach for something else when:

Single documents that fit comfortably in context — chunking only adds failure modes
Questions requiring the whole document at once: totals, counts, or “does this ever say…”
Scanned or low-quality sources without checking OCR first — retrieval cannot fix bad text

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Document Q&A — AI Glossary.

Frequently asked questions

What is Document Q&A?