Not all retrieval-augmented generation is built the same way. The difference between a confident-sounding guess and a verifiable, cited answer comes down to a handful of design choices. If you are evaluating AI document software, these are the five that matter most.
1. Structure-aware chunking
Before anything can be retrieved, documents are split into passages. Splitting blindly - every 500 words - cuts clauses in half and destroys meaning. Good systems chunk along the document's own structure, keeping a clause, a section, or a table intact so the retrieved passage is actually usable on its own.
2. Semantic retrieval, not keyword matching
A keyword search for "liability" misses a clause that says "the supplier shall not be responsible for." Semantic retrieval searches by meaning, so related wording surfaces even when the exact terms differ. The strongest systems then re-rank the candidates so the most relevant passage, not merely the most keyword-dense one, reaches the answer.
3. Strict grounding and citations
This is the pattern that matters most for regulated work. The answer must be built only from the retrieved passages, and every statement must point back to a named source - document, section, page. If a claim cannot be traced to a source, it should not appear. Grounding is what makes an answer checkable.
4. Confidence thresholds - and "I don't know"
When the retrieved material does not actually answer the question, the system should say so rather than improvise. A tool that refuses to guess is more useful in a law firm than one that always produces a fluent paragraph. InVerba treats "I don't know" as a correct answer when the evidence is not there.
5. Access control and audit
In a firm, not everyone should be able to query every matter. A production-grade system enforces who can search which collections, and records every query - who asked what, which documents were returned, and when - so the activity is reviewable.
- Chunking that respects document structure.
- Retrieval by meaning, with re-ranking.
- Answers grounded strictly in cited sources.
- Honest refusals instead of confident guesses.
- Role-based access and a complete query log.
How to use this list
When you evaluate any document-AI tool, ask the vendor how it handles each of these five. The answers tell you quickly whether you are looking at a careful system built for evidence, or a chatbot pointed at your files.