Use an AI summary to locate important parts of a source, not to replace the source for consequential decisions. Compression always chooses what to omit.
Define the job
Specify audience, length and questions. For a contract or study, request section references and separate findings from recommendations.
Check failure points
- Negations and exceptions.
- Dates, amounts and units.
- Attribution.
- Correlation versus causation.
- Footnotes and tables.
- Uncertainty and disagreement.
NIST treats confabulation and integrity as risks. OpenAI notes some document workflows extract text while discarding embedded images depending on plan—important when charts carry evidence.
Two-pass method
- Request source locations.
- Open cited sections.
- Recalculate numbers.
- Ask what was inaccessible.
- Write from verified notes.
A reliable summary is navigation over evidence. Keep the original one click away.
Ask the model to expose its coverage
Before requesting conclusions, ask for the document's headings, page range, tables and unreadable elements. Compare that inventory with the source. If the tool missed an appendix or treated a scanned page as blank, the summary has a defined gap rather than an invisible one.
Use a claim-by-claim table
For each consequential statement, record the summary wording, source location and verification result. Mark supported, contradicted, incomplete or not found. This catches a polished sentence that combines pieces from different sections into a conclusion the author never made.
Example: summarizing a policy
Ask separately for eligibility, deadlines, exceptions, required evidence and appeal routes. Then read each cited section. A general overview may accurately describe the main rule while omitting the exception that controls a particular person's case.
Check quantities independently
Recalculate totals, percentages and date intervals. Confirm units and denominators. If a chart is the only source, inspect it directly rather than trusting extracted prose. Preserve uncertainty ranges and do not convert estimates into exact counts.
Write the final from verified notes
Once checks are complete, produce a concise synthesis that links back to the original. Do not ask the model to rewrite the unchecked summary and call that validation. The second pass must use evidence, not merely different wording.
Use disagreement as a test
Ask the model to identify passages that complicate its own summary and to present the strongest contrary evidence in the source. Then inspect those passages yourself. This does not guarantee neutrality, but it can expose an omitted limitation faster than repeatedly asking whether the first summary is accurate.
For recurring reports, keep the checklist and compare model versions on the same source. A newer system should earn trust through better verified coverage, not through a smoother tone or a shorter response time.
