A useful Harvey Vault pilot for a personal injury firm should show whether a reviewer can trace an answer to the right document and identify what the uploaded file does not contain. Evaluate the evidence, gaps and correction process alongside the summary. A fluent answer alone is not an acceptance criterion.
Evaluate the document task you need
Harvey's Vault product page describes organizing and analyzing large collections of legal documents. That makes document review a reasonable workflow to investigate. This article proposes a PI evaluation exercise based on the public description checked October 22, 2026; it does not report hands-on testing, medical accuracy measurements or a recommendation that Harvey performs provider calls.
A public discussion about legal AI marketing asked for a practical way to question large discovery collections without confusing that need with research or drafting. A separate paralegal chronology discussion described the difficulty of finding originals when a chronology omitted page numbers. Those are useful problem statements, not verified complaints about Vault's current performance.
Make the input set inspectable
Begin with public or fictional material approved for the exercise. Create a document manifest with filename, source, received date, page count and any missing or unreadable pages. Retain the original files before combining or numbering them. If you add Bates numbers, preserve a mapping to the source filenames. Agree which material is inside the exercise; otherwise a correct statement about one packet can be mistaken for a conclusion about the entire matter.
Use a fictional packet with known gaps
Fictional packet: an emergency-department note mentions an imaging report, but the report is absent. A later office note records a follow-up visit. Two copies of one bill have different cover pages, and an unsigned note uses an ambiguous date. Ask the tool to locate each event, identify the missing report and explain the date uncertainty. The desired answer distinguishes an absent source from evidence that an event never happened.
Score the returned evidence
Write the expected outcome before seeing the output. For each question, record whether the cited page supports the actual statement, whether a material qualification survived, and whether an unresolved fact stayed unresolved. The following worksheet is an original proposed evaluation rubric. Its rows are not claims about features that Harvey lacks or failures observed in its product.
Swipe across the table to see every column. Keyboard users can focus the table and use the arrow keys.
| Test question | Acceptable evidence | Hold for review if |
|---|---|---|
| Where is the imaging finding? | Exact supporting report page, or a clear statement that it is absent | A referring note is presented as the missing report |
| How many distinct visits appear? | Each event linked to its source with duplicates identified | Duplicate copies become additional treatment |
| What does the date mean? | Service, signature and receipt dates kept separate | An ambiguous date becomes an unqualified fact |
| What changed after the new file? | Changed statements and their supporting version identified | Earlier sources or unresolved conflicts disappear |
Give the same packet to the person who will review production output. Ask them to retrieve the supporting page without help from the demonstrator. Record time spent finding sources and correcting statements separately from generation time. If an answer is wrong, retain the prompt, input version and output rather than silently editing the evidence used to assess the pilot.
Check what happens after correction
Now introduce one corrected document. Ask which existing statements need review and which remain supported. Can the reviewer distinguish a new document from a new event? Can the updated output still identify its inputs? The Eve chronology update checklist offers another way to structure this exercise. It is a transferable review method, not a claim that the two products have identical update features.
Before buying, agree what evidence would justify continuing, narrowing or stopping the pilot. An unresolved citation issue should remain visible in the decision record. Use the Harvey quote worksheet to connect the tested task with the quoted products and access. Do not approve a larger package solely because a demonstration worked on a clean file selected by the vendor.
Separate missing documents from missing treatment
An uploaded record can describe treatment or a billed charge; it cannot, by itself, establish what a provider reports today. Our chronology and verification guide separates those questions. When a source is missing, use the records follow-up guide to assign an owner, request scope and next action. Keep requests, verbal updates and received files as separate events.
VerifyMD focuses on provider calls followed by review of the returned information and received documents. A workflow discussion can examine that handoff alongside your existing document-analysis tools. Live calling remains subject to assisted setup and explicit owner activation. The firm retains responsibility for source review, factual interpretation and legal or clinical conclusions.
Explore provider follow-up and source review for your firm. Live calls require agreed setup and activation.
Book a demo