All posts

How We Make AI Extract Deal Terms You Can Actually Trust

If you tell a fund manager "AI reads your deal documents and fills in your cap table," the correct first reaction is suspicion. Language models produce fluent, confident, occasionally wrong output, and in fund operations a plausible wrong number is strictly worse than a blank cell. A hallucinated liquidation preference doesn't look like an error; it looks like data, right up until an LP, an auditor, or opposing counsel checks it.

We build TermProof, which turns a fund's legal documents into an auditable system of record: structured deal terms extracted from the instrument family a venture portfolio actually holds — SAFEs, convertible notes, priced equity (SPAs), side letters, warrants, term sheets, charters (SAFEs, SPAs, and side letters run end-to-end today; the rest are in progress, as noted at the end). The name is the design goal: every term, proven. This post is the engineering answer to the suspicion: the specific mechanisms that stand between "a model said so" and "this is now in your fund's official records." None of them is exotic on its own. The system property comes from stacking all six.

The design principle: extraction is a claim, not a fact

Every value the model produces is treated as an unverified claim that must carry its own evidence and survive verification and review before it becomes a record. Concretely, the pipeline is:

upload -> parse -> classify -> extract (with provenance)
       -> verify quotes -> route by confidence -> human review
       -> commit -> auditable ledger

A value can fail out at any stage. Only the last stage writes to cap tables, terms, or provisions.

1. The model can't free-write: schema-forced output

Extraction calls to Claude use forced tool choice against a JSON schema generated from per-document-type field specifications. A SAFE has a different spec than an SPA or a side letter, each defining fields, types, and normalization rules. The model isn't asked to "summarize the document"; it's constrained to return one specific structure, and prose is not an option.

When the response comes back, we validate it with zod before anything is persisted. Malformed or incomplete output fails the extraction run with a typed error and the document is marked failed, retryable. It never partially writes.

The field specs also encode domain traps as extraction hints. The canonical one: a SAFE converting at "80% of the price per share" is a 20% discount. Rates are normalized to 0-1 decimals, dates to ISO format, so the same term extracted from two documents is comparable.

Classification and extraction are split across two models: a small, fast model determines the document type first, and the heavier model runs extraction against that type's schema. A low-confidence classification doesn't guess its way forward; it surfaces as a task asking a human to confirm the document type before extraction proceeds. Getting the schema wrong at step one would poison everything downstream, so it gets the same treatment as any other uncertain claim.

2. Every field carries provenance, enforced by the schema

The schema wraps every extracted field, including nested fields like each purchaser inside an SPA, in the same required envelope:

{
  "value": 8000000,
  "confidence": 0.97,
  "source_page": 3,
  "source_quote": "the Post-Money Valuation Cap of $8,000,000"
}

All four keys are required. A value without a citation is schema-invalid, so the model cannot return one. Quotes must be verbatim and short (under 200 characters), and pages are 1-indexed against the actual document.

This changes the reviewer's job from "re-read the document" to "check one sentence." It's the difference between auditing and re-doing the work.

3. Quotes are verified, not trusted

Requiring a citation is not enough; a model can fabricate one. So after extraction, every source_quote is checked programmatically against the parsed document text: the quote must appear verbatim (whitespace-normalized) in the document, and source_page must be within the actual page count. We parse PDFs page-by-page precisely so this check, and in-context highlighting for reviewers, is possible.

A field that fails verification isn't discarded, because the value might still be right. Its confidence is downgraded and it is force-routed to human review. Fabricated evidence means the fast path is closed.

Being honest about the limit: this catches fabricated citations, not faithful misreadings. The model can quote a real sentence and still interpret it incorrectly. That residual risk is why the next two layers exist.

4. Confidence with a rubric, and routing by threshold

Raw LLM confidence numbers are self-assessments, not calibrated probabilities. We make them useful in two ways.

First, the extraction prompt defines what the numbers mean: 0.95+ for values explicit in the text, 0.80-0.95 for values derived from clear language, 0.60-0.80 for ambiguous ones, and below 0.60 the model must return unknown rather than guess. "Never infer" is an explicit instruction; a missing term should surface as missing.

Second, we treat confidence as a routing signal, not a truth signal. Fields below a configurable review threshold (default 0.85), fields with null values, and anything that failed quote verification open review tasks for a human. High-confidence, verified fields stay marked unreviewed rather than silently promoted, so the review queue holds the fields that actually need judgment instead of flooding reviewers with 60 rubber-stamps per document, which is how review fatigue kills real oversight.

5. A human approves before anything becomes a record

Nothing the model produces touches the ledger on its own. An analyst works the review queue seeing each value next to its verbatim quote and page, and approves or corrects field by field. Only then does the reviewed extraction commit into the fund's official records: issuances, terms, provisions, and from there cap tables and fund analytics (IRR, TVPI, DPI).

This is a deliberate product decision, not a temporary crutch until the model is "good enough." In a system of record, the human approval gate is what makes the AI's speed safe to use. The AI does the reading; a person remains accountable for the record.

6. The audit trail makes trust inspectable

Every official write and every review action lands in an append-only audit log. For any number in the system, you can trace: which document it came from, the page and verbatim quote supporting it, its extraction confidence, who approved or corrected it, and when it was committed.

That matters beyond debugging. LP reporting expectations have gone institutional (ILPA's 2026 templates are the marker), and "an AI extracted it" is not an answer auditors accept. "A named person approved this value, supported by this quote on page 3, on this date" is.

What this doesn't solve

In the spirit of the whole post:

  • Faithful misreads. Verification catches invented quotes, not wrong interpretations of real ones. Human review is the backstop, and it's mandatory, not optional.
  • Confidence isn't calibrated probability. The rubric and thresholds make scores operationally useful; they are not statistical guarantees.
  • Coverage. SAFEs, SPAs, and side letters run end-to-end today. Convertible notes, charters, and investor rights agreements are in progress.
  • Scanned documents. No OCR yet; a PDF without a text layer won't parse.
  • Very long documents. Extremely long closing binders currently hit a head/tail truncation strategy; per-exhibit splitting is on the roadmap.

Try it against your own skepticism

The demo sandbox needs no signup and is seeded with fixture documents, including a multi-document closing set: termproof.com/demo. Run the loop end to end: upload, extract, open the review queue, click a source quote, correct a field, commit, then check the audit log. The system is designed to be distrusted first. That's the point.

Early access is free; accounts are waitlist-gated at termproof.com.