How we measure NotarIA.
Our public claims rest on reproducible measurements, not promises. This page documents the protocol, corpus and limits of our title-examination extraction benchmark, run on real Québec Land Register documents.
What was measured
The benchmark evaluates the extraction of register entries (act nature, dates, parties, amounts, lots) from real Land Register documents. Result: 100% recall and 100% precision across the corpus's 15 entries (7 scored documents), stable over two consecutive runs — at a measured cost of about USD 0.06 per full-corpus run.
The harness (npm run bench:titres) calls the real production chain — no special “benchmark path”. Each extraction is compared to a hand-annotated gold standard; an invented entry counts against precision, an unreadable document counts as a coverage failure. Nothing is removed from the score.
The corpus
Seven scored documents: three Land Register indexes (one an official-format specimen, identified as such) and four real published acts (servitudes, notice of deterioration, notarized sale). Six further documents probe coverage behaviour — pure scans, narrative text, out-of-scope documents: the system refuses to invent and says so.
The limits, honestly
A small corpus
Seven documents and fifteen entries, biased toward public documents; no old handwritten index is scored.
A gold standard under review
The annotation was produced by the development team; review by a notary is under way. Ambiguous fields are not scored.
A measurement, not a guarantee
Results describe this corpus, at a point in time. Every real file is still verified — and concluded — by the notary.
Why we publish our limits
A figure without a protocol is a promise; a figure with a protocol is a measurement. Publishing limits is part of the product: it is the doctrine of our reports — cite the source, mark the reservations, leave the conclusion to the notary.
Frequently asked questions
Is the benchmark reproducible?
Yes. The harness (npm run bench:titres) and the matching protocol are documented, and the corpus comes from public Land Register documents. Two consecutive runs at the same score are required before a result is published.
Who validated the figures?
The gold standard was annotated by the development team, and review by a notary is under way. We publish the limits with the results and will update this page after the review.
Judge on your own documents.
Request a demo on a real property of your choice.
Assistant — not legal advice. The notary reviews, signs and decides on every act.