Evaluation
Your accuracy number is not an evaluation
A single accuracy figure averages away the cases a review board actually asks about. What replaces it is a breakdown by decision, not a bigger test set.
July 14, 2026
Insights
Notes from the data layer
What we have learned building evaluation, ground truth and retrieval for programmes where the record is scrutinized. Written by the people doing the work.
Retrieval
Retrieval is a data problem before it is a model problem
Most retrieval systems that fail in regulated environments fail before the model is involved. The corpus, not the embedding, is usually the thing that needs work.
May 19, 2026