THE INSTRUMENT ROOM
A reliability score is one number off a much larger instrument. The rest of it is below, running, so you can work the buttons yourself.
EIGHT INSTRUMENTS · PRESS RUN ON ANY OF THEM · EACH ONE LABELLED DELIVERABLE OR RESEARCH
Benchmarks test whether a model can pass an exam. The profiler measures where its knowledge runs out, domain by domain.
≥70 WITHIN BASELINE · 50–69 WATCH · <50 ELEVATED · COMPOSITE: 60% RECOGNITION + 40% ACCURACY
An invented dosage or a made-up citation normally goes unnoticed until a person reads it. Fabrication has a shape in the probability distributions, readable at every token.
The strongest signal shows up while the model is still reading your question, ten to fifteen times more per token than during generation. In the lab, gating on it stops a flagged response before it starts. Your report carries the same signal as per-topic recognition scores. Per-model results for the pre-generation phase are in the method paper (DOI: 10.5281/zenodo.21365655).
A provider pushes an update, or a fine-tune shifts behavior, and the model that measured well last quarter is not the one answering today. Scheduled re-measurement puts the change on a chart where someone can see it.
When safety training kicks in, the probability distributions look different from both normal operation and fabrication. In the lab's thirteen-model refusal study, refusals carried two to three times more temporal texture than normal generation, in a region of their own between the two.
From output probabilities alone, the lab reconstructs which concepts the model is weighing, validated on 3B to 16B models in the published geometry paper (DOI: 10.5281/zenodo.19326718). It is where the work on explaining a given answer, without opening the model up, is headed.
Most organizations end up running more than one model. When several of them go blind on the same kind of question, the failures land together.
The same measurement that separates knowledge from fabrication reads as a continuous spectrum, running from pure fabrication to verbatim reproduction.
Put this instrument on your model.
One email gets you the complete sample report. Same-day reply, no call required.