What we mean when we say Joe never invents an answer
Sources, refusals and escalation. The three mechanisms that keep an agent from being confidently wrong.
“It never makes things up” is the kind of claim that should make you suspicious, because on its own it is unfalsifiable marketing. So here is the specific version: three mechanisms, what each one actually does, and where each one still fails.
1. Answers come from sources, not from memory
The model is not asked to recall your prices. It is given them, at the moment of answering, from the knowledge you provided — your services, hours, policies, the documents you uploaded.
The practical difference: if a fact is not in your sources, there is nothing to retrieve, and the honest outcome is a refusal rather than a plausible guess.
Where this still fails: if your sources are wrong or out of date, the answer is confidently wrong and the mechanism cannot save you. Retrieval guarantees the answer came from your material. It does not audit your material. Stale price lists are the single most common cause of bad answers we see, and no amount of model quality fixes it.
2. Refusal is a first-class action
Most systems treat “I don’t know” as a failure state to be minimised. That incentive is exactly backwards, and it is where invented answers come from — a system pushed to always produce something will produce something.
Refusal here is a normal, frequent, successful outcome. When the sources do not cover the question, the agent says so and hands it on. There is no reward for filling the gap.
You can see this in the logs, and you should look: an agent that never refuses is not well-informed, it is over-confident.
Where this still fails: the boundary is fuzzy. A question that is partly covered is the hard case — enough context to seem answerable, not enough to be right. This is the category we watch most closely, and it is genuinely not solved.
3. Hard rules sit outside the model
Anything with a threshold — refund limits, discount authority, cancellation inside a notice window — is not a matter of judgement. Those are deterministic rules evaluated outside the model. They are not phrased as suggestions in a prompt, because a suggestion in a prompt is not a control.
If a request crosses a rule, it escalates. The model does not get a vote.
Where this still fails: only for rules you wrote down. An unwritten policy that lives in a manager’s head is not a rule, and the system cannot enforce it. Most “it did the wrong thing” reports resolve to a policy nobody had ever stated explicitly.
What this adds up to
Not a guarantee of correctness. A guarantee about failure mode: when the system does not know, it stops and tells you, rather than producing something fluent and wrong.
That distinction matters more than accuracy percentages, because a wrong answer delivered confidently to a customer is a commitment your business now has to honour or retract. A refusal costs you a few minutes of someone’s attention.
The honest summary is that we have made the cheap failure much more likely than the expensive one. That is the whole claim.