Deterministic vs probabilistic AI: what’s the difference?
| Deterministic | Probabilistic (LLMs) | |
|---|---|---|
| Same input, same output | Always | Not guaranteed |
| Handles messy, novel input | Poorly | Well |
| Can be audited line by line | Yes | Only by its behavior |
| Typical failure | Consistently wrong rule | Plausible but wrong answer |
A language model is probabilistic even when you ask it not to be. Setting temperature to zero is not enough: in one test, 1,000 runs of the same prompt at temperature 0 produced 80 different completions; the authors trace the variation to batch size, which on a hosted service changes with server load (Thinking Machines Lab, 10 September 2025, read 9 November 2026). If a decision must be repeatable, a model cannot be the thing that makes it.
Why do businesses want deterministic AI?
Because regulators, auditors and customers ask “why did the system decide this?” and expect the same answer tomorrow. Finance approvals, insurance eligibility and access control all need a rule someone can point to. A deterministic decision can be tested once and trusted; a probabilistic one has to be sampled and monitored forever. Deterministic does not mean correct: a wrong rule is wrong every time, which at least makes it easy to find and fix.
How do you combine a model with deterministic decisions?
Split the work. The model reads the email, the scan or the call and returns structured output. Code then validates that output, checks thresholds and decides. A confidence gate sends low-certainty cases to a person instead of guessing. The model interprets; the rules decide. Every action with money, customers or records behind it passes a check the model cannot talk its way past.
How do we decide what stays deterministic in our builds?
We put the cheap deterministic check first. In one agent we operated, a simple check sat behind two slow, throttled lookups, so under load it inherited their worst case and missed its deadline; moving it to the front fixed that. When we replace a model step with code, we first list everything the model was getting right and test the new rules against the full live set of cases, not a sample, because deleted model behavior fails silently. And any field the model can write, including citations and IDs, goes through a fail-closed gate before anything acts on it. Our post on deterministic confidence gates walks through one in production, and LLM output validation covers the checks themselves.