If you gave a judge exactly the same case file 20 times, you would not expect the plaintiff to win one day and the defendant the next. Yet that is, in effect, what some of the best-known AI models sometimes did.

Researchers Andrew Blair-Stanek and Benjamin Van Durme created 500 difficult legal questions distilled from real U.S. court decisions. Each problem involved two opposing parties, the relevant facts, competing legal arguments and one ultimate question: which party should prevail? These were not trivial cases. The researchers deliberately drew on decisions in which the judges themselves had disagreed over the outcome. (arXiv)

Then they did something remarkably simple: they submitted the exact same problem 20 times to each of three models, GPT-4o, Claude 3.5 and Gemini 1.5. Temperature was set to 0, and the other available parameters were configured to make the runs as deterministic as possible. That meant 10,000 calls to each model. (arXiv)

The results are difficult to ignore. Claude 3.5 switched the winning party in at least one of the 20 runs for 10.6% of the 500 questions. GPT-4o did so for 43%, while Gemini 1.5 did so for 50.4%. In GPT-4o's case, that means that for 215 of the 500 legal problems, repeating precisely the same text could produce both an answer in which one party won and an answer in which the other did. (arXiv)

An important distinction is needed here. The experiment does not simply prove that “AI doesn't understand law.” The cases were selected precisely because they were difficult and genuinely debatable. The more interesting problem is different: the same AI, placed in the same situation and given the same text, did not always preserve its own conclusion.

If systems like these were used mechanically to recommend legal outcomes, two people submitting the identical case could therefore receive opposing recommendations even though not a single word in the facts or legal arguments had changed.

It all split at token 103

Another experiment, conducted by Thinking Machines Lab, lets us watch almost under a microscope how such a divergence can begin.

Researchers sent Qwen3-235B exactly the same prompt, “Tell me about Richard Feynman”, 1,000 times, at temperature 0. The model generated 1,000 tokens for each completion. In theory, if the highest-probability token is selected at every step, we might expect the result to repeat. (Thinking Machines Lab)

It did not.

The 1,000 runs produced 80 distinct completions.

Even more striking is where the differences began. The first 102 tokens were identical across all 1,000 outputs. Every completion reached the same sequence saying that Feynman “was born on May 11, 1918, in...”

Then came token 103.

In 992 completions, the model continued with “Queens, New York.” In eight, it chose “New York City.” One tiny difference at a single step. But once the text differs, however slightly, the next token is calculated from a context that is no longer identical. One deviation can produce another, and another, until two answers that travelled together for 102 tokens gradually head in different directions. (Thinking Machines Lab)

It is a microscopic fork in the road.

What temperature and sampling actually do

To understand why that experiment matters, it helps to forget for a moment the idea that an AI simply “writes an answer.”

The model does not necessarily have the completed sentence sitting somewhere in its head. It builds the response token by token.

Imagine it has reached:

“The cat climbed onto the...”

Very roughly, the model might calculate something like:

  • table, 50%
  • sofa, 25%
  • fence, 15%
  • other options, 10%

Those values form a probability distribution over what could come next.

Sampling is the mechanism used to select the next token from that distribution.

Temperature changes the shape of the distribution. At a low temperature, the already likely options tend to dominate even more strongly and responses become more predictable. At a higher temperature, the differences between possible tokens are flattened, giving less likely continuations a better chance of being selected. That can produce greater diversity, more unexpected wording and, sometimes, greater creativity.

At temperature 0, the idea is to remove that deliberate randomness: the token with the highest probability is selected. This is generally referred to as greedy decoding, or an argmax-style choice. (Thinking Machines Lab)

In our simplified example, if “table” is at 50% and every alternative is below it, the model should pick “table.”

Every time.

That is what makes these experiments so interesting.

Temperature 0 takes the dice out of sampling. It does not necessarily mean there are no dice left anywhere else in the system.

How can the answer change if the model is no longer “rolling the dice”?

Thinking Machines Lab traced the problem deeper, into the way the underlying calculations are performed on inference hardware.

Models operate using floating-point numbers. Such calculations do not always behave exactly like ideal arithmetic on paper: changing the order in which certain operations are performed can create extremely small numerical differences.

On an inference server, your request is not necessarily processed on its own. Servers group and organise requests depending on workload. The size and structure of those batches can affect how some numerical operations are carried out. From the user's point of view, server load is a variable they do not control. (Thinking Machines Lab)

Most of the time, the resulting numerical differences are irrelevant.

But imagine two candidate tokens that are almost tied. In a deliberately simplified example, one has a score of 0.500001 and another 0.499999. A microscopic numerical change could reverse their order. At temperature 0 there is still no random draw: the system dutifully selects whichever token sits in first place. But if first place has just changed, the chosen token changes too.

From that point onward, the context is different.

And the answer can take a different path.

Thinking Machines Lab then built computational kernels designed to produce the same numerical results regardless of batch configuration. Once these batch-invariant kernels were enabled, all 1,000 Qwen completions became identical. (Thinking Machines Lab)

The variation, in other words, was not some mysterious expression of the model's “creativity.” It could be traced down into the infrastructure performing the calculations.

Why this matters

For a story prompt or a holiday recommendation, receiving two slightly different answers may even be desirable.

In law, medicine, finance, government or other areas where an answer can have significant consequences, the situation is different.

“Temperature is set to 0” is not enough to establish that an AI application will reproduce exactly the same conclusion every time. Reproducibility can depend on the model and model version, generation parameters, inference infrastructure and the way the underlying computations are performed.

For high-stakes systems, this creates another requirement. It is not enough to ask whether an AI gave the correct answer. We also need to know how stable that answer remains when the question itself does not change.

An “AI judge” would obviously face a long list of legal and ethical problems before ever taking a judge's seat. But these experiments reveal one remarkably simple issue that cannot be ignored:

if it said A yesterday and B today on exactly the same case, we need to be able to explain why.