Test-Time Compute

Test-Time Compute


For most of the last decade the way to a better model was a bigger training run. Test-time compute moves part of that budget to the moment the question is asked. The model is allowed to think longer, sample several candidate answers, or check its own work before replying.

Three mechanisms do most of the work. Extended reasoning chains let the model write out intermediate steps before committing. Parallel sampling generates many attempts and selects among them, either by majority vote or with a separate verifier model. Iterative revision has the model critique and rewrite its own draft.

The empirical finding that made this a research direction is that the returns behave like a scaling law of their own. On problems with checkable answers, such as competition mathematics or code, letting a mid-sized model reason for longer can beat a much larger model answering immediately.

The cost lands on the serving bill and on latency, so production systems usually route only the hard requests down the expensive path.

From generative AI strategy to custom agent development and retrieval architectures, we help you scale AI responsibly.
Discuss your AI project