Maximum Response Length

Maximum Response Length


Maximum response length is the largest number of tokens a model may produce for a single request. The cap is counted in tokens rather than words because the model works on tokenised text. One English word is often one token, but longer or non-English words split into several, so the same cap yields fewer words in some languages than others.

The setting is normally passed through an API parameter such as max_tokens. When the model reaches that count it stops, which is why replies sometimes end mid-sentence. It is worth keeping separate from the context window. The context window is the combined budget for prompt and reply, while maximum response length only governs the output side of that budget.

The cap does two practical jobs. It keeps cost predictable, since billing is per token. It also protects the design constraints of the surrounding product. In a flow that writes product descriptions, a 120-token ceiling stops copy that would overflow its box from being generated at all.

Take an ecommerce integration that drafts category copy. Set the ceiling too low and the text arrives half finished. Set it too high and the model rambles while token spend climbs. Teams usually measure the right ceiling on a handful of samples, then fix it.

Responses also come back with a finish reason. Checking that field tells you whether the model stopped on its own or ran into the cap.

From generative AI strategy to custom agent development and retrieval architectures, we help you scale AI responsibly.
Discuss your AI project