Common Call Parameters

max_tokens, top_p, stream, system — what each parameter means for your product.

There aren't many parameters, but each one maps directly to a problem users can feel. You don't need to tune them expertly — you need to know what happens when they're wrong, because a lot of "the model is bad" complaints trace back to parameters.

Symptoms you'll recognize:

• Answers stop mid-sentence, as if someone cut them off.
• You ask for a fixed format and two times out of ten you get a polite preamble anyway.
• Users say "it's too slow," but the response time in the backend looks fine.

What the model says

system — the system prompt. Placed at the front of the conversation and sent on every round: the product's role, tone, boundaries, and prohibitions. It has higher priority than a normal message and is more stable. The product's "personality" lives here. Since it's resent every round, it's the block most worth caching — put fixed content first and variable content last, and the cache hit rate climbs.

temperature — randomness. Worth its own article; the short version is "turn it down for stable output — the default is usually too high."

top_p — another way to control randomness. Temperature reshapes the probability distribution over candidate words; top_p only samples from the set of candidates whose cumulative probability crosses a threshold. In practice, tune only one of the two — usually temperature. Tuning both at once interferes, and you can't tell which one is doing the work.

How much it says

max_tokens — the output length cap. The most common failure point. It's a hard cutoff: the model is forcibly stopped at this length and doesn't wrap up on its own. The symptom is classic — an answer that stops mid-sentence, and users blame the network or your product. It's also the main knob on your bill, because output is several times more expensive than input. So bigger isn't always better: too small truncates, too large removes the cost guard. Set it per feature rather than globally; state the expected length in the prompt ("answer in three sentences") so the model writes tight on its own, which is more graceful than a hard chop; and detect "stopped because of the limit" — the API tells you, so don't serve the cut-off half to users. openai-api-ref

stop — the stop sequence. Stops generation when a specified string appears. Used for structured output or preventing the model from continuing to ramble on. You won't use it much, but it's good to know it exists.

How it's delivered

stream — streaming output. The single biggest lever on product feel — but it changes perceived speed, not actual speed. Without it, users stare at a spinner for seconds and then get the whole answer at once. With it, the first character arrives in a few hundred milliseconds and the rest trickles out. Total time can be identical, yet the experience is nothing alike: the former feels frozen, the latter feels like thinking. It usually rides on a server-sent events transport. sse-wiki Any scenario where the user is waiting on screen should use it. The cost is engineering: handling the stream and the "generation failed halfway" state.

response_format — forced output format. Use it when you need structured data like JSON. Far more reliable than "please only return JSON" in the prompt — which still produces "Sure, here's your result:" a couple of times out of ten. If your feature pipes model output into a program, this is almost mandatory.

seed — reproducibility. Setting it should give the same output for the same input — in theory. Vendors only promise best effort, not a guarantee. Server-side versions and batching affect results. Use it for test baselines, not for compliance-grade determinism.

Presets for four scenarios

When in doubt, start here, then adjust with real measurements.

These are starting points, not standards. The real configuration comes from running your own samples — the same set, different parameters, compare the results.

One last reminder: the defaults are almost never the setting you want. Vendors tune them for general chat, and you're building a specific feature. Every time you ship a feature, review all three categories of parameters.