OpenAI GPT & o-series

What separates a reasoning model from a general one

Both families take a prompt and return text. The difference is what happens in between.

A general model produces its answer directly. A reasoning model first spends reasoning tokens, working through the problem before it writes anything you see. OpenAI's documentation says this lets the model "plan, use tools effectively, inspect alternatives, recover from ambiguity, and solve harder multi-step tasks."

From your code, the API shape is the same. What changes is latency, cost, and the kind of problem the model can hold together.

Reasoning tokens are billed, invisible, and take up the window

Three facts that arrive together, and each surprises somebody.

Billed as output tokens. You pay output rates for thinking you never read.

Not visible via the API. You cannot inspect the reasoning, so you cannot debug an answer by reading how it got there.

They occupy the context window. Reasoning shares the budget with your prompt and your answer.

That third one produces the confusing failure. A request that fits comfortably under the limit can still return a truncated answer, because reasoning consumed room you did not account for. Leave headroom, and read the field that reports why generation stopped rather than assuming a short answer was a short answer.

The cost effect is easier to predict and easier to underestimate. A task where the model thinks at length before producing two sentences bills like a long response, because it was one.

The effort setting, and why it is not the first thing to reach for

There is a control, reasoning.effort, ranging from none to max. It guides, in the documentation's words, "how much to think when performing a task."

The guidance attached to it is worth more than the parameter:

Treat reasoning effort as a tuning knob, not the primary way to recover quality.

Clear task definition and constraints matter more than turning effort up. That inverts the instinct. When results disappoint, the reflex is to reach for a bigger model or more thinking; the documentation says to fix the task first.

Effort is what you tune once the prompt is right, to trade latency and cost against depth. It is not a repair for a question the model cannot tell how to answer.

Matching a family to a task

Reach for a reasoning model when the work has steps that depend on each other: multi-step problems, coding, scientific reasoning, planning, and tool use where the model has to decide what to do next. These are the cases where quality outweighs latency.

Reach for a general model for routine retrieval, classification, transformation, and anything latency-sensitive such as voice. Paying for thinking on a task that needs none costs money and time for no benefit.

The mixed approach usually wins in production: a general model for the common path, and a reasoning model behind a route that recognizes the hard cases. That keeps the median response fast while the difficult requests get what they need.

Do not pick on a benchmark you read this quarter. Run both on your own task with your own evaluation set, and let the numbers be about your work. Choosing the right model covers how to structure that comparison.

Further reading

• OpenAI, Reasoning: reasoning tokens, billing and visibility, the effort setting, and when to use which family.
• Inference: where latency and cost come from in any model call.
• Google Gemini: the same decision at another provider.