If you are building an AI product, one of the first surprises on your cloud bill is how much the cost of a single request can vary. The same feature can cost a twentieth of a cent on one call and twenty cents on another. That is roughly a 400x spread, and it can decide whether your gross margin works.
This guide looks at that spread from a founder's point of view: how to think about it, how to measure it, and how to build it into product and pricing decisions.
Two dials behind every request
Every model menu comes down to two choices:
- Model size. Larger models have more connections, which gives them better judgment but a higher price per token.
- Thinking level. A thinking model writes hidden notes to itself before answering. Each note is another full pass through the network, so thinking adds care, at the cost of time and tokens.
A useful way to remember it: thinking adds care, size adds judgment. A small model that thinks can sometimes rescue an ambiguous request, such as working out that "bank" in a sentence about a hike means a river. Hard problems still tend to reward the combination of both.
The cost formula to put in your model
Your bill for a request is tokens multiplied by price per token. Tokens come in three kinds:
- Input: what the model reads.
- Thinking: the hidden notes. Your users never see them, but they are billed as output.
- Output: the visible answer.
Thinking level and question difficulty drive the token count. Model size drives the price of each token. Because the two multiply, costs compound rather than add.
In one example receipt, 3,000 thinking tokens cost six cents and were the largest line item. Hidden does not mean free.
What the effort dial looks like
The rough shape of effort levels, as an illustration:
| Effort | Approx. tokens | Approx. wait | Result |
|---|---|---|---|
| Low | ~500 | ~5 seconds | Quick answer |
| Medium | ~3,000 | ~15 seconds | Solid answer |
| High | ~10,000 | ~45 seconds | Checked answer |
| Max | up to ~30,000 | Minutes | Double-checked answer |
Prices also differ by model. Within one provider's family, output prices can sit up to 100x apart, for example from fifty cents for a small model to fifty dollars for the top one.
Why this matters for startups
Founders often pick one model and one setting for the whole product. That is simple, but it can hurt you in two ways:
- Overspending: running a top model on max effort to fix a typo wastes margin on every call.
- Underdelivering: running the fastest model on low effort for work that needs checking, such as scanning an expense report for duplicate receipts, can produce errors that cost you customer trust.
A practical routing approach
Instead of one setting, map your product's tasks to tiers:
- Small model, light effort: typo fixes, formatting, simple classification.
- Mid-size model, medium effort: everyday drafting and summaries.
- Most capable model, high effort: high-stakes analysis where a mistake is expensive.
The rule of thumb is to choose the smallest model and the least thinking that does the job well. Then test it on tasks that resemble your real users' requests, not generic benchmarks. Before raising effort, check that you are on the latest model version, since newer models may close the gap on their own.
Questions to ask before you price
- Which model handles each type of request in my product?
- Does it think harder on hard questions, and what does that cost me?
- Was it evaluated on tasks like my customers'?
- What is my cost per task at the low and the high end, and does my pricing survive the high end?
Track cost per completed task, not just per call, so thinking tokens show up in your numbers from the start.
Where to go next
The underlying mechanics, including how the loop of notes works, are covered in the original walkthrough, Which AI model should you use, Part 2: How AI thinks. If you want hands-on help turning these tradeoffs into an architecture and roadmap, the team behind this studio shares more at the main website.