Token prices fell. Why isn’t your AI bill falling?
The per-token price was never the lever. The real ROI is in tuning the whole loop — reasoning, retrieval, routing, retries — and that loop, built on your own knowledge and judgment, is the one part a competitor can’t rent.
Four months of falling token prices, and the invoice went the other way. The gap is the whole story — and the answer isn’t in the price list.
Per token, AI got dramatically cheaper. Most companies’ bills went up anyway. The reason both are true: the token price was never the thing you were buying.
- The price fell on the one term you don’t control. Your bill is set by the five terms you do.
- Those terms — how many tasks, how many model calls each, how many tokens per call, how often you retry — are decided by your architecture, not the vendor’s price list.
- The ROI is in tuning that loop: evals, a routing governor, caching, tight retrieval, reasoning budgets, first-pass yield.
- That loop, built on your own data and your experts’ judgment, is the durable IP — what Satya Nadella calls the learning loop where “human capital and token capital compound.”
“I think predominantly token usage will be commodity token usage,” Microsoft’s Satya Nadella said on the 20VC podcast this year. He is right, and the price sheets prove it. In July we shipped a calculator for what enterprise AI costs; we just re-priced it against November’s published list prices, and the token — the raw unit of machine thinking — is now a commodity sliding toward zero.
And yet the enterprise AI bill kept climbing. If the input got cheaper and the invoice went up, then the price per token was never the thing you were actually buying. This piece is about what you were.
1 · The token became a commodity
Start with our own lineup, because that is what the calculator is priced on. The economy tier — Claude Haiku — held flat at $1/$5 per million tokens. The workhorse got a third cheaper, the premium tier a fifth, and a new, pricier frontier rung opened above both.
| Claude tier | July 2026 | Nov 2026 | Move |
|---|---|---|---|
| Economy — Haiku 4.5 | $1 / $5 | $1 / $5 | flat |
| Standard — Sonnet | $3 / $15 (4.6) | $2 / $10 (5.5) | −33% |
| Premium — Opus | $5 / $25 (4.8) | $4 / $20 (5.5) | −20% |
| Frontier — Fable (new) | — | $8–10 / $40–50 | new tier |
Anthropic published list prices, per million tokens (input / output). Figures track the current release of each line.
The “AI is 10× cheaper” headline is real, but it lives below everyone, in the budget tiers OpenAI and Google opened: gpt‑5‑nano at $0.05/$0.40, Gemini 2.5 Flash‑Lite at $0.10/$0.40. Against July’s $1/M economy input, a $0.05 input is a ~20× drop at the bottom of the market. At the same time a new frontier ceiling opened at the top — Fable 5.1 and OpenAI’s gpt‑6‑astra at $50 per million output tokens. The market now spans $0.05 to $50: a 100× price range, live today, where in July it was a tidy $1–$5.
This is exactly the commodity Nadella described — the per-token price collapsing and spreading out, the raw material getting cheap and plentiful. Which raises the obvious question.
2 · So why didn’t your bill fall?
Because the price per token is one term in your bill, and it is the only one the vendors control. Write the monthly bill out and the shape of the problem is immediate:
Exactly one factor fell: price per token. Every other factor is set by how you build, and each one rose through 2026:
- Calls per task. An agent that takes actions makes 5–15 model calls per user task, not one. Those patterns went from demo to production this year. Ten calls at a third the price is three times the old bill, not a saving.
- Tokens per call. Reasoning models think before they answer — cheaper per token, far more tokens per task. Longer context, more retrieved knowledge and longer histories push the same way.
- Retries. A failed attempt re-runs the whole turn. A 30% retry rate is a flat 30% tax on everything above it — and it is almost always invisible in the quoted estimate.
So the vendors cut the one term you do not control, and your own architecture inflated the four you do. That is the entire reason cheaper tokens did not reach your bill. It also points straight at the fix: stop trying to buy the token cheaper, and start tuning the loop that decides how many of them you spend.
3 · The ROI is in the loop
Right-sizing AI cost is an engineering discipline, not a procurement decision. Each lever below attacks a specific term in that equation, and they compound:
| Lever | What it does | Term it attacks |
|---|---|---|
| Eval rails | Measure cost per outcome and prove quality held — the backbone that makes every cut below safe to pull. | all of them |
| Model routing | Cheap model first, escalate to a bigger one only on low confidence. With a 100× spread this is the highest-leverage lever there is. | price⁄token |
| STP vs human | A governor that runs straight-through when the model is sure and escalates to a person when it isn’t — so you don’t pay for a call that didn’t need to happen. | calls⁄task |
| Caching | Prompt caching cuts cached input ~90% and latency; semantic caching removes the repeat call entirely. | price⁄token, calls⁄task |
| Tight retrieval | Retrieve the three chunks that answer it, not the twenty that might — stop stuffing the context window. | tokens⁄call |
| Reasoning budgets | Tune how hard the model thinks per task class, and cap output. Don’t leave every call set to maximum. | tokens⁄call |
| First-pass yield | Constrained decoding so malformed output is impossible, not retried — killing the retry tax at its source. | (1 + retries) |
None of these is about finding a cheaper provider. They are about spending tokens only where they change the outcome. And the one at the top — evals — is not just another lever; it is the measurement backbone the rest ride on. You can only down-shift a tier, trim a context or trust straight-through processing if you can prove the quality held. Without evals, every cost cut is a blind quality gamble.
4 · The retry tax, and the eval paradox
Retries deserve their own paragraph because they are the sneakiest line in the bill. They do not shrink any single term — they re-run the whole turn. And here is the trap: the eval rail that saves you money also costs you money. It is what lets you run a cheaper model safely — but every time it rejects an output, it triggers a retry, and each retry is a full-price turn. A naive “reject, regenerate the same thing” loop can erase the entire saving from cheaper tokens.
So the governor’s real job is not just straight-through-versus-human. It is what to do on a failed attempt: retry, repair, escalate, or hand off. Cap the retries at one or two, then move up the cascade or to a person — never loop. Re-running the same cheap model on the input that just failed is pure waste: double the cost, the same likely failure. Repair the broken part with a cheap call instead of regenerating the expensive reasoning behind it. And measure cost per task with retries baked in, because the happy-path number undercounts by exactly your failure rate.
5 · The loop is the moat
Here is the part that turns a cost exercise into a strategy. If the token is a commodity everyone rents at the same falling price, then renting it is not an advantage. What differs between two companies running the identical model is the loop around it — the routing policies, the evals that encode what “good” means in your domain, the retrieval grounded in your documents, the escalation rules tuned by your experts over thousands of real cases. Same model, completely different economics and outcomes, because the loop is different.
That is the point Nadella has been pressing all year. In a long-form post he described the goal as:
“building a learning loop on top of models where human capital and token capital compound.”
— Satya Nadella, Microsoft CEORead that against the equation. “Token capital” is the commodity — the falling price, available to everyone. “Human capital” is your SMEs’ judgment and your institutional, tribal knowledge. The learning loop is the orchestration we have been describing — the thing that pours your people’s expertise into how the models get routed, grounded, checked and corrected, and gets a little better every time it runs. The model is one copy away from commoditisation. The loop, trained on knowledge only you have, is not. It compounds, and it cannot be rented.
Which reframes the whole cost conversation. Tuning the loop is not a tax-management exercise to be delegated to the platform team. It is where the durable advantage is built. The companies that win the next phase will not be the ones that found the cheapest token — that price is converging for everyone. They will be the ones whose loop turns a commodity model into an outcome no competitor can reproduce.
6 · Re-cost it yourself
The calculator below is refreshed to November prices — it shows the token line for your use cases, the easy part of the bill. Treat its numbers as the floor under a real cost model, then remember that your loop decides everything the equation multiplies on top.
The leader’s verdict
- Stop optimising the token price. You don’t control it, it’s already falling, and it’s converging for your competitors too. It is the one term that will not differentiate you.
- Budget on cost per outcome, with retries in. Tokens-per-task × tasks-per-month × the retry rate — not price-per-token — is the number that moves your bill.
- Invest in the loop. Evals first, then a governor that routes between models, between straight-through and human, and between retry and escalate. Then caching, tight retrieval and reasoning budgets.
- Treat the loop as IP. Build it on your proprietary knowledge and your experts’ judgment. It is the part that compounds, and the part a competitor renting the same model cannot copy.
- Mind the currency (India). The rupee slid from ~₹84 to ~₹94 since July, so a 33% dollar price cut is only ~25% in rupees — one more reason the token line is not where your leverage is.
Model prices from Anthropic, OpenAI and Google published pricing pages as of November 2026 (input / output per million tokens). Nadella quotes: “commodity token usage” from the 20VC podcast; the “learning loop… human capital and token capital compound” line from his 2026 long-form post on X. Use-case costs are computed in our estimator with production context overhead held constant from the July model, so the comparison isolates price and tier changes. All figures are indicative — validate against current provider documentation and your own workload before budgeting.