The provider dashboard tells you that you spent $412 last month. It does not tell you that $280 of that came from four accounts, that three of them are on a free tier, or that one integration has been retrying in a loop since the ninth.
Aggregate spend is the least actionable number in the entire system. It tells you something changed and gives you no way to find out what. The fix is small, it costs a few columns on a table, and it has to go in before launch because it cannot be applied retroactively to calls that already happened.
This is one of the six components that define Template 04 in my stack, and it is in the base scope of every AI build I quote rather than something a client can decline.
Why does aggregate spend tell you nothing?#
Because every useful question about AI cost is a question about a subset. Which customers are unprofitable. Which feature is expensive. Whether the increase this month is more users or the same users doing more. Whether the prompt change last week made things cheaper or dearer.
None of those can be answered from a monthly total. You can watch it rise and you cannot act on it, which in practice means the response to a rising bill is either to do nothing or to make a change and hope. Both are common.
The asymmetry is what makes this worth doing early. Attribution costs almost nothing to add at the start and is impossible to add later for calls that have already happened. When the alarming invoice arrives, the data you need to investigate it either exists or it does not, and no amount of urgency creates it after the fact.
What should you log on every call?#
Seven fields, written at the same time as the response. None of them require a separate service, and the whole thing is one insert on a table you already have.
| Field | Why it matters |
|---|---|
| user_id and org_id | The entire point — attribution |
| feature | Tells you what to optimise |
| model and version | Prices and behaviour change over time |
| input_tokens, output_tokens | The actual cost drivers |
| cost_cents | Computed at write time, never later |
| latency_ms | Catches degradation before users report it |
| status | Success, refused, rate-limited, error |
The feature column earns its place faster than people expect. Knowing that a user is expensive is half the answer; knowing they are expensive because of document extraction rather than chat tells you what to change. Without it you get an expensive account and no idea which part of the product is responsible.
status matters for a subtler reason. Refusals and rate-limited calls have a cost profile of their own, and a rising refusal rate is usually the earliest signal that something upstream has drifted. Logging it here means one table answers both the cost question and the quality question.
Why compute cost at write time?#
Because prices change, and a historical record that recalculates itself against today's price list is not a historical record.
If you store only token counts and multiply by current pricing at query time, then the day a provider changes prices, every past month silently re-prices. Your March figures move. Comparisons across the change become meaningless, and you lose the ability to say what anything actually cost when it happened.
Storing cost_cents at write time fixes that permanently. It also makes the analytics queries trivial — a sum rather than a join against a pricing table with effective dates, which is a surprisingly awkward thing to model correctly.
// Written in the same transaction as the response, not after.
await db.aiUsage.create({
data: {
userId, orgId, feature: 'document_chat',
model: 'claude-sonnet-4', modelVersion,
inputTokens: usage.input, outputTokens: usage.output,
// Priced now, with the rates in force now. History stays true.
costCents: priceFor(model, usage),
latencyMs, status: 'ok',
},
});One detail that catches people: log the failed calls too. A request that errored after generating 3,000 tokens still cost money, and a retry storm is composed almost entirely of calls that failed. Systems that only log successes under-report exactly the situations you most need to see.
What is the query that finds your expensive user?#
Once the data exists, the analysis is ordinary SQL. This is the one I run first on any AI feature that has been live for a month.
SELECT u.email,
SUM(a.cost_cents) / 100.0 AS spend,
COUNT(*) AS calls,
SUM(a.cost_cents) / COUNT(*) / 100.0 AS avg_cost,
s.plan_price AS revenue
FROM ai_usage a
JOIN users u ON u.id = a.user_id
LEFT JOIN subscriptions s ON s.user_id = u.id
WHERE a.created_at > now() - interval '30 days'
GROUP BY u.email, s.plan_price
ORDER BY spend DESC
LIMIT 20;The last column is what turns this from interesting into actionable. Spend on its own is meaningless — a customer costing $180 a month is cheap if they pay $2,000 and ruinous if they pay nothing. The ratio is the number worth watching.
The distribution is almost always steeper than expected. On every AI feature I have looked at, the top few percent of users account for a third or more of total spend, and there is usually at least one account whose cost exceeds what it pays.
Run the same query grouped by feature rather than by user and you get the other half of the picture. The two together answer nearly every question that matters: which customers are unprofitable, which part of the product is responsible, and whether the two overlap. When they do — one expensive feature used disproportionately by unprofitable accounts — you have found the thing to fix, and it is usually a pricing decision rather than an engineering one.
It is worth running both queries the week after launch rather than the month after. The patterns are visible early, and acting on them at ten customers is considerably easier than at a thousand, when the pricing conversation involves people who have already been billed the old way.
What should you actually watch?#
Four numbers, and none of them is total spend.
Cost as a share of revenue, per account#
The headline metric. Anything above roughly 20% of what that customer pays deserves attention; anything above 100% needs action this week. This is the only version of the cost question that connects to whether the business works.
Cost per feature#
Tells you where optimisation effort will actually pay. It is common for one feature to account for most of the spend while receiving none of the attention, usually because it is not the one anyone demos.
Cost per call, over time#
This should be stable. When it moves without a deliberate change, something has drifted — a longer prompt, more retrieved context, a model switch, a retry loop. It is the earliest signal available and the cheapest to check.
Free-tier spend as a proportion#
Free users generating a meaningful share of model spend is a pricing problem, not an engineering one. Worth measuring specifically, because it is easy to miss when looking at totals.
How do you cap what a single user can spend?#
Three controls, layered. Together they turn an open-ended risk into a line item.
- A monthly ceiling per user or organisation, checked before the call rather than after. When it is reached, the feature degrades rather than erroring.
- A rate limit per minute, which is what protects you from a loop in your own code — a more common cause of a shocking bill than abuse.
- A global ceiling as the backstop, so no combination of accounts can produce a bill you did not budget for.
Degradation matters more than the limit itself. Hitting a ceiling should mean a shorter context window, a cheaper model, or a clear message with a reset date — not a failure on a feature the user is halfway through. A hard error reads as broken software, and users do not distinguish between a limit and a bug.
Set the ceiling somewhere a normal user will never reach. Its purpose is not to ration ordinary use; it is to bound the worst case. If ordinary users are hitting it regularly, the limit is wrong rather than the users.
When should the alert fire?#
Before the invoice, and somewhere a human actually reads.
A threshold at around 60% of expected monthly spend, checked daily, gives enough warning to investigate without being noisy. Pair it with a second alert on the rate of change — spend doubling week over week is worth knowing about even when the absolute number is still small, because it is small now and will not be in a fortnight.
Send it to the same place your error tracking goes. An alert in a channel nobody has open is indistinguishable from no alert, and cost alerts are exactly the kind that get routed somewhere quiet and forgotten.
One alert worth adding that most people skip: a threshold on cost per call rather than on total spend. Total spend rising because you gained customers is good news. Cost per call rising means something changed in the system, and the two are indistinguishable on a monthly total. Alerting on the per-call figure separates growth from regression, which is the distinction you actually want to be woken up for.
Where should the logging live?#
In your own database, alongside the users it refers to. This is not a case for a separate analytics product.
It has to join to your data#
Every useful query joins spend against users, plans and features. If usage lives in a third-party analytics tool and revenue lives in Postgres, the one question that matters — cost against revenue per account — requires exporting from both and reconciling by hand. Nobody does that twice.
It has to be written transactionally#
The usage record should be written in the same transaction as whatever the call produced. Fire-and-forget to an external service loses records exactly when the system is under stress, which is precisely when the data matters most.
It does not need to be big#
One table, indexed on (user_id, created_at) and (feature, created_at). At a million calls a month this is a rounding error against your existing database, and it queries fast enough for a dashboard without any additional infrastructure.
Retention is worth deciding deliberately. Full detail for ninety days and a daily rollup after that keeps the table small while preserving year-over-year comparisons. The rollup is a scheduled job on the background worker you already have.
What do you do with an expensive account?#
Four options, roughly in order of how often they are the right answer.
Optimise the feature they are using. Frequently the expensive account is simply the heaviest user of something inefficient, and the fix helps everyone. Sending five reranked passages instead of twenty unranked ones cut cost per answer by roughly two thirds on one system while improving accuracy — the retrieval work paid for itself twice.
Move them to a plan that reflects the cost. If the usage is legitimate and valuable, the pricing is wrong rather than the usage. This is a commercial conversation and it usually goes better than expected, because heavy users generally know they are heavy users.
Apply a limit. Appropriate for free tiers and trials, where the account is unlikely to become profitable at that level of use.
Investigate for a bug. The single most expensive account I have found was not a user at all — it was an integration retrying a failing call every thirty seconds for eleven days. Check this first, because it is both the cheapest to fix and the most embarrassing to discover late.
How do you actually reduce the cost?#
Once attribution shows you where the money goes, the reductions are usually unglamorous and large. Four levers, in the order I reach for them.
Send less context#
The single biggest lever, and it usually improves quality at the same time. Generation cost scales with input tokens, and most systems send far more context than the answer requires. Reranking twenty retrieved passages down to the best five cuts input by roughly three quarters.
The instinct that more context is safer is wrong in both directions: it costs more and it dilutes the model's attention across passages that do not answer the question. This is the rare optimisation with no trade-off to manage.
Route by task, not by preference#
Not every call needs your best model. Classification, routing and reranking are latency-sensitive and need no reasoning, so they go to something fast and cheap — Groq, in my stack. Generation, where reasoning quality is visible, goes to Claude or GPT.
Behind a provider-agnostic adapter this is one line of routing, and it commonly takes a third off the bill. Sending everything to the most capable model because it is simpler is the most common avoidable line on an AI invoice.
Cache what repeats#
In most products a meaningful share of questions are near-duplicates. Caching answers keyed on the normalised question plus the corpus version removes that cost entirely, and it makes the repeated questions faster, which users notice.
Be careful with the cache key. It has to include anything that changes the correct answer — the document set, the user's permissions, the model version — or you will serve one user an answer computed from another user's documents, which is a considerably worse problem than the one you were solving.
Shorten the output#
Output tokens usually cost several times more than input tokens. An instruction to answer concisely, plus a sensible max-tokens ceiling, reduces spend directly. It also tends to produce better answers, because the alternative is a model padding toward a length nobody asked for.
Does this apply to small projects?#
More than to large ones, proportionally. A large company absorbs a surprising invoice; a small business does not, and it is exactly the kind of unexpected cost that turns a client off AI features permanently.
The work is genuinely small — a table, an insert, a query and an alert. Roughly a day for a typical feature, and it is the same day whether the product has ten users or ten thousand. There is no scale at which it is not worth doing.
Attribution is the difference between "AI cost us $412 last month" and "one broken integration cost us $280, and here is the account".
What does this look like in the product?#
Usually invisible, and occasionally worth surfacing. For usage-based pricing it becomes a feature: showing customers their own consumption is straightforward once the data exists, and it reduces support load considerably.
Internally, the useful artefact is a single view showing spend by user and by feature over the last thirty days, sorted by cost-to-revenue ratio. That view answers most cost questions immediately, and building it takes an afternoon once the logging is in place.
For anything customer-facing, be careful about exposing raw token counts. They mean nothing to most users and they invite questions that are expensive to answer. Show consumption in units that map to what the customer did — answers generated, documents processed — rather than to what it cost you.
Conclusion#
Log the user, the feature, the model, the tokens, the computed cost, the latency and the status, on every call including the failures. Compute cost at write time so history stays true when prices move. Then run one query joining spend against revenue, and act on the ratio rather than the total.
Add a per-user ceiling that degrades rather than errors, a rate limit that protects you from your own retry logic, and an alert at 60% of expected spend delivered somewhere a person reads.
It is roughly a day of work, it goes in before launch because it cannot be added retroactively, and it converts the most anxiety-inducing property of AI features — an open-ended bill — into an ordinary, bounded line item. That is the whole of it, and it is not an upsell: it is the difference between a feature you can run and one you eventually switch off.