AI API Cost Tracking: Tokens and Model Groups

AI API cost tracking is most useful when every charge can be connected to one request, one model, one model group, and one measured unit of work. A monthly total can show that spend changed; a request-level record can explain why.

Modelflare combines model pricing, measured usage, group policy, and request logs so users can review cost without estimating from visible text alone.

The four layers of a request cost

1. Model pricing

Each model has its own pricing basis. Many text models use input and output tokens, while image, audio, reranking, or other APIs may use different metered units. Some models also use versioned billing expressions for features that cannot be represented by one flat token rate.

Use Models & Pricing for the current model and API-format contract instead of copying a price into application code.

2. Measured request usage

The gateway records the usage returned or derived for the completed request. For token-priced text models, input and output are kept separate because their rates may differ.

Streaming text visible in a terminal is not a reliable cost oracle. Tool items, reasoning, cached input, normalized tokens, or provider-specific usage fields may affect the bill even when they are not displayed as ordinary assistant text.

3. Model-group multiplier

The selected routing group can apply a sales multiplier to base usage billing. For example, a group ratio of 0.9 applies 90% of the base model charge for that request.

The multiplier belongs to usage billing. It does not change how much wallet credit a top-up adds, and it should not be confused with an upstream supplier-cost multiplier.

4. The final request record

The usage log connects the calculated amount to the request's model, group, status, token usage, timing, and other safe operational metadata. Failed or cancelled requests can follow different settlement paths, so the persisted result is more authoritative than a client-side estimate.

What to compare in usage logs

When spend changes, compare requests along these dimensions:

• Model: Did traffic move to a model with a different input, output, or feature price?
• Group: Did the selected group or fallback group use a different multiplier?
• Protocol: Did the request move between Chat Completions and Responses with a different usage shape?
• Input size: Did prompts, retrieved context, files, or tool results grow?
• Output size: Did completion limits or agent loops produce more output?
• Status and retries: Did failures cause additional completed attempts?
• Timing: Did longer generation correlate with more output rather than just queueing?

Use a stable request ID or application-specific API key to isolate a workload instead of comparing unrelated account traffic.

Set practical cost boundaries

Separate API keys by workload

Create different keys for production, development, automation, and personal tools. Each key can have its own name, expiration, group policy, and finite or unlimited quota.

Choose groups intentionally

Do not select a group only by its label. Check live model availability, the group multiplier, access requirement, RPM, and fallback policy. A cheaper primary group with an acceptable fallback can be more predictable than an implicit routing choice that is difficult to explain.

Limit input before limiting output

Large system prompts, repeated conversation history, retrieved documents, and tool results often dominate input usage. Trim context that no longer affects the answer, and avoid resending unchanged data when the client can safely reference or cache it.

Review agent loops

One user action can create several model requests. Track each tool round and retry rather than treating the final visible answer as one API call.

Why client-side estimates drift

A local tokenizer or character count can be useful for planning, but it may differ from billed usage because:

• providers can normalize or count content differently;
• cached, reasoning, image, audio, or tool units may use separate rates;
• gateway settlement uses the actual selected model and group;
• failed requests and refunds depend on the real request lifecycle;
• pricing can change while old requests retain their recorded result.

For financial review, reconcile the persisted usage and wallet records rather than deriving account balance from one display field.

A repeatable review process

1. Filter usage logs to one API key and time range.
2. Group requests by model and selected group.
3. Compare input, output, status, and cost per request.
4. Inspect outliers for retries, long context, tool loops, or fallback.
5. Confirm the live price and group policy before changing routing.
6. Set a finite key quota when the workload needs a hard spending boundary.
7. Recheck the same metrics after the change.

Clear cost control starts with attribution. When model, group, usage, and request outcome remain connected, teams can optimize the actual source of spend instead of guessing from aggregate totals.