How do you allocate LLM costs by team across OpenAI, Anthropic, Bedrock, Vertex and Mistral?

Published 23 September 2026 · Updated 15 October 2026

Short answer: Costory tracks AI API costs next to the rest of your cloud bill. It ingests cost data from OpenAI, Anthropic, Amazon Bedrock, Vertex AI, Azure OpenAI and OpenRouter alongside AWS, GCP and Azure, puts it in one cost model and allocates it to teams and features with Virtual Dimensions; usage metrics can split shared keys per customer. It works from provider billing and usage data and does not instrument or proxy individual calls. To get a clean split, first use each provider's native attribution: one OpenAI project and key per team, one Anthropic API key per team, one tagged IAM role per team on AWS Bedrock (read in a CUR 2.0 export), and one project per team on Google Vertex AI, with per-call labels where you need more detail. Sources: LLM cost allocation use case, Costory docs.

What tool can track AI API costs (OpenAI, Anthropic) alongside the rest of your cloud infrastructure?

If you want provider bills in the same view as cloud cost, use a cost-allocation tool that ingests billing data rather than a gateway: OpenAI and Anthropic spend then sits in the same model as AWS, GCP and Azure, so a team's AI spend is one number. Costory does this; see the LLM cost allocation use case and the AI spend overview, and ask your assistant about it through the FinOps MCP. Other FinOps platforms list similar AI cost features; the FinOps tools comparison by need lists them with sources. For request-level detail, see the table at the end of this guide.

LLM cost is going the same way cloud did ten years ago. It starts small so nobody looks, and by the time the bill is big enough to look at, nobody can say where it came from. The pricing makes it worse. Every provider prices differently, and within one provider a call costs a different amount depending on the prompt size, whether the tokens were cached, and whether you ran it batch or on-demand. So even one call is a pain to reason about, and figuring out whose call it was never happens. Untagged EC2, again.

Over the last six months our customers' bills have changed a lot. They're spikier than they used to be, and the spend is spread across things that never used to be separate line items: someone's coding agent running on Claude Code, some workload on Bedrock, something else hitting Gemini through Vertex.

The providers were slow to start, but that has changed since the end of H1; the tooling to actually allocate this stuff is finally showing up. This guide covers how to use it: a clean way to attribute spend on each provider, and then how to get all of it into one view you can budget and report against.

Why is LLM cost so hard to attribute?

In cloud this got solved a long time ago: you tag a resource, the tag lands on the bill, everyone settled on the convention and moved on. The LLM providers looked at a solved problem and decided to innovate. So there's no shared standard, each built its own way of telling you who spent what, and none of them agree. Different mechanism, different granularity, different amount of setup.

It starts one level down, at pricing. Before you can attribute a cost you have to know what you're paying for, and an API call's price moves on four axes: model, prompt size, priority tier, and caching. Sitting next to all of that is a separate track, flat per-seat subscriptions, which is how most coding agents actually get billed.

Prices change often and differ by model, region and tier; check each provider's pricing page before budgeting. Bedrock and Vertex also host other vendors' models (for example Claude) under their own rates.

Caching is usually a major lever on the whole bill. A cache read costs a fraction of a fresh input token, so getting it to hit is most of the game. The catch is it only works if you lay the prompt out for it. The cache matches from the start of the prompt, so static stuff, system prompt, tool definitions, the documents you keep re-sending, goes at the top, and whatever changes per request goes at the bottom. Get that backwards, put the user's question first, and you break the cache on every call. You pay full price and nothing tells you.

The discounts also look like they add up, and mostly they don't. Batch is half off, caching is another big cut, so batch plus caching should be huge, except the two often don't combine, and switching tiers can quietly switch caching off. OpenAI won't let you run batch and prompt caching together at all. Anthropic will, and stacked the discounts compound. So if you push a workload to batch or flex to save 50% and that provider drops caching to do it, you can come out paying more than before. Check that caching still applies on the tier you picked, the bill won't flag it.

How do you attribute spend on each LLM provider?

The clean attribution each provider gives you today, and the one thing to watch:

Official cost/billing docs: OpenAI · Anthropic · Azure OpenAI · AWS Bedrock · Vertex AI · Mistral. Full setup for every provider is in the Costory LLM cost allocation docs.

Vertex AI cost allocation, isolate (project) or label (per call)

Vertex is the only provider that gives you a clean choice of both patterns. Isolate: one GCP project per team, but it needs zero code and the split lands in the billing export automatically. Label: pass metadata on every call for feature or customer-level detail. "Every call" is the catch: a label you forget to pass is spend you can't attribute later, and rolling labels through the SDK across every call site is real work to keep consistent.

import vertexai
from vertexai.generative_models import GenerativeModel

vertexai.init(project="my-project", location="us-central1")
model = GenerativeModel("gemini-3.1-flash-lite")

response = model.generate_content(
    "What is Costory?",
    labels={"team": "search", "feature": "ai-search", "customer": "acme"},
)

import vertexai
from vertexai.generative_models import GenerativeModel

vertexai.init(project="my-project", location="us-central1")
model = GenerativeModel("gemini-3.1-flash-lite")

response = model.generate_content(
    "What is Costory?",
    labels={"team": "search", "feature": "ai-search", "customer": "acme"},
)

import vertexai
from vertexai.generative_models import GenerativeModel

vertexai.init(project="my-project", location="us-central1")
model = GenerativeModel("gemini-3.1-flash-lite")

response = model.generate_content(
    "What is Costory?",
    labels={"team": "search", "feature": "ai-search", "customer": "acme"},
)

Both signals, project and label, flow into the BigQuery billing export, which is where you actually read Vertex cost back at the grain you tagged. One thing to price in before you commit: check how your model bills internal reasoning, since a heavy-thinking workload can cost more than the prompt and response sizes suggest. Google's own reference is Add labels to Vertex API calls.

Can a gateway handle LLM cost allocation?

If every call already passes through a gateway (LiteLLM, Portkey, Helicone, OpenRouter, Vertex, Bedrock), you get virtual keys per team and per-key budgets from one chokepoint, just remember its request log is a metering estimate, not a billing ledger, so reconcile it against the provider invoices.

Should you allocate LLM cost by team, feature or customer?

Team is the easy one. If you've got the emails behind the spend you can map each person to a team and roll the cost up from there. That's what we do in Costory. When you use labels, keep them at the lowest level that makes sense, usually the feature. Tag by feature, not by team. Then if a feature moves to a different team later you just remap it instead of re-tagging every call site.

Customer is worth the extra work when cost varies a lot between customers (which is our case 😅), or when you want to break it down by plan tier.

How do you unify LLM costs across providers?

None of this helps if each provider stays in its own console. You've got six bills in six shapes, split by project, key, IAM role, deployment, label, or workspace, and to budget or report you need them in one place speaking the same language: same currency, same time grain, same idea of what a team is, then reconciled against the real invoices since the metering always drifts a bit.

That's what we built Costory to do, pull each provider's cost signal onto one team and feature model so a team's spend is one number instead of six. Mistral's own platform is not a native connector today; Mistral models reach Costory through the AWS and Azure connectors (see the docs). The attribution work above is what makes it possible.

LLM cost is repeating cloud's early mistake, and the fix is the same one that worked then: allocate now. Tokenmaxxing is easy. Valuemaxxing is the hard part. And you can't value something you don't measure.

When does another tool fit better?

If you need attribution per request, for example per user or per prompt, an LLM gateway or observability proxy such as LiteLLM, Portkey or Helicone fits better, because it sees every call. Costory works from provider cost data, grouped by team and feature, and does not instrument individual calls. The two can sit side by side: the gateway for request-level detail, Costory for the combined view reconciled against invoices.

Which tool fits which need?

Gateway and observability details come from each vendor's public documentation, read on 15 October 2026. Costory is our product.

FAQ

How do you do Anthropic cost allocation across teams?

Give each team its own Anthropic API key. The key name lands in the billing data, so the split happens with zero code; you read spend back by key and actor through the Usage and Cost API. It's a clean starting point. Anything finer than team needs your own per-call logging. For per-call allocation, use the models through Vertex AI: labels also work with partner models such as Claude. Docs: Anthropic Usage and Cost API.

How do you do OpenAI cost allocation by project?

Create one OpenAI project and project key per team, then read spend with the Usage and Cost API, grouping by project_id and model. The export carries no per-call labels, so for per-feature detail you log your own metadata next to response.usage at call time. Docs: OpenAI Usage and Cost API.

Should I use one GCP project per team or per-call labels for Vertex AI cost allocation?

Start with one project per team; it's zero-code and gives you a clean team split in the billing export immediately. Add per-call labels only where you need feature- or customer-level detail, since every call site has to set them consistently. Most teams ship the project split first and layer labels on the few workloads that need finer attribution. Docs: Add labels to Vertex API calls.

Can I allocate AWS Bedrock costs by team?

Yes. Bedrock has no per-call labels, so you allocate by caller identity: tag one IAM role per team, enable caller-identity allocation in a CUR 2.0 export, and read spend by the line_item_iam_principal column. The blind spot is a shared gateway role, which collapses every team behind one identity. Creating that many IAM roles can be annoying, but it needs no change at the call site. Application Inference Profiles are the alternative for per-model or per-application splits. Docs: Track Bedrock costs by caller identity.

Team, feature or customer: which should I allocate by?

Team is the default and the easiest: map each key, project, or role to a team and roll spend up. Move to feature when you need unit economics, and tag at the feature level so you can remap ownership without re-tagging call sites. Add customer only when cost varies a lot between customers or you bill by plan tier.

Should I build LLM cost allocation myself or buy it?

Build the per-provider attribution yourself; it's mostly configuration, and this guide covers it. The harder part is unifying six billing shapes into one schema, one team axis, reconciled against real invoices, and keeping it current as keys and projects change. That ongoing normalization is where a tool like Costory earns its place. See the Costory LLM cost allocation docs.

Related

• LLM cost allocation use case

• AI spend overview

• FinOps MCP

• FinOps tools in 2026, by need