How we track our AI margin per customer with OpenRouter

TL;DR: We route LLM calls from four providers through OpenRouter presets, one per component, so switching models is a configuration change. Every call carries the customer in the user field and the component and target in trace metadata. OpenRouter’s BigQuery Broadcast export gives us one row per trace, which tells us who consumed the spend; the Analytics API tells us how much we were billed. Costory splits the billed amount using the traces, so we know the AI cost of every customer and feature.

At Costory, our cost tagging was choosing our models for us. We kept most production LLM traffic on Gemini in Vertex AI, not because it was always the best model, but because Vertex is today the only provider that allows to have tags at the LLM call level. Moving behind OpenRouter let us keep that cost per customer while making the model replaceable.

That visibility matters because of how we price. Our base plan is €250 a month and the most expensive tier is €1,200, with AI usage included up to each plan’s limits. Our LLM bill is not enormous, but at those price points a customer we need to monitor it effectively.

Over time, that trade-off became frustrating. We received startup credits from Anthropic, OpenAI and OpenRouter. New open-weight models such as GLM became genuinely interesting for some of our workloads. We could test all of them, but moving a production workload away from Vertex meant giving up the customer-level cost allocation we relied on.

There was no dramatic cost incident, just a constraint we no longer wanted. This is how we set up OpenRouter, where the first version fell short, and how the BigQuery trace export eventually gave us the detail we needed and we productified it for our customers.

One thing to know up front: the trace data is used only for cost attribution, never as the bill itself. Billed amounts come from the provider invoices, GCP and Anthropic for direct traffic and OpenRouter’s billed usage for routed traffic, and those invoices stay the source of truth.

The cost allocation metadata we wanted on every LLM call

We already had the context in the application. We just needed it to survive the trip to the cost data.

domain is the important one for customer allocation. node_name is the useful engineering view. The target fields are there for investigations: if a report suddenly burns through tokens one day, we want its ID rather than a broad “reports” bucket. We have not needed that yet, but recording the ID costs nothing and recovering it after the fact is impossible.

Why we tried OpenRouter presets

OpenRouter gave us startup credits, so we tried it. That is the honest reason the experiment happened when it did. The migration only stuck because it solved the larger problem: one inference interface, access to models from several vendors, and enough metadata to recover the allocation we had on Vertex.

We created an OpenRouter preset for each node_name. The code asks for @preset/report-explanation; the preset holds the model, provider routing, fallbacks and generation parameters.

{
  "model": "@preset/report-explanation",
  "messages": [
    {
      "role": "user",
      "content": "Explain the cost change in this report."
    }
  ]
}

{
  "model": "@preset/report-explanation",
  "messages": [
    {
      "role": "user",
      "content": "Explain the cost change in this report."
    }
  ]
}

{
  "model": "@preset/report-explanation",
  "messages": [
    {
      "role": "user",
      "content": "Explain the cost change in this report."
    }
  ]
}

This paid off sooner than expected. When a GLM model started performing well on several of our use cases, we ran it against representative examples in PostHog, then changed the relevant presets. No caller changed and there was nothing to redeploy. What used to be a small migration became a configuration change.

GLM started as a development-only experiment. It now runs our medium tasks, description generation and the various suggestion features, while investigations stay on stronger models. The presets made that a gradual promotion: switch the preset, keep the option to switch back.

The same routing now powers a product feature. When a customer reaches 95% of their plan’s AI credits, their Slack questions fall back to cheaper models instead of cutting them off entirely. The running total comes straight from OpenRouter, which returns the cost of each conversation as it happens, charged against the customer’s plan AI credits.

First attempt: OpenRouter cost per customer with the user field

For the first version, we worked with the dimensions OpenRouter exposed in Analytics. We put our customer domain in the request’s user field and used OpenRouter app attribution for the high-level application. App attribution comes from the HTTP-Referer and X-OpenRouter-Title headers.

{
  "headers": {
    "HTTP-Referer": "https://agents.internal.costory.io",
    "X-OpenRouter-Title": "agents"
  },
  "body": {
    "model": "@preset/analysis-agent",
    "user": "acme",
    "messages": [
      {
        "role": "user",
        "content": "Find the main drivers of this cost increase."
      }
    ]
  }
}

{
  "headers": {
    "HTTP-Referer": "https://agents.internal.costory.io",
    "X-OpenRouter-Title": "agents"
  },
  "body": {
    "model": "@preset/analysis-agent",
    "user": "acme",
    "messages": [
      {
        "role": "user",
        "content": "Find the main drivers of this cost increase."
      }
    ]
  }
}

{
  "headers": {
    "HTTP-Referer": "https://agents.internal.costory.io",
    "X-OpenRouter-Title": "agents"
  },
  "body": {
    "model": "@preset/analysis-agent",
    "user": "acme",
    "messages": [
      {
        "role": "user",
        "content": "Find the main drivers of this cost increase."
      }
    ]
  }
}

Both dimensions were available through the OpenRouter Analytics API, so we could ingest them into Costory easily. This got the customer split back, which was already a big improvement. But node_name, target and target_id were still missing.

Second attempt: trace metadata for cost per component and target

We started sending the remaining context as custom trace metadata:

{
  "model": "@preset/report-explanation",
  "user": "acme",
  "session_id": "report_01K8Y5T2",
  "trace": {
    "node_name": "report_explanation",
    "app": "app",
    "target": "report",
    "target_id": "report_01K8Y5T2"
  },
  "messages": [
    {
      "role": "user",
      "content": "Explain why database cost increased this week."
    }
  ]
}

{
  "model": "@preset/report-explanation",
  "user": "acme",
  "session_id": "report_01K8Y5T2",
  "trace": {
    "node_name": "report_explanation",
    "app": "app",
    "target": "report",
    "target_id": "report_01K8Y5T2"
  },
  "messages": [
    {
      "role": "user",
      "content": "Explain why database cost increased this week."
    }
  ]
}

{
  "model": "@preset/report-explanation",
  "user": "acme",
  "session_id": "report_01K8Y5T2",
  "trace": {
    "node_name": "report_explanation",
    "app": "app",
    "target": "report",
    "target_id": "report_01K8Y5T2"
  },
  "messages": [
    {
      "role": "user",
      "content": "Explain why database cost increased this week."
    }
  ]
}

The repeated ID is deliberate: for report explanations, one session is one report run, so session_id groups the calls and target_id names the resource they were for.

OpenRouter accepts arbitrary values in the trace object. The awkward part was that these values did not appear in the aggregate Analytics export we were using. We could see them as trace context, but could not yet use them to allocate the bill.

OpenRouter BigQuery Broadcast: one row per trace

The missing piece arrived with OpenRouter’s BigQuery Broadcast integration. It writes one row per trace directly to BigQuery.

Model, token count and total cost are regular columns. Values from the trace object land in a JSON metadata column, while user and session_id are pulled out into typed columns. There is no deduplication or elaborate modelling step required: one trace is one row.

Our allocation table is a simple daily aggregation of that export: cost, tokens and requests per customer, component, app, target and model.

Once this was in BigQuery, the cost data became useful outside the monthly bill. We could ask which reports generated the most spend, compare title-generation cost before and after a model change, or separate ingestion traffic from interactive chat. If a component jumped in cost, we could get down to the customer and resource that caused it.

Reconciling OpenRouter traces with the billed cost

It is tempting to treat the trace table as the bill. We do not. It is excellent allocation data, but the Analytics API remains our control total for billed OpenRouter spend.

Costory ingests credits_usage, including the platform fees charged on BYOK traffic (bring your own key: OpenRouter routes the call through your own provider account, and the provider bills you directly). We deliberately do not add the upstream byok_usage estimate to billed cost. If AWS, GCP or Azure billing is connected too, the actual inference charge already appears there; adding OpenRouter’s estimate would count it twice.

If you only need spend by model and API key, Costory’s OpenRouter connector is enough. It uses a Management API key, imports daily credits_usage, and handles the BYOK distinction above.

For the deeper attribution described here, the pipeline uses the Analytics API to establish how much we spent and the BigQuery traces to work out who consumed it, following the same per-customer pattern as Costory’s LLM cost allocation playbook. We expose the trace aggregation as a BigQuery table and configure it as a BigQuery usage metric. Costory can use that metric in a telemetry-based virtual dimension or a dynamic enrichment.

If customer A generated 70% of a day’s trace cost and customer B generated 30%, the billed OpenRouter cost follows the same split. The metric carries customer, app, component and target together, so we do not end up allocating each dimension independently and producing combinations that never existed.

Any remainder stays unallocated. That remainder is useful: it exposes missing metadata, late traces and export problems. The result is one cost model across direct LLM APIs, OpenRouter, Vertex AI and cloud-provider invoices.

Was the migration worth it?

The obvious win is model routing. Presets let us test a model and move selected workloads without touching each caller. The less obvious win is that we now have a consistent place to attach context before traffic fans out to different providers.

There are still three separate systems involved, but each has a clear job. OpenRouter presets choose the model. Trace metadata records why the call happened. Costory reconciles that usage with the amount actually billed.

If you already use OpenRouter, give Costory a try. Connecting the OpenRouter billing data with a Management API key takes about five minutes. You can start with spend by model and API key, then add the BigQuery trace allocation when you need customer- or feature-level visibility. Costory starts at a flat €250 per month for up to $10 million in annual cloud and AI spend.