Indexed monthly AI cost: the same workloads, at a fraction of metered API rates. Two thirds of that came from subscription plans absorbing our frontier-model work, and the remaining third from routing everything else onto cheaper and self-hosted models, which is what kept the plans from filling up.
Our AI usage grew faster than our ability to manage it. The bill rose every month, but it told us almost nothing about which teams, models, or projects were responsible. Meanwhile, every new model brought another round of keys, configuration changes, and support.
The underlying problem was simple: our tools talked to providers independently, so there was nowhere to apply policy or collect a complete picture of usage.
We built Blade Code to put that control point in the middle. It is an internal LLM proxy, built and run at Sword, that handles AI requests across our coding agents, teams, and providers. This is how we built it, including the parts we got wrong, and how we used it to reduce costs by 94% while attributing spend down to the feature branch.
The whole system on one page: every AI request in the company takes the same path. One gateway, five stages, any provider behind a uniform adapter.
The constraint
We set one constraint before writing a line of code: zero friction for engineers.
Any system that makes developers ask permission, wait for access, or edit a config file to try a new model will be routed around. It will also be resented. Whatever we built had to make the governed path the easiest path, and it had to let them try a brand-new model the day it launched, not the sprint after.
That ruled out the usual answers: a checkout process for API keys, a spreadsheet of approved models, or a monthly access review. Each one adds enough friction that some usage inevitably moves outside the system. We wanted developers to get governance, budgets, and cost attribution simply by using the fastest path to a model. The rest of the design followed from that constraint.
Providers and models
We started with the foundation: a registry of providers and the models we expose. Today that's Anthropic, OpenAI, OpenRouter, and Nebius Token Factory, alongside our own self-hosted endpoints, which also run on Nebius. The important part is that this registry is runtime configuration, not code. Adding a provider or a model (or pointing an alias at one) is a backoffice edit that the live registry reloads atomically, routable company-wide on the very next request. No deploy, no restart, no per-developer setup.
One API surface, many protocols
Every request in the company enters through a single gateway that speaks two wire formats:
- OpenAI Chat Completions (POST /v1/chat/completions) — what most of our tools speak
- Anthropic Messages (POST /v1/messages) — what Claude Code speaks natively
We support both because our tools are split across them. From there, a request takes one of two paths.
When the request and its target provider already share a dialect, say Claude Code hitting an Anthropic model, the gateway passes it through near-verbatim, streaming bytes included, so nothing native is lost in a round trip: cache-control hints, extended-thinking blocks, and tool calls all survive.
When they don't, we normalize the request into a single canonical form, internally the OpenAI shape, and translate it out to the target's dialect: message roles, system prompts, tool-call formats, stop sequences, reasoning parameters. The response is translated back the same way. Streaming is de-framed and re-emitted event by event rather than buffered, so tokens arrive for the developer exactly as they would hitting the provider directly.
A request takes one of two paths through the gateway: the fast passthrough lane when the dialects match, and the translate lane when they don't.
A provider is an adapter behind a fixed contract
Every provider implements the same five-method interface:
- Build the upstream request
- Translate the response back
- Extract token usage
- Translate each streaming event
- Finalize the stream
That is the entire extension point. OpenRouter and Nebius both speak the OpenAI Chat Completions format, so adding them meant configuring new instances of an adapter we already had. We only write a new adapter when a provider introduces a genuinely different protocol. Without that fixed contract, a runtime registry would only move provider-specific code around; with it, a configured provider can actually go live without a release.
Model metadata, priced in integer micro-dollars
Every registered model carries its owning provider and its price per million tokens, broken out by input, cached input, output, and reasoning. All of it is stored and computed as integer micro-dollars, millionths of a dollar, and money never touches a floating-point number until it's rendered at the API response boundary. Rounding drift across millions of requests is precisely the error you don't catch until reconciliation.
Every provider's usage response, however it's shaped on the wire, is normalized into one accounting record: prompt, cached, completion, and reasoning tokens, with a fixed subset relationship between them so reasoning is never double-billed. That normalized record is what the cost analysis further down turns into dollars.
Credentials never leave the gateway
Provider API keys live server-side only. Developers authenticate with a scoped Blade Code key that is hashed at rest and carries their identity and team. Every key is a team key.
Teams, access, and budgets
A gateway that anyone can point anything at is a proxy, not a control plane. The difference is policy, and policy needs an identity to attach to. So the second thing we built was the team model.
Every request carries a team identity
There are no personal keys and no un-teamed usage. Each account belongs to one team, every key a developer holds is a team key, and the gateway rejects anything else at the door. Identity and team therefore travel with the request. Budgets and model access can be enforced before it reaches a provider, and attribution does not have to be reconstructed later from an invoice.
A ceiling admins set, a switch owners flip
We deliberately split model access into two tiers instead of one list:
- The admin defines the ceiling — which providers and models a team is permitted to touch at all
- The owner flips the switches — toggling individual providers and models on and off within that ceiling, without an admin round-trip
The effective policy is the intersection: a model is live for a team only if it is permitted and switched on. A team lead can turn an expensive reasoning model off for a sprint and restore it later without filing a ticket, but cannot grant access to something the organization has not sanctioned. This also lets teams make different tradeoffs. A latency-sensitive product team may keep a tight set of models while a research team opens theirs wider. An empty model list means every registered model from the team's enabled providers is available.
Routing, decided per member
The same team can hold a developer who wants the gateway to pick the provider automatically and one who wants every request pinned to a specific model. Rather than bolt on flags, we made routing a single explicit choice per person:
Each member picks one mode. Two follow the client's intent; two override every request.
The database enforces that exactly one target is ever set, so a member can never be in a contradictory state. This is what lets a data scientist run Fully Automatic while the person next to them pins everything to a self-hosted model: same team, same key, different routing, and no config file between them.
Budgets: a feedback loop, not a gate
Every team has a monthly ceiling; every member can have a tighter personal one; the effective limit is whichever binds first. We spend real engineering effort keeping this cheap to check: spend is cached in-process and nudged upward the moment a batch of requests is written, so budget pressure is felt within seconds rather than at the next cache expiry. A runaway agent loop can't quietly burn a month's budget in the window between reads. And like everything else in the system, budgets are integer math end to end: caps in cents, spend in millionths of a dollar, compared without ever touching a float.
Enforcement is the backstop, not the first purpose of a budget. Most engineers do not know whether their AI usage is unusual until they can see it. We show the number continuously and block only when it reaches the limit.
Cost analysis
With every request flowing through a single proxy, we could finally see token consumption per team and per member, and price it against the model actually used. Getting a number you can trust, one that reconciles with the provider invoice at the end of the month, turned out to be the hard part, and it's where most of the accounting engineering went.
Not all cost is spend
Two very different things both look like "cost" on the way in:
- Billed — a dollar routed through the org's own provider key is real, invoiced money
- Plan-Covered — a dollar's worth of tokens absorbed by a developer's own subscription plan is value delivered at zero marginal cost; it never appears on any invoice
So every row records who paid, and the dashboards report the two as separate figures. Nobody mistakes plan-covered value for money out the door.
Of $155,624 in delivered value, only $7,631 was billed to the org's own keys. The rest was absorbed by developers' own subscriptions: real value delivered at zero marginal cost, and it never touched an invoice.
The axis that can't see a third of the answer
That chart is the one we misread for a while, and the mistake is easy to make. It answers who paid, and it answers it accurately, while being structurally incapable of showing the other way a request gets cheap.
When a request runs on a cheaper or self-hosted model, nothing moves into the plan-covered column. It's still billed to the org's own key; there is simply far less of it. Routing savings shrink the total the chart is drawn from, and never appear as a slice inside it. So the ranking here is real — subscriptions genuinely are the larger track — but a third of what we saved had already left before any of this was priced, and no amount of staring at this axis will surface it. The cheapest token is the one you never sent to an expensive model.
The monthly total was less useful than the distribution underneath it. A few teams and models accounted for far more of the cost than anyone had assumed.
The same month, ranked two ways. Both distributions have a steep head and a long tail, and neither is visible in a monthly total, which is the only number anyone had before.
Both rankings come from the same request records, because each record contains the team and the model that actually ran. We were not comparing two estimates built by different systems. The discussion could move directly to which teams to migrate first and where an expensive default model was doing work a cheaper one could handle.
But we still had two gaps. We could only price the requests that had found their way to us in the first place. And of those, we knew who was spending and which model they were spending it on. We didn't know what they were building.
The desktop app
Blade Code puts routing, usage, activity, and proxy health in one desktop dashboard.
The server-side system only works for traffic that reaches it. We built a small menu-bar app to close that gap. It is installed through device management, can route the whole machine instead of configuring tools one at a time, and attaches the repository context needed for feature-level attribution.
Nobody installs it
The app is pushed to every laptop by our own device management, so it is already sitting in the menu bar before anyone goes looking for it. An engineer joining Sword doesn't find a download page, a setup guide, or an internal doc pinned to a version that went stale two releases ago. They find an icon, sign in with their work account, and flip one toggle.
We chose managed distribution because availability is not adoption. Engineers rarely object to internal infrastructure; installing it is simply less urgent than their current work. Pushing the app makes coverage the default on day one instead of the eventual result of a rollout campaign.
Daemon Proxy
One switch routes supported provider traffic across the machine, with live proxy health visible in the app.
The obvious way to point a coding agent at a gateway is to write that agent's own config for it, and that is what we built first: one toggle per tool (Claude Code, Codex, Open Code, Pi), each minting a scoped key, editing the client's own configuration format on the developer's behalf, and reverting it cleanly on the way out. It works, and then it leaks. Those toggles only cover tools we have written an integration for, and only tools that have a base-URL override; plenty of desktop AI apps simply don't. And they're a setting, which means they drift: someone reinstalls a tool or restores a dotfile from a backup, and their requests quietly go back to talking to the provider directly. Each of those is small. Together they are the same shadow usage we built the gateway to eliminate. So we stopped configuring tools and started governing machines.
The daemon captures the providers we support, and nothing else. A small local service runs on the laptop holding an allowlist of provider endpoints and impersonating them. Any application on that machine that talks to one of them is transparently routed through the gateway and re-keyed with the developer's scoped team key. The developer sets no base URL, pastes no token, and needs no integration written for that specific tool. Every other connection the laptop makes flows exactly as it did before. Streamed responses are relayed as they arrive with no buffering anywhere in the path, so an agent thinking token by token feels identical to talking to the provider directly.
One switch performs the setup. It mints a machine-scoped team key, generates and trusts the local certificate authority, registers the daemon to start at login, and disables any conflicting per-tool toggles. After that, staying on the governed path requires no further action. A newly installed agent is covered without waiting for us to add a specific integration.
The honest part: this means terminating TLS. An encrypted connection cannot be redirected to a server that does not own the provider hostname without being decrypted. Transparent interception therefore requires a locally generated root certificate authority in the machine's trust store.
We limited that authority to the provider hostnames on our allowlist. Other connections, including banking, email, and private repositories, pass through as raw bytes; the daemon reads only the server name needed to decide that the connection is out of scope. The authority is generated for one laptop, never shared, and removed when the feature is disabled. The daemon also strips the developer's provider credential locally and replaces it with the scoped Blade Code key before forwarding the request.
We spent more time debating this than any other part of the system. TLS interception creates a real security boundary, so we treated it as one and kept its scope narrow. We accepted it because the alternative for unsupported applications was continued direct access to providers, with no policy or attribution.
Fail open, always. Our first version violated this rule. The daemon ran as a child of the menu-bar app, while the capture settings survived reboots. If the app stopped, the laptop continued sending provider traffic to a port where nothing was listening. In some configurations that affected more than the AI client.
The daemon now runs as an operating-system service that starts at login and outlives the app. Proxy-aware clients receive a direct fallback, and a daemon that repeatedly fails removes its own capture configuration even when the app is closed. The tray and dashboard show whether it is healthy, degraded, or off.
The acceptable failure is a request that bypasses Blade Code temporarily, loses attribution, and costs the normal provider rate. Blocking the developer's network is not.
Every connection on the laptop takes one of three paths. Only the first is ever decrypted. And if the daemon can't do its job, the third one is automatic.
Context recomputed per prompt, sent as headers
Repository context must travel with the request. The same switch that enables the proxy installs a shell hook that resolves the working directory, branch, remote, and repository before every prompt. It passes that context to the tool as proxy headers:
X-Proxy-Git-Repo: checkout-service
X-Proxy-Git-Branch: feature/streaming-costs
X-Proxy-Workspace: /home/dev/src/checkout-serviceWe originally resolved this context when the shell opened. That produced bad attribution as soon as someone changed directories or branches inside a long-lived terminal: later requests kept the repository that happened to be open that morning. Recomputing the values before each prompt fixed it.
Each tool carries the headers differently. Claude Code and Open Code use custom-header settings, Codex uses its provider configuration, and Pi resolves the values at request time. The gateway handles them the same way and adds the repository and branch to the accounting record it already writes.
Applications that never went through a shell, the GUI ones the daemon exists to capture, are covered from the other end. When an intercepted connection carries none of those headers, the daemon maps the socket back to the process that opened it, reads that process's working directory, and walks the git checkout around it to recover the same branch, remote, and repository name. It fills in only what's missing (a header the client sent explicitly always wins) and leaves blank anything it can't determine. Attribution degrades to an empty field, never to a wrong one.
The whole attribution path: context computed per prompt, carried as headers, stamped on the row, and rolled up into per-feature cost.
"We spent $X on AI last month" is an accounting fact. "This feature cost $Y in tokens to build" is an engineering decision input. It tells you which workflows are worth optimizing, which agentic loops are quietly burning budget, and where a cheaper model would perform identically. You can't make those calls without attribution — and attribution is four headers and a place to put them.
What it actually bought us
The support threads stopped. That's the win we'd point at, and it's the one we never thought to measure: onboarding an engineer to the whole governed stack is now a single switch on their first day, on a laptop where the app was already waiting for them.
The migration that saved 94%
By this point we had flexibility, governance, and analytics. We also had something rarer: a clear, well-instrumented picture of how much we were overpaying at standard API rates, and where. The cost analysis and per-feature attribution didn't just report the problem; they told us which workloads to move and what each move would save before we made it.
So we moved everyone off pay-as-you-go, along two tracks. The subscription track carried about two thirds of the reduction, which is the answer most people guess. The part worth being precise about is the other third, because the smaller track is the one that made the bigger one possible.
Measured against what the same month's work would have cost at frontier pay-as-you-go rates, two thirds of the reduction came from subscription plans and one third from routing work onto cheaper and self-hosted models.
Track one: route the work to the cheapest model that can do it
This is the smaller lever and the least glamorous one, and it had to come first: a great deal of what our agents do does not need a frontier model, and until we could see the breakdown, all of it was running on one anyway.
The by-model ranking earlier in this post is what made the case. Once you can see that the most expensive model in the registry is carrying the plurality of spend, the question stops being "how do we pay less for it" and becomes "how much of this actually needed it." The answer, repeatedly, was: less than we thought. Routine work (summarizing, classification, mechanical refactors, the high-volume inner loops of agentic workflows) moved onto smaller models and onto open-weight models we host ourselves, GLM and Kimi.
The arithmetic is not subtle. A top-tier frontier model in our registry is priced at $15 per million input tokens and $75 per million output; the GLM deployment we route this work to is $1.40 and $4.40. That's roughly ten times cheaper on input and seventeen times on output for the same tokens. Reserved capacity is most valuable when it stays busy. We run that self-hosted capacity on Nebius, and the partnership has made it straightforward to distribute inference traffic across our GPUs. We already pay for those GPUs whether they are serving requests or sitting idle, so whenever they have spare capacity, we route suitable coding workloads to the models running on them. That turns capacity we have already bought into useful inference, avoids metered calls elsewhere, and keeps the GPUs as fully utilized as demand allows.
Self-hosting also wins on a second axis that has nothing to do with money. Teams handling sensitive information need the data to stay inside our environment, and an open-weight model on our own infrastructure is the only version of that which is actually true. That requirement is what got the self-hosted providers built in the first place; the cost result was the larger surprise.
None of this needed new plumbing. These are providers in the registry like any other, behind the same adapter contract, so moving a team's work onto them is a policy edit: change which providers and models the team has switched on, or pin a member's routing at a specific model, and the next request goes somewhere cheaper. Nobody's laptop changed.
Track two: subscription plans for the frontier work that remains
The work that genuinely does need a frontier model still needed paying for, and that's where subscriptions come in. The proxy makes the subscription invisible as a setup step. Engineers sign in to their coding tools as usual; when an eligible request passes through the local proxy, Blade Code carries that existing authenticated session into the gateway and routes the request against the developer's plan. It follows the same routing, logging, and attribution path as any other request, but consumes flat-rate subscription capacity instead of generating a metered API charge.
On the dashboards those requests show up as Plan-Covered: we can still see their full API-rate value, we just never pay it. A single subscription can absorb a volume of tokens that would cost many times its monthly fee at metered rates, and for members opted into cost-saving routing the gateway restricts them to their connected subscription providers, reaching for a billable key only when a plan is exhausted.
The two tracks are multiplicative, not additive
A two-thirds/one-third split reads like a big win and a small one, with the small one optional. It's closer to the reverse. The tracks aren't independent programs whose savings you add up, and the smaller track is what makes the larger one's number achievable at all.
A subscription plan is a capacity, not a discount: it absorbs a bounded amount of frontier-model work per month, and past that boundary you are back on metered rates. Had we left the routine high-volume work on frontier models, it would have consumed those plans within days and everything after would have been billed at list. Track one is what kept the frontier-model share small enough for track two's plans to swallow it whole. Run either alone and you get a fraction of the result; the 94% is what happens when the expensive work is the only work left on the expensive path.
The result: a 94% reduction in our AI costs, and more than $1 million saved this year alone versus what we would have paid at standard API usage rates.
And the migration itself took a fraction of the time it would have before Blade Code existed. That's the compounding payoff of a control plane. Because routing is centralized and reloads live, moving an entire team to a self-hosted model or onto subscriptions was a change we made on the server: not a coordinated rollout, not a support thread, not a config edit on anyone's laptop. Developers kept using the exact same tools they had the week before. Most of them noticed nothing at all, which is exactly what the founding constraint demanded.
Where we landed
One piece of software now gives our teams:
- New models the week they ship — a registry edit makes them routable company-wide on the next request, with no deploy, no setup, and no approval queue
- Budgets they can see — the number shown to them before it's ever enforced against them
And engineering leadership:
- Every dollar of AI spend — by team, by member, by model, reconciled against the invoice, not estimated
- Cost attributed to features, not just cost centers — from four headers computed on every prompt, across every tool on the machine rather than the handful we integrated by hand
- Somewhere to go when the terms change — if a subscription plan is withdrawn or a model's price moves the wrong way, the work moves with it, onto a cheaper provider or onto models we host ourselves, with nobody's laptop touched
That last one is the part that outlives any particular price list. We don't control which plans exist next quarter, or what a frontier model costs six months from now. We do control where every request goes. Moving a workload, a team, or the whole company onto a different model is a policy edit on the server: switch providers and models on or off for a team, or point a routing override somewhere new, and the registry picks it up on the very next request, while overrides that no longer make sense are pruned automatically rather than left to fail. Engineers keep using the tools they already have, pointed at the gateway they were already pointed at, and find out afterwards, if at all. The migration that saved us 94% was the first time we exercised that, not the last.
The lesson we'd pass on is this: the cost problem and the visibility problem look separate, but you can't solve the first properly without solving the second. We didn't find our 94% by negotiating harder, and we didn't find it in a single clever move. It took both tracks, two thirds subscriptions and one third routing, and the smaller one is what made the larger one possible. We found them because instrumentation showed us where the money was going, and centralization meant we could act on it in an afternoon.
If you're running an engineering org and your AI spend is growing faster than your understanding of it, start with the control plane. Everything else gets easier from there.
