Result Per Dollar Per Second: The AI Metric That Pays

Last week Bain put a number on the thing the whole industry has been avoiding. For the AI buildout to pay for itself, the sector has to reach about 6 trillion dollars in annual revenue by 2031. Existing consumer and enterprise AI services get you to roughly 1.8 trillion of that, which leaves a 4.2 trillion dollar hole to fill in five years. Hyperscaler capital spending could hit 780 billion dollars in 2026 alone, close to five times the level of three years earlier. Bain’s own framing is that capex should sit at about a quarter of revenue, so the math only closes if the money coming out is enormous.

Every time I read one of these pieces, the comment sections argue about the same thing: will the models get good enough to justify the spend. I think that is the wrong argument, and I have thought so since well before this particular report. The buildout does not pay back because a model tops a leaderboard. It pays back when a specific job gets done, at a price a customer will bear, fast enough that the customer sticks around. That is a different measurement than the one everyone quotes, and most builders I talk to are not running it.

I have spent the last few years building companies where the “team” is mostly software, and I have made the mistake I am about to describe. I picked models the way you pick a phone from a spec sheet: I looked at the rankings, took the one near the top, and shipped it. It cost me money and it cost me speed, and for a while I could not see it because the scoreboard I was reading said I had made the right call. This post is the scoreboard I use now.

A ninth of a company’s revenue now goes to concrete and power

Sit with the Bain framing for a second, because it reframes what a metric is for. If a quarter of AI revenue has to be plowed back into data centers, memory, networking and power, then the industry is running one of the most capital-hungry business models ever assembled. PwC has tracked tens of trillions of dollars in projected capex across the build-out. Goldman has published its own set of assumptions about how much of that spend is even recoverable. These are not small rounding differences. They are bets that the output of all this compute will be worth paying for at a scale that has never existed.

Here is the part that matters for anyone building on top of the models rather than building the models. The people writing 780 billion dollar checks are, in effect, subsidizing your inference. Prices at the API layer have collapsed while that spend went up. That is a gift and a trap at the same time. It is a gift because your unit costs keep falling. It is a trap because cheap inputs make it easy to stop measuring, and when you stop measuring you drift toward the flashiest model instead of the one that earns.

So the question a builder should ask is not “which model is best.” It is “which model turns the least of my money into the most finished work, quickly enough that my customer stays.” The industry answers the first question every week with a new leaderboard. Almost nobody answers the second question for their own product. I want to give you the metric that does.

The leaderboard measures a lab’s homework, not your P&L

Public benchmarks exist to compare models in a laboratory. They are useful for that. They are close to useless for deciding what to run in production, and the reason is structural, not a matter of picking a better leaderboard.

Start with saturation. MMLU was the general-purpose benchmark of record through 2023, and then frontier models blew past 90 percent and the scores stopped separating anyone. HellaSwag saturated above 95 percent. The pattern is so consistent that reviewers now estimate a static benchmark loses its ranking signal in under two years on average. When every serious model scores in the nineties, the ranking is measuring the last two points of a rounding error, and those two points have nothing to do with whether the model can close a support ticket in your product.

Then contamination. A benchmark is contaminated when its questions, or text close enough to them, end up in the training data, so the model recalls the answer instead of working it out. A systematic review this year concluded that contamination has effectively retired four of the benchmarks people still quote in pitch decks: MMLU, HumanEval, HellaSwag, and the original GSM8K. Top scores on all of them sit in the nineties, and none of them produces a trustworthy ranking for frontier work anymore.

Then gaming. In April 2025, Meta’s chart-topping result on the popular human-preference arena came from an unreleased, customized model rather than the version anyone could actually use. The arena said the presentation should have been clearer and changed its policy. I am not interested in relitigating that episode. I am interested in what it reveals: a public ranking is a competitive surface, and competitive surfaces get optimized. The number at the top is partly a measurement and partly a marketing artifact.

None of this makes benchmarks worthless. It makes them the wrong instrument for a spending decision. A leaderboard tells you how a lab’s model did on a shared test. Your business does not run on a shared test. It runs on your tasks, at your prices, on your clock. You need a scoreboard built from those three things.

Result per dollar per second is the whole scoreboard

The metric is exactly what it sounds like. For a given job that your product needs done, measure how many finished, correct results you get per dollar you spend and per second you wait. Three terms, and each one replaces a habit that a leaderboard trained into you.

Result replaces rank. Instead of asking how a model scored on someone else’s test, you ask how many of your jobs it completed correctly. Dollar replaces the fiction that inputs are free. You count the fully loaded cost of producing a correct result, not the price of a single token. Second replaces the hand-wave of “fast enough.” You put a number on how long the customer waits, and you treat that number as revenue, because it is.

The reason to fuse the three into one metric, rather than track them separately, is that they trade against each other and the trade is where all the money is. A cheaper model that is wrong more often can cost you more per correct result than an expensive one. A faster model that drops quality can lose you customers even though the latency chart looks great. You cannot see any of that if you look at rank, price, and speed on three different dashboards. You see it the moment you divide correct results by cost and by time and compare two options on the same line.

I am deliberately keeping this to one named metric. Everything else in this post is described in plain language, because the last thing this field needs is another capitalized framework. The point is not the phrase. The point is that you compute one ratio, for your own workload, before you spend a dollar or make a promise about speed.

The result is a finished job done right, and a wrong answer scores zero

The numerator is where most teams fool themselves, so start here. A result is not a response. A result is a job your product needed done, completed correctly, with no human having to catch and redo it. If the model returns something plausible that is wrong, that is not a partial result. It is a zero, and often worse than a zero, because someone has to notice it is wrong and clean it up.

The agent benchmarks make this vivid in a way the old text benchmarks never did. On OSWorld, the computer-use benchmark, human testers finish about 72 percent of tasks. When the benchmark first came out, the best agent managed 12 percent. The numbers have climbed fast since then, and on the newer version the leading model clears around 72 percent. That sounds like parity with humans, until you read the reliability work published this year. There is a hard difference between passing a task once and passing it every time. One study framed it as pass at one versus pass to the fourth: an agent that succeeds 85 percent of the time on a single run can still fail one out of every four times you ask it to do the same thing four times in a row. In a customer-facing flow, that is undeployable. The same analysis put a blunt number on the gap: benchmarks reporting 85 percent completion corresponded to about 20 percent completion on real workflows, a 65 point gap between the leaderboard and the job.

This is why the numerator has to be correct results, not attempts. And it is why a wrong answer is not neutral. It carries a retry tax. When a share of your jobs come back wrong, every wrong one costs you the failed attempt plus the cost of the attempt that fixes it plus, sometimes, the cost of a human noticing. The effective cost per correct result climbs faster than people expect.

Notice what the diagram does. It refuses to let you compare sticker prices. Model B is less than half the price per attempt, and at a 70 percent success rate it still comes out cheaper per finished job. But the correctness term is doing the real work: push B down to a 55 percent success rate and the cheaper model becomes the more expensive one, because you are paying for two attempts to get one job done. The sticker price told you B was the obvious choice. The result term told you it depends entirely on how often B is right, which is a fact about your tasks that no leaderboard contains.

The practical move is to define, per job type, what “correct” means and who checks it. For a support reply, correct might mean the customer did not write back confused. For a code change, correct might mean the tests pass and the diff merges without a human rewrite. Once you can label a result right or wrong, you can count it, and once you can count it, the whole metric becomes real. This is the same discipline I wrote about in the accountability trap of selling software as a service outcome: if you are going to charge for a result, you have to be able to say whether you delivered one.

The dollar is your fully loaded cost, not the sticker on a token

The denominator has two traps. The first is measuring the wrong dollar. The second is not measuring at all, because inputs got so cheap that it felt beneath you.

Start with the cheapness, because it is genuinely staggering and it is exactly why people stop counting. Getting output at the quality of the original GPT-4 cost around 60 dollars per million tokens in late 2021. By 2026 you can get that level of performance for roughly 40 cents per million tokens, a drop of somewhere between 150 and 1000 times depending on which model you use. Even at the frontier, API prices fell about 80 percent between early 2025 and early 2026. Budget models now sit around 10 cents per million input tokens. When the input is this cheap, the temptation is to wave the cost line away and assume it rounds to nothing.

It does not round to nothing, for two reasons. Output tokens run about four times the price of input tokens across the market, and reasoning-heavy work generates a lot of output. And the fully loaded cost of a result is much larger than the token bill. It includes the retry tax from the section above. It includes the tokens you spend on context you stuffed into the prompt to make the model reliable. It includes the humans who review the borderline cases. It includes the second and third model calls in an agent chain, each of which has its own token bill and its own failure rate that compounds down the chain. I have watched a workflow where the raw token cost was a few cents and the real cost per finished job was a couple of dollars once the retries and the review time were counted.

So the dollar in the metric is total cost to produce one correct result, all in. That is the number that belongs under the line. If you only have the token bill, you have the smallest and least interesting part of the cost, and you will systematically overrate cheap-per-token models that are expensive-per-result. I went deep on this in the way agent costs invert as you scale, and the short version is that the cost that kills margins is almost never the one on the pricing page.

There is a strategic edge hiding in the denominator, too. When inference keeps getting cheaper, the builders who actually measure cost per result can pass some of the savings to customers and keep some as margin, on purpose, instead of letting the whole windfall leak away. That is the pricing decision I laid out in how to price when inference keeps getting cheaper. You cannot make that decision if you do not know your cost per result to begin with. And you should sanity-check any efficiency you think you are buying, because the claimed savings and the realized savings are often different numbers, which is the whole subject of the efficiency trap.

The second is money, because your customer is watching a spinner

The last term is the one builders treat as a nice-to-have, and it is the one with the cleanest revenue data behind it. Latency is not comfort. It is conversion, retention, and usage frequency, and the web has been measuring the effect for twenty years.

The canonical finding is Amazon’s: every 100 milliseconds of added latency cost about 1 percent of revenue. It has held up well enough that people still quote it as the baseline. Google found that adding 100 to 400 milliseconds of delay to search reduced searches per user, and estimated roughly a 0.6 percent revenue loss per user from the drop in activity. Bing measured a 1.2 percent revenue-per-user hit from 500 milliseconds. Walmart saw the mirror image: shaving 100 milliseconds off load time lifted revenue by around 1.5 percent. Akamai found that going from a two-second to a three-second page load cut conversion by about 7 percent. These are old, boring, replicated numbers, and they describe pages far simpler than an AI feature that thinks for several seconds before it answers.

AI makes the second term heavier, not lighter, because model responses are slow by web standards and getting slower where reasoning is involved. A frontier model that “thinks” can take many seconds to produce a good answer. If that answer is correct and the alternative was a fast wrong answer, the wait can be worth it. But the wait is a cost, and it belongs in the metric explicitly, not as a footnote. The right question is never “is this fast.” It is “how many correct results per dollar do I get at this speed, and does a faster, slightly less accurate option produce more finished value once I account for the customers who leave during the wait.”

Here is a concrete way the second term changes a decision. Suppose a support-triage feature can run on a slow frontier model that is right 92 percent of the time in eight seconds, or a fast mid-tier model that is right 88 percent of the time in one second. On rank, the frontier model wins and it is not close. On result per dollar per second, you have to weigh four points of extra correctness against seven seconds of extra wait on every single ticket, multiplied by the share of customers who abandon a slow interaction. For a high-volume, low-stakes flow, the fast model usually produces more finished, kept business per dollar. For a low-volume, high-stakes flow, the accurate one does. Rank cannot tell you which situation you are in. The metric can, because it forces the trade into a single number.

The number one model is usually the wrong purchase

Put the three terms together on a real comparison and the counterintuitive result falls out on its own. The model at the top of the leaderboard is frequently the wrong thing to run in your product, and not by a little.

Walk the numbers, because the gap is not a trick of the chart. The frontier model is right slightly more often, 94 versus 90 percent, so its cost per correct result is 9.6 cents against the mid-tier model’s 1.3 cents. That alone is a seven-times cost difference for four points of accuracy. Now divide by latency, seven seconds against a bit over one, and the finished-value-per-second gap widens to roughly forty times. On a routine, high-volume classification job, the model nobody ranks first produces dramatically more kept business per dollar per second. The four points of accuracy do not come close to paying for the price and the wait.

The frontier model is the right call in the opposite situation: a low-volume, high-stakes job where a single wrong answer is expensive and nobody is watching a spinner. Drafting a legal clause, diagnosing a production incident, making an irreversible decision. There, the four points of accuracy are worth almost any latency and almost any price, because the cost of being wrong dwarfs both. The metric shows you that too. The point is not that cheap always wins. The point is that rank never tells you which regime you are in, and result per dollar per second always does.

What benchmark rank measures	What result per dollar per second measures
Accuracy on a shared, public test set	Correct completions of your specific jobs
One-shot capability in a vacuum	Reliability across repeated runs, retries included
Nothing about price	Fully loaded cost per finished result
Nothing about latency	Wait time, priced as lost conversion
Comparable across companies	Comparable across your own model choices
Moves when a lab ships	Moves when your product earns

Three ways a benchmark rank lies to a builder

It helps to name the specific failures, because each one has a fix and the fix is always the same shape: measure the thing you actually care about on your own workload.

The first lie is task mismatch. A model can top a coding benchmark and be mediocre at your particular kind of code, or ace a reasoning test and fumble the messy, half-specified requests your users actually send. Public tests are clean and self-contained. Real jobs arrive with missing context, contradictory instructions, and the assumption that the system knows things about your product that no benchmark encodes. The rank is measuring performance on a distribution that is not yours.

The second lie is the reliability gap I described earlier, and it deserves its own name because it is the one that burns people in production. A model with a great single-run score can be undeployable if it cannot repeat the result. The 85-percent-on-the-benchmark, 20-percent-on-the-workflow gap is not an outlier, it is the normal shape of things right now. When you commit to letting a model do a job without a human in the loop, the number that matters is not how well it does the job once, it is how rarely it fails across thousands of runs. That is the whole substance of the delegation threshold, the point where you stop testing a system and start depending on it.

The third lie is quieter and more technical. A model that looks strong on a short, clean prompt can degrade badly as you feed it the long, messy context that real work requires. Performance is not flat across the size of the input, and the effective, useful context is usually far smaller than the advertised window. I wrote about that decay in why more context makes models worse, not better. A leaderboard runs its tests near the model’s best conditions. Your product runs it near its worst, with a bloated prompt full of history and instructions, which is exactly where the rank stops predicting anything.

All three lies dissolve under the same discipline. Build a small, private evaluation set out of your own real jobs, with your own definition of correct, run the candidate models on it at the prices and latencies you would actually pay, and compute result per dollar per second. The public rank becomes a rough prior, useful for deciding which handful of models to even test, and nothing more.

Chasing the top of the leaderboard is the expensive mistake

The instinct to run the best-ranked model feels responsible. You are giving your users the strongest tool. In most products it is the costly reflex, and it hurts in a way that is hard to see on any single dashboard.

It hurts because the frontier carries a premium that a routine task does not need. You are paying frontier prices and frontier latency to get a few points of accuracy on jobs where those points do not change the outcome. Multiply that across every call in a high-volume feature and the waste is enormous, and it never shows up as a line item called “waste.” It shows up as thin margins and a slow product, and you blame the market or the model instead of the choice.

It also hurts because the frontier moves. Tie your product to whichever model is ranked first and you have signed up to re-integrate every few months as the crown changes hands, chasing accuracy gains that your users may not even be able to perceive. The teams I have seen build durable AI products do the opposite. They pick the cheapest, fastest model that clears their correctness bar for each job, they route different jobs to different models, and they only reach for the frontier where a wrong answer is genuinely expensive. That is not a compromise. It is the design that produces the most finished value per dollar per second, which is the only thing the buildout is eventually going to be judged on.

The macro data is starting to say the same thing at the scale of the whole industry. The MIT study that made the rounds this year found that about 95 percent of enterprise generative-AI pilots delivered no measurable return, and the reason was almost never model quality. It was integration, workflow fit, and the gap between a demo and a dependable result. The small group that succeeded, earning something like 3.70 dollars for every dollar spent, did it by wiring agents into real work, not by running the highest-ranked model. That is result per dollar per second at company scale. The winners optimized the finished result. The other 95 percent optimized something closer to the rank.

A one-week instrument for your real cost per result

You do not need a platform to start measuring this. You need one week and a willingness to count things you have been ignoring. Here is the sequence I use when I set it up for a new workflow.

On day one, pick one job that matters and write down what “correct” means for it in a single sentence, plus who or what decides. Not the whole product, one job: the support reply, the classification, the code change, the extraction. If you cannot define correct in a sentence, you have found your real problem, and no model choice fixes it.

On days two and three, assemble fifty to a hundred real examples of that job from your actual traffic, not invented ones, and build a tiny harness that runs a candidate model over all of them and records three things per example: was the result correct, what did it cost in tokens in and out, and how long did it take. Fifty real examples will teach you more than any leaderboard, and you can label correctness by hand at that volume in an afternoon.

On days four and five, run at least three models through the same harness: the frontier model you assume you need, a mid-tier model, and a budget or open-weight model. Compute cost per correct result by dividing each model’s total cost by its number of correct results, so the retry tax is baked in. Then divide by median latency to get result per dollar per second. Put the three on one line and look at the gap. It is usually larger than you guessed and pointing at a model you did not expect. The output of that week looks like this, and the last column is the only one that should decide anything.

Model on your 80 real jobs	Correct	Cost / attempt	Latency	Cost / correct	Results per $ per s
Frontier, ranked 1	94%	$0.090	7.0 s	$0.096	1.5
Mid-tier, ranked 7	90%	$0.012	1.2 s	$0.013	64
Budget open-weight	71%	$0.004	0.9 s	$0.006	198

The budget model wins the raw ratio, but this is exactly where the correctness bar earns its keep. If a 71 percent success rate means nearly a third of your customers get a wrong answer on the first try, the ratio is lying about the experience, and the mid-tier model at 90 percent is the honest choice. That judgment call is yours to make, and it is a real one, but at least you are now making it against three numbers you measured instead of a ranking someone else published. The instrument does not remove the decision. It just makes the decision about your business instead of about a leaderboard.

On days six and seven, make the boring decision the numbers support, then wire in the two habits that keep the metric alive. First, route by job: send routine, high-volume jobs to the cheap fast model that clears the bar, and reserve the frontier for the jobs where a wrong answer is expensive. Second, put cost per correct result and median latency on a dashboard you look at weekly, next to revenue, so the metric moves when your product earns instead of when a lab ships. That is the entire instrument. It costs a week and it will change model decisions you have been making on vibes and rankings for as long as you have been building.

If you want the larger context for where this fits, it is one piece of running a company where software does most of the work, which is the subject of the AI-native founder playbook. And the same instinct that makes result per dollar per second the right model metric makes revenue per employee the right company metric, which I unpacked in why revenue per employee is the number that tells the truth. Both are the same move: stop reading the scoreboard someone else built and start reading the one your business actually runs on.

Questions builders ask about measuring AI

Is benchmark rank useless, then?

No. It is a useful prior for deciding which handful of models are worth testing, and it is a fair way to compare labs. It is the wrong instrument for a production spending decision, because it measures capability on a shared test rather than finished results on your tasks at your prices and your latency. Use it to build a shortlist, not to make the call.

How do I actually put a dollar figure on latency?

Start with the replicated web numbers as a floor: roughly 1 percent of revenue per 100 milliseconds is the Amazon baseline, and a 2-to-3-second slowdown cut conversion about 7 percent in Akamai’s data. Estimate the share of users who abandon a slow interaction in your product, multiply by the revenue those users represent, and treat that as the cost of the extra seconds. It will be rough. Rough and counted beats precise and ignored.

What counts as a correct result?

Whatever means no human has to catch and redo the work. For a support reply, the customer did not come back confused. For a code change, the tests pass and the diff merges without a rewrite. For an extraction, the fields match the source. Define it in one sentence per job type before you measure anything, because the definition is what turns responses into countable results.

Does this still matter if inference keeps getting cheaper?

It matters more. Cheap inputs are exactly what tempt teams to stop counting, and cheapness hides the retry tax and the latency cost, which do not fall just because the token price does. Falling input prices also create the pricing decision of how much of the saving to keep versus pass on, and you cannot make that decision without knowing your cost per correct result.

How is this different from just tracking cost per token?

Cost per token is the smallest part of the picture and the most misleading, because it ignores whether the answer was right and how long the customer waited. A model can be cheap per token and expensive per correct result once you count retries, review time, and the tokens you spend on context to make it reliable. The metric divides by correct results and by latency for exactly that reason.

Which model should I actually default to?

The cheapest, fastest model that clears your correctness bar for each specific job, with different jobs routed to different models, and the frontier reserved for the cases where a wrong answer is genuinely expensive. There is no single default. The default is the routing rule, and the routing rule comes from your own result-per-dollar-per-second numbers.

Do I need a special tool to measure this?

No. Fifty real examples, a small script that records correct-or-not, token cost, and latency per example, and an afternoon of hand-labeling will get you a trustworthy number for one job. Tools help once you have many jobs and want continuous tracking, but the first, decision-changing measurement is a week of work, not a purchase.

Won’t the frontier eventually be cheap and fast enough that this stops mattering?

The frontier keeps moving, but so does the definition of frontier, and reasoning-heavy work keeps latency and output cost high even as older capability gets cheap. More importantly, the discipline is not about a price level, it is about measuring finished value against cost and time on your own workload. That stays true no matter where the frontier sits, because your tasks, your prices, and your clock are always specific to you.

I am a four-time founder building AI-native companies. I write about the founder mindset, AI opportunities, and the mechanics of building with AI at vikasmalpani.com.