AI Gross Margins: Why the SaaS Playbook Breaks

Estimated read: 21 minutes. The short version: SaaS handed you an 80 percent gross margin for free. AI hands you something closer to 52 percent, and it gets worse with every heavy user you add. The companies that survive the current spending correction are not the ones with the best return-on-investment deck. They are the ones who treated gross margin as something you build into the product, not something the accountant reports after the fact.

AI spending is getting cut right now, and the headlines are loud about it. One report this season found that seven in ten executives would slash their AI budgets if the returns do not show up. Seventy-three percent said their AI investments had already missed expectations over the past year. Uber’s operating chief told analysts that token costs were “harder to justify” than the company first planned. Forrester expects enterprises to push a quarter of their planned AI spend into 2027. One CTO admitted his employees were using a frontier model to check the weather.

Everyone is reading this as a demand story. The buyers got disappointed, the buyers are pulling back, the bubble is leaking air. That framing is comfortable because it points the finger outward, at customers who did not get it or hype that got ahead of itself.

I think it is mostly a margin story wearing a demand-story costume. When money was cheap and growth was the only number that mattered, almost nobody asked what an AI feature actually cost to serve. The correction did not destroy demand for AI. It forced the question that a decade of SaaS economics let founders skip: what is your gross margin, really, after the model takes its cut on every single call? For a lot of AI businesses, the honest answer is “lower than I want to say out loud,” and a few are running negative without knowing it.

This is the durable version of the story, the part that will still be true after the current panic fades and the next one starts. I run two companies where AI does most of the production work, and I have watched my own cost-of-goods line behave in ways the SaaS playbook never warned me about. What follows is the model I use to think about it, where the money leaks, and how to get the points back before a customer, an investor, or a cash-flow statement forces the issue.

The 80 percent margin was a one-time gift

Software ate the world partly because software had the best unit economics any industry has ever seen. You wrote the code once. Serving the millionth user cost almost the same as serving the thousandth, which is to say almost nothing. That is why mature SaaS companies post gross margins of 80 to 90 percent, and why investors learned to pay 10 times revenue for them. The marginal cost of one more customer rounded to zero, so almost every new dollar of revenue dropped toward the bottom line.

A whole generation of founders, me included, internalized that as a law of nature. Grow first, worry about profitability later, because the margin is already there waiting for you. Spend to acquire the customer, then harvest 85 cents on every dollar they pay you for years. The playbook said gross margin is a solved problem. Put your attention on growth and retention.

AI breaks the law. Every time your product calls a model, you pay for it, and you pay again on the next call, and the cost does not amortize away as you scale. By 2026, AI products were averaging around 52 percent gross margin against that 80 percent SaaS benchmark. Inference alone, the fee you pay to run the model, was eating roughly 23 percent of revenue at scaling-stage AI companies. For every million dollars of AI product revenue, about 230,000 dollars walked out the door as compute before a single salary got paid.

Here is the part that should keep you up at night. That inference cost does not shrink as you grow. The data shows it sitting near 20 percent of revenue before launch and 23 percent at scale. It went up. The SaaS reflex, “we will grow into our margin,” assumes a fixed cost you spread over more users. Inference is a variable cost that travels with usage, so growing the user base grows the bill in lockstep. There is no economy of scale to grow into. There is only the meter.

Even traditional software companies bolting AI onto an existing product feel it. The typical hit is 12 to 17 points of gross margin, and public SaaS companies have reported margins running 10 to 17 points below their pre-AI baseline. The AI feature that the board demanded, the one that was supposed to be a growth lever, quietly became a margin tax. The cost-first approach to launching an AI product exists precisely because this tax is invisible until you go looking for it.

The Margin Stack: a model for AI unit economics

I needed a way to think about this that was not just “costs are higher now, good luck.” So I started using a three-layer model I call the Margin Stack. It separates what changed, where the money goes, and what you can do about it. Most founders only ever look at the top layer, the price they charge, and never inspect the two underneath.

The first layer is the Inversion. In SaaS, cost of goods sold was a thin sliver at the bottom of the revenue bar and gross margin was the fat block on top. In AI, that flips. Cost of goods becomes a thick block coupled to usage, and gross margin is the thinner slice that survives on top. The diagram above is the whole thesis in one picture. The green did not just shrink. The red changed character, from a fixed cost you forget about to a variable cost that grows every time someone uses the thing you built.

The second layer is the Leak Map. That 48 cents of cost is not one number. It is inference, plus the retries when the first answer is wrong, plus the evals and safety checks you run to keep quality up, plus the serving infrastructure, plus whatever human review sits in the loop. Each of those is a place margin leaks, and each has a different fix. Founders who only see “compute is expensive” miss that half their cost might be rework, not raw inference.

The third layer is Margin Engineering. In SaaS, margin was an outcome. In AI, margin is a feature you design, the same way you design latency or uptime. Routing cheap requests to cheap models, caching repeated answers, distilling a smaller model for the common path, gating retries so they do not spiral, and pricing in a way that tracks cost rather than ignoring it. This is the layer almost nobody builds, because the SaaS playbook never required it. The AI-native founder playbook treats this as core product work, not a finance afterthought.

The rest of this piece walks down the stack. First the Inversion in detail, then the Leak Map with the actual breakdown of where the dollar goes, then the engineering moves that win the points back.

Where every revenue dollar actually goes

“Compute is expensive” is true and useless. It is true the way “rent is expensive” is true. It does not tell you which room is costing you the money. To fix a margin you have to see it broken apart, dollar by dollar, into the pieces you can actually act on.

Here is how a revenue dollar tends to travel through an AI product. The inference number, that 23 cents, is the one the research nails down for scaling-stage companies. The rest is an illustrative split of the remaining cost that gets you to the roughly 52 cents of margin that companies actually report. Your mix will differ. The point is the shape, not the decimals.

Inference, around 23 cents. The raw fee for running the model on the request. This is the floor everyone sees and the one founders fixate on. It is large, but it is not the whole story, and it is often not even the biggest lever.

Retries and rework, call it 8 cents. When the first answer is wrong, you call the model again. Agentic flows make this worse, because a single task can loop through the model many times before it lands. This is the cost that cost per correct task was built to expose. If your first-pass quality is low, you are paying inference twice or three times for one good output, and that doubling lands straight on your cost line. Reliability is not only a quality problem. It is a margin problem.

Evals and safety, maybe 5 cents. The checks you run to keep the product trustworthy are real compute and real money. I treat the AI eval budget as a line in cost of goods, not a research expense, because it scales with traffic the same way inference does. Skipping it does not save the money. It moves the cost to a refund, a lawsuit, or a churned account later.

Serving and human-in-the-loop, the last dozen cents. Vector databases, orchestration, the GPU you keep warm so latency stays low, and any person who reviews outputs before they ship. Each is defensible. Together they decide whether your 52 cents is actually 52 or actually 38.

The reason this matters is that each slice has a different fix. You cut inference with routing and smaller models. You cut retries with better prompts and evals. You cut serving with caching and right-sized infrastructure. If you only ever see one blended “compute is expensive” number, you cannot tell which lever to pull, so you pull none and hope the model providers cut prices for you. Here is the same shift, laid out as the assumptions you inherited from SaaS against the reality AI hands you.

The SaaS assumption	The AI reality	What it changes
Gross margin is 80 to 90 percent	Gross margin averages around 52 percent	Every growth and pricing model built on 85 points is now wrong
Marginal cost per user is near zero	Every call costs money, every time	Usage is a cost driver, not just an engagement metric
We will grow into our margin	Inference holds near 23 percent at scale	There is no economy of scale waiting for you
Your heaviest user is your best user	Your heaviest user may be your biggest loss	Engagement and profit can point in opposite directions
Margin is an accounting outcome	Margin is a product you engineer	Cost work belongs in engineering sprints, not just finance

Your best customer is your worst account

In SaaS, the power user is a trophy. They use every feature, they never churn, they cost you the same as everyone else, which is almost nothing. You build for them and you celebrate them.

In AI, the power user can be the account that quietly drains the company. The same flat 200-dollar-a-month plan that prints money on a light user can run deeply negative on a heavy one, because the heavy user’s cost is not flat. It tracks their consumption, and consumption varies enormously.

This is not a hypothetical. Through 2026, the people who build coding tools said it out loud. Windsurf was reported to run gross margins that were “very negative,” costing more to operate than it could charge. The founder of one rival put it bluntly: “Margins on all of the code-gen products are either neutral or negative. They’re absolutely abysmal.” Cursor, Lovable, Replit, the whole category was caught in the same trap. They had priced a variable cost as if it were flat.

Then the bill came due all at once. In a six-week window in April 2026, GitHub, Anthropic, and OpenAI ended the all-you-can-eat era. GitHub Copilot moved every plan to usage-based credits, and heavy users watched their bills jump from 29 dollars to nearly 750, and from 50 dollars to 3,000. The companies were not being greedy. They were stopping a bleed.

The math is brutal once you look. A median heavy developer on a 200-dollar plan was burning around 140 million tokens a month, which worked out to roughly 254 dollars of subsidy per user, every month, paid by the provider. A 500-million-token user generated about 1,420 dollars of monthly subsidy. And at the far tail, some “inference whales” ran more than 35,000 dollars in compute while paying their 200 dollars, a 175-times subsidy on a single account. That was never a business. It was a venture-funded coupon, and in 2026 the coupon expired.

The driver underneath all of it is that agentic AI consumes 5 to 30 times more tokens than a simple chat. The moment your product stops being a chat box and starts planning, calling tools, and looping through steps on the user’s behalf, your cost per active user can leap by an order of magnitude while your price sits still. The free tier makes it sharper. A free user who runs agentic workflows is not a marketing cost you can wave away. They are a direct, metered loss, and a determined few can run up a serious bill before you notice. Your revenue model for an AI product has to assume the meter exists, because it does, on every account, free and paid alike.

Even the labs miss their own margin targets

You might assume the companies that own the models have this solved. They have the cheapest possible access to inference, after all. They do not have it solved.

OpenAI posted a 33 percent gross margin, down from 40 percent the year before, and missed its own 46 percent forecast. Its inference cost ran 8.4 billion dollars in 2025 and was projected to climb to 14.1 billion in 2026. On a cash basis its cost of goods came in near 67 percent of revenue. Anthropic told a similar story from a different angle. Its inference margin, the spread on serving a model it already paid to train, jumped from 38 to 70 percent in a year, which sounds like a triumph. But its company-level gross margin still landed around 40 percent, roughly 10 points below its own target, even as revenue exploded. Growth was not the problem. The cost of serving the growth was.

Sit with that. The organizations with the deepest model expertise on earth, serving at a scale you will never match, were missing their margin targets because inference costs ran ahead of them. If they cannot fully tame it from the inside, your position as a company that rents the model is structurally thinner. You pay retail for the most expensive and least controllable input in your cost stack, and that input gets repriced and deprecated on a schedule you do not set.

This is why I treat the model as a supplier, not a foundation. A foundation is stable and you build on top of it. A supplier raises prices, changes terms, and sometimes exits the contract. Building an AI business that survives model churn is the same discipline as protecting your margin, viewed from one step back. The company that can swap to a cheaper model next quarter, or run a smaller one for the common path, is the company whose margin is not hostage to a single vendor’s pricing committee. Portability is a margin strategy.

Margin engineering: getting the points back

The good news is that the 48 cents of cost is not fixed either. It responds to engineering, and the levers are well understood now. Margin engineering is the work of designing your system so the cheap path handles most of the traffic and the expensive path is reserved for when it actually earns its cost.

The biggest lever is routing. Most requests are easy and do not need your most capable model. A lightweight classifier, or even a few heuristics, can send the simple traffic to a smaller, cheaper model and reserve the frontier model for the hard cases. Teams that do this seriously cut inference cost by large fractions without users noticing, because the user never wanted the expensive model. They wanted the right answer.

The second lever is caching. A surprising share of requests repeat, exactly or in meaning. Semantic caching recognizes when a new query matches a previous one closely enough to serve the stored answer with no inference at all. Every cache hit is a request you got paid for and did not pay to serve, which is the closest thing AI has to the old SaaS magic of zero marginal cost.

The third lever is distillation and right-sizing. A smaller model distilled from a larger one can capture most of the capability at a fraction of the cost, with distilled models running 10 to 100 times cheaper on inference, and quantized or smaller bases cutting GPU needs by half to three quarters on workloads that tolerate it. The “good enough” model for any given task keeps shrinking. The fourth lever is killing the retry spiral. Low first-pass quality makes you pay for the same output two or three times, so investment in prompts and evals pays back directly as fewer paid retries. The fifth is context discipline, because stuffing the whole window into every call is a quiet tax you pay per token, per request, forever.

The margin leak	Where it hides	The fix
Big model doing easy work	“We just use the best model for everything”	Route by difficulty; cheap model as the default
Paying twice for identical work	Invisible inside a single blended compute bill	Exact and semantic caching; measure hit rate
Retry spirals on bad first answers	Looks like healthy “usage” on the dashboard	Evals and better prompts; cap the loop count
Context bloat	“Bigger context window must be better”	Retrieve only what the task needs; trim aggressively
Flat price on variable cost	“Engagement is up, so things are good”	Meter the heavy tail; add a usage ceiling above the plan

None of this is exotic. It is the AI equivalent of the infrastructure work that SaaS companies did once, a decade ago, and then never had to think about again. The difference is that in AI it is never finished, because inference is now the majority of the cost, more than half of all AI cloud infrastructure spend, and the meter never stops running. Margin engineering is not a project. It is a standing function, like security. The internal AI stack a small team actually runs should have a cost owner the same way it has a reliability owner.

A worked example: the plan that looks profitable and is not

Abstract percentages are easy to nod along to and hard to act on. So let me run a small product through the numbers, the way I run my own. Say you sell an AI research assistant for 30 dollars a month, flat. Three users sign up.

The first is a light user. Fifty requests a month, short prompts, a mid-tier model. Their compute costs you about 3 dollars. You keep 27. That is a 90 percent margin, exactly the SaaS dream, and if every user looked like this you would never think about cost again.

The second is a daily user. Five hundred requests, some of them multi-step, occasionally invoking a tool. Their compute runs about 12 dollars. You keep 18, a 60 percent margin. Still fine, still a business.

The third is your favorite kind of user, the one who logs in every day and tells their friends. They run agentic research loops, the model planning and calling tools and looping through steps, which consumes 20 to 30 times the tokens of a simple query. Their compute costs you 48 dollars. They pay 30. You lose 18 dollars a month on the customer you are most proud of. The reason their cost explodes is the same reason AI agents fail in production: every extra loop is another paid call, and a few of them quietly become a small fortune.

Now blend it. If 70 percent of your users are light, 25 percent are daily, and 5 percent are heavy, your blended margin still looks acceptable, somewhere in the mid-50s, and your dashboard says everything is fine. Here is the trap. The heavy users are the engaged ones. They stay, they expand, they refer. So as your product succeeds, the heavy share grows, the light share shrinks, and your blended margin rots from the inside while your growth chart points up and to the right. Success makes the unit economics worse. That is the sentence that should change how you plan.

Then engineer it. Put in routing and the light and daily users drop to a cheaper model on most calls, cutting their cost by roughly 40 percent. Add caching and the repeated research queries stop costing anything to serve. Your heavy user’s 48 dollars falls toward 29, near breakeven instead of deep loss. Finally, add a usage ceiling: above a defined level of consumption, the heavy user moves to a 99-dollar tier. Now the account you were proud of is also the account that pays its own way, and you can keep celebrating it honestly. Same product, same users, same model. The only thing that changed is that someone decided to engineer the margin instead of hoping it would appear.

What most founders get wrong about falling token prices

Here is the objection I hear every time I raise this, and it is a good one. Tokens keep getting cheaper. The price of a given unit of model capability has fallen roughly 98 percent since the early GPT-3 days, from around 20 dollars per million tokens to well under one dollar. Some people call it a 1,000-times cost collapse. So why panic about a 52 percent margin? Wait two years and the same product will run at 70, the same way bandwidth and storage got cheap and stopped mattering. The model providers will save us.

I want to give that argument its full weight, because the price decline is completely real and it is not slowing down. If your only cost problem were the per-token rate, time really would fix it.

But that is not what happens to the bill. Despite a 98 percent drop in unit price, spending on large language models doubled in a single year, and the average enterprise AI budget went from 1.2 million dollars in 2024 to about 7 million in 2026. Prices fell and bills rose, at the same time, for the same buyers. This is Jevons paradox, the old observation that when a resource gets cheaper, total use can rise faster than the price falls, so you end up spending more, not less.

The mechanism is two-sided. Cheaper tokens make you run more queries, so the team that budgeted for a million calls happily scales to a hundred million while the price per call drops and the total check stays the same or grows. And cheaper tokens make whole new uses economical that were not before, so work that lived in a one-shot chat becomes a multi-step agent that costs dollars where the chat cost cents. Falling prices fund deeper reasoning loops, bigger context windows, and multi-agent workflows. The savings do not reach your margin. Your product spends them on new capability before they ever get there.

So the honest version is this. Per-unit prices will keep falling, and that is a genuine tailwind. But it blows sideways. It lands as more capability and more usage, not as margin, unless you make a deliberate decision to bank some of the decline instead of spending all of it. The founders who win are not the ones who wait for compute to get cheap. They are the ones who treat each price cut as a choice: take it as margin, or reinvest it in capability, on purpose, with eyes open. Waiting for the providers to save your unit economics is the AI version of standing in a field hoping the wind picks your direction. Margin is something you take, not something you are given.

What to do Monday morning

If you build or sell anything with AI in it, here is the install. None of it requires a finance team or a pricing consultant. It requires you to look at numbers you have probably been avoiding.

1. Measure real margin on one feature, not a blended number. Pick your most-used AI feature. Instrument it so you can see, per request, the inference cost plus retries plus eval calls plus serving plus any human review. Get to a true gross margin for that one feature in the next few days. A blended company-level number hides the feature that is bleeding.

2. Find your whale. Sort your users by token consumption. Take the top 5 percent. Compute what each of them costs to serve against what they pay. If some are negative, and they usually are, you have just found the account that scales your losses every time you grow. You cannot fix what you refuse to look at.

3. Put a router in front of the model. Stop sending every request to your most expensive model. Even a crude rule, send short or simple requests to a smaller model and escalate only when needed, captures most of the savings. This is the single highest-return change for most products.

4. Turn on caching and treat hit rate as a margin metric. Start with exact-match caching, add semantic caching next. Every cache hit is revenue you served at near-zero cost. Put the hit rate on a dashboard and try to raise it like you would raise conversion.

5. Re-price to cost, and kill true unlimited. If you sell a flat plan, add a usage ceiling above which heavy users move to metered pricing or a higher tier. You are not punishing power users. You are refusing to subsidize a few accounts into a hole the rest of the business has to climb out of. Price the meter because the meter is real.

6. Put gross margin next to growth, every week, owned by a name. The reason margin rotted is that nobody watched it while everybody watched growth. Add one line to the weekly review, give it an owner, and defend it the way you defend uptime. What gets watched gets protected.

The spending correction is not the end of AI as a business. It is the moment the business gets real. The companies still standing a year from now will not be the ones who told the best return story to a nervous buyer. They will be the ones whose gross margin was built into the product, defended every week, and engineered back every time the cost crept up. Margin was a gift in the SaaS era. In the AI era it is a craft, and the craft is learnable.

Frequently asked questions

What is a good gross margin for an AI startup?

By 2026, AI products averaged around 52 percent gross margin, well below the 80 to 90 percent that defined mature SaaS. A healthy target for an AI-native product is 60 percent or higher, reached through routing, caching, and pricing that tracks usage. Below 40 percent, you are likely subsidizing your heaviest users without realizing it.

Why are AI gross margins lower than SaaS?

Traditional SaaS had near-zero marginal cost, so serving one more user cost almost nothing. AI products pay for inference on every request, a variable cost that scales with usage and does not amortize away as you grow. Inference alone runs about 23 percent of revenue at scale, which is why AI margins sit near 52 percent instead of 80.

How much does inference cost as a share of revenue?

At scaling-stage AI companies, inference consumes roughly 23 percent of revenue, about 230,000 dollars for every million dollars of revenue. It does not fall with scale; it sits near 20 percent before launch and 23 percent at scale. Bolting AI onto an existing SaaS product typically costs 12 to 17 points of gross margin.

Will AI margins improve as token prices fall?

Per-token prices have fallen about 98 percent since early GPT-3, yet total AI spending rose over the same period because cheaper tokens fund more usage and new use cases, which is Jevons paradox. Price declines tend to land as more capability rather than more margin. Your margin improves only if you deliberately bank some of the savings instead of spending all of it on capability.

Why are AI coding tools changing their pricing?

Flat-rate plans were structurally loss-making for heavy users, whose token consumption could cost the provider hundreds or thousands of dollars a month against a fixed fee. In April 2026, GitHub, Anthropic, and OpenAI moved to usage-based pricing within weeks of each other, and some heavy users saw bills rise more than 20 times. The venture-funded subsidy on unlimited usage simply ran out.

How do I reduce my AI product’s cost of goods sold?

The main levers are routing simple requests to cheaper models, caching repeated answers, distilling smaller models for the common path, raising first-pass quality to cut paid retries, and trimming context. Routing and caching usually give the fastest return. Treat margin as a product you engineer, not an outcome you report.

Is usage-based pricing better than flat-rate for AI products?

For products with high usage variance, flat-rate pricing subsidizes the heavy tail and can run negative on power users. A common fix is a flat plan with a usage ceiling, above which users move to metered pricing or a higher tier. The goal is to keep price tracking cost, so a few accounts cannot scale your losses faster than your revenue.

Does gross margin matter for AI startup valuation?

Yes, and more so in a downturn. Investors weight profitability more heavily when conditions tighten, and low-margin companies need high growth to justify their valuation, which makes them fragile when growth slows. Durable gross margin is what lets an AI business survive a spending correction instead of being repriced by it.