Skip to content

Topic

AI inference costs

The compute expense of running AI models at serving time, driven by GPU usage and token consumption, which shapes vendor pricing and margins.

Current stories

productConfirmed3 publishers

Uvi estimates its context layer costs $0.10 to $0.16 an answer against up to $1.41 for in-house agents

Uvi, which raised $7.5 million in seed funding, says its own tests show company-built AI agents cost about eight times as much per answer. Both reports trace the figure to Uvi's testing, so it is a claim to reproduce on a buyer's workloads.

Perspective Coverage

3 publishers
Builder
Builder 43%
Operator
Operator 30%
Investor
Investor 27%

Reality

Evidence30
Adoption20
Hype gap+35
Incentives80
Confidence60
productOne report1 publisher

Cisco recasts Webex as the front end of its single AI platform

Cisco used WebexOne to present Webex as the place where people and agents meet on one AI platform consolidated from roughly 250 acquisitions. Many Webex buyers got their first detailed look at that pitch in a keynote about token budgets and agent network load.

Publishers:siliconangle.com

Reality

Evidence35
Adoption20
Hype gap+35
Incentives75
Confidence40
investOne report1 publisher

Ackman and Wood are measuring different ends of AI spending in their Fed rate dispute

Cathie Wood cites 99.99% annual falls in AI inference costs against Bill Ackman's warning that the Fed's September hike won't slow AI spending. Her figures price AI's output and his worry is demand for its inputs, so both can hold at once.

Reality

Evidence30
Adoption
Not scored
Hype gap+35
Incentives
Not scored
Confidence35
investOne report1 publisher

Honeywell and Ecolab say frontier AI errs too often and costs too much for physical work

Honeywell's CTO put frontier AI at maybe 85% accuracy against the 99.9999% customers demand, and Ecolab said top models can cost more than people. Ecolab has since cut its token costs 70% to 80% by tuning, so cost looks like the easier of the two limits to fix.

Publishers:fortune.com

Reality

Evidence35
Adoption40
Hype gap+5
Incentives55
Confidence40
buildOne report1 publisher

LessWrong cost study prices the median hour of AI task work at 4 cents of GPU time

LessWrong post puts the GPU cost of an AI doing an hour of median human work at about 4 cents, against a $25 US median wage. Current API prices narrow that gap sharply, and for the hardest tasks they lift AI cost to the hourly rate of a skilled engineer.

Publishers:lesswrong.com

Reality

Evidence30
Adoption
Not scored
Hype gap+35
Incentives
Not scored
Confidence35
leadershipOne report1 publisher

Kognitos's founder traces the enterprise token bill to undocumented work

Binny Gill argues in a Forbes council column that firms are paying reasoning-model rates for rule-following tasks. The two studies he cites measure consultants and a research router. Neither one measured an enterprise bill.

Publishers:forbes.com

Reality

Evidence30
Adoption18
Hype gap+35
Incentives78
Confidence38
investOne report1 publisher

McKinsey survey finds just 6% of companies report significant impact from AI so far

A Fortune essay by the authors of the forthcoming When Machines Act argues that the pacing fight misreads how AI reaches the economy. The survey lines it cites put the clock in enterprise data.

Publishers:fortune.com

Reality

Evidence32
Adoption22
Hype gap+18
Incentives62
Confidence36
buildOne report1 publisher

VoxCPM2 trades the per-character speech bill for a GPU and a base-URL change

OpenBMB's 2B-parameter model is Apache-2.0, speaks 30 languages, and serves through vLLM's OpenAI-compatible /v1/audio/speech. The real-time factor quoted for it was measured on a different backend than the one the project recommends for production.

Publishers:dev.to

Reality

Evidence38
Adoption
Not scored
Hype gap+38
Incentives42
Confidence40
investOne report1 publisher

Uber's CTO calls time on tokenmaxxing after quadrupling its frontier AI users

Praveen Neppalli Naga says treating AI cost as an engineering problem lowered Uber's cost per token even as employee use grew. The spending data for the wider enterprise market points the other way.

Publishers:fortune.com

Reality

Evidence42
Adoption55
Hype gap+35
Incentives68
Confidence52
investOne report1 publisher

Adobe's $27.50bn of ARR works out to $27.50 a year for each of its billion monthly users

Adobe reported record third-quarter revenue of $6.76bn and its billionth monthly active user on September 10. By Shantanu Narayen's own split, freemium expansion explains only about half of the lowered ARR outlook.

Publishers:techtimes.com

Reality

Evidence62
Adoption72
Hype gap+20
Incentives65
Confidence55
leadershipOne report1 publisher

Seven cents a million tokens buys the capability that cost twenty dollars in 2022

Stanford's AI Index puts a 142-fold parameter cut and a more than 280-fold price cut behind one fixed MMLU threshold. That narrows where building your own still pays, and it lands in a state-law count that doubled in a year.

Publishers:hai.stanford.edu

Reality

Evidence60
Adoption68
Hype gap+10
Incentives40
Confidence57
productOne report1 publisher

Canva's $7.1bn markdown is an inference bill, not a mood swing

Two separate valuers cut Canva the same way after a third came off its growth forecast. The stated cause was compute cost the company could not pass on to customers.

Publishers:thenextweb.com

Reality

Evidence61
Adoption58
Hype gap+14
Incentives71
Confidence55