Skip to content

Topic

AI inference

Running trained models to answer requests, together with the chips, schedulers and serving software used to do it cheaply and quickly.

Current stories

buildConfirmed4 publishers

Apple's idle Macs pull their own AI jobs from a PostgreSQL queue, with no central scheduler

Apple's internal Minions system moves stalled AI jobs onto idle Macs, cutting some 10-plus-hour GPU queues to as little as minutes. The researchers also saw a common worry about team data running on devices outside the group.

Publishers:arxiv.org computerworld.com machinelearning.apple.com tomshardware.com

Perspective Coverage

4 publishers
Builder
Builder 43%
Operator
Operator 35%
Investor
Investor 22%

Reality

Evidence45
Adoption15
Hype gap+30
Incentives50
Confidence62
productOne report1 publisher

On paper, Maia 300 trails Rubin per chip on bandwidth and leads it per rack

Microsoft's Maia 300 and Nvidia's Rubin cannot be ranked on speed, because their published performance figures use different baselines. TechRepublic's comparison found no independent, workload-matched benchmark for the two.

Publishers:techrepublic.com

Reality

Evidence50
Adoption25
Hype gap+35
Incentives
Not scored
Confidence58
investConfirmed2 publishers

SK hynix locates the next AI inference bottleneck in the data moving between HBM, DRAM and SSDs

SK hynix's Ahn Hyun projects AI infrastructure performance needs will grow roughly 19 to 20 times by 2030, citing inference demand. The company's answer is to tune the whole memory hierarchy, from HBM down to SSDs, and to add 3D stacked DRAM and a NAND-based HBF.

Perspective Coverage

3 publishers
Builder
Builder 38%
Operator
Operator 34%
Investor
Investor 28%

Reality

Evidence30
Adoption
Not scored
Hype gap+20
Incentives65
Confidence70
investOne report1 publisher

Cerebras sinks below its $185 IPO price after its October 1 lockup expiry

Cerebras shares traded around $165 in early October, below the $185 IPO price, after the company's lockup expired on October 1. At that level, buyers who paid the $350 opening price in May hold less than half of what they spent.

Reality

Evidence52
Adoption45
Hype gap+15
Incentives
Not scored
Confidence50
buildOne report1 publisher

Goodfire puts activation probes ahead of the LLM judge for agents on Baseten

Goodfire launched agent monitors on Baseten that it says watched about 1,500 Kimi K3 sessions for $51, against $233 for a model reviewing every step. Every figure so far comes from Goodfire's own tests, so operators still have to measure misses and false alarms on their own traffic.

Reality

Evidence40
Adoption15
Hype gap+25
Incentives65
Confidence55
buildConfirmed2 publishers

Broadcom reportedly arranges more than $50B in lender financing for OpenAI's Jalapeno chip

Broadcom is arranging more than $50 billion in financing for OpenAI's Jalapeno chip, according to Bloomberg. The talks have not settled a borrower or security, so the risk of a chip still in development could sit with OpenAI, Broadcom or the lenders.

Perspective Coverage

4 publishers
Builder
Builder 16%
Operator
Operator 21%
Investor
Investor 63%

Reality

Evidence45
Adoption10
Hype gap+25
Incentives55
Confidence50
buildOne report1 publisher

OpenAI books 750MW of Cerebras inference capacity in three 250MW segments through 2028

OpenAI has contracted 750MW of Cerebras wafer-scale inference capacity, delivered in three 250MW segments by the end of 2028. The dates are contractual, but neither company has published pricing, latency figures or a plan for which customers get access.

Publishers:dev.to

Reality

Evidence45
Adoption25
Hype gap+10
Incentives45
Confidence40
productOne report1 publisher

LlamaIndex builds document models on rented GPUs that have to absorb its customers' paperwork spikes

LlamaIndex runs a workload that is 75% inference on rented GPUs, parsing millions of document pages a day that arrive in bursts. Its CEO's account at the Fully Connected event describes a buyer for whom guaranteed capacity matters more than owning hardware.

Publishers:siliconangle.com

Reality

Evidence35
Adoption30
Hype gap+30
Incentives80
Confidence40
investOne report1 publisher

GMI Cloud borrows two-thirds of its $668 million raise to expand GPU capacity

GMI Cloud raised $668 million, two-thirds of it through a $445 million credit facility led by CTBC and the rest as a $223 million Series B. Repaying debt on that scale depends on the more than $600 million in contracted annual revenue GMI reports and on how quickly it puts GPUs into production to serve it.

Publishers:pulse2.com

Reality

Evidence35
Adoption50
Hype gap+20
Incentives65
Confidence40
productOne report1 publisher

CoreWeave rents out Vera Rubin NVL72 racks backed by one customer's 4.8x throughput claim

CoreWeave now rents Nvidia's Vera Rubin NVL72 racks, with first customer Cognition reporting up to 4.8x the token throughput on SWE-2 inference. That figure is a best case from a single inference workload, and it is the only customer result behind the launch.

Reality

Evidence35
Adoption22
Hype gap+35
Incentives75
Confidence45
investConfirmed5 publishers

Groq raises $350M at half its 2025 price, with Nvidia on the cap table

The inference challenger now sells Nvidia clusters and takes Nvidia money. Buyers who priced in a non-Nvidia alternative should re-read their renewal terms.

Perspective Coverage

5 publishers
Builder
Builder 19%
Operator
Operator 33%
Investor
Investor 48%

Reality

Evidence68
Adoption40
Hype gap+25
Incentives72
Confidence65
productOne report1 publisher

CoreWeave's current customers all plan to expand their use, a 13-buyer study finds

All seven CoreWeave customers running workloads in a 13-buyer SiliconANGLE study expect to grow their spending or workloads. Three of the ten buyers past evaluation are not running on it, and hyperscalers still hold the rest of the application stack.

Publishers:siliconangle.com

Reality

Evidence40
Adoption45
Hype gap+20
Incentives
Not scored
Confidence50
investConfirmed7 publishers

OpenAI's first chip beat merchant silicon in 16 months. The moat was never the fab

Jalapeno's lead is measured against last generation and the volumes are tiny, but a first-pass ASIC clearing Nvidia, AMD and Google parts reprices the design barrier, not the supply chain.

Perspective Coverage

7 publishers
Builder
Builder 30%
Operator
Operator 24%
Investor
Investor 46%

Reality

Evidence50
Adoption8
Hype gap+40
Incentives65
Confidence55
investConfirmed9 publishers

Six customers underwrite Broadcom's $230bn AI chip target for 2028

Custom silicon made up 73% of the $16.7bn AI quarter. Hock Tan says that line doubles twice by fiscal 2028. He has already secured wafers and memory through 2028 to serve promises only six buyers have made.

Perspective Coverage

9 publishers
Builder
Builder 14%
Operator
Operator 21%
Investor
Investor 65%

Reality

Evidence74
Adoption72
Hype gap+22
Incentives62
Confidence70
productConfirmed4 publishers

Crusoe raises $3.9bn to build the five gigawatts it has already sold

Crusoe's Series F values the Denver data centre builder at $30.9bn and pays for capacity customers have already contracted. One gigawatt is running, and the company now makes its own circuit breakers and industrial controls.

Perspective Coverage

4 publishers
Builder
Builder 25%
Operator
Operator 34%
Investor
Investor 41%

Reality

Evidence55
Adoption55
Hype gap+25
Incentives70
Confidence60
investOne report1 publisher

Broadcom's $230bn AI target needs its quarterly chip sales to more than triple

Hock Tan told CNBC that Broadcom's $115bn and $230bn AI chip targets for fiscal 2027 and 2028 are intact after the stock fell 4.8%, and the company has lifted this year's AI guide to $58bn from $56bn.

Reality

Evidence38
Adoption52
Hype gap+34
Incentives78
Confidence44
productOne report1 publisher

An AI chip insider argues inference should get cheap enough that teams stop counting tokens

A SiliconAngle opinion piece written from inside the AI semiconductor business says commodity inference would grow the market. Teams that ration tokens today still have to decide whether their caps are configuration or architecture.

Publishers:siliconangle.com

Reality

Evidence18
Adoption
Not scored
Hype gap+42
Incentives80
Confidence38
buildOne report1 publisher

Anthropic and OpenAI are shopping for 20MW sites at big-campus prices per megawatt

CNBC reports both labs have sounded out powered sites of roughly 20 to 30MW in the UK, the Nordics and the US. A hall that small is usable because inference requests can be routed independently.

Publishers:runtimewire.com

Reality

Evidence58
Adoption38
Hype gap+12
Incentives65
Confidence52
productOne report1 publisher

Qualcomm puts tokens per watt at the centre of its AI infrastructure pitch

Qualcomm's Tony Pialis told the AI Infra Summit in Santa Clara that tokens per watt is the new key metric in the AI war. The vendors in the room disagreed about which memory architecture gets them there.

Publishers:siliconangle.com

Reality

Evidence46
Adoption34
Hype gap+33
Incentives82
Confidence52

Earlier coverage

  1. Hock Tan defends Broadcom's $230 billion fiscal 2028 AI forecast on inference demand

    Invest · September 14, 2026 · Confirmed2 publishers

  2. Qualcomm issues Amazon warrants on 25 million shares to co-design AWS inference chips

    Product · September 11, 2026 · Confirmed2 publishers