Skip to content

Topic

AI Inference Infrastructure

The hardware, chips, and capacity used to run trained AI models in production, as distinct from infrastructure used to train them.

Current stories

investConfirmed3 publishers

Samsung and AMD demonstrate AI servers borrowing memory from a shared CXL pool

Samsung and AMD demonstrated CXL memory pooling at the 2026 OCP Global Summit, letting AI servers borrow idle capacity from a shared pool. Pooling would move memory sizing from each server to the pool, so orders would follow combined cluster load.

Perspective Coverage

3 publishers
Builder
Builder 40%
Operator
Operator 32%
Investor
Investor 28%

Reality

Evidence35
Adoption
Not scored
Hype gap+20
Incentives70
Confidence65
productOne report1 publisher

Clockwork raises $31 million for software that keeps GPU jobs running through hardware failures

Clockwork Systems raised $31 million and launched TorchSnap, a tool that snapshots distributed AI jobs so they can resume after hardware failures. The customer deployments it cites predate the new feature, so its value depends on how long a buyer's own restarts take.

Publishers:siliconangle.com

Reality

Evidence40
Adoption35
Hype gap+30
Incentives70
Confidence45
investConfirmed3 publishers

Nebius buys idle-GPU startup Inferize for an estimated $100 million to $150 million

Nebius bought Inferize, a 10-month-old Israeli startup with 17 staff, for an estimated $100 million to $150 million, according to Calcalist. The money goes on software that keeps inference GPUs from sitting idle, and Nebius is still adding data center capacity alongside it.

Publishers:calcalistech.com nebius.com runtimewire.com

Perspective Coverage

3 publishers
Builder
Builder 38%
Operator
Operator 32%
Investor
Investor 30%

Reality

Evidence55
Adoption12
Hype gap+30
Incentives70
Confidence60
buildOne report1 publisher

Gimlet plans 100 MW of Cerebras wafer-scale capacity to split inference by phase

Gimlet Labs plans 100 MW of Cerebras-powered inference capacity, pairing wafer-scale chips with GPUs for different phases of each request. For buyers, the quoted 3,000 tokens per second is a company target until measured results appear.

Publishers:runtimewire.com

Reality

Evidence40
Adoption15
Hype gap+40
Incentives70
Confidence40
buildOne report1 publisher

Nvidia's MaxLPS trial at Nscale ran 192 GPUs against a 140-GPU static baseline

Nvidia says DSX MaxLPS power sharing fits up to 40% more GPUs in the same approved power budget, citing an Nscale trial of 192 GPUs against 140. The gain holds only while GPUs rarely peak together, and each operator has to measure that on its own fleet before buying hardware against it.

Reality

Evidence45
Adoption12
Hype gap+25
Incentives80
Confidence50
investOne report1 publisher

DeepSeek weighs 160,000 Huawei Ascend chips for a 1GW inference site in Ulanqab

CEO Liang Wenfeng said in investor discussions that Huawei starts delivering AI training chips around Q4 2026 or Q1 2027, and the Inner Mongolia fleet under consideration is 160 times the Ascend cluster DeepSeek ran in June 2026.

Reality

Evidence27
Adoption20
Hype gap+44
Incentives58
Confidence34
buildOne report1 publisher

China Merchants Bank's 60% token-cost cut outpaces the utilization gain it credits

The bank pooled nearly 10,000 heterogeneous accelerator cards under Kubernetes and reports average utilization rising from 35% to more than 60%. That density supplies 1.71 of the 2.5x drop in cost per million tokens.

Publishers:thenewstack.io

Reality

Evidence40
Adoption55
Hype gap+25
Incentives65
Confidence45
productOne report1 publisher

Huawei's 2035 token forecast assigns more than 90% of the traffic to agents

The company published the number days before its Connect 2026 show, alongside ten directions that name the cluster hardware, memory systems and chip method it sells. The agent share is the part a budget has to absorb.

Publishers:thenextweb.com

Reality

Evidence38
Adoption24
Hype gap+48
Incentives86
Confidence57
buildOne report1 publisher

Open-weight safety pact hints Baseten could test controls in its runtime, but no division of labor is confirmed

Base Labs has signed Hugging Face and Goodfire to publish methods for training, evaluating and monitoring open models, with the check running where the activations are. The partners have not published an evaluation suite yet.

Publishers:runtimewire.com

Reality

Evidence42
Adoption10
Hype gap+38
Incentives72
Confidence48
productOne report1 publisher

Delos Data builds its inference server around an OAM socket buyers fill themselves

The startup has closed more than $100 million three months after debuting its first appliance, a four-accelerator box whose 36 OSFP ports are reported to give it network capacity comparable to Nvidia's Rubin systems.

Publishers:siliconangle.com

Reality

Evidence28
Adoption12
Hype gap+45
Incentives70
Confidence45
buildOne report1 publisher

Lambda ran 19 Blackwell nodes on the power budget provisioned for 16

The one measured number behind NVIDIA's tokens-per-megawatt pitch comes from Lambda's Blackwell cluster, where power reclaimed from static provisioning ran three extra nodes. Amazon's Annapurna Labs and d-Matrix got a line each.

Reality

Evidence33
Adoption45
Hype gap+44
Incentives90
Confidence60
investOne report1 publisher

Xunce's parent guarantees every yuan of a 10 billion yuan compute loan

Shenzhen Xunce told the Hong Kong exchange that a subsidiary will seek up to 10 billion yuan of syndicated loans for an AI inference centre, with the parent guaranteeing the same amount because the project company cannot borrow on its own credit.

Reality

Evidence55
Adoption
Not scored
Hype gap+22
Incentives50
Confidence52
productOne report1 publisher

Arm now sells a chip, and its licensees are buying cores from a competitor

The AGI CPU is a 136-core Neoverse part co-designed with Meta. Arm's pitch to custom-silicon customers used to be that it had nothing of its own to sell.

Reality

Evidence40
Adoption22
Hype gap+32
Incentives72
Confidence45
productOne report1 publisher

I Squared's Saragon buys 53MW of ex-Sprint switch sites and calls it an AI inference platform

A $1bn platform seeded with ten converted Cogent facilities takes about a quarter of Cogent's installed power from ten of its 185 sites. The bet is on interconnection, not slab.

Reality

Evidence45
Adoption30
Hype gap+45
Incentives70
Confidence50
investOne report1 publisher

AMD's Korea test lab is a bet that inference stops being a GPU-only purchase

AMD will open an AI Center of Excellence in Korea by year end to co-test its CPUs and GPUs with domestic NPUs. What is committed so far is engineers and a steering committee, not capital.

Publishers:en.sedaily.com

Reality

Evidence50
Adoption12
Hype gap+18
Incentives65
Confidence45
productOne report1 publisher

Inference cost is now a storage and power problem, and it prices differently per workload

A vendor panel at Supermicro's Open Storage Summit argues GPUs no longer set token cost alone. The useful corollary: chat, batch and agentic work each need their own capacity plan.

Publishers:siliconangle.com

Reality

Evidence28
Adoption18
Hype gap+34
Incentives86
Confidence36

Earlier coverage