Skip to content

Topic

AI inference hardware

Purpose-built accelerators and rack-scale systems targeted at inference rather than training workloads.

Current stories

investConfirmed6 publishers

Former Groq engineers ask Delaware to treat Nvidia's $17 billion license as a sale of the company

Two former Groq engineers sued the startup's board in Delaware, saying Nvidia's $20 billion license-and-hire deal was a sale that needed a stockholder vote. A ruling for them would extend that vote to other deals built the same way.

Perspective Coverage

7 publishers
Builder
Builder 19%
Operator
Operator 33%
Investor
Investor 48%

Reality

Evidence60
Adoption35
Hype gap+20
Incentives70
Confidence58
investConfirmed7 publishers

Lisa Su courts 13 Korean AI firms to build inference systems around AMD chips

AMD's Lisa Su met 13 Korean AI firms in Seoul to discuss pairing AMD CPUs and GPUs with local inference chips. So far the plan is talks and planned demonstrations, and AMD's firmer Korean ties from the same trip run through memory supply.

Perspective Coverage

7 publishers
Builder
Builder 19%
Operator
Operator 38%
Investor
Investor 43%

Reality

Evidence55
Adoption15
Hype gap+30
Incentives60
Confidence55
buildOne report1 publisher

d-Matrix models 100 TB/s per card for Raptor, a stacked-DRAM chip due in late 2027

d-Matrix's ISCA paper models up to 100 TB/s per card for Raptor, a stacked-DRAM inference accelerator. The figures come from modeling and the first cards are expected in Nvidia MGX racks in late 2027, so they have no bearing on inference hardware bought in the next year.

Publishers:runtimewire.com

Reality

Evidence38
Adoption4
Hype gap+35
Incentives70
Confidence45
investConfirmed6 publishers

Volantis raises $88 million to link one GPU to 220 memory chips with light

Volantis, a San Francisco chip startup, closed an $88 million Series A to replace the electrical links between AI chips and memory with optical ones. Its A-1 system, promised to customers in 2027, puts a date on the bet that memory, more than compute, now limits AI inference.

Perspective Coverage

6 publishers
Builder
Builder 41%
Operator
Operator 16%
Investor
Investor 43%

Reality

Evidence40
Adoption5
Hype gap+35
Incentives60
Confidence60
buildOne report1 publisher

Cerebras measures its 5x mixed-chip inference gain against its own system count

Cerebras says splitting inference stages across chip types gave 5x more throughput from the same number of its systems without slowing token generation. Because the count covers only Cerebras hardware, the figure does not yet show what a mixed-chip fleet costs per unit of work.

Publishers:runtimewire.com

Reality

Evidence30
Adoption20
Hype gap+30
Incentives75
Confidence40
buildConfirmed2 publishers

Cognition reports 4.8x token throughput on Nvidia Vera Rubin in its own coding-agent test

Cognition says its SWE-2 model produced up to 4.8 times more total token throughput on Nvidia's Vera Rubin NVL72 than on GB200, in a test on CoreWeave. The company-run result points to more agent capacity per system, but whether long coding tasks get cheaper depends on what the new systems cost per hour.

Reality

Evidence40
Adoption30
Hype gap+35
Incentives85
Confidence60
buildOne report1 publisher

d-Matrix's CTO labels Raptor's 1,000-tokens-per-second figure a simulation

d-Matrix CTO Sudeep Bhoja says Raptor's 1,000 tokens per second per user comes from a simulated 72-card system, with full racks not due until Q4 2027. Whole-system speed, power and cost per request are still unmeasured, so buyers are working from early silicon tests and simulations.

Reality

Evidence30
Adoption
Not scored
Hype gap+35
Incentives70
Confidence40
buildOne report1 publisher

OpenAI's hardware head calls Jalapeno's nine months from RTL to tapeout the new baseline

OpenAI took its Jalapeno chip from RTL to tapeout in nine months with its own AI models, against a norm its hardware head puts at 18 to 24 months. The count stops at first tapeout, before a B0 stepping reported to be up to 25% more efficient per watt.

Publishers:tomshardware.com

Reality

Evidence35
Adoption20
Hype gap+35
Incentives65
Confidence40
productConfirmed4 publishers

Etched has silicon, a customer rack running, and $1bn raised in 26 days

The first customer is also the lead investor, which blunts the validation. What is left is a non-Nvidia inference system buyers can benchmark instead of read about.

Perspective Coverage

4 publishers
Builder
Builder 34%
Operator
Operator 24%
Investor
Investor 42%

Reality

Evidence55
Adoption20
Hype gap+40
Incentives70
Confidence60
investConfirmed8 publishers

Marvell just put Google's purchase orders on its cap table

A warrant for 58.97 million shares vests one $500 million tranche of custom-chip revenue at a time, turning a supply agreement into an equity schedule that runs to 2033.

Perspective Coverage

8 publishers
Builder
Builder 14%
Operator
Operator 17%
Investor
Investor 69%

Reality

Evidence82
Adoption30
Hype gap+30
Incentives55
Confidence74
investConfirmed4 publishers

Positron's second tranche pays a 16 per cent step-up inside the same announcement

The $375m Series C priced Positron at $3.5bn before the money, and the up-to-$500m follow-on has to be sitting at $4.5bn pre-money to reach the $5bn headline, which only holds if every dollar of it is drawn.

Perspective Coverage

4 publishers
Builder
Builder 34%
Operator
Operator 21%
Investor
Investor 45%

Reality

Evidence55
Adoption30
Hype gap+35
Incentives70
Confidence60
productConfirmed2 publishers

Apple weighs wiring an M8 Ultra AI server together with Nvidia's NVLink Fusion

The Information reports two configurations, one with two chips and one with four, aimed at developers, businesses and governments, with a possible 2029 launch. Apple already builds servers for Private Cloud Compute and has turned partners away from them.

Perspective Coverage

7 publishers
Builder
Builder 32%
Operator
Operator 32%
Investor
Investor 36%

Reality

Evidence35
Adoption15
Hype gap+20
Incentives55
Confidence40
buildOne report1 publisher

Apple's reported M8 Ultra server would pick up NVLink where UltraFusion stops

The Information reports Apple is weighing Nvidia's NVLink Fusion for an inference server built on two or four M8 Ultra chips in 2029. The engineering question is which part of that package would join the NVLink domain.

Perspective Coverage

3 publishers
Builder
Builder 37%
Operator
Operator 23%
Investor
Investor 40%

Reality

Evidence30
Adoption18
Hype gap+42
Incentives55
Confidence50
buildOne report1 publisher

Taiwan's pre-listing market prices PieceMakers at NT$55bn ahead of its first volume program

A Nanya-backed designer says DRAM hybrid-bonded to the processor belongs between on-die SRAM and HBM for inference. Design fees are almost 40 percent of its first-half revenue, and the first volume program is not booked before 2027.

Publishers:tomshardware.com

Reality

Evidence62
Adoption25
Hype gap+32
Incentives70
Confidence58
scienceOne report1 publisher

Positron raises $875M for an inference chip built around commodity LPDDR5X instead of HBM

The Reno startup says its Asimov ASIC carries 288 GB of on-package LPDDR5X and realizes more than 90% of its memory bandwidth. Both figures come from the company, and no independent benchmark accompanies them.

Publishers:forkast.news

Reality

Evidence26
Adoption5
Hype gap+58
Incentives74
Confidence54
productOne report1 publisher

Positron bets $875m that smartphone memory can carry inference

Positron is now worth $5 billion on a design that swaps HBM for the memory that ships in phones, but Asimov has not taped out, mass production is set for the second half of 2027, and the 26x tokens-per-dollar figure comes from simulation.

Publishers:siliconangle.com

Reality

Evidence28
Adoption4
Hype gap+58
Incentives78
Confidence55
investOne report1 publisher

DOJ weighs Nvidia's $17bn Groq licence against a $119.5m notification threshold

The remedy people familiar with the probe describe is a penalty rather than an unwind, and nobody has put a figure on it, which leaves the cost of routing a $17bn deal around merger review still blank.

Publishers:techtimes.com

Reality

Evidence45
Adoption68
Hype gap+22
Incentives62
Confidence52

Earlier coverage