buildOne report1 publisher Cloudflare starts measuring Verified bots against robots.txt and 429 responses over time
Cloudflare has introduced a system that scores Verified bots on robots.txt compliance and on whether they slow down after an HTTP 429. It reports the result as a trust band, and Cloudflare says it will share scores on Radar later.
Reality
- Evidence35
- Adoption10
- Hype gap+10
- Incentives60
- Confidence65
buildConfirmed6 publishers Amazon cut Meta's Muse off for arriving unannounced, browsing without identifying itself, and appearing to keep customer credentials. Signed-request verification, already running at Cloudflare and CloudFront, answers the middle objection.
Perspective Coverage
7 publishers
- Builder
- Builder 25%
- Operator
- Operator 36%
- Investor
- Investor 39%
Reality
- Evidence68
- Adoption40
- Hype gap+15
- Incentives78
- Confidence64
buildOne report1 publisher Ahrefs and three rival bot-analytics tools run a Cloudflare Worker on every file a page loads
Ahrefs, Known Agents, Profound and Scrunch run a Worker on every request, so a 20-file page spends 21 of Cloudflare's 100,000 free daily invocations. A site with 10,000 page views a day would need more than twice that.
Reality
- Evidence64
- Adoption
- Not scored
- Hype gap+8
- Incentives
- Not scored
- Confidence58
buildOne report1 publisher AI-agent requests on Cloudflare's network grew more than 1,700% in a year
Cloudflare says AI-agent requests on its network grew more than 1,700% in a year as non-human traffic passed half the total. It now wants sites to identify and price agents as customers, after advising last year that new domains block AI training crawlers.
Reality
- Evidence35
- Adoption50
- Hype gap+25
- Incentives85
- Confidence40
buildOne report1 publisher DataDome's threat research VP puts AI login traffic at 1.08 percent of classified requests
The 51.7 percent in DataDome's 2026 bot report is login's share of five sensitive endpoint types, and Jerome Segura told Lets Data Science the same 313 million requests work out to about 1.08 percent of the 29.02 billion requests the company could classify by path.
Reality
- Evidence58
- Adoption42
- Hype gap+18
- Incentives80
- Confidence55
buildOne report1 publisher Google's ARD spec stops at the endpoint, before the agent proves which workload is calling
The spec's own boundary list says a verified catalog entry establishes which domain claims a capability and what trust material it published. It says nothing about whether the calling process is the agent it names.
Reality
- Evidence45
- Adoption18
- Hype gap−10
- Incentives30
- Confidence45
buildOne report1 publisher Paying for a crawl means retrying every priced URL with a transaction hash
HTTP 402 sat in the spec unused since 1991. The x402 flow makes it callable. The work lands inside your crawler: two origin requests per priced page, a funded wallet on the server's chain, and settlement checked before any retry.
Reality
- Evidence44
- Adoption18
- Hype gap+42
- Incentives58
- Confidence46
buildOne report1 publisher Cloudflare attaches a status and a rejection reason to every BotBase submission
The bot submission form moves into Application Security and grows a history tab, which means the entry describing your crawler becomes a record you are expected to keep matching your traffic rather than a form you filed once.
Reality
- Evidence47
- Adoption20
- Hype gap+12
- Incentives72
- Confidence56