For the first time in the history of the public internet, machines generate more traffic to websites than people do. New data from Cloudflare puts automated requests at 57.5% of all web activity, against 42.5% from human visitors — a crossover that Cloudflare’s own CEO had predicted wouldn’t happen until 2027.
The shift is being driven by agentic AI: software that browses and retrieves web pages on behalf of assistants like ChatGPT and Gemini, often visiting thousands of pages in the time it would take a person to open a handful of browser tabs. Cloudflare’s crawler data shows the composition of that traffic changing too. As of June 2026, 52% of crawler requests exist purely to feed AI training systems, up from 22% just over a year earlier. Pure search indexing, the kind that eventually sends a visitor back to the source, now makes up a shrinking share of the total.
The Cost to Publishers
The practical effect is a growing gap between how much a site is crawled and how much it’s actually visited. Content gets pulled into an AI system’s answer, but the reader never clicks through to the page it came from. Cloudflare reports that publishers in the most heavily crawled categories — including software, IT services, and finance — have seen human traffic fall by as much as 40% in under a year.
The behavioral shift behind that number is stark: Cloudflare estimates that of every hour people spend online researching something, only about 15 minutes is now spent on the open web itself. The rest is absorbed into AI-generated answers that never route a visit back to the original publisher.
Cloudflare’s Response
Cloudflare has started giving publishers more granular tools to manage the shift. Crawlers are now split into three categories — Search, Agent, and Training — so site owners can block or charge each independently rather than treating all bots the same way. New default settings, effective September 15, will block Training and Agent bots on ad-supported pages unless a publisher opts out, a change broad enough to catch even Googlebot. The company has also opened a waitlist for a Monetization Gateway, aimed at letting publishers charge AI companies directly for access to their content.
Earlier steps have already shown measurable effects: after Cloudflare introduced a one-click AI-crawler block last year, more than a million customers enabled it, and traffic from ByteDance’s Bytespider crawler fell more than 71% across Cloudflare’s network.
The open question is whether these tools can restore the old exchange — content for traffic — or whether the web is settling into a new model where content is consumed by machines on behalf of humans, with far less of the reciprocity that funded publishing for the last three decades.
- The Bilbao Effect, 500 Years Early: Jerónimos Monastery And The Guggenheim
- Google Frozen v2 AI Chip Could Deliver 10x Efficiency Gains Over Current TPUs
- Israel Says Iran Moved Thousands of Centrifuges to Pickaxe Mountain After 12-Day War
- Tencent in Talks to Acquire SuperPlay From Playtika in Deal Valued at Up to $1.5 Billion
- Empirical Security Raises $25 Million Series A to Expand AI-Driven Threat Prediction
- Tempus AI to Acquire Personalis for $16.25 Per Share in $1.5 Billion MRD Deal
- Unity (NYSE: U) Bets on Coding Agents With Unity 7: Does the Roadmap Move the Stock?
- Unity Unveils Unity 7 With Native AI Agent Support and a CoreCLR Performance Overhaul
- South Korean Retail Investors Face 70% Losses as Leveraged Chip ETFs Crash
- South Korea’s July Chip Exports Surge 180.6% as AI Supercycle Accelerates