Cloudflare's new AI traffic options for customers
The deal between crawlers and website owners that had held up for 30 years — we crawl you, and you get referrals — was no longer true. AI was taking everything and sending back nothing, presenting an existential threat to website owners. And so we launched a one-click "Block AI Bots" option, along with a Pay-Per-Crawl marketplace.A lot has changed in a year.
- ▪The deal between crawlers and website owners that had held up for 30 years — we crawl you, and you get referrals — was no longer true.
- ▪AI was taking everything and sending back nothing, presenting an existential threat to website owners.
- ▪And so we launched a one-click "Block AI Bots" option, along with a Pay-Per-Crawl marketplace.A lot has changed in a year.
Hacker News (AI / LLM) files mainly under ai. We currently carry 2,301 of its stories.
Opening excerpt (first ~120 words) tap to expand
BlogAIAI BotsBot Management+2Show 2 more tags5 TagsShow 5 tagsSelected TagsAIAI BotsBot ManagementContent Independence DayCrawler HintsAll tagsMatching tagsNo tags found1.1.1.12FAAbuseAccessAccess Control Lists (ACLs)AccessibilityAccount TakeoverAcquisitionsAddressingAdvanced Certificate ManagerAdvanced DDoSAdvertisingAegisAfricaAfroflareAgent ReadinessAgentsAgents WeekAIAI BotsAI GatewayAI SearchAI-SPMAI WAFAI WeekAlertmanagerAlways OnlineAMDAMPAnalyticsAnonymousAnti MalwareAnycastAPIAPI GatewayAPI SecurityAPI ShieldAPJCAppleApplication SecurityApplication ServicesArea 1 SecurityArgo Smart RoutingASCIIAsiaAthenian ProjectAtlassianAttacksAudit LogsAustinAustraliaAuthenticationAuthyAutomatic HTTPSAutomatic Platform OptimizationAutomationAutoMinifyAuto RagAwardsAWSBaiduBandwidth…
Excerpt limited to ~120 words for fair-use compliance. The full article is at The Cloudflare Blog.
Discussion
0 commentsMore from The Cloudflare Blog
