Cloudflare goes to war with Microsoft, Google, and OpenAI’s bots, with blanket free tools to block all crawlers

Declare AIdependence, says Cloudflare, as it provides tools to prevent generative AI systems from stealing info.

When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

What you need to know

Last week,Microsoft’s AI chief said that public content on the open web was “freeware,” giving the trillion dollar corporation free license to steal any and all content you’ve publishedon the web to power its premium products. The backlash to the gaffe was significant, and has served as a claxon of sorts for web content providers to reconsider their relationship with companies like Microsoft, who seek to profit from content creators' hard work while giving literally nothing back in return. Cloudflare may have also just handed those same creators a much important defensive weapon in the fight back.

Cloudflare is a global internet service and hosting company, powering roughly 20% of all web traffic. Offering things like DDoS protection from attacks and bot verification checks on websites, Cloudflare has been instrumental in improving the general quality of the world wide web, using its massive server infrastructure as a vast security layer for companies of all shapes and sizes.

Yesterday, the firm announced a new feature it will begin rolling out to all users, even those on its free tier, designed to combat generative AI.

To help preserve a safe Internet for content creators, we’ve just launched a brand new “easy button” to block all AI bots. It’s available for all customers, including those on our free tier. Read our blog post for more details: https://t.co/csWFFgqbKMJuly 3, 2024

Declare “AIndependence,” Cloudflare says on itsblog. Its new system will allow users to opt-in to block AI bots and crawlers from accessing websites, effectively preventing Microsoft, Google, OpenAI, and others from stealing web content for free.

After surveying its users, Cloudflare shared data that over 80% of its customers wanted the ability to shut Microsoft out from stealing their content. “We hear clearly that customers don’t want AI bots visiting their websites, and especially those that do so dishonestly.” Cloudflare continued, “To help, we’ve added a brand new one-click to block all AI bots. It’s available for all customers, including those on the free tier.”

Generative AI training content is becoming lucrative and valuable to companies like Google and Microsoft. Google reportedly paid upwards of $60 million dollars for access to all of reddit’s content for training its models, which also hilarious resultedin sarcasm and trolling appearing in Google search results.

Can Microsoft discover a healthy balance?

I’ve written before howit should be in Google and Microsoft’s interests to create a healthy, symbiotic relationship between human creators and its generative AIefforts. Generative AI undoubtedly has some role to play in the future of tech, but I feel like companies are still struggling with what exactly that looks like for customers.   Right now, generative AI seems best used for the most basic writing tasks, such as producing formal emails or summarising long form text. Even then, it has its issues doing even basic things when you dig into it. I’ve found that it often simply erodes productivity rather than enhances it, given that you have to double check everything AI does to avoid their “hallucinations.”

Get the Windows Central Newsletter

All the latest news, reviews, and guides for Windows and Xbox diehards.

AI is incredibly expensive to run, too.AI queries are battering Google’s emission reduction efforts, and I doubt Microsoft is faring much better here, either. Even when you disregard the climatological impacts, the business model doesn’t exactly seem to work today either. Microsoft gives awayCopilotfor free, and I’m not sure why I’d ever pay for it.

RELATED:Why Microsoft won’t be the company that mainstreams AI with consumers

The low-hanging-fruit feature that Google and Microsoft have opted for very quickly is search summaries. We produce thousands of guides here at Windows Central, andMicrosoft Copilotwill now simply take the content from the article and reproduce it, depriving us of traffic, and thus income. That’s bad for us, but it’s also bad for Microsoft and Google. If human content creators can no longer effectively monetize and earn a living, increasingly large swaths of the internet will become AI-generated. Much like JPEG compression, the quality of the content will decrease as AI starts learning from other AI, rather than human creators. Since ultimately, AI doesn’t “understand” the content its reproducing, and can only infer context by doing comparisons against human content. This phenomenon is called model collapse, and it’s a real concern among serious AI scientists. But right now, all Google, Microsoft, and others are thinking about is getting ahead.

For this kind of tech to actually proliferate, it still needs human intervention. The alarm Microsoft’s AI chief generated with his irresponsible “freeware” comments has contributed to the on-going backlash. And now, companies like Cloudflare are stepping up to help fight back. It won’t be long until others follow, and Microsoft may have to actually reckon with its cavalier attitude towards its industrial-scale content theft.

Jez Corden is the Executive Editor at Windows Central, focusing primarily on all things Xbox and gaming. Jez is known for breaking exclusive news and analysis as relates to the Microsoft ecosystem while being powered by tea. Follow onTwitter (X)andThreads, and listen to hisXB2 Podcast, all about, you guessed it, Xbox!