In a significant move to promote fairness in the artificial intelligence (AI) sector, Cloudflare’s CEO, Matthew Prince, has urged the United Kingdom’s Competition and Markets Authority (CMA) to implement stricter regulations on Google’s operations. Prince’s primary concern is Google’s practice of using its dominant search engine to bolster its AI initiatives, potentially stifling competition.
Earlier this month, the CMA recognized Google’s substantial and entrenched position in the search and advertising markets, granting the regulator the authority to impose more stringent regulations. These regulations could extend beyond search and ads to encompass Google’s AI Overviews, AI Mode, Discover feed, Top Stories, and News tab.
Speaking at the Bloomberg Tech conference in London, Prince emphasized Cloudflare’s unique position in the industry. We don’t have a direct stake in AI; we’re not an AI company, he stated. However, we serve as a network intermediary, with 80% of AI companies as our customers. This perspective, he believes, allows Cloudflare to offer unbiased recommendations to the CMA.
Prince highlighted a critical issue: Google’s use of its existing web crawler, Googlebot, to collect content for both its search engine and AI products. This dual-purpose approach, he argued, provides Google with an unfair advantage over other AI companies that must negotiate access to content. Google asserts a right to all online content without compensation, leveraging its 27-year history, Prince explained. They use the same crawler for search and AI systems, forcing site owners to opt out of both if they wish to exclude one.
This bundling practice poses a dilemma for website owners, especially media outlets that rely heavily on search traffic. Opting out of Google’s crawler could result in a significant loss of revenue. Prince further noted that blocking Google’s crawler also affects the company’s ad safety mechanisms, disrupting advertisements across various platforms.
The crux of the issue lies in Google’s ability to access content that other AI companies, such as Anthropic, OpenAI, and Perplexity, would need to pay for. Prince warned that this could lead to a market dominated by Google, stifling competition. He advocated for a competitive landscape where numerous AI companies can purchase content from a diverse array of media and small businesses.
Cloudflare has provided the CMA with data illustrating how Google’s crawler operates and the challenges other companies face in replicating its success. Prince’s concerns are echoed by industry leaders like Neil Vogel, CEO of People, Inc., who criticized Google’s practices and highlighted the difficulties media companies face due to the bundling of crawlers.
In response, Google spokesperson Ned Adriance mentioned that site owners can opt out of having their content used for training AI products through Google Extended, without affecting their inclusion in Google Search. However, many media companies prefer to fully opt out of AI features, indicating that the issue remains contentious.
This development underscores the ongoing debate about the balance between technological advancement and fair competition, with industry leaders like Prince advocating for regulatory measures to ensure a level playing field in the AI sector.
