The head of the largest digital and print publisher in the U.S. has accused Google of misconduct for crawling its websites to support the search conglomerate’s AI offerings.
Neil Vogel, CEO of People, Inc. (previously Dotdash Meredith), a publisher managing over 40 brands such as People, Food & Wine, Travel + Leisure, Better Homes & Gardens, Real Simple, Southern Living, Allrecipes, and others, stated that Google is not acting fairly since it employs the same bot to index websites for the Google search engine as it does for its AI functionalities.
“Google operates with a single crawler, which means they use that same crawler for their search, where they still send us traffic, and for their AI products, where they appropriate our content,” Vogel remarked at the Fortune Brainstorm Tech conference this week.
He pointed out that three years back, Google Search represented approximately 65% of the company’s traffic, which has now declined to the “high 20s.” (Vogel previously revealed to AdExchanger that Google’s traffic constituted as much as 90% of People Inc.’s web traffic several years ago.)
“I’m not expressing dissatisfaction. We’ve expanded our audience. We’ve increased our revenue,” Vogel informed conference participants. “We’re thriving. What’s inappropriate about this is that: You cannot utilize our content to compete against us.”
Vogel is of the opinion that publishers require greater leverage in the AI age, which is why he believes blocking AI crawlers — automated systems that scan websites to train AI technologies — is essential, as it can compel them into content agreements. His organization, for instance, has an arrangement with OpenAI, which Vogel referred to as a “good actor.”
People Inc. has been utilizing the latest solution from web infrastructure company Cloudflare to prevent unpaid AI crawlers, leading AI entities to approach the publisher with potential content agreements. Although Vogel refrained from naming the companies involved, he mentioned they were “large LLM providers.” No agreements have been reached yet, but Vogel indicated that the company is “much further along” than it was prior to adopting the crawler-blocking solution.
Techcrunch event
San Francisco
|
October 27-29, 2025
However, Vogel highlighted that Google’s crawler cannot be obstructed, as doing so would also hinder the indexing of the publisher’s websites in Google Search, thus cutting off that “20%-ish” of traffic that Google still provides.
“They are aware of this, and they’re not differentiating their crawler. Hence, they are being willfully negligent,” Vogel asserted.
Janice Min, the editor-in-chief and CEO at the newsletter company Ankler Media, concurred, labeling Big Tech firms like Google and Meta as longtime “content thieves.”
“At this moment, I don’t perceive any advantages in collaborating with any AI company,” she stated, adding that her company obstructs AI crawlers.
On the other hand, Cloudflare CEO Matthew Prince, whose firm produces the AI-blocking solution (and who participated on the panel), expressed his belief that there would still be changes in the future regarding the behavior of AI companies. He speculated that such changes could be triggered by new regulations.
The Cloudflare executive also questioned whether legal avenues against the AI companies regarding copyright law, which were established for the pre-AI era, were the appropriate course of action.
“I believe that pursuing that route is a futile effort, as in copyright law, typically, the more derivative something is, the more it is protected under fair use… What these AI companies are doing is actually generating derivatives,” Prince explained. “So if you examine the best legal precedents that have emerged thus far, they have indicated that the use by Anthropic and others — the rationale behind Anthropic’s recent $1.5 billion settlement with all book publishers — was to maintain the favorable copyright ruling they received.”
Prince also claimed that “everything that’s wrong with the world today is, at some level, Google’s responsibility,” since the search giant has taught publishers to prioritize traffic over original content creation, causing publishers like BuzzFeed to write solely for clicks. Nevertheless, he conceded that Google currently finds itself in a challenging position from a competitive perspective.
“Internally, they’re experiencing significant conflicts regarding their actions, and my forecast is that, by this time next year, Google will be compensating content creators for crawling their material and incorporating it into AI models,” he stated.
