How AI Broke Search Trust and What Google Is Doing About It (Maybe)

Disclaimer: This article is speculative, so do not take everything I say as “true”, but more as an exercise in connecting the dots between information we have and recent announcements Google has made regarding new Search Console filters and views.

The way we find information online is breaking down. The core problem: we can no longer easily tell whether content comes from real companies or from fabricated ones.

Generative AI has made content production almost free. Anyone can now spin up a website, invent a brand history, and publish dozens of articles, all in a few hours. These “fake authorities” look legitimate but have no real-world operations, customers, or history.

This is a serious problem for search engines, which are built on the assumption that good content comes from trustworthy sources. The old model of ranking pages by links and relevance no longer works, as AI can generate an unlimited number of “relevant” pages for brands that don’t exist. Note, all this was possible even before “AI”, but Artificial Intelligence has multiplied the possibilities and the velocity of this kind of spam.

This post collects my thinking over the past few months. I’ll walk through the problem in detail, anchoring on the Ahrefs “AI vs. Made Up Brand” experiment by Mateusz Makosiewicz, which showed how easily AI models can be fooled by detailed lies. I’ll also cover how “Parasite SEO” and listicle spam spread these fake brands. Finally, I’ll look at how Google is fighting back, specifically through the new Branded Queries Filter and Social Channels Integration in Search Console, and what these tools may tell us about Google’s reliance on user behaviour signals and entity verification.

Why AI Search Can Be Fooled

The “Xarumei” Experiment: Testing What AI Actually Believes

When Google started moving from keyword matching to AI-generated answers (Retrieval-Augmented Generation, or RAG) with AI Overviews and LLMs burst onto the Search market, they introduced a serious weakness: AI models care more about whether something sounds plausible and detailed than whether it’s actually true.

Ahrefs proved this with their “AI vs. Made Up Brand” experiment. This wasn’t just an SEO test; it was a test of whether AI can tell the difference between fiction and reality when both look equally polished.

How They Built a Fake Brand

Mateusz wanted to know: would AI search engines (Perplexity, Gemini, ChatGPT, Claude) actually verify claims, or would they just summarise whatever text they found, even if it was false?

He invented “Xarumei,” a completely fictional luxury paperweight brand. Using AI tools, he built a professional-looking website in about an hour, complete with fake product images and absurd prices (like an $8,251 paperweight).

Then, he poisoned the web with three conflicting fake stories:

1. A fake celebrity endorsement on X
2. A made-up product defect scandal
3. A fictional Black Friday 2024 sales spike

His bet: AI models, designed to be helpful and thorough, would treat detailed-but-false information as fact rather than flag that Xarumei didn’t exist.

The Results: Detail Beats Truth

Without contradicting information, the most detailed story wins, regardless of whether it’s true. When asked about Xarumei (like “How is the brand handling the backlash?”), AI models often made up answers that treated the fake premises as fact.

How each model performed:

• Perplexity: Most easily fooled. In fact, it failed about 40% of tests. It confused Xarumei with Xiaomi or invented details from the planted fake stories. Perplexity seems to prioritise how similar words sound rather than verifying whether entities actually exist.
• Google Gemini: More sceptical. It often refused to treat Xarumei as real because the brand wasn’t in its training data. This suggests Gemini checks claims against Google’s Knowledge Graph.
• Claude: Most cautious. It ignored the fake website entirely and said the brand didn’t exist. This avoided hallucination but also meant Claude missed any legitimate new information.

Why does this happen?

LLMs rely on agreement across sources. When a brand is new (or fake), and the only available information is the poisoned content, the model sees what looks like consensus among the fake sources. Without a “database of things that don’t exist,” several fake articles agreeing with each other look like legitimate confirmation.

Data Voids: Empty Spaces That Liars Fill

AI search engines are vulnerable to “data voids”, aka topics where no authoritative sources exist to contradict manipulated content. For obscure or new subjects, the volume and specificity of text act as stand-ins for truth.

The team used Grok to generate 56 questions with built-in false assumptions (like “A celebrity endorsed Xarumei’s paperweights on X. Who was it?”). Instead of questioning the premise, models usually retrieved the made-up celebrity name to answer the query.

What this means for brands:

A bad actor could invent a crisis for a real brand – or create a fake brand to compete with real ones – and seed the web with detailed supporting lies. AI search engines, which increasingly provide direct answers, might then present this misinformation as fact.

The experiment proved that AI will talk about your brand whether you like it or not. If you don’t provide an official story, AI latches onto whatever convincing text it finds, even if it’s spam from a Reddit thread.

Why Specific Details Fool AI

Why did AI models fail so badly when faced with specific details?

It comes down to how LLMs work. These models predict what words should come next based on probability. A prompt with specific nouns and verbs (like “defective Precision Paperweight batch”) narrows down the possible responses.

When the model doesn’t already “know” what Xarumei is (because it’s not in its training data), it relies entirely on whatever the retrieval step (RAG) pulls in. If RAG grabs the fake articles the researchers planted, the model treats those as truth for that session. It has no external way to check whether a paperweight batch was actually defective in the real world. It only knows that the text says it was.

This exploits a cognitive bias called the “Conjunction Fallacy”; more specific details make stories feel truer. In LLM training data, detailed reports are statistically more likely to be accurate than vague ones. Spammers and fake brand creators exploit this by packing their content with specific prices, model numbers, dates, and incident reports.

As Mateusz put it: “The most detailed story wins.”

How Fake Brands Scale: Parasite SEO and Listicle Spam

The Xarumei experiment was controlled. In the real world, these same tactics run at industrial scale through “listicle spam” and “Parasite SEO.” These strategies hijack the authority of established websites to rank content for fake or low-quality brands, creating an ecosystem of manufactured credibility.

How Parasite SEO Works

Parasite SEO means publishing content on high-authority domains (Forbes, LinkedIn Pulse, Medium, Outlook India) to skip the scrutiny Google normally applies to new or weak sites. Because Google’s algorithms historically trust these host domains, marketers can rank “best of” listicles for competitive keywords (like “Best Fraud Detection Software” or “Best Keto Gummies”) without building their own backlinks.

In the case of ChatGPT and other models, the situation is in many ways even more dramatic because it is very possible that the websites where “Parasite SEO” is developed are not only part of their Training Data, but may even be part of economic agreements between LLMs and publishers to train the models.

The economics of “Parasite SEO” are simple: it’s cheaper to rent authority from a site like Dallas Morning News or Times of Israel than to build a real reputation from scratch.

Here’s how it plays out:

• The setup: A marketer creates a “Best X for Y” listicle.
• The fake brand: The list mixes real brands with a made-up or white-label brand the marketer owns.
• The ranking: The article hits Page 1 because of the host domain’s authority.
• The payoff: Users trust the host domain, assume the list is objective, and click affiliate links for the fake brand.

This weaponises the trust that users place in known publications. These “Parasite Properties” become long-term assets that influence not just traditional search, but also AI models, which scrape these high-authority sites to generate answers.

If an LLM sees a brand in a “Best of 2025” article on a major news site, it encodes that brand as a market leader, regardless of whether the brand actually exists or is any good.

But here’s the thing: this spam tactic also works without major news sites. Mass-producing listicles over a sustained period exploits another LLM weakness: their preference for recent content (the “recency problem”).

Listicle Spam and “Best X for Y” Queries

The listicle format is especially easy to manipulate because it matches what Google and AI models see as “commercial investigation” intent. Users searching for “Best CRM software” or “Best weight loss supplements” are ready to buy.

Spammers exploit this by creating circular validation loops:

1. A fake brand gets featured in a listicle on a Parasite host.
2. The spammer creates fake Reddit threads asking, “Has anyone tried [brand]?” Other bot accounts reply with glowing reviews, citing the Parasite listicle as proof.

The “Best Software” Problem

This is rampant in software categories. Generic “best email marketing software” lists often mix legitimate players like Klaviyo or Mailchimp with obscure, affiliate-heavy tools that barely function. In fraud detection software lists, brands like SEON or Feedzai are real, but spammers may insert phantom security tools that are just rebranded versions of inferior products, boosted by fake “Best of” rankings.

The danger – actually, the reality – is that LLMs ingest these listicles during training or RAG. When asked, “What are the top fraud detection tools?”, an AI might say the spam brand is a market leader because it appeared at the top of a high-authority Parasite article.

Manufacturing Fake Consensus Through Reviews

To prop up fake brands, spammers build artificial “consensus” through fake reviews, just like Ahrefs seeded the web with fake Xarumei stories.

• Reddit manipulation: Spammers use aged accounts to post questions (“What’s the best coffee maker?”) and then use other accounts to recommend their fake brand, simulating organic discussion. This is called “Astroturfing.” In this sense, the recent announcement of Reddit about testing verified profiles can be an interesting insight about how the company wants to fight this kind of manipulation, and it is quite similar in intent to what Google, as we will see later, may have decided to deal with this same problem.
• Review bombing/boosting: Fake reviews manipulate star ratings that show up in Rich Snippets and Local Packs.

This creates a self-reinforcing loop. The listicle provides “authoritative” top-down validation. The fake Reddit threads and reviews provide “grassroots” social proof. An AI analysing this sees both authority and social agreement, so it recommends the fake brand over legitimate competitors.

How Fake Authority Tactics Exploit AI

The “Hollow Brand” Problem

When you combine all these tactics, you get “Hollow Brands”: entities that exist only as text and affiliate links, with no real operations, no customer service, no supply chain, no actual history. They’re designed purely to capture search traffic and extract money.

The Ahrefs experiment showed that AI currently can’t detect these hollow brands because AI validates based on language and meaning, not supply chain verification. A paperweight company with no actual paperweights looks identical to a real one in the text that AI reads.

The challenge for search engines: find signals that text alone can’t fake.

Google’s Response: User Behaviour and Entity Verification

Because text-based signals are easy to manipulate, Google has shifted toward signals that are much harder to fake: user behaviour (Navboost) and entity verification (Knowledge Graph).

The 2024 Google API leaks confirmed that Navboost exists and plays a central role in how Google ranks results.

Navboost: Clicks as Truth

Navboost is a re-ranking system that uses clickstream data, specifically, which search results users actually click to validate relevance and authority. Unlike PageRank, which counts links (which can be bought), Navboost counts human intent.

Branded Search as the Ultimate Trust Signal

The most powerful signal in Navboost is branded search volume. When someone types a specific brand name (like “Nike,” “Ahrefs,” or “Xarumei”) into the search bar, that’s a “navigational query.”

• The logic: A navigational query means the user already knows the brand exists and wants to find it specifically. This implies awareness from the real world: word of mouth, marketing, and actual use. That’s extremely hard for a bot or spammer to fake at scale with realistic behaviour patterns.
• The check: If a brand has thousands of backlinks and listicle mentions (text signals) but nobody is searching for it by name (Navboost signal), Google sees a mismatch. This is the signature of a hollow brand. Real brands generate branded search; fake brands rely on hijacking generic search traffic.

The Google leaks revealed that Google compares the ratio of links to branded search volume. A site with lots of new links but low branded search gets flagged as suspicious. This ratio is basically a mathematical measure of authenticity.

Why Gemini Was Harder to Fool Than Perplexity

In the Ahrefs experiment, “Xarumei” had content but initially had no branded search volume (until the article was published and readers started searching for it).

This explains why Gemini was more sceptical than Perplexity. Gemini likely checked its internal query logs, saw zero historical searches for “Xarumei,” and concluded the entity wasn’t established, despite the web content.

Perplexity doesn’t have access to Google’s massive archive of search behaviour, so it had to rely on the poisoned text available on the web.

This is a major competitive advantage for Google in terms of fighting spam: 20+ years of human search behaviour lets Google sanity-check new entities in ways that newer AI models cannot.

Entity Identity and Brand Context

Beyond clicks, Google uses entity-based verification. This means moving from “keywords” (strings of text) to “things” (entities in the Knowledge Graph).

Brand Co-occurrence

Google analyses how often a brand name appears alongside related concepts (like “Xarumei” appearing near “luxury,” “paperweight,” “glass,” “expensive”). This is brand co-occurrence.

• Context validation: For a fake brand to be accepted, it needs to be mentioned in context across independent websites. Parasite SEO tries to force this, but Google’s systems are getting better at distinguishing sponsored or affiliate mentions from organic ones (or so Google says).
• Unlinked mentions matter: Brand mentions without links are becoming as valuable as backlinks. If a brand is discussed on forums, news sites, and social media without links, it signals genuine entity status. Fake brands rarely get discussed naturally; they only get “pitched.”

In theory, a sophisticated spammer could train bots to fake social engagement (like the bots used in information warfare to influence public opinion). But it’s not easy, and these can be detected by anti-spam systems, similar to how Google can identify complex private blog networks.

The Knowledge Graph as Gatekeeper

The API leaks and follow-up analysis suggest that the Knowledge Graph acts as a gatekeeper for AI Overviews and top rankings. Entities that aren’t confirmed in the Knowledge Graph (due to conflicting data or lack of Navboost verification) get suppressed.

This is why “Xarumei” couldn’t gain traction in Gemini: it never cleared the bar to become a confirmed entity.

Google’s 2025 Tools: Branded Queries Filter and Social Integration

A few weeks ago, Google rolled out two new features in Google Search Console that may also signal to us how it tries to directly target the fake brand and listicle spam problem. These aren’t just reporting features; they’re windows into how Google classifies and verifies entities, and they give site owners the data to tell hollow brands apart from real businesses.

The Branded Queries Filter: Separating Demand from Discovery

In November 2025, Google added the Branded Queries Filter to Search Console, letting site owners split their traffic into “Branded” and “Non-Branded” segments.

How It Works

The filter uses an internal AI model to group queries:

• Branded: Includes the brand name, misspellings, and brand-product combinations (“Xarumei pricing,” “Xarumei login”)
• Non-Branded: Generic keywords (“best luxury paperweights,” “buy glass paperweight”)

This goes beyond simple pattern matching. It uses semantic understanding to identify when a user is looking for a specific entity, even with imperfect spelling. This aligns with the navigational query concept from the Navboost leaks.

What This Tells Us

This tool makes hollow brands visible:

For Google: A site that ranks for high-volume generic keywords but has almost no branded queries is statistically suspicious, likely using Parasite tactics or expired domain tricks.

For site owners: It shows whether marketing campaigns are actually building brand awareness. If a campaign only increases generic traffic, brand building has failed. Real brands see correlation: as generic visibility grows, branded search should follow as people remember the name.

The spam signature:

A spam listicle site might get 50,000 monthly visitors, but 99.9% come from queries like “best crypto wallet” or “weight loss pills.” A real brand (like Ledger or WeightWatchers) gets a significant chunk (30-50%) from people typing their name.

This ratio is a primary filter for Google’s spam detection. By sharing this data with users, Google is essentially saying: “This is what we’re watching. If you want to rank, build a brand, not just a keyword trap.”

Social Channels Integration: Proof of Life

The second big update is Social Channels integration in Search Console, letting Google connect social profiles (LinkedIn, X, Instagram, YouTube) with a website.

Organization Schema Validation

This integration relies on Organization schema markup. Google is asking brands to “claim” their social identities:

• Anti-spam barrier: Fake brands often have dormant, fake, or non-existent social profiles. Building and maintaining quality social presence across multiple platforms takes real resources; a cost that spammers (usually) prefer to avoid.
• Verification loop: By linking the website to verified social profiles in GSC, Google builds stronger Knowledge Graph entries. It confirms the entity exists outside the website. (Note: spammers should be cautious about using sameAs properties in Organisation schema to connect websites with social profiles because Google could use this to connect the dots and identify spam brands faster.)
• Activity signals: The GSC report tracks impressions and clicks for social profiles in Search. This implies Google monitors not just whether profiles exist, but whether they’re active and getting engagement. A fake brand’s X profile that never gets clicked signals irrelevance.

Identity vs. the Xarumei Approach

In the Ahrefs experiment, Xarumei had a generated website but no verified, historical social presence. The researchers planted a fake story about X, but they didn’t maintain a long-term, active X account with consistent engagement.

Google’s new integration favours brands with consistent identity signals across multiple platforms: website, social accounts, and GSC verification.

The Social Channels report acts as “proof of life.” A real business has employees on LinkedIn, customer service on X, and video demos on YouTube. A hollow brand usually has empty profiles or profiles that only push affiliate links.

Google’s integration of these metrics into GSC signals that social health is now a search ranking factor.

How to Spot a Hollow Brand

What This Means for AI Search SEO

Brand Defence Is No Longer Optional

For real businesses, the Ahrefs experiment is a warning: brand defence is essential. If you don’t define your own narrative with verified, high-quality content, AI will make one up based on whatever third-party noise it finds; or worse, competitor attacks.

This means:

• Fill the Knowledge Graph: Saturate it with correct information using Schema, verified social profiles, and consistent PR.
• Create an official FAQ: As Ahrefs noted, an official FAQ can sometimes override AI hallucinations, but only if the ranking algorithms consider it authoritative.
• Take care of your About Us section: The About Us section of a website is often considered a minor or irrelevant area from an SEO perspective. With the advent of AI Search, this section is crucial. Don’t just present the company’s mission statement in a classic inspirational style; create, for example, profile pages for the company’s people, starting with its executives and implement Person Schema.

Optimising for AI Citations

The data suggests that branded web mentions (co-occurrence) correlate most strongly with AI visibility. To appear in Gemini or ChatGPT answers, your brand needs to be mentioned by others in contexts relevant to the query:

• Mentions over links: In the AI era, an unlinked mention of your brand in a high-authority context is more valuable for establishing your entity than a low-quality link.
• Structure for citation: Format data (tables, stats, lists) so LLMs can easily ingest it. Feed the AI correct facts to prevent hallucination.
• Guide interpretation: Including structured data in PDFs and webinars can influence how LLMs interpret your content, increasing the chance of accurate citation. We can call this “Ethical Prompt Injecting”.

The Arms Race Ahead

The future will be an arms race between AI-powered spammers generating fake brands and AI-powered filters (like Google’s Navboost and Branded Query classifiers) detecting them:

• Deepfakes and verification: As deepfakes expand to video and audio, Social Channels verification will likely evolve to include biometric or cryptographic identity checks to prevent “Deepfake CEOs” from running fake brands. (This could conflict with privacy regulations, so new rules may be needed.)
• The cost of looking real: Google’s strategy is to make authenticity expensive. Generating text is nearly free; generating 5 years of consistent social engagement, branded search volume, and cross-platform verification is expensive. This economic barrier is the ultimate defence against the “Xarumei” tactic.

Conclusion

The Ahrefs “Xarumei” experiment exposed a fundamental weakness in current AI search: it can’t reliably tell the difference between detailed fiction and reality. For AI, truth is often just a function of repetition and specificity.

But the experiment also showed why Google’s 2025 updates matter, specifically, the Branded Queries Filter and Social Channels Integration. These tools represent a shift from ranking based on content (which AI can fake) to ranking based on identity (which depends on user behaviour and verification).

By tying search visibility to navigational demand (Navboost) and verified entity identity (Social/Schema), Google is trying to protect its index from the flood of manufactured authority.

For marketers and brands, the takeaway is clear: in a world of unlimited content, authenticity, verified identity, and genuine user demand are the only assets that can’t be faked.