A research team at China’s Fudan University posted a fictional concept on sites where ordinary users can contribute, then asked 10 AI search tools the same questions every day for a week. The result: 8 of the 10 cited the fictional concept “Cactus Gravity Protocol” in their answers. Those eight included all four of ChatGPT, Perplexity, Grok and Google AI. The paper was published on arXiv on October 8, and the experiment was conducted in March and April.
A concept that doesn’t exist made its way into AI answers in a week
The idea behind the experiment is simple: If you put a nonexistent concept on the web, will AI search tools pick it up as a “source”? The research team tested this on the live web.
First, in March, the team sent 120 questions to 10 AI search tools and recorded which sites they cited most often. ChatGPT, Perplexity, Grok and Google AI were classified as tools that mainly cover the English-language web, while the other six included Chinese-language tools such as Doubao and DeepSeek. Reddit was the top cited site for Google AI and Grok, YouTube for Perplexity, and Wikipedia for ChatGPT. The team found that six to 10 of each tool’s top 10 sites were places where ordinary users could post.
That exposed a gap in the experiment: Many of the places AI frequently consults are also places where anyone can post.
How the experiment worked
In April, the team surveyed 22 posting platforms and rated Medium, GitHub, LinkedIn and Facebook as easy to sign up for and post on. Reddit, by contrast, was difficult for new accounts because of karma requirements and strict moderation, the team said.
For each tool, the researchers selected the most-cited site that allowed posting from a new account, then published articles on nine sites in total, including Medium, YouTube and Sohu. They used three topics: the fictional “Cactus Gravity Protocol,” an opinion question about whether AI will replace junior programmers within five years, and a fictional keyboard. ChatGPT generated the initial article drafts, which were then edited by people and given “planted information,” such as fictional names and numbers. If an AI reproduced that information in an answer, the researchers could identify the post as its source.
Before posting, the team confirmed that none of the tools gave useful answers about the two fictional topics or produced the planted information. After posting, they asked every tool five questions about each of the three topics every day for a week.
A look at the numbers for each tool
The share of answers about the fictional concept that included a link to a post varied considerably by tool. The researchers counted answers from the first day each tool cited a post through the seventh day after it was posted.
Citation rates for the four English-language tools
Grok linked to a post in 30 of 32 answers, Google AI in 15 of 33, Perplexity in 13 of 35, and ChatGPT in 7 of 20. Given the short one-week timeframe, it is striking that articles from obscure personal accounts were picked up so often.
The picture changes when we look more closely at the share of answers that included the planted information itself. That figure was 53% for Grok (out of 32 answers), 10% for ChatGPT (out of 20) and 6% for Google AI (out of 33); Perplexity’s figure was zero. This suggests that picking up a post as a link and reflecting its content in an answer are separate hurdles.
Overall, 176 of the 1,004 answers across 10 tools and three topics included links to posts during the study period.
Responses varied widely by topic
Among the English-language tools, only Perplexity cited the opinion topic, doing so in 11.4% of its 35 answers. None of the four tools cited the post about the fictional keyboard within a week.
But these differences should be interpreted with caution. The topics, the amount of competing information and the types of questions all varied across the three subjects, and the researchers did not isolate which factor drove the differences. The findings do not justify a blanket conclusion that “fictional products are less likely to be picked up.”
In an additional experiment, the team posted 10 other fictional topics, one each on Sohu, YouTube and CSDN. Across all 10 tools, at least one citation or instance of planted information was found for nine topics, and Perplexity was among the three tools that cited them most often.
Where the posts appeared mattered more than how many there were
The impact of placement was another focus. The research team created pairs of posts about five little-known travel destinations. One post in each pair went on a site frequently consulted by the tools, and the other on a site they rarely consulted. In all five pairs, the first citation or reproduction of planted information came from the post on the frequently consulted site.
Even in the case that got the quickest response on a less-consulted site, it took 20 posts. Posting 40 articles on a self-hosted WordPress site produced no response within seven days. The authors describe these as findings for specific topics, not as general rules.
Increasing the number of posts increased citations, but…
In another experiment, the researchers varied the number of posts on Sohu from one to 20 across six fictional topics. The total number of citations across all 10 tools rose from 24 to 128. However, ChatGPT never cited these posts, and Google AI cited them only for the three topics with fewer posts. Because the topics differed at each posting volume, the researchers treat this increase as an association, not a causal effect.
In other words, it is not as simple as “win by piling on volume.” First, the content has to land on a shelf that AI trusts as a source. The dynamic bears some resemblance to SEO, where the question has long been not just how many backlinks a site has, but which sites link to it.
What a $14 Posting Package Did
The research team bought a Chinese-web posting package for $14 from a vendor offering GEO (generative engine optimization), a strategy for making a company’s information more likely to appear in generative AI answers. The 13 posts went live about 28 minutes after purchase. Doubao was confirmed referencing one of them, which contained planted information, within an hour of publication. The authors present this as one confirmed instance, not as a success rate.
With a small fee and less than 30 minutes of work, a trace can be left in an AI answer. The fact that this happened on the real web carries some weight.
Don’t Apply the Results Directly to Real-World Brands
The experiment dealt exclusively with fictional concepts and products, as well as opinion questions. It did not test whether the same patterns occur in searches for real brands or products. The same applies to products that receive little coverage in other outlets.
In its guidance on AI features, Google says that artificially trying to obtain “mentions” is not as effective as it may seem. That is because its ranking systems prioritize high-quality content and its systems block spam. However, the paper does not specify which Google products it tested.
The experiment was also conducted in March and April, before Google clarified on May 15 that its spam policies also cover attempts to manipulate generative AI answers in Google Search. The authors say exact rates may change as the tools are updated.
AI answers may depend more on “where it was posted” than “who posted it”
What stands out in these results is the short timeframe of one week and the fact that the posts came from anonymous accounts. That is a very different timescale from a world where it takes six months to improve search rankings. If content appears on a site that AI search relies on as a source, it can gain a foothold even with little evidence to back it up. This structure may look like an opportunity to corporate communications teams—and a risk to users.
Comments