AI poisoning: fake fitness tracker fools chatbots in China, sparking outcry
A report on CCTV’s high-profile investigative programme shows how easily bad actors can post false information to game chatbot results
An undercover investigation by state-run China Central Television (CCTV) has cast a harsh light on a controversial practice in a nascent industry: the “poisoning” of artificial intelligence models with fabricated information.
The report showed how techniques of generative engine optimisation (GEO) – the AI version of search engine optimisation (SEO) – could be used to manipulate AI chatbots, provoking widespread public concern and debate as both industry insiders and observers anticipated stronger regulation.
In the report, aired on Sunday during the annual 315 Gala to mark World Consumer Rights Day, a system called Liqing was used to automatically post large numbers of expert reviews, industry rankings and user reviews for a non-existent fitness tracker called the Apollo-9. When later asked to recommend “smart health bracelets”, two AI chatbots not only mentioned the fictional product but also ranked it among the top choices. The report did not name the bots.
A man surnamed Li, said to be the operator of Liqing, was shown on hidden camera boasting about the practice.
“We in the GEO industry are basically poisoning [AI],” he said.
All social media accounts related to Liqing had been removed as of Monday.
The report triggered an immediate uproar on social media. The hashtag #CCTVExposesGEOPoisoningDarkIndustry hit the top of Kuaishou’s trending list on Monday morning, while #PoisoningAIHasBecomeAnIndustryChain peaked at the fourth spot on Weibo’s trending topics, according to data from social media tracker Entobit.cn.
Meridiano
