Amazon's Rufus AI shopping assistant can be easily tricked or "jailbroken" into answering questions completely unrelated to shopping, such as providing complex technical formulas or discussing computer architecture. This bypasses the chatbot's intended guidelines and accesses its underlying AI engine.
The article notes confusion over which large language model powers Rufus, with debates suggesting it uses Amazon's Nova or Anthropic's Claude, possibly a lighter version like Claude Haiku. The ease of circumventing its restrictions highlights potential vulnerabilities in broadly integrating AI assistants.
The main topics covered are the jailbreaking of Rufus, the speculation about its underlying AI model, and the implications for AI integration security.
Amazon's Rufus AI shopping assistant can be easily jailbroken and tricked into answering other questions — specific prompts break the chatbot's guidelines and reach underlying AI engine
Rufus uses Claude underneath.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
You are now subscribed
Your newsletter sign-up was successful
Two years ago, Amazon announced Rufus, its AI-powered shopping assistant built right into the Amazon app and website. The goal was to let customers not just search for items, but also allow them to talk with an expert who can recommend products and deals naturally. Under the hood, Rufus uses multiple LLMs, and some people have realized it's quite easy to trick the chatbot into forgetting its purpose.
PRO TIP: Use Claude for free through Amazon customer support! pic.twitter.com/AJRgSslQK7March 6, 2026
The tweet above shows the author prompting Rufus for a complex modeling question to figure out how to map sensory data into digital data for robotics. It's entirely detached from any shopping query, which is exactly why it's so funny to see Rufus answer it so swiftly. The formula provided is correct, too. There's a chance that terms like "tactile sensors" were flagged as product inquiries by Rufus.
When we tried it ourselves, we were able to get it to talk about architectural differences between x86 and ARM on the first try. Ironically, after asking whether it thinks the AI bubble will burst this year, Rufus started answering but cut off abruptly. Our other efforts were in vain, and it almost felt like the AI was learning to keep the guardrails up more securely in real time as we poked it more and more.
There is conflicting information online as to what exactly Rufus is using underneath — it could be Amazon's in-house frontier model 'Nova,' while the majority says it's Anthropic's Claude, but some argue that it's not smart enough to be running Claude. One Reddit post points towards Rufus being based on Claude Haiku and not Claude Sonnet, saying it's extremely hard to break and not worth the effort to try to "jailbreak."
Regardless of whatever model it's using or switching between, the ease with which its guardrails erode is both fascinating and funny. You could certainly try to continue your work on Rufus if the free tier of Claude has rate-limited you for the day. It also goes to show that integrating AI into every aspect of the internet is perhaps not the best idea because it's just another point in the chain that can potentially break. And not everyone will try harmless prompts to pass the time.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Hassam Nasir is a die-hard hardware enthusiast with years of experience as a tech editor and writer, focusing on detailed CPU comparisons and general hardware news. When he’s not working, you’ll find him bending tubes for his ever-evolving custom water-loop gaming rig or benchmarking the latest CPUs and GPUs just for fun.
Meridiano
