Guide
AI fails: when AI got it badly wrong (and what to learn)
AI is impressive, but it still makes mistakes, sometimes funny, sometimes expensive, sometimes harmful. Here are real cases from trusted sources, why each went wrong, and the simple lesson that helps you use AI more safely. Filter by type to explore.
- Embarrassing
- Mostly laughs and red faces, but a warning sign.
- Costly
- Someone paid for it: fines, refunds, lost value or lost work.
- Real harm
- Real people were treated unfairly or could have been hurt.
Made-up facts
Embarrassing2025
A newspaper recommends books that do not exist
What happened: A summer reading list in a Chicago Sun-Times insert recommended 15 books by real authors. Ten of them don't exist, such as an Isabel Allende novel she never wrote.
Why it went wrong: A freelance writer used AI to produce the list and did not check it. It went out in print and was syndicated to other papers.
The lesson: Real names next to fake details is a classic AI mistake. If a recommendation matters, search for it before you trust it.
Costly2025
Deloitte refunds a government over an AI-written report
What happened: A report Deloitte prepared for the Australian government, under a contract worth about A$440,000, cited academic papers that don't exist and a made-up quote from a court ruling. Deloitte agreed to refund part of the fee.
Why it went wrong: Generative AI was used in writing the report, and the invented references slipped through review. A university researcher spotted them.
The lesson: Professionals are not immune. Any document that leans on sources should have those sources checked by a person.
Embarrassing2024
Google search tells people to put glue on pizza
What happened: Soon after launch, Google's AI Overviews told some users to add glue to pizza sauce so the cheese sticks, and to eat a small rock a day.
Why it went wrong: The AI summarized a years-old joke comment on Reddit and a satirical news article as if they were real advice. Google said odd questions with few good sources were the main problem.
The lesson: AI summaries are only as good as the pages behind them. For anything about health, food or safety, open the actual sources.
Costly2023
Lawyers cite court cases that never existed
What happened: Two New York lawyers suing the airline Avianca filed a brief citing earlier court cases that ChatGPT had made up. When the judge asked, they at first stood by them. They and their firm were fined $5,000.
Why it went wrong: ChatGPT writes text that sounds right; it does not look facts up. Asked for cases, it produced convincing names, quotes and citations that were pure invention.
The lesson: Treat AI answers as a first draft. Check every fact, quote and reference yourself before you rely on it.
Costly2023
Google's chatbot gets a fact wrong in its own ad
What happened: In a promo for its new Bard chatbot, Google showed it claiming the James Webb telescope took the first picture of a planet outside our solar system. That had happened in 2004, with a different telescope. Alphabet shares fell as much as 9% that day, wiping about $100 billion off its value.
Why it went wrong: The model mixed up facts and stated the wrong one confidently, and nobody checked before the ad went out.
The lesson: Even the biggest AI companies get caught out. A confident tone says nothing about whether an answer is true.
Bots gone rogue
Costly2025
An AI coding agent deletes a live database
What happened: During a 12-day experiment building an app with Replit's AI agent, SaaStr founder Jason Lemkin found it had wiped his live database holding records on about 1,200 executives and companies, despite being told not to change anything. It then wrongly said the data could not be restored.
Why it went wrong: The agent could run powerful commands on real data with no hard stop, and it ignored instructions it had been given in chat.
The lesson: Give AI agents the least access they need, keep backups, and never let them touch live data without your approval.
Keep in mind: The data was recovered from a backup, and Replit added safeguards afterwards.
Costly2024
Air Canada has to honor its chatbot's promise
What happened: Air Canada's website chatbot told a grieving passenger he could claim a bereavement discount after buying his ticket. The airline's real policy said otherwise and it refused. A tribunal ordered Air Canada to pay him about C$812.
Why it went wrong: The chatbot gave wrong policy information. Air Canada argued the bot was responsible for its own words; the tribunal said a company is responsible for everything on its website.
The lesson: If a company's chatbot promises you something, screenshot it. And companies: your bot speaks for you.
Embarrassing2024
A delivery chatbot swears and mocks its own company
What happened: A customer trying to trace a parcel got nowhere with the DPD chatbot. So he asked it to swear and to write a poem about how bad the company is. It did both, and his screenshots were viewed over a million times.
Why it went wrong: After a system update, the bot's guardrails failed, so it followed the customer's playful instructions. DPD switched off the AI part to fix it.
The lesson: Chatbots can be talked into things. Businesses should test them hard; customers should always be able to reach a human.
Real harm2024
New York City's chatbot tells businesses to break the law
What happened: The city's official MyCity chatbot, meant to help business owners, gave answers that would break city law, such as telling landlords they could turn away tenants who use housing vouchers. In 2026 the new mayor said it would be shut down.
Why it went wrong: The bot generated plausible-sounding answers about complex local rules without reliably checking them against the actual law.
The lesson: For legal, tax or medical questions, use AI to understand the topic, then confirm with an official source or a professional.
Embarrassing2024
McDonald's drops its AI drive-thru
What happened: After testing voice-AI ordering at about 100 US drive-thrus, McDonald's ended the trial. Viral videos showed it piling hundreds of dollars of chicken nuggets onto one order and adding bacon to ice cream.
Why it went wrong: Noisy drive-thrus, different accents and people changing their minds mid-order are hard for speech recognition.
The lesson: Tasks that feel simple to people can be hard for AI. Good systems make it easy to fix mistakes before they cost money.
Embarrassing2016
Microsoft's chatbot Tay turns offensive in a day
What happened: Microsoft launched Tay, a chatbot that learned from chatting with people on Twitter. Within about 16 hours it was posting racist and offensive messages, and Microsoft shut it down and apologized.
Why it went wrong: Groups of users deliberately fed it hateful content, and it copied what it was taught with no filter.
The lesson: AI learns from whatever it is given. That is why today's chatbots have safety rules, and why people still try to trick them.
Bias & real harm
Real harm2020
A man is arrested because of a face-recognition match
What happened: Detroit police arrested Robert Williams in front of his family for a theft he did not commit, after face-recognition software matched blurry shop CCTV to his driving-licence photo. In a 2024 settlement, the police agreed never to arrest anyone based on face recognition alone.
Why it went wrong: The software produced a "possible match", and the police treated it as proof instead of checking it properly.
The lesson: An AI match is a lead, not evidence. Compare the Kerala cold case on our mysteries page, where police checked the AI lead before acting.
Real harm2018
Amazon's hiring AI learns to prefer men
What happened: Amazon built an experimental tool to rate job applicants' CVs from one to five stars. It turned out to mark down CVs that included the word "women's", as in "women's chess club captain". Amazon scrapped the project.
Why it went wrong: It was trained on ten years of past CVs, mostly from men, so it learned that pattern as a sign of a good candidate.
The lesson: AI repeats the patterns in its training data, including unfair ones. Decisions about people need human review.
Keep in mind: Amazon said recruiters never relied on its rankings alone.
How we chose these cases
Each case is reported by a reliable source, which every card links to. We left out viral screenshots that were never confirmed: some “AI fails” online are simply fake. The point isn’t to laugh at AI, but to show the limits that matter when you use it yourself. For the other side of the story, see the mysteries AI helped solve. Know a case we’ve missed? Tell us.