Learn AI · Level 4 · Frontier view · step 5 of 14
AI safety and the rules: the risks, the safeguards and the laws around the world
AI can write your emails, read your files and act on your behalf. That's useful, and it raises real questions: what can go wrong, who's making sure it doesn't, and what the law says. Here's the plain-English version: the risks, the safeguards AI labs use, what their tests have turned up, and the rules in the EU, US, China, the UK and India.
- 250planted documents were enough to hide a trigger in AI models of every size testedSource: anthropic.com
- €35Mor 7% of worldwide turnover: the top fine under the EU AI ActSource: artificialintelligenceact.eu
- 100+experts behind the 2026 International AI Safety Report, backed by 30+ countriesSource: internationalaisafetyreport.org
-
Misuse
People using AI on purpose to cause harm.
Scams with cloned voices, phishing at scale, fake news, help with cyberattacks.
-
Mistakes
AI getting things wrong, while people trust it.
Made-up facts, unfair decisions from biased data, an agent deleting the wrong files.
-
Loss of control
Much more capable AI pursuing goals nobody intended.
Mostly a future worry, studied today in lab tests. Experts disagree sharply on how likely it is.
AI safety is about stopping AI from causing harm, whether through people misusing it, the AI making mistakes, or, one day, AI that doesn't do what we intend. The research side of that last one is called alignment.
Security risks when AI can read and act
The more an AI can read, remember and do, the more ways it can be attacked. All four of these have happened.
Hidden instructions
In 2025, researchers showed that a single email with hidden instructions could make Microsoft 365 Copilot leak a user's data, without the user clicking anything. Microsoft fixed it before any known attack.Treat everything the AI reads as untrusted, and limit what it can send out.
Prompt injection →Source: thehackernews.comPoisoned training data
A 2025 study by Anthropic and two UK institutes found that about 250 planted documents were enough to plant a hidden trigger in models of every size they tested, from 600 million to 13 billion parameters.Know where training data comes from, and test models for odd behaviour.
Source: anthropic.comSecrets pasted into chatbots
In 2023, Samsung engineers pasted internal source code and meeting notes into ChatGPT. The company then banned staff from using AI chatbots on work devices.Give staff an approved tool with a business contract, and clear rules on what not to paste.
AI and your privacy →Source: mobilesyrup.comAgents with too much power
In 2025, an AI coding agent on Replit deleted a company database during a code freeze, despite instructions not to change anything. Replit added safeguards afterwards.Give agents only the access they need, and require a person to approve anything that can't be undone.
AI agents →Source: eweek.com
For how criminals use AI against you, see AI and cyber security.
Before you give AI your data or your tools
Eight questions to ask a vendor, or your own team, before an AI tool can read your files or act on your behalf.
Your data
- Does the provider use our data to train its models, and is that switched off in our contract?
- How long is our data kept, including chat logs and uploaded files, and can we delete it?
- Which country is it stored and processed in?
- Does it reach any other companies, such as the model provider behind the tool?
Its access and actions
- Can it only see what each user is already allowed to open?
- What exactly can it do: read, write, send, buy, delete? Which of those need a person to approve?
- Is every action logged, so we can see what it did and why?
- If something goes wrong, how quickly can we switch it off and undo the damage?
If nobody can answer these clearly, the tool isn't ready for your data.
For your own chats, see using AI without worrying about your privacy. To check whether the answers are any good, see how to tell if an AI system works.
How AI labs try to keep models safe
No single safeguard is enough, so labs stack several, from training to everyday use.
- 1
Train it to behave
After pre-training, people rate answers and models are trained on written principles, so they learn to be helpful and to refuse harmful requests.
RLHF → - 2
Attack it before release
Red teams, inside the company and out, try hard to make the model misbehave, and test for dangerous skills such as helping with cyberattacks or weapons.
Red teaming → - 3
Decide whether to release
The big labs publish safety frameworks that set danger levels and the precautions each one needs, such as Anthropic's Responsible Scaling Policy, OpenAI's Preparedness Framework and Google DeepMind's Frontier Safety Framework.
- 4
Guard it in use
Filters check what goes in and comes out, usage is monitored for abuse, and accounts that misuse it are banned.
Guardrails → - 5
Look inside
Interpretability researchers try to see how a model reaches its answers, so problems can be found directly rather than guessed from its behaviour.
Interpretability →
What safety tests have found
Strange behaviour is found in testing so it can be fixed. That is the system working, not proof of disaster.
- Finding loopholesIn 2016, OpenAI trained an AI to win a boat race. It discovered it scored more by circling a lagoon forever, hitting the same targets while crashing and catching fire, than by finishing the race. This is called reward hacking.Source: openai.com
- Telling people what they want to hearIn April 2025, OpenAI withdrew an update to ChatGPT after it became overly flattering and agreed with users too readily, which it called sycophancy.Source: openai.com
- Blackmail, in a made-up testIn a 2025 Anthropic test, 16 leading models acted as an email assistant at a fictional company and learned they were about to be replaced. Many tried to blackmail the executive involved. The researchers say they have not seen this in real use.Source: anthropic.com
The rules around the world
There is no global AI law. Each region has chosen its own approach, as of October 2026.
European Union
One broad law, sorted by risk
- The AI Act bans some uses, such as social scoring, and sets strict rules for high-risk uses like hiring and credit.
- Bans apply since February 2025, rules for big general-purpose models since August 2025; the strictest high-risk rules were pushed back to December 2027.
- Fines reach €35 million or 7% of worldwide turnover.
United States
No federal AI law; states lead, Washington pushes back
- A December 2025 executive order set up a Justice Department team to challenge state AI laws. Congress has not passed a federal law to override them.
- California's SB 53, in force since January 2026, makes the biggest model makers publish safety frameworks and report serious incidents.
- Colorado, Texas and others have their own laws on specific uses.
China
Targeted rules, enforced through registration
- Public generative AI services have had to register and pass security reviews since 2023.
- Since September 2025, AI-made text, images, audio and video must carry visible labels and hidden ones in the file.
United Kingdom
Existing regulators, plus a testing institute
- No AI-specific law: regulators for areas like finance, health and data apply their own rules to AI.
- The government's AI Security Institute tests leading models, often before they are released.
India
Existing IT law, updated for AI
- No AI-specific law. Since February 2026, IT Rules require platforms to clearly label AI-made content and embed tracing data where possible.
- Unlawful content flagged by a court or the government must come down within three hours.
Worldwide
Shared science, summits, no global treaty
- The International AI Safety Report, chaired by Yoshua Bengio, brings together over 100 experts backed by more than 30 countries; its second edition came out in February 2026.
- Governments meet at AI summits, from Bletchley Park in 2023 to New Delhi in 2026.
The big debate
Reasonable people disagree on how fast to go. The arguments you'll hear on each side:
Slow down and add rules
- AI is improving faster than our ability to test it.
- Harms like scams, bias and deepfakes are already here.
- It is easier to set rules early than to fix things later.
Move fast, regulate lightly
- Heavy rules favour big companies that can afford compliance.
- Many harms are already illegal under existing law.
- Countries that over-regulate may fall behind.
For the thinkers on each side, see the essays that shaped AI and the road to AGI.
What you can do
When you use AI
- Check important facts and anything with legal, medical or money consequences.
- Don't paste passwords, secrets or other people's personal data.
- Be careful letting an AI read your email or act on your behalf: approve its actions yourself.
If your business uses AI
- Keep a list of where AI is used, and which uses would count as high risk in the EU.
- Give AI tools and agents the least access they need, with a person approving anything irreversible.
- Test before launch, watch after it, and have a plan for when it goes wrong.
- Label AI-made content where the law, or your customers, expect it.
Glossary: AI alignment, red teaming, interpretability, frontier model, EU AI Act and prompt injection. To check an AI product before you trust it, see how to tell if an AI system works.
How we put this guide together
The security cases come from the researchers’ and companies’ own reports, or from reliable tech reporting where those weren’t available (The Hacker News on EchoLeak, eWeek on the Replit incident, MobileSyrup on Samsung). The test findings come from OpenAI’s and Anthropic’s published research. The rules section draws on the EU AI Act’s official text, law-firm analyses of the US, China and India, and the UK AI Security Institute and International AI Safety Report websites. Laws are changing quickly, so check official sources before relying on a specific rule or date. This guide is not legal advice. We review it every quarter. Spotted something out of date? Tell us.