Learn AI · Level 2 · How AI works · step 2 of 20 · word 2 of 2
What is data contamination in AI?
Data contamination is when the questions used to test an AI model were already in its training data, so a high score may show memory rather than real skill.
In plain English
A test only works if the student hasn’t seen the answers. AI models are trained on huge amounts of web text, and popular benchmarks are often published online, so test questions can end up in the training data. The model may then score well because it has, in effect, seen the exam.
An example
In 2024, researchers wrote a fresh set of maths word problems in the style of a well-known test (GSM8k). Some models scored up to 8 percentage points lower on the new questions, partly because they had memorised the originals. Many of the strongest models showed little sign of this.
What to watch out for
Treat a single benchmark score with care. Results on fresh, private or recently written questions are more trustworthy.
Learn AI path · Level 2 · How AI works · step 2 of 20
Done? Next stepHow machines learn →See all of level 2