Learn AI · Level 2 · How AI works · step 3 of 16 · word 3 of 5
What is self-supervised learning?
Self-supervised learning makes its own practice questions from raw data, such as hiding the next word of a sentence and learning to guess it. It is how chatbots learn language.
In plain English
Labelling billions of examples by hand is impossible. So the model makes its own quiz: it hides part of the data and tries to fill it in. The hidden part is the answer, so no person needs to label anything.
That is why models can learn from almost the whole public internet.
An example
A large language model reads “The cat sat on the ___” and guesses the missing word. Repeat that trillions of times across books and websites, and it picks up grammar, facts and style. This stage is called pre-training.
Watch out
Guessing the next word well isn’t the same as knowing what is true. The model learns what text usually says, mistakes included, which is one reason hallucinations happen.
Learn AI path · Level 2 · How AI works · step 3 of 16
Next wordReinforcement learning →See all of level 2