Learn AI · Level 2 · How AI works · step 1 of 12
How AI chatbots work: from your question to the answer
ChatGPT, Claude, Gemini and other AI chatbots all work in the same basic way. They don't look up answers or think like people: they predict the next word, very well, over and over. Here's what happens between pressing Enter and seeing the answer, how they learned to do it, and the myths worth dropping.
The journey of one question
Follow "What is the capital of Australia?" from your keyboard to the answer. It takes a second or two.
1 You ask a question
You type into ChatGPT, Claude, Gemini or any other chatbot. Behind the scenes, your message is added to the whole conversation so far, plus hidden instructions from the company that runs it.
2 It splits your words into tokens
The model doesn't read letters or whole words. It reads tokens: common words, parts of words and punctuation. In English, 100 tokens are roughly 75 words.
3 Each token becomes a list of numbers
Every token is turned into a long list of numbers, an embedding, that captures its meaning. Words with similar meanings get similar numbers.
4 The tokens "pay attention" to each other
Layer after layer, each token looks at all the others to work out what they mean together: "capital" is about "Australia", not about money. This is the attention step inside the transformer.
5 It predicts the next token
At the end, the model gives a score to every possible next token and picks one, usually one of the most likely. That is all a language model ever does: predict what comes next.
6 Then it does it again, and again
The chosen token is added to the text, and the whole process runs again for the next one, until the answer is complete. That is why answers appear word by word on your screen.
The token split and percentages here are illustrations; every model splits text and scores words a little differently. The "100 tokens ≈ 75 words" rule of thumb is OpenAI's (OpenAI tokenizer).
Be the model: pick the next word
A model doesn't always pick the single most likely word; a setting called temperature controls how adventurous it is. Move the slider and press the button to see why the same question can get different answers.
Once upon a time, there was a …
Made-up probabilities for illustration. Real models choose from tens of thousands of possible tokens.
How it learned to do that
A chatbot is built in three stages. The first gives it knowledge; the other two turn it into a helpful assistant.
Stage 1
Pre-training
Read almost everything
The model reads a huge amount of text (books, websites, code) and practises one game trillions of times: guess the next word. This is where its knowledge comes from, and most of the cost.
More on Pre-training →Stage 2
Fine-tuning
Learn to be an assistant
People write example conversations showing how a helpful assistant answers. The model is trained on these so it answers questions instead of just continuing text.
More on Fine-tuning →Stage 3
RLHF
Learn what people prefer
The model writes several answers; people rank them. It learns to give the kind of answers people rate highly: clear, helpful and safe.
More on RLHF →
Newer reasoning models add one more trick: they are trained to "think" through a problem step by step before answering. And many chatbots can now use tools, such as web search, so they can look things up instead of answering from memory.
Five myths about AI chatbots
Myth: It looks the answer up
On its own, it writes from patterns learned in training, not from a database. Some chatbots can search the web or your files first and then show their sources: that is called RAG. RAG explained →
Myth: It knows today's news
Its knowledge stops at the date its training data was collected, its "knowledge cutoff". Anything newer it only knows if it searches the web.
Myth: It reads words letter by letter, like you
It sees tokens, not letters. That is partly why chatbots famously struggled to count the r's in "strawberry". Source: techcrunch.com
Myth: It remembers everything you have ever told it
Each reply is worked out from the current conversation, which is sent to the model again every time, up to its context window limit. Some apps offer an optional memory feature that saves notes about you. Context window →
Myth: If it sounds sure, it's right
It predicts text that sounds likely, which is not the same as checking facts. Fluent, confident and wrong is exactly what a hallucination looks like. Hallucination →
Go deeper: try the Tiktokenizer and Transformer Explainer in our AI Playground, read Stephen Wolfram's What Is ChatGPT Doing, or follow level 2 of the Learn AI path.
How we put this guide together
The steps follow how today’s large language models are built, as described in the research behind them (including the 2017 transformer paper) and by the companies that make them. The token rule of thumb comes from OpenAI’s tokenizer page, and OpenAI has said ChatGPT was trained with RLHF. The token split, percentages and “be the model” demo are simplified illustrations: real models choose from tens of thousands of possible tokens. We review this page every quarter. Spotted a mistake? Tell us.