How to run AI on your own computer

What “running AI on your computer” means

When you use ChatGPT, Claude or Gemini, your question travels over the internet to the company’s computers, an AI model there writes the answer, and it’s sent back to you.

Some companies, including Google, Meta, OpenAI and Alibaba, also release open-weight models: AI models anyone can download. An app on your computer can load one and run it there instead. Nothing is sent anywhere, and it keeps working with Wi-Fi switched off.

You need two things:

• An app (Ollama, LM Studio or Jan, above). It downloads models and gives you a chat window.
• A model. This is the “brain”, a single big file, usually between 2 and 20 GB.

Should you bother?

Good reasons to try it

• Privacy. Your chats stay on your computer. Useful for personal notes, health questions or work documents you shouldn’t paste into an online chatbot. (More on this in our AI and your privacy guide.)
• Works offline. On a plane, on a train, or anywhere the internet is bad.
• Free, with no limits. No subscription and no “you’ve reached your limit, try again later”.
• Learning. It’s the best way to understand how AI actually works.

Honest downsides

• Less clever. The models that fit on a laptop are smaller than the ones behind ChatGPT, Claude and Gemini, which you can’t download at all. They make more mistakes and make things up more often.
• Slower, especially on older laptops without a good graphics card.
• No web search by default, so they don’t know recent news.
• Heavy. Models take up gigabytes of disk space and, while running, most of your computer’s memory and battery.

Our advice: keep using an online chatbot for everyday work (see the best AI chatbots), and use a local one for anything private or when you’re offline.

Will it work on my computer?

What matters most is memory (RAM). A model has to fit in memory to run, and it needs a little more than its download size.

• 8 GB: small models only, such as Llama 3.2 or the smallest Gemma 4. Fine for summaries and rewriting.
• 16 GB: the sweet spot. Runs good everyday models such as Gemma 4 12B and Qwen 3.5 9B, and just about fits gpt-oss-20b.
• 32 GB or more: runs bigger, noticeably smarter models (around 27B to 32B).

Mac or Windows?

• Macs with Apple chips (M1 or newer) are very good at this, because the chip and graphics share the same memory. A 16 GB MacBook Air runs everyday models well. Older Intel Macs are slow, and LM Studio doesn’t support them.
• Windows laptops run models fastest on a graphics card (NVIDIA works best). What counts then is the card’s own memory, called VRAM. Without one, models still run on the main processor, just more slowly. Newer laptops with a built-in AI chip (an NPU) don’t make much difference yet for these apps.

How to check your memory: on Windows, go to Settings → System → About and look for “Installed RAM”. On a Mac, open the Apple menu and choose About This Mac.

Get started in 10 minutes

1. Install an app. For the easiest start, download LM Studio or Ollama and install it like any other app.
2. Download a model. In LM Studio, use the search (magnifying glass) and type “gemma”. In Ollama, pick a model from the list in the chat window. On an 8 GB computer, start with Llama 3.2 instead.
3. Wait for the download. Usually a few gigabytes, so allow a few minutes.
4. Start chatting. Type a question as you would in ChatGPT. The first reply can be slow while the model loads.
5. Try something private. Drag in a PDF or text file and ask for a summary. It’s processed on your computer.

If replies are painfully slow or the app warns that the model is too big, switch to a smaller model. That fixes most problems.

The jargon, decoded

Model names look like “qwen3.5:9b-q4_K_M”. Here’s how to read them.

• “9B” (the size). Billions of parameters, the numbers the model learned in training. Bigger usually means smarter, but slower and hungrier for memory. 1–4B is small, 7–14B is medium, and 27B and up is large for a laptop.
• “Q4” (quantisation). A compressed version of the model that rounds its numbers to save space, a bit like an MP3 compared with a CD. Q4 is the usual choice: about a quarter of the full size, with only a small loss in quality. Q8 is higher quality but twice as big.
• Context length. How much text the model can keep in mind at once, such as a long document plus your chat (see context window). Longer settings use more memory.
• GGUF and MLX. File formats. GGUF works everywhere. MLX is a version tuned for Apple Silicon Macs, which is often faster on a Mac.
• Local vs cloud. “Local” means it runs on your computer. Some apps also offer “cloud” models that run on the company’s servers. Those are not private in the same way.

Is it really private?

Mostly yes, with three things to watch:

• Cloud options inside the apps. Ollama offers cloud models (their names end in “cloud”), and Jan can connect to ChatGPT, Claude and others. Chats with those leave your computer. Stick to downloaded models for anything sensitive.
• Where models come from. Download models through the app itself, from its built-in library or Hugging Face, the main public site for AI models. Don’t run model files from random websites or chat groups.
• Your workplace rules. Running a model locally keeps data on your machine, but some employers still restrict which AI tools and models staff may use. Check before using one for work.

Where to go next

• For developers: these apps can also act as a local AI server, so coding tools and your own programs can use the models in place of a paid online service.
• Learn the basics: what a large language model is, and how inference (a model producing an answer) works.

How we chose and keep this guide up to date

We tested the claims against each app’s official website, documentation and privacy policy, and took model sizes from the Ollama model library. Each card shows when it was last checked. Memory guidance is a rule of thumb: real needs depend on the model version, the compression level and how much text you give it. New models come out every few weeks, so we review this guide every quarter. Spotted something out of date? Tell us.