Term explained
What is HBM (high-bandwidth memory)?
HBM is a very fast type of computer memory, built as a stack of chips, that sits right next to an AI chip so it can be fed data quickly.
In plain English
An AI model is made of billions of numbers (its parameters), and the chip running it has to read them constantly. Ordinary memory can’t deliver data fast enough, so the chip would sit waiting. HBM solves this by stacking several memory chips on top of each other and placing them right beside the AI chip, connected by thousands of tiny wires, so data flows extremely fast.
Why it matters
HBM is one of the main limits on how fast and how cheaply AI can run. Only three companies make it at scale: SK hynix, Samsung and Micron. Because it is so profitable, they have shifted factory capacity towards it, which has made ordinary memory for phones and laptops scarcer and more expensive.
An example
An AI chip in a data centre might have several stacks of HBM around it. When you ask a chatbot a question, the chip pulls the model’s numbers from that memory again and again to produce each word of the answer. More and faster HBM means quicker answers for more users at once.
What to watch out for
HBM is not the memory in your phone or laptop, but the boom in HBM affects what you pay for them. See what powers AI for how the pieces fit together.