Learn AI · Level 2 · How AI works · step 8 of 18 · word 2 of 2
What is a recurrent neural network (RNN)?
A recurrent neural network reads a sequence, such as a sentence, one piece at a time, carrying a memory of what came before. LSTMs are a popular kind. Transformers have mostly replaced them.
In plain English
Words only make sense in order. An RNN reads a sentence word by word and keeps a running summary in its memory, so “bank” can mean a river or money depending on what came earlier. LSTMs (long short-term memory networks) are RNNs designed to remember things for longer.
An example
In 2016, Google described the neural system behind Google Translate, built from deep LSTM networks. In side-by-side tests on simple sentences, it cut translation errors by about 60% compared with the older system.
Watch out
RNNs read one word at a time, so they are slow to train and forget the start of long texts. In 2017, transformers solved both problems with attention, and today’s language models are built on them instead.
Learn AI path · Level 2 · How AI works · step 8 of 18
Done? Next stepInside a language model →See all of level 2