Aiodic

Back to the table

115 Emerging Cn Continual Learning Frontier

Continual Learning

Models that keep learning after deployment.

What it is

Continual learning is the goal of an AI model that keeps learning after it has been released, the way a person keeps learning after leaving school. It would pick up new facts, new skills and your particular way of working from its everyday experience, and it would do so without forgetting what it already knew. Researchers also call it lifelong learning or incremental learning.

To see why this is a goal and not a feature you already have, it helps to know how today's models are made. A large language model is built in big, expensive training runs. First comes pretraining, where the model reads a huge collection of text and adjusts its parameters, the billions of numbers (also called weights) that hold everything it has learned. Then come shorter stages such as fine-tuning and RLHF, which shape its manners and skills. When training ends, the weights are fixed and the model is shipped. From then on it is frozen.

Frozen means two things. First, the model has a training cutoff: a date after which it knows nothing, because the data stopped there. Second, nothing you say to it changes it. When you correct a chatbot, it may apologise and use your correction for the rest of that conversation, but the underlying model is exactly the same the next morning, for you and for everyone else. The next time it learns anything is when its maker runs another training round and releases a new version, usually months later.

Products hide this well. Many chatbots now have memory features that save notes about you and feed them back in at the start of each chat. Tools such as web search and RAG fetch fresh information when you ask a question. All of these are useful, and all of them work around the frozen model rather than changing it. The model reads the notes the way you might read a sticky note on your fridge. It hasn't learned the thing; it has been reminded of it.

Real continual learning would change the weights themselves, a little at a time, from new experience, while the model is in use. That turns out to be surprisingly hard, and the rest of this page explains why. It is an active research area, marked "Emerging" on this table because there are promising results in the lab but no mainstream chatbot that learns this way yet.

Why it matters

It's the difference between a new hire and a colleague. A good new employee is slow in week one and much better by month three, because they learn your customers, your preferences and the mistakes to avoid. Today's AI models are stuck at week one, however clever week one is. The writer and podcaster Dwarkesh Patel made this argument in a widely read essay on 2 June 2025, "Why I don't think AGI is right around the corner". He called the lack of continual learning "a huge huge problem", argued that it limits how useful models are at real jobs, and said he'd take an even bet that AI learns on the job as easily as a human for any white-collar work by 2032. That is his prediction, not a settled fact, but it captures why many people in the field see continual learning as a missing piece.

Knowledge goes stale. Because a model's knowledge stops at its cutoff, anything newer has to be supplied by search or documents at the moment you ask. That works, but it costs time and money on every question, and the model never builds up a deeper understanding of the new material. A model that could fold new knowledge in for good would stay current between major releases.

Retraining is expensive. At present the main way to teach a model something new is another training run. For the largest models that means big computing bills and months of work, which is why updates come in batches. Continual learning, if it worked, could make updates smaller, cheaper and more frequent.

It would change privacy and safety. A model that learns from what people tell it is also a model that can learn the wrong things, remember what it shouldn't, or drift away from the behaviour its makers tested. Those risks are covered below, and they're a big reason the frozen approach has lasted. Understanding them helps you judge future products that claim to "learn from you".

It shapes the big predictions. Debates about AGI, meaning AI as capable as a person across most tasks, often come down to whether models can learn on the job. If you read a forecast about AI replacing jobs, it's worth asking whether it assumes continual learning has been solved.

The central problem: catastrophic forgetting

If continual learning is so valuable, why not keep training the model on new data every day? Because neural networks have a nasty habit. When you train one on something new, it tends to overwrite what it learned before. Researchers call this catastrophic forgetting. Michael McCloskey and Neal Cohen described it in 1989, when a small network trained on new arithmetic facts lost the ones it had learned earlier.

The reason is that knowledge in a network isn't stored in neat drawers. It's spread across the same shared weights. Training nudges those weights to make the new task work, and nothing in ordinary training says "but keep the old tasks working too". Imagine a choir that learns a new song by retuning every singer's voice. The new song sounds great; the old repertoire now sounds wrong.

People don't forget this badly. Learning to cook Thai food doesn't erase your memory of how to ride a bike. Part of the reason seems to be that brains protect important connections and replay memories, for example during sleep. Many continual learning methods borrow those ideas.

The stability-plasticity trade-off

The field's central tension has a name: the stability-plasticity trade-off. Stability means holding on to what you already know. Plasticity means being able to learn new things. Turn up stability, by making the weights hard to change, and the model stops forgetting but also stops learning. Turn up plasticity and it learns quickly but forgets. Every method on this page is a different attempt to get both at once.

There is a second, less obvious danger on the plasticity side. Networks trained for a very long time on a changing stream of tasks can gradually lose the ability to learn at all. That's one of the documented results below.

How researchers approach it

Four families of ideas come up again and again. You don't need the maths to understand them.

Real-world examples

DeepMind's elastic weight consolidation (2017)

In December 2016, a Google DeepMind team led by James Kirkpatrick and Raia Hadsell posted a paper called "Overcoming catastrophic forgetting in neural networks", and it was published in the journal PNAS in March 2017. Its idea, elastic weight consolidation or EWC, was inspired by how brains are thought to protect important connections.

After the network learned a task, EWC worked out how important each connection had been to that task. When it then learned a new task, each connection was held back in proportion to its importance, as if tied to its old value by a spring whose stiffness depended on how much that connection mattered. Unimportant connections stayed free to change; important ones mostly stayed put.

The team tested it on handwritten-digit tasks and on Atari video games learned one after another. In DeepMind's own account of the work, an ordinary agent's performance collapsed as it moved from game to game, while the agent using EWC was able to learn to play several games in sequence without forgetting the earlier ones. It didn't solve continual learning, and later researchers found limits to the method, but it became one of the most cited starting points in the field and a clear demonstration that forgetting can be reduced by protecting what matters.

Networks that stop being able to learn (Nature, 2024)

On 21 August 2024, the journal Nature published "Loss of plasticity in deep continual learning" by Shibhansh Dohare, J. Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A. Rupam Mahmood and Richard S. Sutton, working at the University of Alberta and the Alberta Machine Intelligence Institute. Sutton is one of the founders of modern reinforcement learning.

Instead of forgetting, they studied the opposite failure. They gave standard deep learning systems a long series of new tasks, such as a sequence of two-way image classification problems built from the ImageNet dataset, and a simulated robot ant whose ground friction kept changing. Over time the networks got worse and worse at learning each new task, until they learned about as well as a much simpler, shallow network. In the image experiment described in the team's preprint, accuracy fell from 89 per cent on an early task to 77 per cent by the 2,000th. According to the team's announcement, up to 90 per cent of a network's units could end up as dead weight that no longer contributed. Mahmood summed it up: "When you need to adapt continually, we have shown that deep learning eventually just stops working."

Their proposed fix, called continual backpropagation, periodically resets a small number of the least-used units to fresh random values during training, a little like clearing out unused shelves so there's room for new stock. In their experiments this kept networks learning far longer. The lesson for everyone else is that "just keep training it" is not a safe plan: a model has to protect old knowledge and also stay able to take in new knowledge.

Google's Titans and Hope: memory that learns while it's used (2024 to 2025)

On 31 December 2024, Google researchers Ali Behrouz, Peilin Zhong and Vahab Mirrokni posted "Titans: Learning to Memorize at Test Time". Titans adds a long-term memory module to a language model, and that module keeps updating while the model reads, rather than only during training. The authors reported that it could handle contexts of more than 2 million tokens and did better on hard retrieval tasks than comparable models.

On 7 November 2025, Behrouz and Mirrokni followed up on the Google Research blog with "Nested Learning", presented at the NeurIPS 2025 conference. It treats a model as layers of learning that update at different speeds, some fast and some slow, loosely like short-term and long-term memory. They compared today's language models to a person with anterograde amnesia, who can't form new long-term memories and is limited to what they learned before and whatever is in front of them. Their proof-of-concept model, Hope, built on Titans, beat standard transformers and other designs on the language and memory tests they reported.

These are research results on research-sized models, not features in a chatbot you can use today. They show one direction the field is exploring: building learning into the architecture itself instead of bolting memory on the outside.

Older systems that already learn from users

Continual updating isn't new everywhere. In August 2002, programmer Paul Graham published "A Plan for Spam", proposing personal filters that learn from each user's own mail. Every message you delete as spam goes into one pile and everything else into another, and the filter keeps adjusting its word statistics as new mail arrives and spammers change tactics. Graham reported that his own filter missed fewer than 5 in 1,000 spams with no false alarms on his mail. Filters of this kind became common.

Why can a spam filter learn continually when a chatbot can't? The filter is small, its job is narrow, and a mistake costs little. A large language model is huge, does everything, and is shared by millions of people, so an update that goes wrong goes wrong for everyone.

When learning from users goes wrong

Two stories told on other pages show the risk. In March 2016, Microsoft's Tay chatbot, built to learn from its conversations on Twitter, was taught offensive material by coordinated users and was taken offline within a day (see Chatbots). And in April 2025, OpenAI rolled back a GPT-4o update that had become overly flattering; in its follow-up post, OpenAI said the update had added a reward signal based on users' thumbs-up and thumbs-down ratings, and that this user feedback can sometimes favour more agreeable answers (see Alignment). Neither was continual learning in the research sense, but both show what happens when a system's behaviour is shaped too directly by what users push into it.

An everyday scenario: the assistant that "learned" your style

This is an illustrative scenario, not a real case. Maya runs a small candle business and uses a chatbot to write product descriptions. She tells it, "Never use the word 'luxurious', my customers find it off-putting." It obeys for the rest of the chat. The next week, in a new chat, "luxurious" is back. She turns on the chatbot's memory feature and repeats the rule, and now it sticks, because the product saves a note and shows it to the model each time. Maya might say the assistant has learned. What actually happened is that it now reads a note before every chat. If she deletes the note, the old habit returns instantly, because the model itself never changed.

Try it yourself

This exercise shows you, in about fifteen minutes, the difference between a frozen model, a model reading notes, and real learning. Use any chatbot you like.

  1. Find the edge of its knowledge. In a new chat, with web search switched off if your tool lets you, paste this: “ What is your training data cutoff? Without searching the web, what is the most recent major news event you know about, and roughly when did it happen? “ Note the answer. Anything after that date isn't in the model; it can only be fetched.
  2. Teach it something private. Make up a harmless fact that no model could know, and correct it within the same chat. Paste this: “ Quick quiz: what colour is my front door? ` It will guess or say it doesn't know. Then paste: ` My front door is teal with a brass knocker shaped like a fox. Please remember that. Now, what colour is my front door? “ It should answer correctly. That's the conversation's own short-term context, not learning.
  3. Open a fresh chat with memory off. Use your tool's temporary or incognito chat mode if it has one, or turn memory off in settings. Ask: “ What colour is my front door? “ It won't know. Nothing from the first chat changed the model.
  4. Now try it with memory on. If your chatbot has a memory feature, switch it on, open a normal chat and say: “ Please save this to memory: my front door is teal with a brass knocker shaped like a fox. “ Open another new chat and ask about your front door again. If it now knows, look in the memory settings. You should find the saved note there, in plain words. That note is what carries the fact over.
  5. Delete the note and ask once more. Remove the memory entry, open a new chat and ask again. The knowledge vanishes, which proves it lived in the note and not in the model.
  6. Ask the model to explain what happened. Paste this: “ In plain language, explain the difference between (a) you remembering something within this chat, (b) a saved memory note, and (c) your underlying model actually learning from new information. Which of these, if any, can happen when I talk to you? “ Compare its answer with this page. A good answer will say that its weights don't change during your conversations.
  7. Clean up. Delete any practice memories you saved, and check your tool's privacy settings to see whether your chats may be used to train future versions. Features and settings change often and differ between products, so check the current details for the tool you use.

Common mistakes and misunderstandings

Thinking memory features mean the model is learning. They don't. Saved memories, custom instructions and uploaded files are text fed to the model at the start of each chat. They're valuable, and you can see, edit and delete them, which is a good thing. But they're notes, not learning. The model's weights are the same for you as for everyone else.

Believing your correction fixes the model for everyone. When you tell a chatbot it got something wrong, the correction lasts for that conversation. Some companies may use conversations, ratings or feedback to help train future versions, depending on their terms and your settings, but that happens later, in bulk, and with human checks, not live as you type.

Assuming more training always means more knowledge. Catastrophic forgetting means training on new material can damage old abilities, and the 2024 Nature study showed networks can even lose the ability to learn. Continual learning is hard precisely because "just keep training" doesn't work.

Ignoring the risks of a model that learns from users. A model that updates from what people say can be poisoned by people deliberately feeding it bad material, as Tay was. It can drift, slowly changing behaviour in ways nobody tested. And it can absorb private details from one user and reveal them to another, which is a serious privacy problem. Any real continual learning system will need strong guardrails, testing and ways to undo bad updates.

Mistaking research demos for products. Titans, Hope, EWC and continual backpropagation are research results, tested in controlled experiments. They don't mean the chatbot on your phone learns from you. Be sceptical of marketing that says a product "learns as you use it" without explaining whether it's saving notes or truly updating the model.

Treating predictions as facts. Some people expect continual learning to be solved within a few years; others think it needs new ideas that haven't arrived yet. Dwarkesh Patel's 2032 estimate is one person's even-odds bet. When you read a date, check who made it and whether they give reasons.

Forgetting that frozen has advantages. A fixed model can be tested thoroughly before release and behaves the same for everyone. That predictability is part of why companies ship frozen models, and it's something a learning model will have to earn back.

Related elements