Sleep framework: A new way for LLMs to retain knowledge in continual learning
Researchers introduced the Sleep framework, a method enabling large language models to preserve learned information during continual training. By simulating sleep-like consolidation, the system selectively freezes and reactivates parameters, reducing catastrophic forgetting. Experiments show improved retention across diverse tasks while maintaining performance. The approach offers a scalable solution for lifelong learning in AI models. It also reduces computational overhead.