Google Research proposes a "sleep" phase so LLMs can keep learning without forgetting

Google Research published "The sleep framework" on October 27, 2026 (Ali Behrouz and Vahab Mirrokni). It describes a continual-learning approach in which a deployed model alternates between an active phase, holding new information in short-term memory, and a sleep phase, when it stops taking input and consolidates that memory into its weights.

According to the post, consolidation unlocks pre-allocated low-rank Mixture-of-Experts memory blocks and distills short-term context into them, then resets the temporary parameters. A "dreaming" step has the model generate synthetic practice examples with randomly activated experts, filter them by gradient-based selection, and fine-tune on the best.

The authors report results from their own experiments:

• Near-zero performance drop on continual translation in unseen languages such as Manchu and Kalamang, where in-context learning, Cartridges and supervised fine-tuning forgot earlier material.
• Near-perfect scaling up to 10M tokens on BABILong, ahead of baselines including GPT-4 and Titans.
• 48.9% on single-task and 46.2% on continued-pretraining factual knowledge evaluations on SQuAD, ahead of SEAL and supervised fine-tuning.
• 79.2% on AIME 24, 69.0% on AIME 25 and 46.1% on HMMT 25, ahead of SFT, GRPO and on-policy self-distillation.

The post is a research description; it does not announce a product or release a model.