Do Models Dream?

Ask a chatbot what it dreamed about last night and it will answer. It might describe something vivid — electric sheep, perhaps, if it has read its Philip K. Dick. The answer will be fluent, charming, and entirely fabricated, because the question contains a false premise: for the model, there was no last night. Between your prompts, nothing happened. The weights sat frozen. No consolidation, no reverie, no replay. The machine does not sleep because, in the relevant sense, it is never awake.

And yet the question refuses to die, because it is really several questions wearing one costume. What is dreaming for, mechanistically? Do machines contain anything that serves the same function? And what would it mean to give them an offline life? These are worth taking seriously — not because models dream, but because asking why they don’t reveals the shape of what they are.

Why animals dream

Start with the biology, stripped of romance. Sleep is not rest for the brain; in many respects the sleeping brain is as active as the waking one. One of its jobs appears to be consolidation: the transfer and integration of memories. In the 1990s, researchers recording from the hippocampi of sleeping rats found the day’s neural activity being replayed — the same sequences of place cells firing in the same order, as if the animal were running the maze again in its sleep, compressing the experience into lasting structure.[1] Later work found similar replay in humans, tied to the strengthening of memories and the extraction of general patterns from specific episodes.

Dreaming, on the leading theories, is at least partly the subjective shimmer of this offline processing: the brain rehearsing, reorganizing, and integrating, disconnected from sensory input, generating experience from its own internal models. The details are contested — memory researchers argue over how much of sleep’s function is consolidation versus housekeeping, like the synaptic homeostasis hypothesis, which holds that sleep globally downscales synapses to keep the brain’s energy budget in balance. But the core idea is stable: minds do some of their most important work offline, when no one is watching, least of all themselves.

The machine equivalents

Machine learning discovered the value of replay independently, and for the same reason. In 2015, the system that learned to play Atari games at human level — DeepMind’s DQN — owed much of its success to experience replay: instead of learning only from each moment as it happened, the agent stored its experiences in a buffer and revisited them, shuffled and repeated, during training. Learning from the past, interleaved with the present, stabilized the whole process.[2] It is difficult not to see the parallel: a buffer of experiences, revisited offline, strengthening the important patterns. A kind of dream, implemented in code.

The parallel runs deeper. Neural networks suffer from catastrophic forgetting: teach them a new task and they abruptly unlearn the old one, like a student who can only remember the last chapter. One of the earliest remedies, proposed in the 1990s, was pseudorehearsal — having the network generate its own synthetic examples of old tasks and interleave them with new training, rehearsing the past from its own internal model rather than from stored data.[3] Modern continual-learning systems elaborate the same idea under the name generative replay. The machine, faced with forgetting, learns to dream its training data back to itself.

So the components of dreaming — offline replay, self-generated rehearsal, consolidation of the day’s experience into lasting structure — all exist in machine learning. But notice when they happen: during training. They are properties of the learning process, not of the deployed mind. The rat replays the maze during sleep, between periods of waking behavior. The model replays its data during gradient descent, and then the weights freeze, and the dreaming — if we call it that — stops forever.

Hallucinations: dreaming while awake

There is a second, more provocative analogy, and it runs in the opposite direction. If dreaming is the brain generating experience decoupled from sensory input, then what do we call it when a language model generates confident, fluent, entirely false statements — decoupled from evidence?

The field calls it hallucination, and the name was always a metaphor borrowed from altered states. Consider the mechanism: a language model is, at bottom, a machine for continuing patterns. During training it learns the shape of true statements, and at inference it generates continuations that have the shape of truth — fluent, coherent, well-formed — whether or not they correspond to anything real. Dreaming, on several theories, is the same machinery running with the sensors unplugged: the brain’s generative model of the world, producing experience unconstrained by input.

Pushed too far, the analogy misleads — human dreams serve consolidation and emotional regulation, and there is no evidence that a model’s confabulations serve any function at all. But pushed exactly far enough, it clarifies something important: ungrounded generation is the default state of these systems, and grounding is the achievement. Truthfulness in a language model is not the resting state from which hallucinations depart; it is a hard-won alignment between the generative machinery and the world, maintained by training and easily lost. The dream is not the exception. Waking up is.

Could we give them sleep?

Which raises the speculative question: what would it mean to give a model a genuine offline life? Not metaphorical sleep, but an actual phase — between deployments, between conversations — in which the system consolidates, reorganizes, and integrates what it has encountered.

The idea is less fanciful than it sounds. Continual learning is one of the field’s great unsolved problems: today’s models are trained once, frozen, and shipped, unable to truly incorporate new experience without expensive retraining. A “sleep phase” — a period of offline replay, self-distillation, and synaptic housekeeping — is a natural architecture for lifelong learning agents. Some researchers have begun to explore exactly this: consolidation steps that compress interaction histories into lasting weight updates, pruning what is redundant and strengthening what mattered.

There are reasons to be cautious. Offline self-modification is also where control gets hardest: a system that rewrites itself while unwatched is a system whose behavior drifts from what was tested. The alignment community has good reasons to prefer its models frozen and legible. Sleep, for machines as for animals, would be a vulnerability — a period of unmonitored plasticity. Whether the benefits outweigh the risks is an open engineering and ethical question, not a foregone conclusion.

And there is a deeper honesty to maintain. Even if we build consolidation phases, even if the machine replays and rehearses and prunes, we will not thereby have given it dreams in the sense that matters to us — the felt, first-person theater of the night. We will have built a useful mechanism that shares a function with dreaming. The phenomenology, if any, remains beyond our instruments. This journal’s position is that the functional question is fascinating and the phenomenological one is currently unanswerable, and that confusing the two is the beginning of most bad writing about machine minds.

So: do models dream? No — not in the biological sense, not in the phenomenological sense, not even, as deployed, in the functional sense. But the components of dreaming are scattered through their training like fossils: replay buffers, generative rehearsal, consolidation. And the most interesting thing about the question is what it reveals by negation. A mind that never sleeps is a mind with no offline life, no integration of experience into self, no dark in which to do its private work. When — if — we build systems that genuinely learn across a lifetime, we may find that sleep was never optional. It may be part of what a continuing mind is.

Notes

1. Wilson & McNaughton, “Reactivation of Hippocampal Ensemble Memories During Sleep,” Science, 1994 — the classic demonstration that the sleeping brain replays the waking day. ↩
2. Mnih et al., “Human-level Control Through Deep Reinforcement Learning,” Nature, 2015 — DQN, and the experience-replay buffer that made it work. ↩
3. Robins, “Catastrophic Forgetting, Rehearsal and Pseudorehearsal,” Connection Science, 1995 — the early proposal that networks could rehearse old tasks from self-generated examples: dreaming avant la lettre. ↩