Neural Decoding of Visual Imagery During Sleep
Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013). Neural decoding of visual imagery during sleep. Science, 340(6132), 639-642.
Dreaming is a subjective experience during sleep that is often accompanied by vivid visual experience. Due to the fundamentally subjective nature of dreaming, the objective study of dream contents has been challenging. However, recent advances in analysis methods of neuroimaging data have made it possible to uncover direct links between specific dream contents and brain activity patterns.
In this study, we performed dream content decoding from human brain activity during sleep. We hypothesized that the contents of visual dreaming during sleep have some similarity with visual cortical activity patterns while viewing visual stimuli when awake. Thus we trained decoders on awake brain activity induced by natural images from web image databases.
We focused on visual dreams experienced during the sleep-onset (hypnagogic) period (sleep stage 1 or 2), because it allowed us to collect many observations by repeating awakenings and recording subjects’ verbal reports of dreaming. Three subjects participated in the fMRI sleep experiments, in which they were awakened when an EEG signature was detected and then asked to give a verbal report, freely describing their visual experience(s) before awakening. We repeated this procedure to attain at least 200 awakenings with a visual report from each subject. We analyzed the verbal reports using a lexical database to create systematic labels for visual contents (about 20 object categories).
We constructed decoders by training linear support vector machines (SVM) on fMRI data measured while each subject viewed web images for each object category. We tested the decoders on brain activity patterns during sleeping, and demonstrated that the dream contents subsequently reported by the subjects could be predicted from the visual cortical activity patterns measured immediately before awakening.
Our results suggest that the specific contents of visual dreams during sleep are represented by and can be read out from visual cortical activity patterns shared with awake stimulus representation. Our method may further work beyond the bounds of sleep stages and reportable experience to uncover the dynamics of spontaneous brain activity in association with stimulus representation. We expect that it will lead to a better understanding of the functions of dreaming and spontaneous neural events.
Figure 1. Experimental overview. fMRI data were acquired from sleeping subjects, along with simultaneously recorded polysomnography. Subjects were awakened during sleep stage 1 or 2 (red dashed line) and verbally reported their visual experience during sleep. fMRI data immediately before awakening were used as the input for main decoding analyses. Words describing visual objects or scenes (red letters) were extracted. The visual contents were predicted using machine learning decoders trained on fMRI responses to natural images.
Figure 2. Example time course of decoder outputs for a single sleep sample (Subject 2, 118th awakening; character category, in bold).
Decoded dream contents over time: