CBI | ATR NIA Labs

When we see and respond to something, in our brains signal transmissions rapidly occur on the order of 10 milliseconds. Revealing such fast brain dynamics is very important for understanding the mechanisms underlying our brain functions and behaviors.

The Department of Computational Brain Imaging (CBI) investigates and develops methodologies for “multimodal data integration” to elucidate the dynamics of the human brain.

In the following, we explain why multimodal data integration is necessary for revealing human brain dynamics.

Measurement of Human Brain Activity

Neuroscience aims to elucidate how our brain functions (e.g., perception, behavior, and consciousness) emerge from our brain activities.

The first step toward this goal is to measure brain activity, which is the electrical activity of neurons. Measurement of neuronal activity requires sensors (electrodes) to be placed near the cells, making head surgery inevitable. This type of measurement involving surgical procedures is called invasive brain measurement. Since such measurement cannot be applied to healthy subjects, we use the so-called “non-invasive brain measurement,” which does not require surgical procedures.

Currently, there are four types of non-invasive brain measurements: electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and near-infrared spectroscopy (NIRS). By the way, positron emission tomography (PET) also does not require surgery, but it is not considered a noninvasive measurement because it requires inhalation of radioactive material. Because the sensors of these non-invasive brain measurements are placed far from neurons, they can detect signals only when tens or hundreds of thousands of neurons activate simultaneously. The measurement systems of MEG and fMRI are huge, and they are mainly used for basic research. In contrast, the measurement systems of EEG and NIRS are portable and thus are used for basic research, clinical diagnosis, and industrial applications. We can divide these four measurements into two categories based on the object measured.

Measurement of neuronal electrical activity: EEG and MEG

EEG and MEG measure electric and magnetic fields produced by the electrical activity of a neuronal population. EEG is the oldest method of measuring brain activity, discovered by Dr. Hans Berger in the 1920s. Here, electrodes placed on the scalp measure the “potential change.” MEG’s development was initiated in the 1960s. It detects changes in magnetic fields with highly sensitive magnetic field sensors, or SQUID sensors, placed near the head. Recently, a new device, optically pumped magnetometer (OPM), has been put to use. An OPM sensor is small, enabling a wearable MEG measurement system.

Measurement of blood flow response: fMRI and NIRS

Responding to the neuronal activity, blood flow locally increases to supply oxygen and energy. This process is known as “neurovascular coupling.” fMRI and NIRS measure this blood flow response. fMRI uses the BOLD effect, discovered by Dr. Seiji Ogawa in 1990, to detect the amount of magnetic materials (mainly deoxygenated hemoglobin in cerebral blood flow) from the turbulence of nuclear magnetic resonance. fMRI can accurately determine the location of brain activity on the order of a few millimeters by irradiating electromagnetic waves in an inclined magnetic field and measuring the responses with sensors placed outside the head. NIRS was first applied to brain function measurement in the early 1990s. NIRS takes advantage of the fact that near-infrared light effectively penetrates the skin and is specifically absorbed by hemoglobin in the blood. NIRS measures the amount of hemoglobin changes in the light path by detecting light emitted from a light source set on the scalp with an optical sensor several centimeters away.

Inadequacy of Each Measurement

As shown in the figure below, no method satisfies the requirement for both high temporal and high spatial resolution in revealing brain dynamics. The figure below plots the temporal and spatial resolutions of the four brain measurements. Solid lines indicate measurements that can identify the source location from data, and dotted lines indicate measurements that cannot do this.

fMRI has high spatial but low temporal resolution, while MEG and EEG have high temporal but low spatial resolution.  Although NIRS has low spatial and temporal resolution, it is expected to become a method that can measure various targets in various environments due to its portability. fMRI is the only method capable of pinpointing the exact location of brain activity. On the other hand, for MEG, EEG and NIRS, we cannot determine the location from simply observing their data (although we can estimate it).

In other words, no measurement method satisfies all of the following three properties needed for revealing brain dynamics: high temporal resolution of less than a second, high spatial resolution of less than a centimeter, and the ability to localize brain activity.

(Remark) Let us consider the temporal and spatial resolutions sufficient for elucidating human brain functions. Regarding spatial resolution, electrophysiological studies have shown that the brain has functional units called column structures. Since their size is on the 100-micrometer order (one-tenth of a millimeter), a spatial resolution of 100 micrometers is considered sufficient. Since their size is on the 100-micrometer order (one-tenth of a millimeter), a spatial resolution of 100 micrometers is considered sufficient. As for temporal resolution, a human’s shortest response time is known to be around 100 milliseconds. Therefore, to investigate the relationship between brain activities and behavior, a temporal resolution higher than 100 milliseconds would be sufficient. Moreover, the measurement target must be a neural response, not blood flow. In summary, the following brain measurement would be ideal: a method that measures neural responses comprehensively over the entire brain with a spatial resolution of one-tenth of a millimeter and a temporal resolution of milliseconds.

Need for multimodal data integration

Due to fMRI’s excellent advantages of the high spatial resolution, human brain function studies, such as functional brain mapping and macro connectome studies, have relied heavily on it. On the other hand, it is also true that human neuroscience has been weighted toward a way of understanding that ignores temporal information, since this community has been heavily biased by the performance of fMRI measurement. To understand the brain’s information processing as it actually functions, it is important to investigate fast brain dynamics using MEG and EEG, which have excellent temporal resolution. Therefore, combining fMRI with MEG and EEG is considered a promising approach.