Goal

The goal of our study is to develop a walking assistive device and control it by brain activity. We are collaborating with Duke University (Miguel Nicolelis Laboratory) to control CB-i, our humanoid robot, using a monkey’s brain activity. This control framework is considered one BMI technology to control robotic devices without using the body movements of users.

In this study, a monkey’s brain activity is extracted from its primary motor cortex when the monkey is walking on a treadmill(see the movie). The phase information of periodic walking patterns are decoded from the extracted brain activity and used to control our humanoid robot.

Real-time network connection of a monkey’s brain and humanoid robot CB-i

The extracted monkey’s brain activity was transferred from Duke University to ATR in Kyoto. The transferred decoded brain activity was used to generate the leg trajectories of our humanoid robot, CB-i(see IMG1). With this network system, future study includes an experiment in which the monkey learns to control the humanoid robot.

Extracting information related to walking behaviors from brain activity

All neural activities need to be identified by a spike sorting process. In this study, from 150 to 300 neural activities are detected using a 100-electrode array. From these neural activities, we extract information related to walking behaviors.

However, not all neural activities contribute to represent walking behaviors. In addition, BMI parameters for extracting information are designed before on-line decoding of movements from brain activity. Therefore, perhaps pre-designed parameters are not proper parameters for the on-line estimation of movements any more.

In our study, we used Bayesian estimation to estimate information related to walking behaviors and BMI parameters. We also extracted neurons that contribute to represent walking behaviors through a model selection algorithm.

Estimation of phase information from brain activity

Different from existing BMIs that are used to control cursors or arm robots, the BMIs for biped walking need to control underactuated systems. Therefore, if the body dynamics of users and robots are different, it would be difficult to directly use the decoded leg joint angles from users’ brain activity to control the biped system.

In our approach, we propose soft coupling between the detected brain activities with biped systems. We use a biped walking controller that has internal dynamics with an internal state. The detected brain activities are used to modulate the internal dynamics of the biped controller rather than directly modulating such controller output as joint angle trajectories.

We use a coupled phase oscillator system as the biped controller dynamics. By softly coupling the monkey’s brain activity with the coupled oscillator system（see IMG2）, the walking patterns of the humanoid robot were synchronized with the monkey’s walking behaviors (see the movie).