DBI | ATR CMC Labs

Despite remarkable developments in artificial intelligence in these days, there have not been realized any robots that can tidy up entire houses yet.　This is because a high level of generalization ability is necessary in order to respond to diverse situations of room/furniture layout, clutter etc. By developing analysis methods for brain activity and life-log data, we are making a data-driven approach toward a better understanding of the human ability to grasp complicated situations in real world and to take appropriate actions. We also aim at constructing intelligent systems based on such deeper understanding in data-driven human science, and at creating novel services of daily-living assistance.

Research Theme
1. Analysis techniques for multi-sensor data
Humans understand environmental situations by integrating multiple sensory inputs and take appropriate actions. In order to enhance the ability of intelligent systems to grasp situations in real environments, we are developing methods for analyzing multi-sensor data acquired from human subjects during daily activities at home. In [1], we proposed a novel feature representation capturing relations between egocentric video and hands/head motions (accelerations) and constructed a system for memory assistance which can retrieve past egocentric videos including motions similar to gesture queries. In [2], we developed a machine learning method to extract key frames (time intervals containing important information such as daily activities) from stream data of egocentric videos and accelerations acquired simultaneously.
[1] Miyanishi et al., “Egocentric video search via physical interaction”, Proc. of the 30th AAAI Conf. on Artificial Intelligence (AAAI-16), Vol.1, pp.330-336, 2016
[2] Li et al., “Key frame extraction from first-person video with multi-sensor integration”, Proc. of the IEEE Int. Conf. on Multimedia and Expo (ICME), 2017.

2. Characterization of variability in brain activities across subjects and over time
Daily behaviors are influenced by personality and situations. For adapting daily-living assistance to each person, it is necessary to incorporate such information in intelligent systems. We developed interpretable decomposition of variability in brain connectivity across subjects and over time from brain imaging data [3]. We further extend it to the novel method which can estimate module structures in brain networks and variability of connectivity between the modules simultaneously [4]. When applying this approach to a large database of resting-state fMRI, we obtained as a result the following modules: i) default mode network (DNM) which becomes active during rest, ii) sensory networks, and iii) saliency network and a reasonable variability pattern of connectivity between these modules, which coincide with the hypothesis that saliency network may switch between DMN and sensory networks.
[3] Hyvaerinen et al., “Orthogonal connectivity factorization: Interpretable decomposition of variability in correlation matrices”, Neural Computation, Vol.28, No.3, pp.445-484, 2016.
[4] Hirayama et al., “Characterizing variability of modular brain connectivity with constrained principal component analysis”, PLoS ONE, Vol.11, No.12, e0168180.

3. Sensing daily-living environments by mobile robots
In sensing private environments, many surveillance cameras may make people nervous. As an alternative, we are developing intelligent systems with mobile robots for gathering information about daily-living environments. We aim at such robots that can collect information efficiently without disturbing people through better understanding about environmental situations and persons’ conditions.

Members

• Motoaki Kawanabe
• Taiki Miyanishi
• Marc Patrick Zapf
• Jun-ichiro Hirayama (RIKEN AIP / ATR)
• Atsunori Kanemura (AIST / ATR)
• Luis Yoichi Morles (Nagoya U. / ATR)
• Aapo Hyvaerinen (UCL)
• Hiroshi Morioka (UCL / ATR)
• Hande Celikkanat (U. Helsinki)

Projects
– CREST Harmonious Intelligence Area “Construction of artificial consciousness and its implementation in real environments by integrating axiomatic approach in neuroscience and constructive approach in engineering” (PL: Ryota Kanai)
– JST SICORP Japan-Finland Collaboration “Next generation affective life log: machine learning with multimodal sensor networks” (PL: A. Hyvaerien, M. Kawanabe)
– KAKENHI (B) “Transferable BMI by extracting common characteristics across many users and many situations” (PL: M. Kawanabe)
– RIKEN AIP Information Integration for Neuroscience Team (TL: M. Kawanabe)