Our research activities encompass a wide range of basic and applied research areas, including machine intelligence, intelligent robotics, spoken language processing, and machine learning. Recent research topics include domestic service robots, multimodal language understanding, language generation, time series prediction, space and environmental data prediction, imitation learning, and recommendation systems.
1. Multimodal Language Processing
To develop technologies for searching and explaining the real world, we are working across a broad spectrum of fields, from the development of multimodal learning methods based on foundation models and large language models, as well as their applications in robotics, image captioning, autonomous driving, fashion, and art. We won first place for a multimodal instruction-following task at the CVPR 2023 Embodied AI Workshop.
2. Domestic Service Robots
The increasing demand for support services for older and disabled people has spurred the development of domestic service robots as an alternative and credible solution to the shortage of caring labor. We won RoboCup@Home twice, which is the world's largest competition for DSRs.
3. Solar Flare Prediction
It is estimated that the economic loss from a Carrington-class solar flare will be approximately US $163 billion in North America. We developed the Flare Transformer, a solar flare prediction model that handles both line-of-sight magnetograms and physical features, and achieved super-human performance.
4. Explainable AI
Visual explanations for deep neural networks are important in terms of enhancing accountability about image processing and providing scientific insight to experts.
We developed the Lambda Attention Branch Networks, an extension of Lambda Networks that improves the explainability, and outperformed existing explanation generation methods in terms of interpretability.
5. Air Pollutants Prediction
Early deaths from air pollutants (PM2.5, etc.) are estimated to be 3.3 million per year worldwide. However, it is still difficult to predict them accurately. We developed an air pollutant prediction method in 2014, which is one of the earliest introduction of deep neural networks into this field.
6. Cloud Robotics
Cloud computing of robot functions enables low-cost, high-performance processing. We have developed a cloud robotics platform, Rospeex, which was used by more than 50,000 unique users in 2013-18.
7. Imitation Learning and Motion Analysis
Imitation learning provides a user-friendly interface to teach motions to robots. Our studies on human motion analysis and imitation learning can reduce cost for implementing robot motion.
8. Optimal Sensor Placement
Optimizing sensor placement on the robot's body and in the environment enables faster learning and more efficient information collection. We proposed a method that maximizes mutual information using submodularity.
9. User Profiling and Recommendation
There is a demand for recommendations of tourist spots based on ambiguous preferences. We have developed a method to recommend tourist spots based on ambiguous preferences. Our algorithm has been adopted by Kyoto City.
10. Open Data Search System
Many papers and data are available on the Web, but their search accuracy is very low compared to web pages. We developed spatio-temporal pseudo relevance feedback method and released it as a web service.