Language Models with Combined Data Modalities for AI Systems
By Teruko Mitamura
This collaborative project between CMU and the National Institute of Advanced Industrial Science and Technology, Japan (AIST) seeks to develop multimodal systems for advanced human-machine interactions through discussion and implementation. Especially, the CMU team focuses on the temporal reasoning capabilities of multiple modalities. Multimodal systems with advanced temporal reasoning capability will be beneficial in analyzing and understanding different types of data jointly.