Multimodal Live Commentary Generation and Evaluation

Cutting-edge natural language generation (NLG) technology has been applied to live commentary generation that improves the understanding and the excitement of audience through language．Realtime live commentary generation requires 1) multimodal NLG techniques that understand input sensor data and visual data, 2) NLG techniques under time constraints such as “when to say” and “how long”, 3) generation of non-play-by-play utterances like “Currently the player is in the 3rd place”.  4) Multi-party live commentary generation between play-by-play announcer and color commentator, 5) Automatic and human evaluation methodology for live commentary.

This project develops the live commentary generation technology from multimodal input as well as evaluation methodology.