` XCooking

Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks

Xiaofeng Gao1*, Ran Gong1*, Yizhou Zhao1, Shu Wang1, Tianmin Shu2 and Song-Chun Zhu1

Center for Vision, Cognition, Learning, and Autonomy, UCLA1

Massachusetts Institute of Technology2

(*Equal contribution)

Overview Overview

Abstract

Human collaborators can effectively communicate with their partners to finish a common task by inferring each other's mental states (e.g., goals, beliefs, and desires). Such mind-aware communication minimizes the discrepancy among collaborators' mental states, and is crucial to the success in human ad-hoc teaming. We believe that robots collaborating with human users should demonstrate similar pedagogic behavior. Thus, in this paper, we propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations, where the robot builds a hierarchical mind model of the human user and generates explanations of its own mind as a form of communications based on its online Bayesian inference of the user's mental state. To evaluate our framework, we conduct a user study on a real-time human-robot cooking task. Experimental results show that the generated explanations of our approach significantly improves the collaboration performance and user perception of the robot.

Paper and Demo

Paper [Link]

Xiaofeng Gao, Ran Gong, Yizhou Zhao, Shu Wang, Tianmin Shu and Song-Chun Zhu. Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks. IEEE International Conference on Robot Human Interactive Communication (RO-MAN), 2020.

@inproceedings{gao2020joint,
title={Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks},
author={Gao, Xiaofeng and Gong, Ran and Zhao, Yizhou and Wang, Shu and Shu, Tianmin and Zhu, Song-Chun},
booktitle={2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)},
pages={1--8},
year={2020},
organization={IEEE}
}

Demo

Code

Code for the collaborative cooking game is available here.

Please cite this paper if you use the code:

Xiaofeng Gao, Ran Gong, Yizhou Zhao, Shu Wang, Tianmin Shu and Song-Chun Zhu. Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks. IEEE International Conference on Robot Human Interactive Communication (RO-MAN), 2020.

Contact

Any question? Please contact Xiaofeng Gao (xfgao [at] ucla.edu).