XCooking

Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks

Xiaofeng Gao^1, Ran Gong^1, Yizhou Zhao¹, Shu Wang¹, Tianmin Shu² and Song-Chun Zhu¹

Center for Vision, Cognition, Learning, and Autonomy, UCLA¹

(^*Equal contribution)

Abstract

Human collaborators can effectively communicate with their partners to finish a common task by inferring each other's mental states (e.g., goals, beliefs, and desires). Such mind-aware communication minimizes the discrepancy among collaborators' mental states, and is crucial to the success in human ad-hoc teaming. We believe that robots collaborating with human users should demonstrate similar pedagogic behavior. Thus, in this paper, we propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations, where the robot builds a hierarchical mind model of the human user and generates explanations of its own mind as a form of communications based on its online Bayesian inference of the user's mental state. To evaluate our framework, we conduct a user study on a real-time human-robot cooking task. Experimental results show that the generated explanations of our approach significantly improves the collaboration performance and user perception of the robot.

Paper and Demo

Paper [Link]

Xiaofeng Gao, Ran Gong, Yizhou Zhao, Shu Wang, Tianmin Shu and Song-Chun Zhu. Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks. IEEE International Conference on Robot Human Interactive Communication (RO-MAN), 2020.

@inproceedings{gao2020joint,
  title={Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks},
  author={Gao, Xiaofeng and Gong, Ran and Zhao, Yizhou and Wang, Shu and Shu, Tianmin and Zhu, Song-Chun},
  booktitle={2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)},
  pages={1119--1126},
  year={2020},
  organization={IEEE}
}

Demo

Code

Code for the collaborative cooking game is available here.

Please cite this paper if you use the code: