Towards Seamless Human-Robot Collaboration (HRC): Artificial Intelligence (AI)-empowered Robot Learning from Human Demonstration

Currently, a new industrial revolution, also known as Industry 4.0, is transforming manufacturing industries towards higher production efficiency, greater levels of automation, more intelligence in processes, increased product complexity, and enhanced customization flexibility. Human-robot collaboration (HRC) is considered pivotal for this transition, wherein robots work alongside their human partners to automate repetitive, physically demanding tasks and to replace humans in hazardous or extreme working environments. On the other hand, integrating humans with robots enhances the flexibility and adaptability of manufacturing tasks, given that the high-level cognitive and decision-making abilities of human workers still surpass those of robots. However, the current level of collaboration between humans and robots is far below what is expected for human-like collaboration. From a human perspective, current methods for collaborating with robots are unintuitive, time-consuming, and lack intelligence. It is challenging for robots to understand human workers' high-level intentions and respond appropriately. Moreover, human expertise, cognitive abilities, and decision-making capabilities have not been effectively transferred to and utilized by robots. To tackle these issues, Dr. Zhang has delved into a multidisciplinary research area that intersects multiple domains, including Human-Computer Interaction (HCI), AI, Robotics, and Computer Vision. In this talk, Dr. Zhang will present his research progress in three aspects: 1) Extended Reality (XR) interfaces and associated new paradigms to enable human communication and demonstration; 2) AI-enabled robot grasp planning learned from human demonstrations; and 3) Visual language model-based multimodal robot learning from human instructions and demonstrations for long-horizon tasks. A few future research directions will also be discussed.

Dr. Yunbo "Will" Zhang is currently an Assistant Professor in the Department of Industrial & Systems Engineering at the Rochester Institute of Technology (RIT). He is also affiliated with the School of Information (iSchool) at RIT. His research interests include Human-Robot Collaboration, Human-System Interaction, AI and Machine Learning in Engineering, and Smart Manufacturing. He has actively taken on leadership roles in editing papers and organizing technical sessions in his field, including: Program Chair for the Symposium on Solid and Physical Modeling (SPM) in 2024; Associate Editor for the Frontiers in Robotics and AI starting from 2024; Associate Editor for the Journal of Intelligent Manufacturing (JIMS) starting from 2022; and Chair of the Virtual Environments and Systems (VES) Technical Committee within the ASME CIE Division for 2023–2024. He has also led several guest editorships in the fields of HRC and XR, including: the ASME Journal of Computing and Information Science in Engineering (JCISE), Special Issue on Human-Robot Collaboration in Industry 5.0, starting from January 2024; ASME JCISE, Special Issue on Extended Reality in Design and Manufacturing, from August 2022 to March 2024; and the Journal of Manufacturing Systems (JMS), Special Issue on Augmented Reality Applications in Smart Manufacturing, from August 2019 to August 2020.