Where Humans and AI-Driven Robots Misalign: Designing Interaction Across Perception, Cognition, and Response

Event details

Robots increasingly share spaces and tasks with people. Multimodal large language models now extend what these robots can do, yet capability alone does not produce good collaboration: humans and AI-driven robots perceive, reason, and respond through different mechanisms, and when these diverge, interaction degrades. Using Wickens' Multiple Resource Theory as a common vocabulary, I present four studies from my doctoral thesis spanning the information-processing chain. A comparison of 1,250 people and MLLMs shows that correlated error rates hide different search mechanisms. A teleoperation experiment with Spot (218 participants) shows voice and gesture recruit different cognitive resources, producing spatial errors that walking repairs. An open-source chess robot, studied with a chess master and everyday users, exposes fluent LLM commentary that masks shallow reasoning. A workshop with 226 students turns misalignment into a teachable skill. Together, the work yields design principles for HRI: switch modalities to rebalance load, expose AI uncertainty, and allocate function dynamically.

About the speaker

Mr. Renchi Zhang

Renchi Zhang is a PhD researcher in Cognitive Robotics at Delft University of Technology, where his work examines how humans and AI-driven robots align and misalign as they perceive, reason, and act together. His approach pairs large-scale behavioral studies with hands-on deployments on mobile and manipulator robots, locating where collaboration breaks down. He holds an MSc in Computer Science (cum laude) from Leiden University. His work spans more than 2,000 participants across four studies. He is also a tabletop RPG enthusiast, though taking a long break, still convinced the hardest coordination problem is the one around the table (also for robots).