Asking Follow-Up Clarifications to Resolve Ambiguities in Human-Robot Conversation
summary
An interactive system asks targeted yes/no clarifications to disambiguate object references using Grad-CAM activation heatmaps to find candidate regions, k‑means clustering, an object detector, a relation presence network and a relation informativeness network to pick the best spatial question, and a transformers-based module to read undetectable targets. Tested on a Pepper robot with 63 participants and up to two clarifications, it produces more correct identifications, fewer conversational turns, fewer unknown trials, and higher perceived ease and competence.
AI-written summary · about