Binghamton, United States – A video shows a robotic guide dog speaking to visually impaired participants as it leads them through a large office environment. The AI-powered robot provides real-time verbal guidance, announcing routes, obstacles, and surrounding features.

The system uses large language models to plan safe paths and verbally communicate with users before and during travel. This spoken interaction gives participants situational awareness that would otherwise be limited without vision.

Seven legally blind participants tested the robot, selecting preferred routes to a conference room. The robot guided them step by step, narrating corridors, obstacles, and other scene details along the way.

“This is very important for visually impaired or blind people, because situational and scene awareness is relatively limited without vision,” said Shiqi Zhang, associate professor at the Thomas J. Watson College of Engineering and Applied Science.

After the trial, participants rated the system’s usefulness and ease of communication. The combination of planning explanations and real-time narration was preferred.

Researchers plan to increase the robot’s autonomy and expand navigation for longer indoor and outdoor routes. Participants expressed excitement about the technology and its potential for everyday use.

“They were super excited about the technology, about the robots,” Zhang said. “They really see the potential for the technology and hope to see this working.”

Podcast: Play in new window | Download