TouchPilot
A Guidance System that Assists Learning Complex 3D Structures
We present TouchPilot, a guidance system that allows blind users to learn a complex 3D structure independently through the guidede exploration.
It guides users step by step through navigation, exploration of hierarchical elements, and confirmation of their entire areas.
Step Control
Navigation
Information and Confirmation
Abstract
Making complex structures accessible to blind people is challenging due to the need for skilled explainers. Interactive 3D printed models (I3Ms) have been developed to enable independent learning of 3D models through activating audio labels. However, they present single-layered information and require users to identify interactive elements through a pinpointing action, which might be insufficient for learning complex and unfamiliar subjects. In this paper, we investigate I3Ms for complex structures. We propose TouchPilot, a guidance system designed based on a study that observed learner-explainer interaction styles. TouchPilot guides users step by step through navigation, exploration of hierarchical elements, and confirmation of their entire areas. A follow-up study found that the guidance system led to better learning outcomes and higher independence compared to a pinpointing system. Feedback suggests that being primed by the guidance system systematically, followed by pinpointing freely for review, is preferred for learning complex structures.
Video Figure [2:30]
Video Presentation [6:00]
Publication
Xiyue Wang, Seita Kayukawa, Hironobu Takagi, and Chieko Asakawa. 2023. TouchPilot: Designing a Guidance System That Assists Blind People in Learning Complex 3D Structures. In The 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23), October 22–25, 2023, New York, NY, USA. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3597638.3608426
Aknowledgement
We thank all participants who took part in our user studies. We also thank Hitoshi Matsuoka, Chief Science Communication Specialist, as well as Kodai Sano, Natsumi Nakano, Ruriko Nagashima, and Bunsuke Kawasaki, the Science Communicators at Miraikan -- The National Museum of Emerging Science and Innovation, for their contributions to the system design and related studies.
Contact
If you have any question about the project, or if you hope to have a hand-on experience of our system at Miraikan, please contact Xiyue Wang (wang.xiyue[at]lab.miraikan.jst.go.jp). We are happy to discuss the project with you.