Journal / International Conference Papers (peer-reviewed)

2023

1. ASSETS

   TouchPilot: Designing a Guidance System That Assists Blind People in Learning Complex 3D Structures

   Wang, Xiyue, Kayukawa, Seita, Takagi, Hironobu, and Asakawa, Chieko

   In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility 2023

   Abs DOI Project Page

   Making complex structures accessible to blind people is challenging due to the need for skilled explainers. Interactive 3D printed models (I3Ms) have been developed to enable independent learning of 3D models through activating audio labels. However, they present single-layered information and require users to identify interactive elements through a pinpointing action, which might be insufficient for learning complex and unfamiliar subjects. In this paper, we investigate I3Ms for complex structures. We propose TouchPilot, a guidance system designed based on a study that observed learner-explainer interaction styles. TouchPilot guides users step by step through navigation, exploration of hierarchical elements, and confirmation of their entire areas. A follow-up study found that the guidance system led to better learning outcomes and higher independence compared to a pinpointing system. Feedback suggests that being primed by the guidance system systematically, followed by pinpointing freely for review, is preferred for learning complex structures.

2022

1. ASSETS

   BentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors

   Wang, Xiyue, Kayukawa, Seita, Takagi, Hironobu, and Asakawa, Chieko

   In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility 2022

   Abs DOI Project Page

   Obtaining information before a visit is one of the priority needs and challenges for blind museum visitors. We propose BentoMuseum, a layered, stackable, and three-dimensional museum map that makes complex structural information accessible by allowing explorations on a floor and between floors. Touchpoints are embedded to provide audio-tactile interactions that allow a user to learn the museum’s exhibits and navigation when one floor is placed on a touch screen. Using a tour design task, we invited 12 first-time blind visitors to explore the museum building, chose exhibits that attracted them, and built a mental map with exhibit names and directions. The results show that the system is useful in obtaining information that links geometric shapes, contents, and locations to then build a rough mental map. The connected floors and spatial structures motivated users to explore. Moreover, having a rough mental map enhanced orientation and confidence when traveling through the museum.

2021

1. CHI

   Can Playing with Toy Blocks Reflect Behavior Problems in Children?

   Wang, Xiyue, Takashima, Kazuki, Adachi, Tomoaki, and Kitamura, Yoshifumi

   In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems 2021

   Abs DOI PDF Video

   Although children’s behavioral and mental problems are generally diagnosed in clinical settings, the prediction and awareness of children’s mental wellness in daily settings are getting increased attention. Toy blocks are both accessible in most children’s daily lives and provide physicality as a unique non-verbal channel to express their inner world. In this paper, we propose a toy block approach for predicting a range of behavior problems in young children (4-6 years old) measured by the Child Behavior Checklist (CBCL). We defined and classified a set of quantitative play actions from IMU-embedded toy blocks. Play data collected from 78 preschoolers revealed that specific play actions and patterns indicate total problems, internalizing problems, and aggressive behavior in children. The results align with our qualitative observations, and suggest the potential of predicting the clinical behavior problems of children based on short free-play sessions with sensor-embedded toy blocks.

2020

1. IMWUT

   AssessBlocks: Exploring Toy Block Play Features for Assessing Stress in Young Children after Natural Disasters

   Wang, Xiyue, Takashima, Kazuki, Adachi, Tomoaki, Finn, Patrick, Sharlin, Ehud, and Kitamura, Yoshifumi

   Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2020

   Abs DOI PDF Video

   Natural disasters cause long-lasting mental health problems such as PTSD in children. Following the 2011 Earthquake and Tsunami in Japan, we witnessed a shift of toy block play behavior in young children who suffered from stress after the disaster. The behavior reflected their emotional responses to the traumatic event. In this paper, we explore the feasibility of using data captured from block-play to assess children’s stress after a major natural disaster. We prototyped sets of sensor-embedded toy blocks, AssessBlocks, that automate quantitative play data acquisition. During a three-year period, the blocks were dispatched to fifty-two post-disaster children. Within a free play session, we captured block features, a child’s playing behavior, and stress evaluated by several methods. The result from our analysis reveal correlations between block play features and stress measurements and show initial promise of using the effectiveness of using AssessBlocks to assess children’s stress after a disaster. We provide detailed insights into the potential as well as the challenges of our approach and unique conditions. From these insights we summarize guidelines for future research in automated play assessment systems that support children’s mental health.

2. AHs

   SpotlessMind: A Design Probe for Eliciting Attitudes towards Sharing Neurofeedback

   Elagroudy, Passant, Wang, Xiyue, Stemasov, Evgeny, Hirzle, Teresa, Shishkovets, Svetlana, Mehrotra, Siddharth, and Schmidt, Albrecht

   In Proceedings of the Augmented Humans International Conference 2020

   Abs DOI PDF

   Mutual understanding via sharing and interpreting inner states is socially rewarding. Prior research shows that people find Brain-Computer Interfaces (BCIs) a suitable tool to implicitly communicate their cognitive states. In this paper, we conduct an online survey (N=43) to identify design parameters for systems that implicitly share cognitive states. We achieve this by designing a research probe called "SpotlessMind" to artistically share brain occupancy with another while considering the bystanders’ experience to elicit user responses. Our results show that 98 percent would like to see the installation. People would use it as a gesture of openness and as a communication mediator. Abstracting visual, auditory, and somatosensory depictions is a good trade-off between understandability and users’ privacy protection. Our work supports designing engaging prototypes that promote empathy, cognitive awareness and convergence between individuals.

International Conference Poster / Demo / Workshop (peer-reviewed)

2019

1. UIST Poster

   An Investigation of Electrode Design for Physical Touch Extensions on a Capacitive Touch Surface

   Wang, Xiyue, Ikematsu, Kaori, Fujita, Kazuyuki, Takashima, Kazuki, and Kitamura, Yoshifumi

   In The Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology 2019

   Abs DOI PDF

   A simple way to prototype touch interaction is to extend electrodes from a capacitive touch screen to off-screen areas. With that we aim to develop a toolkit that transforms a user-designed layout into a layout of screen-extension electrodes that realizes touch for rapid prototyping. Nevertheless, this kind of extension cannot detect touch if the physical properties of the electrodes become large. In this work, we decompose the physical design properties of the extension electrode into two factors, the target area, and line bridge, and investigate the limitation of each separately. While revealing some factors, such as area, is extremely limited in term of designing freely, we look into the causes by measuring the capacitive charge on-screen and on-extensions.

2. CHI Workshop

   Asian CHI Symposium: Emerging HCI Research Collection

   Fujita, Kazuyuki, Sari, Eunice, Kim, Juho, Tedjasaputra, Adi, Do, Ellen Yi-Luen, Liu, Zhengjie, Lee, Uichin, Monserrat, Toni-Jan Keith Palma, Matsufuji, Akihiro, Miyafuji, Shio, Takada, Ryosuke, Wacharamanotham, Chat, Ghazali, Masitah, Wang, Xiyue, Chintakovid, Thippaya, Seo, Kyoungwon, Kim, Jinwoo, and Kitamura, Yoshifumi

   In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems 2019

   Abs DOI PDF

   This symposium showcases the latest work from Asia on interactive systems and user interfaces that address under-explored problems and demonstrate unique approaches. In addition to circulating ideas and sharing a vision of future research in human-computer interaction, this symposium aims to foster social networks among academics (researchers and students) and practitioners and create a fresh research community from Asian region.

2018

1. CHI Late-Breaking

   Designing Action-Characterizing Toy Blocks for Behavior Assessments

   Wang, Xiyue, Ishikawa, Miteki, Takashima, Kazuki, Adachi, Tomoaki, Sharlin, Ehud, Finn, Patrick, and Kitamura, Yoshifumi

   In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems 2018

   Abs DOI PDF Video

   Playing with toy blocks reveals patterns in children’s play that are valuable for therapy and assessment. Following the 2011 Tohoku Earthquake and Tsunami in Japan, we witnessed young survivors expressing post-trauma stress in block play. Motivated by the limitations in assessing this behavior using traditional methods, our paper describes the design rationales of AssessBlocks, an action-characterizing system using smartwatch embedded toy blocks. Utilizing a smartwatch’s Inertial Measurement Unit (IMU) and capacitive screen, a monitor is able to receive, visualize and document sequential and quantitative play actions that were empirically selected from a preliminary block therapy study on children’s post-disaster stress. We also propose our vision of a multi-dimensional behavioral assessment system using actions obtained by AssessBlocks.

2. CHI Workshop

   Children’s Blocks: Machine Learning and the Analysis of Motion During Play

   Wang, Xiyue, Ishikawa, Miteki, Takashima, Kazuki, Adachi, Tomoaki, Sharlin, Ehud, Finn, Patrick, and Kitamura, Yoshifumi

   In CHI’18 Asian CHI Symposium 2018 Best Demo/Poster Award