Overview

This STS will focus on emerging technologies capable of sensing environmental features for applications in accessibility technologies for persons with visual impairment, including low vision and blindness. The development of environmental sensing technologies (ESTs) and the study of their potential to support the activities of daily living for visually impaired persons is progressing at a rapid pace, and engages many disparate research fields, including computer vision, machine learning, wearable sensors, ubiquitous computing, crowdsourcing, man-machine interfaces, and human factors. It is important to create a forum to forge interdisciplinary links among practitioners in these fields and to promote research into ESTs that will have the greatest likelihood of having a real impact on the lives of people with visual impairment.

Background and Significance

ESTs such as computer vision and mobile sensors hold the promise to greatly extend the capabilities afforded by more mature technologies such as GPS, widely used in commercial systems for visually impaired persons to provide outdoor navigation and wayfinding assistance. For instance, computer vision has the potential to detect and recognize informational signs normally accessible only to those who are sighted. Indoor localization systems are also being developed that can provide persons with visual impairments with wayfinding information where GPS is unavailable. Modern smartphones come equipped with screen readers, high resolution cameras with focusing capabilities, and multiple on-board sensors, providing an ideal platform to host many of these ESTs. Apps such as Seeing AI and Lookout use deep learning to recognize the scene captured by the camera, while services such as Be My Eyes and Aira rely on remote sighted assistants to provide real-time scene interpretation and description. Meanwhile, auxiliary hardware such as wearable cameras and other sensors add additional capabilities and improved form factors to the smartphone platform, which can be augmented by bone conduction headphones and portable VR/AR headsets to provide fully immersive experiences.

While ESTs are developing at a rapid pace, harnessing their capabilities in accessibility technology for the community of people with visual impairment poses a variety of technological, design and human factors challenges. For example, while traditional computer vision applications are normally designed for imagery taken by a normally sighted user or acquired by a rigidly mounted camera, images taken by visually impaired individuals are normally highly unconstrained, especially for blind users who may not know where to aim the camera. Such applications often demand “vision without sight” – the ability of an access tool to provide a user interface (UI) to actively direct the search of a user with no useful vision towards likely targets of interest. Indeed, UI design – which demands a realistic understanding of the possibilities offered by the available interfaces (visual, audio, tactile) – is of paramount importance in the design of access tools, yet is often neglected or minimized when focus is placed solely on the design and optimization of the ESTs themselves. In addition to UI design, research on applications for the visually impaired population requires informed knowledge of many other human factors issues, including the actual problems impairing the activities of daily living of these users and an awareness of existing assistive aids. Finally, researchers must adopt a system-level approach to the design of algorithms and hardware that takes into consideration practical factors such as form factor, size, speed and cost.

Call for Contributions

The development of ESTs that meets these challenges demands participation by experts from multiple and disparate disciplines, including rehabilitation, clinical science, orientation and mobility, user interfaces, accessibility, human factors and computer vision/sensors – the very range of expertise and approaches that is invited to contribute to this STS.

Contributions to the STS have to be submitted using the standard submission procedures of ICCHP.

Chairs

• 
  James Coughlan, Smith-Kettlewell Eye Research Institute
• 
  Roberto Manduchi, UC Santa Cruz