Individuals with visual impairment can 'see' through device that turns digital images into physical sensations

HaptImage co-founders Ting Zhang (left) and Shruthi Suresh (right) use their system to “touch” and interact with digital blood cells on screen. The HaptImage device translates digital images into real-time haptic feedback using a novel algorithm developed by Zhang. (Purdue Research Foundation image/Oren Darling) Download image

Device helps people study STEM-based images that mimic the would-be surface of digital objects and improve learning

WEST LAFAYETTE, Ind. — David Schwarte, an assistive technology specialist on the Innovative Learning Team at Purdue University, knows what it is like to learn in a classroom when challenged with a visual impairment.

Schwarte, who has a visual impairment, says it is nearly impossible to understand what a professor is talking about when teaching from a PowerPoint on a large screen. To understand what is happening, the student has to depend on either an audio description or a 3D-printed mockup of the image on screen.

Now a new device designed by HaptImage LLC, a Purdue-affiliated startup, is assisting students of all ages with visual impairments to “see” and learn about what is on the screen.

HaptImage LLC has developed a novel haptic device that converts digital graphics into real-time physical sensations. Here, inventor Ting Zhang works with a HaptImage prototype to explore the blood smear pictured on screen. (Purdue Research Foundation image/Oren Darling) Download image

“There are many areas in science, engineering and technology where the end-user needs to visualize a bunch of different graphics,” Schwarte said when he used the HaptImage device. “This system has the advantage of helping people visualize the graphics more instantly. HaptImage’s technology is important because students can come into a very technical field and still be able to succeed.”

These students are not alone. According to the World Health Organization, there are an estimated 1.3 billion people living with some form of visual impairment across the globe. HaptImage has a better alternative: an image-accessing system with novel haptic capabilities. A video about the device is available here.

Ting Zhang, a doctoral student in the School of Industrial Engineering, created the system with her co-advisors, Juan Wachs, a professor in the School of Industrial Engineering and Bradley Duerstock, a professor in the School of Industrial Engineering and the Weldon School of Biomedical Engineering.

She was inspired by a colleague who voiced difficulty in seeing blood smear slides during lab. He often needed a partner and didn’t feel independent.

Zhang co-founded HaptImage with Shruthi Suresh, a doctoral student in Weldon School of Biomedical Engineering, in order to commercialize the technology which helps individuals with blindness or visual impairments study images related to science, technology, engineering and mathematic topics in real time.

Already, the startup has received $47,500 in entrepreneurial investments through its involvement with the Purdue Foundry and participated in the Fall 2017 National Science Foundation I-Corps cohort. Read more about I-Corps at Purdue here.

“There is no instantaneous method of helping a student with blindness understand an image,” Suresh said. “In STEM, so much information can come from an image. When you don’t have access to that digital image, or a viable alternative, you feel discouraged to pursue a career in that STEM-related field.”

The HaptImage device uses a novel algorithm to convert digital images into physical sensations that mimic the would-be surface of the depicted object. Users hold a pen-like joystick, and as they move it across the allotted space, the joystick creates vibrations and resistance corresponding to the digital object’s shape and texture.

"We, as sighted people, take images for granted and don't realize how valuable accessible visual alternatives are for someone with impaired sight. I think having that access is valuable," Zhang said.

To help students with visual impairments, instructors commonly use tactile graphics or 3D printouts as stand-ins for digital images, but this process can be time-consuming, expensive and limiting for the students. However, the HaptImage device streamlines this process, so that students can interact in real time and directly with an image.

"The device combines tactile feedback through resistance with friction and sounds to give a person the idea of what the image 'looks' like," Suresh said. "They can feel the height and shape of the image, and they feel where the shape changes. That comes across a lot easier for someone who is blind or visually impaired."

The technology gives students with visual impairments the ability to “touch” a digital object due to its sophisticated haptic responses and makes these images instantly accessible. The founders hope this software can encourage more people living with visual impairments to pursue STEM-related careers.

“The algorithm translates an image into different intensities of vibration, different pitches or amplitudes of sound and haptic feedback, which allows you to feel that shape,” Zhang said. “It’s adding a whole other dimension of understanding for these students.”

Currently, the startup led by Duerstock, is preparing to launch its device through a tiered subscription in the fall. HaptImage is currently developing a portable prototype that will allow individual students to interact with digital images on their smart devices.

HaptImage’s technology is licensed through Purdue Office of Technology Commercialization.

The technology aligns with Purdue's Giant Leaps celebration of the university’s global advancements made in health, space, artificial intelligence and sustainability as part of Purdue’s 150th anniversary. Those are the four themes of the yearlong celebration’s Ideas Festival, designed to showcase Purdue as an intellectual center solving real-world issues.

About Purdue Office of Technology Commercialization

The Purdue Office of Technology Commercialization operates one of the most comprehensive technology transfer programs among leading research universities in the U.S. Services provided by this office support the economic development initiatives of Purdue University and benefit the university's academic activities. The office is managed by the Purdue Research Foundation, which received the 2016 Innovation and Economic Prosperity Universities Award for Innovation from the Association of Public and Land-grant Universities. For more information about funding and investment opportunities in startups based on a Purdue innovation, contact the Purdue Foundry at foundry@prf.org. For more information on licensing a Purdue innovation, contact the Office of Technology Commercialization at innovation@prf.org.

Writer: Kelsey Henry, 765-588-3342, kehenry@prf.org

Purdue Research Foundation Contact: Tom Coyne, 765-558-1044, tjcoyne@prf.org

Sources: Shruthi Suresh, suresh9@purdue.edu

Ting Zhang, zhan1013@purdue.edu

David Schwarte, david-schwarte@purdue.edu