We live in a visual world and build environments that rely heavily on visual perception. Want to find somewhere? You look on a map or read a road sign. Perhaps the GPS on your touch screen smartphone can help you. It probably couldn’t help you find your mislaid house keys or help you choose between a red and green apple but we don’t need it to, we have our eyes to do that.
Well, most of us do. For the 40m people who are legally blind worldwide, functioning in these visual environments is much more problematic. The signs can’t be read, and the apples, without tasting, are the same. At least the GPS will “talk” to me but the touch screen keyboard is awkward.
Efforts to increase accessibility to the environment for the visually impaired are not new – white stick, guide dog, Braille – but rapid technological advancements in the past three or four decades have facilitated not only new assistive devices but also techniques to restore a visual percept in the blind.
One option is surgical. Electrode arrays implanted in the retina of the eye electrically stimulate surviving retinal cells to elicit a basic visual sense. Research using implants such as the Argus II – the only approved retinal implant in the US and Europe – has demonstrated functionality in simple tasks, but implants suffer from limitations – technologically (it has very low resolution) and in availability.
Sensory substitution
There is another way, however, which requires no surgery and works with technology you may have in your home. Rather than replace the damaged part of the visual system this method tries to provide the “visual” information in another way – by using a different sensory system. This is sensory substitution.
Substitution devices can be divided into two groups based on whether the output is auditory or tactile. Modern tactile devices include the Brainport, which uses a camera, device and display unit to convert visual stimuli into tiny electrical signals on a person’s tongue. The main auditory devices, which convert the stimuli into sound, include The vOICe, PSVA and Eyemusic. The differences between them come down to the algorithms they use to convert the environment into sound.
Sight into sound
For auditory devices the technology can be basic: you need a camera to extract information from the environment, a PC or smartphone to run the conversion algorithm, and headphones to relay the converted signal back to the user – but the magic in how it works lies in how the brain processes sensory information and how this is used to inform the algorithm.
And it does work. The vOICe substitution device (the middle three letters stand for “oh I see”) was developed by Dutch engineer Peter Meijer and uses a three principle conversion algorithm to tell the user where something is in the visual scene and how bright it is using auditory features such as pitch, volume, and stereo scan. If an object is high up, on a shelf perhaps, then it has a high pitch. If it is to the left, you hear it in the left headphone; visually bright then it is aurally loud.
Even with minimal training users can display high acuity and perform far above chance on tasks such as object localisation and recognition due to the brain naturally understanding what is happening.
Better performing than retinal implant?
In a recent study published in Multisensory Research, we evaluated how much information is needed to successfully recognise simple objects using the vOICe. The level of recognition was more precise than simulations for retinal implants.
In the future the two could be used in partnership – The vOICe and a retinal implant – with each device no doubt better for specific tasks. But in terms of performance, we think that sensory substitution devices currently perform better.
Why it works
Train people on these devices and the story becomes even more remarkable. Due to the plasticity of the brain, areas of the cortex – usually thought of as “visual” part of the brain – show activation in users of sensory substitution devices. This is fascinating as it implies that rather than processing information specific to a sense/or sensory system, areas of the cortex are in fact processing information specific to the task.
This applies to all of us, not just those with visual impairment. But it is with visual impairment where it is most applied of course. An increasing understanding of how the brain processes perceptual information, facilitated by studies using sensory substitution devices, is allowing the development of more effective training and new algorithms to provide further information such as colour and depth.
While full visual restoration is some distance away, the advancements in invasive and non-invasive rehabilitation techniques signal a positive future for increasing inclusiveness and accessibility for the visually impaired population through simple technology. And if you want to try the vOICe for free, click here.