A man blind since birth is taking up a surprising new hobby: photography. His newfound passion is thanks to a system that turns images into sequences of sound. The technology not only gives “sight” to the blind, but also challenges the way neurologists think the brain is organized.
In 1992, Dutch engineer Peter Meijer created vOICe, an algorithm that converts simplegrayscale images into musical soundscapes. (The capitalized middle letters sound out “Oh, I see!”). The system scans images from left to right, converting shapes in the image into sound as it sweeps, with higher positions in the image corresponding to higher sound frequencies. For instance, a diagonal line stretching upward from left to right becomes a series of ascending musical notes. While more complicated images, such as a person sitting on a lawn chair, at first seem like garbled noise, with enough training users can learn to “hear” everyday scenes.
In 2007, neuroscientist Amir Amedi and his colleagues at the Hebrew University of Jerusalem began training subjects who were born blind to use vOICe. Despite having no visual reference points, after just 70 hours of training, the individuals went from “hearing” simple dots and lines to “seeing” whole images such as faces and street corners composed of 4500 pixels.
Amedi’s team recently released a successor to vOICe, called EyeMusic, as a free iPhone app. The new algorithm produces more pleasant tones and can even provide color information.