Software Engineers Figured Out How to Turn Charts Into Music for the Blind
zlikovec/Shutterstock.com
There is a vexing gap between data’s possibilities and its accessibility.
Charts are something of a religion here at Quartz. Done right, they’re succinct, attractive illustrations that convey volumes at a glance. Yet they are also inherently excluding. A visual representation of data, no matter how elegant, is a useless tool for someone who is not able to see.
Ed Summers is intimately familiar with this problem. Summers is a software engineer at the North Carolina-based data analysis company SAS. Diagnosed with the degenerative eye condition retinitis pigmentosa at the age of 10 and now legally blind, he has spent the last several years trying to close the vexing gap between data’s possibilities and its accessibility.
“The real unfortunate aspect of technological innovation in education and the workplace is the accessibility of charts and graphs really hasn’t kept up,” Summers said.
In February, under Summers’s guidance, SAS released a tool called the Graphics Accelerator. The free browser plug-in scans web pages for graphs or charts and translates them into sound—specifically, piano notes.
An Atlas chart showing the number of mortgage loans in the US, for example, sounds like a scale that crescendos in July 2008 and then trips back down.
The tool is an example of sonification, the art and science of turning data into sound. The ear is particularly good at taking in patterns, subtle changes, and multiple streams of information at once, says Mark Ballora, an associate professor of music technology at Pennsylvania State University.
He gives the example of Wanda Diaz Merced, a blind astronomer who uses a NASA-developed software called xSonify to aurally observe X-rays. She detected X-rays by listening that sighted colleagues missed on visual graphs.
Sonification has broad application for both the sighted and blind, Ballora says. Given that we have more data available to us than ever before, why not deploy all senses available to take it all in?
“I keep asking myself about the value of raising kids to think of science and math as things you study not just with your eyes, but with your ears,” he says. “This [SAS tool] has a lot of implications for what science education could look like in 20, 30 years.”
Speaking of students, the melodic notes used by the Graphics Accelerator are the result of direct feedback from students at the Governor Morehead School for blind students in Raleigh, NC, a few miles from the SAS campus in nearby Cary. Students testing the program balked at the tinny, robotic noise of earlier prototypes.
“Some of the kids that have perfect pitch were really irritated by our work in this space because it wasn’t in tune,” Summers says. “They were like, ‘You’ve got to fix that. I can’t listen to that all day.’”