Exploiting the data revolution

Digital technologies have irrevocably changed business in recent years, and life in the era of big data inherently signifies that business decisions may never again lack for data.

But this big-data age, where crunching petabytes isn’t uncommon and compute can be purchased on-demand, is just "the warm-up act," according to Andrew McAfee, co-author of "The Second Machine Age: Work, Progress and Prosperity in the Age of Machines.”" 

"We’re nowhere near done with innovation and transformation, and I firmly believe during our lifetimes, we’ll witness the biggest transformation since the Industrial Revolution,” said McAfee, speaking at VMWare’s Government Innovation Summit on March 6. McAfee is a principal research scientist at the MIT Sloan School of Management.

"The Industrial Revolution helped us overcome the limitations of our muscles. We are now overcoming the limitations of our minds," McAfee said. "Over the next 50 years economies will go through the biggest change since the Industrial Revolution."

Having ridden in Google’s autonomous car and hung out with Baxter, a robot billed to have behavior-based common sense, McAfee said advances in digital technology, robotics and automation are beginning to change the world.

The changes have been rapid, limited only it seems by Moore's Law, money and imagination.

Jeopardy champion Ken Jennings was defeated by IBM's Watson less than two decades after former chess champion Garry Kasparov was defeated by IBM's Deep Blue computer, highlighting an important point: humans can now build machines that are better at tasks outside of arenas like number-crunching. Machines have always owned number-crunching. Now  they’re getting better than humans at other activities.

Machines can produce and sift through increasingly deep seas of data as fast as people can blink, leading to all sorts of "weird performance changes" that apply across the private and public sectors.

But the biggest changes occur not from machines or people, McAfee said, but from the combination of people and technology. McAfee used freestyle chess as an example. While the fastest chess-playing supercomputer can defeat the best human player, that supercomputer is dominated by a team of chess players and a slower computer.  

"[Machines and people] don’t have to be head to head. When we bring people and technology together correctly, the results are amazing," McAfee said.

The most obvious example in IT of teamwork between humans and machines is the use of computer algorithms to make predictions and data-driven decisions.

McAfee referenced Nate Silver, formerly of the New York Times, whose data-driven model predicted the presidential race correctly in all 50 states in 2012. Machines are great at producing data, compiling it and storing it, but there still is no substitute for human ingenuity in finding what best to do with data. And increasingly, organizations are releasing their big datasets to "the geeks of the world" for help. Allstate, for example, ran a 12-week contest for data nerds in hopes of improving its liability predictive prowess. In three months, the data teams had improved Allstate’s predictive baseline by 340 percent.

"In the era of big data that we're in, data is the lifeblood of science. It is how you get smarter about the world," McAfee said.

Because of this influx of technology and data, organizations and federal agencies face enormous opportunities. At the same time, they face daunting challenges handling the kind of disruption that’s only going to make the IT world seem more chaotic. Government sometimes struggles with innovation because of its traditionally slow-moving procurement process, while technology seems to evolve every few months.

"How we react to [the disruption] will be the defining issue for businesses, organizations, the economy and society for the rest of our lives," McAfee said. "Let's not screw this up, shall we?"