AI is coming

In the 1930s a renowned Austrian writer named Stefan Zweig fled the Nazis and headed to Brazil; a few years later he produced a book with the title, Brazil: Country of the Future. The predicted ascendency of Brazil never occurred, and after a while Brazilians started saying (they still do today), "Brazil is the country of the future and always will be." (It didn't help that shortly after the original book was published, Zweig committed suicide.)

I bring up this old story because the observation about Brazil reminds many people of the development over the decades of artificial intelligence. Like Brazil, AI has been hyped for a long time.  Yet somehow the reality has never lived up to the promise.

As the Economist recently wrote, "The term ‘artificial intelligence' has been associated with hubris and disappointment since its earliest days. … AI research became notorious for promising much more than it could deliver."

Now, however, it appears that huge increases in the amount of data available through the Internet (which is essentially a raw material for developing AI product) and advances in how to "train" the computer to process that data are helping AI to finally come into its own. U.S. investment is skyrocketing, and the Wall Street Journal reported recently that China is putting major investments into AI as well.

Recently, however, Kate Crawford and Meredith Whittaker, the two organizers of a recent White House conference on public policy issues associated with AI , published a cautionary post  on Quartz with the somewhat alarming (alarmist?) title, Have we given artificial intelligence too much power too soon?

Crawford and Whittaker write about the use of AI for decision-making or decision support -- uses which, along with facial recognition, are arguably the most important applications for AI in government.

AI applications produce algorithms (a word that has now migrated from high school math classes into widespread usage in the media), which are decision rules that have grown out of patterns seen by analyzing large amounts of data. The concern is that using these algorithms may yield biased or incorrect decisions.  "AI systems," the authors write,  "are already making problematic judgments producing significant social, cultural and economic impacts in people's everyday lives."

Much of what I read in Crawford and Whittaker's article was new to me, so I took the time to read the links to which they referred. For me, it was an eye-opener.

The authors begin with a recent story from the Wall Street Journal about Facebook having deleted the famous Vietnam-era photo of a small girl, naked, fleeing an American air attack using napalm, because the photo featured child nudity. The photo got caught in Facebook's body recognition algorithm that forbids child nudity, and was then automatically deleted. (After an uproar in the Norwegian media, the country's prime minister wrote to Mark Zuckerberg, and the deletion was overridden.) The point was that the mechanistic, automatic application of the algorithm had produced an absurdity.

The article's other examples involved the use of algorithms in the criminal justice system, to provide information for judges to use in sentencing or parole decisions, based on a prediction, growing out of a large number of cases, of how likely the person is to commit more crimes. These systems, which are actually in use in parts of the U.S. in different ways, have been criticized both for doing poorly in actually predicting future crimes and for tagging blacks as having higher risks for re-offense. The point here is that government should go slow in fielding such applications, especially where we are not sure about disparate impact.

It strikes me that the areas where caution is most in order about fielding AI for decision support is where the decisions involve the government deciding on a benefit or cost that can't go to everyone (not everyone can be given a lighter sentence, for example), and therefore where issues of fairness in making such decisions are most important. But there is a lot of variation in the extent to which different AI applications pose potential dangers of unfair treatment of individuals.

Last week, a Department of the Army scientist received a Sammie award from the Partnership for Public Service for an artificial intelligence system that collects and interprets soldiers' vital signs during transport to treatment centers so that medical personnel can be alerted if the patient will require an immediate blood transfusion on arrival.  This seems to pose minimal threats of unfairness.

Similarly, with using AI to find patterns in medical data that could help doctors diagnose diseases and suggest treatments to improve patient care and health outcomes, there is not really a danger that AI will direct benefits unfairly across different individuals (unless diseases affecting a specific segment of the population are chosen for AI analysis).  In cases such as these, it would seem appropriate to field AI solutions as soon as they have been developed. 

However, for other examples in the Crawford and Whittaker post, more caution is in order before the results of AI efforts are used.  I am reminded of the viewpoint that government should not be an early adopter of new technology. As a general matter, this is not a viewpoint for which I have much sympathy, since it too often seems to tie the government's hands. However, for AI-guided decisions that have a risk of treating people unfairly, I think caution about government not being an early adopter is warranted. 

Let's try to learn more, including from experiences regarding non-governmental use of AI for decision-making. But at this point, this is too new, and there are too many unanswered questions, to rush.

I will confess that the more I read about these issues, the more confused I became about what the right answers might be. An obvious solution to problems such as the napalm photo might appear to be to bring in human curators to do quality control of algorithmic results. But this may typically be too expensive for government to do, and furthermore it brings its own problems. Facebook got into trouble last spring after it was revealed that employees curated algorithmic results for their "trending" news feed, fueling accusations of liberal bias. (As if to illustrate the point that there is no perfect solution, Facebook later fired its trending news curation team and moved to using algorithms only, but within days reports came out that a story about "a man masturbating with a McChicken sandwich" had appeared on the trending feed.

A bigger question about algorithms is "compared to what." Some algorithms may display racial or gender bias, but decision-making by individual humans is probably much more prone to this. And humans often have other biases that algorithms don't have -- in one study,  judges became far more lenient in parole decisions after they'd eaten lunch. Decisions by humans are also frequently marred by inconsistency, which algorithms don't show.

In other cases, the amount of data that would need to be analyzed to make a non-algorithmic system work -- such as in screening out certain kinds of photos on Facebook -- might be so large that the practical alternative might be between algorithmic screening and no screening at all.  Algorithmic screening of surveillance video photos to produce facial matches will surely produce some errors, with pain for those incorrectly tagged. But if the number of errors is modest, should the police forsake using this tool at all?

I came out of the time I spent reading the Crawford and Whittaker post and the accompanying links with a lot of questions, but also with the view that issues for government and society raised by the use of AI for decision support, not the least in a government context itself, are going to become more and more important. From now on, this issue is most definitely going to be on my screen, and I think it should be on yours as well.