Agile development: A case in point

At a recent Kennedy School executive education program, I was lucky to have a chance to meet and talk with Mark Schwartz, CIO at the Homeland Security Department's Bureau of Citizenship and Immigration Services.

Mark has a great background, coming to government for the first time after a fascinating career in private-sector IT.  He studied computer science at Yale, he went to film school, dropped out, and worked first at a film production company, then in software consulting, then lived abroad for five years. After all this, he went back to school to get an MBA at Wharton, and then became CIO of a company called Intrax Cultural Exchange, which sets up different kinds of exchange programs, including au pair jobs for foreigners working in US families. 

 “Then,” he says, “I got this idea that it would be interesting to work for government. My organization brought 60,000 people a year to come to the US. Citizenship and Immigration Services receives seven million applications a year to come. DHS seemed to have a lot of problems – I thought maybe I could help.”

And here's something that many will regard as amazing: He applied for the CIO job on USAJobs – and eventually got it (although six months passed before the agency acknowledged his application).
 
Since coming to CIS, Mark’s major challenge has been to take charge of the agency’s major IT transformation effort to digitize agency benefit application processes (including for visas). The project had been going on for several years, burned through hundreds of millions of dollars, and not yet produced any capabilities – in other words, a prototypical IT program in trouble.
 
With his private-sector experience, Mark has chosen to see if he can turn this program around through agile software development methods – under which software is developed in very small increments by small teams, rather than having complex and detailed requirements that take years to develop, and often never get developed, with the aim of getting incremental capabilities out much faster.
 
To Schwartz, the potential advantages of agile were fairly clear. When the software being developed is complicated, requirements changes (inevitable in large numbers in any major project) themselves become complicated because they involve rewriting more already-developed software. When problems are discovered during an old-fashioned “gate review” after a lot of work has been done, they similarly involve huge amounts of rework of already-developed code. The long lead time before any capability actually exists delays and complicates user feedback that is so essential for a new system.
 
I asked Schwartz what the problems he perceived in introducing agile into government and his approaches for dealing with them. One issue, he said, was that project overseers like to see well-developed requirements, which are not available early on for an agile project. His approach has been to present high-level system capabilities rather than more detailed requirements (which, it might be added, often significantly change anyway).

A similar problem involves the way the test and evaluation/independent verification and validation community do product testing at gateway reviews, in ways that are often time-consuming and contrary to the agile spirit of getting capabilities out fast. His approach has been to involve the testers in small tests while the software is being developed, rather than waiting for a formal gate review.
 
What’s his most difficult challenge? Agile involves development by small teams that are empowered to make decisions and even tradeoffs to move things forward fast. In government, with many stakeholders with different interests, and often less ability to have a common metric (such as return on investment) to judge tradeoffs, it is difficult to give as much authority to a small team as an agile effort in the private sector would typically have. He doesn’t feel he has solved this problem, but is toying with the idea of getting teams empowered to make tentative decisions, that are then – like elements of a new agile release in general – submitted to reaction from users, realizing that the smaller increments mean less rework costs if something doesn't go over well.

In May the new approach bore modest fruit as CIS debuted an online visa account form. Pretty modest when you think about it for a second – my reaction was “didn’t they have this already?” – but at least that the effort seems to be turning around. Better late than never, and hopefully there will be lessons learned here for expanding the role of agile development in the government.