An IT moonshot: Can government IT development come to look like SpaceX?

In my last blog, I discussed the challenge posed by Elon Musk’s firm SpaceX to traditional ways of government contracting for satellite launches, represented by the Boeing-Lockheed United Space Alliance. Unlike traditional satellite (and IT project) development, SpaceX has taken up the challenge of introducing competition into launches and to develop satellites that are offered to the government at a fixed price. (In the case of SpaceX, that fixed price is about half of what the traditional contractor is charging).

If we changed the way we developed new IT systems to introduce competition in the ongoing development process (rather than our current mode of one upfront competition followed by the winning contractor being a sole-source supplier), and if development work occurred on a fixed-price basis, could we reproduce SpaceX’s achievements in government IT? Consciously imitating a metaphor taken from the space world, I will refer to such a possible transformation as a government IT moonshot.

Could this actually happen? And if it did, would it produce changes similar to the difference between the United Space Alliance and SpaceX?

Well, like most moonshots, realizing either and especially both outcomes would be a long shot. And clearly, an important part of SpaceX’s success has to do with the energy and brilliance of Elon Musk. Simply imitating institutional features such as competition and fixed-price development does not guarantee the same results as he has achieved.

However, my view is that, if there are to be dramatic improvements in IT development in government, the best chance is through some form of agile development approach. (Note that the analogy to SpaceX’s successes here is figurative not literal; you can’t use agile IT software development methods for a satellite. But SpaceX uses innovative approaches appropriate to the product being developed.)

I have blogged in the past about the potential of agile to increase IT project success rates by dividing up the work into a large number of brief sprints. This approach allows much earlier, more frequent course corrections compared to traditional waterfall projects, which have huge sunk costs that occur before the government really knows if it’s on a good path. That can be done, however, using only one contractor, and doesn’t require that any individual sprint be done on a fixed-price basis.

I want to put out there for discussion in the community what we might want to call agile 2.0 – which introduces multi-vendor competition for agile sprints and explores having individual sprints done on a fixed-price basis. Something close to this has been pioneered by Mark Schwartz, the CIO of U.S. Citizenship and Immigration Services, and by work 18F has done with the State of California on a new case management system for child welfare programs. In preparing this blog, to get and test ideas I spoke first with my Harvard colleagues Nick Sinai and David Eaves (Eaves is completing a Kennedy School case on the California system), and then with Schwartz and with Greg Gershman, who has worked both for traditional government contractors and for the government, and now runs Ad Hoc LLC, a small firm in the agile space.

I actually started to be interested in ongoing competition for IT systems development when I was in government in the 1990s, suggesting multiple awards for overall development contracts and competition for individual task orders under those contracts. My reasoning that this would provide both performance and price pressure not present in the traditional system.

However, at the time I was shot down on the grounds that this was not feasible, because code developed by one contractor could not easily be made to talk with that another contractor would develop. Nice idea, Steve, I was told at the time, but it won’t work.

Well, a key piece of news for the ability to improve government IT development is that for the last five years or so, this has no longer been the case. Jeff Bezos at Amazon was an early pioneer of using what became known as application programming interfaces, standardized protocols to allow communication among code developed in different parts of the company, conceptually similar to code developed by different contractors. Combined with continuous integration, where the work of multiple developers is integrated in real time, using multiple contractors on a project has now become very feasible -- though its actual use in government is still in its infancy. Furthermore, much of the existing use of multiple contractors in IT projects involves a sort of parallel play, where developers are working on very different elements of a system, and the code for these different elements is then integrated, rather than a truly competitive environment where different vendors are working on similar things.

In an agile world (which did not exist when I was thinking about this in the 1990s), there is an additional advantage to multiple contractors -- if the approach one is using has run into trouble, and agency can easily switch to another before too much damage is done. It adds to the risk reduction advantages of agile over waterfall development.

Performance and productivity pressures in a multiple award world come from the ability to jigger around the amount of work given each of the multiple award contractors based on their performance in earlier sprints. This is key to what has been done at USCIS, where the agency reassigns among contractors every six months the number of teams assigned to the contract based on earlier performance. (It should be noted that such reassignment is easier for large than small business, because the former have an easier time reassigning staff. The government may need to be more generous about guaranteeing a minimum amount of work in a small business environment.)

The last piece is moving toward a fixed-price environment. The dramatic shortening of the size of chunks of work -- sprints or a suite of sprints -- makes bidding a fixed price much more practical. Uncertainty in an agile environment about what the “requirement” is sets a limit to that: What one can hope for might be a fixed-price level of effort bid, where the government evaluates afterwards how much different contractors accomplished for the fixed price. As a project moves from initial stages and people get more experience with what is going on, actual firm fixed prices for a requirement, particularly for very short sprints, may become more feasible.

Could these approaches, taken together, revolutionize government IT development? To return to the space metaphor, I am hesitant to promise the moon. And many details must still be worked out in practice. But I would really like to see a dialogue about whether we ought to be moving much more in this direction. Comments to this blog, and other forums for engaging this question, are welcome.