How many of your IT projects fail? Too many
People who follow IT project management have probably seen the Standish Group 2006 survey report (CHAOS) that suggested 67% project failure rate in IT projects. The Impact of Size and Volatility on IT Project Performance (overview only, full text is members only) by Chris Sauer, Andrew Gemino and Blaize Horner Reich in the November 2007 Communications of the ACM suggests just the opposite: 67% project success rate.
So, which is it? Does it matter? A one in three failure rate is still too high.
Here are some of the highlights from this Communications of the ACM article:
- Any more than one change in project management leadership or sponsorship leads to a greater than 50% risk of under-performance.
- Risk of under-performance is closely correlated to effort (person-months).
- Risk of under-performance isn't correlated to project budget, and there is minimal correlation to duration. Team size was only important for very large teams (in this study 20 FTE's is large).
- Regressing the volatility data with the size data, they shows a correlation between changes in project manager and schedule (longer), budget (higher), and scope (less).
- Of the studied projects, they found that 25% of project fail along many different axes.
- Survey respondents were fairly experienced: 9 years as a project manager.
In short, the survey suggests the most important thing to do is keep the (experienced) project manager on the project to the end. The other suggestions have to do with keeping the size in the acceptable-risk range.
Note: I haven't taken the time to dive into the details of the CHAOS Report to speak to the differences. I would love to see someone compare this survey of 412 IT projects to the studies of thousands of projects that Standish has been doing in order to see where the differences lie. InfoQ has an interview with Jim Johnson of Standish Group, talking about their latest report and the work in general. InfoQ also has a discussion about Standish CHAOS Report Methods Question which talks about some of the concerns over the Standish report voiced by Robert L. Glass in a recent CACM article.
5 Comment(s)
said:
There are two takes on the failure rate. Mine was under assumption that these projects are (presumed) money-makers or operationally required. Essentially failed projects = lost revenue. I assume that you are thinking along the lines of innovation, where we should be trying as many different ideas as possible, rather than trying to analyze the idea to death before giving it a whirl. Here a failed project would not be lost revenue, but gained experience. Sure there is the expense associated with trying, but that has to be there.
Where else would you want to be increasing the number of "at bats" and be willing to lower your batting average?
I was thinking in terms of the overall portfolio of projects and risk assessments at both the individual project level and the overall portfolio level. And risk, of course, is a dynamic thing.
If you have no risk in your project portfolio, you should likely have no project failures. You're also likely to be outsourced as an IT organization as you aren't adding value to the organization (no risk = no reward). If you are taking risks, then you will have project failures. One question for an IT organization is whether your project management practices contribute to raising or lowering the overall risk of a project.
said:
One more test comment. Should not get automatically published, right?
said:
Thanks again, Jim.
I hadn't thought along these dimensions. Risk is clearly important. No risk, and "anyone can do it." Too much risk, and you don't have any customers. Where does the risk appear in accepting projects? Is it risk that the project will work at all, or risk in that some specific features / capabilities will make it?
At some point the organization needs to be making money... What does a high IT failure rate say about the ability to make money today -- an in the future? I'd find it difficult to believe anyone would hire a company that has 33% or 67% failure rates.
I've generally been quite skeptical of the Standish data.
As for failure rates, I actually think that a 1/3 failure might be about right and that most organizations ought to be shooting to increase their failure rate (essentially by increasing their overall number of "at bats").