As part of our software estimation series, we explored the importance of managing expectations as part of any software project. This article is going to focus on the cone of uncertainty, a powerful tool for mitigating the risks involved in the scoping and estimation process. At its core, it is a depiction of the uncertainty about the time required to complete a project, based upon the amount of knowledge (or lack thereof) at the beginning of a project. It ultimately accounts for the risk of the project blowing out due to unknowns, and the fact that the further into the future you try to estimate, the more likely you are to be wrong. It is an interesting paradox where we want to predict the future accurately, however we cannot truely be accurate because it is only an estimation.
In case you haven’t already, I’d recommend that you familiarise yourself with a previous experiment we conducted here at Codebots to discover how we could best manage risks in software. The motivation for that exercise was the frequent overoptimism from our software teams when it came to conducting estimations.
Having equipped ourselves with the best procedure to propose an experiment, we identified a problem in our fixed scope estimations. Every quote contained a minimum 80% variation to the work done, no matter the length of the project. Since we didn’t apply a cone of uncertainty, our ability to provide our customers with realistic estimations and manage their expectations was significantly hampered.
Step 1: Understand the problem
As mentioned above, we encountered a problem in our estimations, occurring when we provide a fixed scope price at the quoting stage. (Fixed scope meaning delivering the exact requirements given, but for a flexible length of time). No matter the length of the project, we always went over due to having a large amount of variation.
The root cause - we didn’t have a cone of uncertainty. We didn’t account for the fact that the further into the future you try to predict, the larger the increase in time variance. This is due to the presence of a number of specific risks, with the most severe ones being inaccurate estimations, scope variations and end-user engagement. Because it was all estimated at the beginning, there was no opportunity to alter the estimation to account for these discovered risks. Instead, the impact of these risks are compounded over the lifetime of the project, meaning the longer the project, the more you try to predict, therefore the more likely you are to be wrong.
Step 2: Develop a hypothesis
Having digested the problem, we next worked on developing a hypothesis to test how a cone of uncertainty could be accurately applied to our fixed scope estimations.
From this, it was hypothesised that a formula could be used to calculate size of the Cone of Uncertainty, given the length of a project.
Step 3: Plan the experiment
Equipped with our hypothesis, we proceeded to plan an experiment to test our proposition of the Cone of Uncertainty. To do this, we generated a quadratic function to simulate our cone, with a parameter representing the number of predicted weeks of development. We could then apply this multiplier to our estimations to account for the time variance, based upon how far into the future we were trying to predict.
If we apply this to estimation data from previous projects, we could then determine if a fixed scope price with the variation would have been more accurate.
Step 4: Collect the data
We collected data from a variety of projects, including information on time allocated and taken, stories completed, time until completion, including additional notes on the project and risks encountered.
Taking the estimations, we calculated a hypothetical fixed scope value for each project, using the cone of uncertainty. Using this information, we were able to examine whether the time which we actually took to develop, finishing both the initial work and the additional variations, was comparable to the new estimation values generated using the cone of uncertainty. Turns out, the improved estimation and the time actually taken were very similar!
Step 5: Make a decision
In the end, the data demonstrated that if we applied the Cone of Uncertainty to a fixed scope, using a formula that takes into account the length of a project, we would end up with an estimation which would take into account the variance that occurs over the development of the project.
Ultimately, we recommend that rather than trying to estimate the entire project, a development team only focuses on and estimates a small amount of work. This will help reduce the impact of the unknowns, and therefore the size of the cone of uncertainty. This also means that as the team works on the project they learn more about it, and when it comes to estimating the next chunk of work, they can reduce the uncertainty and make more accurate estimations.
Written by Eban Escott
Founder of Codebots
Dr Eban Escott received his Doctorate from UQ (2013) in Model-Driven Engineering and his Masters from QUT (2004) in Artificial Intelligence. He is an advocate of using models as first class artefacts in software engineering and creating not just technologies, but methodologies that enhance the quality of life for software engineers.
