Story Points Estimation

A Story Point is an estimation metric used by many IT-development teams. Story Points are often recommended in Agile Project Management frameworks and methods such as Scrum, Agile-Kanban or Extreme Programming (XP). The Story Point estimation tool separates itself from conventional tools by being an arbitrary unit of measure that estimates required effort instead of the more tangible and well-known Hours. Proponents of the tool argues that Story Points encompass a range of dimensions including, complexity, uncertainty and risks involved with the completion of the estimated task. It remains, however, a controversial tool, often because the concept is not well understood; the benefits are abstract and effective use requires prerequisite work as well as medium or long project run-times.

One clear benefit of using Story Points is that it enables the project team to measure the velocity of the team: An hour is always an hour, and in a perfect world the burndown-chart burns with exactly the amount of work-hours you assign to your tasks. This is, of course, almost never the case. Project teams works more efficiently in some periods than other. Overhead, such as time- or resource waste also varies through the different phases of the project. By assigning Story Points based on effort to tasks you create an effective tool to measure the current speed of the team. An added benefit is that the perceivable metric often creates a common goal to increase the pace within the project team.

This Wiki-article is aimed specifically at groups or organizations operating with Agile frameworks or methods. The Story Point estimation tool is not easily applicable for other (especially linear) project execution methods i.e. Waterfall Model. A prerequisite for using the Story Point estimation tool is an iterative development process like Sprints, which is a prescribed Scrum practice: A Sprint is a relatively short period of time (1-4 weeks) in which certain project deliverables (i.e. tasks or stories) are completed in a focused effort by the project team. Each sprint is usually followed by a grace-period of 0.5 to 1 week during which a sprint review and sprint retrospective is usually held. New tasks/stories are then added into the scope of the next sprint in preparation. In these type of projects, typically IT/software-development projects, deliverables are usually formulated as User Stories. For future reference, below is an example of a User Story:

"The Admin logs into the system and is presented with a list of users and a search bar.

The Admin searches for a user by typing the user's sir-name.

The Admin is then presented with a list of users with the typed sir-name."

It is the relative required effort of implementing these types of User Stories that encompass a Story Point.

History

Although there is no definite data available of when the Story Point practice was birthed (citation needed), there exist strong indications that the core thoughts behind the tool originate from the Delphi Method developed by the RAND Corp. at the beginning of the Cold War in the 1940's. Later, the Wide-band Delphi Method from the 1970's facilitated increased communication over the Delphi Method. Some of the practices stemming from the Wide-band and Delphi methods have later implemented themselves as common practices in the "Agile Methodology"; introduced by software engineers in the late 80's and early 90's. In 2001 we see thoughts from the Wide-band Delphi Method cemented in the prominent "Agile Manifesto", developed by 17 distinguished software developers to serve as a guideline for Agile Project Teams i.e.:

"Business people and developers must work together daily throughout the project."

Scrum, being one of the most widely used Agile frameworks, prescribes both time-boxed iterations as well as velocity estimations. And although the Story Points estimation tool is not prescribed in Scrum, the fact that Story Points satisfies both of the prescribed guidelines while also keeping in line with the overall Agile perspective, has made Story Point estimation a widely used, although somewhat controversial, tool in recent years.

Application and Use

Figure 1: Example Step-by-Step guide to estimating process using Story Points, Source: Jacobsen, Ian Thobias (user: s113735)

A Story Point is an arbitrary unit of measure, and so, the precise method of application is not well defined, and may vary greatly between different projects and teams. It is, however, very important that a shared understanding and application of method exists within any team estimating using Story Points.

The template method below provides an example application of the Story Point estimation tool. It is in no way prescribed, and any truly Agile team should aim to develop their own approach:

• Step 1: Identify base stories.

Most Agile software development teams will have either general- or technical User Stories made available to them at the beginning of the project. From the backlog of stories, identify one or more base-stories to serve as reference-stories to help estimate the relative size of the backlog.

• Step 2: Make individual "bids" on stories.

Have each team member (including testers, front- and back-end developers, project owners etc.) write down a reasonable "bid" for the story and keep it secret for now. This could for instance be done using the Fibonacci sequence: 1, 2, 3, 5, 8, 13, 21... The bid should represent each member's personal estimation of the effort required to complete the given story. Make sure to underline the importance of not only taking into account the number of hours they believe the task will take to complete, but to consider the task's complexity, risks, size and so on.

• Step 3: Have each team member explain the reason for their bid.

Taking turns, have each team member reveal and then explain his reasons for the bid. Afterwards, discuss in the group the required effort to complete the task and reach a consensus before moving on. Especially pay attention to the outliers and never simply average the bids. An example of why this is important has been outlined below:

"In an Agile team estimating using Story Points, a back-end developer with a lot specialized knowledge has made a bid on a User Story he is very likely to be responsible for developing some time in the future. Using his considerable knowledge about this kind of story he has decided to make a bid of 13 Story Points. Meanwhile, the project owner, who was in charge of writing this particular User Story has failed to see the technical difficulties in implementing this (he thinks) rather simple story. For that reason, he has made a bid of just 3 Story Points. After some discussion between the developer and the project owner it becomes apparent that what the project owner actually wanted delivered was far less extensive than what the back-end developer had read/understood from the User Story. Following, the User Story was quickly rewritten, another round of estimations took place and the team reached a consensus of 5 Story Points for the Story."

• Step 4: Reach consensus for assigned Story Points.

For reasons outlined in Step 3 above: Through group discussion the team should reach a consensus value for each story. This particular endeavor gets easier as the project progresses; this is because the amount of reference stories will grow, making relative estimation faster and less demanding. With enough reference (base) stories the project team will simply have to reach a consensus if the new story requires more, less than or the same amount of effort than a couple of the base stories.

• Step 5: Break down stories.

Once in a while, the team will reach the conclusion that a story requires an extraordinarily large amount of effort to implement i.e. 13, 21, 34... Story Points. This is a good indication that the story can be broken down into two or more smaller stories. As an example a story of 21 Story Points could probably be broken down into 2 stories of 13 and 8 story points, or maybe even 3 stories of 8, 8 and 5 Story Points. The reason an Agile team should always stride towards breaking down large tasks is because smaller tasks become more manageable, while it also facilitates easier Sprint-Planning, reduces the risk of over commitment and diminishes some estimation uncertainty.

• Step 6: Use base stories to estimate the size of the backlog.

For each story in the backlog, make a rough estimation using the base stories as reference. This will give the team a relative size of the backlog and a gauge of the required effort to implementing the entire project. Before committing a story to a sprint go through step 2-5 to ensure the team is continually working on the same page. For reasons explained later in the article, estimation review and re-estimation are key tools for project success.

Cost/Benefit

Figure 2: Cone of Uncertainty; an Example of quick uncertainty reduction when estimating using Story Points. Legend: Middle Line (red): Estimation Error of Team, Outer Lines: Upper and Lower Bounds of the "Cone of Uncertainty", Source:

Figure 3: Traditional Estimation Funnel; a general Example of the Traditional Estimation Funnel when estimating Hours. Legend: Middle Lines (red): Estimation Error (interval) of Team, Blue Zone: Upper and Lower Bounds of the "Cone of Uncertainty", Source: .

Historically speaking, IT projects have failed at a comparative rate much higher than ordinary projects and estimation accuracy seem to be a core factor, where studies show more than 68 % of projects fail to meet original estimates. It is therefore paramount, that any estimation tool's merits and drawbacks are thoroughly understood before use.

Consider all of the costs and benefits below before deciding if Story Points would be suitable for your team:

Costs

• Story Points take a lot of time in the beginning.

Although estimating is fast once the base stories have all been estimated and completed you might only really benefit from easy and fast estimation after a couple of sprints/development iterations. The Story Point tool is considered a high-level estimation measure, while it excels at long lasting projects it lacks merit for short-lived projects.

• Story Points are tough to understand.

Story Points are not hours, neither are they an estimate of complexity or how many people should be involved in the completion of a task. Rather, Story Points symbolize the required effort for the completion of a task. People who have successfully estimated using hours might have difficulties adapting to using the somewhat more vague Story Points as a estimation and progress-tracking metric.

• Management often care about hours.

Many aspects of a business depends heavily on estimated hours; a work day is measured in hours, consultants gets paid by the hour and a lot of KPI's are often related to hours as well. Realise that management often cares more about when the product will be delivered rather than how many hours still needs to be completed - if leveraged correctly this point of view may be turned into a benefit.

Benefits

• People are inherently bad at estimating time, but good at relative approximation.

Estimating using Story Points versus estimating using hours might improve estimation accuracy noticeably as depicted in Figure 2 and Figure 3. Although estimation inaccuracy might be equal for both methods at first, the iterative and reflective nature of Story Point estimation in an agile team, eliminates more estimation inaccuracy faster. This is because the Story Point tool discourages optimistic estimations, while the velocity-metric inherent to the Story Point tool makes progress-tracking easier.

• Story Points makes tracking velocity easy.

Because tracking velocity is prescribed in some Agile frameworks the tool is an obvious choice when using one of such frameworks.

• Story Points often strengthen team cohesiveness and promotes knowledge sharing.

Inherent to the estimation procedure people get to discuss different problems and solutions related to the tasks of the project. The method facilitates strong task-relationship as well as perceived unity (because project owners, developers, managers and testers all work together) which are two of the four metrics related to Group Cohesiveness.

• Story Points are a long-term measure.

Using Story Points is especially effective in large project planned to run for a long period of time. Consider using the tool if you have a large backlog and plan to run for at least a couple of sprints. The tool only becomes effective after a few iterations. The more base-stories the team completes, the more accurate future predictions become. This could arguably be both a benefit and a cost.

Alternatives

There are a number of ways to estimating and measuring size of software projects alternative to using Story Points. Among these alternative ways are measuring Lines of Code (LOC), Thousand Lines of Code (KLOC) or Ideal Days. Another obvious alternative is simply to estimate using Hours.

Lines of Code

Using LOC or KLOC simply estimates the size of the project in regards to the lines that needs to be coded. These methods don't take into account the complexity of the tasks, but have shown to be decent indications of required effort. While these two tools are fairly easy to use, the number of Lines Coded has almost no correlation with program functionality or quality, which often results in the tools being seen as a less desirable practice. LOC is an intuitive tool that is easy to use and most can relate to. However, unlike Story Points, only developers take part in estimating LOC. While this reduces overhead considerably, at the same time it provides some inherent project uncertainty.

Ideal Days

With Ideal Days you apply an absolute measure of required effort to implementing tasks. The main idea is you provide an estimate for the number of unobstructed development days or hours needed for completing a task. While this is also very easy and intuitive to use, it can be implemented with almost the same benefits as estimating with story points can. This is because the metrics can often be used interchangeably: Imagine a team estimating using Ideal Days having a backlog of tasks requiring 0.5 day, 1 day, 2 days etc. Now, convert each of these estimations into Story Points using a 1:1 scale: 0.5 Ideal Days = 0.5 Story Points, 1 Ideal Day = 1 Story Point and so on... Although it doesn't necessarily fit with the Fibonacci sequence the team will still be able to commit a certain number of point to each sprint, measure velocity, get an idea of current overhead, break down large tasks into smaller etc. just like when using the Story Point estimation tool. A story thought to take an Ideal Day typically takes a lot longer to implement than one work day. This is because the tool doesn't take into consideration interruptions such as meetings or unexpected problems such as computer-failure i.a. The team should take special care when estimating Ideal Days, unlike Story Points, that takes input from the whole team, Ideal Days may vary greatly between developers; a task that would take one developer half a day might take another 2 days to implement. People not only have different schedules that might affect the number of Ideal Days, but also has different skill levels to take into consideration.

Hours

Estimating using Hours is probably the most common practice in the industry. Estimating using hours can definitely have its merits as it is easier to align with core-business characteristics. Take special caution when estimating using hours, as this statistically seem to be one of the biggest pitfalls for project success. General caution is advised when estimating using hours:

1. Do not place much confidence in your first guess.
   • People are notoriously bad at gut-feel estimations. If you are estimating alone, make up at least a couple of bids before deciding.
2. Use a formula to improve your estimate.
   • A general formula could be (OD*1 + ED*4 + PD*1)/6, where OD is the Optimistic Duration, ED is the Estimated Duration and PD is the Pessimistic Duration.
   • Using this method it is possible to weight different estimations to get a more realistic bid.
   • If the team later discovers they're being overly optimistic or pessimistic, they can easily change the weights in the formula.
   • See also the PERT Article from the Wiki.
3. Revisit estimates often.
   • Once the project progresses, review hourly estimates often using the new knowledge you've acquired. This will help keep track on the future delivery date.
   • This is generally also a good idea when estimating using Story Points. Completion of some tasks, especially in IT projects, may positively influence the remaining work in another, unrelated task.
   • This is because possible code-redundancy becomes more apparent when some tasks have already been developed.
4. Remember estimation uncertainty is often log-normally distributed (citation needed).
   • This has the implication that estimates are often skewed to the right.
   • This simply means that estimations generally are overly optimistic with a larger portion of estimates being lower than reality. Use this when developing your estimation formula.
5. Compare and share your estimates with stakeholders and other developers.
   • Teams are much better at estimating than individuals. Agile development also emphasize that estimation is a Team Sport.
   • Don't be afraid to tell the product owner your real hourly estimate for a task; maybe the healthy discussion will align some expectations or clear up some miscommunication or documentation.

For further reading on human uncertainty see the Wiki: Mindfulness and Cognitive Biases in Project Management.

Mixed Approach

Mixing hours and Story Points is not a recommended practice. This is because hours and Story Points don't measure the same thing. There exist no good trade-off metrics as a Story Point might pertain to 8.3 Hours of work one week, 4 Hours the next and 7.2 Hours the week after that. That being said, the practice is not unheard of:

Story Points are, as elaborated, a relative measure good for long-term estimation and progress-tracking. But it may lack some short-term benefits. In the referenced examples(see also Annotated Bibliography), two teams have decided to use Story Points as a high level measure, while using hours as a low level measure for Sprint-planning. In both examples it is argued that Hours give a better chance of committing "to capacity" for a sprint so that you don't end up having idle developers towards the end of an iteration.

Annotated Bibliography

For further reading it is suggested to check out:

Books:

Henrik Kniberg and Mattias Skarin (2010), Kanban and Scrum - Making the most of Both, C4Media Inc., USA. ISBN 9780557138326

This short book gives an outline of the Agile framework Scrum and the Agile Kanban-method. The book touches upon different Agile tools and practices, describes which is prescribed in Kanban and Scrum and explains how they differ.

The book is recommended for anyone interested in Agile Project Management, as well as new IT development teams deciding upon tools, KPI's and work practices.

Articles and Web-pages:

Sutherland, Jeff et al. (2001), The Agile Manifesto

This web-page includes the Agile Manifesto, produced in 2001 by prominent and early practitioners of Agile development. It includes the 12 core practices of Agile development as well as the names and information of many renowned scholars and practitioners of Agile, who have since signed the Manifesto. The 12 practices is a recommended read for anyone trying to understand or uphold core Agile practices.

Cohn, Mike (2007) Why I Don't Use Story Points For Sprint Planning and Cheng, Chai Wei (2012) Story Points Versus Task Hours

These two articles independently argue the drawbacks of using Story Points for Sprint planning. The two authors reach the same conclusion, that Story Points finds merits mostly in their ability to easily estimate and measure velocity as a high-level measuring tool while, in fact, Hours in many cases are a better fit when planning sprints. Both authors argue that the quite fluctuating velocity of Agile teams makes it too hard to estimate to capacity in Agile Sprints.

Radigan, Dan Agile Estimation, Atlassian web-page

Atlassian is a market-leading company that sells online tools specifically to facilitate easy Agile Project Management for both small and large IT companies. Atlassian's web-page has a huge collection of peer-reviewed articles and guides about Agile Project Management that is highly informative and serves as a great introduction to applied software development. The linked article outlines some core considerations when estimating in an Agile environment including corroborating with the Product Owner, estimating as a Team and considering estimation errors in Sprint-Retrospectives.

Case Studies and Research:

Georgsson, Anna (2011) A Case Study at Swedbank IT

This interview-based study of a department at Swedbank IT thoroughly examines various Agile tools and practices with a special focus on Story Points and improving Estimation certainty. The case-study is yet another example of Scrum implementation and core findings include improved task understanding and decreased unnecessary work. The Master Thesis provides a well-referenced list of advantages of Scrum-practices and tools.

Williams, Laurie. Scrum Practices: Experiences of Three Microsoft Teams

This case-study research is based on the empirical data from three Microsoft development teams that has transitioned to Agile Project Management and a Story Point estimation-practice. The study gives a brief introduction to the Scrum framework and various Agile tools and practices applied by the three teams. Core findings include a drop of efficiency the first 3 iterations of the implementation followed by a significant improvement in productivity without having a cost on quality. Another core finding is the merit of implementing a "Planning Poker"-practice. Recommended reading for anyone doubting the eligibility, purpose or implementation of Scrum.