Friday, March 25, 2011

Questioning Story Points

What is a Story Point ?

A Story Point is a subjective unit of estimation used by Agile teams to estimate User Stories.

What does a Story Point represent ?

Story points represent the amount of effort required to implement a user story. Some Agilists argue that it is a measure of complexity, but that is only true if the complexity or risk involved in implementing a user story translates into the effort involved in implementing it.

Here is an article by Mike Cohn that explains this in detail. Do make sure you read the comments.

What is included within a Story Point estimate ?

A story point estimate should include the amount of effort required to get the story done. The definition of done here should ideally involve development effort as well as testing effort required to implement a story.Once the user story is implemented, it should be usable in a production like environment.

Why are Story Points better than estimating in hours or days ?

Story point estimation is done using relative sizing by comparing one story with a sample set of already sized stories. Relative sizing across stories tends to be much more accurate over a larger sample, than trying to estimate each individual story for the effort involved.

A simple analogy is that it is much easier to say that Delhi to Bangalore is probably twice the distance of Mumbai to Bangalore than actually trying to put a number such as 2061 kms against Delhi – Bangalore.

Teams are hence able to estimate much more quickly without spending too much time in nailing down the exact number of hours or days required to finish a user story.

How do we estimate in points ?

The most common way of estimating stories is to categorize them into 1,2,4,8,16 points and so on. Some teams are more comfortable with using a Fibonacci series of 1,2,3,5,8 as the point scale. Once the stories are laid out in index cards, the team can start with sizing the first card which the team feels is of a smaller complexity.

As an example , a team might pick the Login user story and call it a 2 point story and subsequently pick a customer search story and call it a 4 point story as it probably involves double the effort to implement than the earlier login story. This exercise is continued till all stories have a story point attached to them.

Who should be involved in Story Point estimation ?

The team which is responsible for getting a story done should ideally be part of the estimation. If the team has QAs to test stories and do test automation, they should also be part of the estimation exercise along with the developers.

The QAs should be able to call out if story has less development effort and more testing effort involved. For example building a customer search screen might be a 4 point story, supporting it on 2 new browsers might be a 1 point development effort but a lot more from a testing perspective. In this scenario QAs who are part of the estimation exercise should call this out and size the story to reflect the adequate testing effort which in this example might be 2 points.

How do we convert Story Points into hours or days ?

This should not be done at a story point level. Story Points go hand in hand with Velocity and hence Velocity at the end of every iteration should be measured in the number of Story Points done by the team.

Should we do a best, likely , worst case estimate even when we are estimating in points ?

Giving 3 estimates for a User story can still be done using story points by just providing 3 different points for the best, likely and worst case scenarios. This is quite effective when estimating a large sample set of stories probably during the first release of the project where little code has been written.

Doing this provides a range across which estimates may vary depending on outcomes of certain assumptions which the team has made. For example a best case estimate for the Login story could e 2 points assuming integration with a local LDAP server, but if that assumption changes to a 3rd party provider integration the worst case could be 8 points.

How do we plan/schedule a project using Story Points ?

To convert story points into schedule, the team needs to calculate their velocity in terms of number of points the team can deliver in an iteration. This is typically done using yesterdays weather by averaging the velocity achieved by the team in last 3 iterations.

If the team is starting afresh , then a raw velocity exercise should be done. In a raw velocity exercise , the team decides how many stories can it finish in an iteration period. This is done by repeatedly picking different sample sets of already sized stories which can be done within an iteration period. The total points across different picks are averaged and that is taken as the velocity the team will achieve each iteration.

For example if the result of 3 picks was 6,8 and 10 points for a 2 week iteration then (10+8+6)/3 = 8 points is the raw velocity for the team for 2 weeks. A schedule can then be laid out assuming the team finishes 8 points in a 2 week iteration.

Can Story Points be standardized across various teams ?

Different teams will end up having a different measure of Story Point depending on the the sample set of stories they are sizing. Unless they are building the same system the amount of effort required to finish a 1 point story by team A will be different to the amount of effort required by team B to finish a 1 point story in their system. And this difference will ultimately show up in velocity of team A and team B.

If there is a large program of work which is split into multiple teams building certain areas of a large system, it is quite tempting to attempt to standardize the point scale across these teams. This again defeats the purpose of estimating using story points and it being a unit of measure subjective to a team.

How do we estimate spike stories in points ?

Spike stories are stories which a team plays to better understand how to implement a particular feature. It can also be used as a proof of concept. Since in a spike very little is known about the amount of effort involved, it is typically time boxed with an outcome that the team can agree upon. This can be approximately converted into points by looking at the velocity trend. For example, if it is required to plan a week long spike, and the velocity of the team has been 16 points, then we can attach 8 points against the spike story.

Is there a way we can calculate cost per point ?

Cost per point will typically be ( Cost of an iteration ) / (Velocity per iteration (in points) ) / . In cases where there is an additional stabilization sprint or regression iteration, the cost of that iteration should also be included to calculate the cost per point.

Are story points an excuse for teams not being able to estimate correctly in days/hours?

It is not an excuse but a reality that it is a waste to attempt arriving at an accurate number in days or hours for a User Story. The amount of effort and time required to arrive at such a number trades off against the benefits of estimating in days/hours.

Moreover estimating in days/hours often puts pressure on the team to deliver within the stipulated number of days and the team ends up burning themselves to meet such false commitments. This results in the team never reaching a sustainable pace over a period of time.

Do Story Points relate to Business Value ?

Story points are an internal measure of effort involved in implementing a user story. It does not, in any way, reflect the amount of business value a user story provides. There might be cases where 1 point story might provide a lot of business value versus a 4 point story in the same system. Business value is best left for the product owner and business stakeholders to be able to determine.

Here is an article which talks about measuring business value in much more detail.

How do we know if the team is getting better at estimation when it is estimating in points ?

It is a popular belief that if the team were to estimate in ideal days, then it is much easier to track if the estimation is good , by checking the actual days elapsed on a story and the progress against it. This is however counter productive as the team spends hours to estimate few stories to arrive at the magic number of days before being pressurized to deliver on that magic number.

When a team is relatively sizing stories in points, a trend slowly starts emerging where similar sized stories start showing similar time to implement them. If there is a bad estimate , then that bubbles up automatically as an exception

Should developers change their story point estimation as they learn more about the system they are building ?

If a story A was classified in the 2 points bucket, a similar story B coming in months later should be classified in the same bucket. If the team has learnt more about implementing them between when story A and story B were played, this will show up as an increase in velocity of the team.

It is better to setup a relative sizing triangulation board for the team which has placeholder stories from the initial estimation session, so that later on the team can relate to it while sizing a new story.


  1. What is the relation between the sprint backlog (in story points) and the burndown chart (in hours)? In other words, at some point the conversion is needed.

  2. You wrote:
    "unless they are building the same system the amount of effort required to finish a 1 point story by team A will be different to the amount of effort required by team B to finish a 1 point story in their system."

    I guess it might even be different even if they are building the same system. It might be same only if same team is building the same system. Anyway, it doesn't matter as long as the estimates are relatively consistent.

    Am I correct here?


  3. @Paolo If the estimation in done in points, then it is better to have the burn down chart to burn down points rather than hours.

    I do not believe in the burn down in hours and I had earlier blogged about it here

  4. @3rd guy If I were to clarify with an example, if there was a Login story which is classified as a 1 point story, and a Search story is a 2 point story by team A for web system, then a team B should also classify it with the same relative sizes for the same web system. The velocity of the team in points will ultimately show how slower or faster the team A or B is.

    This should be the same even if in team A , a new developer pair comes in and starts estimating. In that case they should use the relative sizes provided by the team earlier to compare and size the remaining stories rather than coming up with their own point system.

  5. I have a few of issues with story points. First, in a situation where someone cares how long (in real world terms) something will take, it's not possible to prevent them from saying "Oh you burned down have the story points in six months, so you'll be done in a year.", which would defeat some of the benefits listed.

    The second issue is on a team with people doing estimates separately (a necessity on a large project), it would be difficult to ensure consistency since there's no common point of reference. On the other hand, everyone knows how long a day is. Estimation doesn't necessarily get more accurate when the units get more abstract.

    If you put these two together, you could possibly do better by estimating in 'ideal days' and calculating an actual velocity.

    The trick to any estimating attempt is to avoid doing stupid things to meet a date that any thinking person would understand is non-deterministic. That requires education, communication, political ability. Metrics 'sleight of hand' where you substitute one scale for another won't cut it.

  6. @larry

    If one understands velocity of a team per iteration (in points) and the overall scope (in points), then I do not think in the real world there is a lot of difficulty in talking about how many iterations required to complete the overall scope.

    I agree that on larger projects getting consistency across the team is difficult. One method to achieve that is use a triangulation board and keep example stories which have been sized previously on it. Use this to size further stories relatively.

    Even with a few new people on the team , I have felt that asking them to size stories relatively in points is much quicker, than asking them to estimate in days and make them feel a lot more pressurized to deliver within that time ( which a lot of times does not happen).

  7. @Anand Ok well, I'm not entirely convinced, but you've convinced me to give story points a more thorough investigation. Thanks.

  8. How do we estimate voluminous work in agile.
    For example, in localization projects the work will be similar for different languagues but the time consumed will be more.
    When we compare it with a complex story the Story-points for the localization story will be very less. But if we add the story points depending on the effort then the concept of using story point estimation changes. How can we estimate story point for complex and voluminous stories?

  9. Nice post!
    Can you also add a point around consistency around story point estimation -
    A dev: 'When the project had begun this story was 2 points but now that we know how to do it, it is just 1 point!' I'm trying to highlight the fact that the effort on a story doesn't change based on how much a developer/QA is comfortable with the system. The sizing should always remain the same just that the team does things faster.

  10. @Disha Valid point ;-). Have updated the post. Thanks !

  11. Hi Anand,
    I am late to the post..:)
    I was trying to understand the estimation using story points in agile projects... Your article was really interesting and provided me a great insight...
    In between I have few more queries..With story point estimation everything is done in terms of points ( not days)... It means, we have got a product backlog with an estimated points (number) of stories . Then we prepare the sprint back log from the product back log. So what should be the measurable thing in sprint back log.. again is it like, the number of stories taken in a sprint..? So is it mandatory that a story has to be there in each sprint...If it is a set of tasks ( not a story) what should be the measurable thing..?
    Finally we will track the progress using velocity ( story points /sprint)..
    Now another question how will we quote to the customer..i.e. work duration in calendar days and effort in person days..??


  12. Anand, I came by your blogpost while searching for articles by Martin Fowler on Story Points.

    But first I read this article with Martin Fowler's picture and name next to it:

    Can you clarify if you wrote it, Martin Fowler wrote this, or did you two collaborate in writing it?

    I ask because I want to reference this article in my discussions and want to make sure I get the attribution right.


  13. @Murali - I wrote this as a blog post and Martin carried a link to it on his website. Dzone has aggregated Martin's link I guess.