Tuesday, June 23, 2009

Dealing with Bottlenecks

Whether it is a manufacturing plant, or a software product team, one can always find bottlenecks. Either they are human resources, machines, tools that we use or processes that are flawed.

2 important things that "The Goal" puts into perspective is that

* The capacity of the plant/team is always equal to the capacity of the bottleneck

In simple terms, for an iteration, if your Dev Capacity is 10 points, your QA capacity is 10 points, but your Customer Acceptance teams' capacity is 5 points, then ultimately your throughput is going to be just 5 points, even if the Dev and QA team work at their full capacity. The customer acceptance team is the bottleneck.

* If you have a bottleneck in between the flow, soon your inventory will go up

In the above scenario, within an iteration, you can easily have a 5 point story hangover/inventory pile up before the Customer Acceptance team. If we continue to sign up for 10 points every iteration, the inventory will easily go up by 5 points each time


What to do with such a bottleneck ?

* In the example scenario, the easiest option to minimize inventory would be to either increase the Customer Acceptance team's capacity. If this is not possible then it is better to sign up for 5 points in each iteration rather than building a hangover story inventory which will have its own maintenance cost. This will make the system more efficient

* Look at hidden costs and try to minimize defects going through the bottleneck. In this case, since we know the Customer Acceptance team is maxed out of capacity, it is better to actually do as much quality testing as possible before the Customer Acceptance stage. This will ensure really low probabilities of a defective story coming out of the acceptance (bottleneck) stage , which can prove really costly.

Another example from my previous project where we applied this was, when we had a major build system crisis The massive Cruise environment we used had hardware issues , and also some issues with software upgrades. Our Quality Analysts heavily depended upon builds from the Cruise machine to start their testing but in this case, the Cruise itself became a bottleneck. On of the measures we adopted was to make sure Business as well as Quality Analysts started testing exhaustively on developer boxes (machines with checked out code). This ensured that defects were caught much before the bottleneck and the bottleneck capacity (which is really expensive) is not wasted on churning out defective pieces for the next stage.

4 comments:

  1. Nice post. I think it is mainly about moving (removing is not possible) the constraint and maintaining the flow. I used same concepts to write about how to measure QA velocity? (a trick question :). http://tinyurl.com/nxgr4w.

    By the way is it just me or is the comment text box function is broken?

    ReplyDelete
  2. wow...really nice post and helpful contents, thanks for sharing.

    ReplyDelete
  3. I go back to my dominant theme these days. S/w development is not manufacturing. In manufacturing, the job is small and clear and repetitive enough to determine the exact time it would take. Don't underestimate the relevance of Parkinson's law (work expands to fill the time) in s/w dev.
    Therefore, reducing the output of the preceding phase (from 10 to 5) is probably not a meaningful decision as the throughput of the acceptance testing team is much more amenable to change (than a task on the assembly line).

    ReplyDelete
  4. Continuing - Your other solutions aimed at doing just that. Improve the capacity of the acceptance test team by changing the way we work in the preceding phases. So, that is much more relevant, imho, to s/w dev.

    ReplyDelete