Tuesday, June 23, 2009

Dealing with Bottlenecks

Whether it is a manufacturing plant, or a software product team, one can always find bottlenecks. Either they are human resources, machines, tools that we use or processes that are flawed.

2 important things that "The Goal" puts into perspective is that

* The capacity of the plant/team is always equal to the capacity of the bottleneck

In simple terms, for an iteration, if your Dev Capacity is 10 points, your QA capacity is 10 points, but your Customer Acceptance teams' capacity is 5 points, then ultimately your throughput is going to be just 5 points, even if the Dev and QA team work at their full capacity. The customer acceptance team is the bottleneck.

* If you have a bottleneck in between the flow, soon your inventory will go up

In the above scenario, within an iteration, you can easily have a 5 point story hangover/inventory pile up before the Customer Acceptance team. If we continue to sign up for 10 points every iteration, the inventory will easily go up by 5 points each time

What to do with such a bottleneck ?

* In the example scenario, the easiest option to minimize inventory would be to either increase the Customer Acceptance team's capacity. If this is not possible then it is better to sign up for 5 points in each iteration rather than building a hangover story inventory which will have its own maintenance cost. This will make the system more efficient

* Look at hidden costs and try to minimize defects going through the bottleneck. In this case, since we know the Customer Acceptance team is maxed out of capacity, it is better to actually do as much quality testing as possible before the Customer Acceptance stage. This will ensure really low probabilities of a defective story coming out of the acceptance (bottleneck) stage , which can prove really costly.

Another example from my previous project where we applied this was, when we had a major build system crisis The massive Cruise environment we used had hardware issues , and also some issues with software upgrades. Our Quality Analysts heavily depended upon builds from the Cruise machine to start their testing but in this case, the Cruise itself became a bottleneck. On of the measures we adopted was to make sure Business as well as Quality Analysts started testing exhaustively on developer boxes (machines with checked out code). This ensured that defects were caught much before the bottleneck and the bottleneck capacity (which is really expensive) is not wasted on churning out defective pieces for the next stage.

What should we measure on a project ?

While reading "The Goal", the author clearly states that there are 3 important measurements in a system ( a manufacturing plant, which can be extended to a software delivery team)
  1. Throughput - Which in a software project would be the No. of story points actually delivered at the end of the cycle (Iteration or Release)
  2. Operational Expense - Which in a software project would be the cost incurred (eg: billable consultants, development kit etc..) to actually deliver features (or story points)
  3. Inventory - Which again in a software world would translate to the number of stories sitting in transition states and not yet customer signed off (Eg: In Development, In QA, On Hold).
The more important point is that one cannot improve on one of these measures in isolation.

Why I say it is important, is because in teams delivering software we often tend to focus on "Increasing the team velocity" without looking at how we can actually reduce the number of "Hangover/In Flight stories".

It is also common for teams to actually sign up for more Story Points in an Iteration or Release in order to show improvement in throughput, which results in more stories in hangover because of some unresolved bottleneck.

The Goal .. is a refreshing read

I have started reading "The Goal". I am just half way through it and am really impressed by the writing style the author has adopted, and how effectively the essence of Lean and Theory of Constraints have been delivered.

The fact that the book is written in a style of a fast paced thriller novel makes it superb. If you are planning to read a book on Lean, this is one of the books I would highly recommend.