Why is a Story narrative signoff and Showcase acceptance from a Product Owner important every iteration ?

A proper development iteration will involve the development team working on a user story, and towards the end of it, showcasing what the team has completed. This is not only critical to get feedback from the product owner, but also from a scope management standpoint, which most teams tend to neglect.

It is the responsibility of a Product Owner to sign-off on a User Story narrative before the beginning of the iteration, and see the working software during the showcase. Sometimes Product Owners do not end up doing this because of various reasons, and this can result in the following smells

Story requirements keep changing constantly within an Iteration

A narrative signoff is an agreement between the Product Owner and the Development team around the Acceptance Criteria of the story mentioned in the narrative. Sometimes Product Owners shy away from doing the same because they are not sure what they exactly want in the given story. They also want to work in a mode where they can propose changes to the same story within an iteration, since in their mind, as the team is doing agile , it is supposed to embrace changes at any point in time.

This setup results in story changes leaking into a development iteration and causing a lot of churn, and hence wait in the form of code and testing effort. This also results in a team not achieving sustainable pace with every iteration, because of constant changes in user stories,

A fix to this problem,is to coach the product owner on writing/signing-off User Story narratives well before the Iteration starts. What is also required is to make him/her understand that a change made later in the iteration will come at extra cost. One also needs to highlight the amount of waste generated because of changes being made, and the hidden costs associated with thrown away code etc...The team should ideally work in a mode, where if a story narrative is not signed off within an iteration, it should not be played at all. And once development has kicked off for the story, any change to it, should come in as additional scope request which is up for negotiation.


Stories do not get accepted at the end of an Iteration, and the team does not have a clear definition of "Done" for stories.


If Iterations are planned correctly then towards the end of every iteration, stories developed in that time frame should be showcased to the Product Owner and accepted. Sometimes Product Owners are averse to the idea of "Acceptance of a User Story" and insist on a feature being complete to sign-off. Taking such a stand results in many stories within a release pending acceptance till the very end of a feature. Some Product Owners lean towards this claiming a complete feature is much more testable and makes logical sense than a single user story. If this is happening, then it is either a case of your User Stories not following the INVEST principle or the product owner wanting to delay commitment for various reasons.

Again this very problem should be fixed as early as possible in a project. The practice of acceptance of stories every iteration ensures that the product owner has validated the working software as per his spec every iteration. The feedback cycle is cut short and any change beyond this point on requirement is a new story altogether. More importantly, this builds a lot more confidence in the development team that they are on the right path. Ideally, "Customer Accepted" should be the definition of "Done" of a User story, and only when a story is "Customer Accepted" should it be counted in velocity of the team,

The Product Owner role is critical for an Agile team to work smoothly. A good product owner should not only have an excellent view of the product and domain, but also understand the importance of some of his responsibilities towards the delivery team.

Sprint backlog in Scrum is evil

As is defined by Mike Cohn Scrum encourages teams to breakdown stories in the Product Backlog into tasks which are 8-16 hours worth of effort and which become a part of the Sprint Backlog. These are the tasks which the team commits to completing at the end of a Sprint (which can be 2-4 weeks).

The problem with breaking a good user story into tasks like "Create the UI", "Develop the Service", "Test the Service" etc... and maintaining them separately in a backlog , is that it encourages Scrum Masters to allow different people in the team to pick these bits of tasks of a big user story. So a scenario where Bob (and his pair ?) works on the "Create the UI" task and Ron (and his pair? works on "Develop the Service" in parallel, in order to finish work fast is encouraged in the team.

Scrum templates like those proposed by Ken Schwaber, take it even further by asking people to track hours logged on these tasks. Not only is tracking tasks like these unnecessary, but if a developer does not develop a story end to end, he completely misses out the bigger picture of the architecture. This is where a team ends up with UI experts who know only JavaScript/HTML with no clue of the services layer and Services experts who just develop services without knowing their consumers.

To me, a good user story which has business value, is the only unit of work I would like to track. Developer pairs in a team should be encouraged to work on complete user stories so that they understand the architecture end to end , and have the satisfaction of adding business value to the product they are working on. This is extremely important but unfortunately I have seen many adopters of Scrum mislead by popular Scrum literature available.

Managing Agile team chaos with smaller sub teams

Agile practices work really well when you are dealing with smaller teams. Small teams are bound to be much more efficient and focused. It allows people to know each other well, while having fun at work. Simply put, it reduces the amount of bloat or waste, and it becomes quite evident when you see the team working.
Ideal Size

I prefer a team to be a max of 2-3 developer pairs. Any number more than this calls for a team to be split into smaller sub teams.

Team composition

Ideally based on features or themes. If you have 4 pairs of developers you are bound to be working on more than one theme or feature. A good Business Analyst or a Product Owner should be able to identify these themes in the functionality being developed. If you do not have 2 broad features, try to club the smaller features based on a pattern or a theme. Never split the team based on infrastructure (eg: UI team, Services team)

Align a BA and a QA to be focused on specific themes. If the features are functionally less complex, then a BA and a QA might be shared across these teams.


Split the meetings, making them shorter and more focused

The idea of reducing bloat also applies to the team meetings. Have separate standups among the smaller sub teams. When you have a max of 6-8 people in a standup, everyone will be attentive, interested and will remember what the first person said in the standup when it ends.

Split the Iteration Planning meetings also according to teams which are focused on tasking and estimating features they are developing. You can even have mini retrospectives with sub teams on a regular basis.


Split the card wall

The story card wall tends to clutter up with cards for 6 pairs of developers. Splitting the card wall into separate ones for each sub team helps in keeping the card wall cleaner and motivates the team to keep it up to date.


Handling dependencies across teams.

There will always be dependencies between sub teams. The teams might share some areas in the code base, builds, QA integration boxes etc... A PM or a lead of the whole team plays a vital role here. A PM will have to be part of both the standups regularly and ensure that dependency blockers are removed by teams communicating effectively. As long as the teams are sitting closely, once can always shout out if there is a show stopper !

The whole team can talk about technical dependencies and showstoppers in impromptu developer huddles/technical standups as and when required.


Managing people rotation between teams

People will get bored working on the same feature. Developers will be bored if they were working in a team just fixing bugs or small enhancements. similar is the case with BAs and QAs.Rotation is also important to allow team members to pair with members of the other sub team.

Any rotation between teams can be done at the beginning of an iteration. 1 person swapping in between two 2-pair teams would be ideal. A PM can also choose to limit the number of rotations based on constraints such as knowledge around a particular area of functionality etc...


Better feedback, pair rotation, and self organization

In a 8-10 people team sitting across a table, people can quickly learn about each others strengths and weaknesses. Knowing each other well automatically allows for a better feedback culture within the team. Pair rotation within 4-6 people allows people to pair with everyone in the sub team.

In a small team people have the tendency to self organize themselves. You will see people take on more responsibilities, sometimes a BA playing a PM hat and proactively planning stories for the next iteration. People will not wait for a retrospective to speak their mind about a certain issue, but shout across the table !


Stitching it all together

The Project Manager of the whole team plays a key role in keeping the sub teams focused and on track. For this purpose a PM can conduct an Iteration Close/Planning meeting with the whole team, showing team members where the team stands on velocity, and scope burn down, along with any team rotations for the next iteration. This could be followed by letting the whole team know the scope planned for next iteration.

Needless to say that team outings and other fun activities would involve the whole team !

Needless time tracking in Scrum

On a new assignment, while the client is trying to slowly adopt Scrum (and hence Agile), I have been bothered by the needless time tracking that is proposed by some of the Scrum templates and related literature .

Tracking hours remaining at a task level, or even %age task completion to me is useless. Either a user story is done or not done, simply because you cant do much with a half baked story. A story which is done contributes to the velocity or throughput. Everything else is inventory.

Services have business value

When you are on a project which is about evolving a set of services from an existing application, or building from scratch, there is always a lot of talk on the architecture and service governance aspects.

But if you are a Business Analyst on this piece of work, and you are asked to write stories for these services, you will wonder.. "well how can we showcase just services ?". If you have never done this before, you might even conclude that writing stories without a consumer of the service is not such a good idea. A lot of Agilists might argue that having a services story is a bad user story.

This happens because the fact that "Each Service has business value" does not become very apparent. If you look at Amazon WebServices it becomes immediately evident that each of the services have a set of features which can be sold to customers.

Services are a reality in today's world on the web, and people make money by selling services. A story which builds a service to search a book based on ISBN has business value. It does not need to have a pretty front end consumer for you to write a good user story. A temporary client/UI would be sufficient or maybe even functional tests/specs. It is still a good user story. This essence needs to be spread across the entire team, so that BAs and Devs are on the same page on "Good User Stories" for this kind of project.

Both BAs and QAs have to embrace this along with the developers on the team, and look at ways to apply Agile principles considering this, rather than being dogmatic in their approach towards Agile and SOA.

Challenges in applying Theory Of Constraints to Agile Software Development

After finishing Goldratt's first book, the Goal, I immediately jumped to David Anderson's book on Applying TOC in Agile Management. Preetam also has left some interesting comments on my blog on the differences between s/w and manufacturing which should not be neglected.

One important thing which is still puzzling me at this stage is, if you have to account for throughput, how do you measure value. Value here is the value of a story or a feature. If you are a services shop building an application for a customer, it is upto the customer to determine this business value of a story. Quite often this is not easy to determine, specifically for people like Subject Matter Experts you are dealing with to gather your requirements, who do not deal with sales at all.

In a manufacturing plant, if you have signed a contract to deliver 10 cars for say $100,000. The cost of an engine still waiting in the queue (work in progress) is almost equal to the value of 1 car, $10,000. However the similar analogy is not easily applicable to a software development team. If a story is in progress, it cannot be easily translated to how much $$ is part of inventory.The story might be part of a bigger feature. Also, determining the business value for that feature in $$ itself can be tricky

David's book touches upon in-house IT shops providing IT services (eg: SaaS) or implementing a software product. In these cases, revenue from Sales can be directly mapped to what is being developed. Which is what should happen, in order to increase the throughput of the system, and hence increase the revenue. This is not the case when you are providing IT services to other clients (not in-house).

Some might say you can use story estimate (ideal days or points). Again this represents cost and not value. The iterative nature of Agile also makes it difficult to apply the same principles as a manufacturing plant.

While I am still looking for some answers, the learning on how to deal with bottlenecks, optimizing the whole system, and applying the same to software development, from both the books, are still quite useful.

Dealing with Bottlenecks

Whether it is a manufacturing plant, or a software product team, one can always find bottlenecks. Either they are human resources, machines, tools that we use or processes that are flawed.

2 important things that "The Goal" puts into perspective is that

* The capacity of the plant/team is always equal to the capacity of the bottleneck

In simple terms, for an iteration, if your Dev Capacity is 10 points, your QA capacity is 10 points, but your Customer Acceptance teams' capacity is 5 points, then ultimately your throughput is going to be just 5 points, even if the Dev and QA team work at their full capacity. The customer acceptance team is the bottleneck.

* If you have a bottleneck in between the flow, soon your inventory will go up

In the above scenario, within an iteration, you can easily have a 5 point story hangover/inventory pile up before the Customer Acceptance team. If we continue to sign up for 10 points every iteration, the inventory will easily go up by 5 points each time


What to do with such a bottleneck ?

* In the example scenario, the easiest option to minimize inventory would be to either increase the Customer Acceptance team's capacity. If this is not possible then it is better to sign up for 5 points in each iteration rather than building a hangover story inventory which will have its own maintenance cost. This will make the system more efficient

* Look at hidden costs and try to minimize defects going through the bottleneck. In this case, since we know the Customer Acceptance team is maxed out of capacity, it is better to actually do as much quality testing as possible before the Customer Acceptance stage. This will ensure really low probabilities of a defective story coming out of the acceptance (bottleneck) stage , which can prove really costly.

Another example from my previous project where we applied this was, when we had a major build system crisis The massive Cruise environment we used had hardware issues , and also some issues with software upgrades. Our Quality Analysts heavily depended upon builds from the Cruise machine to start their testing but in this case, the Cruise itself became a bottleneck. On of the measures we adopted was to make sure Business as well as Quality Analysts started testing exhaustively on developer boxes (machines with checked out code). This ensured that defects were caught much before the bottleneck and the bottleneck capacity (which is really expensive) is not wasted on churning out defective pieces for the next stage.