Measuring progress

Measuring progress in software development has always been up for debate. There was the waterfall era of Gantt Charts created using MS Project which used to breakdown effort into smaller work items with dependencies. These were easily outdated the very next day they were created.

The Agile community brought in the idea of user stories and doing them in small iterations. We then measured progress using a burn-up or burn-down chart which burnt either story points or ideal days. Soon, we started feeling the pain of the inventory we carried between releases which happened in a span of 3-6 months. This prompted us to measure and limit our work in progress and measure lead and cycle times across the delivery workflow.

Releases started becoming more frequent with technology advances in the cloud, virtualisation and deployment automation. The continuos delivery era came through with examples of people deploying almost everyday to production. Startups really needed this to validate their ideas in the market and course correct themselves based on market feedback. A/B testing became common place and measuring validated learning and cost of delay in deploying a feature made more sense for these new businesses.

Anyone can easily get confused trying to decide how one should measure progress or at least start with, in their project.  There are 2 key variables in making this decision, the kind of relationship IT has with the Business, and the type of product ultimately being built.

So here is the quadrant I am talking about.

Outsourced / Utility (bottom left)

To the extreme left on the relationship axis is outsourcing your IT. Trust levels are fairly low. It could be a new IT vendor who has not delivered for the business. Work is probably happening outside the office or even the country where the business exists.

The extreme bottom of the product axis sit utility/internal applications which are mainly for keeping the business lights on. (Time Sheets, Payroll, Company intranet etc...). There is no market advantage to be gained by these. They are well defined internal business problems.

Start with measuring progress in these cases using effort estimates such as Ideal Days, Story Points, T-Shirt sizes etc... A burn up chart should be a good indicator of where the project stands. Try to work on building enough trust which enables you to move to the right axis in terms of relationship.

Partnership / Utility (bottom right)

Moving right on the relationship axis is only possible by building trust. And that is possible by delivering frequently to the business and also maintaining the people relationships at the right levels. A vendor who is treated as a mere supplier can become a trusted partner for well defined applications and their maintenance as they prove themselves to have the required expertise.

Move progress measurements from effort based units to delivery efficiency based units. Optimise the workflow till production by minimising inventory you are carrying and reducing lead times. A Cumulative Flow Diagram  (CFD) can provide the same view.

Partnership / Strategic (top right)

This is really the sweet spot. This is where the business has engaged IT to build a strategic product which is probably completely new to the market. It could also be a product that is struggling and badly in the need for new ideas. The IT here is a closely trusted partner which the business can rely on.

Given the volatile nature of what needs to be built, the need for agility is the highest here. Being able to deploy a new feature and rolling back as quickly as it was deployed, if hypothesis fails, is equally important.

Measure progress here using business outcome based units such as validated learnings and measuring cost of delay of a feature and the impact it has on the business. 

Partnership / Outsourced (top left)

This hardly ever happens where a strategic business need for technology is outsourced to a less trusted supplier !

Build enough trust before you can stop estimating

A lot of people in the Agile community these days are talking about estimation being wasteful and suggesting not estimating projects at all.

One radical view amongst all of this is to tell clients that as a vendor of a project

"I will try and deliver the best possible outcome based on resources I have at my disposal. I will give you frequent production quality software almost every other day for you to track how I am progressing. I will charge you per day or even per hour. Don't sign a huge contract with me but sign me up for a shorter span , say a month, review my software regularly to see what I have built and renew the funding for another month if you are happy. If you are not happy you can please kick me out. "

That might sound like the best thing to do or probably the most absurd depending on how your relationship is with your client. What people fail to articulate while suggesting this is that the entire premise is built on trust. And a whole lot of TRUST ! You tell a new client your are not giving an estimate and suggest the above dialogue you are bound to be frowned upon.

This view of stopping estimation is built upon the belief that "if we release software frequently, we do not need to worry about estimates". And this belief can be incorrect many a times.

For instance a new client who has never worked with you would want to compare bids from other similar vendors before choosing one. The issue becomes bigger in an outsourcing scenario where the main reason for choosing a vendor is cost. High level estimates on cost and time would definitely help the client make that assessment.

While releasing software frequently builds trust with the product owner(s), it might not build the same levels of trust with say the CXO group within the organisation. Maybe the CIO who is funding your project does not have enough time to attend your showcases or UAT sessions.

Asking a client to fund one month and fund the next month based on the product delivered might again sound good on paper but the reality is for a lot of big clients this means a lot of administrative overhead to manage. Choosing a vendor is a hard ask in itself and now being ready to switch vendors almost every month is compounding the problem.

Taking a hard stance by saying no to estimation will be highly immature in the above scenarios.But this does not mean all this is entirely impossible.  It is possible, but only after a period of time of working with the client. When a new project starts you have to bite the bullet and estimate. The important thing is to start moving towards a place where estimates become more and more irrelevant as you do more projects or more releases with the same client.

And this is where delivering value continuously will build trust with the client. Fostering people relationships at all levels with the client on top solid delivery will slowly nudge the client to start thinking about the product they are building rather than managing the vendor. This will then allow you as a vendor to focus way less on estimation and much more on building a great product.

Most "products" in the enterprise software world start as "projects" unfortunately. Projects are funded based on budget and the estimates are required. However changing this scenario is in the hands of the project team. It is the responsibility of the project team to slowly help the client focus on product building than project tracking. And that is only possible when there is enough trust built between a client and vendor, not always from day one !

Books that helped me build better software delivery teams

I have learnt a lot by making mistakes on the job but there have been books that have influenced my thinking to a great extent. I will try to list the ones that were the most influential in changing my perspective towards managing projects and teams.

Classic General Management

• The Mythical Man Month - Essays on Software Engineering
• The Fifth Discipline - The Art and Practice of a Learning Organisation
• Peopleware - Productive Projects and Teams
• Waltzing with the Bears - Managing Risk on Software Projects
• What did you Say - Art of Giving and Receiving Feedback

Theory of Constraints

• The Goal
• Agile Management - Applying Theory of Constraints

Agile

• Extreme programming explained - Embrace change
• User Stories Applied
• Agile Retrospectives

Useful Link : Top 100 Agile books of 2011

Lean and Kanban

• Implementing Lean Software Development - From concept to cash
• Kanban
• Principles of Product Development Flow

Technical

• The Pragmatic Programmer - From Journeyman to Master
• Working Effectively with Legacy Code
• Refactoring
• Continuous Delivery

The Lean Startup

• The Lean Startup
• Business Model Canvas

There are many more great books out there but I would highly recommend the minimum reading list above for building teams to deliver great software.

Prefer partnering with craftsmen over contractors

If you are in an IT department in one of the bigger organisations working in the major verticals of banking, telecommunication, insurance, retail etc... you are probably looking at a portfolio of work which has major bumps in the curve when starting a big software initiative and a steady state when those initiatives move to a steady state of enhancements and bug fixes (a.k.a Business as Usual (BAU) ).

If you are someone responsible for delivering this portfolio of work you probably are tied under the typical IT budget constraints of a big organisation. Hiring highly skilled people quickly to deliver the bursts of projects that come along is going to be difficult. Given this scenario it is more than likely that your attention goes to the right end of the curve where you see the amount of work in the portfolio reducing within a number of years. This is when you think "Why do I need to hire more permanent staff, I can get things done by hiring contractors who can then leave when most of the work is done."

This might probably work if the contract staff form a very small percentage of your overall organisation. You might even be able to find some real good contractors who specialise in certain niche areas which might be temporarily beneficial to your company.

But the moment you start relying on contractors to do most of the work in the pipeline, it is the beginning of a big world of problems. Because here is what will most likely happen.

• You decide on hiring contractors to get some major pieces of work done.
• Since these contractors are temporary, you ensure that you have enough permanent staff to oversee the work the contractors are doing. You have most of the permanent staff in all the important decision making roles.
• The contractors come in and start working alongside your permanent staff. The good ones want to drive new initiatives within these projects but soon get frustrated by the red tape approval process created by your permanent staff.
• Your permanent staff start feeling insecure about their jobs once they see some contractors who are better at the job than themselves.
• You might try to hire these brilliant contractors which creates an ugly set of dynamics at work.
• The contract staff becomes frustrated and start working just the hours required for their contract. If you hired really cheap contractors then they are probably in this mode from day one.
• The permanent staff has issues with the quality of work done by these contractors and sometimes vice versa.
• You add rigorous shortlisting processes for contractors in a bid to weed some bad ones out. But you still end up with some bad contractors because of budget constraints.
• Your project ultimately might go live but chances are that the quality of the architecture, codebase, user experience , performance is worse than ever. Your business stakeholders are unhappy.
• The contractors leave and you inherit a horrible quality system.
• Your permanent staff feels terrible maintaing this poor quality system. They either suggest a complete rewrite of the system again or suggest outsourcing the maintenance of the system to contractors.
• Another project comes along and you are probably still choosing contracting firms in the never ending grind of work.

So is there a better solution ? Let us look at the assumptions again

• Most systems you are building are of strategic importance to your business.
• There is more work in the pipeline than the permanent staff can chew and to be delivered in an aggressive timeframe which does not allow you to hire more permanent staff.
• The inflow of work might reduce after most IT systems are up and running in a few years.

and of course the more important long term goals that

• You need to be build a high quality system or platform which scales well to serve the business needs.
• You need to build an IT department with skilled technical people who are able to deliver and maintain the systems required for the business

So it is in your best interest as someone tasked out for delivery to build good software and at the same time set out to build an organisation which can do the same repeatedly in the future.

This is best done by hiring good people in first place, but if for various reasons you cannot, then at least try to partner with people who are passionate about the craft of building software than people who worry about billing the 8 hours of work.

To scale, try to find companies with whom you can partner with for their delivery services knowing how culturally passionate the partner company is about delivering the work for you. (And believe me , there are many good consulting companies out there which can help you out).

And by partnering, you engage with them at levels where they have a decent voice in decisions made on how to deliver the project. There is transparency and a drive towards building a win win collaborative culture within the team. You treat them as strategic partners rather than augmented resources in your delivery organisation .

It takes time to find such companies to partner with. You need to validate their delivery capability and culture fit over a period of time. But once you have done that, try and build a lasting relationship with them because in reality you will always have a backlog of work given the dynamic nature of your business.

After all, you are building long term assets for your company and it can best be done by partnering with people who care about it the same way you do. So invest in finding a good software delivery partner that can work for you instead of risking long term failure by using low cost/quality contractors.

When Agile meets yearly IT budget

There is a well-known problem when Agile projects meet a client’s yearly IT budgets. The problem with the yearly budget is that

1. It is largely incorrect as it has been planned with minimal information

Most big organizations have a budget review only once a year where a CIO and his team have to decide what their future looks like for the year. This is the time when everyone looks into their magic crystal ball to estimate the amount of funding required for their projects in the financial year.

These numbers are largely incorrect purely due to the scale at which they are done and with the limited amount of information available upfront.

2. Fosters project thinking and deliver under budget than building the right product.

Once a ballpark yearly budget figure is reached after minimal planning by project teams, the teams are encouraged to plan in more detail to secure their exact funding required.

Project teams then start building their detailed product backlog of stories with estimates and break it down into releases through the major part of the year to come up with their funding numbers which are then secured.

This not only results in a lot of upfront planning even in Agile, but also focuses the team on delivering that huge backlog of stories under the project budget, than focusing on building the right product fit for the market. Needless to say that these big backlog of stories often get outdated within the first few iterations and so does the estimates and hence the budget plan.

So is there an alternative?

The good news is that a lot of companies are beginning to recognize this problem. And these are companies who view their products as “Strategic IT” investments and not an operational overhead. The solution requires changes in 2 key areas, budgeting and project planning and tracking.

Rolling wave or Quarterly budgeting

A year is clearly too long to forecast and budget for upfront. One option is to budget for the first couple of iterations and then budget the third and fourth just before the completion of the second iteration based on real data. This ensures that at any point in time you have budget for the next couple of iterations.

If a couple of iterations are too less for long-term visibility, the other option is to do it quarterly.

The advantages of this approach are

1. More informed hence accurate budgeting based on actual data on the ground.
2. Encourages people who own the high level budget to engage with the team on the ground at least once every quarter.

Track projects to measurable goals and not to a product backlog.

Not having to forecast a yearlong budget is a boon to an Agile team. The team can then focus on building just enough of the backlog for their product to be able to validate it with end users and then pivot or preserve as necessary.

This is a mindset shift to the traditional Agile/Scrum technique of doing discovery workshops and then building a backlog of user story phrases that are then elaborated every sprint.

Instead, the team needs to come up with clear and measurable goals on how they measure the success of the product. This could be things like increasing user uptake of the product by releasing some new features, improved customer service, launching a new brand / service etc.… As some of these goals might be long term, it is important to break them further down to a level that can be measured every month or a quarter. An example would be to validate an idea for a web page with 5 users before building 3 similar web pages in the subsequent iterations and validating with 50 users.

The owners of the budget in this case can decide based on the progress towards goals of the first few iterations to fund / invest in the next few iterations. This also gives them on option to pivot or preserve based on what they see built so far.

To summarise, the advantages of this approach are

1. Less of time spend in estimation of effort and more time thinking how to build something valuable
2. Keeps stakeholders who make budgetary decisions more engaged in the product.
3. Makes product owner more responsible to think and show results building the right product than forcing the team do deliver a backlog

References

The Budgeting Black Hole - Johanna Rothman

Annual Budgeting and Agile IT - Ross Petit

Lean Startup – Eric Ries

Implementing Beyond Budgeting – Bjarte Bogsnes