In an agile project requirements are often discovered, adapted and/or changed completely as the project progresses and very little up front design work is performed as a result, unlike traditional methods that work on the belief that requirements can be defined up front and then fixed for the duration of the project.

For project managers who need to determine the ROI on a project the move to an agile methodology presents new challenges.

Using a waterfall SDLC, project managers can look at the scope of the deliverable, calculate what the expected cost will be and express that cost as a Return On Investment.

An agile SDLC however does not have a fixed scope, instead relying on the principles of adjusting development priorities based on changing needs and uncertain requirements.  How then does a project manager calculate the ROI for an agile project when scope is so uncertain?

The answer lies in the product backlog.  A first up ROI calculation can be done as follows:

  • Gather the initial requirements for the project and place them in the product backlog
  • Get the team to estimate sizes for the requirements individually at a high level.  Use story points, comparison points, ideal days, whatever unit of measure you are comfortable with, but keep it high and estimate quickly.
  • Now, examine the overall size of the product backlog.  If the size seems too large at the start, then it probably is.  Before proceeding either look to cull items from the backlog, or cancel the project.
  • Next, work out your starting velocity (i.e. how many points/ideal days of work your team completes in a day). If possible, get approval to develop one item (or a few) as a proof of concept exercise - this often helps determine what the initial velocity of the project is likely to be.  Remember to ensure that the POC is developed using the same techniques that the project proper will use.   If you can't do a proof of concept then try to use previous projects as a guide for working out the estimated initial velocity for this project.
  • Extrapolate from your initial velocity how many days of effort will be required, and thus what the cost will be.
  • Use the estimated time and cost figures to determine your ROI.

There are a few things to keep in mind:

After the sizing exercise if the job seems to big don't pressure the team into reducing their sizing estimates so that the ROI looks better.  This is a perfect way to set the project up for failure.

If the initial velocity from the proof of concept is low, don't start into the project with the hope that velocity will increase as the team progresses.  This rarely happens and even when it does, the velocity increase is often marginal.  Your velocity is your velocity - accept it and make decisions on the assumption that it won't improve.

 

Now at the start of the project your ROI is a known quantity, however like all agile projects you should constantly reassess this.  As time progresses scope will likely change and the initial velocity the ROI was based on will stabilise into the projects average velocity.  As this happens reassess the ROI of the project regularly, especially early on in the project.  If ROI starts trending downward it may be necessary to make a hard decision and cancel the project.

As a note - the philosophy behind waterfall projects is one that says requirements can defined up front in detail, however it does not ensure that those requirements are estimated correctly and far more often than not, the estimates given are treated as commitments of actual time when everyone knows they are likely wrong.  Project managers also typically treat ROI calculations as a one-time exercise to get a project started and never revisit them after the project commences.  As reality bites and actual development times diverge from estimates or the project scope changes, the ROI fundamentals are altered.  It's entirely possible that the ROI will degrade so badly that the project should be canceled, however without constantly reassessing the inputs PM's are blind to the problem.

Agile projects are not immune to ROI degradation, however  it is more likely that any such problem will be picked up early based on the monitoring of project velocity, the forward planning elements of agile projects in general, the expected openness of communication and the reassessment of timelines during the life of the project.  Early detection implies early corrective action, and if that means cancellation, then it's better to cancel the project early than find out there's a problem when it's too late to do anything about it.