Agile Meets Waterfall

This morning I’ve been looking at the release schedule for a major project launching next year. Chances are that you will be affected by it somehow. No names disclosed though. It’s a replacement for an existing capability and will use an off the shelf, quasi-cloud product (that is, software that is already in use by others and so can, in theory, be configured but that, in practice, will need some development to make it do what is really needed).

The top level project plan looks like this

Spring 2019Discovery
Summer 2019Design and Build
Spring 2020Testing
Summer 2020Go-live
Spring 2021Transition completed
ThereafterContinuous improvement

Plainly the team want to be agile. They want to iterate, adding features and capabilities and seeing how they land before improving them – going round the discovery / alpha / beta / live process repeatedly. But, at the same time, they have a major existing system to replace and they can’t replace just part of that.

They also, I suspect, know that it’s going to be harder than everyone expects, which is why they’re sticking to those nebulous seasonal timeframes rather than talking about particular months, let alone actual dates. The rhythmic cadence of Spring/Summer milestones perhaps suggests an entirely made up plan.

Ever since I first joined the Inland Revenue and heard a team saying that they “hoped to be up to speed by the Autumn” I’ve laughed whenever seasons are mentioned in the context of delivery. It’s as if there are special watches issued with 4 zones marked on them – one for each season.

What do you do when meshing the need to replace an existing widely used system that is no longer doing what it needs to do and that needs radical upgrades, with a new capability that can’t be introduced a bit at a time. This is not a start up launching a new product into the market. They’re not Monzo who were able to start with a pre-paid debit card before moving into current accounts, credit cards etc.

These kinds of projects present the real challenge in government, and in any large organisation:

How do you replace your ageing systems with new ones whilst not losing (too much) capability, keeping everyone happy and not making an enormous mistake?

At the same time, how do you build a big system with lots of capability without it going massively over-budget and falling months or years behind plan?

There are some options to consider:

  • Is a NewCo viable? Can a new capability be set up in parallel with the old, and customers, users or businesses migrated to that new capability, recognising that it doesn’t do everything. This is ideal MVP territory – how much do we need to have to satisfy a given group of customers?Ideally have less complicated needs than the full set of customers, and we can shore up the NewCo with some manual or semi-automated processes, or even by reaching into the old system
  • Can we connect old and new together and gradually take components away from the old system, building them in the new world? This might particularly work where a new front end can be built that presents data far more clearly and adds in data from other sources. It might require some re-engineering of the old system which perhaps isn’t used to presenting data via APIs and has never been described as loosely coupled.
  • Is there standalone capability that will make for a better experience, perhaps using data from the old system (which can be extracted in lots of different ways from cumbersome through to smooth), with that new capability gradually expanded?

None are easy, of course, but they are all likely better than a big bang or the risk of a lengthy project that just gets longer and more expensive – especially as scope increases with time (people who have time to think about what they want will think of more things, and industry will evolve around them giving them more things to think about).

There are, importantly, two fundamental questions underpinning all of these options (as well as any others you might come up with):

1. What do we do today that we don’t need to do tomorrow. Government systems are full of approaches to edge cases. They are sedimentary in nature – built over the years (and decades) with layer after layer of policy change. Today’s system, and all of the capability it has, is not the same as what is needed. This allows a jackhammer to be taken to the sedimentary layers so that the new capability isn’t trying to replicate everything that went before.

2. What do we need to do tomorrow that we don’t do today. This gets the policy arm the chance to shape new capabilities, including simpler policies that could have fewer edge cases, but still cater for everyone. It will also allow new thinking about what the right way to do something is, considering everything that has gone before but also what everyone else is doing (in other industries, other government departments and other countries).

Asking those questions, and working the answers really hard, will help ensure that the solution reflects the true, current need, and will also help get everyone off the page of “we need to replace what we have” when, in reality, that’s the last thing that anyone should be trying to do.

There is, though, one giant caveat to all of this which I will take a look at tomorrow.

Leave a Reply