When the dragons grow too mighty
To slay with pen or sword
I grow weary of the battle
And the storm I walk toward
When all around is madness
And there’s no safe port in view
I long to turn my path homeward
To stop a while with you
from the song “Madrigal” by Rush, published at MetroLyrics
The Scene Setter: a typical Transformation Project
Many of us will have been here before and seen this story play out …
A customer, faced with the problem that existing systems do not provide support for their new direction, issues an RFP for the new system. A number of suppliers respond. Some may have been deeply involved in identifying issues with the existing system, some may be new to the process but aggressively seek to sell and market their products and services. A decision is made, a contract is awarded and work begins. Expectations are high.
Some time later, perhaps a few years, things are not going so well. The new system is late, perhaps over-budget, perhaps some features have been compromised. Expectations are not being met.
Why is this?
There are many different reasons and “experts” out there who can tell us what failed. Some of them have “successful projects” on their resume and will use those experiences to drive “success” on this project, like a cut and paste exercise. Vendors and clients alike try to institutionalise a “lessons learnt” process from previous projects so they are better able to identify what corrective actions to take. My personal view is that we now have a sufficiently well-developed body of knowledge that, while it may seem counter-intuitive to some, will help us succeed and I’d like to explore this view during the course of this and the following 2 blog posts.
Some Typical Reasons
One of the most common reasons is different expectations as to how each party will adapt to changing requirements.
When the project was conceived the customer’s IT team probably spent a lot of time talking to business users, gathering requirements, perhaps they even hired some consultants who are experts in business process to help them. They did a thorough job but, in general, they were documenting a melange of (1) what people thought they needed; with (2) what people actually did; combined with (3) what the IT people thought was possible.
What people thought they needed (1) is one interpretation of the future. Interestingly, what we believe is possible (3) is another. Often they are different. And often what people actually do (2) is left out because the people doing the work are not always consulted. Also when (3) is tested that can result in (1) changing and, eventually, (2) also changes. The fact that these 3 elements of the melange are different, are interdependent and subject to change over time make managing requirements in a multi-year transformation project difficult. There is an article by Paul Ralph titled “The Illusion of Software Requirements” that makes interesting reading in this context.
One project that I was involved with had a very extensive and well documented set of requirements that had been written by a set of very smart consultants 5 years ago. The CEO of the company admitted in an exec steering meeting that her business had changed in the past 5 years so she needed future releases of the system (of course it was a phased project) to deliver new functionality that could not have even been conceived 5 years prior. Whilst that was not a problem and was well understood by all parties what was the source of the problem was the discussion of “how to include it in the contract” – it was a classic case of two sets of lawyers trying to protect their respective clients by crafting a compromise agreement that felt like it benefited no-one a few years later!
Another reason might be “unmet promises over time” in the delivered product.
“Unmet promises over time” reflects the fact that the underlying software products are subject to change. Often a specific feature meets a need at the time of order, but when detailed requirements are written the feature is no longer suitable … something has changed. Often the result of the melange of requirements specified by the many stakeholders in an enterprise system results in significant customisation work to be done.
Then, during the course of a transformation project, the software vendor wants to issue an upgrade. They do their best to make it backwards compatible but there is always some point in the life cycle of a software product where they need to rework “old” features into a new paradigm, a new way of doing things.
Hence, with all the best will in the world, it is likely that somehow, in some way the software delivered will be different to what was originally conceived. But that difference is not necessarily bad … what if the business was prepared to adapt to changes ? Hmm, that’s a radical thought, but not easy to accomplish in a more typical waterfall-style development!
Another common problem is the typical end-user resistance to change.
Those pesky end-users (horrible term “end-user”, isn’t it?) have a job to do and often they are under considerable pressure to do it right and do it efficiently (think about the demands of life in a call centre). So when a new system comes along they can often struggle to cope with the change.
Even after all the training, after all their demands for changes in the user interface, and the workflows have been taken into account, the fact is this is still “new stuff” and anything new or changed may slow them down or cause them to make a mistake. One project I worked on had a set of requirements written in to the spec to ensure that invoice processing was run in exactly the same way as the legacy system (to avoid changes in some set routines for the finance department). Funnily enough that routine bypassed all the checks that the new system normally applied and thereby rendered the new system completely useless – effectively it emulated the old system!
There is a famous cartoon that seems to describe this process (see everystockphoto):
Where Does This End?
One project I was involved had the company lawyer attend every steerco. Funnily enough it had the desired effect and there was a renewed sense of collaboration between the client and the supplier. Perhaps we should bring the legal team early into the project as a way to get collaboration, focus and productivity?
The recognised best practice in consulting thinking with regards to solving these problems has tended to focus on business process analysis as a means to create solid requirements. Interestingly this work dates back to the work done by Hammer and Champney in Re-Engineering the Corporation (1993) and the work done by James Martin and IBM in creating a BPR method as part of Information Engineering (also in the early 1990’s). Today even moderately mature systems integrators are aware of the need for business process management and organisational change management in implementing new systems.
Despite advances over the years there is still a belief that 70% of transformation projects are prone to failure. There has to be a better way and the last 20 years of thinking in this space has yielded a lot of ideas. The big question remains for both customers and suppliers is how to make it work for the benefit of all.
A Way Forward?
The next post in this series will look into the thoughts I have for going forward but let me summarise them for you:
First there is the perspective of the Software Developers who, faced with the problem of developing software with imperfect and changing requirements, invented Agile Development. In doing so they challenged the project and contract management community and so a debate over Agile Contracts ensued. Even today it is a rare procurement team that feels comfortable contracting with suppliers for Agile Development in large transformation programs.
As Customer Service and Operations sought to deal with the problems that Agile Development caused them, as they sought certainty and tried to react to rapid change, so they invented DevOps. Whilst this is still an approach that is in early stages of maturing it shows great promise.
Meanwhile the increased focus on the online channel (web, mobile) has allowed a new class of design-focussed web developers to embrace UX. As UX expands beyond its roots and takes its place in mainstream software development we also see Agile moving out of the software development space and into business thinking.
Which brings us to the concept of Lean, whether it be Lean Six Sigma or Lean Thinking or Lean Startup, there is considerable focus on eliminating waste and this can apply equally well to service industries and transformation projects. When I refer to the business (or more precisely the end-users and project owners) being adaptable to change I do see the Lean Startup concepts as good ways to initiate and encourage that change.
However let’s see if we can make sense of this mess!