What we often call process improvement is actually change management. The fact that most process improvement initiatives fail or disappoint is primarily due to the lack of appreciation for what matters when attempting to drive change in an organization. It has nothing to do with suggesting new practices or telling people what to do.
In the software world, we have a bad track record of adopting wholesale change, or swinging this way or that way with fads that come and go. We can all chase statistics that would claim that the industry is not getting appreciably better or that the industry is doing a lot better than it used to be. Regardless of which stats we chose to believe, regardless of how successful we feel or what our bottom line is telling us, most teams struggle with how to change for the better. As an industry, we are looking at it from the wrong angle: it's about the people first, not the practices.
First off, we can't expect any change to be successful if we don't understand where we are moving from. Good or bad, our current practices, attitudes and results constitute a status quo. While the practices are easy to discern (though the perception of which practices are actually applied will differ, even within the same team), the attitudes will vary greatly across the team, as will the perception of the results (as very few teams quantify success in multiple dimensions). Some will feel that things are going OK while others will see the same projects as a disaster.
The better we understand all this, the better we are equipped to initiate change. With knowledge of the range of perceptions in the group and awareness of the motivating factors that drive the entire team, we at least have a chance of visualizing what a better future might look like. What is critical here is that better future will look different from each set of eyes. With each person that we neglect in this stage, our chances for overall success will diminish significantly. Everyone needs to be able to see where they are headed with the initiative, what is in it for them. Interestingly, there are times when the best alternative for change is not actually change itself, but driving a more consistent common understanding of what practices are currently in place: this will set the stage for an easier facilitation of change downstream, with fewer different perspectives to manage.
Anecdotally, we tend to perceive that things are going fairly well (or that those challenging spots are just the way it is in the software industry). When we compare that perception against the prospect of adding new practices into the fray, we have a strong bias towards keeping things as they are. We have an inertia towards remaining in our status quo. A key part of motivating the team, helping everyone understand what's in it for them, is to expose the inefficiencies that exist today. Few measure the cost of the weak components of current practices, often called the cost of non-quality, and without these measures we tend to see the world through rose-coloured glasses. In any team, even the most successful I have encountered, the existing costs of inefficiency will outweigh the costs of effective change.
Given baseline measures and an understanding of the range of motivations across the team (and knowing that for some, it is these baseline measures that are the prime motivating factors), we can start to identify what would be reasonable to do differently. If we have done our job well to this point, we have already lowered many of the barriers to change that would otherwise exist, and the challenge becomes more a matter of constraining the things we would take on at this point, through a triage of potential changes. We need to recognize that we won't go from the status quo to a perfect situation in one change cycle. The most effective route is to carefully select a few small changes, and the best way to do this is to help the group to decide on their own what makes sense to bite off.
Each potential thing we can change has its own cost-benefit relationship to what we are doing at the moment. Some will be highly disruptive to the team, and may actually result in a negative overall impact to our performance. Others, the ones we are looking for, will be a natural transition from what we are doing now, but will provide significant improvement in our results. It is impossible to identify the prime candidates for change without understanding the current situation: the practices, the results, the culture of the existing team. Despite this, there is no shortage of consultants that are willing to sell their pet approach to you without this knowledge.
The benefits of selecting a few small things to change need to be emphasized here. Culture shock is dramatically reduced, the time required to realize the benefits of the change is reduced, and these benefits will in turn demonstrate to the group that effective change does not have to be a painful experience (sowing the seeds for ongoing evolution). With the baselines that we have gathered, we can quantify the value of the effort, which will further motivate (and educate) those that would otherwise suggest that they can't afford to take the time for improvement.
Finally, we need to see these changes through to the point where they become part of a new status quo. This is the point where people recognize the value of these new practices, and will continue to work in this way, particularly when pressures rise on a project. How often have you been involved in a project where the deadline looms, and the team abandons all manner of practices in the name of meeting their goals? Peer reviews will be dropped, end-product testing will be reduced, even rigorous change management will be short-circuited. In situations like these, we haven't made it through the complete change cycle. We haven't improved until our new way of doing things is ruthlessly preserved, particularly when the pressure is on.
Overall, process improvement has everything to do with understanding the people involved and working to ensure their needs are met. Effective change management is all about helping people find their own path to a better way of doing things.
Take advantage of special member promotions, everyday discounts, quick access to saved content, and more! Join Today.