Home > Blogs > Communication Chasm

Communication Chasm

I read an article from Cutter's Agile E-mail Advisor this week that sparked my interest. In it, Jens Coldewey concluded that "rather than being an opposition to software engineering, agile development is an alternative draft of software engineering". In the context of Jens' article and some presentations earlier in the week at the PNSQC, here was my response.

It is interesting to read your article, and others like it that come from the agile community these days. I agree with the principles of agile, that it can be considered an alternative draft of software engineering. I was at the PNSQC this week, and listened to Ron Jeffries and Chet Hendrickson talk about Quality Dynamics of Agile Software Development. To summarize, a lot of very strong software engineering concepts, sprinkled with a number of suggestions that these are ideas that are new and that make agile unique and different from 'those other approaches' (there is often the suggestion that if the project is bad, it can't be agile). I strongly disagreed with that that implication, and fear that it overpowered their more important message that personal skills matter. It is not that agile is new, and your Advisor agrees with that by quoting challenges from 40 years ago that still exist today, it is rather, as you suggest, a re-packaging of strong ideas that have been around a while.

The parallel I see between software engineering from 40 years ago and agile today is the huge gulf that remains between the state of knowledge and the state of practice. Fred Brooks said that "The gap between the best software engineering practice and the average practice is very wide - perhaps wider than in any other engineering discipline." Compare that to something Peter Coffee said a couple of years ago: "Auto mechanics, it has been observed, invest more of their time and money than the average IT pro puts into maintaining professional competence, and it shows."

I agree with both of these quotes, regardless of when they were originally spoken.

The challenge I see both with software engineering and agile is not the collection of ideas, but the difficulties in reasonably disseminating these ideas into the masses. In both cases, what we often see is superficial implementation of the specific practices, rather than a deep understanding of the principles behind these practices. The educational institutions must bear some of the blame, but much of the responsibility for this deeper dissemination of these ideas lies with the evangelists.

Back in July 2004, around the time that Jim Highsmith released Agile Project Management, I wrote an article for the Cutter IT Journal titled Beyond the Hype of a New Approach. What I saw then (and still see today, as agile has, if nothing else, set records for remaining high on the hype curve) is that these great ideas have been challenged in making it across the chasm, in truly making inroads beyond the early adopters. This remains true for agile today. The evangelists need to step up and be responsible for fostering a deeper knowledge in the community, rather than the pithy comments that agile is better, suggesting that there were no project successes before a bunch of people got together in Utah a few years back. I do not see this happening quickly enough.

My experience with a wide range of companies is twofold. When working with the large, established companies in the private sector, or large public sector IT shops, agile is either not on the radar whatsoever, or is just generating interest in some early-adopter groups, indicating that the movement is only now approaching this chasm. This is contrasted with my experience with small startups, the early adopters. In most of these shops where individuals suggest they are using agile techniques, the state of practice is no better than before. Indeed in several companies, the business value being generated by the technology teams has diminished significantly. There are more instances of this in my experience than there are of success using agile techniques. I do not market myself as an Agilist (which I believe exposes me to a broader audience than those that strongly promote agile practices), but I end up correcting far too many agile misconceptions in the field. The practice rarely meshes with the theory, particularly at the principle level.

The strong ideas behind agility: strong visibility into progress, wide open communication, early delivery of value, adaptation to current knowledge, personal skills matter, are being lost in the cacophony of things that the developers want to hear: index cards over specifications, refactoring over good design. "Don't worry, we'll get you something" becomes a mantra that developers like, but business owners dread. The core ideas are being stated, but they are not being heard. This has been true for 40 years or more.

If we can understand and address this communication gap (by taking ownership for closing this gap), then there is the possibility that Software Engineering (or agility in its current guise), may actually make a difference in our lifetimes.

If we continue to simply rant from the pulpit, though, the important message will still miss the masses, and agility will fail to cross that same chasm that software engineering has failed to cross, even after 40 years.

Become an InformIT Member

Take advantage of special member promotions, everyday discounts, quick access to saved content, and more! Join Today.