Risk-Averse Organizational Attitudes
For a second, think about your organization in terms of its attitude toward failure. What really happens if you fail? Whether we're talking about a failed project, being part of a failed initiative, or being tied to an idea that didn't work, how dire are the consequences of that failure?
Some organizations simply don't tolerate failure. Despite having "embrace failure" wording in the mission statement or in some HR newsletter, the reality there is shown by what happens to someone who tries something new and fails. Is the employee fired? Demoted? Disrespected and generally shunned?
How an organization handles failure is usually a good indicator of how willing it is to take on risk. "Risky" inherently means having a reasonably high chance of failure. If the organization punishes those who take a risk and fail, risk-taking tends not to occur.
Going Agile means taking a risk. It's the same for any action whose long-term benefit comes at the perception of a short-term cost. Behavioral economics includes a concept called loss aversion: "Losses loom larger than corresponding gains." This concept plays out in software development activities when the gains from code maintenance in the long term are not considered worthwhile because of the necessary short-term investment in writing maintainable code.
To counter this risk aversion in your organization, remember that Agile techniques are designed to reduce risk. Waterfall projects only provide the illusion of reduced risk during the horizon of the project, when you can claim progress against a set of stale requirements. If you use a legitimate measuring stickthat is, the ability of software to meet the requirements at the end of the projectyou really take a far greater risk with a waterfall methodology, because only in a waterfall system do you ask a business analyst to predict requirements a year in advance from the time when those requirements will be implemented.
Put another way, predictability of software delivery should not be valued above solving real business problems. You must be able to write code now that can adapt to change in the future.