Upgrade Your IT Using Agile IT Organization Design
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Efforts to improve enterprise IT application development (strategic IT) performance usually focus on one or more of the following:
- Adoption of the latest tools and better engineering techniques
- Adoption of lighter-weight processes
- Individual capability development through training
Although these changes are necessary, they fall short of what is needed to improve IT responsiveness as a whole and make a visible difference to the business (see Figure 1). Without a visible difference, enterprise IT will continue to suffer a poor reputation. Despite IT's growing strategic relevance, IT leadership won't be invited to the business strategy table. What's more, sluggish IT puts business at risk. The digital wave is upon us, and the days are gone when a business could somehow make do with lackluster IT capability.
)
Figure 1 Engineering and process agility are necessary, but insufficient to improve overall IT responsiveness. (Image copyright Sriram Narayan. Used with permission.)
What's Missing?
The missing competence for IT is organizational agility—inter-team, inter-function, and inter-department agility, rather than just intra-team agility. Organizational barriers to agility are structural, operational, cultural, and political in nature. They manifest in the form of team structures, organizational habits of decision-making, and management and governance playbooks and controls.
A number of approaches to overcome these organizational barriers have already been proposed. Some of these approaches to "scaling Agile" (even ones that are quite popular) try to "color within the lines" of old-school IT governance and management, which is all about scaling using these techniques and attitudes:
- A regimen of activity-oriented metrics, targets, and incentives
- Standardized documentation, reports, and rolled-up red-amber-green dashboards
- Optimizing utilization of staff without worrying about how it affects end-to-end cycle time
- Obtaining funds for and executing IT project-by-project
- Fixating on delivering to plan
- Abdicating responsibility for delivering value to the business or to the product-management organization
The ethos of the above approach may be summarized as follows:
- Governing for predictability, without concern for value
- Organizing teams in order to maximize utilization of people and simplify staffing, without concern for end-to-end responsiveness
- Designing policies and reward programs that act as extrinsic rather than intrinsic motivators for better performance
Reworking IT for Digital Success
Chasing predictability and utilization is the opposite of what is needed for governing strategic IT. Strategic IT must deliver value; it cannot justify itself by achieving sustainable, good-enough velocity, or even by delivering to plan. It has to make a difference to business outcomes. To get there, strategic IT must set itself up to make a difference to business outcomes.
Here are three overarching principles you can use to set up strategic IT for success:
- Govern for value over predictability
- Organize for responsiveness over cost-efficiency
- Design for intrinsic motivation and unscripted collaboration
Notice that the first two principles are relative value statements. The objective on the right isn't unimportant, but it must be traded for the objective on the left when the two are in conflict.
Old-school IT governance may recoil at the relative deprioritization of predictability above. IT has long been obsessed with delivering to plan, and this obsession persists despite a miserable track record. The CHAOS Manifesto 2013 reports average cost and time overruns for sub-million-dollar projects since 2004. Average overruns have never dipped below 45% for cost and 70% for time, and larger projects fare worse! Such overruns might be overlooked if the delivered value ultimately made up for the overrun, but that hasn't been the case. The debacle at healthcare.gov (until a last-minute rescue effort) is an infamous case in point.
But what do these principles really mean in terms of actionable changes to IT? Let's take them one by one.
Principle 1: Govern for Value over Predictability
The upshot of the first principle is that we refocus governance from the pursuit of predictability to the pursuit of value (see Figure 2). We deemphasize conventional mechanisms that are used in the pursuit of predictability, in favor of mechanisms that allow the pursuit of value with low downside risk.
)
Figure 2 How governing for value works. (Image copyright Sriram Narayan. Used with permission.)
Make Smaller Bets
Funding for IT development work is usually secured on the back of large, upfront, speculative financial business cases. For example, such a business case may claim that spending $2.2 million over the next two years on developing a set of new features with a team of 10 people will result in extra revenue of $12 million from year 2 to year 6.
Unfortunately, there generally is no way of determining actual benefits realized in these situations, and getting this information three or four years down the line (after all the money has been spent) isn't very useful. In addition, these business cases are usually dressed up with unrealistic estimates and assumptions in order to win against other business cases competing for the same capital expenditure budget.
All in all, large upfront financial business cases aren't very Agile; they deliver feedback late in the day, whereas Agile focuses on early feedback. With fast and early feedback, we don't have to commit a lot of funds upfront; therefore, we can go ahead and experiment without elaborate business cases. If early feedback is negative, we can pivot or shut down the effort without wasting further resources.
Go Product-Centric
When the vehicle of IT funding is a large upfront financial business case, the vehicle of IT execution is the project. A project-centric outlook puts a premium on delivering to plan, regardless of value. By contrast, strategic IT takes a value-first attitude, like the focus in many leading-edge tech companies and new-generation independent software vendors.
These teams don't ramp up and ramp down, as is the case with projects; instead, they're stable and long-lived. They work out of a continuously evolving roadmap owned by a product manager or product owner. They practice continuous delivery—where every check-in/commit (or set of them) progresses through an automated build-and-deployment pipeline until it fails for some reason or results in a release candidate. They work in a continuous flow that is not check-pointed by release plans, or even sprints or iterations, although they may host regular showcases for stakeholders.
The project manager on these Agile-styled teams is more of a delivery manager (and maybe a people manager) who anticipates and resolves delivery bottlenecks and external dependencies, and who chips in with execution when possible. Teams, rather than projects, are funded against business objectives, rather than project plans, and the product manager/owner is accountable for the business outcomes (key performance indicators, or KPIs).
Principle 2: Organize for Responsiveness over Cost-Efficiency
When we structure teams and departments to maximize utilization of specialists such as developers, testers, data engineers, architects, and UXers, we end up organizing for cost-efficiency. An IT matrix organization is a common consequence of this approach. Each group of specialists reports to a VP/Head of that specialization on a long-term basis. For day-to-day project work, each specialist also reports to a project manager. On its face, the project team may appear to be a single cross-functional team. But in reality, the style of working may be influenced by specialist departments and their departmental KPIs.
Some organizations don't even pretend to have a single cross-functional team. Work is simply handed off from one specialist team to another. Handoffs between teams are usually much more expensive than those within a team. Expensive handoffs discourage piecemeal workflow and encourage batching. Rather than a single story flowing from user experience design to business analysis to development to testing, all the stories that comprise a feature are batched together for handoff. Increased batch size reduces cycle time and hurts responsiveness.
By contrast, organizing for responsiveness locates all critical specialist skills within the same reporting chain and team (see Figure 3). Try to minimize part-time assignment of specialists, even if it means they aren't fully utilized in their teams. Over time, these specialists might acquire adjacent skills (for example, an analyst may learn deployment skills, or a UXer may gain testing skills), which in turn will help them to discharge their specialist roles better. Organizing along lines of specialization—not specialization itself—is the problem.
)
Figure 3 How organizing for responsiveness looks. (Image copyright Sriram Narayan. Used with permission.)
Principle 3: Design for Intrinsic Motivation and Unscripted Collaboration
One of the core principles of Agile is to emphasize individuals and interactions over processes and tools. Agile at scale needs motivated individuals and unconstrained interactions at scale. In his book Drive: The Surprising Truth About What Motivates Us (Riverhead Books, 2011), Dan Pink establishes that we need an organizational climate of autonomy, mastery, and purpose for intrinsic workforce motivation at scale.
Unconstrained interactions refers to the sort of low-ceremony interactions that happen within a team. By contrast, inter-team interactions tend to require more orchestration—setting up meetings, keeping managers in the loop about inter-team agreements and commitments, and so on. For overall agility, inter-team collaboration needs to be less scripted—hence the term unscripted collaboration.
How do we design an organization for intrinsic motivation and unscripted collaboration? Focus on three primary areas: autonomy, mastery, and purpose.
Autonomy
Autonomy is a powerful motivator, but without mechanisms to ensure alignment with strategy, it can lead to silos. Set up teams with the right goals before granting autonomy. Outcome-oriented teams (such as a product team) make better candidates than activity-oriented teams (such as a UX team) for autonomy. To prevent runaway autonomy, we need counterbalancing measures to encourage alignment with strategy and accountability for decisions and results.
Mastery
Cross-functional teams are arguably less suited than specialist teams for mastery of a particular skill, but mastery can be encouraged by setting up communities of practice led by experts. On the other hand, narrow targets, KPIs, and incentives hinder mastery because they discourage performance in unmonitored areas. They can turn useful informational metrics into counterproductive, extrinsically motivational metrics, undermining intrinsic motivators such as mastery. Functional organization often requires scripted collaboration (meetings, release calendars, etc.), whereas cross-functional teams allow for greater unscripted collaboration and therefore better responsiveness.
Purpose
Teams work better when they can relate to a clear purpose. This purpose cannot be as broad and vague as the company vision or mission; it must be concrete and capable of being directly influenced by the team. For example, a product team can work toward greater sales, lower churn, or fewer support tickets. Within enterprise IT, an application development team can work toward increasing adoption of the application and greater effectiveness of its users. Teams organized around outcomes have a more meaningful purpose to work toward than teams organized around activities such as UX or testing.
Final Thoughts
Organizational agility can be improved by deliberate application of the three principles explained here. These principles are describe in greater, more actionable detail in my book Agile IT Organization Design: For Digital Transformation and Continuous Delivery.
References
Daniel H. Pink, Drive: The Surprising Truth About What Motivates Us. Riverhead Books, 2011.
Agile IT Organization Design: For Digital Transformation and Continuous Delivery. Addison-Wesley, 2015.
For more on scale-out management, see "Do you scale Agile vertically or horizontally?" Retrieved July 13, 2015.