Practical Approach to Large-Scale Agile Development, A: How HP Transformed LaserJet FutureSmart Firmware

Product Author Bios

Gary Gruver is formerly the Director of Engineering for HP's LaserJet Core Firmware Lab, and he worked at HP for 22 years. He is currently VP of Release, QA, and Operations at macys.com. Any major initiative needs a true business sponsor—someone who has truly caught the vision of agile, and who can make the business and financial decisions necessary to get huge breakthroughs to happen. Gary has also been able to bring a “manage to metrics” approach that rallies everyone to common measurable objectives without requiring lots of meeting and coordination overhead. Of course, his most critical role is buying lunch during particularly busy sprints for anyone working weekends to finish off key features. His favorite hobbies are cycling and skiing with family (he’s married with two daughters).

Mike Young is the program manager directing day-to-day efforts across our many distributed teams at HP's LaserJet Core Firmware Lab. Mike has been involved in development of HP LaserJet Printers for 18 years, and he previously designed satellite control systems for Hughes Aircraft Company. He also is one of the strongest advocates of agile approaches and helped get the organization started down this path before anyone really knew we were doing agile. His hobbies are family (he’s married, with two daughters and two sons) and playing racquetball. In agile, we’ve found that a program manager should spend most of his/her time watching the metrics and quietly coordinating behind-the-scenes to cater to the bottleneck. In our sprint checkpoints, we tend to minimize slideware and maximize problem solving and demos of new user stories.

Pat Fulghum is architect of the HP LaserJet FutureSmart firmware and its development team's agile toolset. Pat’s been at HP for 24 years. He found out during the past few years that his favorite escape is scuba diving in Maui with his family (he is married and has a son and a daughter). A large-scale agile initiative requires a central architect who can help maintain architectural integrity amid many pressures to do otherwise (which keeps the system enabled for the future) and who has the vision for making sure the architecture supports both firmware development and qualification. Pat still loves to get in and dig deep to solve vexing technical challenges. He also loves to find developer productivity improvements (build time, triage time) and has been the passion behind our “10x productivity improvement” vision.
Practical Approach to Large-Scale Agile Development    

Today, even the largest development organizations are turning to agile methodologies, seeking major productivity and quality improvements. However, large-scale agile development is difficult, and publicly available case studies have been scarce. Now, three agile pioneers at Hewlett-Packard present a candid, start-to-finish insider’s look at how they’ve succeeded with agile in one of the company’s most mission-critical software environments: firmware for HP LaserJet printers.

This book tells the story of an extraordinary experiment and journey. Could agile principles be applied to re-architect an enormous legacy code base? Could agile enable both timely delivery and ongoing innovation? Could it really be applied to 400+ developers distributed across four states, three continents, and four business units? Could it go beyond delivering incremental gains, to meet the stretch goal of 10x developer productivity improvements?

It could, and it did—but getting there was not easy.

Writing for both managers and technologists, the authors candidly discuss both their successes and failures, presenting actionable lessons for other development organizations, as well as approaches that have proven themselves repeatedly in HP’s challenging environment. They not only illuminate the potential benefits of agile in large-scale development, they also systematically show how these benefits can actually be achieved.

Coverage includes:
• Tightly linking agile methods and enterprise architecture with business objectives
• Focusing agile practices on your worst development pain points to get the most bang for your buck
• Abandoning classic agile methods that don’t work at the largest scale
• Employing agile methods to establish a new architecture
• Using metrics as a “conversation starter” around agile process improvements
• Leveraging continuous integration and quality systems to reduce costs, accelerate schedules, and automate the delivery pipeline
• Taming the planning beast with “light-touch” agile planning and lightweight long-range forecasting
• Implementing effective project management and ensuring accountability in large agile projects
• Managing tradeoffs associated with key decisions about organizational structure
• Overcoming U.S./India cultural differences that can complicate offshore development
• Selecting tools to support quantum leaps in productivity in your organization
• Using change management disciplines to support greater enterprise agility

Online Sample Chapter

Tuning Agile to Your Business Objectives

Table of Contents

Foreword by Jim Highsmith, ThoughtWorks     xiii
Preface     xv
Chapter 1  Agile Principles versus Practices     1
The Principles of the Agile Manifesto     2
Our Take on Agile/Lean Principles     3
A Quick Tutorial: Agile versus Waterfall     6
Summary     8
Chapter 2  Tuning Agile to Your Business Objectives     9
Background: HP FutureSmart Firmware Case Study     10
Cost and Cycle-Time Drivers Prior to HP FutureSmart Firmware     11
Value Proposition of Re-Architecting the HP FutureSmart Firmware and Processes     13
Establish Development Objectives from the Business Analysis     15
Summary     16
Chapter 3  Aligning Architecture with Business Objectives     17
Challenges with Existing Architecture     18
Architecting for the Business: Dynamic Variability and Forward Compatibility     19
Keeping an Architecture Current and Sustainable     22
Summary     25
Chapter 4  How to Establish a New Architecture Using Agile Concepts     27
Re-Architecting Iteratively     28
Making Progress     28
The Thin-Slice Model     30
Creating Cultural Shifts Through Architectural Demos     31
Summary     33
Chapter 5  The Real Secret to Success in Large-Scale Agile     35
Change for People’s Sake     36
Metrics Are a Conversation Starter     38
Iterative Model of Agile Management     39
   Mini-Milestone Objectives     40
   Cascading Objectives to Track Progress     41
   Conversations     42
   Learning     43
   Agile Adjustments     44
Summary     44
Chapter 6  Continuous Integration and Quality Systems     45
Reducing Build Resources and Build Time: Continuous Integration     46
Achieving High Quality with CI: Automated Multilevel Testing     55
   L0 Testing     57
   L1 Testing     58
   L2 Testing     58
   L3 Testing     59
   L4 Testing     60
   Continuous Improvement of the Deployment Pipeline     60
Productivity Results of Our Automated Delivery Pipeline     61
Special Considerations for Enterprise Software Systems     63
Summary     65
Chapter 7  Taming the Planning Beast     67
Predict by Ballparking and Trend Watching     69
   Ballpark Prediction: R&D Early Response to High-Level Initiatives     70
   Trend Watching: Quick Response to All Feature Requestors (Where They’re Likely to Land)     70
Clear Prioritization     73
Just-in-Time User Story Definition     76
   Invest in System Engineering     77
   Put Marketing in Charge of a Unified 1-N List     80
   Involve the Technology Architects     81
   Use Project Managers as “Feature Leads”     81
   Reuse Requirements and Test Tags for Scalability     82
   Commit by Delivering, Not by Estimating     83
Convincing the Business: Agile Planning Is Okay     86
Summary     88
Chapter 8  Unique Challenges of Estimating Large Innovations     91
Waterfall Approach and Challenges     92
Agile Approach     92
Challenging Situations with the Agile Approach: Large Architectural Efforts     95
Change Management and Integrating with the Business     98
Summary     100
Chapter 9  Our Take on Project Management for Large-Scale Agile     101
Oversight and Priority: Program Managers     102
Accountability: Section Managers     103
Robustness and Scalability: Architects     104
Putting It All Together     104
Summary     105
Chapter 10  Organizational Approach: Managing to Disadvantages     107
Test Ownership Organization     108
Component versus Feature Organization     111
Traditionally Managed Project Teams versus Self-Managed Scrum Teams     114
Summary     116
Chapter 11  Effective Agile Development across U.S. and Indian Cultures     117
Lesson 1: Permission to Ask     118
Lesson 2: Ensure Time to Explore     119
Lesson 3: Have Small Wins First     119
Lesson 4: Exploit the Time Difference     120
Lesson 5: Take Time to Train–Always     121
Lesson 6: Remember a Team Is about People     121
Organizing for the Highest Leverage of Offshore Teams     122
Summary     125
Chapter 12  The Right Tools: Quantum Leaps in Productivity     127
Common Development Environment     128
Simulation and Emulation Environment for Automated Testing     129
Test Architecture for Scalability: Common Test Framework (CTF)     131
The Most Important Part of Test Automation: Virtual Machine Provisioning System (VMPS)     133
Real-Time Metrics and Tracking     136
Integrated Toolset     137
Cool Toys Worth Investing In     138
Summary     139
Chapter 13  Real-World Agile Results: HP FutureSmart Firmware     141
Resources Moved from Overhead to Innovation     142
R&D and Developer Productivity     144
Improvement in Current Product Support     146
Summary     147
Chapter 14  Change Management in Moving Toward Enterprise Agility     149
Impacts on Other R&D Groups and System Qualification     150
Impacts on Product Program Teams     151
Impacts on Non-R&D Product Generation Activities/Teams     154
Where to Draw Boundaries with Coordinating Organizational Agility     155
Change Management of the HP FutureSmart Firmware Transformation     156
Summary     158
Chapter 15  Differences in Our Perspective on Scaling Agile     159
A Difference in Perspective     160
Focusing on Agility Rather Than Team Operations     161
Changing the Deployment Pipeline     162
Embracing the Uncertainty of Agile     163
Enterprisewide Tracking and Incremental Improvements     164
Summary     164
Chapter 16  Taking the First Step     167
Figuring Out First Steps     168
What’s Next for FutureSmart?     169
Determining Your First Steps     171
Summary     172
Appendix A  Twelve Principles of Agile Software     173
Bibliography     175
Index     177