"Jill Dyché does an expert job of describing the varied uses of data warehouses and data marts--not only in marketing, but across lines of business."
-- Martha Rogers, Ph.D.
Over the last ten years the strategic use of detailed data has changed the face of business. This change was made possible through the use of data warehouses, which are now widely accepted for their role in the delivery of decision-support and business-intelligence applications. Today's data warehouses are the critical hubs of such burgeoning strategic initiatives as e-commerce, knowledge management, database marketing, and customer relationship management. Given this, a working knowledge of the fundamentals of data warehousing is essential for today's executives, managers, and other professionals who must maximize the power of data warehousing in both existing business contexts and future strategic initiatives.
Written especially for these business professionals, e-Data: Turning Data into Information with Data Warehousing covers data warehousing and its surrounding technologies in a straightforward and engaging way, illustrating how companies are leveraging their data warehouses to serve a wide range of business needs. This book clearly lays out what business people should know about data warehouse implementation and the best techniques for evaluating and justifying new data warehouses and data marts. This book provides:
Inspirational real-world case studies and staff profiles appear throughout, showcasing data warehousing's "vanguards"--companies that have succeeded in achieving long-term financial and strategic benefits. Included are Bank of America, Charles Schwab & Co., Qantas Airways, GTE, Royal Bank of Canada, Sears, and Twentieth Century Fox.
e-Data provides invaluable information about data warehousing as a whole, its development and strategic value, the technologies that support it, and its effect on corporate decision making--information that will enable you to turn a gold mine of raw data into valuable information, position your company for market leadership, and enhance customer satisfaction.
In an industry that still relies on gut feel for many of its business decisions and on focus groups for market research, Justin Yaros might well sit on his laurels after throwing together some Web sites and deploying corporate e-mail. But Yaros, senior vice president and chief information officer for Twentieth Century Fox, has other things in mind. And decision support is one of them.
"There are a number of areas where DSS is beginning to take root," says Yaros. "Principally, it's in sales and marketing analysis. While we're not yet as advanced as other consumer packaged goods companies, we're learning a lot from our data."
Log on to Fox.com and you'll see why business intelligence is critical. The company markets merchandise ranging from X-Files mouse pads to videos of Dr. Doolittle to The Simpsons refrigerator magnets. Who buys this stuff, anyway? Plenty of people, it turns out. And Fox's ability to track these purchases can help the company better understand its supply chain, as well as construct profitable licensing and merchandising agreements.
But don't assume that an entertainment organization is nothing more than a high-gloss consumer packaged goods firm focused on product movement. In this business, the money's in the theatrical area, and so is the data warehouse.
Fox uses an Oracle data mart with DSS Agent to analyze box office results together with marketing expenditures relative to specific films. "On a Monday morning," says Yaros, "we can look at a movie's box office performance that weekend. We can examine our marketing expenditures, not just for the film as a whole, but for specific DMAs [demographic marketing areas] and even for individual theaters." Although in its early stages, this type of analysis can help Fox evaluate its marketing expenses for a given film as well as assist the company in determining its distribution strategies, potentially dictating which regions or theaters can sell the most tickets.
This analysis strategy will help Fox establish optimal distribution channels and enable it to develop relationships with high-volume theaters, perhaps inducing them to hold over a film that's doing well. Moreover, using its data warehouse, Fox can present the theater owner with the data to back up its claims. It's a new spin on CRM, and one that promises improved revenues.
Yaros and his team are also considering data-mining technology, especially predictive models. Yaros hints at the ability to predict the performance of certain films or genres prior to their release, potentially saving the company millions. "There are people here who have been in the business for so long that they can feel how a film will perform," he says. "Our data warehouse has to be as or more reliable than their intuition."
Data warehousing and the accompanying analytics will ultimately allow Fox to invest its marketing dollars more wisely, as well as schedule release dates and test out distribution scenarios well before the fact. And there's little doubt that the technology will also help the company forecast just how many Homer Simpson fridge magnets to have on hand.
You may not have heard of Allsport, one of the world's leading sports photography providers. But you've probably heard of its parent company, Getty Images (yes, that Getty). Allsport, based in London and Santa Monica, California, distributes sports images to a high-profile client base. From advertising agencies to magazines to shoe companies to the NBA, Allsport is the acknowledged source for high-quality sports images.
In the early 1990s, the company's legacy accounting and reporting system began running out of steam, and information provision was woefully inadequate. For an organization with a 30% annual growth rate, business data was critical and flexibility key.
Rather than continuing with its proprietary system, Allsport opted for the clean-slate approach. "Management was willing to basically start over," says Greg Walker, Allsport's CEO. "We identified the key legacy data and business needs, and basically just began again."
Building a new system from scratch was a leap of faith, and a risky one. But it allowed Allsport to furnish a solid infrastructure to support continued double-digit growth while responding to the evolving needs of its business users. Sales staff needed data about which types of images were selling and which clients were requesting them. Executives were lobbying for flexible sales and revenue reports, and not just by specific clients or dates. The company had no way of tracking photographers' commissions in real time. Allsport needed customer information so that it could target specific markets, while increasing its understanding of how and where its products--sports photographs--were actually being used.
"With our old system, we would have never been able to get the variety of information we needed," says Matt Schoen, Allsport's vice president of information systems. With the authority to start anew and the wherewithal to design a more nimble system, Schoen and his team elected to go with Web-based decision support.
"We didn't want to go with a client/server solution because of the platform issues involved," Schoen says. "For one thing, we were opening so many new offices, we didn't want to have to worry about different desktop technologies. We have some die-hard Mac users here, including executives. We weren't about to change what they were comfortable using." Allsport selected an HP-Oracle data mart on Windows NT with a Netscape browser at the front end.
The gamble has paid off. Allsport has now delivered Web-based decision support reporting to a cross-section of users from London to Santa Monica to Sydney. The browser front end has reduced training time and provided a common "look and feel" across the company, which has been able to open additional offices without having to increase support staff. Moreover, the resulting reports have become both more elaborate and more useful.
For one thing, the company can now profile the images its customers are buying, from the type of sport most in demand to various regional trends. Account executives can discover that, for example, images of extreme sports seem to be selling well among European magazine publishers, and they can even drill down to specific photographic criteria such as wide-angle live-action shots. In addition to their bread-and-butter profit and loss reports, managers can request daily sales breakdowns and revenues via a variety of dimensions, from photographer to sport code. "With this information, we can now aggressively attack specific markets," claims Walker.
Understanding who is buying what also lets Allsport enhance its "Concepts" line, photographs the company actually stages itself. Understanding sales trends can pinpoint emerging sports, allowing the company to plan photo shoots according to anticipated demand and thereby supplement its library with the optimal set of images.
Allsport is gradually barcoding its stock photographs so that it can track each of its 6 million physical and half-million digital images. Eventually, it will offer its extensive image library over the Web.
Indeed, Schoen's vision is for a complete Web-enabled order-processing and fulfillment environment, made even more efficient by the elimination of the sales and research efforts that are now involved in order provisioning. This will not only cut costs but also maintain the customer knowledge so integral to the company's market leadership. "We've seen the industry actually modify its deadline structure because of our ability to deliver over the Web," he claims.
While Schoen and Walker are thrilled with the business improvements engendered by decision support, they go one step further, implying that their innovative technology strategy may have actually been a factor in the company's purchase by Getty in 1998. Whatever the reason, Allsport and Getty are now teammates fortified by customer knowledge and sprinting into the millennium.
From an imposing office tower in the heart of downtown San Francisco, Christopher Kelly considers the last three years at Bank of America. That his title is "senior vice president of affluent and consumer database marketing" is a testimonial to the bank's belief in the potential magnitude of database marketing's business impact. That his organization has already saved the bank millions of dollars is a fact not lost on its new parent company, NationsBank.
Kelly manages a group of marketing analysts responsible for poring over Bank of America's two-terabyte customer database. The varied analysis methods and advanced technologies his department employs are proof that Kelly's group is as much a strategy center as it is an analysis organization.
"Our job is to execute marketing strategies that create relevance with our customers," he explains. With 30 million households resulting in 300 million banking transactions monthly, Kelly's staff wisely combines a state-of-the-art data warehouse with a variety of analysis techniques in order to understand customer behavior better.
Among a variety of analysis strategies is the segmentation Bank of America's massive customer database, the results of which are leveraged by the bank's retail division. Typical customer segments include the following:
The resulting business activities involve everything from reallocating marketing and sales resources around segments to designing unique, segment-based customer communication strategies. The bank also evaluates product development and pricing against certain segments. "We use a 'Rubik's cube' approach to segmentation," says Kelly. "We slice and dice data in different ways to identify homogeneous customer niches."
Chris Kelly is proudest of his group's accomplishments in CRM (see figure 3-2). The bank is using CRM to identify customer behavior "triggers," such as what prompts customers to purchase new products or close accounts. Life-stage marketing is the exemplar of the group's analysis skills. "We're tracking certain customer life events," Kelly explains, "in order to know what a customer might buy next, and to be able to offer such a product in a proactive way."
The intent of Kelly's organization is not only to provide retail banking staff with the data and findings they need in order to come up with new marketing programs, but to ensure that the findings ultimately result in valuable business actions. To this end, the bank regularly updates its data warehouse with the results of its marketing programs.
What's singular about Chris Kelly and his team of marketing analysts is that they aren't just hacking away at data in order to arrive at "neat to know" findings that are never deployed beyond the desk of the knowledge worker--an all too common phenomenon in advanced analysis organizations. "The idea is to create a perpetual marketing process," he claims. "We certainly have the customer and behavioral trigger data to do just that."
Bank of America's recent acquisition by NationsBank has involved predictable reorganizations and departmental mergers. Nevertheless, Kelly's group has remained largely intact. They will continue to mine the two banks' combined customer base for interesting and actionable marketing knowledge. Says Kelly, "We're trying to create a unique experience for the customer, and with it, a unique value proposition: one that only BofA can offer."
About the Author.
The Book and Its Purpose.
You the Reader.
Part I: Getting the Value.
Part II: Getting the Technology.
Part III: Getting Ready.
A Case Study Sneak Preview.
I. GETTING THE VALUE.1. What Is a Data Warehouse Anyway?
The Data Warehouse Defined.
Data Warehousing, Decision Support, and Business Intelligence.
The Data-Warehousing Bandwagon and Why Everyone Jumped on It.
Some Trite Data-Warehousing Aphorisms.
Venus and Mars: How IT and Businesspeople Communicate.
Some Other Buzzwords and What They Mean.
Some Lingering Questions.2. Decision Support from the Bottom Up.
The Evolution of Decision Support.
Standard Query: The Workhorse of DSS.
Multidimensional Analysis: The Power of Slice 'n' Dice.
Modeling and Segmentation: Analysis for Knowledge Workers.
Knowledge Discovery: The Power of the Unknown.
Some Real-Life Examples.
Modeling and Segmentation.
Wherefore Data Mining?
Data Warehousing in the Real World.
What It Takes to Get to the Top.3. Data Warehouses and Database Marketing.
Customer Relationship Management.
Individual Customer Analysis.
Case Study: Bank of America.
A Word about CRM Technology.
Popular Database-Marketing Initiatives and What They Mean.
Sales Analysis and Forecasting.
Market Basket Analysis.
Customer Retention and Churn Analysis.
Customer Value Measurement.
Sales Contract Analysis.
Database Marketing Lessons Learned.
Some Lingering Questions.4. Data Warehousing by Industry.
Uses of Data Warehousing in Retail.
Market Basket Analysis.
In-Store Product Placement.
Product Movement and the Supply Chain.
The Good News and Bad News in Retailing.
Case Study: Hallmark.
Uses of Data Warehousing in Financial Services.
The Good News and Bad News in Financial Services.
Case Study: Royal Bank of Canada.
U.S. Local Service Carriers.
U.S. Long-Distance Carriers.
International Long-Distance Carriers.
Uses of Data Warehousing in Telecommunications.
The Good News and Bad News in Telecommunications.
Case Study: GTE.
Travel Packaging and Pricing.
The Good News and Bad News in Transportation.
Case Study: Qantas.
The Good News and Bad News in Government.
Case Study: State of Michigan.
Uses of Data Warehousing in Health Care.
The Good News and Bad News in Health Care.
Case Study: Aetna U.S. Healthcare, U.S. Quality Algorithms.
Uses of Data Warehousing in Insurance.
The Good News and Bad News in the Insurance Industry.
Case Study: California State Automobile Association.
Case Study: Twentieth Century Fox.
Some Lingering Questions.
II. GETTING THE TECHNOLOGY.5. The Underlying Technologies: A Primer.
Data Warehouse Architecture.
The Operational Data Store.
Two-Tier Versus n-Tier.
Databases and What They're Good For.
Disseminating the Information: Application Software.
Graphical User Interfaces.
A Word about the Web.
Development Definitions and Differentiators.
Data Modeling and Design Tools.
Data Extraction and Loading Tools.
Management and Administration.
Putting It All Together.
Some Lingering Questions.6. What Managers Should Know about Implementation.
What You Should Know about Data Warehouse Methodologies.
Evaluating a Methodology.
The Data Warehouse Implementation Process.
The Steps in Data Structure and Management.
The Steps in Application Development.
Who Should Be Doing What?
Development Job Roles and Responsibilities.
Consultants Versus Full-Time Staff.
The Lost Fine Art of Skill Delineation.
Good and Evil Square Off:A Tale of Two Project Plans.
Executive Involvement on the Project.
Profile: Hank Steermann of Sears, Roebuck and Co..
Some Lingering Questions.7. Value or Vapor? Finding the Right Vendors.
The Hardware Vendors.
Five Questions to Ask Your Hardware Vendor.
The Database Vendors.
Five Questions to Ask Your Database Vendor.
The Application Vendors.
Five Questions to Ask Your Application Tool Vendor.
Data-Mining Tools: A Breed Apart.
Ten Questions to Ask Your Data-Mining Vendor.
The Big Guys.
The Little Guys.
A Word about the Analysts.
A Word about the Vendors.
Five Questions Your Consultant Should Ask You.
The RFP Process.
The Components of a Good RFP.
A Sample Table of Contents.
Some Lingering Questions.
III. GETTING READY.8. Data Warehousing's Business Value Proposition.
Return on Investment.
Hard ROI: The Tangible Benefits.
Soft ROI: The Intangible Benefits.
Budgeting for the Data Warehouse.
Obtaining Funding — But Not Too Much!
Data Warehouse Operations Planning.
Developing an Operating Plan.
Are You Ready for a Data Warehouse? A Quiz.
Data Warehouse Readiness Score.
Some Lingering Questions.9. The Perils and Pitfalls.
The New Top 10 Data-Warehousing Pitfalls.
Pitfall #1: The Data Warehouse as Panacea Syndrome.
Pitfall #2: They Talked to End-Users--But the Wrong Ones!
Pitfall #3: Too Much Time Spent on Research, Alienating Constituents.
Pitfall #4: Bogging a Good Project Down by Creating Metadata.
Pitfall #5: Being Sidetracked by "Neat to Know" Analysis.
Pitfall #6: Adopting Decision Support Without Supporting Decisions.
Pitfall #7: Greediness on the Part of Development Organizations.
Pitfall #8: Lack of "Internal PR".
Pitfall #9: Failing to Acknowledge That DSS Applications Are Finite.
Pitfall #10: Overemphasizing Development and Ignoring Deployment.
Thinking of Outsourcing?
Data Warehousing's Dirty Little Secrets.
The Politics of Data Warehousing.
The Top 10 Signs of Data Warehouse Sabotage.
The Vanguards of Data Warehousing.
Case Study: Charles Schwab & Co., Inc..10. What to Do Now.
If You Need a Data Warehouse.
Establish Up-Front Success Metrics.
Research External Staff.
Prepare Your Environment.
Classify Your Stakeholders.
Ramp Up Support Capabilities.
Profile: Philippe Klee, Qantas Airways.
Look Outside Your Box.
Solicit a Request for Information.
If You Already Have a Data Warehouse.
Establish a Formal Postmortem Process.
Inventory Existing Applications.
Spring for an Audit.
Improve Customer-Facing Business Processes.
Establish a Closed-Loop Process.
Go Web, Young Man!
Case Study: Allsport.
Consider Branching Out Vertically.
Consider Branching Out Horizontally.
If You Have a Data Mart or Marketing Analysis System.
Share Your Toys.
Migrate to Enterprisewide.
An Insider's Crystal Ball.
Enterprise Resource Planning.
Extending the Data Warehouse to External Vendors.
Customized Web Portals.
The Whole Truth.Appendix: Haven't Had Enough? Suggested Reading.
E-commerce. Knowledge management. CRM. ERP. Smart cards. Data mining. It's true that in the vast realm of technology, the term data warehousing has recently ceded ground to some whiz-bang buzzwords. Even with data warehouse adoption rates steadily increasing by 30 percent a year, the Web and its patois have drowned out discussions of even more advanced technological developments, rendering state-of-the-art technical breakthroughs a second-page story.
At first, I was a bit depressed by all the Web hype. Inasmuch as data delivery was critical to the enterprise, you still needed to store that data someplace, Internet or no. With all the hullaballoo about Y2K and Web portals, had data warehousing simply faded away?
Recent customer experiences quickly shook me awake. Not only have data warehouses not faded away, they've assumed center stage again. While certain terminology might ebb and flow--data warehouses are now synonymous with "decision support" and "business intelligence" and naturally symbiotic with all things Web--the data warehouse is in fact the hub in the wheel when it comes to many companies' most important strategic initiatives.
Attend any conference these days, whether focused on industry, marketing, or technology, and there's bound to be a presentation on customer loyalty programs, retention, or customer relationship management (CRM). Sometimes supply chains and even business process reengineering still rear their heads. The point here is that regardless of what the business initiative is, the data warehouse will likely play a central role in its execution by making key data available to a cross-section of the business.
In this book, the term e-data refers to data that has been intelligently modeled, cleansed, and consolidated into a data warehouse so that it's meaningful and useable by business people. The fact is, e-data is more important than ever. Whether you have a data warehouse, a data mart, or a decision support application (chapter 1 defines the differences), or are considering stepping up your CRM, e-commerce, or target marketing programs, having clean, consolidated information is no longer a nice add-on; it's a necessity. This book explains how e-data and a data warehouse can solve a wide range of business problems and provides real-world examples from a diverse set of industries, countries, and companies.
The Book and Its Purpose
This is not one of those books.
Rather, it's a book for those of us who aren't interested in lofty technical dissertations but whose work nevertheless touches corporate data in some way. Those who are keen on getting the information the data warehouse can deliver, are hiring staff who will use it, and are interfacing with their technical colleagues in making it all work.
We need to understand what data warehouse technology really does, in common terms, and why it's right for our companies. While we're not interested in implementing it, we'd like to differentiate the well-worn buzzwords. We'd appreciate some implementation scenarios as they pertain to data warehouses and why they're used, and checklists of success criteria. We want to know how e-data can aid in marketing, assist our companies in winning customers--sometimes for the second time--and help us eat our competitors' collective lunch. We want trenchant examples and are hungry for tips from those who've realized the vision. We want to understand what data warehouses will do for us, as well as what they will not.
In short, this is a book for the rest of us.
You the Reader
Executives and managers will glean a lot of practical information from this book, both in terms of how to tell whether a data warehouse is the right solution for the business problem at hand and how to determine whether an existing data warehouse is living up to its value proposition.
Businesspeople whose thirst for new information alone is often justification for a data warehouse will be interested in how data warehouses are being used in various industry and marketing capacities. The book introduces concepts and terms that managers and end-users alike can learn in order to speak the same language as their information technology (IT) colleagues, ensuring that their business requirements are understood and addressed, and offers several checklists against which to gauge data warehouse readiness.
Marketing experts, including product managers, merchandisers, and strategic planners, can read about key corporate initiatives that directly leverage data warehouses.
IT managers will find this book a practical tool in confirming the requirements for successful data warehouse delivery. The book includes a variety of metrics and success factors with which technology management can measure its efforts or bolster its preparatory activities.
Consultants, too, will find this book useful; they can employ the various checklists and matrices in order to evaluate staff and review delivery success metrics, as well as to prepare their practices for what's on the horizon. Project managers, both administrative and technical, can translate the information for their own implementation strategies, supplementing both their project plans and the methodologies that drive them.
Finally, technical practitioners and implementation team members can use this book for review; in the process they may discover a thing or two about how other companies are implementing their data warehouses and as a result refine existing development activities.
Certain readers might surmise that the book focuses on the technology platform, the data warehouse hardware itself. This approach would be like writing a book about television and discussing the electronic circuitry of the television set rather than the actual shows. While the book does explain the underlying technology involved in data warehousing by way of framing the picture, it nevertheless focuses more on "what's inside" the data warehouse, not to mention the prevalent audience. In short, the book is about what data warehouses do for a business.
The book is divided into three parts, which categorize the chapters into high-level areas. Below is a thumbnail sketch of the book's organization and contents.
Part I: Getting the Value
Chapter 2, "Decision Support from the Bottom Up," presents an e-data analysis taxonomy for the data warehouse. It describes the four main types of business intelligence that call for data and offers some examples on their usage.
Chapter 3, "Data Warehouses and Database Marketing," outlines both the popular and the emerging database marketing applications that focus on customers while leveraging data, explaining their origins and business benefits.
Chapter 4, "Data Warehousing by Industry," covers the gamut of industry sectors and what they're doing with e-data and data warehouses, using case studies to illustrate various usage scenarios from real-world companies.
Part II: Getting the Technology
Chapter 6, "What Managers Should Know About Implementation," exposes the often arcane world of data warehouse development, the methodologies it employs, and some of the well-worn staffing mistakes that get development managers into trouble. In addition, it offers some tactical hiring guidelines for you to use when conducting your next round of interviews.
Chapter 7, "Value or Vapor? Finding the Right Vendors," presents metrics for assessing the data warehouse solutions that fit best with your organization and its unique needs, including hardware, database, application, and consulting evaluation criteria.
Part III: Getting Ready
Chapter 9, "The Perils and Pitfalls," presents several sets of metrics in order to outline why some customer data warehouses succeed while others fail. Not content with offering the negatives, this chapter concludes with a list of what the "vanguards of data warehousing"--those companies attributing improvements of several orders of magnitude to their data warehouses--have in common when it comes to successful decision support delivery.
Chapter 10, "What to Do Now," provides some advice from the trenches on how to continue your data warehousing journey, whether you're a seasoned traveler or are just breaking in your boots.
The appendix of supplementary reading material provides a guide to other recent works for those readers who want to learn more about either the business or the technology side of e-data.
A Case Study Sneak Preview
Most technology books abstain from discussions of specific vendors for valid and practical reasons. However, because of this book's heavy emphasis on real-world examples, specific vendors pop up here and there, particularly in chapter 4, where most of the case studies mention the company's chosen data warehouse platform.
New companies and technologies are emerging every day, and I apologize to those vendors that may have slipped through the cracks. The technologies discussed in the book are those of particular interest to the primary audience, that is, businesspeople, and thus a mere nod of the head to the many worthy data warehouse software companies that target--and are of greater interest to--the IT side of the house.
From time to time the vendor discussion will be updated and supplemented. For an updated discussion of emerging data warehouse topics, keep an eye on this book's corresponding Web site: http://www.baseline-consulting.com/e-data.