Building the Data Warehouse
Data Warehousing expert Laura Brown discusses how to build a successful Data Warehouse.
Building the Data Warehouse
By Laura Brown
What is Data Warehousing? You may have heard the term and wondered what's involved, or you may even be considering a Warehouse implementation for your organization.
Practice and Origins of Data Warehousing
The typical IS organization supports an environment that was built over time, with computer applications that were designed to support the day-to-day operation of the company's primary business. There are separate applications (or computer systems) for several categories of use. Some are centralized systems, used for back-office functions like Financial, Payroll, and General Ledger updates. Others run in localized environments, such as those for Field Office, Personnel, and Sales & Marketing initiatives. Still others are run in multiples—every branch running its own version. Order processing and customer information storage systems may fall into the category of multiples.
Often these applications are built on differing platforms by many different and separate development teams, without a "city plan" or architecture for fitting them all together. Each department just "does their own thing", without worrying about what other departments are doing.
To complicate matters further, over time we see a proliferation of needs for reporting extracts from various quarters, such as:
- Special Projects
- Process Reengineering Initiatives
- Year 2000 Projects
- Marketing Campaigns
- Financial, Pricing, & Profitability Studies
- Auditing Concerns
What ensues is an increasing competition for resources, including:
- Information
- Knowledgeable Technical and Subject Matter Experts
- Processing Time (Mips)
This occurs along with a decreasing window of opportunity for accessing those resources. Particularly, time in the overnight processing cycle is in demand. Often, there exists no more than a two-hour window of opportunity for extracts. In a real-time environment, or what's referred to as 7 by 24, the problem gets worse.
To avoid degrading the processing of day-to-day operations in a business, you must remove the information access from the critical path of information processing. Thus, the practice of making "Shadow Copies" of data repositories was born, and termed Data Warehousing. These copies typically are taken daily, weekly, or monthly, and in some instances much more frequently.
The Data Warehousing industry grew up around the real business need to create a "Corporate Information Architecture"—a "city plan," that is—to support informational processing for management decision support and integrated analysis. The organization and integration of data into this Architecture, as well as the creation of data access mechanisms, are the goals of many a DW Implementation Project.