Table of Contents
- Microsoft SQL Server Defined
- Microsoft SQL Server Features
- Microsoft SQL Server Administration
- Microsoft SQL Server Programming
- Performance Tuning
- Practical Applications
- Professional Development
- Application Architecture Assessments
- BI Explained
- Developing a Data Dictionary
- BI Security
- Gathering BI Requirements
- Source System Extracts and Transforms
- ETL Mechanisms
- Business Intelligence Landscapes
- Business Intelligence Layouts and the Build or Buy Decision
- A Single Version of the Truth
- The Operational Data Store (ODS)
- Data Marts – Combining and Transforming Data
- Designing Data Elements
- The Enterprise Data Warehouse — Aggregations and the Star Schema
- On-Line Analytical Processing (OLAP)
- Data Mining
- Key Performance Indicators
- BI Presentation - Client Tools
- BI Presentation - Portals
- Implementing ETL - Introduction to SQL Server 2005 Integration Services
- Building a Business Intelligence Solution, Part 1
- Building a Business Intelligence Solution, Part 2
- Building a Business Intelligence Solution, Part 3
- Tips and Troubleshooting
- Additional Resources
A Single Version of the Truth
Last updated Mar 28, 2003.
In previous entries in this series I've explained how important it is to involve the business in developing your Business Intelligence (BI) landscape. In the next few tutorials I'll cover more of the technical and physical aspects of the process, but we need to talk about a concept you will run into when you in your design process: "A single version of the truth". This phrased is most often used by either the business side of your organization in meetings or by a vendor of a BI system.
Let's first examine the intent of this statement and then dig a little further into the reality you'll face when you try to implement it.
Many businesses decide they need an analytical system (BI) when they realize how much data they are storing, and begin to understand that they may not be getting the most information out of it that they can. As I've mentioned, as a technical professional it's important that you steer this conversation towards these general stages:
- Solid data strategy
- Solid reporting strategy
- Solid analytical strategy
I was a consultant for a few years, and in almost every case a business that wanted (or thought they wanted) an analytical system did not have the first two strategies mapped out or implemented properly. In the data strategy, you're thinking about what you store, where you store it, how it is accessed, and how it is protected. You must start here because if the base data are incorrect, any numeric calculations made on them will be even more incorrect. This is often called a "drift error", and you'll see it quite often in statistics. Taking the average of two incorrect numbers is even further from the truth than the incorrect numbers themselves, and this error is made even worse if the consumer does not know that the original numbers are wrong.
The data strategy includes archiving, access and protection – all of these play into the analytical system, since the BI landscape derives its information from the base data. If the data isn't there, or can be tampered with, once again the subsequent decisions will be incomplete or incorrect.
The second strategy of reporting is equally important. About 40-50% of the engagements I've been on could be solved with solid reporting, rather than implementing a BI solution. In most cases management (and even the whole organization) wasn't even aware of what reports were available, who was using them, or how. A good reporting strategy, in fact, can form the basis of your BI system. Knowing which questions can be answered by a simple report and which need an analytical approach is one of the first steps in your design.
This brings us to the analytical system and the desire on the part of management for that single version of the truth. By that they normally mean a system of record – one that shows the exact amounts of something or at what time of the month things happen and so on. They mean that when we say we have 300 widgets in stock we really have 300 physical widgets on a shelf somewhere.
This kind of information is highly desirable. You and I do this all the time when we make our grocery lists: we see if we have any pasta on the shelf, and if we don't, we buy more. That knowledge makes our decision for us. You can imagine that in a complex organization of thousands of people spread around the world, with regulations, laws, accounting standards and so on that you would really want to be able to know absolute information about inventory, headcount, income and expenses and so on.
But as desirable as this information might be, it isn't always possible. While you and I can look in our cabinet to see if we have any pasta, imagine trying to count the boxes of pasta in all of the cabinets in your entire city, or even just in your neighborhood. What if the boxes are already opened? Do you count those? What if they only have a very small amount? Do you combine all of the mostly-empty boxes in all the cabinets to make a new one? And what is pasta, anyway? Do the cans of pre-prepared dishes with pasta in them count?
You can see even in this simple sample that it is less intuitive than you might think to count up a single item of inventory. Now what if you are a manufacturing plant, and you count the parts you build things with. What do you count? The completed units, the individual parts, or both? How do you account for which stage of the manufacturing process they are in when they are neither a single part nor a completed unit?
And this doesn't even deal with things like accounting, where you have currency differences among countries and so on. Do you take the spot price, the daily average, an agreed-on "leveling" currency or some other method?
And even if you answered these questions definitively, you should resist the urge to do so. The time will come when that definition won't work anymore. You'll absorb another company, the laws or even the governments will change.
This is difficult to explain to a management team, but it can be done, especially if you use examples. Since the "pasta" example might be deemed too silly for a high-level meeting (or perhaps not), you can lead them through an exercise of discovering this for themselves. It all depends, of course, on the type of organization you're in, but once you've convinced them to consider the other two strategies (data and reporting) you can ask them to begin to define terms like "currency", "inventory" and "headcount" very specifically. Take notes, and then read that back to them, prodding them with questions like: "so headcount is the number of people employed by the firm on a given date. Does that include contractors and part-time employees? In all situations?" You'll normally get a slight pause while they think that through, and then come to the understanding that "perhaps not" is the right response.
You can't just leave them there, however. Once they understand what the issue is, you can make two broad suggestions for dealing with the issue.
The first method you can use is to break apart the single concepts into multiple views, reports, or slices of data. You can say: We have a single version of the truth, but we have a lot of qualifications of that truth. for instance, you might have a view that shows the number of full-time employees, another that shows the number of contractors, and so on. You might even want to put those as "levels" (which we'll study later) and then roll them up to a single report of "headcount". The analyst can then drill down into those levels to see the qualifications of that "truth".
This becomes a bit ridiculous after a while, however. Suppose you're asked to attend a dinner at a friend's house. You reply: "Yes, but only if it doesn't rain, the dinner is after 6:00, my daughter doesn't have any homework, I feel like it and my wife will go. Then I'll come." Your friend would have no idea if you're coming or not!
Normally the better solution is to accept that there may be multiple versions of the truth. You'll design your information so that it fits the best definition of the common understanding that the organization has of the concept, and then create more views as warranted. You're not looking to eliminate ambiguity; you're trying to minimize it.
So what does this mean? Let's take the example of current cash-on-hand. The major problem with this number in a global organization is multiple currencies. Depending on how you convert those currencies (since they fluctuate against each other all the time) you can end up with different numbers. So what you do is pick the conversion rate (average, spot, moving average, etc.) that the users of the particular view or report care about the most. You may have one view for accounting, another for management, another for budgeting and so on. Each of these might report a slightly different number, which suits the question at hand.
Which brings us all the way around to the start again – you have to do two things before you embark on any kind of analytic system: get those requirements nailed down carefully and force the business to define their terms. There is simply no way you can do this for them. If they will do this, you can deliver a great system that they can trust – with or without a "single version of the truth."
InformIT Tutorials and Sample Chapters – A Single Version of the Truth
This book seems to be an interesting take on BI: Internet-Enabled Business Intelligence, by William A. Giovinazzo.
Online Resources – A Single Version of the Truth
There's a great reference for all things BI here.