- Content at Asset
- What Is Content?
- Measuring Return on Content
- Case in Point: HMO Customer Service
This chapter is from the book
This chapter is from the book
This chapter is from the book
Measuring Return on Content
In the volumes of literature on knowledge management (KM), the study of return on investment has yielded less than compelling results. For some, the issue is no longer relevant; while others have tried to attack it from myriad perspectives—both quantitative and qualitative. As we discussed in the Preface, the connection between content management and knowledge management is obvious. As a result, the ROI problems of knowledge management have been carried over to content management, and to understand them we need to look at why the ROI on knowledge management has been so difficult to nail down. With that understanding, we will try to cut through this dead-end discussion to arrive at a more useful view of how effective management of content (as a form of knowledge) can yield positive qualitative and quantitative returns.
Health maintenance organizations (HMOs) are struggling with very thin profit margins (if they are making any profits at all) because of the rising costs of healthcare. These costs are largely out of the HMOs' control, so they must look elsewhere in the organization for cost containment. Administrators struggle mightily with information management. Every area, from sales to contracts to service, is intensely dependent on accurate information. More often than not, this information should be shared across the departments, but it isn't.
For example, 200,000 letters communicating a policy change have gone out in mass mailings to HMO's insured members, but member services knows nothing of the change, not to mention the letter. Confused (and irate) members call looking for clarification, while member services representatives remain in the dark. Table 1.1 details the total costs of communicating the policy change.
Table 1.1 Communicating Policy Change Cost
|
Number of letters |
200,000 |
|
Production and mailing cost of one letter |
$1.50 |
|
Total Cost of Mailing |
$300,000 |
|
Number of calls to call center |
10,000 |
|
Duration of call |
15 minutes (.25 hours) |
|
Hourly cost of member service representative |
$22 |
|
Total Cost of Response |
$55,000 |
|
TOTAL |
$355,000 |
A simple process change in which member services is notified three days before a mailing goes out can help the rep prepare for the inevitable spike in phone calls. Better yet, preparing a Frequently Asked Questions document ahead of time can make it easy to answer members' questions, further reducing the additional $55,000 of lost productivity.
Knowledge Management and Measurement: A Literature Review
The origins and development of knowledge management as a business topic have hindered a fruitful discussion of how knowledge can yield bottom line results that can be validated through accurate measurement. A look at the seminal books on this subject helps us understand why this is the case.
The first problem that knowledge management faced was its presentation, inevitably, as a new management philosophy akin to TQM, re-engineering, learning organizations, and the like. This was because the key books on the topic were preoccupied with convincing readers that knowledge needed to be treated as an asset that should be managed just as land, labor, and capital were. While the management techniques and "best practices" for managing these familiar assets were mature, those for knowledge were introduced as something wholly different and autonomous.
Thomas H. Davenport and Laurence Prusak wrote about "knowledge markets" in Working Knowledge: How Organizations Manage What They Know; Ikujiro Nonaka and Hirotaka Takeuchi elaborated on the "knowledge spiral" in The Knowledge-Creating Company. Along with Thomas A. Stewart's Intellectual Capital: The New Wealth of Organizations, these books set the parameters for how knowledge management would be subsumed into the vocabulary of management in the United States. As a result, they profoundly affected our ability to understand and measure its ROI.
The language of knowledge management has been remarkably consistent. The topic came into vogue after 1995 with the publication of The Knowledge-Creating Company. "Intellectual Capital" appeared in 1997 and was quickly followed in 1998 by "Working Knowledge."1 In these books and articles, the central argument was that the collective intelligence (i.e., knowledge) of an organization was a new kind of asset that must be managed for the organization's strategic benefit. The focus was therefore on describing the special management techniques necessary to harness this new asset to create "new wealth" and "innovation."
With this focus, the authors had no choice but to start with some discussion of the nature of knowledge. The first chapter of Working Knowledge, What Do We Talk about When We Talk about Knowledge? spends a great deal of time separating data, information, and knowledge. The Knowledge-Creating Company begins with a long discussion of the competing epistemologies of Plato and Aristotle, moves through the Enlightenment, and ends up with a brief discussion of Heidegger's notion of Dasein. At issue was grounding the discussion in knowledge as something unique and special. Only by establishing this could discussion of the proper management techniques for this most unique asset begin.
Because these books spent so much time defining the qualities of knowledge, business strategy took a back seat, creating confusion as the topic matured. For instance, Davenport and Prusak, after defining knowledge, make a fairly simple statement: "Knowledge can and should be evaluated by the decisions or actions to which it leads." Here was a step in a necessary direction: the need to attach knowledge management practices to strategic business objectives. Knowledge is valuable only insofar as it leads to decisions and actions deemed valuable and productive to the business. The authors continue with their central argument that knowledge is a unique asset that needs special management techniques: "We've observed and analyzed over a hundred attempts to manage knowledge in organizations. To the managers of most of them we've posed the question, 'How do you make the distinction between data, information, and knowledge?' " (p. 6). However, if their interest was truly with the value of knowledge as an asset, we would have expected a different question: "How do you know what knowledge is of most value to the organization?"
Stewart makes a similar move in Intellectual Capital: "There's a vital lesson here: Knowledge assets, like money or equipment, exist and are worth cultivating only in the context of strategy" (p. 71; emphasis in original). But this point is immediately dropped and no systematic argument is made for showing how to link these assets (i.e., intellectual capital) to business strategy. In fact, his next chapter, Recognizing Tacit Knowledge, abruptly returns to the central concern, which is to define the essential qualities of knowledge.
We aren't criticizing these books for not pursuing a link between knowledge management and strategy. Their authors were making the necessary first steps in introducing knowledge as a strategic asset, and they rightfully spent their time discussing the special qualities of intellectual capital and how to manage it. Still in their attempts to add force to their arguments, the authors were unnecessarily confused and confusing in the new assets' positioning. They elevated knowledge in such a way that it could be seen as more important than strategy. Stewart claims, for instance, that "managing intellectual capital should be business's first priority" and that "Knowledge has become the preeminent economic resource—more important than raw material; more important, often, than money" (p. 6).
Davenport and Prusak add to the confusion. They begin by talking about their involvement with managers who "were admitting that they didn't have any effective methods and approaches for managing and understanding how to better use information themselves." They then turn their research to "why it isn't managed well, what managing information actually means, and what kinds of specific improvements our clients could make in how they obtained and used it" (p. xii). These are very reasonable problems to address when the focus is to understand the unique nature of knowledge and information as assets. However, in citing economist Sidney Winter, Davenport and Prusak state, "If 'knowing how to do things' defines what a firm is, then knowledge actually is the company in an important sense" (p. xiii; emphasis in original).
So, the discussion of knowledge management as it was introduced simultaneously offered promise and confusion. The promise was that knowledge and information are intimately related to strategy and that, with new technology for better managing them, companies could improve their top and bottom lines. The confusion came with statements about knowledge as somehow more important than strategy. As a result, knowledge management could be thought of as an independent undertaking within a business—as if the management of knowledge could proceed unattached from strategic concerns.2
At that point, measuring ROI became nearly impossible. Typical financial metrics seemed inappropriate—return on capital employed (ROCE) being the classic example. Proving that a new piece of capital equipment could yield cost reductions through improved productivity was easy, showing that automating accounts receivable could reduce errors and shorten "days sales outstanding" (DSO) was a no-brainer. However, the benefits of knowledge management were touted as competitive advantage, employee learning and retention, and other so-called intangibles. In fact, in many cases effects such as collaboration and communication were offered as valuable ends in and of themselves rather than as the means to other more strategic objectives. But how do you measure the benefit of improved collaboration? Email systems allow more prolific communication, but how do you calculate their ROI? In fact, conventional measures would put email systems at a complete loss with no positive financial return.
Content Services and Business Performance Measurement
Fortunately, the art and science of business performance measurement have matured over the last few years. The work in this area—balanced scorecards (BSC) and Six Sigma specifically—has helped us refocus on the link between strategy, performance, measurement, and management. These new disciplines thus help us intelligently resituate the ROI discussions of knowledge management (and consequently content services).
Beyond ROI
First, we need to distinguish between performance measurement and ROI. On the one hand, performance measurement is a discipline that focuses on identifying the metrics to precisely gauge the execution of strategy. On the other hand, ROI makes no necessary connection between the investment being measured and the direction of the business; ROI studies tend to be done in isolation. The key question is always whether or not the specific investment has returned more financial benefit to the organization than its cost. A positive return alone is assumed to be beneficial.
The essential challenges in determining ROI are twofold:
The boundaries around the investment itself. Can we completely account for all costs associated with the investment—human, opportunity, capital, and so forth?
The boundaries around the benefits of the investment. Can we completely understand all of the benefits—financial and nonfinancial—associated with the investment?
If you can adequately answer these challenges, you can determine whether or not your gain is more than your investment. However, this kind of analysis doesn't take into account the overall benefit to the business. For instance, a new investment in upgrading a piece of equipment would allow for an increase in the capacity of a factory to turn out product, but what if the sales force's compensation plan didn't provide adequate incentives to move the additional volume? The ROI analysis would show a positive return—fewer inputs to produce greater outputs—unless you took into account the additional inventory carrying costs necessary to handle the additional output that wasn't being sold.
ROI analyses tend to let management off the hook. In the example we just cited, what if this investment were made without knowing whether or not the market could absorb the additional capacity? A narrow view would indicate a positive impact; a broader view would show a possible negative impact as inventory stagnated in the warehouse. Neither view—narrow or broad—would reveal whether or not the investment contributed to an overall plan.
Recent work in performance measurement has pointed out the shortcomings of financial indicators like ROI and ROCE as measures of business performance. It isn't that they are irrelevant but rather that they need to be part of an overall strategy. Instead of being isolated numbers in a report, they should become key performance indicators (KPIs) that tell management how well the business is doing in executing its chosen strategy to achieve its objectives. Translating this into the example above, the ROI on the capital investment makes sense insofar as the company has determined that they are going to actively grow the market share of the product. They have set up a sales and marketing plan, which the investment in the equipment directly supports; so the ROI will be measured appropriately against two KPIs:
Manufacturing capacity. Did the investment allow for meeting the capacity requirements set forth in the plan?
Market share. Was the desired market share increase achieved?
The first KPI lets you know if the manufacturing manager hit his objectives. The second tells you whether or not investment in the new equipment materially benefited the business.
Return on Investment versus Business Performance
A DVD player manufacturer's customer support center has implemented a new call-tracking system for $1.5 million, and within the first year the result has been a 40 percent reduction in talk time. Another investment of $750,000 will complete the job with an expected drop in callbacks of about 75 percent. The problem is that customer satisfaction has dropped precipitously over the same time. Studies show that customers feel rushed through the support process and are showing up in the stores to get their problems fixed. This is preventing sales associates from helping new customers.
Measuring Content Services Performance
As we discussed in the Preface, content services is a form of knowledge management. Consequently, it is trapped in the shortcomings of the ROI on knowledge management discussions we have been analyzing. Let's rehabilitate this discussion in the context of content services at this point.
The challenge of determining the return on content services is understanding how it affects the organization's financial KPIs. Therefore, we need to establish a cause-and-effect relationship between content services and financial performance, and the only way to do this is to first know the financial goals of the organization and the operational plan and KPIs that support them. Then and only then can we begin to make sense of how content services can contribute to financial performance.
There are many ways to create business strategy, as the ever-growing literature on this topic illustrates. Our starting point is the balanced scorecard as developed originally by Robert S. Kaplan and David P. Norton in The Balanced Scorecard: Translating Strategy into Action. We like the BSC because it provides a clear method for translating goals into metrics at all levels of the business. Kaplan and Norton are only tangentially concerned with how content can impact performance; but we see in their approach a way to, first, link knowledge management and content services initiatives to strategic initiatives and, second, measure that performance appropriately.
Kaplan and Norton articulate the different levels of strategy that allow you to set up reliable KPIs:
Financial. What does the company hope to achieve in the given timeframe—revenue growth, profitability, cash flow neutrality?
Customers. What segment or segments of the target customer base will be addressed to achieve financial goals?
Internal business processes. What business processes are key to the organization in achieving financial and customer objectives?
Innovation and learning. What employee skills and technical infrastructure need to be in place for the strategy to be successful?
We are going to use these levels to show how content services can be a key contributor to business strategy and thus yield a measurable and meaningful benefit. We will do this by showing how and where better access to relevant, accurate, and useful content positively affects the performance measures linked to financial goals.
Figure 1.2 shows this cause-and-effect relationship. In it we have groupedthe financial and customer perspectives as strategic and the Internal process and innovation and learning perspectives as operational , which highlights the different roles that content services play depending upon which perspective is relevant. At the strategic level, content services performance is measured largely by the ability to deliver necessary information (e.g., metrics, reports, competitive intelligence, customer satisfaction surveys) to decision makers. It is also measured by the ability to communicate strategy and metrics to those who need to know (e.g., shareholders, employees, partners).
)
Figure 1.2 Measuring the Performance of Content Services
At the operational level, content services is responsible for lower-level KPIs as well as for timely, accurate content that promotes the following:
Improved productivity, timeliness, flexibility, and quality of key processes
Improved innovation and decision making at the front lines
Alignment of employee skills with strategic direction
Each of these is measurable, depending on the specific goals of the organization.