Home > Articles > Networking > Storage

This chapter is from the book

Utilization and Yield

A fundamental piece of the storage TCO equation is utilization and its direct correlation to what can be referred to as the storage yield. If one assumes that the average company used at best 50 percent of their storage assets between 1999 and 2002 (which is itself a conservative number), then, based on the worldwide revenues shown in Table 1-2, we can estimate that over $35 billion dollars in storage assets went unutilized during that time.

NOTE

In this section, I borrow two terms from different fields—the COPQ (from Total Quality Management and Six-Sigma) and yield (from manufacturing and agriculture)—and I apply those terms to the discussion of storage utilization.

Storage utilization is the most important storage management issue today: Poor utilization wastes millions of dollars a year in unused storage assets.

Understanding utilization is crucial for the introduction of ROI, Net Present Value (NPV), and TCO in Chapter 3, "Building a Value Case Using Financial Metrics." This material is required to build the financial models with which the business case for storage networks can be justified.

A close analysis of storage yield and the COPQ demonstrates how increased utilization helps lower the overall storage TCO.

The Cost of Poor Quality and the Storage Problem

The Cost of Poor Quality, in terms of quality and yield management, typically refers to the costs associated with poor or undesirable performance of a product over the course of its economic usefulness.26

A high COPQ implies higher manufacturing, operations, and labor costs, and consequently, lower revenues. Couching the value of an IT solution in terms of quality management, the COPQ can be said to be the dollar value of how a product, service, or solution performs relative to its expectations. In terms of financial analysis, this figure equates to a negative ROI.

Just as the buildup of IT capacity and subsequent downturn was the outcome of macroeconomic events, the move to storage networks is part of many corporations' efforts to raise their storage yield over time and lower the COPQ (and the TCO) for their storage infrastructure.

Storage Yield

In manufacturing operations, the term yield refers to the ratio of good output to gross output.27 In storage operations as in manufacturing, the yield is never be 100 percent as there is always be some waste. The goal of a storage vision is to increase not only storage yields, which can be measured in dollars or percent of labor, but also to increase operational yields (or "good output") as much as possible. Ultimately, a storage vision built on a storage utility model helps increase a company's storage yield, the amount of storage capacity allocated and then used efficiently to create and sustain business value.

A tiered storage infrastructure is required to fully increase storage yield and gain true economies of scale. In Table 1-5, each tier has a different capability model and different direct and indirect costs associated with it. The goal is for the COPQ to be as insignificant as possible (shown here as a percentage of $1,000,000 in revenue), and ideally for the accompanying tiers to be appropriately matched to the level of business impact or business revenue of the associated applications. A typical tiered storage infrastructure might look something like this:

  • Tier One—Mirrored, redundant storage devices with local and remote replication

  • Tier Two—RAID-protected, non-redundant storage devices with multiple paths

  • Tier Three—Non-protected, non-redundant, near-line storage devices (for example, SATA drives used as a tape replacement)

Table 1-6 Cost of Poor Quality as a Percentage of $1,000,000 of Revenue for 1000 GB

Storage Type

Cost per MB

GB

Total Cost

Allocated

Utilized

Tier 1

$0.05

1000

$51,200

80%

75%

Tier 2

$0.03

1000

$30,720

80%

60%

Tier 3

$0.01

1000

$10,240

80%

90%

Storage Type

Allocated Yield

Utilized Yield

Realized Yield

Cost of Poor Quality

COPQ % of Revenue

Tier 1

$40,960

$30,720

60%

$20,480

2.05%

Tier 2

$24,576

$14,746

48%

$15,974

1.60%

Tier 3

$8192

$7373

72%

$2867

0.29%


As seen, a low storage yield has a corresponding high COPQ and indicates an overall higher total cost of storage ownership. A more complete discussion of tiered storage solutions (and Information Lifecycle Management) is presented in Chapter 5.

NOTE

The difference between allocated and utilized storage is discussed in the section titled "Utilization."

Obstacles Inherent in DAS

As the predominant storage architecture to date in terms of terabytes deployed, DAS has served the storage needs for millions of environments around the globe. Small Computer Systems Interface (SCSI), DAS is a standard, reliable method of presenting disk to hosts. DAS also presents many challenges to the end user including failover and distance limitations, as well as the increased expense associated with poor utilization.

Failover Limitations

Although some DAS environments are Fibre Channel, large storage environments in open systems datacenters have historically been direct-attached SCSI. SCSI is a mainstream technology that has worked well and has been widely available since the early 1980s. SCSI provided the necessary throughput and was robust enough to get the job done. One disadvantage, however, has always been the inability of the UNIX operating system and most databases to tolerate disruptions in SCSI signals, thus limiting the capability to failover from one path to another without impact to the host. In addition, logical unit number (LUN) assignments are typically loaded into the UNIX kernel when the system is booted up, requiring allocation or de-allocation of storage from the host to be planned during an outage window. If the storage unit in question is shared between different clients with mismatched service-level agreements and different maintenance windows, then negotiating an outage window quickly becomes a hopelessly Sisyphean task.

Distance Limitations

Another significant factor hampering the flexibility of SCSI DAS is that SCSI is limited in its capability to transfer data over significant distances. High Voltage Differential (HVD) SCSI can carry data only up to 25 meters without the aid of SCSI extenders. This limitation presents difficulties for applications requiring long-distance transfer, whether for the purposes of disaster recovery planning, application latency, or just for the more physical logistics of datacenter planning.

Expense

Aside from the technical limitations of DAS, the primary drawback of DAS is, without a doubt, its expense. Ultimately, the storage frames themselves constitute a single point of failure, and to build redundancy into direct-attached systems, it is often necessary to mirror the entire frame, thereby doubling the capital costs of implementation and increasing the management overhead (and datacenter space) required to support the environment.

The expense of DAS also stems from poor utilization rates. A closer look at the two primary types of storage utilization further illustrates the nature of the cost savings inherent in networked storage solutions.

Utilization

When considering the impact of managing storage in general and the financial disadvantages of DAS in particular, the primary variable to monitor is storage utilization. Poor utilization leads to a decreased storage yield and a high COPQ, whereby the storage capital asset purchased to provide a service, fails to perform at an optimal level.

Storage utilization has been a marketing hot-button since Fibre Channel SANs began to gain momentum, and as such, utilization is now laden with many different meanings, all of which are often (and unfortunately) used interchangeably. To prevent further confusion, I prefer to use Jon William Toigo's terminology of efficiencies. Toigo clearly delineates between "allocation efficiency" (broadly referred to as "utilization") and "utilization efficiency," which typically reflects the effects of storage usage policies.28

Allocation Efficiency

Due to the physical constraints of the solution, DAS environments are intrinsically susceptible to low "allocation efficiency" rates that cost firms money in terms of unallocated or wasted storage. Let us look at one example of the financial impact of poor allocation efficiency.

Imagine a disk storage system (containing 96 73-GB disk drives) with six four-port SCSI (or Fibre Channel) adapters capable of supporting up to 24 single-path host connections. This system is capable of providing approximately 7008 GB of raw storage, or 3504 GB mirrored. Under most circumstances, hosts have at least two paths to disk, so this particular environment can support a maximum of twelve hosts. In a typical scenario, shown in Figure 1-5, this frame hosts the storage for a small server farm of six clustered hosts (12 nodes).

Figure 5Figure 1-5 Sample DAS Configuration

If each cluster hosts six similar applications using 500 GB each (an allocation efficiency rate of 85 percent), almost 500 GB remains unallocated due to the frame's port limitations. With a purchase price of $0.10 per MB, or $100 per GB, there is a loss of $105,120.00 associated with the unutilized disk on that frame.

NOTE

Keep in mind that as this frame is formatted for mirroring, the value of the unallocated storage is the cost of the total non-mirrored storage. In other words, the 500 GB of unallocated storage is still 1 TB raw, which must be valued at its purchase price. Also note that use the $0.10 per MB for quick math. The average purchase price of the disk might be significantly lower.

This loss can be considered the COPQ and reflects the costs of additional storage required to provide the expected capacity. An allocation efficiency rate of 85 percent for a DAS environment, however, is significantly higher than the normal average. In June, 2001, a joint study published by McKinsey & Company and Merrill Lynch's Technology Group, titled "The Storage Report—Customer Perspectives & Industry Evolution," estimated the average utilization rate for DAS environments to be 50 percent.29

Fred Moore of Horison Information Strategies has an even more dismal view of allocation efficiency. According to Moore, surveys of clients across various industries indicate allocation efficiencies of 30–40 percent for UNIX and Linux environments and even less for Windows environments, which Moore says frequently see allocation efficiency rates as low as 20 percent.30

Using the same environment shown in Figure 1-5 as an example, if the allocation efficiency is only 50 percent, then the loss widens significantly to $350,400, or half the purchase price of the frame. Figure 1-6 shows the costs associated with poor utilization in this environment.

Figure 6Figure 1-6 Utilization Rate and Associated Costs—Cash Basis

Most firms depreciate the cost of storage over the course of its useful life (assuming the storage is purchased and not leased), so the actual COPQ might vary according to depreciation schedules.

Given the rapid progress of technological advancement, in most cases, depreciation is carried out over three years. If the straight-line method of depreciation is used over a period of three years, the asset value or purchase price of the frame is divided by three with the assumption that one-third of its usefulness is consumed each year. The impact of the loss, or the COPQ, is then spread across the span of the economic usefulness of the asset. In other words, one third of the COPQ affects the firm's bottom line each year.

Low utilization does not increase or decrease the estimated life of the hardware, nor does this loss change the asset's value in accounting terms. Low utilization does, however, decrease the storage yield of the asset and increases the COPQ, which, in turn, increases the overall TCO. Regardless of the method of depreciation used, poor utilization detracts from the firm's bottom line.

Whether or not the storage units themselves are depreciated, the net effects of poor allocation efficiency are similar: Low allocation efficiency increases the rate of frequency of additional storage purchases. A real life parallel is buying a full tank of gas and being able to use only half of the purchased fuel. As long as you need to drive the car, you will need to purchase more fuel. If more fuel is not consumed, you will be forced to stop at the gas station more often.

Similarly, as long as the firm operates, it needs to purchase storage. The idea that a firm can delay purchasing storage indefinitely by constantly increasing the utilization rate is, to put it bluntly, misinformed. The long-term key to financial success in terms of storage management is optimizing storage usage to minimize the frequency and magnitude of storage purchases. A high allocation efficiency rate helps decrease the size and number of storage purchases, as does a high utilization efficiency rate.

NOTE

Capacity on-demand programs are alternative procurement strategies aimed at alleviating the frequency and number of storage purchases. Although these "pay-as-you-go" methods are quite successful at easing the purchase and planning process, they do little to address the rate of consumption or poor utilization found in many environments.

Utilization Efficiency

There might be environments in which the allocation efficiency is at a desirable rate, but the allocated storage is misused, unusable, abandoned, or even hoarded. This is what Toigo refers to as poor utilization efficiency, whereby the storage itself might be highly allocated, but poorly utilized. In fact, in many open systems environments in which the storage capacity is efficiently allocated, utilization efficiency might be extremely poor, with many applications needlessly consuming data that is rarely, if ever, used.

To resolve these types of issues, a targeted program or project aimed at reclaiming allocated—but lost or poorly used storage—is needed. A project of this magnitude requires a significant time investment and an energetic executive sponsor who is capable of ensuring the proper alignment of goals and initiatives. A storage reclamation project also requires extensive use of a combination of off-the-shelf storage resource management (SRM) software and home-grown scripts dedicated to tracking storage consumption.

Despite the many obstacles that are known factors in implementing DAS, the majority of disk units sold in the last five years are still connected to hosts in a direct-attached fashion. Most companies—even early adopters of storage networking technologies—are still in the implementation phase of building SANs, and therefore have at least a partial mix of SAN and DAS technologies in the datacenter.

Although it is difficult to determine the exact percentage of DAS and SAN storage currently installed world-wide, estimates based on the sales of disk and Fibre Channel gear indicate that nearly three quarters of all disk storage units installed still utilize the direct-attached architecture. As shown in Table 1-1, DAS storage units made up nearly 70 percent of all storage sales in 2003 (with NAS and SAN storage together comprising approximately 30 percent). As these figures indicate, there is still a long way to go before the majority of storage environments currently deployed are networked storage solutions.

In addition to the recently installed DAS, a mountain of DAS that was purchased during the market upswing and it still carries a sizable net book value. As shown in Table 1-1, nearly one million DAS units were shipped between 2001 and 2003, indicating significant depreciation expense for customers when considering the corresponding low utilization rate (and the high COPQ) for DAS.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020