Home > Articles > Software Development & Management

An HP UX-11i Performance Management Methodology

Optimizing the performance of a system can save your company a considerable amount of money. In this chapter, you'll see how to streamline your system for maximum efficiency.
This chapter is from the book

Although performance management and crisis management (including performance problem resolution) require different techniques and data collection, the same basic methodology can be used for both. Performance management uses the following steps:

  • Assessment

  • Measurement

  • Interpretation and Analysis

  • Identification of Bottlenecks

  • Tuning or Upgrading the System

The flow chart in Figure 3-1 summarizes the performance methodology that will be discussed in detail, step by step.

03fig01.jpgFigure 3-1. Performance Management Methodology

3.1 Assessment

Assessment often involves asking lots of questions, as the following example shows.

From Bob's Consulting Log— I spent the entire morning asking people a lot of questions about the application, the system configuration, and how users perceived the performance problem. At lunch, the person who hired me said he was surprised that I had not yet logged onto the system. I told him my approach is to ask questions first and collect system data second. Later that day, I did obtain some data from the system, but the most important clue to the nature of the problem came from interviewing the users.

These questions are necessary to help you understand your limits as a performance professional, as well as those things that you can change. The following items must be learned during the assessment phase:

  • System configuration

  • Application design

  • Performance expectations

  • Known peak periods

  • Changes in the system configuration or the application

  • Duration of the problem, if applicable

  • The options

  • The politics

3.1.1 System Configuration

System configuration includes both hardware and software configuration. The number of disk drives, how data are distributed on the disks, whether the disks are used with mounted file systems or used in raw mode, the file system parameters, and how the application makes use of them—all of these factors are examined during a system configuration assessment. Memory size, the amount of lockable memory, and the amount of swap space are scrutinized in assessing the virtual memory system configuration. You will need to know the processor type and the number in the system. Finally, the kernel (operating system) configuration must be understood, and the values of tunable kernel parameters should be identified. Knowing all these items in advance will make carrying out the various performance management functions much easier.

3.1.2 Application Design

Understanding the design of the application is equally important. It may not be possible for you as a performance specialist to thoroughly understand the design and workings of the application. Optimally, there should be someone with whom the application design can be discussed.

Such things as inter-process communication (IPC) methods, basic algorithms, how the application accesses the various disks, and whether the application is compute-or I/O-intensive are examples of the knowledge you will need. For instance, with relational databases, it is important to understand whether or not the Relational Database Management System (RDBMS) software supports placing a table and the index that points to it on different disk drives. Some RDBMS products support this and some do not; when this capability is present, it is an important tuning technique for improving database performance. Modern design techniques such as creating multi-threaded applications and the deployment of applications on multi-processor systems require new skills to analyze how the application works and its use of system resources.

You should expect consumer complaints and comments to be in terms of the application. However, measurements will be based upon the available metrics, which are mostly kernel-oriented. If you can't translate system measurements into application-specific terms, it will be difficult to explain in a meaningful way what the measurements indicate in terms of necessary changes in the system.

3.1.3 Performance Expectations

You will need to learn the performance expectations of the system's users. It is very important to know, in advance, the measurable criteria for satisfactory performance. This is the only way to know when performance tuning is successful and when it is time to monitor performance rather than actively attempt to improve it. Objective rather than subjective measures must be elicited. Being told that response time must be less than two seconds and is currently five seconds or more is much more useful than being told that performance is "lousy."

Understanding performance expectations is a complicated task. The perception of actual system performance and the definition of satisfactory performance will change depending upon one's perspective and role. Understanding expectations includes the need for eliciting success criteria. In another words, you need to know whether the users or owners of the application and system will agree with you that you have finished tuning the system and application.

3.1.3.1 Response Time

Users of the system typically talk about poor response time. They are mostly concerned with how long it takes to get a response from the system or application once the enter key is pressed. For instance, a user who is entering invoices into an accounts payable system expects to have data verification and data insertion completed within several seconds, at most. Engineers who use a computer-aided design application expect to see the image of the part being analyzed rotated in real-time. Otherwise, they get frustrated with what they consider to be poor performance. Response time is the most commonly used measure of system performance and is typically quoted as one of the following:

  • An average of a fractional second (sometimes called sub second) or seconds

  • A certain confidence level

  • A maximum response time

Typical confidence levels are 90%–98%. For example, one would say that the desirable average response time is two seconds, 95% of the time, with a maximum transaction response time of five seconds.

What is a good value for response time? While the automatic answer "It depends" is certainly true, it is useful to have a range of values for response time to use as a guideline. Users of a text editor or word processing package don't want to have to wait to see the characters echoed on the screen as they are typed. In this case, sub-second response time of approximately 250 milliseconds would be considered optimal. In a transaction processing environment one must understand the environment before developing good response time values. Some of the factors are as follows:

  • Transaction complexity

  • Number of users

  • Think time between transactions

  • Transaction types and ratios

Transaction Complexity

This deals with the amount of work which the system must perform in order to complete the transaction. If the transaction is defined as requiring three record lookups followed by an update, that transaction is much more complex than one that is a simple read. The complexity associated with typical Computer-Aided Design (CAD) applications is very high. Many CPU cycles and perhaps disk I/Os are necessary for simple interactive tasks, such as rotating a model of an object on the display.

Number of Users

The number of users influences the sizing of any system that is required to support a given workload and response time.

Think Time

As the think time between transactions increases, more work can be supported by the system in the idle periods between transactions. Heads down environments provide almost no think time between transactions. This is a typical data input environment. Conversely, customer service environments often provide very long think times, since most of time the customer service representative is speaking with the caller by telephone and casually accessing various databases as the telephone conversation proceeds.

Transaction Types and Ratios

It is necessary to look at the types of transactions and the ratios of the number of each transaction type to the total. Read-intensive applications can provide rapid response times to queries. Insert-and particularly update-intensive applications require more CPU cycles and often disk I/Os to complete the transaction. Table 3-1 gives guidelines for acceptable response times.

Table 3-1. Typical Application Response Times (based on the author's experience)

Transaction Type

Acceptable Response Time in Seconds

Interactive CAD applications

<1

Text editing or word processing

1/4

Read-intensive, low complexity

<1

Read-intensive, medium to high complexity

1–2

Update-intensive, low to medium complexity

5

Update-intensive, high complexity

5–15

Long think-time environments

2–3

Batch run

N/A

Users perceive performance as poor when update response time exceeds 5 seconds and when there is no preparation to be done by the user to get ready for the next transaction. Read performance must be no more than 1–2 seconds to keep users satisfied.

After installing the computer system and the application, some system administrators or performance managers have been known to create dummy workloads on the systems before letting any users access the applications or the system. The initial users perceive a certain response time following their inputs. As more users are added, the dummy workload is reduced, thus providing a constant response time to the users. This trick attempts to address another issue with the perception of actual performance. Users prefer consistent responsiveness rather than variable responsiveness. If someone decides to run some resource-intensive batch jobs while interactive users are on the system, interactive performance will typically degrade. End-of-month processing will usually consume a very large amount of system resources, making it necessary to either keep the interactive users off the system while it is being run, or to run it in off-hours.

Users can tolerate and accept consistently poor response time rather than response that is good one minute and poor the next. The acceptable variance in response time changes as the average response time itself changes. Users will tolerate an average response time of 1.5 seconds with a variance of ± 1 second much less than they will tolerate an average response time of 3 seconds with a variance of ± .25 second. One problem associated with attempting to prevent variability in performance is that when throughput is favored in tuning the system, the chance of experiencing variability in response time is greatly increased.

Predictability is another way of looking at consistency of performance. Predictability makes the job of the performance professional easier when forecasting future resource consumption. It also allows the appropriate setting of expectations for performance, as the following analogy shows.

The public transportation department announces that buses on a particular bus route are scheduled to arrive at each stop an average of every ten minutes. In a given thirty-minute period, three buses arrive all at once and the next one arrives forty minutes later. This schedule meets the stated criteria. However, it will make the people waiting for the fourth bus very unhappy. It would be much better for the waiting passengers if the buses were to arrive consistently ten minutes apart. It would also be perceived well if the buses were to arrive predictably at certain clock times.

3.1.3.2 Throughput

Information system management personnel are typically interested in throughput, rather than response time. If the demands of the organization require that a certain amount of work must be processed in an average day, then management is concerned whether the system can process that workload, rather than caring whether response time is one second or two seconds. Throughput is often quoted as work per unit time. Examples of throughput measures are:

  • Three thousand invoices processed per hour

  • Five CAD modeling runs per day

  • All end-of-month processing must complete within three days

  • Overnight processing must complete in the nine off-peak hours

It is not possible to develop guidelines for good throughput values. Throughput is driven by business needs, and the system must be sized to support those requirements. Capacity planning is done to ensure that as business requirements grow, the system will be able to handle the workload. It should be easy to predict in advance whether a system will be able to provide a specified throughput. The workload is considered a batch workload and the average time to complete a unit of work can be measured with a simple benchmark.

In reality, the situation is never that simple. Users are typically very vocal, and poor response time often reaches the ears of management. The point of this discussion is that the definition of performance may change from person to person; attitudes about response time and throughput must be examined to determine what users consider to be acceptable performance. Although the overall methodology is the same, tuning for response time and for throughput are different. Another way of putting this is that there is one strategy for approaching performance, but there are many different tactics.

3.1.4 Known Peak Periods

It is useful to identify known peak periods in advance, so that unusual spikes in the data can be readily explained. For instance, it is often mentioned that resource utilization in an office environment peaks at 11:00 a.m. and between 2:00–3:00 p.m. during the normal work day. Processing requirements typically grow at the end of the month or the end of a quarter. Expecting peaks at these times can save time when analyzing the data. Additionally, if the peaks are absent, it may be a clue that something unusual is preventing full system utilization.

3.1.5 Sudden Changes

Anyone who has worked in technical support has experienced callers complaining that the system "suddenly" is no longer working correctly or that performance has suddenly degraded. The following is a familiar dialogue:

From Bob's Consulting Log

Consultant: Has anything changed in the application or the system?

Client: No, nothing has changed.

Consultant: Are you sure that nothing has changed?

Client: I'm quite sure nothing has changed.

(Three days later, after a lot of investigation ... )

Consultant: Did you notice that your database administrator (DBA) dropped some indexes?

Client: Oh! I didn't think those changes would make a difference.

The task of investigating changes that may have been made to the system is quite an art. However, the importance of this part of the assessment should not be minimized.

3.1.6 Duration of the Problem

When doing performance problem diagnosis, identifying the duration of the problem involves several issues:

  • How long does the performance problem last?

  • When does the performance problem occur?

  • How long has the performance problem existed?

  • When did it begin?

It is important to understand the length of time that the problem lasts to detect whether it occurs only sometimes, because of a spike in resource utilization, or constantly. In each of these situations, tuning requires a different approach.

Knowing when the performance problem occurs means that data collection can be planned and minimized, as an alternative to collecting performance data for days or weeks to capture data when the problem manifests itself.

Finally, the length of time that the performance problem has existed influences the probability of determining if anything in the system or application has been changed. If the problem has existed for a long time (weeks or months), it is very unlikely that any changes will be discovered. One can also question the seriousness of the situation if the users have been living with the problem for months.

3.1.7 Understanding the Options

Understanding the options lets you determine what recommendations should be offered. If there is no capital budget for purchasing computer hardware, you can look for other ways to resolve the performance problem. Perhaps tuning the operating system or application is a viable alternative to upgrading the CPU to a faster model. In contrast, if time constraints dictate that the problem must be resolved quickly and deterministically, then upgrading the CPU to a faster model would probably be more expeditious than spending an unpredictable amount of time attempting to improve or redesign the application.

3.1.8 Understanding the Politics

It may be necessary to understand the politics of the organization before revealing the cause of the problem or before recommending the changes to be implemented. Knowledge of the politics may help narrow the scope of the question you are trying to answer. It may be that the user organization wants to gain more control of the system, and it is trying to gather evidence to support this cause. The Information Technology (IT) department (also called Information Systems in some organizations) may be trying to allocate computing resources fairly, and it may not be possible to make changes that improve the situation for only one group. Finally, you may have been called in simply to provide objective data to justify the purchase of a larger, faster system.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020