Home > Store

Techniques for Optimizing Applications: High Performance Computing

Register your product to gain access to bonus material or receive a coupon.

Techniques for Optimizing Applications: High Performance Computing

Book

  • Sorry, this book is no longer in print.
Not for Sale

Description

  • Copyright 2002
  • Edition: 1st
  • Book
  • ISBN-10: 0-13-093476-3
  • ISBN-13: 978-0-13-093476-5

This book is a practical guide to performance optimization of computationally intensive programs on Sun UltraSPARC platforms. It is primarily intended for developers of technical or high performance computing (HPC) applications for the Solaris(tm) operating environment. This audience includes both independent software vendor (ISV) developers and noncommercial developers. It can also be used by end-users of HPC applications to help them better understand how applications utilize system resources.

The book presents information so that it follows logical stages of the process for application development and optimization. We pay special attention to issues related to parallel applications and to using appropriate performance measurement tools. Wherever applicable, sections are illustrated with code examples that show benefits of methods described.

Unless otherwise noted, topics in this book are not limited to a particular programming language, parallelization method, software version, or hardware product. However, emphasis is on techniques relevant to applications written in Fortran 77, Fortran 90, and C, because these languages are most commonly used in HPC and technical applications. Most topics can be applied to C++ programs; however, we do not address performance optimization issues specific to object-oriented programming.

Sample Content

Table of Contents



Acknowledgments.


Preface.


Who Should Read This Book.


How This Book Is Organized.


Additional Resources.


Code Examples.


Typographical Conventions.

I. GETTING STARTED.

1. Introduction.

Performance Components. Hardware. Software. Optimization Process Overview. Serial Optimization. Parallel Optimization.

2. Overview of Sun UltraSPARC Solaris Platforms.

UltraSPARC-Based Desktop and Server Product Line. UltraSPARC-Based Workstations. UltraSPARC-Based Servers. Sun Technical Compute Farm. Solaris Operating Environment. Sun WorkShop and Forte Developer Tools. HPC ClusterTools Software. Summary.

3. Application Development on Solaris.

Development Basics. Standards Conformance. Binary Compatibility. Source Code Verification Tools. Checking C Programs. Checking Fortran Programs. Additional Source Code Analysis Tools. 64-bit Development and Porting. Fortran Porting. Language Interoperability. Fortran 95 and Fortran 77. C and Fortran. Linking Mixed Languages. Summary.

II. OPTIMIZING SERIAL APPLICATIONS.

4. Measuring Program Performance.

Measurement Methodology. Benchmarking Guidelines. Measurement Tools. Program Timing Tools. Timing Entire Program. Timing Program Portions. Fine-Grained Timing Measurement. Program Profiling Tools. Profiling With prof and gprof. Profiling With tcov. Profiling Tools in Forte Developer 6. Process and System Monitoring Tools. /proc Tools. Process Tracing Tools. System Monitoring Tools. Hardware Counter Measurements. Monitoring Tools. Hardware Counter Overflow Profiling. Code Instrumentation With libcpc Calls. Summary.

5. Basic Compiler Optimizations.

Compilation Overview. Structure of Sun Compilers. Using Sun Compilers. -fast and -xtarget Options. Basic Guidelines. -xarch. Specifying Target Architecture. Generation of Conditional Move Instructions. Creating 64-bit Binaries. -xchip. -xO Optimization Level. -xinline, -xcrossfile. -xdepend. - xvector. -xsfpconst. -xprofile=collect, use. -xprefetch. Summary.

6. Advanced Compiler Optimizations.

IEEE Floating-Point Arithmetic. Binary Storage Format. Trap Handling and -ftrap. Gradual Underflow and -fns. -fsimple. -dalign. -xsafe= mem. Pointer Alias Analysis Options. -xrestrict. -xalias_level. -stackvar. Compiler Directives and Pragmas. pragma pipeloop. pragma opt. pragma prefetch. pragma pack. pragma align. Pointer Alias Analysis Pragmas. Summary.

7. Linker and Libraries in Performance Optimization.

Linking Overview. Static and Dynamic Linking. Structure of an ELF Binary. Solaris Linker Usage. Linking Static and Dynamic Libraries. Weak Symbol Binding. Linker Mapfiles. Linking Optimized Math Libraries. Creating Architecture-Specific Libraries. $PLATFORM and $ISALIST Linker Tokens. $ORIGIN Token. Runtime Linker in Profiling and Debugging. Interposing Libraries. Using LD_PROFILE and LD_DEBUG. Summary.

8. Source Code Optimization.

Overview of Memory Hierarchy. Memory Levels. Memory Organization of UltraSPARC-Based Systems. Memory Hierarchy Optimizations. Cac

Preface

Preface

This book is a practical guide to optimizing performance of computationally intensive applications on Sun UltraSPARC platforms. It offers techniques for improving performance of applications that are predominantly compute-intensive or CPU-bound.

We wrote this book with a general enough scope so that it would be useful to as many developers of technical applications on Sun platforms as possible. Also, we made the material practical by showing developers how to use each optimization method.

For information on related topics such as system configuration and tuning, or improving the I/O and network performance, we refer readers to other resources.

This book differs from other books and technical documents written aboutperformance optimization of high performance computing (HPC) applications. In many cases, other resources either give a detailed description of a product or provide general recommendations that are sometimes difficult to apply to practical tasks. In addition, some older resources are not as useful because of changes in technology.

Though many of the techniques we offer apply to other platforms, we limited the scope of this guide to Sun compilers and UltraSPARC-based Solaris systems. We address new features in Sun compilers and in the Solaris Operating Environment, and we show readers how to use these products to get maximum performance on Sun hardware.

Who Should Read This Book

This guide is primarily for developers of technical or HPC applications for Solaris. This audience includes both independent software vendor (ISV) developers and non-commercial developers.

Developers creating or optimizing applications in the following fields may benefit from reading this book:

  • Mechanical computer-aided engineering (MCAE)
  • Electronic design automation (EDA)
  • Computational chemistry
  • Bioinformatics
  • Operations research
  • Financial modeling
  • Reservoir simulation and seismic modeling
  • Mechanical computer-aided design (MCAD) modeling
  • Graphics rendering and imaging
  • Climate and weather modeling

This book may also be helpful to technical application end-users in understanding the principles of HPC and how an application utilizes system resources.

We assume the reader has:

  • familiarity with development basics in UNIX environments
  • a working knowledge of programming in C and Fortran languages
  • familiarity with computer architecture
  • experience in parallel programming
  • a basic knowledge of SPARC assembly (desirable)

Unless otherwise noted, topics in this book are not limited to a programming language, parallelization method, or software version. However, emphasis is on techniques relevant to applications written in Fortran 77, Fortran 90, and C, because these languages are most commonly used in HPC and technical applications.

Most topics can be applied to C++ programs; however, we do not address performance optimization issues for object oriented programming. We refer readers to other resources.

How This Book Is Organized

This book presents information so that it follows logical stages of the process for application development and optimization. We pay special attention to issues related to parallel applications and to using appropriate performance monitoring tools.

Wherever applicable, sections are illustrated with code examples that show benefits of methods described.

Part I - Getting Started

Chapter 1 "Introduction," introduces optimization for HPC applications. We describe the basics of the optimization process and illustrate it with flow charts for serial and parallel optimization.

Chapter 2 "Overview of Sun UltraSPARC Solaris Platforms," describes the available "tools of trade" for HPC developers using Solaris platforms. It gives an overview of Sun hardware and software products for technical computing. Also, the chapter introduces software development tools.

Chapter 3 "Application Development on Solaris," considers development and porting issues on Sun platforms. It includes sections on binary compatibility between platforms, standards conformance, code verification tools, language interoperability, and 64-bit porting issues.

Part II - Optimizing Serial Applications

Chapter 4 "Measuring Program Performance," focuses on tools that measure application performance. Accurate measurement of performance is crucial in tuning. We describe accurate timers available on Solaris, profiling tools, Forte Developer 6 Performance Analyzer, hardware performance counter access tools on UltraSPARC processors, and other system monitoring tools.

Chapter 5 "Basic Compiler Optimizations," introduces basic compiler optimizations and how to use compiler flags correctly. Options covered in this chapter are safe and generally can be applied without knowledge of any specifics of the application. The impact of using these flags is illustrated with examples, and analysis of the generated code with and without the options is presented.

Chapter 6 "Advanced Compiler Optimizations," extends Chapter 5 and gives an overview of techniques that enable aggressive compiler optimizations. These often result in additional performance gains but may also lead to incorrect answers or spurious side-effects. Also, we cover performance related compiler pragmas and directives, which can be inserted in a program. Information about a program can be passed to the compiler, allowing additional optimizations.

Chapter 7 "Linker and Libraries in Performance Optimization," highlights optimized libraries and features of the Solaris linker that can be used for application optimization. We describe the platform-specific optimized math libraries whose use can result in significant performance gains. We show linker techniques that allow linking of these platform-specific libraries in a portable fashion.

Chapter 8 "Source Code Optimization," provides an overview of tuning techniques at the source code level. The techniques were selected from the point of view of better utilizing the underlying architectural features of UltraSPARC systems. We pay special attention to memory hierarchy utilization such as cache blocking and reducing the translation lookaside buffer (TLB) misses. We present ways of simplifying the code to allow better compiler optimizations, such as alias disambiguation in C programs, to take place.

Chapter 9 "Loop Optimization," focuses on optimizing loops, one of the most commonly used constructs in scientific and HPC programs. We discuss ways in which developers can help the compiler control loop fusion and fission, as well as perform loop peeling. We show examples of register-tiling and consider loops with branches.

Part III - Optimizing Parallel Applications

Chapter 10 "Parallel Processing Models on Solaris," introduces concepts of parallel programming and different parallelization models available on Solaris/SPARC systems: automatic compiler parallelization, directives-based parallelism, explicit multithreading, UNIX fork/exec, message passing model, and hybrid programming (combined directives and message-passing).

Chapter 11 "Parallel Performance Measurement Tools," details the tools for performance measurement and monitoring of parallel programs. Similar to Chapter 4, we focus on accurate timers for timing parallel programs, tools for measuring synchronization and communication overheads, tools for measuring hardware counters, and tools for multiprocessor system monitoring.

Chapter 12 "Optimization of Explicitly Threaded Programs," provides an overview of explicit multithreading of programs using P-threads and Solaris threads. An overview of thread scheduling models in Solaris and their relevance to HPC programs is given and techniques for decreasing synchronization overheads are described.

Chapter 13 "Optimization of Programs Using Compiler Parallelization," covers support and optimization techniques for automatic and directive-based parallelization in Sun compilers. Special emphasis is given to tuning OpenMP programs using the Fortran 95 compiler. OpenMP programming styles and data-scoping issues are illustrated with examples. Comparisons between OpenMP and P-threads approaches are presented.

Chapter 14 "Optimization of Message-Passing Programs," describes message-passing models and how to tune MPI programs. We present an overview of message-passing programming models, compiling and linking programs using Sun MPI, and using Sun MPI environment variables. This chapter describes approaches for optimizing point-to-point and global communication with Sun MPI, using the S3L scientific library and using a hybrid OpenMP/MPI model.

Part IV - Appendices

Appendix A "Commands That Identify System Configuration Parameters," lists useful Solaris commands that identify system configuration parameters.

Appendix B "Architecture of UltraSPARC Microprocessor Family," gives an overview of architectural features of the UltraSPARC microprocessor family.

Appendix C "Architecture of UltraSPARC Interconnect Family," describes the architecture of interconnect technologies for UltraSPARC systems.

Appendix D "Hardware Counter Performance Metrics," shares some useful performance metrics that can be derived from hardware counters on UltraSPARC systems.

Appendix E "Interval Arithmetic Support in Forte Developer 6 Fortran 95 Compiler," gives an overview of interval arithmetic support in the Forte Developer 6 Fortran 95 compiler.

Appendix F "Differences in I/O Performance," considers the performance of different I/O techniques.

Additional Resources

To keep the scope of this book manageable, we intentionally omitted many subjects related to performance optimization. Our criteria was to omit subjects that were not applicable to a wide range of applications. Many of these subjects are presented in other documentation for Sun products. The following is a list of publications you may find useful for more narrowly focused subjects:

  • Numerical Computation Guide
  • Fortran Programming Guide
  • Fortran User's Guide
  • Analyzing Program Performance with Sun Workshop
  • C User's Guide
  • Forte 6 update 1 C User's Guide Supplement
  • Linker and Libraries Guide
  • Sun Performance Library Reference
  • Multithreaded Programming Guide
  • Programming Utilities Guide
  • 64-bit Developer's Guide
  • Solaris Tunable Parameters Reference Manual
  • Sun HPC ClusterTools 3.1 Performance Guide
  • Sun MPI 4.1 Programming and Reference Guide
  • Prism 6.1 User's Guide
  • Sun HPC ClusterTools 3.1 Installation Guide

All these publications are available online at http://docs.sun.com. We strongly recommend that developers visit this site, because nearly all published Sun documentation is available there. Printed versions are available from Sun Documentation Center at Fatbrain:

http://www1.fatbrain.com/documentation/sun

The following publications are related to UltraSPARC microprocessors:

  • UltraSPARC I and II User's Manual
  • UltraSPARC IIi User's Manual

These publications are available at:

http://www.sun.com/microelectronics/manuals

Other sites of great use for developers are http://www.sun.com/developers/ and http://soldc.sun.com/, which contains current information for the Sun developer community.

A description of Sun product lines is available at http://www.sun.com/desktop and http://www.sun.com/servers. Sun products and solutions for the HPC are listed at http://www.sun.com/hpc.

For specialized books and additional theoretical information on application optimization, we refer readers to other sources. There are many excellent books on topics such as optimizing compilers, software tuning techniques, and efficient parallelization. The following are some helpful resources:

  • J. Hennessy, D. Patterson - Computer Architecture: A Quantitative Approach, Second Edition; Morgan Kaufmann Publishing, 1996
  • K. Dowd, Ch. Severance - High Performance Computing, Second Edition; O'Reilly & Associates, 1998
  • D. E. Culler, J. P. Singh, A. Gupta - Parallel Computer Architecture: A Hardware Software Approach, Morgan Kaufmann Publishing, 1999
  • S. S. Muchnick - Advanced Compiler Design and Implementation; Morgan Kaufmann Publishing, 1997.
  • S. Kleinman, D. Shah, B. Smaalders - Programming with Threads; SunSoft Press, A Prentice Hall Title, 1996.
  • W. Gropp, E. Lusk, A. Skjellum - Using MPI: Portable Parallel Programming with Message-Passing Interface (Scientific and Engineering Computation Series); Second Edition, MIT Press, 1999.
  • R. Chandra, L. Dagum, D. Kohr, D. Maydan, J. McDonald, R. Menon - Parallel Programming in OpenMP; Morgan Kaufmann Publishing, 2000.

More resources are listed in the References section at the end of this book.

Readers interested in "hands on" training should check with the Sun Educational Services to determine if a class is scheduled and enrollment is open.

Code Examples

Many sections in this book are illustrated with code examples that show benefits of optimization techniques and coding practices. The code examples can be downloaded from the Sun BluePrints site:

http://www.sun.com/blueprints/tools

A makefile is provided for each chapter so that the examples can be run with a single make command.

Unless otherwise noted, all examples and results presented in this book use Forte Developer 6 compilers and the HPC 3.1 ClusterTools release. The results for serial runs were obtained on Sun Ultra 60, Sun Ultra 80, and Sun Blade 1000 systems. The results for parallel runs were performed on Sun Enterprise 4500, Sun Enterprise 10000 servers, and a Sun technical compute farm. The system parameters are listed in TABLE P-1 and TABLE P-2, respectively. More information about Sun platforms is in Chapter 2.

Updates

Submit Errata

More Information

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020