NEW TITLE RELEASE ANNOUNCEMENT
See the latest about a temporary release delay on new titles. Learn more.
A unique resource that shows Linux kernel AND Linux application programmers how to write bug-free code that's optimized for performance.
° Debugging tools and techniques for the Linux kernel itself and Linux applications
° Performance tools and techniques that show how to supercharge the kernel and the applications that run on it.
° Sophisticated profiling techniques that teach programmers how to uncover application bottlenecks
Learn Linux debugging and optimizationat kernel and application levelshands-on!
This is the definitive guide to Linux software debugging and performance optimization at both the kernel and application levels. Using extensive Linux code examples, Steve Best systematically introduces open source tools and best-practice techniques for delivering bug-free, well-tuned code.
Drawing on his exceptional experience optimizing Linux systems for IBM, Best covers issues ranging from memory management and I/O to system processes and kernel bug messages. You'll walk through real debugging sessions, discovering the strategies experts use to debug even the most complex application- and kernel-related problems. You'll master sophisticated profiling techniques for identifying and resolving bottlenecks more quickly and learn how to capture the right data in the event of trouble. Coverage includes
Debuggers: gdb, kgdb, and KDB
/proc kernel data analysis
System process monitoring
Oops bug messages
Syslog and event logging
Profiling kernel behavior
Crash dump analysis
Linux® Debugging and Performance Tuning will be indispensable for every developer who needs to supercharge the Linux kernel and applications, and for every administrator and support specialist who must resolve Linux reliability or performance issues.
© Copyright Pearson Education. All rights reserved.
Download the Sample
Chapter related to this title.
About the Author.
2. Code Coverage.
3. GNU Debugger (gdb).
4. Memory Management Debugging.
5. System Information (/proc).
6. System Tools.
7. System Error Messages.
8. Event Logging.
9. Linux Trace Toolkit.
10. oprofile: a Profiler Supported by the Kernel.
11. User-Mode Linux.
12. Dynamic Probes.
13. Kernel-Level Debuggers (kgdb and kdb).
14. Crash Dump.
The book includes many sample programs that demonstrate how to use the best profiling and debugging tools available for Linux. All the tools are open-source and continue to be enhanced by the open-source community.
The goal of the book is to provide you with the knowledge and skills you need to understand and solve software problems on Linux servers. It discusses techniques and tools used to capture the correct data using first failure data capture approaches.
Chapter 1, "Profiling," discusses methods to measure execution time and real-time performance. Application performance tuning is a complex process that requires correlating pieces of data with source code to locate and analyze performance problems. This chapter shows a sample program that is tuned using a profiler called gprof and a code coverage tool called gcov.
Chapter 2, "Code Coverage," discusses coverage code that can be used to determine how well your test suites work. One indirect benefit of gcov is that its output can be used to identify which test case provides coverage for which source file. Code coverage during testing is one important measurement of software quality. Like an X-ray machine, gcov peers into your code and reports on its inner workings.
What would debugging be without a debugger? Chapter 3, "GNU Debugger (gdb)," looks at the GNU debugger. You can debug by adding printf statements to a program, but this is clumsy and very time-consuming. A debugger like gdb is a much more efficient debugging tool.
Chapter 4, "Memory Management Debugging," looks at the APIs for memory management, which, although small, can give rise to a large number of disparate problems. These include reading and using uninitialized memory, reading/writing from memory past or in front of (underrun) the allocated size, reading/writing inappropriate areas on the stack, and memory leaks. This chapter covers four memory management checkers: MEMWATCH, YAMD, Electric Fence, and Valgrind. We'll review the basics, write some "buggy" code, and then use each of these tools to find the mistakes.
The /proc file system is a special window into the running Linux kernel and is covered in Chapter 5, "System Information (/proc)." The /proc file system provides a wealth of information for the Linux kernel. It offers information about each process to system-wide information about CPU, memory, file systems, interrupts, and partitions. Some of the utilities that use /proc entries to get data from the system include iostat, sar, lsdev, lsusb, lspci, vmstat, and mpstat. Each of these utilities is covered in the chapter.
Chapter 6, "System Tools," looks at various tools that can be used to pinpoint what is happening to the system and to find which component of the system is having a problem. The ps command is a valuable tool that can be used to report the status of each of the system processes. Three other process tools are coveredpgrep, pstree, and top. The strace command lets you trace system calls. The magic key sequence can provide a back trace for all the processes on the system. The lsof tool can be used to list the open files on the system. Finally, the network debugging tools ifconfig, arp, ethereal, netstat, and tcpdump are covered. They can help solve network-type problems.
Many kernel bugs show themselves as NULL pointer dereferences or other values to pointers that are incorrect. The common result of such a bug is the Oops message. Chapter 7, "System Error Messages," covers where an Oops message is stored, how to analyze the Oops, and finding the failing line of code.
An important goal of a Linux systems administrator is to ensure that his or her systems are functioning and performing 100 % of the time. Applications producing error messages, file systems not having free space available, network adapter failures, hard drives producing errors, and the kernel producing errors are just a few types of problems that could possibly stop a system, impacting that goal. Chapter 8, "Event Logging," helps administrators grapple with these issues by describing Syslog and event logging.
Chapter 9, "Linux Trace Toolkit," shows how an execution trace shows exactly what scheduling decisions are made and how various management tasks are done. It captures how they are handled, how long they take, and to which process the processor has been allocated. The trace facility provides a dynamic way to gather system data. Application I/O latencies can also be identified, as well as the time when a specific application is actually reading from a disk. Certain types of locking issues also can be seen by tracing.
In short, tracing can be used to
The Linux Trace Toolkit (LTT) differs from strace or gprof in that LTT provides a global view of the system, including a view into the kernel.
Chapter 10, "oprofile: a Profiler Supported by the Kernel," covers the kernel profiler called oprofile. Profilers are software development tools designed to help analyze the performance of applications and the kernel. They can be used to identify sections of code that aren't performing as expected. They provide measurements of how long a routine takes to execute, how often it is called, where it is called from, and how much time it takes. Profiling is also covered in Chapter 1; one profiler in that chapter is called gprof. Another topic covered in Chapter 10 is ways to minimizing cache misses. Cache misses can be a cause of applications not performing as expected.
User-Mode Linux (UML) is covered in Chapter 11, "User-Mode Linux"; it is a fully functional Linux kernel. It runs its own scheduler and virtual memory (VM) system, relying on the host kernel for hardware support. The benefits of UML from a debugging point of view are that it lets you do kernel development and debugging at the source code level using gdb. The UML technology can be a powerful tool to reduce the time needed to debug a kernel problem and development kernel-level features.
Chapter 12, "Dynamic Probes," explains dynamic probes (Dprobes), which is a technique for acquiring diagnostic information without custom-building the component. Dynamic probes can also be used as a tracing mechanism for both user and kernel space. It can be used to debug software problems that are encountered in a production environment that can't be re-created in a test lab environment. Dprobes are particularly useful in production environments where the use of an interactive debugger is either undesirable or unavailable. Dprobes also can be used during the code development phase to cause faults or error injections into code paths that are being tested.
Chapter 13, "Kernel-Level Debuggers (kgdb and kdb)," covers two kernel-level debuggers: kgdb and kdb. kgdb is an extension to gdb that allows the gdb debugger to debug kernel-level code. One key feature of kgdb is that it allows source code-level debugging of kernel-level code. The kdb debugger allows kernel-level debugging but does not provide source-level debugging.
There are multiple ways for Linux to support a crash dump. Chapter 14, "Crash Dump," covers the different types of crash dumps. It discusses Linux Kernel Crash Dump (LKCD), Netdump, Diskdump, and mcore. Crash dump is designed to meet the needs of end users, support personnel, and systems administrators needing a reliable method of detecting, saving, and examining system problems. There are many benefits of having a bug report and dump of the problem, since the dump provides a significant amount of information about the system's state at the time of the problem.
Download the Foreword
file related to this title.
Download the Index
file related to this title.