Home > Store

Software Build Systems: Principles and Experience

Register your product to gain access to bonus material or receive a coupon.

Software Build Systems: Principles and Experience

Book

  • Sorry, this book is no longer in print.
Not for Sale

Description

  • Copyright 2011
  • Dimensions: 7" x 9-1/8"
  • Edition: 1st
  • Book
  • ISBN-10: 0-321-71728-7
  • ISBN-13: 978-0-321-71728-3

“This book represents a thorough and extensive treatment of the software build process including the choices, benefits, and challenges of a well designed build process. I recommend it not only to all software build engineers but to all software developers since a well designed build process is key to an effective software development process.”

Kevin Bodie, Director Software Development, Pitney Bowes Inc.

“An excellent and detailed explanation of build systems, an important but often overlooked part of software development projects. The discussion of productivity as related to build systems is, alone, well worth the time spent reading this book.”

John M. Pantone, Objectech Corporation, VP, IT Educator and Course Developer

“Peter Smith provides an interesting and accessible look into the world of software build systems, distilling years of experience and covering virtually every type of tool in the build engineer’s toolbox. Well organized, well written, and very thorough; I would recommend this book to anyone with a build system under their responsibility.”

Jeff Overbey, Project Co-Lead, Photran

Software Build Systems teaches how to think about building software. It surveys the tools and techniques for building software products and the ways things go wrong. This book will appeal to those new to build systems as well as experienced build system engineers.”

Monte Davidoff, Software Development Consultant, Alluvial Software, Inc.

Inadequate build systems can dramatically impact developer productivity. Bad dependencies, false compile errors, failed software images, slow compilation, and time-wasting manual processes are just some of the byproducts of a subpar build system. In Software Build Systems, software productivity expert Peter Smith shows you how to implement build systems that overcome all these problems, so you can deliver reliable software more rapidly, at lower cost.

Smith explains the core principles underlying highly efficient build systems, surveying both system features and usage scenarios. Next, he encapsulates years of experience in creating and maintaining diverse build systems–helping you make well-informed choices about tools and practices, and avoid common traps and pitfalls. Throughout, he shares a wide range of practical examples and lessons from multiple environments, including Java, C++, C, and C#. Coverage includes

• Mastering build system concepts, including source trees, build tools, and compilation tools

• Comparing five leading build tools: GNU Make, Ant, SCons, CMake, and the Eclipse IDE’s integrated build features

• Ensuring accurate dependency checking and efficient incremental compilation

• Using metadata to assist debugging, profiling, and source code documentation

• Packaging software for installation on your target machine

• Best practices for managing complex version-control systems, build machines, and compilation tools

If you’re a developer, this book will illuminate the issues involved in building and maintaining the build system that’s best for your team. If you’re a manager, you’ll discover how to evaluate your team’s build system and improve its effectiveness. And if you’re a build “guru,” you’ll learn how to optimize the performance and scalability of your build system, no matter how demanding your requirements are.

Sample Content

Online Sample Chapter

Software Build Systems: Make

Table of Contents

PREFACE     xxi

Why Do Build Systems Become Complex?     xxiii

The True Cost of a Build System     xxiv

The Focus of This Book     xxvii

Who Should Read This Book?     xxviii

How This Book Is Organized     xxix

Summary     xxxi

ACKNOWLEDGMENTS     xxxiii

ABOUT THE AUTHOR     xxxv

PART I THE BASICS     1

Chapter 1 BUILD SYSTEM OVERVIEW     3

What Is a Build System?     3

Compiled Languages     4

Interpreted Languages     6

Web-Based Applications     6

Unit Testing     7

Static Analysis     8

Documentation Generation     9

Components of a Build System     10

Version-Control Tools     10

Source and Object Trees     11

Compilation Tools and Build Tools     13

Build Machines     14

Release Packaging and Target Machines     15

The Build Process and Build Description     16

How a Build System Is Used     18

Build-Management Tools     19

Build System Quality     21

Summary     21

Chapter 2 A MAKE-BASED BUILD SYSTEM     23

Calculator Example     23

Creating a Simple Makefile     26

Simplifying the Makefile     28

Additional Build Targets     29

Using a Framework     31

Summary     33

Chapter 3 THE RUNTIME VIEW OF A PROGRAM     35

Executable Programs     36

Native Machine Code     36

Monolithic System Images     37

Full Program Interpretation     38

Interpreted Byte Codes     39

Libraries     40

Static Linking     41

Dynamic Linking     42

Configuration and Data Files     43

Distributed Programs     44

Summary     46

Chapter 4 FILE TYPES AND COMPILATION TOOLS     47

C/C++     48

Compilation Tools     49

Source Files     50

Assembly Language Files     52

Object Files     53

Executable Programs     56

Static Libraries     57

Dynamic Libraries     58

C++ Compilation     59

Java     60

Compilation Tools     61

Source Files     62

Object Files     63

Executable Programs     65

Libraries     67

C#     68

Compilation Tools     68

Source Files     69

Executable Programs     71

Libraries     74

Other File Types     76

UML-Based Code Generation     77

Graphic Images     79

XML Configuration Files     81

Internationalization and Resource Bundles     81

Summary     82

Chapter 5 SUBTARGETS AND BUILD VARIANTS     83

Building Subtargets     84

Building Different Editions of the Software     86

Specifying the Build Variant     87

Varying the Code     90

Building Different Target Architectures     94

Multiple Compilers     94

Platform-Specific Files/Functions     95

Multiple Object Trees     96

Summary     98

PART II THE BUILD TOOLS     99

Chapter 6 MAKE     107

The GNU Make Programming Language     108

Makefile Rules to Construct the Dependency Graph     109

Makefile Rule Types     110

Makefile Variables     112

Built-In Variables and Rules     114

Data Structures and Functions     116

Understanding Program Flow     119

Further Reading     122

Real-World Build System Scenarios     123

Scenario 1: Source Code in a Single Directory     123

Scenario 2(a): Source Code in Multiple Directories     125

Scenario 2(b): Recursive Make over Multiple Directories     126

Scenario 2(c): Inclusive Make over Multiple Directories     130

Scenario 3: Defining New Compilation Tools     137

Scenario 4: Building with Multiple Variants     138

Scenario 5: Cleaning a Build Tree     140

Scenario 6: Debugging Incorrect Builds     142

Praise and Criticism     144

Praise     144

Criticism     146

Evaluation     148

Similar Tools     149

Berkeley Make     149

NMake     150

ElectricAccelerator and SparkBuild     151

Summary     153

Chapter 7 ANT     155

The Ant Programming Language     156

A Little More Than “Hello World”     157

Defining and Using Targets     158

Ant’s Flow of Control     161

Defining Properties     162

Built-In and Optional Tasks     164

Selecting Multiple Files and Directories     168

Conditions     170

Extending the Ant Language     172

Further Reading     173

Real-World Build System Scenarios     174

Scenario 1: Source Code in a Single Directory     174

Scenario 2(a): Source Code in Multiple Directories     175

Scenario 2(b): Many Directories, with Multiple build.xml Files     175

Scenario 3: Defining New Compilation Tools     179

Scenario 4: Building with Multiple Variants     183

Scenario 5: Cleaning a Build Tree     188

Scenario 6: Debugging Incorrect Builds     188

Praise and Criticism     191

Praise     191

Criticism     191

Evaluation     193

Similar Tools     193

NAnt     194

MSBuild     194

Summary     196

Chapter 8 SCons     197

The SCons Programming Language     198

The Python Programming Language     199

Simple Compiling     202

Managing Build Environments     206

Program Flow and Dependency Analysis     210

Deciding When to Rebuild     212

Extending the Language     214

Other Interesting Features     218

Further Reading     219

Real-World Build System Scenarios     219

Scenario 1: Source Code in a Single Directory     219

Scenario 2(a): Source Code in Multiple Directories     219

Scenario 2(b): Multiple SConstruct Files     220

Scenario 3: Defining New Compilation Tools     222

Scenario 4: Building with Multiple Variants     224

Scenario 5: Cleaning a Build Tree     226

Scenario 6: Debugging Incorrect Builds     226

Praise and Criticism     229

Praise     230

Criticism     231

Evaluation     231

Similar Tools     232

Cons     232

Rake     233

Summary     235

Chapter 9 CMAKE     237

The CMake Programming Language     238

CMake Language Basics     239

Building Executable Programs and Libraries     240

Control Flow     243

Cross-Platform Support     246

Generating a Native Build System     248

Other Interesting Features and Further Reading     254

Real-World Build System Scenarios     255

Scenario 1: Source Code in a Single Directory     255

Scenario 2: Source Code in Multiple Directories     256

Scenario 3: Defining New Compilation Tools     257

Scenario 4: Building with Multiple Variants     259

Scenario 5: Cleaning a Build Tree     260

Scenario 6: Debugging Incorrect Builds     260

Praise and Criticism     261

Praise     261

Criticism     262

Evaluation     262

Similar Build Tools     263

Automake     263

Qmake     264

Summary     264

Chapter 10 ECLIPSE     267

The Eclipse Concepts and GUI     268

Creating Projects     269

Building a Project     276

Running a Project     282

Using the Internal Project Model     285

Other Build Features     286

Further Reading     288

Real-World Build System Scenarios     288

Scenario 1: Source Code in a Single Directory     288

Scenario 2: Source Code in Multiple Directories     290

Scenario 3: Defining New Compilation Tools     291

Scenario 4: Building with Multiple Variants     292

Scenario 5: Cleaning a Build Tree     295

Scenario 6: Debugging Incorrect Builds     296

Praise and Criticism     296

Praise     297

Criticism     297

Evaluation     298

Similar Build Tools     299

CDT for Eclipse, C/C++ Development Tooling     299

Summary     301

PART III ADVANCED TOPICS     303

Chapter 11 DEPENDENCIES     305

The Dependency Graph     307

Incremental Compilation     307

Full, Incremental, and Subtarget Builds     308

The Problem with Bad Dependencies     310

Problem: Missing Dependencies Causing a Runtime Error     310

Problem: Missing Dependencies Causing a Compile Error     311

Problem: Unwanted Dependencies Causing Excess Rebuilding     312

Problem: Unwanted Dependencies Causing Failed Dependency Analysis     312

Problem: Circular Dependencies     313

Problem: Implicit Sequencing As a Substitute for Dependencies     314

Problem: The Clean Target Doesn’t Clean Everything     315

Step 1: Computing the Dependency Graph     315

Gathering Exact Dependencies     316

Caching the Dependency Graph     319

Updating the Cached Dependency Graph     320

Step 2: Determining Which Files Are Out-of-Date     324

Time Stamp-Based Methods     324

Checksum-Based Methods     326

Flag Comparison     328

Advanced Methods     329

Step 3: Sequencing the Compilation Steps     330

Summary     333

Chapter 12 BUILDING WITH METADATA     335

Debugging Support     336

Profiling Support     338

Coverage Support     340

Source Code Documentation     341

Unit Testing     344

Static Analysis     348

Adding Metadata to a Build System     349

Summary     350

Chapter 13 SOFTWARE PACKAGING AND INSTALLATION     351

Archive Files     352

Packaging Scripts     353

Other Archive Formats     356

Improvements     356

Package-Management Tools     359

The RPM Package Manager Format     360

The rpmbuild Process     361

An Example RPM Spec File     363

Creating the RPM File from the Spec File     369

Installing the RPM Example     371

Custom-Built GUI Installation Tools     373

The Nullsoft Scriptable Install System (NSIS)     374

The Installer Script     376

Defining the Pages     379

The License Page     380

Directory Selection     381

The Main Component     381

The Optional Components     383

Defining a Custom Page     385

The Installation Page and the Uninstaller     387

Summary     388

Chapter 14 VERSION MANAGEMENT     391

What Should Be Version-Controlled     392

Build Description Files     393

References to Tools     395

Large Binary Files     400

Source Tree Configurations     401

What Should Not Be in the Source Tree     402

Generated Files in the Source Tree     402

Generated Files Under Version Control     404

Build-Management Scripts     405

Version Numbering     406

Version-Numbering Systems     406

Coordinating and Updating the Version Number     407

Storing and Retrieving the Version Number     410

Summary     411

Chapter 15 BUILD MACHINES     413

Native and Cross-Compilation     414

Native Compilation     414

Cross-Compilation     415

Hybrid Environments     416

Centralized Development Environments     416

Why Build Machines Differ     418

Managing Multiple Build Machines     421

Open-Source Development Environments     424

GNU Autoconf     428

The High-Level Workflow     428

An Autoconf Example     430

Running autoheader and autoconf     434

Running the configure Script on the Build Machine     435

Using the Configuration Information     437

Summary     438

Chapter 16 TOOL MANAGEMENT     441

Rules for Managing Tools     442

Tool Rule #1: Take Notes     442

Tool Rule #2: Use Version Control for the Source Code     443

Tool Rule #3: Periodically Upgrade Tools     444

Tool Rule #4: Use Version Control for the Tool Binaries     445

Breaking the Rules     448

Writing Your Own Compilation Tools     449

Custom-Written Tools with Lex and Yacc     450

Summary     453

PART IV SCALING UP     455

Chapter 17 REDUCING COMPLEXITY FOR END USERS     457

Build Frameworks     458

Developer-Facing Portion of the Build Description     459

Framework Portion of the Build Description     460

Convention over Configuration     461

Maven: An Example Build Tool     462

Reasons to Avoid Supporting Multiple Variants     463

You’ll Have More Variants to Test     463

Source Code Becomes Messy     465

Build Times Can Increase     465

Higher Disk Space Requirements     466

Various Ways to Reduce Complexity     466

Use a Modern Build Tool     466

Automatically Detect Dependencies     467

Keep Generated Files out of the Source Tree     467

Ensure That Cleaning a Build Tree Works Correctly     468

Abort the Build After the First Error     468

Provide Meaningful Error Messages     470

Validate Input Parameters     470

Don’t Overengineer Build Scripts     471

Avoid Using Cryptic Language Features     471

Don’t Use Environment Variables to Control the Build Process     472

Ensure That Release and Debug Builds Are Similar     473

Display the Exact Command Being Executed     474

Version-Control References to Tools     475

Version-Control the Build Instructions     475

Automatically Detect Changes in Compilation Flags     475

Don’t Invoke the Version-Control Tool from the Build System     476

Use Continuous Integration as Often as Possible     476

Standardize on a Single Type of Build Machine     477

Standardize on a Single Compiler     477

Avoid Littering Code with #ifdefs     477

Use Meaningful Symbol Names     478

Remove Stale Code     478

Don’t Duplicate Source Files     479

Use a Consistent Build System     480

Scheduling and Staffing Build System Changes     480

Summary     482

Chapter 18 MANAGING BUILD SIZE     485

The Problem with Monolithic Builds     486

Component-Based Software     488

Advantages of Using Components     491

What Exactly Is a Component?     493

Integrating Components into a Single Product     498

People and Process Management     502

Development Team Structure     503

Component Line-Up Management     505

Managing the Component Cache     507

Coordinating New Software Features     509

Apache Ivy     512

Chapter 19 FASTER BUILDS     515

Measuring Build System Performance     516

Measuring Performance in the Start-Up Phase     516

Measuring Performance in the Compilation Phase     526

Performance-Measurement Tools     531

Fixing the Problem: Improving Performance     534

Build Avoidance: Eliminating Unnecessary Rebuilds     535

Object File Caching     536

Smart Dependencies     539

Other Build-Avoidance Techniques     544

Parallelism     545

Build Clusters/Clouds     546

Parallel Build Tools     546

Limitations of Scalability     547

Reducing Disk Usage     548

Summary     551

REFERENCES     553

INDEX     559

Updates

Submit Errata

More Information

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020