Learning Core Audio: A Hands-On Guide to Audio Programming for Mac and iOS

Product Author Bios

Mike Lee, founder and CEO of United Lemur, has played a role in creating many of the world's most popular iPhone applications, including Obama '08, Tap Tap Revenge, Twinkle, and Jott. Prior to iPhone, he cut his teeth - and won an Apple Design Award - at Seattle-based Delicious Monster Software. Kevin Avila has spent eight years focused on audio at Apple, and may understand Core Audio better than anyone else on Earth. Nearly every iPhone application uses Avila's audio code - either from his years at Apple's Developer Technical Support, or via Apple's sample code, which he wrote.

Audio can affect the human brain in the most powerful and profound ways. Using Apple’s Core Audio, you can leverage all that power in your own Mac and iOS software, implementing features ranging from audio capture to real-time effects, MP3 playback to virtual instruments, web radio to VoIP support. The most sophisticated audio programming system ever created, Core Audio is not simple. In Learning Core Audio, top Mac programming author Chris Adamson and legendary Core Audio expert Kevin Avila fully explain this challenging framework, enabling experienced Mac or iOS programmers to make the most of it. In plain language, Adamson and Avila explain what Core Audio can do, how it works, and how it builds on the natural phenomena of sound and the human language of audio. Next, using crystal-clear code examples, they guide you through recording, playback, format conversion, Audio Units, 3D audio MIDI connectivity, and overcoming unique challenges of Core Audio programming for iOS. Coverage includes: mastering Core Audio’s surprising style and conventions; recording and playback with Audio Queue; synthesizing audio; perform effects on audio streams; capturing from the mic; mixing multiple streams; managing file streams; converting formats; creating 3D positional audio; using Core MIDI on the Mac; leveraging your Cocoa and Objective-C expertise in Core Audio’s C-based environment, and much more. When you’ve mastered the “black arts” of Core Audio, you can do some serious magic. This book will transform you from an acolyte into a true Core Audio wizard.

Downloads

Download the source code for the projects in each chapter of the book here

Related Article

A Tour of the iOS Media Frameworks

Online Sample Chapter

An Overview of Core Audio

Sample Pages

Download the sample pages (includes Chapter 1 and Index)

Table of Contents

About the Authors     xiii

Foreword  by Mike Lee  xv

Introduction     1

   Audience for This Book     2

   What You Need to Know     3

   Looking Up Documentation     3

   How This Book Is Organized     5

   About the Sample Code     9

PART I: UNDERSTANDING CORE AUDIO

Chapter 1  Overview of Core Audio     13

The Core Audio Frameworks     14

Core Audio Conventions     15

Your First Core Audio Application     16

   Running the Example     19

Core Audio Properties     22

Summary     23

Chapter 2  The Story of Sound     25

Making Waves     25

Digital Audio     27

DIY Samples     32

Buffers     40

Audio Formats     40

Summary     41

Chapter 3  Audio Processing with Core Audio     43

Audio Data Formats     43

   Example: Figuring Out Formats     46

Canonical Formats     51

Processing Audio with Audio Units     53

The Pull Model     55

Summary     55

PART II: BASIC AUDIO

Chapter 4  Recording     59

All About Audio Queues     59

Building a Recorder     60

A CheckError() Function     63

Creating and Using the Audio Queue     64

Utility Functions for the Audio Queue     71

   The Recording Audio Queue Callback     75

Summary     78

Chapter 5  Playback     81

Defining the Playback Application     81

Setting Up a File-Playing Audio Queue     83

   Setting Up the Playback Buffers     85

   Starting the Playback Queue     88

Playback Utility Functions     89

   Handling the Magic Cookie     89

   Calculating Buffer Size and Expected Packet Count     90

The Playback Audio Queue Callback     91

Features and Limits of Queue-Based Playback     94

Summary     95

Chapter 6  Conversion     97

The afconvert Utility     97

Using Audio Converter Services     100

Setting Up Files for Conversion     102

   Calling Audio Converter Services     105

   Implementing the Converter Callback     109

Converting with Extended Audio File Services     112

   Reading and Converting with Extended Audio Files     116

Summary     118

PART III: ADVANCED AUDIO

Chapter 7  Audio Units: Generators, Effects, and Rendering     123

Where the Magic Happens     123

How Audio Units Work     124

Sizing Up the Audio Units     126

Your First Audio Units     129

   Building the main() Function     131

   Creating an Audio Unit Graph     133

   Setting Up the File Player Audio Unit     137

Speech and Effects with Audio Units     141

   Building Blocks of the Speech Synthesis Graph     142

   Creating a Speech Synthesis AUGraph     144

   Setting Up a Speech Synthesizer     146

   Adding Effects     147

Adding Your Code to the Audio Rendering Process     150

   The Audio Unit Render Cycle     150

   A Custom Rendering Example     151

   Creating and Connecting Audio Units     154

   The Render Callback Function     155

Summary     160

Chapter 8  Audio Units: Input and Mixing     161

Working with I/O Input     161

   Connecting Input and Output Units     164

   Creating an AUHAL Unit for Input     168

   Writing the Input Callback     176

   Building an AUGraph to Play Samples from a CARingBuffer     178

   Writing the Play-Through App’s Render Callback     181

   Running the Play-Through Example     182

Mixing     183

Summary     189

Chapter 9  Positional Sound     191

Sound in Space     191

The OpenAL API     193

Putting a Sound in Space     196

   Setting Up the Example     197

   Using OpenAL Objects     200

   Animating the Source’s Position     205

   Loading Samples for an OpenAL Buffer     206

Streaming Audio in OpenAL     210

   Setting Up the OpenAL Streaming Example     210

   Setting Up an ExtAudioFile for Streaming     215

   Refilling the OpenAL Buffers     217

Summary     220

PART IV: ADDITIONAL TOPICS

Chapter 10  Core Audio on iOS     223

Is That Core Audio in Your Pocket?     223

Playing Nicely with Others: Audio Session Services     224

An Audio Session Example     227

   Setting Up the App     227

   Initializing the Audio Session and Audio Queue     231

   The Tone Generator Method     234 

   Handling iOS Interruptions     236

Audio Units on iOS     238

   Building an Audio Pass-Through App with the iOS RemoteIO Unit     239

   Setting Up the Pass-Through Example     241

   Setting Up the RemoteIO Audio Unit for Capture and Play-Out     244

   The RemoteIO Render Callback     249

Other iOS Audio Tricks     253

   Remote Control on iOS     253

   IOS Hardware Hazards     254

Summary     254

Chapter 11  Core MIDI     257

MIDI Concepts     257

Core MIDI     258

   Core MIDI Architecture     258

   Core MIDI Terminology     258

   Core MIDI Properties     260

   MIDI Messages     260

Instrument Units     261

Building a Simple MIDI Synthesizer     262

   Connecting to MIDI     265

Handling MIDI Notifications and Events     267 

   Playing Your AUGraph     269

Creating MIDI Events     269

   Setting Up the MIDIWifiSource Example     269

   Setting Up MIDI over Wi-Fi     271

   Sending MIDI Messages     273

   Setting Up Your Mac to Receive Wi-Fi MIDI Data     275

Summary: MIDI Mastery … but Mobility?     277

Chapter 12  Coda     279

Still More Core Audio     279

Next Steps     280

   Digital Signal Processing     280

Lion and iOS 5     281

   AUSampler     281

   Core Audio on iOS 5     285

The Core Audio Community     286

Summary: Sounds Good     287

Index     289

Errata

Download the errata

Download additional errata

Download additional errata(1/23/13)