Until now multimedia developers had to program directly to hardware in order to maximize application performance. DirectX, RDX, RSX, and MMX technology are new advancements that enable programmers to write applications that take advantage of hardware acceleration without direct hardware programming.
Written by Intel experts who are developing and applying these new technologies, DirectXAE, RDX, RSX, and MMXoTechnologyo: A Jumpstart Guide to High Performance APIs takes a hands-on approach to illustrate the latest technologies from Microsoft, Intel, and Progressive Networks. This book:
If have have any questions, concerns or suggested errata, please feel free to E-mail the author, Mike Hawash at firstname.lastname@example.org
I. SURVEYING MULTIMEDIA.1. Overview of Media on the PC.
Graphics Device Independence.
Motion Video under Windows.
Multimedia Gaming under Windows 95.
3D Video Architectures on the PC.
Audio Architectures on the PC.2. Processor Architecture Overview.
II. SPRITES, BACKGROUNDS, AND PRIMARY SURFACES.3. Simple Sprites in GDI.
Graphics Device Interface (GDI) Overview.
Transparent Blts with GDI.
How Fast Does GDI Draw Sprites and Backgrounds?4. Sprites with a DirectDraw Primary Surfaces.
Introduction to Microsoft’s DirectDraw.
Features of DirectDraw.
Before You Get Overly Excited.
Instantiating a DirectDraw Object.
Querying and Creating a Primary Surface.
Implementing a Simple Sprite Class.
Drawing a Sprite on the DirectDraw Primary Surface.
Redrawing Backgrounds on a DirectDraw Primary Surface.
How Fast Can We Draw Sprites and Backgrounds?
Compositing Objects on a DirectDraw Primary Surface.5. Hardware Acceleration via DirectDraw.
Creating an Offscreen Surface.
Drawing a Sprite on the DirectDraw Offscreen Surface.
How Fast is OffScreen Surface Drawing?
Finding Hardware Acceleration.
Setting Up for Hardware Acceleration.
How Fast is CSurfaceVidMem Drawing?
Accelerating Offscreen to Primary Transfers by Page Flips.
What is Graphics Page Flipping?
DirectDraw Page Flipping Model.
Does the Hardware Support Page Flipping?
Setting Up DirectDraw to Use Page Flipping.
“Rendering” Flippable Surfaces.
How Fast is CSurfaceBackBuffer Drawing?
Hardware Acceleration to Blt Sprites.
How Fast is CSpriteGrfx (and CBackgroundGrfx) Drawing?6. RDX: High-Performance Mixing with a High-Level API.
Introduction to Intel’s RDX Animation Library.
Features of RDX.
Before You Get Overly Excited.
Generic Objects with RDX.
The Programming Model.
Working with RDX.
Creating an RDX Surface.
An RDX Sprite Class.
Drawing the RDX Sprite.
How Fast Does CSurfaceRdx Draw?
Hardware Acceleration with RDX.
Full Screen Mode with RDX.
How Fast Does CSurfaceRdx Draw in Full Screen Mode?
Accelerating Objects with RDX.
III. MAKING THE MEDIA MIX.7. Video under Windows.
Concepts of Motion Video.
Capturing and Compressing Video.
Windows Multimedia Architectures.
Overview of Video Codecs.8. DirectShow Filters.
What’s a Filter Graph?
Understanding the Mighty Filter.
An Overview on the Samples.
Creating a Source Filter.
The Source Filter Class.
Create an Instance of the Source Filter.
The Source Stream Class.
The Connection Process.
Starting and Stopping.
Moving the Data.
Creating a Transform Filter.
Creating a Rendering Filter.
Adding Your Own Interface.
Adding Property Pages to Filters.
Adding the Property Interface to the Filter.
Implementing the Property Page Interface.
Adding a Filter to the Registry.
Using a Registry File is Not Recommended.
Using Filter Self-Registration is Recommended.9. DirectShow Applications.
DirectShow Mechanisms for Working on Filter Graphs.
COM: Automatic Construction of Filter Graphs.
COM: Manual Construction of Filter Graphs.
Adding Source Filters.
Adding Transform and Rendering Filters.
Connecting the Two Pins.
COM: Accessing Custom Interfaces.
COM: Showing Filter Property Pages.
Creating Events under DirectShow.
ActiveX: A Simple Way to Control DirectShow.
Playing a File Using the ActiveX Interface.
Controlling the ActiveX Control from Your Application.
ActiveX: Handling Events.10. Mixing Sprites, Backgrounds, and Videos.
Introduction to Mixing.
Mixing Sprites with Video.
Mixing Animation with Video.
Mixing with RDX.
Playing Video with the RDX DirectShow Interface.
Mixing a Sprite on Top of Video.
Mixing Video on Video.11. Streaming down the Superhighway with RealMedia.
Overview of RealMedia.
The RealMedia Plug-in Architecture.
Data Flows: Server to Client.
Data Management Objects.
IRMABuffer: Dynamic Memory Allocation Object.
IRMAValues: Indexed List Object.
IRMAPacket: Packet Transport Object.
RealMedia Asynchronous Interfaces.
Common Requirements for All Plug-ins.
Building a File-Format Plug-in.
Initializing the File-Format Plug-in.
File and Stream Headers.
Let the Streaming Begin!
Building a Rendering Plug-in.
RealMedia Audio Services.
Playing a Simple Pulse Coded Modulation (PCM) Audio File.
Pump Up the Volume.
IV. PLAYING AND MIXING SOUND WITH DIRECTSOUND AND RSX 3D.12. Audio Mixing with DirectSound.
Overview of Audio under Windows 95.
Playing a WAV File Using DirectSound.
Creating Sound Buffers.
Playing the Sound.
Mixing Two WAV Files.
Controlling the Primary Sound Buffer.
Initializing to Get Control of the Output Format.
Creating a Primary DirectSound Buffer.
Demo Time.13. Realistic 3D Sound Experience: RSX 3D.
RSX 3D Features.
Creating an RSX 3D Object.
Creating the Cached Emitter and the Direct Listener Objects.
Play One WAV File.
Mixing Many WAV Files.
RSX Goes 3DNTrue 3D Sound.
Setting Up 3D Sound with RSX 3D.
Adding Special Sound Effects with RSX 3D.
The Doppler Effect.
The Reverberation Effect.
Audio Streaming in RSX 3D.
V. WELCOME TO THE THIRD DIMENSION.14. An Introduction to Direct3D.
Some Background on 3D on the PC.
Introduction to Direct3D.
A Taste of Direct3D’s Retained Mode.
Direct3D’s Immediate Mode.
Before You Get Overly Excited.
Direct3D and DirectDraw.
Direct3D Rendering Engine.
Revving Up Direct3D.
The Starting Point: IDirect3D Object.
Creating an IDirect3DDevice.
Preparing a Palette.
Extending the Surface for 3D.
Mapping from a 3D Model to the 2D Surface Using Viewports.
Talking to 3D Devices Through Execute Buffers.
Operations Used to Render a Simple Triangle.
Executing the Execute Buffers.
Seeing Results from 3D Devices.
Demo Time.15. Embellishing Our Triangle with Backgrounds, Shading, and Textures.
Continuing Our Look into Direct3D.
Repainting the Background Using Direct3D.
Looking at Direct3D Materials.
Creating a Direct3D Background.
Bltting a Direct3D Background.
Controlling Shading Options.
Looking at Some Render States and Their Default Values.
Coloring a Pixel in Direct3D.
Shading with the RGB Color Model.
Shading with the Ramp Color Model.
Changing Default Render States.
Texture Mapping with Direct3D.
Creating a Texture Map.
Setting Up Triangle Vertices for Texture Mapping.
Setting Up Render Operations for Texture Mapping.
Handling “Lit” Texture Maps.
Z-Buffering with Direct3D.
Why Bother with Z-Buffering?
Setting Up for Z-Buffering.16. Understanding and Enhancing Direct3D Performance.
How Fast Does Our Triangle Run?
Stages of Rendering Our Triangle.
Measuring the Rendering Stages of Our Triangle.
Trimming Some Fat from the Rendering Stages.
Measuring Shading Options.
Measuring the Performance of Shading Options in Our Triangle.
Measuring the Performance of Texture-Mapping in Our Triangle.
Adding a Z-Buffer to the Recipe.
Getting Perspective: Comparing 3D (RGB Mode) to 2D.
Improving Performance Using the Ramp Driver.
Loading the Ramp Color Model Driver.
Using the Ramp DriverNThe First Try.
Creating Materials for the Ramp Driver.
Rendering a Triangle with the Ramp Driver.
How Does the Ramp Driver Perform?
Optimizing Texture Mapping.17. Mixing 3D with Sprites, Backgrounds, and Videos.
Mixing a 3D Object on a 2D Background.
Our 3D Surface is Also a 2D Surface.
Measuring Background Performance.
Mixing in Sprites.
Using RDX to Mix in Sprites.
Adding RDX Objects at Front and Back.
Mixing in Video.
Using Video as a Texture Map.
VI. PROCESSORS AND PERFORMANCE OPTIMIZATION.18. The Pentium Processor Family.
Basic Concepts and Terms.
The Pentium Processors.
The Pentium Processor.
The Pentium Pro Processor.
The Pentium Processor with MMX Technology.
The Pentium II Processor.
Identifying Processor Models.19. The Pentium Processor.
Instruction and Data L1 Caches.
Branch Prediction and the Branch Target Buffer.
A Closer Look at the BTB.
Dual Pipelined Execution.
Pentium Integer Pairing Rules.
Address Generation Interlock (AGI).
Revisiting Our Sprite Sample.
Overview of the Assembly Version of CSprite.
Analyzing the Performance of Our Sprite Sample.
Do I Really Need to Schedule My Code?20. The Pentium Processor with MMX Technology.
A Look at MMX Technology.
The Pool of Four Write Buffers.
MMX Uses Floating-Point Registers.
EMMS to the Rescue: How to Mix MMX and FP Instructions.
MMX Technology Data Types.
The MMX Instruction Set.
Using MMX Technology to Render Our Sprite Sample.
MMX Technology Optimization Rules and Penalties.
MMX Exceptions to General Pentium Rules.
MMX Instruction Pairing Rules.
MMX Instruction Scheduling Rules.
Performance Analysis of Our Sprite.
MMX versus Integer Implementation of the Sprite.21. VTune and Other Performance Optimization Tools.
Overview of Performance Counters.
VTune Static Analysis.
Tune Dynamic Analysis.
Systemwide MonitoringNTime-and Event-Based Sampling.
Read Time Stamp Counter.
The PMonitor Event Counter Library.22. The Pentium II Processor.
The Life Cycle of an Instruction on the Pentium II.
Comparing the Pentium II with the Pentium Pro Processor.
Comparing the Pentium II with the Pentium with MMX Technology Processor.
Instruction and Data Caches.
Instruction Fetch Unit.
Fetch Performance with Event Counters.
Branch Prediction and the Branch Target Buffer.
Branch Performance with Event Counters.
Register Alias Table Unit.
Reorder Buffer and Execution Units.
Rendering Our Sprite on the Pentium II.
Speed Up Graphics Writes with Write Combining.
Performance Considerations.23. Memory Optimization: Know Your Data.
Overview of the Memory Subsystem.
Memory Pages and Memory Access Patterns.
Architectural Differences among the Pentium and Pentium Pro Processors.
Architectural Cache Differences.
Write Buffer Differences.
Data Controlled Unit Splits on the Pentium Pro Processor.
Partial Memory Stalls.
Maximizing Aligned Data and MMX Stack Accesses.
The Pitfalls of Unaligned MMX Stack Access.
Accessing Cached Memory.
Writing to Video Memory.
Using Aligned Accesses to Video Memory.
Spacing Out Writes to Video Memory with Write Buffers.The Finale.
The Spiral Continues.
The Hardware Spiral.
The Software Spiral.
Remote Multimedia (a.k.a. Internet Multimedia).
Multimedia on the Internet.
Evolving Hardware for the Internet.
Better, Faster, Cheaper 3D.
3D Hardware Spiral.
3D Software Spiral.
Emerging Application Areas.
Multimedia in the Home.
Demo Time.Index. 0201309440T04062001
Why Read This Book?There's Lots of New Stuff to Learn
In the past few years, the pace of technology growth has been exhilarating. Microsoft launched Windows 95. Intel debuted the Pentium, Pentium Pro, and MMX technology processors. Netscape burst the Internet pipe with a new class of applications and architectures. These companies and others paraded out a slew of new multimedia architectures. And you've never before felt so lost in space.
Maybe, you're familiar with programming for Windows 95 and now want to deliver Windows 95 multimedia applications, and you're wondering where to start. Or maybe, you've programmed multimedia for DOS/Windows 3.1, and now you're scrambling to learn Windows 95, learn the new computing environment, and then learn to deliver high-performance multimedia in this environment.
Well, several new architectures have been introduced to help you deliver high-performance multimedia under Windows 9x, such as DirectDraw*, DirectSound*, Direct3D*, DirectShow*, RealMedia*, Realistic Sound Experience (3D RSX), Realistic Display Mixer (RDX), and so forth. But now you've got to learn these new architectures, and you've got this steep learning curve on your hands.
On the hardware frontier, the power of personal computers has increased at a dramatic paceNboth in processor and peripheral power. The Pentium, Pentium Pro, Pentium II, and MMX Technology processors, the Accelerated Graphics Port (AGP) bus, and the various graphics hardware accelerators are recent hardware advancements that affect multimedia performance. Surely your applications would sizzle if you mastered these advancements. But mastering these advancements only increases the learning curve.
And, of course, the Internet adds yet another dimension to the puzzle. The new programming space includes Internet browsers and their plug-ins; programming languages such as Java, HTML, and VRML; Internet architectures such as ActiveX, RealMedia, and a huge list of applications such as Internet Phones and Chat Worlds. More to learn, more to wade through, more time to spend.Lightening the Learning Burden
As multimedia developers, we constantly investigate, evaluate, or learn these new technologies. Our typical sources are technical reference manuals and sample applications. With so many recent products, we've got a huge quantity of material to wade through. When time is precious, as it invariably is, just getting started can be an overwhelming problem. Spending time getting started eats away from time allocated for finishing touches and product testing. And overall quality suffers when we've spent too much time just getting up to speed.
Wouldn't it be nice, if there were a simple way to just get started? To grasp the bare essentials and leave the esoteric stuff for on-the-job training (those need-to-know moments)? To steer clear of performance pitfalls? Well, do we have a deal for you. We, the authors, have been involved in various aspects of multimedia development on the PC for five long years. Through our employment at Intel and through our relationships with Microsoft and other key players, we've had the privilege to influence the architectures of processors, peripherals, platforms, and software components toward the betterment of multimedia on the PC. During that time, we've done our fair share of defining, reviewing, and implementing numerous multimedia architectures, both software and hardware.
With this book, we hope to use our internal vantage point to give you a jump start to high-performance multimedia development for Windows 9x. We'd like to help you cut to the chase; focus on the bare necessities; stick to the essentials; and jump-start a variety of offerings. What's more, we're hoping to take you a step beyond getting startedNto extracting performance.
We hope to provide you with a quick start to a wide spectrum of multimedia advancements for Windows 9x. We hope to answer questions like Where do I start? What do I really need? How little can I get away with? How do I get it to run faster?
A dose of caution: there's more than one way to get jump-started and more than one way to extract performance. We'll share our experiences with you, show you "a" way. We hope you'll come away with some tricks, of course, but more important, we hope you'll come away with a thought processNan approach.
We've tried to maintain a light flavor. We hope you'll have some fun along the way.