Home > Store

Deep Reinforcement Learning and GANs LiveLessons: Advanced Topics in Deep Learning

Deep Reinforcement Learning and GANs LiveLessons: Advanced Topics in Deep Learning

Your browser doesn't support playback of this video. Please download the file to view it.

Online Video

Register your product to gain access to bonus material or receive a coupon.


  • Copyright 2018
  • Edition: 1st
  • Online Video
  • ISBN-10: 0-13-517124-5
  • ISBN-13: 978-0-13-517124-0

6+ Hours of Video Instruction

An intuitive introduction to the latest developments in Deep Learning.


Deep Reinforcement Learning and GANs LiveLessons is an introduction to two of the most exciting topics in Deep Learning today. Generative Adversarial Networks cast two Deep Learning networks against each other in a “forger-detective” relationship, enabling the fabrication of stunning, photorealistic images with flexible, user-specifiable elements. Deep Reinforcement Learning has produced equally surprising advances, including the bulk of the most widely-publicized “artificial intelligence” breakthroughs. Deep RL involves training an “agent” to become adept in given “environments,” enabling algorithms to meet or surpass human-level performance on a diverse range of complex challenges, including Atari video games, the board game Go, and subtle hand-manipulation tasks. Throughout these lessons, essential theory is brought to life with intuitive explanations and interactive, hands-on Jupyter notebook demos. Examples feature Python and Keras, the high-level API for TensorFlow, the most popular Deep Learning library.

Skill Level

  • Intermediate

Learn How To
  • Understand the high-level theory and key language around deep reinforcement learning and generative adversarial networks
  • Architect GANs that create convincing images in the style of human-drawn illustrations
  • Build deep RL agents that become adept at performing in a wide variety of environments, such as those provided by OpenAI Gym
  • Run automated experiments for optimizing deep reinforcement learning agent parameters, such as its artificial-neural-network configuration
  • Appreciate what the current limitations of “artificial intelligence” are and how they may be overcome in the near future

Who Should Take This Course
  • Perfectly suited to software engineers, data scientists, analysts, and statisticians who want to further their understanding of deep learning.
  • Code examples are provided in Python, so familiarity with it or another object-oriented programming language would be helpful.
  • Previous experience with statistics or machine learning is not necessary.

Course Requirements
  • The author’s earlier Deep Learning with TensorFlow LiveLessons, or equivalent foundational Deep Learning knowledge, are a prerequisite.

Lesson Descriptions

Lesson 1: The Foundations of Artificial Intelligence
This lesson starts off by examining what the term “AI” means and how it relates to deep learning—particularly the advanced Deep Learning topics we’re covering in these LiveLessons. It continues by discussing cutting-edge applications of generative adversarial networks and deep reinforcement learning algorithms that have recently revolutionized the field of machine learning. We then quickly review how to run the code in these LiveLessons on your own machine as well as the foundational deep learning theory that is essential for building these advanced-topics specializations upon.

Lesson 2: Generative Adversarial Networks (GANs)
Lesson 2 begins by covering the high-level theory of what GANs are and how they are able to generate realistic-looking images. Next is the “Quick, Draw!” game, which is used as the source of hundreds of thousands of hand-drawn images from a single class—such as apples, rhinoceroses, or rainbows—for a GAN to learn to imitate. The bulk of the lesson is spent developing the intricate code for the three primary components of a GAN: the discriminator network, the generator network, and the adversarial network that pits them against each other.

Lesson 3: Deep Q-Learning Networks (DQNs)
Lesson 3 is the first of three lessons that explore deep reinforcement learning algorithms. It introduces a simple game called the Cartpole Game that is used throughout the rest of the lessons to train your deep reinforcement learning algorithms. Next, the lesson delves a bit into the essential theory of deep reinforcement learning as well as DQNs, a popular type of deep reinforcement learning agent. With that theory under your belt, you’ll be able to understand at an intuitive level the code that you subsequently develop when you define your own DQN Agent and have it interact with The Cartpole Game within a handy library called OpenAI Gym.

Lesson 4: OpenAI Lab
In the previous lesson, a deep Q-learning network was used to master the Cartpole Game. This lesson builds upon those deep reinforcement learning foundations by using the OpenAI Lab both to visualize your DQN agent’s performance in real-time and to straightforwardly modify its hyperparameters. You learn how to automate the search through hyperparameters to optimize your agent’s performance and to gauge your agent’s overall fitness.

Lesson 5: Advanced Deep Reinforcement Learning Agents
The previous two lessons covered deep reinforcement learning largely through the lens of the deep Q-Learning network. In this lesson, your arsenal of deep reinforcement learning algorithms expands. The lesson begins with coverage of policy gradient algorithms, and then these are combined with Q-learning to discover the so-called actor-critic algorithms. The lesson closes out by covering why deep learning is reshaping software in general and by returning to the discussion of artificial intelligence—specifically addressing the limitations of contemporary deep learning approaches.

About Pearson Video Training

Pearson publishes expert-led video tutorials covering a wide selection of technology topics designed to teach you the skills you need to succeed. These professional and personal technology videos feature world-leading author instructors published by your trusted technology brands: Addison-Wesley, Cisco Press, Pearson IT Certification, Prentice Hall, Sams, and Que Topics include: IT Certification, Network Security, Cisco Technology, Programming, Web Development, Mobile Development, and more. Learn more about Pearson Video training at  http://www.informit.com/video.

Sample Content

Table of Contents


Lesson 1: The Foundations of Artificial Intelligence
1.1 1.1 The Contemporary State of AI
1.2 1.2 Applications of Generative Adversarial Networks
1.3 1.3 Applications of Deep Reinforcement Learning
1.4 1.4 Running the Code in these LiveLessons
1.5 1.5 Review of Prerequisite Deep Learning Theory

Lesson 2: Generative Adversarial Networks (GANs)
2.1 2.1 Essential GAN Theory
2.2 2.2 The “Quick, Draw!” Game Dataset
2.3 2.3 A Discriminator Network
2.4 2.4 A Generator Network
2.5 2.5 Training an Adversarial Network

Lesson 3: Deep Q-Learning Networks (DQNs)
3.1 3.1 The Cartpole Game
3.2 3.2 Essential Deep RL Theory
3.3 3.3 Essential DQN Theory
3.4 3.4 Defining a DQN Agent
3.5 3.5 Interacting with an OpenAI Gym Environment

Lesson 4: OpenAI Lab
4.1 4.1 Visualizing Agent Performance
4.2 4.2 Modifying Agent Hyperparameters
4.3 4.3 Fitness
4.4 4.4 Automated Hyperparameter Experimentation and Optimization

Lesson 5: Advanced Deep Reinforcement Learning Agents
5.1 5.1 Policy Gradients and the REINFORCE Algorithm
5.2 5.2 The Actor-Critic Algorithm
5.3 5.3 Software 2.0
5.4 5.4 Approaching Artificial General Intelligence



Submit Errata

More Information

Unlimited one-month access with your purchase
Free Safari Membership