Modern Data Engineering Essentials (Video Course)

Description

Almost 2 Hours of Video Instruction

This video equips you with the knowledge and skills needed to engineer data for todays large systems.

Overview

Modern Data Engineering Essentials presents the concepts and issues surrounding todays big data and AI systems. You also gain hands-on experience with data tools and workflows. The video finishes with a discussion of education, experience, and credentials needed to develop your skills in the modern data world.

Learn How To

• Apply data engineering concepts
• Load data into DuckDB
• Set up and use dbt
• Orchestrate data engineering workflows using Airflow, Dagster, and Prefect
• Put it all together integrating Dagster and dbt
• Maximize your education, experience, and skills to work in the modern data engineering world

Who Should Take This Course

Developers, data scientists, and engineers who are interested in the data side of big data systems

 Course Requirements

• Previous experience with data and data models
• Some knowledge of SQL and Python

Lesson Descriptions

Lesson 1: Modern Data Lakes

Lesson 1 lays the groundwork for the course by discussing the history of big data and the modern data landscape. It explores data warehouses and data lakes in the context of todays AI. By the end of the lesson, you will have more of a big-picture understanding of how data engineering fits into todays modern systems.

Lesson 2: Fundamentals of Database Engineering

In Lesson 2 you dive into a discussion of fundmental concepts in database enginerring: data models and data stores. This sets you up for the hands-on portion of the lesson: loading data into DuckDB, an open source, column-oriented relational database management system.

Lesson 3: Data Transformation

Lesson 3 introduces data transformation. Unfortunately, data in its raw format is typically not read for processing and analysis. It needs to be transformed into something usable, and that is where the extract, transform, and load concepts come in. Christina also discusses SQL and Python in this context. The lesson finishes up with another hands-on demonstration, introducing dbt, a tool for building, testing, and deploying robust data pipelines.

Lesson 4: Workflow Orchestration

Lesson 4 tackles workflow orchestration, the coordinating of multiple tasks across applications. Christina discusses tools such as Airflow, Dagster, and Prefect before turning to a hands-on demo on Dagster assets. She finishes up by helping you put it all together with Dagster and dbt.

Lesson 5: Future of Data Engineering

Christina finishes the video discussing where data engineering is going and helping tie that into where you are going with your data engineering career. She discusses topics like whether or not you need a degree related to computer science and/or data analytics or whether it is more important to dig in and get experience with data. She finishes up by briefly discussing next steps for you.

About Pearson Video Training

Pearson publishes expert-led video tutorials covering a wide selection of technology topics designed to teach you the skills you need to succeed. These professional and personal technology videos feature world-leading author instructors published by your trusted technology brands: Addison-Wesley, Cisco Press, Pearson IT Certification, and Que. Topics include IT Certification, Network Security, Cisco Technology, Programming, Web Development, Mobile Development, and more. Learn more about Pearson Video training at http://www.informit.com/video.

Video Lessons are available for download for offline viewing within the streaming format. Look for the green arrow in each lesson.

Sample Content

Table of Contents

Lesson 1: Modern Data Lakes

1.1 A Brief History of Big Data

1.2 The Modern Data Landscape

1.3 Data Warehouses and Lakehouses in the Context of AI

Lesson 2: Fundamentals of Database Engineering

2.1 Relational versus Non-relational Data Models

2.2 Row versus Columnar Data Stores

2.3 Hands-on DemoLoad Data into DuckDB

2.4 Design for Performance

Lesson 3: Data Transformation

3.1 Extract, Transform, Load versus Extract, Load, Transform

3.2 SQL or Python

3.3 Hands-on Demodbt Core Setup

3.4 Write Efficient Queries

Lesson 4: Workflow Orchestration

4.1 Airflow, Dagster, and Prefect

4.2 Hands-on DemoDagster Assets

4.3 Putting It All TogetherDagster Plus dbt

4.4 Alternatives

Lesson 5: Future of Data Engineering

5.1 Degree or Decree

5.2 Range: Why Generalists Will Triumph Over Specialists

5.3 Next Steps

Updates

Submit Errata