Register your product to gain access to bonus material or receive a coupon.
Cleaning Data for Effective Data Science (Video Course)
- By David Mertz
- Published Jun 6, 2025 by Addison-Wesley Professional.
Online Video
- Your Price: $319.99
- List Price: $399.99
- About this video
Video accessible from your Account page after purchase.
Description
- Copyright 2025
- Edition: 1st
- Online Video
- ISBN-10: 0-13-545408-5
- ISBN-13: 978-0-13-545408-4
4+ Hours of Video Instruction
Walks you through the real work of data science--getting to clean data
Overview:
Cleaning Data for Effective Data Science introduces you to the first stage in the data science process--establishing useful data. This can often be the most important, as well as the most time-consuming, stage in the data science process.
Learn How To:
- Distinguish between different types of data formats
- Work with tabular data formats
- Work with hierarchical data formats
- Extract and ingest data from repurposed sources
- Detect and mark anomalous data
- Assess data quality
- Remediate missing or problematic data
- Identify and address outliers
- Impute plausible values for missing or unreliable data
- Utilize sampling principles
Who Should Take This Course:
- Developers, data scientists, and engineers who are interested in improving the quality of datasets
Course Requirements:
- Proficiency in any programming language used for data processing and machine learning
Lesson Descriptions:
Lesson 1: Data Ingestion--Tabular Formats
Lesson 1 is about tabular data formats. In most data science, the data we wish to work with has a tabular format, ideally a so-called tidy format. Sometimes that data starts out in physical formats, which are basically tabular, but many formats have characteristic pitfalls. CSV or fixed width data starts as plain text and often has data typing ambiguities. Spreadsheets are used to record data but are also prone to problems when performing more precise analysis. Other more specialized tabular formats avoid most of the problems but are used less often.
Lesson 2: Data Ingestion--Hierarchical Formats
Lesson 2 looks at the common hierarchical formats XML and JSON, and within NoSQL databases. A first goal is to convert the underlying data to a more tabular representation. But the specific formats also each have some characteristic dangers. XML is fairly complex though good tools exist for pulling out the data points from XML documents or streams. JSON has become ubiquitous and is generally a simple and useful format. Sometimes JSON takes the form of JSON lines, which represents many small documents streamed in sequence. JSON is loosely structured, which often merits using JSON schema to create more controlled hierarchies. Data also lives in NoSQL databases, sometimes called document databases. This lesson looks at utilizing data from those sources.
Lesson 3: Data Ingestion--Repurposing Data Sources
Sometimes data is embedded inside web pages, PDF documents, or within images. These common formats require initial work to pull out information for analytic visualization or modeling purposes. This lesson covers useful techniques for data extraction.
Lesson 4: Anomaly Detection
Collected data often contains anomalous data points. Detecting and marking these problem data is an important element of the data science pipeline. Data are sometimes simply missing, though how varies considerably among formats and sources. This lesson examines several ways absence is marked, including a discussion of various sentinels that are often used and how to work with those. Although sentinels can require work to recognize and transform, a trickier question arises with data points that are detectably wrong but have the general form of good data. These bad data points can sometimes result from direct miscoding. Other times, problems can be deduced based on known bounds of a data field or by extreme variance from expectations within the data. Each of these ways that data can go wrong is discussed in a sub lesson, as well as simple outliers, which can be identified by relatively easy statistical tests. Sometimes outliers need multivariate analysis to locate. The final sub lesson of this lesson addresses multivariate outliers.
Lesson 5: Data Quality
This lesson looks for systematic trends in the availability of data. Many trends can be characterized in statistical bias, which may result from sample bias. Sometimes bias is detectable from data distributions and at other times from domain knowledge that provides expectations. Benford's Law is also discussed as a mechanism for detecting a certain kind of bias. Remediation is addressed as well, including class imbalance in datasets. This may result from bias or may reflect the underlying distribution of data but nonetheless warrant sample weighting. A final sub lesson looks at normalization and scaling, which is also helpful in preparing data for many kinds of analysis and modeling. This lesson provides the tools to create a well-rounded data set for final steps in your data science pipeline.
Lesson 6: Value Imputation
The final lesson discusses value imputation. Earlier lessons may have identified or marked data points as missing or unreliable. Often a follow up to such marking is imputing new plausible values to that missing data. The sub lessons look first at imputation of typical values that might be either global or based on the parameter space locality. As a more sophisticated technique, imputation might also reflect trends identified within the data. Many trends are temporal, with timestamped data points tending to resemble those with similar timestamps. However, many trends are also non temporal, both by spatial location and following more abstract continuities. The final sub lesson reviews oversampling and undersampling. Sampling is a mechanism to impute data based on the data we already have. Oversampling can use techniques more nuanced than mere duplication of existing data rows.
About Pearson Video Training
Pearson publishes expert-led video tutorials covering a wide selection of technology topics designed to teach you the skills you need to succeed. These professional and personal technology videos feature world-leading author instructors published by your trusted technology brands: Addison-Wesley, Cisco Press, Pearson IT Certification, Sams, and Que Topics include IT Certification, Network Security, Cisco Technology, Programming, Web Development, Mobile Development, and more. Learn more about Pearson Video training at http://www.informit.com/video.
Video Lessons are available for download for offline viewing within the streaming format. Look for the green arrow in each lesson.
Sample Content
Table of Contents
Introduction
Lesson 1: Data Ingestion--Tabular Formats
1.1 CSV
1.2 Spreadsheets Considered Harmful
1.3 Other Formats
Lesson 2: Data Ingestion--Hierarchical Formats
2.1 XML
2.2 JSON
2.3 NoSQL Databases
Lesson 3: Data Ingestion--Repurposing Data Sources
3.1 Web Scraping
3.2 Portable Document Format
3.3 Image Formats
Lesson 4: Anomaly Detection
4.1 Missing Data
4.2 SQL
4.3 Hierarchical Formats
4.4 Sentinels
4.5 Miscoded Data
4.6 Fixed Bounds
4.7 Outliers
Lesson 5: Data Quality
5.1 Missing Data
5.2 Biasing Trends
5.3 Benford's Law
5.4 Class Imbalance
5.5 Normalization and Scaling
Lesson 6: Value Imputation
6.1 Typical Value Imputation
6.2 Trend Imputation
6.3 Sampling
Summary
More Information
Other Things You Might Like
Python Data Visualization: Create impactful visuals, animations and dashboards (Video Course)
- Online Video $239.99