Register your product to gain access to bonus material or receive a coupon.
This eBook includes the following formats, accessible from your Account page after purchase:
EPUB The open industry format known for its reflowable content and usability on supported mobile devices.
PDF The popular standard, used most often with the free Acrobat® Reader® software.
This eBook requires no passwords or activation to read. We customize your eBook by discreetly watermarking it with your name, making it uniquely yours.
The Example-Rich, Hands-On Guide to Data Munging with Apache HadoopTM
Data scientists spend much of their time “munging” data: handling day-to-day tasks such as data cleansing, normalization, aggregation, sampling, and transformation. These tasks are both critical and surprisingly interesting. Most important, they deepen your understanding of your data’s structure and limitations: crucial insight for improving accuracy and mitigating risk in any analytical project.
Now, two leading Hortonworks data scientists, Ofer Mendelevitch and Casey Stella, bring together powerful, practical insights for effective Hadoop-based data munging of large datasets. Drawing on extensive experience with advanced analytics, the authors offer realistic examples that address the common issues you’re most likely to face. They describe each task in detail, presenting example code based on widely used tools such as Pig, Hive, and Spark.
This concise, hands-on eBook is valuable for every data scientist, data engineer, and architect who wants to master data munging: not just in theory, but in practice with the field’s #1 platform–Hadoop.
Coverage includes
Data Munging with Hadoop is part of a larger, forthcoming work entitled Data Science Using Hadoop. To be notified when the larger work is available, register your purchase of Data Munging with Hadoop at informit.com/register and check the box “I would like to hear from InformIT and its family of brands about products and special offers.”
For more information about the larger, forthcoming book entitled Data Science with Hadoop, visit the following: http://www.informit.com/store/data-science-with-hadoop-9780134024141
Preface vi
About the Authors viii
Data Munging with Hadoop 1
Why Hadoop for Data Munging? 2
Data Quality 2
What Is Data Quality? 2
Dealing with Data Quality Issues 3
Using Hadoop for Data Quality 8
The Feature Matrix 9
Choosing the “Right” Features 10
Sampling: Choosing Instances 10
Generating Features 12
Text Features 13
Time-Series Features 16
Features from Complex Data Types 17
Feature Manipulation 18
Dimensionality Reduction 19
Summary 22