Analytic Data Storage in Hadoop

Video description

Analytic Data Storage in Hadoop

Video description

In this Analytic Data Storage in Hadoop training course, expert author Ryan Blue will teach you about typical storage and ingest patterns in Hadoop. This course is designed for users that are already familiar with Hadoop.

You will start by learning how to create the dataset, load sample data, and query a dataset. From there, Ryan will teach you about partitioning, formats, and Avro. This video tutorial also covers parquet, bulk data drops, and database snapshots and mirroring. Finally, you will learn about event stream processing, including how to build a test pipeline and move to production.

Once you have completed this computer based training course, you will have gained a solid understanding of typical storage and ingest patterns in Hadoop. Working files are included, allowing you to follow along with the author throughout the lessons.

Publisher resources

Download Example Code

Introduction

Introduction And What To Expect 00:03:27

About The Author 00:04:43

Introduction To The Movielens Dataset 00:01:41

Getting Started With Hadoop

Your First Hadoop Dataset - CSV To SQL Query 00:06:45

Describing Your Data With A Schema 00:10:06

Creating The Dataset 00:10:20

Loading Sample Data 00:09:56

Querying A Dataset 00:05:58

Partitioning

Introduction to Partitioning 00:07:25

Testing Partition Strategies 00:17:13

Partitioning Patterns 00:13:17

Formats

Introduction To File Formats 00:14:02

Hadoop File Formats - Why Splitability Matters 00:05:48

Hadoop File Formats Review 00:13:44

Avro

Avro 00:03:46

Avro File Format 00:07:32

Avro Shemas 00:10:01

Avro Object Models 00:11:05

Avro Tools 00:07:44

Parquet

Parquet 00:14:57

Parquet File Formats 00:12:01

Parquet Object Models 00:05:15

Parquet Tools 00:10:24

Bulk Data Drops

Application Pattern Overview - Bulk Data Drops 00:08:34

Apache Nifi - Part 1 00:07:35

Apache Nifi - Part 2 00:06:39

Using Apache Nifi 00:05:13

Preparing The Datasets 00:11:24

Building The Data Flow - Part 1 00:15:10

Building The Data Flow - Part 2 00:14:07

Database Snapshots And Mirroring

Application Pattern Database Snapshots And Mirroring 00:06:14

Introduction To Apache Sqoop 00:08:51

Table Snapshots 00:10:57

Incremental Mirroring - Part 1 00:08:11

Incremental Mirroring - Part 2 00:10:55

Moving To Production 00:04:18

Event Stream Processing

Application Pattern - Event Streams 00:07:54

Introduction To Apache Flume 00:06:51

Building A Pipeline - Part 1 00:09:18

Building A Pipeline - Part 2 00:14:51

Moving to Production 00:14:14

Conclusion

Wrap Up 00:00:51

Start your Free Trial

Self paced

Go to the Course
We have partnered with providers to bring you collection of courses, When you buy through links on our site, we may earn an affiliate commission from provider.