Analyzing Data Using Spark 2.0 DataFrames With Python

Video description

Analyzing Data Using Spark 2.0 DataFrames With Python

Video description

Apache Spark 2.0 has become the gold standard for processing large datasets. This course, designed for learners with basic Python programming experience, takes you on an introductory journey into the world of big data analysis using Spark 2.0, Python, and the Spark DataFrame API.

Beginning with an overview of Spark 2.0 and Python, and then moving into a detailed examination of DataFrames, you'll learn about using SQL with DataFrames, DataFrame dates and timestamps, DataFrame aggregate operations, and about DataFrames and missing data. The course includes a hands-on data analysis exercise using real stock data. Learners should have Python and Spark installed on their computers before starting the class.

Gain a core understanding of Spark 2.0 and Spark DataFrames
Learn how to use Python with Spark DataFrames
Gain big data experience analyzing stock data with Python and Spark DataFrames

Jose Marcial Portilla is Head of Data Science at SF Bay area based Pierian Data, where he creates and delivers data science and Python training courses for Fortune 500 clients such as Credit Suisse, General Electric, and The New York Times. Jose holds degrees in Mechanical Engineering from Santa Clara University.

Publisher resources

Download Example Code

Introduction

Welcome To The Course

About The Author

Course Curriculum Overview

Overview Of Spark 2.0

What Is Spark

Why Spark 2.0 DataFrames

DataFrame Basics

Jupyter Notebook Overview

Python Review Part One

Python Review Part Two

Creating A DataFrame

Data Input

Data Output

Getting DataFrame Information

Selecting Columns And Rows

Creating and Renaming Columns

Using SQL With DataFrames

Filtering The Data

Spark DataFrame Dates And Timestamps

Introduction To Date And Timestamps

Working With Dates

Working With Timestamps

Spark DataFrame Aggregate Operations

Introduction To Aggregate And GroupBy Concepts

Spark GroupBy Method

Spark Built In Aggregate Methods

Sorting And Ordering

Spark DataFrame Working With Missing Data

Introduction To Missing Data

Dropping Data

Filling Missing Data

Spark DataFrame Exercises

Introduction To Exercises

Exercise Solutions

Thank You

What Is Next And Where To Go From Here

Start your Free Trial

Self paced

Go to the Course
We have partnered with providers to bring you collection of courses, When you buy through links on our site, we may earn an affiliate commission from provider.