9 - ** Learning Spark **

 Casey

 Amish

1-3

4

5

6

7

8

9

10

11

12

3

For converting DBC files to ipynb (jupyter notebook) files, Amish found the following solution via a google search during our meetup - Convert Databricks DBC notebook format to ipynb Not tested yet, but it looks good. The book recommends that you sign up for the Databricks Community Edition, but Patrick and I had found that the website forces you to a 14-day free trial instead now. Hopefully this conversion solution will allow us to convert the dbc file that is in the book’s code repo so that we can work with the Jupyter notebooks independently.
Upon further investigation, this conversion requires a Databricks personal access token, which in turn requires you to sign up for the Databricks service.

Spark: What’s Underneath an RDD?
Structuring Spark
The DataFrame API
- Spark’s Basic Data Types
- Spark’s Structured and Complex Data Types
- Schemas and Creating DataFrames
- Columns and Expressions
- Rows
- Common DataFrame Operations
- End-to-End DataFrame Example
The DataSet API
DataFrames Versus Datasets
Spark SQL and the Underlying Engine

Casey, Amish, Patrick (discussion of the chapter as a whole)

The Genesis of Spark
What is Apache Spark

Casey

Unified Analytics
The Developer’s Experience

Amish

Step 1: Downloading Apache Spark
Step 2: Using the Scala or PySpark Shell
Step 3: Understanding Spark Application Concepts

Amish

Transformations, Actions, and Lazy Evaluations
The Spark UI

Casey

Your First Standalone Application

Casey