Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics, offering information and knowledge of the Big Data.

cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
cta
HDP Developer: Enterprise Apache Spark I

cloud Upcoming Courses

Schedule

Overview

This course is designed as an entry point for developers who need to create applications to analyze Big Data stored in Apache Hadoop using Spark. Topics include: An overview of the Hortonworks Data Platform (HDP), including HDFS and YARN; using Spark Core APIs for interactive data exploration; Spark SQL and DataFrame operations; Spark Streaming and DStream
operations; data visualization, reporting, and collaboration; performance monitoring and tuning; building and deploying Spark applications; and an introduction to the Spark Machine Learning Library.

Duration

4 days

Format

50% Lecture/Discussion

50% Hands-on Labs

Prerequisites

Students should be familiar with programming principles and have previous experience in software development using either Python or Scala. Previous experience with data streaming, SQL, and HDP is also helpful, but not required.

Target Audience

Software engineers that are looking to develop in-memory applications for time sensitive and highly iterative applications in an Enterprise HDP environment.


Course Schedule

Hortonworks University provides an immersive and valuable real world experience in scenario-based training Courses. Our classes are available both in classroom or online, from anywhere in the world.

COURSE OBJECTIVES

At the completion of the course students will be able to:

*

Describe Hadoop, HDFS, YARN, and the HDP ecosystem

*

Describe Spark use cases

*

Explore and manipulate data using Zeppelin

*

Explore and manipulate data using a Spark REPL

*

Explain the purpose and function of RDDs


*

Employ functional programming practices


*

Perform Spark transformations and actions

*

Work with Pair RDDs

*

Perform Spark queries using Spark SQL and DataFrames

*

Use Spark Streaming stateless and window transformation

*

Visualize data, generate reports, and collaborate using
 Zeppelin


*

Monitor Spark applications using Spark History Server

*

Learn general application optimization guidelines/tips

*

Use data caching to increase performance of applications

*

Build and package Spark applications


*

Deploy applications to the cluster using YARN


*

Understand the purpose of Spark MLlib

LAB CONTENT

Labs can be performed using either Python or Scala


*

Use common HDFS commands


*

Use a REPL to program in Spark

*

Use Zeppelin to program in Spark


*

Perform RDD transformations and actions

*

Perform Pair RDD transformations and actions


*

Utilize Spark SQL

*

Perform stateless transformations using Spark Streaming


*

Perform window-based transformations


*

Use Zeppelin for data visualization and reporting

*

Monitor applications using Spark History Server

*

Cache and persist data

*

Configure checkpointing, broadcast variables, and executors


*

Build and submit a Spark application to YARN


*

Run Spark MLlib applications

Certification

The demand for Big Data skills is increasing every day. Hortonworks offers a comprehensive Certification program to help establish your credentials. Get trained, Get Certified, Get Hired!

Hortonworks University

Hortonworks University is your expert source for Apache Hadoop training and certification. Public and private on-site courses are available for IT professionals involved in implementing big data solutions.