cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
cta
HDP Developer: Real-time Development

cloud Register For Upcoming Courses

Schedule

OVERVIEW

This 4 day training course is designed for developers who need to create real-time applications to ingest and process streaming data sources using Hortonworks Data Platform (HDP) and Hortonworks Data Flow (HDF) environments. Specific technologies covered include: Apache Hadoop, Apache Kafka, Apache Storm & Trident, Apache Spark and Apache HBase as well as Apache NiFi and Solr. The highlight of the course is the custom workshop-styled labs that will allow participants to build complete streaming applications with Storm and Spark Streaming.

PREREQUISITES

Students should be familiar with programming principles and have experience in software development. Java programming experience is required. SQL and light scripting knowledge is also helpful. No prior Hadoop knowledge is required.

TARGET AUDIENCE

Developers and data engineers who need to understand and develop real-time / streaming applications on HDP.

Day 1: HDP Real-Time Architecture and Components

Objectives

icon6.png

Real-time architecture & overview of the class

icon6.png

Identify the relevant HDP/HDF components

icon6.png

Kafka architecture

icon6.png

HBase architecture

Hands-on Labs

icon6.png

Creating Kafka topics from CLI and publishing & consuming messages from Java

icon6.png

Creating & accessing HBase tables from HBase shell and from Java

Day 2: Real-Time Processing with Storm

Objectives

icon6.png

Storm architecture

icon6.png

Building Storm topologies

icon6.png

Extending Storm with Trident

icon6.png

Integrating Kafka with Storm

Hands-On Labs

icon6.png

Storm WordCount

icon6.png

Integrating Kafka with Storm

icon6.png

Interactive workshop: Consuming a Kafka topic with a Storm topology and publishing results to HBase

Day 3: Real-Time Processing with Spark Streaming

Objectives

icon6.png

Spark ecosystem overview

icon6.png

RDD Programming

icon6.png

Spark Streaming

icon6.png

Integrating with Kafka

Hands-On Labs

icon6.png

Spark RDD WordCount

icon6.png

Spark Streaming WordCount

icon6.png

Interactive workshop: Consuming a Kafka topic with a Spark Streaming application and publishing results to HBase

Day 4: HDF/NiFi and Solr

Objectives

icon6.png

Compare & contrast workshop technical implementations

icon6.png

Introduction to HDF / NiFi

icon6.png

Introduction to Solr

Hands-On Labs / Demos

icon6.png

NiFi User Interface

icon6.png

Building a NiFi Data Flow

icon6.png

Demo: Indexing and searching with Solr