Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button

57th Bay Area Hadoop User Group HUG Meetup

Apache Hadoop YARN is the modern distributed operating system for big data applications. It morphed the Hadoop compute layer to be a common resource management platform that can host a wide variety of applications. Many organizations leverage YARN in building their applications on top of Hadoop without themselves repeatedly worrying about resource management, isolation, multi-tenancy issues, etc.

In this talk, we’ll start with the current status of Apache Hadoop YARN – how it is used today in deployments large and small. We’ll then move on to the exciting present and future of YARN – features that are further strengthening YARN as the first class resource management platform for data centers running enterprise Hadoop.

Speakers: Vinod Vavilapalli / Wangda Tan

HDFS’s evolution and Ozone Introduction
HDFS has several strengths: horizontal scaling of IO bandwidth over petabytes of storage. Further it provides very low latency metadata operations and scales to over 60K concurrent clients. Apache Hadoop 3.0 recently added Erasure Coding, Multiple NameNode support and HDFS federation improvements. We will talk about the latest HDFS enhancements in Apache Hadoop 3.0 and what the road ahead might look like…

One of HDFS’s limitations is scaling number of files and blocks in the system. We describe a radical change to Hadoop’s storage infrastructure with the upcoming Ozone filesystem. It allows Hadoop to scale to tens of billions of files. Ozone fundamentally separates the namespace layer and the block layers. Further, the use of RAFT protocol has allowed the storage layer to be self-consistent. We will provide a high-level overview of the Ozone architecture.

Speakers: Jitendra Pandey / Arpit Agarwal

Wednesday, August 29, 2018
Hortonworks HQ: 5470 Great America Parkway · Santa Clara, CA