Hadoop Summit Reflections

Thank you for your support

We’re finally catching our breath after a phenomenal Hadoop Summit event last week in San Jose.  Thank you to everyone that came to participate in the celebration of Hadoop advances and adoption—from many of the organizations that shared their Hadoop journey with us that fundamentally transformed their businesses, to those just getting started, to the huge ecosystem of vendors. It is amazing to be part of such a broad and deep community that is contributing to making the market for everyone.

If you didn’t have the opportunity to see them live, the keynotes have already been posted here and we’ll be posting the videos from all of the individual sessions as soon as we get them back.

In the meantime, I wanted to share some quick and simple reflections from the event.

Momentum, Momentum, Momentum

Based on the attendance —3,100+ individuals from 1,000 unique organizations— it was a special community conference and the buzz at the event made it clear that adoption is accelerating.  Vendors? Yup: 88 unique event sponsors, including Platinum Sponsors AT&T, Microsoft, SAP and Teradata—a sign that Hadoop is here to stay as a core component of the Modern Data Architecture.

And if the impromptu job board that popped up at the event is any indication, things are going to pick up pace in the coming months!


YARN Enables a Modern Data Architecture

An Apache Hadoop platform has always referred to a collection of Apache Projects, and YARN—the key addition in Hadoop 2.x —has clearly become the architectural center of Hadoop.

In many respects, Hadoop 2 has caused the inflection point in its adoption. Users who had deployed “traditional” Hadoop to underpin a particular application can now add applications 2, 3 and 4 on that same cluster—spanning batch, real-time and interactive use cases that are now enabled by YARN.  This was certainly clear from the user panel on Day 3 entitled Hadoop in the Enterprise with representatives from AIG, BNY Mellon, British Gas / Centrica, Kohls, Rogers, Target and TrueCar —all of whom are well down the path of their Hadoop 2 journey.

Not only does YARN introduce the ability to run multiple applications on a common infrastructure, it also provides the common operating system to enable a consistent approach to plug in the core requirements of operations, data access, data management, security and governance.

In the manner that UNIX liberated us from a single-process, single-user operating system to multi-user, multi-process operating system, so does YARN liberate us from single-mode, single-load to multi-load, multi-apps data operating system.

In case you missed it, in order to simplify the on-boarding of components to run in Hadoop, we introduced a number of tools for ISV partners to get started with YARN.


And to ensure that the applications our customers use are deeply integrated, we introduced the YARN-Ready program: YARN Ready applications have been certified by Hortonworks to work with YARN—meaning that they can be safely deployed on a common cluster.

The Blueprint for Enterprise Hadoop

As adoption has accelerated, an even greater expectation has been placed on Security, Data Access & Data Management, Operations and Governance within the core platform.  And the good news is that through both community and vendor contributions these capabilities have been added at a phenomenal pace and will make life even simpler for mainstream users.

We think this “Blueprint for Enterprise Hadoop” represents the core capabilities that are expected of any data platform, and in the case of the Hadoop ecosystem, they are being uniquely delivered in open source.


What’s Next?

A fascinating week: if you weren’t there, stay tuned for the videos that will be posted in the coming weeks.  And if you were: thank you for participating with us and hope to see you again next year!

Categorized by :
Administrator CIO & ITDM Data Analyst & Scientist Developer Hadoop in the Enterprise Modern Data Architecture New Features YARN

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Join the Webinar!

Big Data Virtual Meetup Chennai
Wednesday, October 29, 2014
9:00 pm India Time / 8:30 am Pacific Time / 4:30 pm Europe Time (Paris)

More Webinars »

HDP 2.1 Webinar Series
Join us for a series of talks on some of the new enterprise functionality available in HDP 2.1 including data governance, security, operations and data access :
Contact Us
Hortonworks provides enterprise-grade support, services and training. Discuss how to leverage Hadoop in your business with our sales team.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.