Hadoop Summit Reflections

Thank you for your support

We’re finally catching our breath after a phenomenal Hadoop Summit event last week in San Jose.  Thank you to everyone that came to participate in the celebration of Hadoop advances and adoption—from many of the organizations that shared their Hadoop journey with us that fundamentally transformed their businesses, to those just getting started, to the huge ecosystem of vendors. It is amazing to be part of such a broad and deep community that is contributing to making the market for everyone.

If you didn’t have the opportunity to see them live, the keynotes have already been posted here and we’ll be posting the videos from all of the individual sessions as soon as we get them back.

In the meantime, I wanted to share some quick and simple reflections from the event.

Momentum, Momentum, Momentum

Based on the attendance —3,100+ individuals from 1,000 unique organizations— it was a special community conference and the buzz at the event made it clear that adoption is accelerating.  Vendors? Yup: 88 unique event sponsors, including Platinum Sponsors AT&T, Microsoft, SAP and Teradata—a sign that Hadoop is here to stay as a core component of the Modern Data Architecture.

And if the impromptu job board that popped up at the event is any indication, things are going to pick up pace in the coming months!

billboard

YARN Enables a Modern Data Architecture

An Apache Hadoop platform has always referred to a collection of Apache Projects, and YARN—the key addition in Hadoop 2.x —has clearly become the architectural center of Hadoop.

In many respects, Hadoop 2 has caused the inflection point in its adoption. Users who had deployed “traditional” Hadoop to underpin a particular application can now add applications 2, 3 and 4 on that same cluster—spanning batch, real-time and interactive use cases that are now enabled by YARN.  This was certainly clear from the user panel on Day 3 entitled Hadoop in the Enterprise with representatives from AIG, BNY Mellon, British Gas / Centrica, Kohls, Rogers, Target and TrueCar —all of whom are well down the path of their Hadoop 2 journey.

Not only does YARN introduce the ability to run multiple applications on a common infrastructure, it also provides the common operating system to enable a consistent approach to plug in the core requirements of operations, data access, data management, security and governance.

In the manner that UNIX liberated us from a single-process, single-user operating system to multi-user, multi-process operating system, so does YARN liberate us from single-mode, single-load to multi-load, multi-apps data operating system.

In case you missed it, in order to simplify the on-boarding of components to run in Hadoop, we introduced a number of tools for ISV partners to get started with YARN.

YARN-Services-APIs

And to ensure that the applications our customers use are deeply integrated, we introduced the YARN-Ready program: YARN Ready applications have been certified by Hortonworks to work with YARN—meaning that they can be safely deployed on a common cluster.

The Blueprint for Enterprise Hadoop

As adoption has accelerated, an even greater expectation has been placed on Security, Data Access & Data Management, Operations and Governance within the core platform.  And the good news is that through both community and vendor contributions these capabilities have been added at a phenomenal pace and will make life even simpler for mainstream users.

We think this “Blueprint for Enterprise Hadoop” represents the core capabilities that are expected of any data platform, and in the case of the Hadoop ecosystem, they are being uniquely delivered in open source.

blueprint

What’s Next?

A fascinating week: if you weren’t there, stay tuned for the videos that will be posted in the coming weeks.  And if you were: thank you for participating with us and hope to see you again next year!

Categorized by :
Administrator CIO & ITDM Data Analyst & Scientist Developer Hadoop in the Enterprise Modern Data Architecture New Features YARN

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Join the Webinar!

YARN Ready: Developing Applications on Hadoop with Scalding
Thursday, September 18, 2014
12:00 PM Eastern / 9:00 AM Pacific

More Webinars »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.