We’re finally catching our breath after a phenomenal Hadoop Summit event last week in San Jose. Thank you to everyone that came to participate in the celebration of Hadoop advances and adoption—from many of the organizations that shared their Hadoop journey with us that fundamentally transformed their businesses, to those just getting started, to the huge ecosystem of vendors. It is amazing to be part of such a broad and deep community that is contributing to making the market for everyone.
If you didn’t have the opportunity to see them live, the keynotes have already been posted here and we’ll be posting the videos from all of the individual sessions as soon as we get them back.
In the meantime, I wanted to share some quick and simple reflections from the event.
Based on the attendance —3,100+ individuals from 1,000 unique organizations— it was a special community conference and the buzz at the event made it clear that adoption is accelerating. Vendors? Yup: 88 unique event sponsors, including Platinum Sponsors AT&T, Microsoft, SAP and Teradata—a sign that Hadoop is here to stay as a core component of the Modern Data Architecture.
And if the impromptu job board that popped up at the event is any indication, things are going to pick up pace in the coming months!
In many respects, Hadoop 2 has caused the inflection point in its adoption. Users who had deployed “traditional” Hadoop to underpin a particular application can now add applications 2, 3 and 4 on that same cluster—spanning batch, real-time and interactive use cases that are now enabled by YARN. This was certainly clear from the user panel on Day 3 entitled Hadoop in the Enterprise with representatives from AIG, BNY Mellon, British Gas / Centrica, Kohls, Rogers, Target and TrueCar —all of whom are well down the path of their Hadoop 2 journey.
Not only does YARN introduce the ability to run multiple applications on a common infrastructure, it also provides the common operating system to enable a consistent approach to plug in the core requirements of operations, data access, data management, security and governance.
In the manner that UNIX liberated us from a single-process, single-user operating system to multi-user, multi-process operating system, so does YARN liberate us from single-mode, single-load to multi-load, multi-apps data operating system.
In case you missed it, in order to simplify the on-boarding of components to run in Hadoop, we introduced a number of tools for ISV partners to get started with YARN.
And to ensure that the applications our customers use are deeply integrated, we introduced the YARN-Ready program: YARN Ready applications have been certified by Hortonworks to work with YARN—meaning that they can be safely deployed on a common cluster.
As adoption has accelerated, an even greater expectation has been placed on Security, Data Access & Data Management, Operations and Governance within the core platform. And the good news is that through both community and vendor contributions these capabilities have been added at a phenomenal pace and will make life even simpler for mainstream users.
We think this “Blueprint for Enterprise Hadoop” represents the core capabilities that are expected of any data platform, and in the case of the Hadoop ecosystem, they are being uniquely delivered in open source.
A fascinating week: if you weren’t there, stay tuned for the videos that will be posted in the coming weeks. And if you were: thank you for participating with us and hope to see you again next year!