Although the Hadoop Summit San Jose 2014 has come and gone, the invaluable content—keynotes, sessions, and tracks—is available here. We ’ve selected a few sessions for Hadoop developers, practitioners, and architects, curating them under Apache Hadoop YARN, the architectural center and the data operating system.
In most of the keynotes and tracks three themes resonated:
Whether your data are at rest or in motion, the demands from enterprises for data access patterns, multiple workloads, and efficient resource management have driven Hadoop’s evolution in the last two years. At the core of its evolution is YARN, a data operating system with a pluggable framework that manages resources for all the data access engines running natively on it.
YARN has fundamentally changed Hadoop. In a way, YARN’s evolution and potential reminds me of how UNIX liberated us from a single-process, single-user model to a multi-process, multi-user pluggable platform—think of shared libraries (.so) in UNIX, think of pluggable Linux kernel modules (.ko), think of myriad data applications running on top of the same platform. In this case, the platform is a Hadoop 2 cluster.
In short, YARN supports multiple programming paradigms. You are not confined only to MapReduce. Instead, you can implement purpose-built programming paradigm, implemented as YARN native apps, atop YARN for your distributed tasks.
These couple of tweets sum up the YARN sentiments:
Here are a few keynotes and sessions that speak to these themes:
|Unlocking Hadoop’s Potential||Video|
|Enterprise Hadoop for Pools, Ponds, Clouds and Beyond||Video|
|Apache Hadoop YARN: Present and Future||Video||Slides|
|YARN: The Key to Overcoming the Challenges of Broad-based Hadoop Adoption||Video||Slides|
|One Grid to Rule them All: Building a Multi-tenant Data Cloud with YARN||Video||Slides|
|Lessons Learned from Migration of a Large-analytics Platform from MPP Databases to Hadoop YARN||Video||Slides|
We cherry picked these few tracks, because they explore the potential of YARN. However, you can always peruse through all the tracks on the schedule’s session description along any time slot, on any day, that piques your curiosity. For example, when you hover and click on a session description, a popup will display in which you can either elect to watch the video or view the slides.
You can learn more about YARN by