Build YARN Apps on Hadoop with Apache Slider: Technical Preview Now Available

Apache Slider enables on-boarding, provisioning, and managing YARN based apps with greater ease

A significant reason for the increased adoption of the Hortonworks Data Platform by customers and partners has been Apache Hadoop YARN. This major advance, introduced last October in HDP2, allows Hadoop to move from many single-purpose clusters to a versatile, integrated data platform that hosts multiple business applications.

YARN has become the architectural center of Hadoop. We intend to make it easier for applications to work in a YARN environment, and benefit from the integrated capabilities and technologies that form the blueprint for enterprise Hadoop. For that reason, we’re very excited about Apache Slider.

Introducing Apache Slider

In April, Hortonworks submitted Slider to the Apache Software Foundation as an incubator project. Two months later, we’re happy to make Slider available as a Technical Preview.

Slider provides a set of platform services that enables  other applications to quickly run in the YARN environment. It provides four key benefits:

    1. Simplified on-boarding of existing apps to Hadoop YARNWith Slider, distributed applications that aren’t YARN-aware can now “slide into YARN” to run on Hadoop – usually with no code changes.
    2. Full capabilities of a YARN applicationSlider allows applications to be provisioned, started, stopped, restarted on failure, monitored, expanded & shrunk, all while the application is running. It allows users to create and run multiple instances of applications, even with different application versions if needed.
    3. Integrated participation in enterprise HadoopSlider allows applications to participate in a Hadoop ecosystem in an integrated and cooperative way, leveraging Hadoop’s data and processing resources, as well as benefiting from the security, governance, and operations capabilities of enterprise Hadoop.
    4. Automated lifecycle managementSlider automatically makes applications manageable through Apache Ambari without any additional work.

Slider allows long-running applications, real-time services and online applications to easily integrate into a Hadoop environment. It complements Apache Tez, which is quickly gaining adoption as the batch and interactive engine of Hadoop. These engines, depicted in the diagram below, provide targeted mechanisms for developers to create YARN-based applications.

slider

Running HBase, Accumulo and Storm under YARN

Along with the Slider technical preview, we are including three Apache projects as samples for Slider:

  • Apache HBase
  • Apache Accumulo
  • Apache Storm

Customers who have been eager to run these projects within their Hadoop cluster are encouraged to try them out. For developers, we’ve included the setup and configuration information for each of these projects to assist in understanding how to “slide” existing distributed applications onto YARN.

Managing Slider Applications

As Slider brings new applications to the Hadoop ecosystem, the applications will need to be managed – not just by YARN, but also by Hadoop operators and administrators. The answer, of course, is to leverage the open source Apache Ambari project. Slider automatically makes the application manageable by Ambari without further work on the application’s part. Today we’re also releasing a Slider View for Ambari Technical Preview that can manage Slider applications.

Innovating the Core of Enterprise Hadoop

As YARN took Hadoop 2 beyond batch, Slider will further accelerate that promise by bringing new applications to Hadoop, and lowering the barrier to run in a YARN environment. It’ll do so in a way that treats the applications as an integrated part of the Hadoop ecosystem.

Slider continues Hortonworks’ commitment to innovating at the core of 100% open source Enterprise Hadoop, while enabling partners to expand the range of solutions that make up the modern data architecture. We think Slider will change the face of Hadoop, and we’re excited for you to try it today.

Discover and Learn More

Categorized by :
YARN

Comments

|
July 17, 2014 at 2:05 am
|

Wow this sounds impressive – the potential is huge – I can’t wait to try this out… +1 for vision and keeping it Apache Foundation open source!

Leave a Reply

Your email address will not be published. Required fields are marked *

If you have specific technical questions, please post them in the Forums

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Resources

Join the Webinar!

Big Data Virtual Meetup Chennai
Wednesday, October 29, 2014
9:00 pm India Time / 8:30 am Pacific Time / 4:30 pm Europe Time (Paris)

More Webinars »

Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.