Hadoop, The Perfect App for OpenStack
The convergence of big data and cloud is a disruptive market force that we at Hortonworks not only want to encourage but also accelerate. Our partnerships with Microsoft and Rackspace have been perfect examples of bringing Hadoop to the cloud in a way that enables choice and delivers meaningful value to enterprise customers. In January, Hortonworks joined the OpenStack Foundation in support of our efforts with Rackspace (i.e. OpenStack-based Hadoop solution for the public and private cloud). [
Today, we announced our plans to work with engineers from Red Hat and Mirantis within the OpenStack community on open source Project Savanna to automate the deployment of Hadoop on enterprise-class OpenStack-powered clouds.
Why is this news important?
Because big data and cloud computing are two of the top priorities in enterprise IT today, and it’s our intention to work diligently within the Hadoop and OpenStack open source communities to deliver solutions in support of these market needs. By bringing our Hadoop expertise to the OpenStack community in concert with Red Hat (the leading contributor to OpenStack), Mirantis (the leading system integrator for OpenStack), and Rackspace (a founding member of OpenStack), we feel we can speed the delivery of operational agility and efficient sharing of infrastructure that deploying elastic Hadoop on OpenStack can provide.
Big Data and Cloud Computing are Top Initiatives for IT Executives
A year ago, Barclays published its April 2012 CIO Survey where they stated “most CIOs rated the challenges of data growth and “Big Data” as the No. 1 trend driving IT spending decisions. We believe that “Big Data” is quickly becoming one of the biggest challenges within IT infrastructures given the shift to cloud computing and growth in unstructured data.”
Fast forward to CIO magazine’s 2013 State of the CIO Survey, we see that cloud computing and big data continue as major themes. Of the IT executives surveyed, 39% expect to complete cloud computing initiatives and 37% expect to complete big data initiatives this year. On top of that, 59% of those surveyed classify their organization as late majority or laggards when it comes to adoption of big data initiatives.
Translation? Big data and Hadoop have crossed the chasm and are accelerating into the mainstream enterprise, which reinforces our assertion made back in January.
Hadoop and OpenStack Sitting in a Tree…
Hadoop is the leading open source platform for storing, processing and accessing large data sets across clusters of computers, and OpenStack is the leading open source framework for building and managing private, public and hybrid Infrastructure-as-a-Service (IaaS) clouds.
According to Gartner, big data will drive $232 billion in IT spending through 2016. The benefits to organizations for adding big data to their information management and analytics infrastructure will force a more rapid cycle of replacing existing solutions. Since Hadoop is net-new workload for most organizations, Hadoop on OpenStack provides the perfect “greenfield” use case for those looking to start anew on a platform that makes sense.
Moreover, Hadoop and OpenStack are open source technologies designed for scale-out architectures that can be cost effectively deployed. Finally, since Hadoop can be complex to get started with, taking advantage of the operational agility and deployment choice that OpenStack enables will go a long way to jumpstarting those interested in deploying big data on cloud.
…First Comes Love, then Comes Marriage
At Hortonworks, our strategy is founded on the unwavering belief in the power of community driven open source software, and when it comes to platform technologies like Hadoop and OpenStack, we believe that community-driven open source will always outpace the innovation of a single group of people or single company. We feel this is why both Hadoop and OpenStack have attracted major ecosystem players such as IBM, Red Hat, HP, Rackspace, Intel, and many others.
Our news today means we intend to marry, if you will, two of the largest open source movements in order to accelerate the perfect use case of big data on cloud. Specifically, we will be working within the OpenStack community on Project Savanna, originally introduced as an OpenStack project by Mirantis. The goal of the project is to enable OpenStack users to easily provision and manage elastic Hadoop clusters on OpenStack.
An important design point for Savanna is to provide an integration point for Hadoop provisioning and management frameworks, so we will focus on making sure Apache Ambari is well integrated. This will enable our enterprise customers to provision the Hortonworks Data Platform very quickly via OpenStack APIs and OpenStack’s Horizon Dashboard while managing their Hadoop cluster in a familiar way.
For those who want to learn more about Savanna, Mirantis has a blog post and video that provides a great overview.
And finally, we will be demonstrating the initial fruits of our labor at the Hadoop Summit on June 26, 2013 in San Jose California, so we encourage you to sign up for the conference!