Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
October 10, 2014
prev slideNext slide

Riding the Elephant: Enterprise-Grade Hadoop Deployments With Apache Ambari 1.7 and Syncsort

Syncsort is a certified Hortonworks Technology and YARN Ready Partner and our guest blogger. Here, Tendu Yogurtcu, vice president of engineering at Syncsort, expands on Syncsort’s recent news about their integration of DMX-h and Ambari.

As Apache Hadoop YARN has transformed Hadoop from being a data processing solution to being a true data processing platform, requirements for provisioning, managing, and securing the platform have changed dramatically.

Stability, security, easy deployment, performance, management and monitoring are among many of the key attributes that make a data management platform enterprise-grade. YARN allows Hadoop to become a single platform for data storage and processing, on which multiple workloads, batch or real-time or interactive, can run. What does this mean for the Apache Hadoop stack? Does this inevitably mean more components and a higher level of complexity?

Apache Ambari addresses this challenge of managing a variety of workloads, processing engines and different access patterns. This week, we at Syncsort are excited to announce the deployment of Syncsort’s native Hadoop data integration product DMX-h via Apache Ambari. Syncsort is the first data integration solution to use Ambari Stacks, creating a unified user experience for deployment, management and monitoring across Hadoop clusters. This builds on our commitment to accelerate enterprise adoption of Hadoop by providing an intuitive user experience and a highly-secure, industrial scale data integration engine.

This integration between Syncsort DMX-h and Apache Ambari brings several benefits to the enterprise:

  1. Ambari Stacks enables the enterprise to have a single pane of glass for provisioning and deploying all components of the Hadoop stack, including Syncsort DMX-h for high-performance transformations and connectivity.
  2. Syncsort’s integration brings new, powerful services to Apache Ambari for blending data from sources as varied as the mainframe, traditional database platforms, web log generators, social media sources, and message queues, loading “nearby” analytical systems (e.g. HP Vertica) and creating rich visualizations (e.g. with Tableau).
  3. Syncsort provides an industrial-scale data integration engine running natively in Hadoop with a drag and drop graphical user interface, enabling anyone to be a Hadoop developer without the need for any coding experience.

Jeff Sposetti’s blog post after this year’s Hadoop Summit discussed Ambari’s focus in three specific areas: operate, integrate and extend. With the upcoming release of Ambari 1.7, the community has been active developing on all three fronts. Ambari 1.7 allows the community to create custom Stack definitions to plug-in new services that can co-exist with Hadoop. This makes it easy for ISV applications to be deployed on Hadoop clusters, while maintaining the stable, predictable operational capabilities that an enterprise expects. Ambari 1.7 will also introduce Ambari Views, a technology that further allows ISV applications to tailor the user experiences for operators and end users.

Syncsort DMX-h extended Ambari with a custom Service definition. Integration was quite straightforward, the operational controls as defined in the Stack literally “just worked” — we easily implemented a DMX-h Service and exposed it using the Ambari REST APIs. The DMX-h Service is automatically picked by Ambari and displayed in the Ambari 1.7 Web UI, and our joint customers can now use a single Hadoop management platform for all tools in the Hadoop stack.

sync_sort_1

Many thanks to the Hortonworks team for their collaboration, and feedback on the integration.

What’s Next?

We expect to see ~5,000 people in next week’s Strata+Hadoop World 2014, a true testament to the growing ecosystem and number of Hadoop deployments. Syncsort and Hortonworks are committed to help enterprises overcome challenges in transitioning to the future of data management.

You can download DMX-h Service package for Apache Ambari via the Syncsort portal and try it with a Hortonworks Sandbox. Stop by our booth at Strata+Hadoop World to see a demo of the integration.

Learn More

 

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *