Apache Hadoop Meets Informatica Data Parsing

As the framework architects and developers of Apache Hadoop MapReduce, we are always looking for ways to simplify the complex tasks associated with large-scale processing of data. We want users and organizations to spend their time on analyzing their growing data to gain valuable insights, not on menial tasks such as massaging their data for consumption or tediously parsing complex structures in their data. The Informatica HParser technology is extremely valuable in this regard.

For those new to Apache Hadoop, MapReduce is a parallel computing framework for processing large volumes of data. It deals with the four V’s of big data (as Forrester described) that present challenges to existing data systems, namely: volume, velocity, variety and variability. Together with the Hadoop Distributed File System (HDFS) and a handful of other important Apache Hadoop projects, it provides a massively scalable and highly reliable platform for storing, processing, managing and ultimately analyzing the ever-increasing data coming not only from transactional systems but also unstructured data in the form of server logs, customer interaction records, social media updates, email, PDFs, CDRs and so forth.

While extremely powerful and adept at providing insights into huge volumes of data, Apache Hadoop can still present challenges to enterprises trying to implement it. For example, with complex data types come complex tasks required to be able to write applications to analyze the data. One of the areas of pain has always been in the area of parsing deeply hierarchical data like XML or JSON coming from clickstream or logs. Previously, you needed to manually script tasks in MapReduce to handle this and it could take weeks or even months to accomplish the feat in large organizations.

Informatica has addressed this challenge head on with HParser. This solution runs within MapReduce applications to exploit the inherent parallelism. It turns manually scripted parsing tasks into standardized tool-based ones. This means that the user community can become significantly more productive. It also means they can focus much more of their time on writing valuable applications that deliver more and better analytical insight to their business users. Informatica has also released a community edition of HParser that is available for free and compatible with the Hortonworks Data Platform Powered by Apache Hadoop.

As the lead of the Next Generation of MapReduce in Apache Hadoop, I am excited that industry leaders like Informatica are creating solutions that simplify complex Hadoop tasks. As a co-founder of Hortonworks, I am excited that our two companies are working together to accelerate the adoption of Apache Hadoop. Please visit the Hortonworks and Informatica partnership page for more details, including a link to the free community edition of HParser.

~ Arun C Murthy
@acmurthy

Categorized by :
Apache Hadoop Hadoop Ecosystem MapReduce

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Join the Webinar!

Discover HDP 2.2: Comprehensive Hadoop Security with Apache Ranger and Apache Knox
Thursday, October 23, 2014
1:00 PM Eastern / 12:00 PM Central / 11:00 AM Mountain / 10:00 AM Pacific

More Webinars »

HDP 2.1 Webinar Series
Join us for a series of talks on some of the new enterprise functionality available in HDP 2.1 including data governance, security, operations and data access :
Contact Us
Hortonworks provides enterprise-grade support, services and training. Discuss how to leverage Hadoop in your business with our sales team.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.