The Hortonworks Blog

Posts categorized by : HDP

We are excited to welcome Blackrock and Passport Capital as Hortonworks investors who today led a $100M round of funding with continued participation from all existing investors.

This latest round of funding will allow us to double-down on our founding strategy: to make open source Apache Hadoop a true enterprise data platform. To that end we are focused in two areas:

1. Lead the innovation of Hadoop. In open source, for everyone.…

In February 2014, the Apache Storm community released Storm version 0.9.1. Storm is a distributed, fault-tolerant, and high-performance real-time computation system that provides strong guarantees on the processing of data. Hortonworks is already supporting customers using this important project today.

Many organizations have already used Storm, including our partner Yahoo! This version of Apache Storm (version 0.9.1) is:

  • Highly scalable. Like Hadoop, Storm scales linearly
  • Fault-tolerant. Automatically reassigns tasks if a node fails
  • Reliable. 

LDAP provides a central source for maintaining users and groups within an enterprise. There are two ways to use LDAP groups within Hadoop. The first is to use OS level configuration to read LDAP groups. The second is to explicitly configure Hadoop to use LDAP-based group mapping.

Here is an overview of steps to configure Hadoop explicitly to use groups stored in LDAP.

  • Create Hadoop service accounts in LDAP
  • Shutdown HDFS NameNode & YARN ResourceManager
  • Modify core-site.xml to point to LDAP for group mapping
  • Re-start HDFS NameNode & YARN ResourceManager
  • Verify LDAP based group mapping

Prerequisites: Access to LDAP and the connection details are available.…

If there’s one thing my interactions with our customers has taught me, it’s that Apache Hadoop didn’t disrupt the datacenter, the data did. The explosion of new types of data in recent years has put tremendous pressure on the datacenter, both technically and financially, and an architectural shift is underway where Enterprise Hadoop is playing a key role in the resulting modern data architecture.

Download our Whitepaper: Hadoop and a Modern Data Architecture.

Due to the flourish of Apache Software Foundation projects that have emerged in recent years in and around the Apache Hadoop project, a common question I get from mainstream enterprises is: What is the definition of Hadoop?

Download our Whitepaper: Hadoop and a Modern Data Architecture.

This question goes beyond the Apache Hadoop project itself, since most folks know that it’s an open source technology borne out of the experience of web scale consumer companies such as Yahoo!, Facebook and others who were confronted with the need to store and process massive quantities of data.…

This blog post originally appeared here and is reproduced in its entirety here. Part 1 can be found here.

The HBase BlockCache is an important structure for enabling low latency reads. As of HBase 0.96.0, there are no less than three different BlockCache implementations to choose from. But how to know when to use one over the other? There’s a little bit of guidance floating around out there, but nothing concrete.…

Luminar is one of Hortonworks’ original customers. Apache Hadoop is a pillar of their modern data architecture, and since choosing Hortonworks in 2012, the Luminar team became expert users of Hortonworks Data Platform version 1.

They were eager to migrate to HDP2 after it launched in October 2013.

I recently spoke with Juan Manuel Alonso, Luminar’s Manager of Insights. Juan Manuel worked with the Hortonworks professional services team to plan and execute the migration from HDP1 to HDP2.…

The Apache Tez community has voted to release 0.3 of the software.

Apache™ Tez is a replacement of MapReduce that provides a powerful framework for executing a complex topology of tasks. Tez 0.3.0 is an important release towards making the software ready for wider adoption by focussing on fundamentals and ironing out several key functions. The major action areas in this release were

  • Security. Apache Tez now works on secure Hadoop 2.x clusters using the built-in security mechanisms of the Hadoop ecosystem.
  • Compuware is a Hortonworks Technology Partner and this week announced the availability of the newest release of APM for Big Data.  This release provides enhanced support for Hadoop 2.0 and Hortonworks Data Platform (HDP) 2.0

    Compuware’s APM for Big Data now provides greater visibility into Hadoop job details with out-of-the-box dashboards that require no configuration. The graphical dashboards expand insight and ease of analyzing Hadoop deployments.  With the Hadoop focused dashboards, customers can get information about any Hadoop cluster and summarized overviews of cluster utilization across users, jobs, pools, queues and more.…

    Elasticsearch’s engine integrates with Hortonworks Data Platform 2.0 and YARN to provide real-time search and access to information in Hadoop.

    See it in action:  register for the Hortonworks and Elasticsearch webinar on March 5th 2014 at 10 am PST/1pm EST to see the demo and an outline for best practices when integrating Elasticsearch and HDP 2.0 to extract maximum insights from your data.  Click here to register for this exciting and informative webinar!…

    Today, the Forrester WaveTM: Big Data Hadoop Solutions, Q1 2014 was published by Forrester Research and while not surprised, we are delighted that this leading analyst firm recognized us as a clear leader in the Hadoop market. We could not be prouder of our unwavering strategy and hard work that is propelling us to the forefront of this burgeoning Hadoop market.

    Download and review the report here.

    Forrester evaluated nine vendors across a range of criterion from strategy to product and market presence and we scored a very balanced report across all categories and are way “up and to the right”.…

    This blog post originally appeared here and is reproduced in its entirety here.

    HBase is a distributed database built around the core concepts of an ordered write log and a log-structured merge tree. As with any database, optimized I/O is a critical concern to HBase. When possible, the priority is to not perform any I/O at all. This means that memory utilization and caching structures are of utmost importance. To this end, HBase maintains two cache structures: the “memory store” and the “block cache”.…

    With over 230 JIRA tickets resolved, the Apache HBase community released 0.98.0 yesterday which is the next major version after 0.96.x series.

    HBase 0.98.0 comes with an exciting set of new features with keeping the same stability improvements and features on top of 0.96. Additional to usual bug fixes, some of the major improvements include:

    • Reverse Scans (HBASE-4811): for use cases where both forward and reverse iteration is required, HBase now allows to perform scans in reverse mode.

    Actuate is a Hortonworks Technology Partner and founded and co-leads the BIRT open source project, which is used by more than 2.5 million developers around the globe and serves as the foundation of Actuate’s commercial offerings. Applications built with BIRT and BIRT iHub deliver more business and consumer insights to more people than all BI companies combined. 

    The deployment of Big Data architectures has become more prevalent as organizations realize the power of what Big Data can bring to their businesses and to their profitability. …

    Hadoop can be a great complement to existing data warehouse platforms, such as Teradata, as it naturally helps to address two key storage challenges:

    The purpose of this article is to detail some of the key integration points and to show how data can be easily exchanged for enrichment between the two platforms.

    As a data integrator who is familiar with RDBMS systems and is new to the Hadoop platform, I was looking for a simple way (i.e.…

    Go to page:« First...34567...10...Last »