We are thrilled to announce that Hortonworks Data Platform (HDP) version 2.6 is now available – both on pre-premise and in the cloud. For the first time, we are also making this available on IBM Power System in addition to the x86 chipset.
During 2016, we have seen many of Hortonworks’ customers deploy more and more mission critical workloads onto Hadoop. There are still some traditional, batch-style workloads. However, the number of interactive and real-time use cases have increased dramatically in the past year. Apache Hive with LLAP functionality will bring an improved performance booster for interactive queries.
Apache Hive with LLAP and ACID Merge GA
One of the most significant features included in the HDP 2.6 release is Apache Hive with LLAP, which means “Low Latency Analytical Processing” or as our engineers and Star Trek fans named it, “Live Long and Process”.
LLAP enables sub-second SQL analytics on Hadoop by intelligently caching data in memory with persistent servers that instantly process SQL queries. Since LLAP is an evolution of the Hive architecture, it does all this with the same comprehensive ANSI standard SQL support and proven scale.
In this release, we have also taken a huge step forward toward true data management by introducing SQL-standard ACID Merge to Apache Hive. This provides the possibilities for fast updates and queries on the same engine.
Lastly, thanks to the addition of SQL features like enhanced subquery support and new set operators, Hive is now able to run all 99 TPC-DS queries out-of-the-box with only trivial rewrites required, thanks to the addition of new set operators and multi subquery support. Migrating your SQL workloads to Apache Hive 2.1 within HDP platform has never been easier.
For details, please check out our follow up two blogs on this topic in the next couple of days.
What Else is New in HDP 2.6? Spark and Data Science Workloads Become Front and Center
HDP 2.6 ensures deploying Spark within an enterprise environment is secure and easy. Data scientists using Spark with R language can deploy their favorite R package with their Spark job. Additionally, the technical preview of SparkSQL with Ranger integration deliver the same level of access control available previously only with Apache Hive. Now security administrators can apply fine grained security to data accessed with SparkSQL.
Since the GA of Apache Zeppelin, it has been widely used by many of our customers. To recap, Apache Zeppelin is a web-based notebook for analysts and data scientists that enables interactive data analytics. You can make beautiful data-driven, interactive and collaborative documents with SQL, Scala, Python and more. Within HDP 2.6, Apache Zeppelin can now be configured to authenticate users using the industry standard LDAP directory, enabling easy integration with existing IT security infrastructure. In addition, authentication can now be done using an encrypted SSL connection, ensuring username/password credentials was sent over the network securely.
A Few Other Enhancements for HDP 2.6
We develop the entire Hortonworks Data Platform to ensure our customers not only can adopt the latest innovation from the broader open source community, but also enjoy some of the enterprise ready and easy of use functionalities packaged within HDP 2.6. For example, we have expanded the dynamic tag-based policy support to HDFS, Kafka and HBase and the new mass import/export of Ranger security policies will highly reduce the IT overhead.
At this time, Hortonworks would like to thank everyone within the Apache community for all of their efforts and contributions. The pace of innovation continues to be truly amazing. We are grateful for the opportunity to work with such a unique group of dedicated professionals. We look forward to continuing our work with you and the ever vibrant community!
At Hortonworks, we believe in driving innovation. This is at the core of all of the engineering activities we engage in and we know that we are the stewards of open source community. We will continue to listen to our customers and partners as HDP continues to evolve. We are very happy about the milestones we achieved with the release of HDP 2.6 and look forward to hearing your feedback.
For further information, please check out: