Analysts and data scientists⎯not to mention business executives⎯want Big Data not for the sake of the data itself, but for the ability to work with and learn from that data. As other users become more savvy, they also want more access. But too many inefficient queries can create a bottleneck in the system.
From the Dev Team
Follow the latest developments from our technical team
This is the third post in a series exploring recent innovations in the Hadoop ecosystem that are included in Hortonworks Data Platform (HDP) 2.2. In this post, we introduce the theme of supporting rolling upgrades and downgrades of a Hadoop YARN cluster.
HDP 2.2 offers substantial innovations in Apache™ Hadoop YARN, enabling Hadoop users to efficiently store and interact with their data in a single repository, simultaneously using a wide variety of engines.…
As a data scientist working with Hadoop, I often use Apache Hive to explore data, make ad-hoc queries or build data pipelines.
Until recently, optimizing Hive queries focused mostly on data layout techniques such as partitioning and bucketing or using custom file formats.
In the last couple of years, driven largely by the innovation of the Hive community around the Stinger initiative, Hive query time has improved dramatically, enabling Hive to support both batch and interactive workloads at speed and at scale.…
The Apache HBase community has released Apache HBase 1.0.0. Seven years in the making, it marks a major milestone in the Apache HBase project’s development, offers some exciting features and new API’s without sacrificing stability, and is both on-wire and on-disk compatible with HBase 0.98.x.
In this blog, which is a cross post from from Apache HBase Blog, we look at the past, present and future of Apache HBase project.…
Hortonworks Data Platform’s YARN-based architecture enables multiple applications to share a common cluster and data set while ensuring consistent levels of response made possible by a centralized architecture. Hortonworks led the efforts to on-board open source data processing engines, such as Apache Hive, HBase, Accumulo, Spark, Storm and others, on Apache Hadoop YARN.
Since our founding in 2011, Hortonworks has had a fundamental belief: the only way to deliver infrastructure platform technology is completely in open source. Moreover, we believe that collaborative open source software development under the governance model of an entity like the Apache Software Foundation (ASF) is the best way to accelerate innovation that targets enterprise end users since it brings the largest number of developers together in a way that enables innovation to happen far faster than any single vendor could achieve and in a way that is free of friction for the enterprise.…
As a core component of the Modern Data Architecture (MDA), organizations rely on the Hortonworks Data Platform (HDP) for their mission critical functions which demand high availability and performance. Key to these organizations is simplified and consistent Hadoop Operations.
Join us for this workshop where we’ll cover the operational concerns of System Administrators & DevOps Engineers including installation, configuration, maintenance, security and performance topics.
Key Highlights include:
- Hardware Recommendation and Sizing
- OS tuning guide
- Rapid and consistent deployment of clusters using Apache Ambari Blueprints
- Cluster setup validation
- Multi-tenancy with YARN
- HA and Business Continuity
The workshop will be a combination of slides plus demonstrations of the code in action.…
In August 2009, the Facebook Data Infrastructure Team published a white paper that outlined a warehousing solution over Hadoop. They called it Hive. And since that time, this project has not only emerged as the defacto standard for SQL in Hadoop, but with the help of the Stinger initiative it has progressed from a batch only framework with limited SQL interface to a near SQL:2011 compliant, fully interactive SQL query engine.…
Big data and cloud computing are top priorities in enterprise IT today. Organizations are adopting these two disruptive technologies because of the promise of lower cost, flexibility, portability and ease of management.
Today’s blog is another in a series discussing Apache Hadoop in the cloud as a key deployment option. Our guest blogger today is Sean Anderson, Manager of Data Service at Rackspace, the managed cloud company.
In 2012, Rackspace and Hortonworks partnered to expand the capabilities of Enterprise HadoopTM to both public cloud utility services and private clouds utilizing the popular open-source cloud platform Openstack.…
This is the second post in a series that explores recent innovations in the Hadoop ecosystem that are included in HDP 2.2. In this post, we introduce the theme of running service-workloads in YARN to set context for deeper discussion in subsequent blogs.
HDP 2.2 brings substantial innovations in Apache Hadoop YARN, enabling users of Apache Hadoop to efficiently store their data in a single repository and interact with it simultaneously using a wide variety of engines.…
Hortonworks Data Platform (HDP) provides Hadoop for the Enterprise, with a centralized architecture of core enterprise services, for any application and any data. HDP is uniquely built around native YARN services to enable a centralized architecture through which multiple data access applications interact with a shared data set. Apache Hive is one of the most important of those data access applications—the defacto standard for interactive SQL queries over petabytes of data in Hadoop.…
This guest blog post is from Alyssa Jarrett, product marketing manager at Splice Machine. Splice Machine is a Hortonworks Certified Technology Partner and provides one of the only Hadoop RDBMS to power a new generation of real-time applications and operational analytics. With its recent Certification with HDP, Splice Machine offers a 10x price/performance improvement over traditional relational databases.
This is the first post in a series that explores recent innovations in the Hadoop ecosystem that are included in HDP 2.2. In this post, we introduce themes to set context for deeper discussion in subsequent blogs.
HDP 2.2 represents another major step forward for Enterprise Hadoop. With thousands of enhancements across all elements of the platform spanning data access to security to governance, rolling upgrades and more, HDP 2.2 makes it even easier for our customers to incorporate HDP as a core component of Modern Data Architecture (MDA).…
In our series on Data Science and Hadoop, predicting airline delays, we demonstrated how to build predictive models with Apache Hadoop, using existing tools. In part 1, we employed Pig and Python; part 2 explored Spark, ML-Lib and Scala.
Throughout the series, the thesis, theme, topic, and algorithms were similar. That is, we wanted to dismiss the misconception that data scientists – when applying predictive learning algorithms, like Linear Regression, Random Forest or Neural Networks to large datasets – require dramatic changes to the tooling; that they need dedicated clusters; and that existing tools will not suffice.…
On December 18th, 2014, Hortonworks presented the last of 8 Discover HDP 2.2 webinars: Apache HBase with YARN & Slider for Fast NoSQL Access. Justin Sears, Jeff Sposetti and Mahadev Konar hosted the last webinar in the series.
After Justin Sears set the stage for the webinar by explaining the drivers behind Modern Data Architecture (MDA), Jeff Sposetti and Mahadev Konar introduced Apache Ambari and discussed Ambari innovations now included in HDP 2.2:
- Configuration Enhancements, including Versioning & History
- Ambari Administration, including Views Framework
- Ambari Stacks “Stacks Advisor”
Here is the complete recording of the Webinar
Here are the presentation slides.…