Earlier this month, the Apache Ambari community released Apache Ambari 1.6.1, which includes multiple improvements for performance and usability. The momentum in and around the Ambari community is unstoppable. Today we saw the Pivotal team lean in to Ambari, and this is the sixth release of this critical component in 2014, proving again that open source is the fastest path to innovation.
Many thanks to the wealth of contribution from the broad Ambari community that resulted in 585 JIRA issues being resolved in this release.
The new features in Ambari 1.6.1 help operate enterprise-grade Hadoop at scale on very large clusters supporting batch, interactive and real-time workloads running through YARN. Ambari 1.6.1 also delivers the usability guardrails an enterprise expects from an operations platform.
YARN allows more Hadoop users to access data in different ways. Larger groups of users create more demand for data in Hadoop, which accelerates the rate of cluster growth. That means that many enterprise clusters must handle thousands of hosts, and Ambari 1.6.1 makes this increased scale easier to manage.
Ambari 1.6.1 includes an optimized REST API that makes requests as efficient as possible. It only loads the most critical information into the Ambari Web interface. To match the API improvements, additional improvements to the Ambari Server backend cache make operational information readily available. In addition, better tuning of Ganglia and Nagios facilitate metrics collection and alerts.
The Apache Ambari team has tested and verified Ambari Server against a live 2000-node cluster. We presented those test results at Hadoop Summit in June. Watch the presentation or review that slide deck.
In a number of regulated industries, we see users installing clusters with limited or no Internet access. Performing that “local install” means that HDP Stack repositories are available and ready to use on the local network. But it also means that Java should be installed and available on the various hosts prior to install.
Ambari has supported local installs for some time and it also handles environments with a custom JDK installed. Ambari 1.6.1 does more checking for validity of a custom JDK path and reporting of discrepancies across all hosts. This ensures that installed services do not run into problems when they start up, and it also allows Hadoop Operators to identify and correct JDK installation issues.
Ambari supports configuring Hadoop services that require an external RDBMS, allowing them to work with some of the most popular databases including Oracle, MySQL and PostgreSQL. Ambari 1.6.1 makes those configurations easier.
Ambari 1.6.1 also makes it easier to get your JDBC driver in place within the cluster and also tests the database connections to make sure that your configuration is correct and working properly. These improvements reduce the number of manual setup steps, decrease errors and eliminate downstream triage.
The previous Ambari release introduced Ambari Blueprints. As more and more services are introduced to the Hadoop Stack, tracking the number of components (and the relationship between those components) can be a challenge. Blueprints are an API-driven method to install and configure clusters in a consistent and repeatable fashion.
Ambari 1.6.1 addresses the tracking challenge by allowing you to perform a topology validation based on dependency and cardinality information provided in the associated Ambari Stack definition. In addition to validation, some components are auto-deployed if not present in the topology. Optionally, validation can be turned-off while making the API call.
While installing Hadoop, one must make sure that the network is configured properly for each node within the cluster. Ambari 1.6.1 has added more network checks to help with that.
The “hostname check” enhancements include checking for reverse hostname lookup. During cluster installation or while adding hosts, host check will report any errors related to reverse hostname lookup. Ambari 1.6.1 users can also perform hostname resolution checks on each host to ensure that other hostnames are resolvable and reachable.
The Ambari community is already pushing forward towards the next release. Top priority items on the roadmap include: