With Ambari 2.5, our focus was to continue to improve Hadoop operator’s day-to-day cluster management experience. The community’s goal with Apache Ambari is to provide the most intelligent, easy-to-use, and extensible Hadoop operations experience, and with 2.5 we’ve made important improvements in the following areas:
To learn more about Ambari 2.5, join us on August 15, to hear about the latest updates and see a demo. Register here,
With the Service Auto Start (AMBARI-2330, documentation) capability, operators can now set policies to ensure that components are automatically restarted if they fail, and ensure components are started when the underlying host is rebooted.
The host-based Ambari Agent detects when a managed component abnormally exits and will restart that component based on the policy defined by the operator. On boot, the Ambari Agent will consult that policy and automatically start components as required.
For large production clusters, we’ve added the capability to easily Add or Remove JournalNodes (AMBARI-7748, documentation). This feature guides operators through the process of adding, removing, and moving journal nodes and takes the manual effort and guesswork out of the process.
Hadoop components produce plenty of logs and configuring log rotation used to require operators to be familiar with how Log4j works. We’ve made it a lot easier to configure the number of backup files to be saved, and the size of each backup file with the Simplified Log Rotation Configuration (AMBARI-16880, documentation) capability. Now operators can simply specify how many files should be kept, and at what point the logs should be rotated (based on file size).
Ambari Log Search (Tech Preview) has been one of our most popular features, and in this release has seen UI, and backend refreshes based on customer feedback. Log Search is planned for GA with the next major Ambari release, Ambari 3.0 in which the UI will be simplified, and the backend will have more robust log retention and scaling capabilities.
Ever made a single change in Ambari that turned into multiple configuration changes across different services? The Ambari stack/service advisor was built to help teams ensure Ambari updated the appropriate dependent configurations when configuration changes occurred. This can be very helpful. But if you haven’t had a chance to review the configuration changes we’re making, helpful isn’t the right word to describe your feeling. With the Configuration Change Communication (AMBARI-19572, documentation) feature, we’ve scoured the product to identify each point that configuration recommendations are being made by the stack/service advisor. In addition, we have added pop-ups and dialogs to make sure operators have the opportunity to see the recommendations and opt out or in where possible based on preference.
Many of our customers have downstream applications that require exports of the client configuration for the components those applications are integrating with. With the Download All Client Configurations (AMBARI-19275, documentation) feature we’ve made it easy to export a tgz file with ALL of the client configurations for each service that Ambari is managing.
HDFS is important, and when you’re trying to troubleshoot why the NameNode is under stress and who or what may be causing the problem, it’s extremely helpful to see who are the top 10 most frequent users making requests to the NameNode, and the top 10 most frequent operations being requested of the NameNode over time. With the HDFS TopN User & Operation Visualization (AMBARI-19320, documentation) those metrics are now visible in Grafana, making troubleshooting that much easier and helping reduce your MTTB, (mean time to blame) to find who’s responsible for the problem.
The Ambari Metrics System has been a great resource for Hadoop operators, allowing them to have access to a wide array of metrics for the underlying operating system, and components managed by Ambari. As more metrics are introduced, and more users are consuming those metrics using AMS’s built in instance of Grafana, the need to scale AMS out has become a focus of our engineering team. Ambari 2.5 has introduced AMS Collector High Availability (AMBARI-15901, documentation) as a Tech Preview. This means that the feature is available, but not yet recommended for production deployments. This feature has been built for large clusters that require multiple metrics collectors to cope with reading, writing, and aggregating metrics. Benefits include distributed metric aggregation and metric writes/reads, and moving or adding a Metrics Collector no longer requires the restart of cluster components. This architecture will help our largest customers get maximum value out of AMS for troubleshooting and managing their environment.
For those operators in large enterprise environments, integrating Ambari Alerts with your enterprise event management platform just got a lot easier. The default SNMP notifications now use a Built-In SNMP MIB (AMBARI-19257, documentation), making the integration process much simpler.
For those operators in high-security environments, you’ll appreciate the Password Credential Store Management (AMBARI-18650, documentation) for Hive, Oozie, Ranger, and Ambari Log Search. This capability makes it much easier to encrypt key passwords on disk for Hadoop services.
Many Ambari Views require a user to have a valid home directory in HDFS. We’ve made the Post-user-creation script hook (AMBARI-18722, documentation) to simplify the process of creating home directories. With this feature enabled, Ambari will automatically create home directories for users that are either created in Ambari or imported using LDAP.
Many of our customers use the Ambari REST API for automation, integration, and development. When it comes to taking those integrations to production, many have asked for the capability to use Kerberos to authenticate to the Ambari REST API. With the Ambari SPNEGO support (AMBARI-18365, documentation) that’s now possible.
Nobody wants to take downtime for an upgrade, and even fewer want to rush through one and risk data loss. Ambari’s Rolling Upgrade provides a middle ground that avoids downtime during upgrades and provides a conservative approach that ensures that there is no risk of data loss during the upgrade. With this approach, the total duration of a Rolling Upgrade is longer than an Express Upgrade. In Ambari 2.5, we’ve helped operators by giving them the ability to hit ‘Pause’ during a Rolling Upgrade and regain operational control of the cluster to take care of anything that may come up. A common ask is this, “if the Rolling Upgrade could take a few days, what happens if my HiveServer2 instance has a problem halfway through and I need to get it restarted?” This is why we added the ‘Pause’ capability. You simply hit the ‘Pause’ button, minimize the upgrade window, and then you have full control of the cluster to restart your HiveServer2. Once you’ve taken care of getting Hive back up and running, just hit ‘Resume’ to continue with the upgrade when ready.
Script Alert: When talking to customers, I frequently get questions around how Ambari can be configured to send alerts to HipChat, or Slack, or Pager Duty. A little-known feature within Ambari is the Script Alert. This feature allows you to define scripts that should be called whenever an alert is fired in Ambari, allowing you to easily integrate with external systems. So next time you want to send an Ambari Alert to <insert system here>, give the Script Alert a shot.
Hortonworks Documentation Search: Our documentation team has spent a ton of time improving the quality, relevance, and overall search experience on docs.hortonworks.com, I encourage you to give our search a second shot and let us know how you feel about the improvement.
To learn more about Ambari 2.5, listen to our latest August 15 webinar on Ambari 2.5, SmartSense 1.4 and our newest Flex Support Subscription offering here,