Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
April 14, 2015
prev slideNext slide

Ambari 2.0 for Deploying Comprehensive Hadoop Security

Hortonworks Data Platform (HDP) provides centralized enterprise services for comprehensive security to enable end-to-end protection, access, compliance and auditing of data in motion and at rest. HDP’s centralized architecture—with Apache Hadoop YARN at its core—also enables consistent operations to enable provisioning, management, monitoring and deployment of Hadoop clusters for a reliable enterprise-ready data lake.

But comprehensive security and consistent operations go together, and neither is possible in isolation.

We published two blogs recently announcing Ambari 2.0 and its new ability to manage rolling upgrades. This post will look at those innovations through the security lens, because security, like operations, is a core requirement for enterprise-ready Hadoop.

Security in Hadoop Today

HDP offers comprehensive security, across all batch, interactive, or real-time workloads and access patterns. Hortonworks is focused on delivering comprehensive security across 5 pillars, namely centralized administration, authentication, authorization, audit, and data protection.

sec_1

HDP provides comprehensive security by way of three key services:

  • Kerberos is an MIT standard adopted by the open source community to authenticate users attempting to access Hadoop.
  • Apache Ranger provides centralized security administration for HDFS, Hive, HBase, Storm and Knox as well as fine-grain access control.
  • Apache Knox provides perimeter security for API access and REST services.

Security Setup with Ambari 2.0

Ambari 2.0 represents a significant milestone in the community’s ongoing work to make Hadoop enterprise-ready with easy security setup and administration. Now Ambari 2.0 can help administrators automate Kerberos setup for a cluster, install KDC and create service principles. Administrators can also use Ambari to install Ranger admin and enable the Ranger plugin with a few clicks.

Automated Kerberos integration

Before Ambari 2.0, the Kerberos integration in Hadoop required a combination of manual steps to install and manage these important components:

  • KDC (key distribution center),
  • User and service principles (identities) and
  • Respective keytabs (tokens).

With Ambari 2.0, the entire Kerberos setup process is automated, now with the following:

  • A step-by-step wizard to setup the Kerberos infrastructure
  • Integration with existing MIT KDC or Active Directory infrastructure
  • Deployment, configuration and management of Kerberos Clients
  • First time setup as well as ongoing management for adding new services or nodes
  • Automated creation of principals
  • Automated generation and distribution of keytabs
  • Support for regeneration of keytabs

Ambari 2.0 can automate Kerberos deployment and management for existing clusters already using Kerberos, as well as for users looking to install a new cluster.

Figure 1: Initial screen for Kerberos setup
Figure 1: Initial screen for Kerberos setup

This Kerberos Overview documentation for Ambari 2.0 contains an overview and step-by-step details on Kerberos setup.

Automated Ranger deployment

Hortonworks introduced Apache Ranger to deliver the vision of coordinated security across Hadoop with centralized administration, fine-grain access control and audit. Apache Ranger’s first release included enhancements to existing capabilities in the original code base developed at XA Secure and added support for audit storage in HDFS, support for Apache Storm and Knox authorization and auditing, and also REST APIs for managing policies.

With Ambari 2.0, administrators can now easily add comprehensive security through Ranger to either an existing or new cluster. Ambari 2.0 adds in the following benefits to Ranger:

  • Automated install of Ranger policy administrator and user sync. The policy database (mySQL or Oracle) can be configured and user sync can be integrated with LDAP/AD or Unix.
  • Easy one-click setup of the Ranger plugin for HDFS, Hive , HBase, Storm and Knox
  • Ability to start/stop services through the Ambari UI
  • Ability to disable plugins through the Ambari UI

The following screen shots show a user adding Ranger service via Ambari.

Figure 2. Ambari screen to add Ranger service
Figure 2. Ambari screen to add Ranger service
Figure 3: Ambari screen showing already installed and running Ranger service
Figure 3: Ambari screen showing already installed and running Ranger service

Hortonworks continues to lead open-source innovation to enable comprehensive data security for Hadoop—making it easier for security administrators to protect their clusters. With Ambari 2.0, we added the automated install and administration of the HDP cluster’s security infrastructure, with support for installing Kerberos, Apache Knox and Apache Ranger.

This innovation highlights what Hortonworks customers appreciate about our 100% open-source Apache Hadoop platform. HDP provides centralized enterprise services for comprehensive security and consistent operations to enable provisioning, management, monitoring and deployment of secure Hadoop clusters.

Hadoop is ready for the enterprise—providing any data, for any application, anywhere.

More About Comprehensive Security and Consistent Operations in HDP

Read recent Ambari posts

Learn more about the Apache projects

Tags:

Comments

  • MIT KDCs seem more suited to smaller PoC clusters with limited users in terms of administration since you’d have to integrate an LDAP backend for yourself to tie in user ID and group information … how about support for FreeIPA as per the jira AMBARI-6432? (FreeIPA doesn’t support use of the kadmin interface)

  • After installing ambari 2.0 and adding Ranger service to it, I am unable to enable/diable plugins via UI. I do not see that option.

    Also I tried to enable hdfs plugin via commandline but the UI still shows this as disabled. As such my policies are not working.

  • The biggest issues with Ambari 2 and this new “feature” that it *only* is able to configure a new kerberos and removes references to existing kerberos installation. This of course is a showstopper and we need to go to manual route once again. This will be fixed in Ambari 2.1, which is light years away. Well done who would have thought of that of a “enterprise” hadoop version.

  • Leave a Reply

    Your email address will not be published. Required fields are marked *

    If you have specific technical questions, please post them in the Forums

    You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>