Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
October 28, 2014
prev slideNext slide

Discover HDP 2.2: Comprehensive Hadoop Security with Apache Ranger and Apache Knox

Last week Hortonworks presented the first of 8 Discover HDP 2.2 webinars: Comprehensive Hadoop Security with Apache Ranger and Apache Knox. Vinay Shukla and Balaji Ganesan hosted this first webinar in the series.

Balaji discussed how to use Apache Ranger (for centralized security administration, to set up authorization policies, and to monitor user activity with auditing. He also covered Ranger innovations now included in HDP 2.2:

  • Support for Apache Knox and Apache Storm, for centralized authorization and auditing
  • Deeper integration of Ranger with the Apache Hadoop stack with support for local grant/revoke in HDFS and HBase
  • Ranger’s enterprise readiness, with the introduction of REST APIs for policy management, and scalable storage of audit in HDFS

Vinay presented Apache Knox and API security for Apache Hadoop. Specifically, Vinay covered how Apache Knox securely extends the reach of Hadoop APIs to anyone in an organization, using any type of device. Vinay also walked through new innovations in Knox that are included in HDP 2.2:

  • Support for the YARN REST API
  • Support for HDFS HA
  • Support for SSL to Hadoop cluster services (WebHDFS, Apache HBase, Apache Hive and Apache Oozie)
  • The Knox Management REST API
  • Integration with Apache Ranger for service-level authorization

Here is the complete recording of the Webinar, including Balaji’s demo of Apache Ranger.

And here are the Presentation slides.

Attend our next Discover HDP 2.2 on Thursday, October 30 at 10am Pacific Time: Even Faster SQL Queries with Apache Hive and

Or register for all remaining webinars in the series.

We’re grateful to the many participants who joined the HDP 2.2 security webinar and asked excellent questions. This is the complete list of questions with their corresponding answers:

Question Answer
Does Apache Ranger affect Apache Hive’s performance?

No. Ranger manages the policy centrally, but then it pushes enforcement down to the local component for enforcement. So for Apache Hive authorization, the policy is managed by Ranger but enforced by Ranger plugin running within Hive. So the integration of Ranger does not impact Hive’s performance.

But Apache Ranger brings in great value with the ability to centrally manage access policies across different components in the Hadoop platform

Does Apache Ranger manage YARN ACLs? Not yet. Externalizing YARN ACL through Ranger is in the works, but it is not available today in HDP 2.2.
How does Ranger hook into various services to enforce authorization? Do Hive and HBase provide necessary hooks for Ranger policies? Yes. Apache Ranger provides plugin which embed within processes of various components. These plugin use authorization hooks to enforce access control for user requests.
In the work we’ve done for HDP 2.2, we’ve made these hooks even better. Apache Hive has a new Hive authorization API, and Ranger has an implementation of that. In the case of HBase, it also has an authorization method where an external co-processor can be used. Ranger provides its own co-processor that is invoked as part of the HBase process and used for authorization. Also, in the case of Apache Knox and Apache Storm, we have used similar authorization hooks within those components.

That’s the idea of Ranger. We don’t want to change anything within the components, but we want to use those hooks to externalize the management of the authorization.

How do ODBC and JDBC drivers talk with the Knox API Gateway in a secure way?

See this blog for a detailed answer to the question: Secure JDBC and ODBC Clients’ Access to HiveServer2.

In summary, with Hive, using Beeline, when you configure HiveServer2‘s thrift gateway, ODBC and JDBC calls can be routed over HTTP. Then it becomes an HTTP call and Knox can secure those calls.

The main question is: how do you provide ODBC/JDBC access over HTTP? You enable Thrift Server calls and route those calls through Apache Knox. Knox then provides authentication, wire encryption and authorization (through Apache Ranger).

What protects Apache Ranger’s audit data from intentional alteration or corruption? With Apache Ranger in HDP 2.2 we can store audit in MYSQL, Oracle DB & HDFS, and we can apply an internal process to protect the audit from being altered by any user other than Apache Ranger. Audit information is accessible only through the Ranger Admin Portal, for specific users with privileges.
How do you secure access to the Management REST API? You can support authentication through Knox. You can put Knox in front of these REST APIs. The other model is direct REST API access, if you are not using Knox. You can directly access Ranger’s REST API and use the standard security methods such as SSL.
How does the integration work with SiteMinder for secure single sign-on (SSO)? With Knox, we support SSO, so for all the REST APIs that you expose to your Hadoop end users, you can support the SSO through Knox. For example, when you deploy Knox, it supports CA SiteMinder, Oracle Access Management Suite or Tivoli Access Manager. You can deploy Knox with an Apache HTTP Server and leverage its integration, or you can directly integrate with Knox.

Visit these pages to learn more


Leave a Reply

Your email address will not be published. Required fields are marked *

If you have specific technical questions, please post them in the Forums