Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
March 21, 2014
prev slideNext slide

Hadoop GroupMapping – LDAP Integration

LDAP provides a central source for maintaining users and groups within an enterprise. There are two ways to use LDAP groups within Hadoop. The first is to use OS level configuration to read LDAP groups. The second is to explicitly configure Hadoop to use LDAP-based group mapping.

Here is an overview of steps to configure Hadoop explicitly to use groups stored in LDAP.

  • Modify core-site.xml to point to LDAP for group mapping
  • Re-start HDFS NameNode & YARN ResourceManager
  • Verify LDAP based group mapping

Prerequisites: Access to LDAP and the connection details are available.

Step 1: Modify core-site.xml to point to LDAP for group mapping

Back up your core-site.xml before making modifications to it. Below is a sample configuration that needs to be added to core-site.xml. You will need to provide the value for the bind user, bind password and other properties specific to your LDAP and make sure object class, user & group filter match the values specified in your LDAP.

[xml]
<property
<name>hadoop.security.group.mapping</name>
<value>org.apache.hadoop.security.LdapGroupsMapping</value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.bind.user</name>
<value>cn=Manager,dc=hadoop,dc=apache,dc=org</value>
</property>
<!–
<property>
<name>hadoop.security.group.mapping.ldap.bind.password.file</name>
<value>/etc/hadoop/conf/ldap-conn-pass.txt</value>
</property>
–>
<property>
<name>hadoop.security.group.mapping.ldap.bind.password</name>
<value>hadoop</value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.url</name>
<value>ldap://localhost:389/dc=hadoop,dc=apache,dc=org</value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.url</name>
<value>ldap://localhost:389/dc=hadoop,dc=apache,dc=org</value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.base</name>
<value></value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.search.filter.user</name>
<value>(&amp;(|(objectclass=person)(objectclass=applicationProcess))(cn={0}))</value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.search.filter.group</name>
<value>(objectclass=groupOfNames)</value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.search.attr.member</name>
<value>member</value>
</property>
<property>
<name>hadoop.security.group.mapping.ldap.search.attr.group.name</name>
<value>cn</value>
</property>
[/xml]

While group mapping configuration supports reading password from a file, in the above example relevant configuration is commented out due to this bug (HADOOP-10249) .

Step 2 : Re-start Hadoop

Follow the instructions in the Hortonworks Data Platform documentation to re-start HDFS NameNode & YARN ResourceManager.

Step 3: Verify LDAP group mapping

Run hdfs groups command. This command will fetch groups from LDAP for the current user. Note with LDAP group mapping configured, the hdfs permission can leverage groups defined in LDAP for access control

Conclusion

Since there are two ways in Hadoop to use groups in LDAP, a basic question is when to use each way. The OS based group mapping is a Linux/Unix method and won’t work on Windows. The explicit group mapping covered in this post will work on both Linux & Windows.

Let me know if you run into any issues with the steps in this post or have any comments on this post. In the next post I will cover configuring OS to read group information from LDAP.

Tags:

Comments

Charles Slovak says:

How does this work with LDAP connecting to the AD from Windows ??

Ram Baskaran says:

hadoop.security.group.mapping.ldap.search.filter.group
(objectclass=groups)

Ravi says:

In most cases, your user search filter should be (&(|(objectclass=person)(objectclass=applicationProcess))(sAMAccountName={0})) for AD

Veronica says:
Your comment is awaiting moderation.

The sample services.ldif file mentioned in Step1 is not accessible. Kindly make it available.

Sravanthi says:
Your comment is awaiting moderation.

Can you provide me a sample sevices.ldif file for adding hadoop service accounts to LDAP?I need it little urgently as my task is based on the same?It would be great of you if u can provide me ASAP.

And I have one more query. Should the value for “hadoop.security.group.mapping.ldap.bind.password” is LDAP Server authentication password ?

Sravanthi says:
Your comment is awaiting moderation.

Can u provide a sample file for services.ldif file to ass hadoop user accounts to LDAP server.It would be very greatful of you as it require for task urgently.

Alex McLintock says:

> Here is an example services.ldif

File not found

Venkat says:

@Alex : File is available at : https://web.archive.org/web/20140811040637/https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2014/03/services.txt

Laurent Edel says:

Hi Alex, here it is : https://gist.github.com/laurentedel/60c8d02254d7439a7ef7

Blake says:

> Here is an example services.ldif
File not found for me either. Can someone post this?

Kyle Dunn says:

To use LDAPS, you’ll also need to import the cert from the KDC into the default JDK keystore:

keytool -importcert -file rootCA.pem -alias kdc -keystore /usr/java/jdk1.8.0_73/jre/lib/security/cacerts

Ruslan says:

Does LDAP authentication have similar LDAP groups to local groups mapping mechanism like auth_to_local in Kerberos?

Ruslan says:

Correction: Does LDAP authentication have similar LDAP principals to local user mapping mechanism like auth_to_local in Kerberos?

Mike R says:

This is basic, but where is the proper place to update core-site.xml in an ambari managed cluster?

Sedat Kestepe says:

This setting is on core-site.xml file of HDFS.
Does it provide group mapping functionality for authorization for other applications? Like Hive, HBase? Is the HDFS the only level where it was checked? How does it work?

Sedat Kestepe says:

Can you also confirm if value of hadoop.security.group.mapping.ldap.base must be empty?

Richard Venus says:

The sample ldif provided won’t do the mapping correctly as the group map will use same group for all user which is not correct. I notice that on group adding side of ldif it user uid in the member: field and in user add part I I notice that it only register cn, so how would the system read the correct group map?

Leave a Reply

Your email address will not be published. Required fields are marked *

If you have specific technical questions, please post them in the Forums