Hadoop is a business-critical data platform at many of the world’s largest enterprises. These corporations require a layered security model focusing on four aspects of security: authentication, authorization, auditing, and data protection. Hortonworks continues to innovate in each of these areas, along with other members of the Apache open source community. In this blog, we will look at the authentication layer and how we can enforce strong authentication in HDP via Kerberos.…
The Hortonworks Blog
IBM InfoSphere Guardium has certified with HDP 2.1. The Hortonworks Certified Technology Program simplifies big data planning by providing pre-built and validated integrations between leading enterprise technologies and HDP.
Kathryn Zeidenstein, InfoSphere Guardium Evangelist, is our guest blogger and describes security, Hadoop, and the Guardium solution.
Those of us in the data security and privacy space tend to worry a lot. With each new breaking story on the latest data breach, and with the subsequent fallout, people higher and higher up the food chain are also worrying a lot.…
This week we hosted a webinar entitled HDP Advanced Security: Comprehensive Security for Enterprise Hadoop. Over 135 people attended, prompting an informative discourse and a series of questions.
The speakers outlined the HDP Advanced Security features and benefits in Hortonworks Data Platform and gave a demo. Thanks to our presenters Justin Sears (Hortonworks’ Product Marketing Manager), Balaji Ganesan (Hortonworks’ Senior Director, Enterprise Security Strategy), and Don Bosco Durai (Hortonworks’ Enterprise Security Architect).…
Two months ago, we announced the acquisition of XA Secure. and at that time we stated that the software would be generally available by the end of June. We are happy to announce that we have delivered as promised and the solution is available for download for everyone today. Also, if you are an HDP Enterprise Plus Subscription customer, additional support for these new functions is now provided.
HDP Advanced Security expands on the solid security features already found in HDP to provide central administration and coordinated enforcement of enterprise security policy for a Hadoop cluster.…
It has been an exciting few weeks for the XA Secure team. We formally joined Hortonworks on 5/15 and have received a warm welcome from our new peers. Even more exciting are the numerous discussions we have had with current customers and prospects on how we can bring together a comprehensive and holistic security capabilities to HDP. We now begin the journey to incubate our XA Secure functionality as a completely open source project governed by the Apache Software Foundation.…
Customers’ Hadoop Journey
We’ve all had two weeks to reflect on Hadoop Summit 2014. One of the biggest differences that stood out in this year’s Summit (as compared to Summit 2013) was the presence of large enterprise customers that are using Apache Hadoop as an important part of their modern data architectures.
Hadoop has gone beyond its original Yahoo use case—indexing the web via a nightly batch MapReduce process —and into the mainstream of daily data processing and analytics with real-time, online, interactive, and batch applications at many notable companies.…
Enterprises are using Apache Hadoop powered by YARN as a Data Operating System to run multiple workloads and use cases instead of using it just as a single purpose cluster.
A multi-purpose enterprise wide data platform often referred to as a data lake gives rise to the need for a comprehensive approach to security across the Hadoop platform and the workloads. Few weeks back Hortonworks acquired XA Secure to further execute on our vision to bring a holistic security framework to the Hadoop community irrespective of the workload.…
Today we’re delighted to announce our acquisition of XA Secure to provide comprehensive security capabilities for Enterprise Hadoop. Please join us in welcoming XA Secure to the Hortonworks family.
Hortonworks Data Platform has seen phenomenal adoption across an ever-growing number of organizations. As part of that adoption, and thanks to Apache Hadoop YARN, businesses are moving from single-purpose Hadoop clusters to a versatile, integrated data platform hosting multiple business applications – combining data sets with diverse processing needs in one place.…
Last week Vinay Shukla and Kevin Minder hosted the first of our seven Discover HDP 2.1 webinars. Vinay and Kevin covered three important topics related to new Apache Hadoop security features in HDP 2.1:
- REST API security with Apache Knox Gateway
- HDFS security with Access Control Lists (ACLs)
- SQL security and next-generation Hive authorization
Here is the complete recording of the webinar.
Here are the presentation slides: http://www.slideshare.net/hortonworks/discoverhdp21security
Attend our next Discover HDP 2.1 webinar tomorrow, Thursday, May 15 at 10am Pacific Time: Interactive SQL Query in Hadoop with Apache Hive
We’re grateful to the many participants who joined and asked excellent questions.…
Rainstor is a Hortonworks Certified Technology Partner and provides an efficient database that reduces the cost, complexity and compliance risk of managing enterprise data. RainStor’s patented technology enables customers to cut infrastructure costs and scales anywhere; on-premise or in the cloud and natively on Hadoop. RainStor’s customers are 20 of the world’s largest communications providers and 10 of the biggest banks and financial services organizations.
Rainstor’s Mark Cusack, Chief Architect, writes about the benefits of certification on HDP 2.1.…
The Apache Knox Gateway team is pleased to announce Knox’s first release as an Apache top-level project: Apache Knox Gateway 0.4.0. The team resolved approximately 100 JIRAs for this release and Knox Gateway is now better positioned to provide complete security for REST API access to a Hadoop cluster.
The new features in Knox Gateway 0.4.0 are the features that enterprise security officers expect in a gateway solution:
- Perimeter security for a Hadoop cluster
- Support for enterprise group lookup
- Audit log of all gateway activity
- Command line tooling for CMF provisioning
- Protection for web application vulnerabilities
- Pre-authentication via SSO token
- And many more…
As a top-level project, Apache Knox Gateway is fully endorsed by the Apache Software Foundation, and this improves coordination between development of Knox and the other core Hadoop projects with which it interacts.…
Securing any system requires you to implement layers of protection. Access Control Lists (ACLs) are typically applied to data to restrict access to data to approved entities. Application of ACLs at every layer of access for data is critical to secure a system. The layers for hadoop are depicted in this diagram and in this post we will cover the lowest level of access… ACLs for HDFS.
This is part of the HDFS Developer Trail series. …
The pace of innovation within the Apache Hadoop community is truly remarkable, enabling us to announce the availability of Hortonworks Data Platform 2.1, incorporating the very latest innovations from the Hadoop community in an integrated, tested, and completely open enterprise data platform.
What’s In Hortonworks Data Platform 2.1?
The advancements in HDP 2.1 span every aspect of Enterprise Hadoop: from data management, data access, integration & governance, security and operations. …
LDAP provides a central source for maintaining users and groups within an enterprise. There are two ways to use LDAP groups within Hadoop. The first is to use OS level configuration to read LDAP groups. The second is to explicitly configure Hadoop to use LDAP-based group mapping.
Here is an overview of steps to configure Hadoop explicitly to use groups stored in LDAP.
- Create Hadoop service accounts in LDAP
- Shutdown HDFS NameNode & YARN ResourceManager
- Modify core-site.xml to point to LDAP for group mapping
- Re-start HDFS NameNode & YARN ResourceManager
- Verify LDAP based group mapping
Prerequisites: Access to LDAP and the connection details are available.…
Encryption is applied to electronic information in order to ensure its privacy and confidentiality. Typically, we think of protecting data as it rests or in motion. Wire Encryption protects the latter as data moves through Hadoop over RPC, HTTP, Data Transfer Protocol (DTP), and JDBC.
Let’s cover the configuration required to encrypt each of these protocols. To see the step-by-step instructions please see the HDP 2.0 documentation.
The most common way for a client to interact with a Hadoop cluster is through RPC. …