This is a guest blog from Stefan Kupstaitis-Dunkler, Accenture Technology Solutions GmbH.
I’ve been working at Accenture for almost a year and last month I was invited to attend the partner masterclass on HDP 2.3 Security. The classroom setting was a great forum for interactive discussions and a showcase of the security capabilities in the newest version of the Hortonworks Hadoop distribution, HDP 2.3.
Sean Roberts, Hortonworks Solution Engineer and hadoop operations expert in EMEA, guided the attendees through a demonstration of what Hortonworks Data Platform does to integrate the security aspects into Hadoop.
The masterclass was also a very practical approach to introduce the audience to “The Five Pillars of (not solely Hadoop) Security”. Furthermore, Sean suggested a few non-security related features of HDP 2.x, such as Ambari “Plugins”. The class was delivered by giving everyone access to a personal virtual one-node cluster to experiment with – this was pre-installed and configured with Apache Ambari 2.1 and HDP 2.3.
Without going too deep into the technical details, here’s a flavour of what was covered.
Hortonworks provides solutions for all of the above and the participants received some great hands-on experience. We started with Apache Ambari, which is not a security tool, but needs to be secured in order to avoid a weak link. Apache Ranger was introduced as the key to solve three of the above mentioned security problems, i.e., authorization, authorization administration and auditing. Apache Ranger had great improvements and added many features from HDP 2.2 to HDP 2.3 – including providing new plugins for three more services, Apache Hadoop YARN, Apache SolR and Apache Kafka. Ranger can also push audits into SolR in order to visualize the audit data and make it more accessible in general.
Other topics covered were HDFS TDE (transparent data encryption) and the introduction of the related new Ranger KMS (key managing store). Apache Knox was not forgotten as the tool to secure the perimeter of a Hadoop cluster. Lastly, Kerberos was covered which highlighted the simplicity of installation and activation. On one hand, Ambari creates keytabs and the corresponding service principals in a KDC of choice for all hosts and services managed in Ambari which saves time. On the other hand, the critical task with Kerberos and AD is still the configuration, which no tool can take over.
The masterclass was an useful, intensive 8 hours of fun and I will definitely be diving deeper into topics like “How to develop applications in or for secure Hadoop clusters”. Sean referred us to other Hortonworks developers, who were present in the classroom in the event we wanted to explore this topic in more detail. I specifically took the opportunity during the breaks to ask HBase related questions and hopefully these will be answered via email.
One of the most awesome new things that I discovered at the masterclass was not necessarily security related. The so-called Hortonworks Gallery which is a collection of plugins, views, blueprints and other Hortonworks related Hadoop stuff. I actually told every person I met since the masterclass about this gallery. Another useful online resource is Sean’s github repo “ambari-bootstrap”.
In summary, I enjoyed the day at the Hortonworks Security Masterclass and I’ll be looking forward to the next masterclass on “Data Governance” in September. A very informative, interesting session coupled with some great opportunities to make new contacts. It’s important to protect sensitive client’s data. Nothing less and nothing more. HDP 2.3 provides these important capabilities to the enterprise.