Extending Apache Accumulo Support in Hadoop with Hortonworks HDP and Sqrrl
Apache Accumulo is gaining momentum in markets such as government, financial services and health care for its enhanced security and performance. Hortonworks has a long history with this technology and has multiple committers to the Accumulo project on staff – at least one of whom literally helped to write the book on Accumulo. This has enabled Hortonworks to provide enterprise support for Accumulo within the Hortonworks Data Platform for some time now. For those interested, more specifics can be found in our support datasheet.
Since many users have very advanced requirements when working with Accumulo, we often work closely with Sqrrl, who have built extensions to Accumulo adding enterprise-grade functionality that have wide appeal. Here’s what Ely Kahn, vp of business development at Sqrrl has to say.
Cell-Level Security for Health Care
Regulated industries such as health care are very interested in security features due to HIPAA compliance demands and data requirements that stem from the Affordable Care Act. Health care providers are investigating solutions that Sqrrl offers because of the cell-level security and access control that allow them to do new things like data sharing while maintaining compliancy, and data encryption, both at rest and in motion.
These same features have captured the interest of the financial and telecommunication sectors as well. Due to some added analytic capabilities in Sqrrl Enterprise, Hortonworks and Sqrrl partnered recently to develop a “Big Data Security Analytics application” for a large telecommunications provider.
Big Data Security Analytics for Telecom
ESG analyst Jon Olstik qualifies Big Data Security Analytics with the following:
“Security and IT operations tools spit out an avalanche of data like logs, events, packets, flow data, asset data, configuration data, and assortment of other things on a daily basis. Security professionals need to be able to access and analyze this data in real-time in order to mitigate risk, detect incidents, and respond to breaches. These tasks have come to the point where they are difficult to process using on-hand data management tools or traditional (security) data processing applications.”
This Big Data Security Analytics application enables the telecommunications provider to analyze massive amounts of cybersecurity data in real-time. This is data that would be too burdensome and expensive to analyze with traditional tools. The joint Sqrrl/Hortonworks solution relies on Sqrrl’s real-time NoSQL capabilities and Hortonworks’ enterprise-grade distribution, Hortonworks Data Platform (HDP).
Like many open source software solutions (Hadoop included), Accumulo can be challenging to get started for many users. Sqrrl provides added software to help would-be Accumulo downloaders with a user experience expected in enterprise level software.
Sqrrl provides critical extensions to Accumulo that include:
- Secure search which enables integration of Accumulo’s cell-level security capabilities with a variety of search capabilities, such as:
- Secure SQL search to enable real-time aggregations of multi-structured data
- Secure full-text search, using the Lucene syntax to enable keyword search
- Secure graph search, to enable exploration of how data is connected
- JSON support, to enable development of document-style data models
- High concurrency to power applications supporting large numbers of users
- A policy engine and labeling engine to simplify the application of fine-grained security labels to datasets and to enable both Attribute Based and Role Based Access Controls.
HDP and Sqrrl Enterprise in the Modern Data Architecture
Sqrrl Enterprise is powered by Accumulo and utilizes the Hadoop Distributed File System for storage of multi-structured data. The integration between Sqrrl Enterprise and the Hortonworks Data Platform is seamless and enables organizations to take advantage of the advanced features available in Accumulo. Here is how Sqrrl Enterprise fits in the Modern Data Architecture with HDP:
For a more detailed Sqrrl/Hortonworks reference architecture, you can download it here.