With YARN as the architectural center of Apache Hadoop, multiple data access engines such as Apache™ Accumulo interact with data stored in the cluster. Accumulo is a low-latency, large table data storage and retrieval system with cell-level security. Accumulo is based on Google’s Bigtable and it runs on YARN, the data operating system of Hadoop. YARN provides visualization and analysis applications predictable access to data in Accumulo.
Cell-level access control is important for organizations with complex policies governing who is allowed to see data. It enables the intermingling of different data sets with access control policies for fine-grained access to data sets that have some sensitive elements. Those with permission to see sensitive data can work alongside co-worker without those privileges. Both users can access data in accordance with their permissions.
Without Accumulo, those policies are difficult to enforce systematically, but Accumulo encodes those rules for each individual data cell and controls fine-grained access.
Hortonworks Focus for Accumulo
The Apache Accumulo community is working on these improvements:
Recent Progress in Apache Accumulo
What Accumulo Does
Accumulo was originally developed at the National Security Agency, before it was contributed to the Apache Software Foundation as an open-source incubation project. Due to its origins in the intelligence community, Accumulo provides extremely fast access to data in massive tables, while also controlling access to its billions of rows and millions of columns down to the individual cell. This is known as fine-grained data access control.
Here is a list of some of Apache Accumulo’s most important features:
|Table design and configuration||
|Integrity and availability||
How Accumulo Works
Accumulo stores sorted key-value pairs. Sorting data by key allows rapid lookups of individual keys or scans over a range of keys. Since data is retrieved by key, the keys should contain the information that will be used to do the lookup.
- If retrieving data by a unique identifier, the identifier should be in the key.
- If retrieving data by its intrinsic features, such as values or words, the keys should contain those features.
The values may contain anything since they are not used for retrieval.
The original Big Table design has a row and column paradigm. Accumulo extends the column with an additional “visibility” label that provides the fine-grained access control.
Accumulo is written in Java, but a thrift proxy allows users to interact with Accumulo using C++, Python or Ruby. A pluggable user-authentication system allows LDAP connections to Accumulo. An HDFS class loader loads JARs from Hadoop Distributed File System (HDFS) to multiple servers. Accumulo also has connectors with other Apache projects such as Hive and Pig.