This guest blog comes from Attivio, a certified Hortonworks Partner with the Governance Ready Badge, indicating that the Attivio Platform is recognized by Hortonworks as an application that integrates with Atlas APIs to access and create business taxonomical metadata like data lineage and tags. Here, Joe Lichtman, VP of Product Management at Attivio shares the goals and requirements of building a Modern Data Architecture that includes enterprise-level governance.
Most large companies have been using Hadoop for a while, but for some it’s been more of a sandbox than a production environment. More often than not, its use has been limited to data scientists. But now as Hadoop has matured with enterprise-level distributions like Hortonworks Data Platform, organizations have begun to adopt it as their new data storage platform. And that means they need to connect this new platform to legacy data sources like the enterprise data warehouse. Otherwise, Hadoop simply becomes another data silo.
It’s this need that underpins the modern data architecture. A modern (or hybrid) data architecture stores data as is. It handles the volume, velocity, and variety of Big Data, without pre-modeling (schema on read). Of course, once you have an architecture that accommodates modern and legacy data stores, you need a way to access that data without moving it. That’s where a semantic data catalog comes in.
With a data catalog, a Hadoop platform such as HDP has capabilities with unstructured and data source discovery and management that didn’t exist before without a lot of custom coding and configuration. With Attivio’s data catalog, Hadoop can be integrated into the enterprise data supply chain. And HDP helps Attivio process massive data sets in an environment that has become the de facto enterprise standard for any organization undertaking Big Data initiatives.
Adding Enterprise-Level Governance to Hadoop
Attivio is one of the first products to integrate with Apache Atlas—the data governance initiative. Atlas is critical because it’s a metadata store that facilitates governance—making it simultaneously more agile and robust. We’re using our data source discovery expertise and understanding of the origins of data and metadata to help improve the performance of Atlas and automate governance in ways that weren’t possible before.
Especially for companies that operate in regulated industries, the value of trusted governance can’t be underestimated. With Atlas as part of HDP, Hortonworks now delivers analytical power coupled with robust governance, which makes it a true enterprise solution for Big Data.
Putting It All Together
The maturing of Hadoop governance is coming in the nick of time. The biggest use cases we see in Hadoop these days come from the risk and compliance functions of large banks. These global Financial Services Institutions need to establish an effective data ecosystem under strict governance and regulatory requirements.
Join us on December 1st at 10 am PT, 1pm ET for a webinar that details the effort of one global banking leader to build an effective, data-driven anti-money laundering solution.