Can you identify the unused data in your data warehouse? Are you using your “big data” efficiently? Are your data migration projects cost effective? Is your data in compliance with industry regulations? If you answered “no” to any or all of these questions, then you may want to learn more about how to optimize your data warehouse.
On April 23rd at 11:00 am PST, Adis Cesir, Big Data Solution Engineer at Hortonworks, Ramu Kalvakuntla, Principal at RCG Global Services Big Data Practice, and Santosh Chitakki, Director of Product Management at Attunity, will be telling us more about rebalancing data warehouses and integrating your current enterprise data warehouse with a Modern Data Architecture.
In this blog, they provide answers to some of the most frequently asked questions they have heard on the topic.
1. When we talk about “optimizing of the value associated with the EDW”, what do you mean exactly?
Adis: There are three primary drivers to EDW optimization.
First, it is about storage optimization. It consists in offloading data on to the right platform based on its usage patterns. For example, data not being used for analytics but required to be loaded and maintained for regulatory purposes may require a different storage strategy than data that is very frequently used for daily operational reporting.
Second, companies optimize data processing: typically, a large portion of the EDW usage is for low value transformational workloads. Many of these can be transitioned away from the EDW and into Hadoop. This process frees up significant resources from the EDW.
Finally, Hadoop can be used to capture new types of data that can then be refined and used within the context of the analysis of your EDW, introducing wholly new analysis and insight.
2. How would I know what data to offload off of my enterprise data warehouse?
Santosh: With help of Attunity Visibility, a software solution that provides data usage analytics, organizations can get factual information on data usage patterns. The information provided by Attunity Visibility can be easily shared with the business stakeholders to quickly develop a roadmap to offload data from the enterprise data warehouse.
3. Can you give examples of companies that have modernized their enterprise data warehouse and ROI they have achieved?
Ramu and Santosh: We have done the analysis for two large retailors where offloading projections are between $6M to $10M by offloading unused data and ELT processing. We are also working with a large financial company where we are offloading 60TB of RAW data from Teradata into Hadoop along with their decision batch processing, with projected savings around $10M.
4. How does EDW optimization enable advanced analytics? How are the two related?
Adis, Ramu, Santosh: EDW Optimization is usually the first step of building the data lake for advanced analytics platform. It usually provides immediate ROI. Organizations can bring all their data right from source systems within enterprise and outside their enterprise to build one version of truth in the Data Lake. In doing so, they can build an analytics platform within weeks and months – compared to traditional EDW that take multiple months and years before users can access the data for analytics. At the same time, IT can get cost savings by offloading all ETL work within Hadoop and moving only limited data into their traditional EDW/MPP systems for optimal performance.