Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
January 19, 2017
prev slideNext slide

Big Data Analytics At A Tipping Point

Last year Hadoop celebrated it’s tenth birthday, young in the land of data technologies. But the growth in popularity of Apache Hadoop is not slowing down anytime soon. In fact, results from the 2016 Big Data Maturity Survey indicates 97% of respondents plan to do more big data initiatives in the next 3 months.

The Rise Of Big Data Analytics or Business Intelligence

More interesting in the survey though is the rise in business intelligence (BI), overtaking ETL as the primary workload. Over the years we have grown accustomed with the data science and data preparation workloads running on Big Data technologies, but now 75% of respondents will be running BI on their Big Data deployments.

So, why do I view this as the Big Data Tipping Point?  In my view it means Big Data is growing up, becoming mission critical, is becoming trusted by the business. A few key factors are shaping this.

Fast SQL

For Big Data to really become come of age and be the mission critical data platform for BI, it first needs to support interactive SQL style queries to support an iterative OLAP style analysis. These fast Big Data SQL technologies are now readily available through Hive LLAP, Spark SQL, AtScale and others.

Run Hadoop Anywhere

The switch for enterprises to run their applications in the cloud has dominated IT strategy for the past number of years, but BI in the cloud has not exactly followed the same path. That is until now. In the Big Data Maturity Survey 72% of respondents this year indicate they will run Big Data workloads in the cloud compared to only 53% that currently run in the cloud. Offerings like Microsoft Azure HDInsight and more recently Hortonworks Data Cloud for AWS mean there is a growing list of cloud native big data services to tap into for all kinds of common data use cases.

Importance of Governance

Along with the rise in BI (more users all over the enterprise) and running this in the cloud, the need for better security and governance is paramount. In fact the survey bears this out with Governance as the fastest growing concern, up 21% over last year. Over the past 12 months many of the Apache Foundation efforts to bring Apache Ambari, Apache Atlas and Apache Ranger to the next level are clear indications that the voice of the enterprise customer is being heard and understood in the community.

Parting thoughts

One last statistic that caught my eye in this year’s survey was who drives the big data initiative.  And it won’t come as a surprise that IT (at 57%) is still the main driver, but interestingly, when business executives drive they are 19% more likely to achieve tangible value. (BTW Did you know Hortonworks can help you drive big data success with tutorials, training, guidance, industry experience and support?


Want to get more insights from the report? Get your copy of the 2016 Big Data Maturity Survey.  


  • We are in agreement. If you watch the demo you will see pretty good interactive OLAP style analysis. Overall, this has been a focus for us with Hive. Recently we shared some of our own benchmarks with Hive with LLAP which essentially is an in-memory add-on to Hive. And the query performance is impressive and really get us into that interactive, iterative style of analysis.

    • The survey is executed by our partner AtScale. The 2016 survey was executed late in 2016. My expectation is that they will do the 217 version later this year.

  • Big Data is finding more applications in various fields now and hence also growing at an unprecedented pace. With this amazing growth a huge gap is growing between the demand and supply of big data professionals.

  • Nice and very interesting blog to know big data analyticS. The big data were growing everywhere which is quite difficult to analyze for the business folks. The business intelligence techniques was quite helpful to analyze data instantly.

  • Leave a Reply

    Your email address will not be published. Required fields are marked *