7 Key Drivers for the Big Data Market

I attended the Goldman Sachs Cloud Conference and participated on a panel focused on “Data: The New Competitive Advantage”. The panel covered a wide range of questions, but kicked off covering two basic questions:

“What is Big Data?” and “What are the drivers behind the Big Data market?”

While most definitions of Big Data focus on the new forms of unstructured data flowing through businesses with new levels of “volume, velocity, variety, and complexity”, I tend to answer the question using a simple equation:

Big Data = Transactions + Interactions + Observations

The following graphic illustrates what I mean:

Big Data Diagram

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ERP, SCM, CRM, and transactional Web applications are classic examples of systems processing Transactions. Highly structured data in these systems is typically stored in SQL databases.

Interactions are about how people and things interact with each other or with your business. Web Logs, User Click Streams, Social Interactions & Feeds, and User-Generated Content are classic places to find Interaction data.

Observational data tends to come from the “Internet of Things”. Sensors for heat, motion, pressure and RFID and GPS chips within such things as mobile devices, ATM machines, and even aircraft engines provide just some examples of “things” that output Observation data.

With that basic definition of Big Data as background, let’s answer the question:

What are the 7 Key Drivers Behind the Big Data Market?

Business

  1. Opportunity to enable innovative new business models
  2. Potential for new insights that drive competitive advantage

Technical

  1. Data collected and stored continues to grow exponentially
  2. Data is increasingly everywhere and in many formats
  3. Traditional solutions are failing under new requirements

Financial

  1. Cost of data systems, as a percentage of IT spend, continues to grow
  2. Cost advantages of commodity hardware & open source software

There’s a new generation of data management technologies, such as Apache Hadoop, that are providing an innovative and cost effective foundation for the emerging landscape of Big Data processing and analytics solutions. Needless to say, I’m excited to see how this market will mature and grow over the coming years.

Key Takeaway

Being able to dovetail the classic world of Transactions with the new(er) worlds of Interactions and Observations in ways that drives more business, enhances productivity, or discovers new and lucrative business opportunities is why Big Data is important.

One promise of Big Data is that companies who get good at collecting, aggregating, refining, analyzing, and maximizing the value derived from Transactions, Interactions, and Observations will put themselves in a position to answer such questions as:

What are the behaviors that lead to the transaction?

And even more interestingly:

How can I better encourage those behaviors and grow my business?

So ask yourself, what’s your Big Data strategy?

~ Shaun Connolly

Categorized by :
Apache Hadoop Big Data Industry Happenings

Comments

Raakesh Kaul
|
January 10, 2014 at 6:18 am
|

Shaun,

Beautifully summarized to help understand the value of Big data given the ever changing consumer behavior, albeit very few CIO’s have either adopted or leveraged the benefits regardless of nature of companies business. .

|
September 13, 2013 at 8:01 am
|

I agree and appreciate everyone’s comment posted here. I feel we need to highlight the importance of distingushing “Data” from “Information”. Data is RAW and Information is PROCESSED DATA which carries some meaning. We should be particular about the sanctity, accuracy, reliability, compliance, accountability, security etc of Big Data. Mere Big Data does not give insight. Secondly, Big Data outcome should be in sync with the varieties of sources and processes/ methodologies from where data is churned out to facilitate meaningful decisions. Business world keeps changing along with Business rules. Therefore, we should guide the industry with some charter and guidelines to educate and channelize them in the right direction to build frameworks of Big Data Analytics. Patterns with successful results could be shared across domains to adopt Best Practices. Investments on Big Data should yield meaningful return….Partha Mohanty, e-Governance Research, MGRM

|
May 24, 2012 at 7:00 am
|

Shaun,

Great blog post. Many articles are focusing on the issue of Big Data, but few illustrate the tangible benefits that this revolution provides.

We are focusing on the benefits for businesses by helping them use Big Data for sales insight. (Please see our chapter, “Finding Big Growth in Big Data” in McKinsey & Company’s new book: http://pages.lattice-engines.com/mckinsey-book-sales-growth.html)

We’ve seen the tremendous difference between organizations that capitalize on big data and those who are left behind. Our customers are in the first category and are blowing their competitors away, surfacing to the top of their industries. Hopefully, many more will continue to follow this trend.

The opportunity for growth is extraordinary.

- Alicia Brayboy, Lattice Engines, @Lattice_Engines

|
May 20, 2012 at 7:06 am
|

Shaun:
Good post; the key element that’s increasingly become more important to businesses is the ability to rapidly (repeat, rapidly) do the deep dive analytics on the vast amounts of data. Volume and Velocity are absolute keys.

Whether it’s adding an analytical accelerator to an existing EDW, or designing a system from scratch that will seamlessly pull in data from multiple, disparate sources, time-to-results has become the imperative. It’s what companies of all sizes are demanding, because they know it’s become increasingly available at a price point that’s no longer prohibitive.

|
May 15, 2012 at 9:52 am
|

Thanks for the comment Doug.

I checked out the link you provided and in the comments of that post you mention “I’ve recently expanded to 12V’s that cover the full spectrum of big data challenges and opportunities”. Did you publish anything on those 12 V’s yet? I’m always looking for additional layers of detail that can help people understand the topic.

|
May 15, 2012 at 8:01 am
|

Great piece Shaun. I like the equation which equates to what I’ve been calling “subtransactional data.” Also cool to see the industry finally adopting the “3V”s of big data over a decade after Gartner first defined them. For future reference, and a copy of the original article I wrote in 2001, see: http://blogs.gartner.com/doug-laney/deja-vvvue-others-claiming-gartners-volume-velocity-variety-construct-for-big-data/. –Doug Laney, VP Research, Gartner, @doug_laney

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.

Thank you for subscribing!