Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
prev slide
Trusting Open Source Tools to Drive Innovation
July 13, 2017
Big Data in Insurance: How Analytics Will Change the Game
Next slide

Big Data in Oil and Gas: Drilling Down for Insights

Digital technology is transforming some sectors more than others. Information and services companies are rapidly innovating and driving new efficiencies into their business using big data analytics. The traditional energy sector has been slower to embrace new technologies, but now companies are beginning to use big data in oil and gas applications to change the way they operate.

Data in the Energy Field

The fossil fuels energy sector is a notoriously slow adopter of new technology. Its complex, highly regulated workflows are built atop manual, mechanical systems rather than digital ones. Its data loads are also intense. It consumes a vast range of information—from large geological and seismological engineering files to intricate operational data communicated from sensors in the field.

Much of this data is gathered in real time via supervisory control and data acquisition (SCADA) systems that reach to every corner of the globe. This equipment has been installed in multiple phases by different teams, leading to a complex network of different industrial sensor and communications equipment.

Gathering that data is difficult enough; making sense of it all has been an even greater stumbling block because it’s locked into data silos, which are isolated from the rest of the organization. This makes it difficult to mine data for trends and patterns that could be useful in refining an energy company’s field operations.

Pressure Is Building

Nevertheless, oil and gas companies are moving toward a digital future in which analytics systems consume this data and surface new efficiencies for companies in the field. A perfect storm of business pressures is forcing this move.

On the business side, they’re facing increased commercial pressure from alternative forms of energy. This, in combination with depressed oil and gas pricing, is forcing traditional energy companies to find more efficiencies. On the operating side, increased regulatory scrutiny is forcing oil and gas companies to increase the monitoring and control options at their facilities. Many of these facilities are located in remote areas with slow connectivity links to central computing facilities, making the challenge harder.

Data volumes are also increasing. Historical data continues to build up, creating a wealth of information on which oil and gas companies could draw and marry with real-time data—if only they could easily unify it and see it in aggregate.

Benefits of Big Data in Oil and Gas

An oil and gas company with a greater command of its data could dramatically improve control over its infrastructure, say industry experts. By using analytics, companies could tune and automate some of their control loops, driving new efficiencies into their systems. Companies in this challenged sector could also use big data to marry control system and maintenance data, creating new opportunities for reducing downtime. This increases productivity and cuts costs.

How can oil and gas companies make their siloed data available, unlocking it so that they can create operational analytics initiatives with it? It begins with integrating operations technology and information technology.

Utilizing Connected Sensors

Part of this challenge involves connecting devices that aren’t feeding information into analytics systems. Proprietary, nonstandard data networks implemented over the years have stopped companies from collecting useful data. A cheap operational sensor in the field that’s connected to a wireless mesh network can make such data available to a centralized data analytics platform half a continent away, providing an oil company with more data about local conditions.

Using these techniques, oil and gas companies that have had limited visibility into flow rates, pressure, and temperature can now see those metrics with more granularity and more frequency. Marrying these with analysis of historic data allows the use of predictive analytics to detect future infrastructure issues.

Unifying Data

Another innovation involves extracting data that’s already in legacy enterprise data warehouses and moving it to data platforms that are built on open architectures. By unifying data this way, upstream, midstream, and downstream energy companies can work with modern analytics technologies, joining different data sets to improve their operations.

Aggregating data into a single data platform enables oil and gas companies to take advantage of modern analytical tools. One key tool is parallel processing, which can slice complex data sets and queries into multiple parts to execute them more quickly. This faster processing provides more democratic access to data across a wealth of data sets, so that experts from different operations and engineering teams can mine it for insights.

Revamping sensor and data infrastructures will prepare oil and gas companies with powerful predictive analytic capabilities. And this could be a game changer, particularly in 2017. It’s the year that industry professionals expect the sector to begin its rebound, according to Deloitte’s 2017 Oil and Gas Industry Survey: a perfect time to be looking toward the future.

To learn more, check out this video on predictive analytics and how it relates to the oil industry.

Leave a Reply

Your email address will not be published. Required fields are marked *