Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
March 01, 2017
prev slideNext slide

Why Does Cybersecurity Need Big Data and Machine Learning?

Big Data for Cybersecurity

Modern information security encompass broader data sets than in the past, in order to create context and generate a complete picture of network data, user behaviour pattern and business data  – all combined together so that a trendline of normal operations can be created. Then from that, it is possible to determine deviations from the norm (Reference  “To detect the abnormal I need to predict the normal” by Michael Schiebel.

Manual Processes Can’t Match Rapidly Evolving Patterns

As the hyperconnected digital world produces cybersecurity data at a volume and rate that fits the definition of big data, the next step of processing that data needs to be addressed. Hackers have the power of nation states and are constantly attacking enterprises for the information they contain – credit cards, passwords, healthcare data, and more.  These threats are no longer isolated point attacks in time, but are carefully choreographed and orchestrated, sometimes laying stealthy traps of advanced persistent threats that are not simple to uncover and mitigate through existing point solutions and manual processes.

Machine Learning for Cybersecurity

It is impractical for security personnel to manually piece together all the relevant security data to detect threats, and modern cybersecurity solutions need to lean on the automation of manual tasks of processing very large sample sets made possible through big data and machine learning. And as the hackers constantly evolve their game, security teams need to adapt at the same time, at the same rate, to efficiently detect and interpret the signs of the most relevant threats which require further investigation, and to quickly respond to evolving threats.

A Faster Path to Innovation: Don’t Re-Invent the Wheel

There is incredible energy and innovation targeted at applying big data and machine learning to the realm of security right now. However, consuming that innovation is difficult to scale as innovators are solving the same data engineering problems over and over. After all, cybersecurity isn’t the only industry that has a deluge of data to be processed  – businesses in a range of industries face similar changes in processing large volumes of streaming data in real-time. Thus the open source community initiated Apache Metron , a big data cybersecurity platform that enables a single view of diverse, streaming security data at scale to aid security operations centers in rapidly detecting and responding to threats.

Apache Metron: A Big Data Cybersecurity Platform

Apache Metron is a faster path to innovation, encapsulating a decade of big data science and streaming analytics experience into a curated package of technology for security teams to utilize. It gives security personnel a platform to quickly leverage machine learning for real time profiling and statistical analysis, without the need to expend resources on the repeatable data engineering problems involved in creating the baseline infrastructure for an enterprise ready machine learning cybersecurity platform. In the end, this means accelerated times to respond to the ever-evolving threats generated by nation states intent on attacking and breaching enterprises assets.

Learn More: Why Cybersecurity Needs Big Data Webinar

To learn more check out Hortonworks cybersecurity solution or join our upcoming webinar with Michael Schiebel, General Manager, Cybersecurity, and James Sirota, Apache Metron committer and Director of Security Solutions on March 14, 2017 to learn more on why cybersecurity needs big data.



Ryan Persaud says:

Look forward to hearing the state of the project.

Anna Yong says:

Thanks for the interest! During the webinar please do let us know if you have any specific questions about the status you are interested in.

Youngman says:

Look forward to hearing the state of the project.

Anna Yong says:

Thanks for the interest! During the webinar please do let us know if you have any specific questions about the status you are interested in.

Leave a Reply

Your email address will not be published. Required fields are marked *