Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
January 21, 2016
prev slideNext slide

`>echo “Hello, world.”

Hello everyone and welcome to the start of my blogging adventure. I’m Mike Schiebel, Cybersecurity Strategist at Hortonworks where I’m focused on cybersecurity to inject enterprise level security features into the Hadoop ecosystem and provide input into the Apache Metron open source project.  I figured introductions are in order, to explain the where and why behind my blog series.

Who am I?

I’ve taken a long and twisting road before ending up at Hortonworks. I’m a military veteran who served as an aircraft electrician during Desert Shield, Desert Storm, and Provide Comfort.  I’ve been an IT consultant, freelance Microsoft Certified Trainer (MCSE, MCT), Cyber Incident Response Analyst (GCIA, GCIH, CREA), Computer Forensic Investigator (EnCE), Security Engineer (RHCT, Linux+, Network+, Project+), Security Architect (CISSP), and a Security Executive. I’m heavily involved with several not-for-profit cybersecurity organizations, presenting at conferences and helping mentor many cybersecurity professionals.

In the 15 years that I’ve been in cybersecurity, I’ve been on the frontline of every area, working to solve the challenges in medium and large healthcare and financial services enterprises. I’ve designed and managed security systems, developed enterprise security architectures, written business plans justifying annual budgets, and developed multi-year security strategies and roadmaps.

The profession has grown but: frequency and scope of breaches are at epidemic levels

I’ve had the fortune to see the state of the profession move from GRC checkboxes and militarized CERT processes, through the advances of risk quantification using FAIR (Factored Analysis of Information Risk), to embedding sound risk based management into application architectures using TOGAF based processes.  For me, this has been both an amazing yet frustrating experience. While the state of the profession has grown by leaps and bounds, the frequency and scope of computer breaches has grown to truly epidemic levels.  When the common jaded comment from a security practitioner is “There are two types of companies; those that know that they’ve been hacked, and those that don’t”. Clearly, I as a security practitioner, couldn’t just do more of the same tried and true approach of buying more security point solutions for every new style of attack and slowing down IT & development with an ever larger list of non-functional security requirements.  Growing a reputation, as a business leader once told me, “security, the people who put the ‘no’ in innovation” wasn’t helping solve the core business challenges.

My first light bulb moment: A rules-based system approach to detection was dead

The light bulb moment for me was during a penetration test engagement. It was a full red/white/blue team exercise simulating a targeted attack.


  • The red team, the attackers, were an external consultancy with zero inside knowledge; they would only have publicly accessible knowledge to plan and execute against the goal we provided them.
  • The blue team, the defenders, consisted of our internal security operation center (SOC) and IT folks who had no knowledge or warning that they were going to be tested.
  • The white team, the observers, were a combination of external consultants and internal people with full visibility to the actions of both the red and blue teams.  Their job was to time the detection and response of the blue team, and measure the effectiveness of the security controls in blocking and detecting the red team activities.

I remember sitting as part of the white team and realizing that the total visibility to the entire attack was two intrusion detection events over the period of two hours.  Those two events were buried within tens of thousands of other events and never followed up by the blue (defense) team.  After those two events the red (attacker) team had gained access to an internal user context and appeared to be a normal user on the network. At that moment, I realized that the rules based system approach to detection was dead; I needed a strategy to migrate to an anomaly based detection approach. Unfortunately, the products in the marketplace were lacking; there were some interesting ideas, however, the features to price ratio just wasn’t something I was willing to fight the battles to get funding for.

Effective and cost-efficient options are not available today

Not seeing a product in the market that met the fundamental requirements I believed were required, I gave a presentation at a security conference detailing the features and requirements, illustrating why they mattered. Over two years without seeing those requirements being implemented, I continued to fill out my design and create an incident response architecture with security analytics at the core.  I again gave a presentation as a 101 level tutorial on how basic data analytics concepts such as time series predictive analytics could be leveraged to make the SOC folks lives easier. I ended the talk challenging folks to come together and create an open project to make it happen.

I went back to my day job building a business case to build the incident response architecture internally as part of a multi-year security strategy.  I realized that the creation of an isolated security data grid was hard to justify. Most of the data required was the same as the business and IT areas needed and keeping multiple copies of the same petabytes wasn’t a good use of company resources.  What was really needed as an integrated approach that allowed for the data to be shared – an enterprise data vault that could collect and secure all enterprise data, regardless of sensitivity, allow analytics on the raw data, and provide desensitized versions of the results based on the principle of least privilege.  In building the business case, I realized that the security need was the anti-pattern to the business needs.  To detect fraud – anomalous customer behavior – I first needed to predict normal customer behavior.  That was my second lightbulb moment so let me repeat that: To detect the abnormal I need to predict the normal. 

My second light bulb moment: To detect the abnormal I need to predict the normalmichaels2

Predicting the normal is the core business & IT feature needed to create efficient & agile enterprises.  It would be a waste of resources to not leverage both sides of the analytic value chain.  By creating an integrated data analytic platform business and security needs could be addressed by all analytic development with a simple feedback loop.  How could this predictive business model improve anomaly detection?  How could this new security anomaly detection provide greater business prediction?

Working Together For An Open Cybersecurity Platform

During this time I encountered the open source project that has become Apache Metron (incubating). I knew then I needed to get involved. In short, that is why I decided to join Hortonworks and start this blog series.  I hope this blog will be a place we can all come together and create a vision for cybersecurity in the new hyper converged cloud centric paradigm. I ask you to help me make this a success, please send me your comments and feedback on where you believe we are and where you think the state of the industry should go.  Just generating a place for a healthy debate will help move us all forward; applying the open source mantra of many eyes makes problems shallow isn’t just for software. My hope is the creation of an open and interoperable platform for the new paradigm we find ourselves in. I firmly believe Hortonworks is well positioned in the industry with the right open sourced focused culture to make this happen. I’m really excited to be here. Are you?

michaels3Michael Schiebel, Cybersecurity Strategist

Connect with me on LinkedIn

Leave a Reply

Your email address will not be published. Required fields are marked *