June has got off to a great start – and not only because it seems like summer has arrived in London! Yesterday, our team gathered in our International HQ a stone’s throw from Liverpool Street Station for a session with Mike Schiebel, our cyber security strategist, who is visiting from the west coast. We have numerous visitors, including our own executives, customers, partners and wider members of the community coming into our London office so I’m going to start nabbing 5 minute videos to share their insights on the world of Connected Data Platforms!
The role of data in helping organisations manage the constant threat of cyber-attacks is a growing conversation, not least due to projects like Apache Metron, with our customers and I found Mike’s insights incredibly helpful. Before joining Hortonworks 6 months ago, Mike was on the frontline customer-side for around 15 years as analyst, trainer, investigator, engineer, architect and executive so I’m somewhat comfortable (forgive my British understatement…!) with his real-world vigour when it comes to helping us all get to grips with building a business case for data helping the ever-present risk of cyber-security.
At a basic level, we all know that any business can only take on so much risk before it fails. It struck me as Mike was talking through the role of the CISO and his/her team that the crux of it comes down to enabling the business to be well-balanced and agile enough to manage risk – enough to enable its success yet not too much to threaten its demise.
Despite the criticality of their role, I can imagine it can be disheartening to be sometimes mistaken as a cost centre – a team which is responsible for protecting the business but one which is also often playing catch up to the IT team and the lines of business. Ironically, it’s a position not aided by the cyber security experts doing a great job. As Mike points out, if they are doing their job well then absolutely nothing happens!
Therefore, the role of data is about making the invisible, visible. Unlocking the metrics and analytics that helps justify the required investments. Helping to quantify the avoided risks in hard £’s, €’s and $’s rather than an unmeasurable “we stopped bad things happening”.
A lot earlier in my career I worked with a couple of clients in the world of security – first for a traditional security vendor and secondly for an IT services company. It was around 2001-2 and they talked a lot about hacking, ethical hackers and, of course, anti-virus tools. It’s clear a lot has changed since my last exposure to the market! It’s not about hacking anymore; it’s about economics. And that black market is BIG business. It’s about getting access to your organisation and putting a price on selling that access.
These might be US stats but Verizon’s 2016 data breach investigations report found that 82% of breaches happen within minutes yet an advanced security breach goes unnoticed for an average of 8 months!
So what role does Hadoop have in all of this? The challenges a CISO faces include it being too expensive to keep data for long enough, not having enough data to provide context and it being too expensive to collect that data. On top of that, the amount of events their team deals with can be staggering – up to 100,000 in a given 24 hour period. When you consider each ‘event’ can take 30 minutes to diagnose and there’s normally only 3-4 people in the team, they can barely scratch the surface of even 1% of the threats. The 8 months of going undetected suddenly becomes a lot more understandable… and the impact of overcoming that stacks up hugely.
At Hortonworks, we are working with our customers to help them wrap their arms around the role of data ingestion (more so from a people and process perspective, rather than technology) and to demonstrate the maturity of analytics to help with data provenance. The chain of custody and nonrepudiation is crucial because without those assurances, the computer forensics team are lost in a puzzle of who touched or moved or accessed what and when between the bad guys, the IT guys and the security team themselves.
Considering how Apache Nifi (which our Hortonworks DataFlow is based on) can take weeks off of the data provenance process or advancements with Apache Metron, I’ve learnt a lot today about the role of Connected Data Platforms when it comes to cybersecurity.
If you can forgive the amateur camera work, I did manage to grab 5 minutes with Mike Schiebel and Laurie Maclachlan, director of our international emerging products group, in our office to give you a little more (expert!) insight than my post allows – enjoy!