Earlier this month, Hortonworks had the pleasure of joining Yahoo! in hosting the 8th Annual Hadoop Summit, the leading conference for the Apache Hadoop community. Summit is always an important and exciting experience, bringing together thought leaders, technologists, and data specialists from throughout the community to explore and advance the art and science of Big Data.
This year’s event came at a pivotal time for Hadoop and Hortonworks, with news about Open Enterprise Hadoop and the launch of the newest version of Hortonworks Data Platform (HDP 2.3™) poised to transform the way large organizations in every industry process data.
I’d like to congratulate all the participants and attendees who joined us in San Jose, and highlight a few of the things that made this year’s Summit so special.
First of all, we’re thrilled with the enthusiastic crowds who converged at Summit. The event drew more than 4,000 people—a 30 percent increase over last year—and I can’t count the number of people who sought me out to tell me that it was the best industry show they’d ever attended. (That same energy from the open Hadoop community was on proud display in the 40 intrepid individuals who rose before dawn for the latest in our series of Summit Bike Rides).
This year at Summit we heard the clear voice of Hadoop practitioners who are using the platform to transform their businesses. Nearly half of our 165 sessions were led by end-users, vivid proof that Hadoop is both pervasive and enterprise-ready. On Thursday morning, leaders from our customers Home Depot, Rogers, Symantec, Schlumberger and Verizon shared their transformational stories on the customer panel.
We also heard from many of our partners in the ecosystem—the Big Data innovators doing important work with Open Enterprise Hadoop in every part of the industry. In addition, more than forty analysts came to hear firsthand about our launch of HDP 2.3 and why it is the most transformational Hadoop distribution yet.
The excitement is easy to understand. With the launch of HDP 2.3, we’re accelerating the traction of Hadoop in the enterprise by giving businesses what they need to drive transformational outcomes—specifically, these improvements in ease of use, enterprise-readiness, and proactive support:
You can learn more about the latest innovations in HDP 2.3 in our blog post here.
Our day-one keynote focused on the enterprise-readiness of Hadoop today. Hortonworks CEO Rob Bearden explored the state of the market and the ways businesses are using Open Enterprise Hadoop to cope with the incredible volume and diversity of data flooding into their organizations. Not just cope—thrive, as guest speakers from Microsoft and Forrester Research talked about the new possibilities Open Enterprise Hadoop is bringing into reality. Managing and processing unstructured data more effectively is important and valuable, but when you start talking about integrating Hadoop within your applications, then you’re talking about true disruption.
On day two, we invited key Hortonworks partners to the stage, including Microsoft, EMC, SAP, Teradata and HP Software, to talk about complementary technologies that make Hadoop adoption-ready at enterprise scale. The power of Hadoop lies in its open source origins and development. When enterprise customers combine that with infrastructure and systems from the world’s top technology providers, they get the best of both worlds in complete solutions to drive the digitalization of their businesses.
In one of my favorite moments from Summit, author David Epstein used the “Moneyball” era of sports to illustrate how a rising tide of data doesn’t necessarily translate into success unless you’ve got the tools and expertise to discover the essential truths hidden within Big Data.
Day three focused on Hortonworks customers—perhaps the most concise proof of the enterprise-readiness of HDP. Within the Fortune 100 alone, we serve half the banks, more than two-thirds of the retail companies, and three-quarters of telecom companies. Move out to the Global 1000 and you’ll find many, many more organizations using our technology.
Of course, to thrive in the mainstream, a technology must meet core enterprise requirements for rigor that any mature organization expects, in order to weave Hadoop into their established practices. It was inspiring to hear how we’ve been doing from customers including United HealthCare Group, GE, Webtrends, TrueCar, Verizon, Schlumberger, Home Depot, Rogers Communications, and Symantec. Here is some of what they shared on stage (which you can watch here, along with the rest of the keynotes):
We see it as 50 billion machines getting connected in roughly the next 5-10 years. As that happens, we’re going to see the same level of transformation and business model innovation as we saw when a billion people got connected through the consumer Internet…With that comes the need for industrial data management at scale…To do it well, we believe you need an open standard approach and we are excited about the role that Hadoop plays in that as part of the ecosystem.
–Vince Campisi, CIO
We went from being able to spit out all this vehicle intelligence once a day on the core vehicles that we cared about to being able to do that every thirty minutes across the entire set of vehicles that we’re bringing in…Our goal is to acquire everything—literally every piece of data that we can in the automotive industry and synthesize it within 15 minutes…
…Is Hadoop going to work in the enterprise?…We talk about data governance and master data management and security. Of course. It could do it two, three, four years ago, it just was harder than it is going to be in something like HDP 2.3.
–Russell Foltz-Smith, General Manager and SVP for Data Products
With a wireless company as big as we are—over a 100 million customers—any small incremental change in churn percentage means significant changes in revenue…We were able to increase the scoring of our churn modeling by a factor of 20% in predictability…
In partnership with Hortonworks…we are now ingesting over 250 billion records a day…we have about 1,000 nodes in play today, driving value.
–Rob Smith, Executive Director
Symantec processes security logs globally for just about everybody…The security system itself used to get backed up and there would be cases where, on average, the time to detection was [3-4 hours]…On the Hadoop ecosystem, we’re able to reduce that latency down to seconds.
…Going back, I would say, kill the fear. Just kill the fear. When it comes to figuring it out: smart people, cool tech, figure it out. Haters to the left, kill the fear, just go for it. Get it started and go.
–David Lin, Cloud Platform Engineering – Leader and Evangelist
Immediately after the customer panel Geoffrey Moore—yes, that Geoffrey Moore—offered his own take on the kinds of buying decisions customers are making today and how they compare with the roadmap he laid out at Summit just three years ago. Geoff declared that Hadoop has crossed the chasm and he discussed important use cases for mainstream adoption.
If you’re seeing a recurring theme here, it’s that Hadoop has never been more ready for business. Hortonworks is proud of our leadership role defining the emerging category of Open Enterprise Hadoop. With the amount of data in the world set to increase twentyfold over the next five years—most of it unstructured—enterprises must find new ways to manage and analyze it at scale. We never stop working to deliver the capabilities businesses need to drive transformational outcomes, and it was thrilling to see it all live and on stage at Summit. If you made it to San Jose, I hope you enjoyed it as much as we did. If you didn’t—or if you’d just like to relive the magic—you can find our keynote videos here.
Is it too soon to start looking forward to Hadoop Summit 2016?