Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
February 11, 2015
prev slideNext slide

Open Source Communities: The Developer Kingdom

Since our founding in 2011, Hortonworks has had a fundamental belief: the only way to deliver infrastructure platform technology is completely in open source. Moreover, we believe that collaborative open source software development under the governance model of an entity like the Apache Software Foundation (ASF) is the best way to accelerate innovation that targets enterprise end users since it brings the largest number of developers together in a way that enables innovation to happen far faster than any single vendor could achieve and in a way that is free of friction for the enterprise.

In the past, open source development was thought of as a bunch of hackers living in basements or working out of garages. Fast-forward to 2015 and, as Stephen O’Grady puts it, developers “have become the most important constituency in business seemingly overnight.” They are some of the highest paid and most valued assets in any modern organization. Developers are the new kingmakers.

No matter if they are individual developers or rockstars in a mega corporation, many of these key talents have found each other in a common and very open community. They are contributing and building out key platform technologies such as Apache Hadoop.

But why? Why contribute? Why give away such important intellectual property?
With platform technologies, the developer just wants it to work and the mega corporation just wants to pursue adjacent opportunities built on it. Every company is becoming a software company and the platform on which they build needs to be stable, reliable and complete. So, in the end, what is good for the platform is GREAT for all.

None of this can happen without the ASF

The ASF is critical in providing the right environment where meritocracy rules and platform technologies can be advanced. It provides valuable stewardship and guide-rails for projects interested in attracting the broadest community of involvement as possible.

The ASF recently published the Apache Project Maturity Model document that provides a GREAT overview of “how Apache projects operate, in a concise and high-level way”. It is an important piece of work in this new world of software development, and we encourage open source projects without a clearly defined governance model like the ASF’s to read through and use some of these items in support of your own project’s goals.

Harmonizing upstream project innovation and downstream enterprise product

The Hortonworks development model complements what the ASF provides by defining how the dozens of ASF projects included within the Hortonworks Data Platform (HDP) should work together in concert as part of an enterprise-viable data platform. Understanding and satisfying enterprise requirements that span the ASF projects and/or making tradeoffs about which projects are better suited for specific use cases are ways we enable value above and beyond what the ASF focuses on.

Our development model enables us to get the latest stable innovation into our end users’ hands quickly. It’s been designed to enable a virtuous cycle of open source innovation that flows naturally from upstream ASF projects into downstream enterprise grade HDP releases. This minimizes and/or eliminates drift from the corresponding stable ASF project releases. We also work diligently to ensure bug fixes included in HDP 1.x and 2.x maintenance releases are managed in a way that avoid regressions in future releases.

Our certification process is underpinned by a test suite unique to Hortonworks that’s comprised of 10’s of thousands of functional, system, and integration tests that are run on some of the world’s largest clusters within our joint engineering partners’ environments across a wide range of Linux and Windows systems. By doing all feature development and bug fix work in the upstream ASF projects, our downstream HDP process can cleanly harmonize a set of stable ASF project releases into a rigorously tested and certified platform suitable for the enterprise.

Focusing on enabling an equitable balance of power, not lock-in

End users have learned that open source can not only meet their requirements, but can also enable them to get out from under the yoke of any single vendor’s agenda. Moreover, active participation in open source is not just the domain of web-scale end users like Yahoo!, Google, Facebook, Netflix, and LinkedIn.

Mainstream enterprises such as Aetna, Merck, and Target are participating in open source initiatives that address such needs as data governance since they are vital to their businesses and industries. While our customers are normally downstream consumers of HDP and our subscription services, efforts like the Data Governance Initiative, are enabling customers to move upstream into the ASF process to help prioritize features and develop/test the ASF project code so the released technologies are highly aligned with their requirements.

Since we make HDP available free of charge and derive the majority of our revenue from annual support subscriptions, we realize that “every year is an election year” for our customers. We feel our open source development model yields unmatched efficiency and customer satisfaction since it provides a direct way for customers and partners to participate, drive value, and capture adjacent opportunities.

And we feel our business model establishes an equitable balance of power that de-risks investments and focuses on mutual success.

A Benevolent Benefactor

We at Hortonworks have been part of the open source Apache Hadoop movement from its very beginnings. Our team members have helped incubate many new open source technologies and are well versed in how communities work and how organizations can benefit from them. We endeavor to help guide and extend the Hadoop ecosystem as a benevolent benefactor, ensuring all those in the data center can benefit from it.

Another key principle we were founded on was to enable the ecosystem to adopt and extend Hadoop. We do so in the way we help architect solutions. We do so in the way we partner with the technology leaders. We do so in the way we get involved and represent our customer in the community. And it’s all fueled by the rock star developers we employ and the communities in which they work.

If you’d like to see the community in action for yourself, then please join us at Hadoop Summit Brussels and San Jose where you can engage with end users and the broader Hadoop community to help shape the future of Apache Hadoop!


Leave a Reply

Your email address will not be published. Required fields are marked *

If you have specific technical questions, please post them in the Forums