We’re finally catching our breath after a phenomenal Hadoop Summit event last week in San Jose. Thank you to everyone that came to participate in the celebration of Hadoop advances and adoption—from many of the organizations that shared their Hadoop journey with us that fundamentally transformed their businesses, to those just getting started, to the huge ecosystem of vendors. It is amazing to be part of such a broad and deep community that is contributing to making the market for everyone.…
The Hortonworks Blog
Enterprises are using Apache Hadoop powered by YARN as a Data Operating System to run multiple workloads and use cases instead of using it just as a single purpose cluster.
A multi-purpose enterprise wide data platform often referred to as a data lake gives rise to the need for a comprehensive approach to security across the Hadoop platform and the workloads. Few weeks back Hortonworks acquired XA Secure to further execute on our vision to bring a holistic security framework to the Hadoop community irrespective of the workload.…
Apache YARN, Apache Slider, and Docker
Join us June 19 at 6 pm at the Hilton Fort Worth, Texas for an educational workshop hosted by Hortonworks and Sendero Business Services. The topic is “The Key To Success is Consistently Making Good Decisions & The Key To Good Decisions is Good Information.” The speaker is Don Hilborn, Solutions Engineer at Hortonworks.
Don will introduce the paradigm of
- Efficiency – double processing in Hadoop on the same hardware while providing predictable performance and quality of service; and
- Resource sharing – providing a stable common set of shared resources across multiple, coordinated workloads in Hadoop.
This is the second in the series of blogs exploring how to write data-driven applications in Java using the Cascading SDK. The series are:
Historically, programming languages and software frameworks have evolved in a singular direction, with a singular purpose: to achieve simplicity, hide complexity, improve developer productivity, and make coding easier. And in the process, foster innovation to the degree we have seen today—and benefited from.
Anyone among you is “young” enough to admit writing code in microcode and assembly language?…
With the release of Apache Hadoop YARN in October of last year, organizations are moving from single-application Hadoop clusters to a versatile, integrated Hadoop 2 data platform hosting multiple applications — eliminating silos, maximizing resources and bringing true multi-workload capabilities to Hadoop. Many enterprises have adopted YARN as the architectural center of a set of integrated technologies and capabilities that form the blueprint for enterprise Hadoop.
YARN Enabling the Ecosystem Technologies
Hortonworks is making it easier to develop YARN applications through a number of technologies. …
More and more solution providers are integrating with Hortonworks Data Platform to provide their customers with enterprise Hadoop.
As part of our HDP 2.1 certification series, I would like to introduce Greg Benson, Chief Scientist at SnapLogic. In this blog, Greg provides some insights about the value of obtaining HDP 2.1 certification and the benefits of integration platform as a service (iPaaS).
SnapLogic provides a cloud-based service for performing a wide range of data and application integration tasks.…
Apache Ambari has always provided an operator the ability to provision an Apache Hadoop cluster using an intuitive Cluster Install Wizard web interface, guiding the user through a series of steps:
- confirming the list of hosts
- assigning master, slave, and client components to configuring services, and
- installing, starting and testing the cluster.
With Ambari Blueprints, system administrators and dev-ops engineers can expedite the process of provisioning a cluster. Once defined, Blueprints can be re-used, which facilitates easy configuration and automation for each successive cluster creation.…
Since the partnership between Hortonworks and Splunk and the release of Hunk last year, we have created some awesome assets (i.e., Hunk sandbox tutorial, 360-degree customer view webinar) that have enabled Hadoop and Big Data enthusiasts’ hands-on training with Big Data. You can find more details around our partnership and resources here: http://hortonworks.com/partner/splunk/
As part of our HDP 2.1 certification series, I would like to introduce Brett Sheppard, Director of Product Marketing for Big Data at Splunk.…
We recently hosted the fourth of our seven Discover HDP 2.1 webinars, entitled Apache 2.4.0, HDFS and YARN. It was very well attended and a very informative discourse. The speakers outlined the new features in YARN and HDFS in HDP 2.1 including:
- HDFS Extended ACLs
- HTTPs support for WebHDFS and for the Hadoop web UIs
- HDFS Coordinated DataNode Caching
- YARN Resource Manager High Availability
- Application Monitoring through the YARN Timeline Server
- Capacity Scheduler Preemption
Many thanks to our presenters, Rohit Bakhshi (Hortonworks’ senior product manager), Vinod Kumar Vavilapalli (co-author of the YARN Book, PMC, Hadoop YARN Project Lead at Apache and Hortonworks), and Justin Sears (Hortonworks’ Product Marketing Manager).…
Informatica is a Hortonworks Certified Technology Partner. This partnership makes it possible for organizations to use all the data internal and external to an enterprise to achieve the full predictive power that drives the success of modern data-driven businesses.
That is why we’re excited to have John Haddad, Senior Director, Informatica to be our guest blogger. In this blog, John explores the benefits of certification on HDP 2.1.
When I was in high school, one of my best friends had a water ski boat we often took out on California lakes (what are friends for?).…
Julian Hyde will present the following talks at the Hadoop Summit:
What to do with all that memory in a Hadoop cluster? The question is frequently heard. Should we load all of our data into memory to process it? Unfortunately the answer isn’t quite that simple.
The goal should be to put memory into its right place in the storage hierarchy, alongside disk and solid-state drives (SSD).…
We are less than a week away from start of the seventh annual Hadoop Summit San Jose. With all of the final preparations underway, we wish to highlight some of the not to be missed activities in and around the event. The event is filling fast, but you can still register here.
Here are a few things you don’t want to miss!
Many thanks to all of the contributors in the Apache Ambari community for the collaboration to deliver 1.6.0, especially with Blueprints, a crucial feature that enables rapid instantiation and replication of clusters.
Each release of Ambari makes substantial strides in providing functionality to simplify the lives of system administrators and dev-ops engineers to deploy, manage, and monitor large Hadoop clusters, including those running on Hortonworks Data Platform 2.1 (HDP).…
In this blog, Paul Phillips, EMEA Sales Director at Hortonworks, discusses the importance of extending big data science courses to PhD students and scientists. This joint venture with KPMG provides an opportunity to “bring excellent basic skills that are useful in data science and this programme aims to commercialize these skills and ease the path to a data science profession.”
At Hortonworks, we encourage our team members to innovate and as the Open Source community grows, it is also vital that we play our part to ensure the community is continually reinvigorated with new ideas and innovation. …
According to New York Observer, there were couple of major social reasons that spurred the genesis and growth of Meetup.com. First, it was Robert Putman’s book Bowling Alone, in which he talks about the collapse of communities in America. And the second was an event that not only changed the world but changed New York: it was the aftermath of September 11, where strangers cared about greeting, meeting, and talking.…