The Hortonworks Blog

In the era of consumer-centric “agile” supply chain strategies, manufacturers are forced to act more like retailers in terms of how they capture, analyze and use consumer data. This gives visibility to internal and external supply chain partners on how products are made, sold and used.

But that visibility demands more data from more points across the supply chain. In our information-intensive world, the old ways of ingesting, processing and storing data aren’t going to be sufficient.…

One of the fastest ways for customers to realize the benefit of Hadoop technology is by using it to solve relevant business challenges especially as they pertain to improving outcomes in their industry. Customers can accelerate their journey and obtain greater ROI by leveraging solutions and/or services that address challenges unique to their business.

For example, in the Financial Services industry, mitigating risk is critical. The Healthcare industry is searching for solutions to access new data for cutting-edge medical research.…

The modern enterprise requires a comprehensive end-to-end data management solution capable of leveraging advanced machine learning to identify and manage risk; as well as a repository capable of capturing and processing the data necessary to support this solution.

Now more than ever, organizations are subject to privacy and data security laws and complying with these regulations is exceptionally challenging given the complexity of data that enterprises now have to manage. However, one only needs to pick up a newspaper to read about the dire consequences when companies fail to take proper safeguards to comply with privacy and data security laws.…

Our guest blogger today is Nyla Beth Gawel, Manager of Booz Allen Hamilton’s Internet of Things Practice. Booz Allen Hamilton, one of our strategic System Integrators, describes how they help customers with their IoT analytics using Hortonworks DataFlow (HDF), powered by Apache NiFi.

Consumer Internet of Things (IoT) has taken off in forms ranging from wearable technology to smart home devices to remote patient care. On the other hand, Enterprise IoT is still struggling to realize the promise of the connected workplace.…

Our guest blogger is Bob Taylor, Alliances Director at Concurrent, a Hortonworks Technology Partner. In this blog, Bob describes three factors that helped in the success of HomeAway in their big data initiative and are applicable to all projects. HomeAway is a customer of Hortonworks and Concurrent.

HomeAway is a great example of an organization that has found value from their Big Data investment because of three factors. One HomeAway initiative gathers customer preference data from dozens of websites and uses it to refine their marketing and, in turn, increase bookings.…

Earning the prestigious Teradata EPIC award is no easy feat. Partners who would like to have a shot at winning the top recognition need to demonstrate how their solution provides a unified, high-performance big data analytics system for an enterprise and show measurable return on investment. After receiving Teradata’s EPIC award recognition for Big Data Intelligence in 2013 and 2014, Hortonworks, yet again, has been recognized as the leader by winning this award for the third year in a row.…

Apache Spark’s momentum continues to grow and throughout 2015 we saw customers across all industries get real value from using it with the Hortonworks Data Platform (HDP). Examples include:

Insurance Optimize their claims reimbursements process by using Spark’s machine learning capabilities to process and analyze all claims. Healthcare Build a Patient Care System using Spark Core, Streaming and SQL. Retail Use Spark to analyze point-of-sale data and coupon usage. Internet Use Spark’s ML capability to identify fake profiles and enhance products matches that they show their customers.…

When you count on your Hadoop environment to power business-critical applications, you can’t afford to let problems get in the way of performance. By getting ahead of issues before they lead to cluster degradation or downtime, you can deliver the Big Data insights your business relies on with the speed and reliability competitive markets demand. That’s the thinking that led Hortonworks to fundamentally change our support model with the introduction of Hortonworks SmartSense®, a collection of tools and services that’s quickly becoming part of the standard operating procedure of many of our customers—with impressive results.…

We are very excited to announce that Grant Bodley will speak about Big Data, the Internet of Anything (IoAT) and the Connected Car at this year’s West Coast Automotive Data Event on Wednesday October 27th in San Diego.

Telematics West Coast is the premier event that explores and develops new ideas and business practices opening up with Big Data. Hortonworks is thrilled to be a Gold Sponsor at the event.

Join Us at Telematics West Coast

Attend Grant’s session at 10am, The Information Superhighway for Automotive Transformation to learn more about Big Data, the Internet of Anything (IoAT) and how the Connected Car has created a new Information Superhighway that fundamentally changes the relationship between automakers and car buyers.…

Hackathons, Hackfest, and Codefests have an initial air of invincibility. They challenge participants, even veterans—not if the attendees work together or if the community collaborates and innovates together. That air of invincibility quickly dissipates.

Last Saturday, because of such camaraderie and collaboration, a harmony of innovative ideas flourished and came to fruition at an Ambari Hackfest.

Open Data Platform Initiative (ODPi) founding partners Hortonworks and Pivotal co-hosted and co-sponsored an Ambari Hackfest at the Pivotal site near the scenic Foothills in Palo Alto.…

Geospatial data is pervasive—in mobile devices, sensors, logs, and wearables. This data’s spatial context is an important variable in many predictive analytics applications.

To benefit from spatial context in a predictive analytics application, we need to be able to parse geospatial datasets at scale, join them with target datasets that contain point in space information, and answer geometrical queries efficiently.

Unfortunately, if you are working with geospatial data and big data sets that need spatial context, there are limited open source tools that make it easy for you to parse and efficiently query spatial datasets at scale.…

Is a Lake Big Enough to House Your Ocean of Data?

Contrary to popular belief, Hadoop was not the elephant-in-the-china-shop that marauded and disrupted the data center. The real culprit is data and how it has exploded in volume. The past two or three years have seen a rise in the number of successful Hadoop projects in enterprises to tackle this explosion of big data. These large volumes of data, the emergence of the Hadoop technology and the need to store all the siloed data in one place have prompted the phenomenon called the Data Lake among enterprises.…

Our guest blogger today is Rob Rosen, Senior Director Partner Solutions at Platfora, describes how to help customers achieve strategic advantage through data discovery.

While many people have heard the notion of “known unknowns” and “unknown unknowns,” it may surprise you to discover that the concept was first popularized by a NASA scientist. In a presentation given at TEDx GeorgeMasonU, Dr. Kirk Borne described how he used the concept of “known unknowns” (things that we knew might exist, but hadn’t seen evidence of) and “unknown unknowns” (things that we could discover and knew nothing about, but would truly surprise us), and how they relate to the concept of Big Data.…

The advent of connected manufacturing has ushered in an era where low-cost machine sensors take thousands of measurements per second at many points across the manufacturing process. This stream of sensor data enables manufacturers to quickly detect emerging anomalies and solve issues before they impact yield and quality.

Big Data insights enable predictive analytics for those rapid, proactive process adjustments. Manufacturers can capitalize on this opportunity by following an approach that combines the power of Teradata with Hortonworks Data Platform’s storage and compute efficiencies at extreme scale.…

I recently had the pleasure of visiting with Arvind Battula, Sr. Data Scientist at Schlumberger. We discussed his background as a chemical and mechanical engineer and his move onto the Data and Analytics team as a data scientist. The following is a transcript of my conversation with Arvind. We discussed his background, his interesting focus areas for data science in oil and gas, and technologies that he believes will help transform the industry.…