Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
June 12, 2017
prev slideNext slide

HDF Series Part 3: Announcing the general availability of Hortonworks DataFlow 3.0

Developing Streaming Analytics Applications without writing a single line of code for Hortonworks DataFlow

We are thrilled to announce the general availability of Hortonworks DataFlow version 3.0.  In this release, Hortonworks has introduced two innovative, open source product modules – Streaming Analytics Manager and Schema Registry.  

In our previous blog series, we described the ease of building a fully functional streaming analytics application and the need for an integrated schema registry.  Please read the details in our blog series Part 1 and Part 2 .

Both modules enable our customers to design, develop, test, deploy and maintain streaming analytics applications with minimal special skills and training needed.

To realize the full potential of modern data applications, organizations need to capture both rich, historical insights from data at rest and perishable insights from data in motion.  Currently, flow management tools are available to help gather, route, filter and transform data from any source.  But companies have lacked equivalent tools for building the analytics apps needed to extract insight from streaming data. Hortonworks has addressed this need with the release of Streaming Analytics Manager and Schema Registry.

Below is an illustration of the entire and updated Hortonworks DataFlow platform:

HDF Data In Motion Platform
Hortonworks DataFlow 3.0 – Data In Motion Platform

Here are more details on Streaming Analytics Manager and Schema Registry.

Streaming Analytics Manager:

Using Streaming Analytics Manager, users can write complex streaming analytics apps without writing a single line of code. Eliminating the need for specialized skill sets, Streaming Analytics Manager provides a graphical programming paradigm with a drag-and-drop interface to build streaming apps for event correlation, context enrichment, and complex pattern matching.  In addition, analytical aggregations with automated alerts become available when insights are discovered.

Using this solution, Hortonworks customers will get a similarly rich experience for building streaming analytics applications that they already enjoy building flow management applications.  Furthermore,  Streaming Analytics Manager brings these apps to market considerably faster, at a lower cost, to accelerate time to value and strategic impact.  

Last, Streaming Analytics Manager provides powerful tools to meet the needs of three big data personas: developers, business analysts and IT operations teams.

Hortonworks Streaming Analytics Manager
Hortonworks Streaming Analytics Manager

Below are some great screenshots to showcase how Hortonworks Schema Registry works:

  1. Streaming Analytics Application Dashboard to showcase all the applications 

    Streaming analytics app dashboard
    Streaming analytics app dashboard
  2. Build a sophisticated streaming analytics application without writing a single line of code

    A working streaming analytics application
    A working streaming analytics application
  3. Real-time analytics dashboard for the streaming analytics application

    HDF Dashboard
    HDF Dashboard

Schema Registry:

Schema Registry improves end-to-end data governance and operational efficiency by providing a centralized registry, supporting version management and enabling schema validation.

There are three major benefits for this new module:

  • Centralized registry – A shared repository of schemas eliminates the need to attach a schema to every piece of data. Applications can flexibly interact with each other in order to save or retrieve schemas for the data they need to access. Fully integrated with the flow management component of HDF, including Apache NiFi, Schema Registry allows schemas created using Apache NiFi to be easily managed and reused by the entire platform.
  • Version management – Defines relationships between schemas and enables schemas to be shared between HDF components and applications.  Schema Registry supports schema evolution so that a consumer and producer can understand different schema versions but still read all the information shared between them.
  • Schema validation – Schema Registry supports schema validation by enabling generic format conversion and generic routing to ensure data quality.

We are very excited about the new innovations made available to the public with the announcement of HDF 3.0.

For more information:


Felipe says:

Are HDF 3.0 and SAS Event stream processing complementary products? Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *