Get Started


Ready to Get Started?

Download sandbox

How can we help you?

closeClose button

Maximize the Value of Data-in-Motion
With Big Data from the Internet of Things

play video button video

cloud Ready to Get Started?

Download HDF
Hortonworks DataFlow (HDF™)

Hortonworks DataFlow (HDF)

HDF makes streaming analytics faster and easier, by enabling accelerated data collection, curation, analysis and delivery in real-time, on-premises or in the cloud through an integrated solution with Apache NiFi, Kafka and Storm.

Powering the Future of Data


Integrated data-source agnostic collection platform

Integrated collection from dynamic, disparate and distributed sources of differing formats, schemas, protocols, speeds and sizes such as machines, geo location devices, click streams, files, social feeds, log files and videos.

More Info:

Fast, Easy, Secure

Real-Time Response to Streaming Data

Real-time evaluation of perishable insights at the edge to determine what is pertinent or not, and executing upon consequent decisions to send, drop or locally store data as needed through a visual user interface with real time operational visibility and control. Unprecedented operational effectiveness is achieved by eliminating the dependence and delays inherent in a coding and custom scripting approach.

More Info:

  • How streaming data managed through a real-time visual user interface of Apache NiFi increases operational effectiveness.
    Watch Video
Real-time decisions

Secure from source to destination, with real-time data chain of custody

Secure end-to-end routing from source to destination, with discrete user authorization and detailed, real-time visual chain of custody and metadata (data provenance). The ability to equally support security and encryption on small scale, JVM-capable data sources from the edge of the Internet of Things, to large scale enterprise clusters in support of the Internet of Anything ensures high trust of analytical outcomes and its underlying data.

More Info:

  • See how HDF enables high trust of your data
    Watch Video
Trust your data

Real-time business depends upon capturing perishable insights from data in motion.

HDF supports stream processing to aggregate and analyze event data in order to dynamically recognize data patterns and detect outliers. The real-time, high volume event processing for immediate action and response to streaming is supported through an integrated enterprise offering of Apache NiFi, MiNiFi, Kafka and Storm to initiate data.

Apache NiFi and MiNiFi provide dynamic, configurable data pipelines, through which all sources, systems and destinations communicate. Kafka adapts to differing rates of data creation and delivery while real-time streaming analytics with Storm creates immediate insights at a massive scale.

More Info:

  • Processing Streaming Data
    Learn More
    Realtime Event Processing in Hadoop with NiFi, Kafka and Storm
    Learn More
Stream Processing

Create New Insights By Securely Sharing Select Pieces of Data

HDF provides secure access and control of data to enable informed business decisions. Enabling sharing of specific pieces of information, and removing role-based data access that grants blanket access to an entire silo of data at once, HDF allows enterprises to dynamically and securely share select pieces of pertinent data and gain new business insights.

More Info:

  • See how HDF reduces risk while enabling data democratization
    Watch Video
Unlock the power of your data

Optimize log analytics with content based routing at the edge

Real-time edge analytics eases integration with log analytics systems such as Splunk, SumoLogic, Graylog, LogStash, etc. for easy, secure and comprehensive data ingest of log files. By cost-effectively increasing volumes of data collected with content based routing, enterprises can accelerate trouble-shooting and improve anomaly detection with a more comprehensive view across all available machine data.

More Info:

  • Optimize log Analytics from the Edge
    Read More
Log analytics optimization

A scalable, extensible platform for the IoAT, inclusive of IoT

Equally well designed to run on the small scale data sources that make up the Internet of Things as well as on large scale clusters in today's enterprise data centers, HDF is designed to support the Internet of AnyThing. HDF securely moves data from wherever it is, to wherever it needs to go, regardless of size, shape, or speed dynamically adapting to the needs of the source, the connection, and the destination.

More Info:

Designed for the IoT

Encryption and routing on demand

Use the visual user interface of HDF to drag and drop dataflows to encrypt streaming data, route it to Kafka, configure buffers and congestion management so that data can be dynamically prioritized and securely sent from source to destination, with immediate responsiveness to fluctuating conditions that commonly occur at the edge.

More Info:

  • See how live dataflow is encrypted and routed to Kafka in minutes
    Watch Video
Encryption and prioritization

What’s New in HDF

parallax slide
  • New UI: More intuitive interface for easy and time-saving dataflow creation, management, tuning and control of real-time data flows
  • Increased productivity: Multiple actors can manage different parts of a single dataflow shared across an entire enterprise, connecting data silos and teams with fine grained component level control
  • Enterprise Readiness: Ease of deployment and management of NiFi, Kafka and Storm through integrated Ambari interface and integration with Ranger for authorization management
  • MiNiFi: New MiNiFi architecture option in JVM for expanded scalability that addresses the challenges of secure, first-mile and bi-directional communication with millions of edge devices (IoT) in less than 40 MB footprint
  • NiFi     Kafka     Storm

Why was HDF invented?

The real-world challenges of securely and efficiently moving distributed data from everywhere
HDF User Guides

Get HDF release notes; guides for users, developers and getting started.


The industry’s best support for Apache NiFi, Kafka and Storm in the enterprise. Connect to our team experts to help guide your journey.


Real-world training from the Big Data experts. Available in person or on-demand whenever you need us.