Hortonworks DataFlow (HDF) is a scalable, real-time streaming analytics platform that ingests, curates and analyzes data for key insights and immediate actionable intelligence. DataFlow addresses the key challenges enterprises face with data-in-motion—real-time stream processing of data at high volume and high scale, data provenance and ingestion from IoT devices, edge applications and streaming sources.
Drastically reduce your data integration development time
Imagine a no-code approach to building complex data pipelines with minimal effort. HDF offers a simple visual user interface for building sophisticated data flows to accomplish major data ingestions, transformations and enrichment from a variety of streaming sources. Powered by Apache NiFi, HDF can ingest data from a range of data sources—devices, enterprise applications, partner systems or edge applications generating real-time streaming data.
Manage and secure your data from edge to enterprise
HDF enables high volume data collection at the edge, even from edge devices using Minifi. Now you can set up widely distributed IoT deployment models for regional data collection with ease using NiFi with Minifi to stream data from the edge. Tight integration with Apache Ranger gives HDF the unique advantage of seamless security across all your data-in-motion and data-at-rest.
Get real-time insights and actionable intelligence faster than ever
Real-time insights mean you can act sooner. Using the powerful streaming platform Apache Kafka, HDF can process several million transactions per second, identify key patterns, compare against machine learning models and offer predictive or prescriptive analytics to help business leadership make key decisions and seize opportunities.
Streaming Analytics Manager provides a visual way to build complex streaming applications, enabling data analysts and data scientists to understand key insights and gain actionable intelligence from real-time data.
Build a data architecture that adapts to IoT-scale
HDF is 100% open source technology – so you can design a future-proof architecture without any vendor lock-in. This solution is a proven technology hundreds of customers have chosen for its prowess in mission-critical use cases. Customers can implement IoT solutions for sectors such as automotive, manufacturing, transportation, utilities, retail and public sector. You can adopt a data strategy to handle highly diversified and large data volumes at high velocities.
Stay compliant with full governance of any streaming data
HDF is the only product in the industry offering data provenance and edge-to-enterprise data governance out of the box. In the age of GDPR and other regulatory compliance laws, it’s important to track data lineage, even for streaming data. NiFi within HDF offers data provenance tracking without any extra configuration or setup. With tight integration of Apache Atlas, you have a complete governance of data from the edge to the enterprise.
Clearsense is a smart data organization based in Jacksonville, Florida that is re-imagining and simplifying data analytics to help healthcare organizations realize measurable value from their data. They have developed a...
TechnipFMC is a global leader in oil and gas projects, technologies, systems, and services to provide their clients with deep expertise across subsea, onshore/offshore and surface projects. The company’s vision is to...
Johns Hopkins University is an American private research university, founded in 1876 and located in Baltimore, Maryland. It is considered the first research university in the United States, and is organized into 10...
Apache, Hadoop, Falcon, Atlas, Tez, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie, Phoenix, NiFi, Nifi Registry, HAWQ, Zeppelin, Slider, Mahout, MapReduce, HDFS, YARN, Metron and the Hadoop elephant and Apache project logos are either registered trademarks or trademarks of the Apache Software Foundation in the United States or other countries.