Get Started

Ready to Get Started?

Download sandbox

1.408.675.0983

Hortonworks Community

Get Answers

Score with Big Data

Get my score

Save with SmartSense

Calculate My Savings

How can we help you?

December 12, 2016

Announcing the Availability of Hortonworks DataFlow (HDF) 2.1

Wei Wang & Haimo Liu

We are pleased to announce that Hortonworks DataFlow (HDFTM) Version 2.1 is now generally available. You can download the latest version here!

HDF 2.1 (powered by Apache NiFi, Apache Kafka and Apache Storm) brings enterprise readiness, platform stability and ease of use to the next level.

Apache NiFi for dynamic, configurable data pipelines, through which all sources, systems and destinations communicate.
Apache Kafka for high throughput distributed messaging with pub sub semantics to operate at speed on big data volumes that adapt to differing rates of data creation and delivery
Apache Storm for real-time streaming analytics to create immediate insights at massive scale

Here are the 3 top features we would like to highlight within HDF 2.1

1.Extend HDF support in the cloud

In the release of HDF 2.1, users can now interact with AWS Kinesis, Microsoft Azure WASB (Windows Azure Storage Blob) and ADLS (Azure Data Lake Storage). This significantly extends user’s flow management capability in the cloud.

2. Ease of Use Enhancement

In the release of HDF 2.1, users now can further simplify its policy management task with a user centric policy view, further enhancing across enterprise collaborations in a multi-tenancy manner. Please see the following graph as an example:

screen-shot-2016-12-11-at-12-47-41-pm

3. New policy to restrict processor (potentially dangerous) access

screen-shot-2016-12-11-at-12-48-42-pm

In the release of HDF 2.1, data flow administrators within the enterprise can identify that in order for certain potential processors to be added to a working data flow system, additional authorization would be required.

In addition, HDF 2.1 supports over 180 processors including newly introduced Connect/Listen/PutWebSocket, Put/FetchElasticsearch5, ValidateCsv, etc.

We would like to thank Apache NiFi, Kafka and Storm community to help us make this a very successful release.

To learn more:

Leave a Reply Cancel reply

BLOG

11.17.17

Building a global data lake for...

Financial institutions need to leverage all the information they can gather to guide future investments, reduce risk and detect fraud. These objectives directly influence an institution’s bottom line and have become more challenging with the rising volumes and varieties of Big Data. To keep up, financial institutions are continuously adapting their data architectures to support…

learn more

BLOG

9.11.17

Strata Data Conference New York –...

The annual Strata Data Conference will make its next stop in New York the week of September 25-29. The core theme of the conference this year is around driving business transformations through the power of data. And in the world of data, few topics excite as much as data science, machine learning, and deep learning.…

learn more

BLOG

9.7.17

Top 5 Use Cases of HDF...

With the explosion of the Internet of Things (IoT), businesses need to reevaluate their existing data strategies and adopt a more modern data architecture. Building a near-real time streaming application that take advantage of data-in-motion can be a daunting journey to undertake. Most streaming applications require a considerable amount of coding, testing and time to…

learn more

BLOG

9.6.17

Hospitality and the Big Data Window...

What eventually drives us to purchase something online? This isn’t a hypothetical or philosophical question, as businesses are forced to find answers every day. Consumers across every industry have access to almost anything they can find on a search engine, however with such a big supply of goods, businesses must manage the Big Data that’s…

learn more

BLOG

8.30.17

How the Leading Transportation Tech Provider...

Last week we published a story on how the leading transportation tech provider is able to solve the last mile problem for the carrier industry. This article expands on this and how companies need to have clean data. How Clean Is Your Data? By now, we all know that data is Big. We know that…

learn more

BLOG

8.29.17

Denodo Platform 6.0 Certification with HDP...

This guest blog is from our partner Denodo who is a leader in data virtualization and a Hortonworks partner for many years. Denodo helps customers who have data from multiple, heterogeneous sources to quickly, easily and cost-effectively integrate it to derive business insights and positively change their strategy to become more data-driven. In addition to…

learn more

BLOG

8.28.17

How Data Fuels the Breakneck Speed...

The world is moving faster than ever before, and companies are finding themselves scrambling to keep up. Our society has become centered on convenience and everything happening in real time. Data insights need to happen in real time as well so that companies can maximize their ability to better serve their customers. Last week I…

learn more

BLOG

8.25.17

How the Leading Transportation Tech Provider...

In many industries, the last mile problem is the most difficult problem to crack. This is especially true in the carrier industry, where 91% of the total carriers operate fleets of six vehicles or less. TMW Systems (TMW) solves the last mile problem, and an array of other transportation and business challenges, by consolidating the…

learn more

BLOG

8.24.17

GDPR Compliance Presents New Business Opportunities

The General Data Protection Regulation (GDPR) is creating new business opportunities by making it easier for companies worldwide to operate throughout the European Union. GDPR compliance will force companies to review their data-handling practices, and to analyze and understand their customer data types. Now, the conversation is changing from the burden of compliance to the…

learn more

view all events view all webcasts

Contact Sales?

Apache, Hadoop, Falcon, Atlas, Tez, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie, Phoenix, NiFi, Nifi Registry, HAWQ, Zeppelin, Slider, Mahout, MapReduce, HDFS, YARN, Metron and the Hadoop elephant and Apache project logos are either registered trademarks or trademarks of the Apache Software Foundation in the United States or other countries.

How can we help you?