Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
March 07, 2018
prev slideNext slide

Accelerating Data Science and Real-Time Analytics at Scale

Guest blog written by Steve Roberts, Power Big Data Offering Manager, IBM

Until now, organizations looking to get the most business value possible from big data have focused primarily on deploying cost-efficient storage and performing deep analytics across diverse data sources. However, the rules of the game are changing: the growing volume of Internet of Things (IoT) devices is creating a new class of streaming data, while artificial intelligence (AI) is making it possible to capitalize on that data like never before.

To keep up in this changing world, businesses must be able to catch insights not only inside the data lake but also at the edge of the network and take action immediately. They can accomplish this by accelerating development of machine learning models with data science, leveraging Hadoop storage and accelerated computing, and then applying those models in real-time within streaming data flows to drive immediate actions based on model predictions. Data science and real-time edge analytics can support use cases across a variety of industries, including:

  • Logistics: Monitor trucking fleets in real time to mitigate driving infractions (Related Hortonworks Blog)
  • Retail: Analyze and visualize social media data about particular products to support real-time promotions
  • Energy and utilities: Monitor transmission lines with drones and smart meters to predict and prevent failures
  • Finance: Protect credit card customers from fraud
  • Insurance and healthcare: Provide personalized policies and treatments

To achieve the best business outcomes, IBM Power Systems is proven to be the ideal server platform to support the latest data science and analytics solutions, including:

  • IBM Data Science Experience (DSX) that makes data scientists more productive and effective. It gives data scientists the ability to select the tools and capabilities that best meet their needs, including the most popular open source tools. DSX also provides a social environment, allowing data scientists to collaborate with one another to solve data challenges, while sharing their expertise. Organizations that run DSX on IBM Power Systems can complete model training in half the time required by comparable x86 systems providing the advantage of accelerated business insights.
  • IBM PowerAI that makes deep learning, machine learning and AI more accessible. The solution is a software distribution built specifically to help organizations use deep learning, machine learning, and AI to their full potential. Exclusively available on Power Systems, the Best Server for Enterprise AI, it’s even easier for organizations to start unlocking the value of ML/DL applications. PowerAI is integrated as part of IBM DSX on Power Systems.
  • Hortonworks DataFlow (HDF) that enables real-time analytics of data in motion. HDF is the only end-to-end streaming data platform on the market today. Built with 100 percent open source components, HDF empowers organizations to collect, curate, analyze, secure, govern and act upon data at the edge of the network in real time.Once streaming data is processed and acted upon to capture any perishable insights, a version of the data, often filtered or aggregated, must be stored in a secure, reliable enterprise data lake so it can extend the data set for future modeling and deep insights. Hortonworks Data Platform (HDP) on IBM Power Systems is the perfect combination of openness, reliability and performance for this data at rest.

Organizations that can harness the insights from streaming data to better serve and protect their clients and business goals will ensure they can continue to compete and prosper. Building a data science practice to take advantages of AI techniques such as deep learning and applying dataflow management and analytics at the edge of the network are critical steps in this journey.

To learn more, please visit:

Leave a Reply

Your email address will not be published. Required fields are marked *