Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
August 16, 2016
prev slideNext slide

Demo #3: Play-by-Play: Data Hacks & Demos @ #HS16SJ

Use IoT to get real-time feedback on customer preferences and respond to them

During the 3rd demo of the Data Hacks & Demos session, at Hadoop Summit San Jose, it was audience participation time! Kay Lerch demonstrated how to interact with the audience, through specific twitter and SMS messages sent to a specific phone number, or tweet to @HadoopSummitDemo. The messages were routed through to the light boxes which would flash and light for each message received, a vote for each question and hashtag represented.

So what did Joe and Kay tell the audience?

Real-time voting scenario with Hadoop audience simulates how customer can share real-time feedback with a retailer.

This demo simulated how a customer can share their real-time feedback with the retailer about their preferences and opinions. In real life consumers may express their opinion digitally, (Ie on social media such as Twitter) rather than providing feedback in the brick and mortar store. Being able to digitally interact with them and understand their experience provides timely information that can influence their purchasing decision.

In this case, the audience was asked to vote on this demo:

  1. A Does this demo make you more excited about your future experience as a consumer – that a retailer could know you better? If so, vote “More excited”


  1. B, does this demo his make you nervous with respect to its privacy implications? If so,  vote “More Concerned”


As it turned out, the audience was about evenly split between nervous or excited about such tech. And in the end, the system processed almost 3000 votes in total, in less than 3 minutes!)

After which, the batteries for lighting up the LED lightboxes were exhausted!

Demo 3 AWS IoT Apache NiFi Live Voting Hack Demos Hortonworks HS16SJ

How did real-time voting with Apache NiFI, AWS IOT work?

Behind the scenes there was an Apache NiFi instance hosted on Amazon EC2 orchestrating a dataflow. Data was ingested from Twilio (an SMS provider), and the Twitter Streaming API. The Apache NiFi instance parsed the information and passed the vote over to AWS IoT.

AWS IoT  was used as an IoT Gateway to communicate over MQTT with the two Raspberry Pis sitting inside the Lightboxes and these Raspberry Pis controlled the 32×32 LED panels. The lightboxes interpreted incoming SMS message as red flash, and an incoming tweet as a green flash.

Also connected to the Raspberry Pi were temperature and humidity sensors. The data collected by these sensors is sent back to AWS IoT.and stored in the so-called “thing shadow”.  The shadow was then read out by NiFi for each one of your messages.

This sensor data was sent back the same way back to the mobile phone the way the vote information was delivered to the lightboxes. The Raspberry Pi sends the sensor data to AWS IoT. over MQTT, persisted in a thing shadow of AWS IoT, then processed by NiFi and returned to Twilio in response to its webhook resulting in instant real-time feedback as a response SMS.

That was the 3rd demo of Data Hacks & Demos at Hadoop Summit San Jose. The 4th demo  – using streaming analytics to create a single buyer identity is up next in this blog series. In the meantime, to get started with building something like this yourself, check out these links



Leave a Reply

Your email address will not be published. Required fields are marked *