How To Perform Spatial Analytics with Hive and Hadoop

One of the big opportunities that Hadoop provides is the processing power to unlock value in big datasets of varying types from the ‘old’ such as web clickstream and server logs, to the new such as sensor data and geolocation data.

The explosion of smart phones in the consumer space (and smart devices of all kinds more generally) has continued to accelerate the next generation of apps such as Foursquare and Uber which depend on the processing of and insight from huge volumes of incoming data.

In the slides below we look at a sample, anonymized data set from Uber that is available on Infochimps. We step through basics of analyzing the data in Hive and  learn how a new using spatial analysis decide whether a new product offering is viable or not.

Specifically, we look at:

  • The Uber dataset itself, which contains more than 1.1 million GPS readings covering 25,000 Uber trips.
  • New SQL windowing features in Hive 11 that make slicing and dicing datasets simple.
  • The Spatial Framework for Hadoop from ESRI, and how it makes analyzing geospatial data including GPS signals simple.
  • Apply spatial analytics to understand basic facts about the Uber data, including average trip length.
  • Use more sophisticated spatial analytics to determine the viability of a possible new product.

You can test out the whole tutorial using your friendly neighborhood Hadoop-in-a-box: Hortonworks Sandbox.

 Learn more about Hive here, and other types of data fueling the app ecosystem here.

Categorized by :
Business Analytics Hive

Leave a Reply

Your email address will not be published. Required fields are marked *

If you have specific technical questions, please post them in the Forums

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Try it with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.

Get Sandbox

Join the Webinar!

Discover HDP 2.2: Even Faster SQL Queries with Apache Hive and
Thursday, October 30, 2014
1:00 PM Eastern / 12:00 PM Central / 11:00 AM Mountain / 10:00 AM Pacific

More Webinars »

HDP 2.1 Webinar Series
Join us for a series of talks on some of the new enterprise functionality available in HDP 2.1 including data governance, security, operations and data access :
Contact Us
Hortonworks provides enterprise-grade support, services and training. Discuss how to leverage Hadoop in your business with our sales team.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.