cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
Hortonworks Customer
Prescient

Modern Applications for Traveler Safety, Running on Hortonworks Connected Data Platforms

Prescient is a global risk management company providing full-spectrum intelligence solutions to corporate, federal and international clients. The company offers data services to address its clients’ complex global security challenges related to enterprise-wide due diligence, investigations and (most recently) traveler risk management.

Prescient powers its services with the Connected Data Platforms developed and supported by Hortonworks. Hortonworks DataFlow (HDF)—powered by Apache™ NiFi—collects, curates and delivers data from more than 49,000 sources prioritized by the Prescient team.

Prescient’s algorithms and analysts count on that steady flow of data to identify emerging threats to traveler safety. The company then delivers easily-digestible alerts to travelers on the road and other security stakeholders.


Of course, the ability to identify threats at any present moment requires a sense of history. Where have prior crimes occurred? What locations do criminal groups usually chat about online?

Hortonworks Data Platform (HDP)—powered by Apache™ Hadoop®— stores the wellspring of historical data that Prescient uses to evaluate emergent patterns. Predictive analytics evaluate those patterns to determine when Prescient delivers safety alerts to its customers.

Both the company’s employees and customers benefit from the virtuous cycle powered by Hortonworks Connected Data Platforms:


  1. HDF delivers data for analysis,

  2. HDP provides a framework to separate meaningful signals from the chatter, and also to determine where more data would be useful,

  3. HDF captures that additional data, strengthening HDP’s predictive power.

Modern Security Applications Deliver Data-in-Motion

Prescient’s roots are in the federal sector, so its first clients were within the US federal government. In 2014, Prescient began developing and launching similar services for the private sector.

From the beginning of its expansion into the private sector, Prescient wanted to make its products relevant and usable. Company leaders knew that their target segment was comprised of busy executives on the road. Most didn’t have time to consume the long printed reports that were the industry norm. By the time a report reached a traveler and that traveler had time to read it, the data was generally stale.

Prescient knew that it needed to disseminate information as soon as it detected a threat and that it needed to constantly refresh that information. The company also needed data provenance—the ability to show the lineage of any data that led it to identify a region as “high threat.” In order to dramatically speed its ability to identify and ingest streaming data that might highlight a local security risk, and to also demonstrate the underlying data that determine those risk profiles, Prescient turned to Apache NiFi.

Using NiFi, the company aimed to sift through data moving around the Internet, identify looming risks and provide travelers with actionable intelligence about:


  • Physical threats from terrorism, crime, strikes or armed conflicts,

  • Health-related threats like unsafe drinking water or the Zika virus, and

  • Environmental threats such as earthquakes or floods.

Banner

When we have forty-nine thousand sources of information, it's more about determining what should be pushed out to individuals using our mobile app or secure dashboard. We monitor emergent threats and their proximity to people and places our customers care about.

Mike Bishop, Prescient Co-Founder and Chief Systems Architect

Connected Data Platforms for Real-Time Alerts and Predictive Analytics

Apache NiFi proved that it could handle Prescient’s data ingest challenge. It ingests all types of unstructured data from sources like RSS feeds, news sites and blogs. Prescient can also correlate that streaming data with an individual traveler’s geospatial information and flag threats related to his or her location. If ever asked, data provenance in Hortonworks DataFlow lets Prescient show its customers the reasons for a high-risk classification.

When Prescient started implementing scripts (called “processors”) to analyze data in native Apache NiFi, the company quickly started seeking ways to optimize their data management process. They faced challenges maintaining scripts that were servicing widely varied types of sources.

This was one of the reasons that Prescient turned to Onyara for support with NiFi. After Hortonworks acquired Onyara, Prescient subscribed for HDF support, giving it access to HDF’s inbuilt “processors”. These pre-built utilities (such as HTTP processors and SQL processors) helped reduce development effort and ongoing maintenance costs, and they saved the company the $500,000 it had planned to invest in additional development.

As Prescient built up its capacity to ingest growing streams of real-time threat data, this success created its own challenge. As Prescient’s Chief Systems Architect, Mike Bishop, told us, “We recognized early on that this data was going to grow in a way that demanded some deep, post-collection analytics. We needed to have a framework with a very rich toolset, that would allow us to develop those analytic workflows over time.”

Prescient adopted HDP to meet that growing need for highly scalable data storage and processing. With HDP, Prescient has a five-petabyte data lake that interconnects with EMC technologies like Isilon, XtremIO and DataDomain, as well as SAP Hana and MongoDB.

Hortonworks’ dogged adherence to a 100% open-source strategy created a collaborative data center ecosystem that makes it relatively easy for Prescient to leverage value from HDF and HDP with datacenter technologies provided by Hortonworks partners.

Banner

HDF is very robust and easy to use. No other solution in the market provides the same capabilities as HDF.

Jordon Kopp, Prescient Software Engineer

The Results: A 700% Improvement in Productivity of Prescient’s Analysts

Prescient Traveler has demonstrated the ability to process 19,000 location updates per second in order to send location-based alerts to millions of subscribers.

Aside from the $500,000 in cost savings from adopting HDF, the real value came from higher-quality automated threat assessments. HDF presents threat analysts with more refined data, and it drastically reduces the amount of work required to assess specific geographic areas, events and threat domains.

Per-analyst productivity has improved seven-fold. Prior to Prescient’s adoption of HDF, a geospatial threat analyst could complete one assessment every 3-4 days. With HDF, that same analyst is now able to complete an average of two detailed assessments per day. This efficiency improves Prescient’s business margins.


Prescient’s most recent offerings have attracted new customers from the public sector, luxury travel companies, academic institutions and pharmaceutical companies (with more than 320,000 employees in 120 countries).

Prescient’s Plans for the Future of Traveler Risk Management

As Prescient’s service offerings evolve, the company expects to push the boundaries of innovation and expand its use of untapped components already running within its Hortonworks Connected Data Platforms.

Prescient believes that its services will transform the travel industry. The company foresees a day when organizations that book hotel rooms for its employees will not only search hotel prices and service reviews, but also consult Prescient Traveler to find hotels in safer neighborhoods with better public infrastructure.

Prescient still faces significant manual effort to filter out real threats from false positives. The company plans to leverage machine learning in Apache Spark™ at scale to create self-teaching algorithms that will sift through many false positives and further reduce the manual effort needed to identify actual threats.

Prescient also has plans to use Apache Ambari to monitor operations across its entire technology stack. With a comprehensive infrastructure dashboard that would provide real-time processing metrics, the company’s IT team will be able to optimize the data processing rate-of-data of each cluster. This will help to balance workloads, improve system performance and further enhance Prescient’s industry leading predictive analytics for traveler safety.

ABOUT PRESCIENT

As a premier global risk management firm, Prescient combines leading-edge technological solutions and analysis to humanize intelligence and help clients mitigate enterprise-wide risks. Prescient’s commercial solutions consist of a comprehensive suite of due diligence, traveler safety and risk management services that cultivate informed decision-making and situational awareness. The company’s federal solutions support U.S. agencies with a variety of National Security, compliance, and research and development programs. Leveraging the expertise of its principal staff - former intelligence, federal law enforcement & military, and business professionals - Prescient’s capabilities are suited for corporate, federal and international clients.