Prescient’s roots are in the federal sector, so its first clients were within the US federal government. In 2014, Prescient began developing and launching similar services for the private sector.
From the beginning of its expansion into the private sector, Prescient wanted to make its products relevant and usable. Company leaders knew that their target segment was comprised of busy executives on the road. Most didn’t have time to consume the long printed reports that were the industry norm. By the time a report reached a traveler and that traveler had time to read it, the data was generally stale.
Prescient knew that it needed to disseminate information as soon as it detected a threat and that it needed to constantly refresh that information. The company also needed data provenance—the ability to show the lineage of any data that led it to identify a region as “high threat.” In order to dramatically speed its ability to identify and ingest streaming data that might highlight a local security risk, and to also demonstrate the underlying data that determine those risk profiles, Prescient turned to Apache NiFi.
Using NiFi, the company aimed to sift through data moving around the Internet, identify looming risks and provide travelers with actionable intelligence about:
Apache NiFi proved that it could handle Prescient’s data ingest challenge. It ingests all types of unstructured data from sources like RSS feeds, news sites and blogs. Prescient can also correlate that streaming data with an individual traveler’s geospatial information and flag threats related to his or her location. If ever asked, data provenance in Hortonworks DataFlow lets Prescient show its customers the reasons for a high-risk classification.
When Prescient started implementing scripts (called “processors”) to analyze data in native Apache NiFi, the company quickly started seeking ways to optimize their data management process. They faced challenges maintaining scripts that were servicing widely varied types of sources.
This was one of the reasons that Prescient turned to Onyara for support with NiFi. After Hortonworks acquired Onyara, Prescient subscribed for HDF support, giving it access to HDF’s inbuilt “processors”. These pre-built utilities (such as HTTP processors and SQL processors) helped reduce development effort and ongoing maintenance costs, and they saved the company the $500,000 it had planned to invest in additional development.
As Prescient built up its capacity to ingest growing streams of real-time threat data, this success created its own challenge. As Prescient’s Chief Systems Architect, Mike Bishop, told us, “We recognized early on that this data was going to grow in a way that demanded some deep, post-collection analytics. We needed to have a framework with a very rich toolset, that would allow us to develop those analytic workflows over time.”
Prescient adopted HDP to meet that growing need for highly scalable data storage and processing. With HDP, Prescient has a five-petabyte data lake that interconnects with EMC technologies like Isilon, XtremIO and DataDomain, as well as SAP Hana and MongoDB.
Hortonworks’ dogged adherence to a 100% open-source strategy created a collaborative data center ecosystem that makes it relatively easy for Prescient to leverage value from HDF and HDP with datacenter technologies provided by Hortonworks partners.
As a premier global risk management firm, Prescient combines leading-edge technological solutions and analysis to humanize intelligence and help clients mitigate enterprise-wide risks. Prescient’s commercial solutions consist of a comprehensive suite of due diligence, traveler safety and risk management services that cultivate informed decision-making and situational awareness. The company’s federal solutions support U.S. agencies with a variety of National Security, compliance, and research and development programs. Leveraging the expertise of its principal staff - former intelligence, federal law enforcement & military, and business professionals - Prescient’s capabilities are suited for corporate, federal and international clients.