This is the third part of a series written by Guest Blogger Charles Boicey from UC Irvine Health (part 1, part 2). The series will demonstrate a real case study for Apache Hadoop in healthcare and also journal the architecture and technical considerations presented during implementation.
During the summer of 2012 I made a commitment to the folks at Hortonworks to produce a three-part blog series about my experiences applying Hadoop and other Big Data technologies to solve healthcare problems. The first was published in August of 2012 and can be found here. I was at UC Irvine Health at the time and had been working with Apache open source Hadoop for two years and had recently moved the project on to the Hortonworks Data Platform (HDP).
My interest in applying Hadoop and related Big Data technologies to healthcare stemmed from my observations of how Google, LinkedIn, Facebook, Twitter, and Yahoo were able to leverage large and disparate data types without the headaches found in the Relational Database Management System (RDBMS) world. The ability to store healthcare data in its natural state and in its entirety without a schema in 2010 was unheard of, let alone the ability to apply streaming data from physiological monitoring devices to an analytic model to detect early signs of a patient deteriorating.
Between the years 2010 and 2012 we were able to demonstrate that Hadoop is capable of ingesting, storing, and processing any data types generated within the healthcare environment including, but not limited to, the following:
In April of 2013 I wrote the second installment that can be found here. In this post we covered the monitoring of patients, both inpatient and outpatient, in real time utilizing the streaming analytics capabilities inherent in the Hortonworks Data Platform.
Also covered was how Hadoop plays well with others. We can bring in graph databases, NoSQL databases, event processing, messaging and other technologies into the ecosystem. Hadoop is adjunctive to the EMR as well as the enterprise data warehouse; it is not a rip and replace proposition, rather a much-needed technology that fills the data and analytics gaps inherent in both systems.
Next week the Health Information Management System Society (HIMSS) will be hosting their annual Healthcare IT conference where 40K attendees are expected and over 1,400 vendors will be displaying their products. Analytics solution providers are at the top of the attendee’s list of vendors to check out so I thought this would be a great opportunity to publish the third, albeit late, installment describing some areas of advancement from April 2013 to present.
An area of concentration for healthcare over the past few years has been the need to better our quality scores. To do this we need to understand in real time how the care we are providing measures up. Retrospective quarterly reports no longer cut it, as we cannot improve on something that happened in the past. As a result of the Hadoop ecosystem’s capability to process data in real time, we now are able to provide for both the inpatient and outpatient environment’s Continuous Quality Monitoring of how well care providers are doing in meeting quality measure goals. Our patients can now be discharged from our care while meeting all of the applicable quality measures.
We are now ingesting and processing social determinates of health data, social media data, geographic information system data, open data, as well as medical IoT data, to better understand our patient populations and intervene during or before an event occurs.
The ability to deploy advanced analytic models in the Hadoop ecosystem has led to the development of Patient Situational Awareness applications, that allow for the monitoring of streaming data, picking up subtle changes that could lead to an undesirable outcome and passing the information off to clinicians in near real-time. The application of this technology is not limited to just the highly monitored areas (such as the ICU), but is also deployable in the home or mobile environment.
During this time other organizations answered the call and began their healthcare journeys in into Hadoop and supporting Big Data technologies.
Last summer I joined the team at Clearsense for the sole purpose of accelerating the development and adoption of a healthcare centric, cloud based, multi-tenant advanced analytics ecosystem that serves the needs of healthcare organizations large or small.
“The combination of supported open source Hadoop software such as Hortonworks Data Platform (HDP) deployed in a secure HIPAA compliant cloud, makes it possible to offer a rural safety net hospital the same advanced analytics as a large integrated delivery network”
– Richard Proctor GM, Global Healthcare at Hortonworks
Analytics provided at scale as a subscription ensures that healthcare organizations are able to deliver data driven solutions to caregivers that ultimately deliver to their patients the highest level of care. It has been an amazing six years leading the charge and it has been even more rewarding to see how healthcare has adopted these technologies to provide data driven car.