Posts by Kim Rose:


Getting the most out of business analytics

One of the most prevalent uses of Hadoop architecture by enterprises is to create business intelligence and analytics tools that can be leveraged to identify areas that could be improved to foster greater efficiency and productivity. According to a study jointly conducted by Gartner and the Financial Executives Research Foundation, business intelligence and analytics were top areas of focus among surveyed CFOs. Overall, 15 of the top 19 processes that were identified as needing improvement by the study's participants could be addressed through the use of these resources. In addition, 59 percent of the survey's respondents cited the ability to facilitate operational decision making processes as an area that required more technological advancement. Furthermore, half of all participants stated that the capacity to effectively monitor business performance was an investment need as well.

The report indicated that enterprises could improve their business analytics deployment by facilitating communication between data scientists and the C-suite executives who make operational decisions. Specifically, executives should be aware of how these tools work and how to best utilize them to maximize their effectiveness.

An IT executive recently presented several steps companies can take to avoid common analytics pitfalls and optimize their business intelligence initiatives, including:

  • Enterprises should broaden the focus of a business analytics program to the entire enterprise to find new connections and relationships. This includes expanding data collection efforts and taking a holistic view of analytics projects.
  • Companies should look inward to find their data analytics leader. Many executives may be tempted to find a high profile hire who will jump-start operations, but an existing employee will already be familiar enough with the company's needs and culture to foster a successful data-driven culture.

Enterprises can attain significant benefits from their Hadoop analytics and business intelligence programs. However, getting the most out of these processes requires a broad vision, internal communication and a strong business culture dedicated to the pursuit of data analytics.

Hadoop big data gets personal

One of the reasons Hadoop big data analysis is particularly valuable is that much of it is fundamentally about people – what they do, what they buy, what they think about or what they want, etc. Analysis gleaned from big data offers a clearer picture of human conditions and enables organizations to anticipate needs and respond to wants. Many big data projects offer a more objective view of the way humanity functions and can trigger insight into how to make improvements.

The newest trend in big data insight is analysis directly targeted at people, reported Bloomberg Businessweek. In particular, 'People Analytics' is an approach through which organizations turn the focus inward and use big data comprised of external research and intercompany observations to make operational improvements and interact better with employees.

"Because most communication and collaboration happens face to face, the data are critical for people analytics to take that next leap forward and become a transformative organizational tool," wrote Businessweek contributor Ben Waber. "By combining precise data from both real and virtual worlds, we can understand behavior at a previously unimaginable scale."

According to Businessweek, examples of company solutions that resulted from such a combination of data and observation include changing workday structure to improve morale and reduce attrition, as well as reducing the number of coffee stations to facilitate impromptu interactions among personnel.

Hyperpersonal analytics focus Hadoop big data inward
As Hadoop data becomes increasingly critical for facilitating measured objectivity to personal pursuits , more will start to apply it to their own activities. Harvard Business Review correspondent H. James Wilson wrote about 'auto-analytics,' a semantic category for the process of applying big data insights and visualization to daily actions.For example, according to a study by the Pew Researcher Center, 69 percent of people use a self-tracking mechanism when they exercise. Twenty-one percent of these people use smartphone apps or other data-based functions to make self-tracking more objective and precise.

Wilson wrote that people are seizing upon data-based technological resources to make their own lives and practices more efficient and informative. Technology consultants ABI Research projected that by 2018, more than 485 million wearable devices, from smart watches to various articles of smart clothing, will have been shipped worldwide. The trend toward data-based self-analysis parallels one of the important advances in Apache Hadoop, which offers its users a highly customizable, personalized approach to big data filtration and analysis.

The importance of data accuracy for Hadoop banking analytics

Business analytics solutions, such as those built upon Hadoop architecture, can be a major resource for members of the financial industry. With these tools at their disposal, banks leadership could gain major insights into their operations and market places as well as improving their efforts to effectively engage potential clients.

However, these organizations need access to accurate customer data to gain the full benefits of business analytics solutions. According to a recent Experian QAS survey, many financial institutions have struggled to ensure the accuracy of the data they gather, Credit Union Journal reported. Ninety-one percent of the organizations that participated in the survey suspected that the information they collected was inaccurate in some fashion. While respondents reported that as much as 18 percent of their data could not be ensured for accuracy on average, 27 percent of the total number of participating enterprises could not say how much of their information was compromised.

There are several steps that financial institutions can take to increase the accuracy of their data:

  • Establish regular database maintenance tasks to manage files
  • Integrate automated verification tools to ensure that client and prospect data is up to date
  • Create a full data workflow to prioritize high-volume entry points

Information services expert Thomas Schutz noted in Bank Systems & Technology that banks and other financial institutions could improve the accuracy of their collected information by condensing the number of databases they maintained. This will prevent duplicate entries from being entered into multiple systems. One of the problems with operating multiple databases is that updated information may not be spread to each system, leaving some with inaccurate consumer data. In addition, Schutz recommended that banks place more emphasis on training personnel to enter and access data in a streamlined and uniform process. This will eliminate entry errors and inconsistencies, maintaining the accuracy of gathered data across the enterprise and ensuring that it maximizes the effectiveness of their Hadoop initiatives.

3 steps to making Hadoop data aerodynamic

Comparing Hadoop big data analytics to an aerodynamic vehicle produces a fairly apt parallel – both are modern concepts that harness lots of information and figure out how to concentrate it for optimal results. Big data can produce a lot of figurative weight and drag if insights aren't directed with the right focus, and the friction caused by retrieval lags can torpedo organizational growth. Here are three ways to make big data analytics soar.

1) Conquer the air
This step might sound silly, but it comes in the spirit of believing that the sheer amount of available information can be conquered. Big data arrives constantly and from various sources, continuously regenerating and offering new perspectives. According to DotNetNuke CEO Navin Nagiah, data streams will continue to get more crowded, but that doesn't have to mean analytics efforts must become cloudier. On the contrary, having the data and being able to use it will be paramount to success. 

"In the business world, it is the company that has the data that has the power," wrote Nagiah.

2) Concentrate the power
Big data possession is important, but analytical insight will plateau if companies aren't using the right tools. Whether big data analytics are directed toward strengthening customer relationships, synchronizing business operations or innovating solutions, organizations will benefit from information immersion. The inherent capabilities that Hadoop architecture offers analytics users are a good starting point because they're naturally geared toward data accrual.

3) Put big data in motion
The third step is the implementation of Apache Hadoop, the software that makes data effective for complex insights and ones made in real time. A recent TechRepublic report looked at data in motion, the kind of real-time analysis that offers significant advantages for its users.  Components like Hadoop Hbase and Hive enable instantaneous, integrated insights by allowing ad hoc users to interface directly with databases, producing solutions that can be seamlessly applied without slowing down incoming data streams.

Bringing big data to the racing world

NASCAR may dominate the U.S. racing scene, but elsewhere in the world Formula One is king. From Italy to China, F1 is viewed as the pinnacle of competitive motor sports, drawing hundreds of millions of viewers and generating massive revenue streams. According to the latest figures provided by the organization's global broadcast report, worldwide viewership reached 515 million in 2011, Reuters reported. After factoring in race hosting and television airing rights fees, trackside advertising and sponsorship revenue and ticket sales, the annual revenue generated by F1 totals approximately $1.5 billion, according to Autoweek.

With that much at stake, it's no wonder that the companies responsible for building race cars would take every available advantage to improve their vehicles' performance. In the case of Britain's McLaren Group, that includes deploying big data analytics to get the most out of their automobiles, ZDNet reported. The company's researchers outfitted each of its race cars with approximately 160 sensors that typically create one gigabyte of raw data during each event. That information is automatically sent in real time both to the team's garage and to a company technology center. The racing team can leverage that data to tweak the performance of their race cars in the midst of an ongoing race to address emerging structural issues or take advantage of track conditions.

Building state-of-the-art racing machines
Meanwhile, McLaren's team of engineers utilize the data gathered during races to enhance their efforts to build state-of-the-art competitive racing machines. Leveraging data analytics tools is especially important to the development process because the company builds nearly entirely new race cars from the ground up each year.  According to McLaren Group CIO Stuart Birrell, only about 5 percent of the parts used to create new vehicles are from earlier models. With the expectation of creating new high-performance vehicles each year without relying on past technology, engineers can significantly benefit from the insights provided by big data solutions.

As McLaren researchers go through the design process, building models and subjecting them to fluid dynamics, wind tunnel and live track testing, data analytics software provides feedback on a vehicle's performance every step of the way. The cars are equipped with roughly 300 sensors to collect information regarding how it responds to various conditions. Even after engineers have created a finished product, the company's race cars are continually examined using big data as the racing season rolls on. Birrell told the news source that the organization's current data analytics efforts are focused on making micro adjusts to improve the speed and performance of their racing machines and identify any issues that may be affecting their ability to compete.

Like members of many other industries, McLaren engineers have just begun to realize the benefits of Hadoop big data tools. Given more time to pursue their data analytics programs, researchers will continue to find new applications for the burgeoning technology.

Improving movie scripts with data analytics

Over the past few decades, Hollywood producers have increasingly become dependent on blockbuster movies to generate the revenue streams they require to rationalize investing millions of dollars into a single film, with blockbuster movies like "The Avengers" and "The Dark Knight Rises" forming the backbone of the filmmaking industry for some time now. According to the Economist, successful releases aimed at a wide audience were in large part responsible for box-office revenues reaching a record $10.8 billion in 2012. Individual movies are raking in more money than ever before. The recently released "Iron Man 3" has already earned $175.3 million after the first three days of its North American release, CNN reported. Last year's top earner, "The Avengers," made $623.4 million in the United States alone.

The rising costs of film development
However, the cost of funding these projects has skyrocketed as well. For example, "Iron Man 3" cost Disney approximately $200 million, according to CNN. With the amount of money being invested in potential blockbuster movies, Hollywood production companies have looked at every possible method to ensure the release of a lucrative hit. One potential solution filmmaking executives have been pursuing is data analytics.

More studios have begun employing big data solutions during the script writing process to eliminate factors that may prevent a film from connecting with an audience. For instance, many producers have hired the services of a former statistics professor named Vinny Bruzzese to analyze their scripts and suggest changes that would make them more palatable to a wider audience, The New York Times reported. Some of his recommendations would seem to fall in line with commonly held thoughts such as that audiences like sympathetic sidekicks.

Digging into audience data
However, he has also gleaned some unusual insights into factors that characterize successes and flops. For instance, his research concluded that movies containing scenes with characters bowling tend to bomb at the box office. In addition, Bruzzese determined that audiences prefer demons that target a specific character than those summoned by a Ouija Board in horror movies.

While some in the film industry – writers, specifically – have shown some resistance to the burgeoning use of data analytics in the script development process, many producers, studio executives and financiers have praised its application.

"It takes a lot of risk out of what I do," producer Scott Steindorff told the news outlet. "Everyone is going to be doing this soon."

Bruzzese's recommendations may veer from the insightful to the seemingly bizarre. However, with hundreds of millions of dollars being invested in blockbuster releases, Hollywood producers will take any opportunity to mitigate the financial risk of developing a film. With Hadoop big data tools, studios can craft scripts that are more likely to engage an audience and generate ticket sales. 

Hadoop adoption is key to understanding big data

The Hadoop explosion continues, with the number of organizations adopting the platform growing at a compound annual growth rate of about 60 percent, according to the International Data Corporation. However, ReadWrite's Matt Asay wrote that many of the companies adopting it are still acclimating to Hadoop and aren't using it at its optimal capacity. Currently, most organizations that use Hadoop take advantage of its storage and ETL (extract, transform, load) features, but aren't taking the crucial steps for optimizing big data analysis.

"The fact that most enterprises have yet to get to analytics in any meaningful way is simply a description of where we are in the Hadoop market's evolution," Asay suggested.

Part of the lag time, Asay asserted, is that its many components can make some users unwilling to spend time discovering what it has to offer. A recent report by CIO Insight illustrated the stratification of Hadoop users. Based on a survey of 107 data professionals, the report found that 68 percent used Hive, 57 percent employed MapReduce, 34 percent used Pig and 15 percent utilized Native SQL. These discrepancies, said research analyst Matt Aslett at Hadoop Summit, can be ironed out with more concentration on the enlightened process of Hadoop.

"Attempting to fast forward to analytics, missing out on the processing/integration stage, creates silos and will result in disillusionment," he observed.

Taking the steps toward optimizing Hadoop
High-functioning Hadoop, wrote Aslett, makes the most effective use of the big three of big data: volume, velocity and variety. Ultizer's Jonathan Gershater wrote that Apache Hadoop's open source approach directly impacts the efficacy of the big three to work for an enterprise. Additionally, the growing impact of Hadoop software decreases a firm's costly reliance on hardware.

"Because Hadoop is distributing the processing task, it can take advantage of cheap commodity hardware – compare this to processing all the data centrally on big expensive hardware," wrote Gershater.

Eventually, wrote Aslett, the big three become the singular concern: the totality of big data. Total data is the smart form of big data, a strategy of examining big data from all angles so that everything is seen from different perspectives and nothing falls through the cracks. Optimized Apache Hadoop allows big data to be stored, processed and integrated most effectively. Aslett employed the idea of an ecosystem created by Hadoop, a complex distribution of information and resources that accounts for influences from and is influenced by other factors.

"The Hadoop ecosystem is vibrant, with strength in depth and breadth," Aslett wrote.

Reduce travel stress with big data

Recent developments in big data applications use information about consumer habits and demographic trends to develop stress-reducing solutions for travelers. Travel management company CWT recently developed the Travel Stress Index, an algorithm-based tool that they hope will translate information into actionable recommendations, reported InformationWeek’s Ellis Booker.

“We had the transactional data and we had some traveler profile data, but it was scattered and we had to bring them together,” Catalin Ciobanu, CWT big data analyst, told InformationWeek.

Harnessing big data for pointed results involved a multi-faceted approach. They gave surveys to over 7,000 travelers from a variety of backgrounds, who were asked to rank 33 activities on a 1-10 scale, according to the stress they generated. Three categories of stress-inducers came to the forefront: lost time (like not being able to work in locations without wireless or planes without access), surprise (like lost luggage) and interruptions of daily routines (like altered sleeping schedules or a lack of healthy foods). This data was also broken down by demographics, from gender and age to industry and even job title.

“The goal was to calculate the productivity hit caused by stressful travel for different types of travelers,” Booker reported.

Using big data to provide travel solutions
The data CWT analyzed prompted several conclusions. While they observed what Ciobanu called “an irreducible component of stress” that caused nearly 70 percent of surveyed anxieties, they found that 32 percent could be reduced with their help. Using the data collected, they could create sample profiles of what a consumer might be stressed about and respond more directly and personally to a customer’s needs. These possible resources, Booker reported, include offering advice about connecting flights to those made particularly anxious by time constraints and recommending carriers based on lost luggage data for the especially surprise-averse.

According to a CWT press release, advice that confronts the intangibles of business travel can have appreciable financial benefits. The lost time for a firm with an average of 5,000 business trips a year could be as much as $3.3 million, a third of which could be saved through increased traveler productivity.

Other tools, like American Airlines’ ‘abandoned cart’ program, use big data to keep users informed about trips they’ve looked at but haven’t purchased. In addition, Farecast uses databases to predict the optimal time to book a flight and respond to consumer concerns about the cost of air travel. Like CWT’s Ciobanu, business leaders in just about every industry are seeing the value of using big data to develop tools like the Travel Stress Index that utilize clients’ reactions to best meet their needs.

The humanitarian power of big data

Big data can provide insightful solutions for businesses in many different industries. As a few recent projects show, big data might be able to benefit humanitarian life-saving strategies and bring peace to embattled parts of the world. As researchers and experts in many different fields are able to access and utilize big data from many different sources to inform their work, they are discovering ways in which analytics can create positive socio-cultural change.

International humanitarian advocate Patrick Meier recently wrote on Forbes about 'crisis maps,' tools for solving social problems with data. Innovations like crisis maps are fairly new, developed using open source big data insights. Real-time social communications converge with ethnographic and geographic research to provide insight during disastrous events.

"Crisis-mapping platforms display eyewitness reports submitted via e-mail, text message, and social media. The reports are then plotted on interactive maps, creating a geospatial record of events in real time," wrote Meier.

One platform cited by Meier for its influence in responding to global conflict was Ushahidi, an interactive-mapping platform that was used to gather and plot eyewitness reports for 2008 protests in Kenya that followed presidential elections, and again after the 2010 earthquake in Haiti.

"Once these reports were manually collated and plotted on the Ushahidi platform, they became a live crisis map of urgent humanitarian needs. For example, the map showed exactly where victims lay buried under the rubble of collapsed buildings, and where medical supplies needed to be delivered," wrote Meier.

Can big data help to keep the peace?
The precarious balance of thwarting conflict from starting and escalating is a difficult task, but CNN's Tara Kangarlou reported that the U.S. State Department has been using information gleaned from big data analysis in its strategies. The recently created Conflict and Stabilization Operations office uses sourced information to target areas where physical violence might present a threat.

"CSO analyzes 'large data sets' as well as 'civil society' generated data - essentially the sum of patterns, human behaviors, electronic signals, social media elements and everything tangible that creates masses of technological and non-technological data," Kangarlou reported.

These big data sets have furnished the CSO with fairly accurate information about many potential dangers, like timing of defections and growing factions – as well as predictions like when allies would offer support. These insights allow military operatives to quash potential violence preemptively, as a result saving lives, energy and money. The big data approach has improved communications and strategy in a top-down manner, Kangarlou wrote.

Big data’s potential for higher education

One of the many arenas where officials could benefit from big data deployment is higher education. With their large volumes of student information, including enrollment, academic and disciplinary records, universities have the datasets needed to benefit from a targeted analytics project.

Recent reports have suggested that administrators across the United States have become to realize the potential of analytics and have invested more resource into big data projects. Gartner's "Big Data, Bigger Opportunities: Investing in Information and Analytics" survey found that 42 percent of IT professionals across all sectors reported investing in big data projects or planning to do so in the coming year, reported Campus Technology. The report also identified higher education as field with great promise for big data application, citing the availability of various sources of data.

More data brings greater insights
Once universities begin investing more in Hadoop big data projects, they benefit from its many applications. SmartData Collective contributor Mark van Rijmenam argued that with the advent of massive open online courses, higher education administrators have an entirely new pool of data to access, which could provide even greater insights into a college's operations. With these tools, school officials could enhance various aspects of campus life, including student success and academic performance.

One of the ways educators can utilize big data tools is to analyze the performance and skill level of individual students and create a personalized learning experience that meets their specific needs. MOOCs could be especially beneficial in this instance, as the heaps of structured data contained within their records could be easily gathered and processed by data analytics tools. With more sophisticated analytics software, professors could monitor many different factors regarding student performance beyond simple right or wrong answers, including the amount of time needed to answer questions as well as any connections regarding the types of test questions that were skipped over. Education IT researchers have been developing adaptive learning software that can process this information during an assessment or lesson and recommend further exams or coursework based on a student's performance.

According to Mark van Rijmenam, big data tools could even analyze group dynamics, looking at the various strengths and weaknesses of individual students to determine the optimal arrangement. Group projects would have a more fairly distributed workload as the individuals involved would compliment each other's skill sets.

Ultimately, by enhancing the learning experience and improving student performance across the board, universities will be able to reduce dropout rates and increase their graduation numbers. In addition to the purely academic benefits, colleges could also expect a greater and potentially more loyal alumni base that would be more generous with its donations. Hadoop big data software can provide higher education IT teams with tools to build analytics programs customized for their specific needs and goals.

Building better cars with big data

When car companies across the United States saw their funds quickly dwindle at the end of 2008, some predicted the end of the American auto industry. Several of the nation's largest automakers stood on the brink of bankruptcy and slashed jobs in order to avoid complete financial collapse. According to information presented by the White House, at the height of the automotive crisis, nearly 150,000 industry workers lost their jobs in a single quarter. When economic circumstances become dire and markets fluctuate, businesses need to pursue new methods of operation to weather the financial storm. In the face of economic turmoil, the Ford Motor Company turned to big data analytics to help it regain its footing in the global auto industry.

Integrating customer demands with the design process
Big data, along with the development of increasingly sophisticated sensor equipment, has allowed Ford engineers to more effectively analyze the performance of their automobiles and build vehicles based on a balance of quality and customer demands. GigaOM reported that the automaker has deployed a series of internal analytics projects to help engineers improve the design of vehicles as well as integrate external big data analysis to inform those efforts. For instance, researchers recently combed through social media sites to decide whether their latest line of Escape sport-utility vehicles should come standard equipped with a manual or power liftgate. Analysis of these sources found that consumers overwhelmingly favored the latter.

Easing customer concerns with big data
One of the ways Ford has leveraged data analytics is by easing anxieties relating to its burgeoning Energi line of plug-in hybrid cars. With any new and unfamiliar product, consumers will invariably worry about performance. With hybrid cars, the common concern has been the range of operability and possibility that a motorist will become stranded because he or she will not be able to locate a recharge station when the vehicle's battery is depleted. Using data gathered by the Energi's onboard wireless module, Ford engineers can provide drivers with a range of useful information, including the car's performance, battery life and the nearest charging stations.

Ford's commitment to big data has helped the company enjoy some its most robust sales figures in years.  According to Autodata Corp., sales for Ford's Fusion line of midsize sedans has necessitated a projected 9 percent increase in North American production in the second quarter, The Associated Press reported. In order to meet the increased production demands, the automaker will add a shift of 1,200 workers to the plant responsible for the vehicle's production. Matching customer desires with production changes, Ford has been able to roll out cars that are more palatable to current consumer tastes. By leveraging Hadoop big data, companies can provide customers with the products they truly want.

Big data takes on the role of human resources

Since the recent recession hobbled the economy, Americans have struggled with unemployment. Bob Eisenbreis, a chief monetary economist with New York's Cumberland Advisers, said that although the unemployment rate is currently recognized as 7.7 percent, the true level is 14.3 percent when factoring in the underemployed, reported Forbes. 

One of the hardest hit demographics in the recession has been young adults between 16 and 24. According to the Center for American Progress, the unemployment rate for that demographic stands at 16.2 percent. Furthermore, the organization found that experiencing an extended period of unemployment early in a career could have an adverse effect on an individual's earning power later in life. A prospective worker who has been unemployed for at least a period of six months stands to lose $22,000 over the course of the ensuing decade. The organization's analysis concluded that young Americans will lose approximately $20 billion in potential earnings in the next 10 years.

Using big data to remove extraneous information
Many capable applicants have found themselves passed over for many jobs as traditional hiring methods overlook their potential in favor of flashy credentials that do not necessarily predict future success. However, big data has emerged as a possible solution to this issue. The New York Times reported that Luca Bonmassar and his analytics firm Gild have devised a program that attempts to quantify more abstract qualities in an applicant. The basic goal is to ignore criteria such as an applicant's alma mater and simply identify whether he or she can effectively take on the duties of the vacant position. 

Bonmassar's software specifically focuses on the adeptness of computer programmers. With the internet as a data pool, the program searches for data that would suggest whether an applicant's code is respected among other programmers and how often it gets reused. The tool also combs social media sites in an attempt to ascertain how well a potential employee will interact with current staff members.

Addressing hiring bias concerns
Gild's chief scientist, Vivienne Ming, believes that data analytics software can be used to successfully eliminate human bias from the application process.  The existence of bias across many industries has been well documented. For instance, a study conducted at Princeton University found that female applicants for a managerial position were less likely to be hired because they were viewed as less competent than identically qualified male applicants.

Developing Hadoop big data tools for the application process is a win-win scenario. For qualified applicants who have languished in unemployment or have been passed over for promotions, data analytics tools can finally showcase their potential. Businesses, on the other hand, can benefit by hiring more capable and fitting employees without the blight of bias.

Big data’s potential drives healthcare spending for epidemiology and genomes

Big data has shown enormous potential in the healthcare industry and enterprises are beginning to respond. According to the latest study released by TechNavio, the global big data market in the healthcare sector is expected to increase at a compound annual growth rate of 32.96 percent through 2015.

Researchers found that one of the major drivers behind the market growth has been increased access to data. The advent of electronic health records and the federal government's push to standardize their usage has resulted in more organizations obtaining the information needed to launch a successful data analytics program. The study also discovered that more medical professionals were deploying analytics tools to enhance their epidemiology research efforts and uncover new insights into the patterns and causes of various ailments.

More sophisticated tools lead to greater insights
The potential for data analytics to improve patient care and save lives has fostered excitement from many within the medical community. Computerworld reported that analytics tools like Hadoop big data solutions can potentially process both structured and unstructured data to help physicians create more effective treatment practices. Analytics software has been able to crunch structured data for years, as anything contained in a digital format is essentially fair game. In contrast, unstructured data such as handwritten physician notes and medical charts has presented more challenges. However, the technology is quickly developing and many researchers have already made gains processing unstructured data. Once analytics tools can process digital and physical data with equal ease, members of the medical community expect the real promise of healthcare big data will be fulfilled.

"With big data, what happens in a doctor's office is going to be vastly different from what we see today," Robert Walker, director of health innovation for the U.S. Army Surgeon General, told the news outlet. "The top five or 10 things that people die from in America are life-style induced. That's absurd. Maybe instead of vital signs, I'm just going to look at what you buy in a grocery store."

Deciphering the human genome
Medical researchers are particularly excited about the potential for big data tools to help prevent genetic diseases. The costs of mapping out an individual's genetic code is rapidly dropping. Once those expenses have fallen to an acceptable levels, hospitals could conceivable use big data tools to identify flaws in a person's genetic code that could suggest the development of diseases such as Alzheimer's later in life. 

For now, healthcare researchers are busy leveraging big data tools with a variety of resources including genetics analytics to provide better treatment any way they can. For instance, data analytics tools can comb through a number of factors to determine how patients will react to certain courses of treatment, providing physicians with an additional layer of support that could prevent costly errors from being made. The more Hadoop big data tools mature, the greater resources physicians have for improving patient care and fighting disease.

Big data takes to the high seas

Piracy off the coast of Somalia has grabbed a lot of headlines in recent years. Stories of ships being boarded by pirates on the Indian Ocean and the Gulf of Aden have captured the attention of people around the globe. While piracy may be a mere curiosity to some, it has very real ramifications. The Economist recently reported that the cost alone in ransom payments made to ensure the safety of captured sailors was estimated to be $53 million each year. The exact total cost of piracy to the world economy is still up for debate, but experts agree that it is significant. Oceans Beyond Piracy's analysis concluded that the annual cost of these crimes totaled $6 billion. That figure pales in comparison to the World Bank's estimates, however, which place the annual cost of piracy at $18 billion.

Somali government officials have grappled with the problem for years. Recently, they have begun to seek out new methods to quell piracy on its seas.  Sabahi Online reported that authorities had launched an outreach program for pirates, offering them guidance and opportunities for employment. By providing avowed pirates with a financially viable alternative, officials hope they will turn away from a life of crime on the high seas.

Big data plots a safer course
One of the newest weapons government agencies have to combat piracy is big data. According to Business Insider, organizations have begun deploying analytics software tools to determine where pirates may strike next. Law enforcement and government agents have had trouble predicting pirate activity because Somali buccaneers have consistently changed their tactics and altered courses. Authorities have a wealth of information to leverage, but very little of it is in a structured and easily quantifiable format. However, developments in big data technology have allowed researchers to process unstructured data with greater ease.

Culling information from sources as expansive as incident reports, pirate communications and interviews as well as postings on social media sites, officials have leveraged big data software to predict where pirates are likely to attack and – perhaps more importantly – where they are not. With this information, ships can stay on safer courses, traversing the waters off the eastern coast of Africa without worrying that they will run afoul of a band of pirates. Hadoop big data has typically been lauded for its capacity to address the issues of tomorrow, but it turns out it can take on the problems of the past just as effectively. 

Hadoop tools benefit medical IT

Hadoop big data tools can provide healthcare providers with numerous benefits both by enhancing patient care and driving down operational expenses. With the current state of healthcare costs in the United States, this should come as a welcome relief. According to analysis conducted by Aon Hewitt, the average healthcare premium increases are predicted to rise by 6.3 percent in 2013, bringing the average health plan premium cost for each employee to $11,188. With these rising costs showing no signs of abating, there has never been a more pressing need for the cost-saving applications of data analytics.

Charles Boicey, a member of the IT team at the University of California – Irvine Medical Center, recently spoke to eWeek about his organization's successful implementation of Hadoop architecture. Boicey was intrigued by the technology's ability to provide superior data analytics functions with little to no latency. Seeing an opportunity to deploy these resources using the medical center's extensive electronic records-keeping systems, Boicey began integrating a Hadoop Apache framework to glean actionable insight.

When considering his options for big data platforms, Boicey needed a framework that provided an open-source platform without intrusive and needlessly expensive proprietary features. He found his solution in Hortonworks' Hadoop big data tools, which offered the flexibility researchers would need to create their custom applications.

Once the framework was in place, researchers were able to pull data from a wide variety of  sources, including electronic medical records, financial data and even building ventilator sensors. The medical center's 20-year backlog of legacy data covering 1.2 million patients and more than 9 million records was also made available. Boicey and his staff could gather and process both structured and unstructured data types with the Hadoop platform.

Researchers at the medical center have already begun to leverage their resources into actionable results. They have established a data analytics-based feedback loop that has allowed hospital staff to gain insight into the concerns of their patients and improve the hospital experience accordingly.

Go to page:12345