Big Data Insights
Last week I had the pleasure of attending a workshop at Imperial College London on The Future of Big Data Management. This was organized by some of the CERN physicists who were interested in bringing together scientists across different fields together with those of us in the computing industry working on some of the same problems.
CERN’s Hadron Collider Experiments -ATLAS and CMS being the big two- are the latest in a long line of particle detectors that have always stressed the computing, network and storage technologies of the time. This workshop was set up not so much to look at the challenges of those teams (which are significant!), but on the impact that Big Data would have on other sciences -and what the issues would be.
As a result, there was a very interesting attendee list, from the people building the latest generation of the UK academic network, SuperJanet 6, to people just starting to think about the impact that large, invariably machine generated, datasets will have on their diverse sciences.…
Circumventing big data hurdles with Apache Hadoop and Hortonworks
Big data analytics have gained in popularity over the last few years, yet many operations continue to struggle with their implementation. Not all analytics platforms are created equal and businesses may find that a hastily chosen application is unsuitable for their needs and existing capabilities. For instance, only the most high-quality solutions will facilitate the use of unstructured data. The scope of a data analytics project can be greatly improved by incorporating difficult-to-quantify information sources such as audio and video recordings, allowing researchers to dig deeper into the data pool and glean more actionable insights.
The need for greater functionality has driven many organizations to choose Apache Hadoop as the platform from which to launch their big data initiatives, spurring market growth. According to a report released by MarketsandMarkets, the global Hadoop market is predicted to increase at a compound annual growth rate of approximately 55 percent through 2016.…