Top 3 IT Challenges to Delivering a Successful Big Data Solution

In a previous post we spoke about the anticipated growth in Big Data and Analytics over the next 5 years, according to research from the IDC. Continued on from this, here are the Top 3 IT Challenges that the IDC considers could hinder the success of a move Big Data and Analytics if not sufficiently addressed:

1. Lack of Sufficiently Skilled IT Staff and Cost of Technology
IT Directors today need to show a good return on investment from new technologies and so must have a very clear idea of what the business can achieve with Big Data. In addition, employing and maintaining the right skills in a highly competitive market also needs to be factored into the cost of embarking on a Big Data project and so, again, IT Executives must have a clear idea of these costs to assess the return (and indeed the viability) of Big Data.

2. Managing Data Quality
The information you get out of any system is only as good as the information you put in. Data quality problems usually arise because no one person is responsible for the complete “data picture”. Without a central custodian ensuring that all data, across all systems is correct, consistent and up to date, the quality of data can vary significantly from one system to another. Of course, this isn’t a problem that’s new to Big Data – but nevertheless it is an important point to consider when assessing the viability of a Big Data project.

3. Data Integration
Another age-old problem that IT functions will be very familiar with is that most organisational data is highly fragmented. This creates challenges at several levels when embarking on a data integration project of any nature: syntactic (how do we define a common format for our data?), semantic (what are the agreed definitions?) and political (who ‘owns’ the data?).

While the above factors do indeed pose a challenge to CIO’s, they are not insurmountable. Indeed, the benefits that Big Data offers will, for most organisations, considerably outweigh these challenges.

It is often quoted that some 90% of the data that exists today has been created in the last 6 years. This data explosion, combined with the rising sophistication of analytical tools and the speed offered by Big Data technologies, means companies should be addressing these challenges sooner rather than later to ensure they can maintain competitive advantage.

Next week we will be looking at the top ten verticals signing up for big data and analytics. Sign up to receive this next post straight to your inbox.