Big Data… Is it really Dirty Data?


Big Data…or a Big can of Worms!

Big Data…well, it really is beyond question, potentially a huge Big Can of Worms.  And, if you are not in the business of being one of the very few organisations that can actually benefit from it, you could well do with asking yourself; “Do we really need it“?

Everybody is scrambling for Big Data and yet they are doing so blindly. When they get it, few understand what they have and even less understand what they are seeing! And a massive problem of having Big Data is not directly to do with the Data itself, but with the World in which we find ourselves today…

There are many reasons to avoid Big Data and, perhaps surprisingly, only a few reasons to embrace it. In a series of mini-articles we will explore the advantages, and disadvantages, of Data, both Big and Micro and look at which may be best for your company.

So, to begin our mini-exploration, let us look at what Big Data actually is, and how it came to be…

Well, what is it?

There are really two ‘types’ of Big Data and, just to add a bit of confusion, there can be an overlap between the two!

Simply speaking, Big Data can be Big because it is crushing large amounts of data from across a previous period in time.  This can be historical data, accrued from the past ten years of trading of a company. Equally it could be a large amount of data accrued during an acquisition of companies by a single organisation.  Whatever the sources and origins of the data, large amounts of data can now, relatively easily, be bundled together into large repositories – databases.

The second definition of Big Data is applied to sets of data that are derived from multiple, but very different sources.  Data is now frequently digested from various Social Media streams, retail shopping feeds, online shopping feeds and a myriad of other input channels.  This concatenation of data is becoming easier to collect, but certainly not easier to correlate!

Now, it does not really take too much understanding to see the immediate two problems with either, or both of, the above data groups.

  • Organising the data from all of those different feeds and corporate configurations
  • Extraction of the combined data into any sensible form of usable information

…and where did it come from?

One of the primary reasons that Big Data is now available to virtually anybody is the huge decline in the cost of data storage.  Without the availability of cheap, and easily accessible, storage, Big Data would remain available only to the top 1% of companies.

In our next adventure into the world of Big, or micro, data we will explore what it takes to ‘group’ your data and keep it fresh and ready for consumption..  We may even delve into the potentially horrendous situation of acting upon, misunderstood or incorrectly analysed information!!

Get Intelligent

Do you have the Intelligence?  Give us a call and we can give it to you!!