Big Data or Smart Data?

The hot topic at any marketing, insight and data analytics conferences at the present time is Big Data. In our increasingly technological driven world we are generating enormous volumes of data, some estimates suggest the world is adding several quintillion (1018) bytes of data each day. And as marketeers and analysts we see this as a goldmine of information and we are instinctively driven to find ways of making use of it.

In terms of marketing activity, “Big Data” relates to the ever growing data sets created by our customers and prospects, not just through the traditional marketing database or CRM, but also from retail and eCommerce activity, from social media sites such as Facebook, Twitter, LinkedIn, Flickr and Foursquare. We can also access data from sources such as Google Analytics where we can see how visitors find our websites and what they then do once there, and media can include the full range of digital formats such as pictures, video,audio, CCTV.

For example a financial lender may wish to  look at an applicants Facebook site as part of the credit checking processes to see if they have pictures of holidays every other month and parties every night – would they be a safe bet to lend to?  We have all heard stories of employees checking out social media sites of potential employees as to their suitability.

IMG_0986

Would you give a pay-day loan to someone boasting this shoe wardrobe on Facebook?

The immediate consequence of wanting big data is that you need big servers and big software to process it and at this point the IT Team takes over, rub their hands with glee and an obsession develops on gathering all the data that can be found. Little consideration is given as to why the data is being collected, other than “it will help our marketing to be more effective” and planning what is going to be done with it once its gathered is often non existent. And with this data hunter gatherer approach, it seems that the sound statistical and analytical approach to extracting knowledge and insight disappears and in consequence it becomes easy to produce results from looking at the data that at best can be skewed an,d even worse, can be totally unsound. And there is also an implication that big data with its big servers and big software will cost big bucks so making it an exclusive tool for the large multinational Company, leaving us small and medium enterprises behind.

I would argue however that what is really needed to make best use of the data available is not Big Data but Smart Data. Concentrate on looking carefully at what data is available, extracting only data that is potential directly relevant and then using sound statistical sampling techniques to analyse and unlock the insight and knowledge within the data.

In a sense the term “Big Data” is a strange term as it is used to imply that size is in some way a measure of quality and value and I believe in a few years time we will find it odd to talk about big data. Size in itself doesn’t matter – what matters is having relevant data that helps us solve a problem or address the questions we have and so I prefer the term “Smart Data”. Smart data is available to all organisations small and large. It can use existing tools and techniques to evaluate and gain insight.  And when we want to bring in the additional data sets that the likes of social media and Google Analytics offer us then the way to do this is through creating and integrating small data sets using the techniques and systems we already have in place and not through building big data monoliths and creating massive centralized data warehouses.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s