Sifting Through All the Big Hype About Big Data

Today, it is hard to escape all the chatter about Big Data. Every time I read an article or a news story around Big Data, I can imagine the frustration every marketer must face as they try to decide what the best course of action is in this brave new world that is rife with contrarian viewpoints. Hence, I thought it would be valuable to take a critical look and distill some fundamental rules that should govern marketers' actions in the big, bad world of Big Data.

Expensive technology solutions are not the panacea: Much of the Big Data conversations today originate from technology vendors and involve terms like the cloud, Hadoop, Map Reduce, Analytic appliances, Massively Parallel computing, etc. These conversations have little, if anything, to do with the business case/ROI of Big Data. Although all of these concepts are important, just like in the CRM era, focusing only on technology solutions and trying to build a 360-degree view of the customer will lead to frustration. Instead, one should start with identifying the value of the various sources of new data through small focused efforts. These efforts can be implemented at the grassroots level without investing significant amounts in technology.

Invest in the right skills before technology: More important than technology is having the right skills. Three key skills are required. First is the ability to frame and ask the right business questions of the data with a clear line of sight as to how the insights will be used. Second is the ability to use disparate open source software to integrate structured and unstructured data. Third is the ability to bring the right statistical tools to bear on the data to perform predictive analytics and generate forward looking insights.

Data Poor, Insight Rich is much better than Data Rich, Insight Poor: Never before has this been as true as it is now. The risk of data overload without commensurate insights is at its peak. The reality is that most organizations have barely leveraged the information they already had, even before the world of Big Data. Those who have generated insights have barely scratched the surface of being able to implement and act on those insights at the frontline, where they really matter. Generating meaningful insights and acting on them should be the first order of business, before new sources of data are exploited.

Be prepared to deal with new challenges: In the small data world, a lot of emphasis was placed on having complete data without any holes at the individual level. In the new world of Big Data, be prepared to deal with vast holes of information at the individual level; new analytic techniques are required to deal with this information effectively. Also, due to privacy reasons, several streams of Big Data (e.g. social media data) will never be traceable to a specific individual, limiting the use of this information for individual targeting purposes.

Real time is sexy, but usually not necessary: Ask yourself the following questions before falling prey to the real-time hype. Can you collect data in real time? Can you analyze it in real time? Can you make decisions in real time? Can you act on those decisions in real time? How much difference in customer value is created if you do all of the above in real time? Only when the answers to the first four questions is “Yes” and the answer to the fifth question is significantly larger than the cost of making things happen in real-time, should you consider real time as a viable option.