You are here
There’s been plenty of buzz about big data: the analysis of information that is too large to store on a single computer. MIT’s Sloan Management Review calls big data analytics a “mandate.” Five years ago a McKinsey article referred to big data as “the next frontier for innovation and competition,” and Kroger Chairman David Dillon referred to data analytics as the retailer's “secret weapon.” Big data has been hailed as a revolution, yet 70 percent of organizations are not using big data at all. What gives?
PG readers with long memories may recall another so-called revolution: the arrival of a technological breakthrough that promised to transform the shopping experience for consumers, provide retailers and suppliers with invaluable insights to optimize the supply chain, and inform collaborative planning. It promised to save you money and make you money. Like big data, this revolution languished for years and was labeled a “failure.” The revolution? Grocery store scanners. The time? The 1970s! With all the big data hype, it’s easy to forget that grocery experienced a big data revolution 40 years ago. A second revolution is on its way.
There’s a reason for the chatter. The motivation behind big data is simple: bringing more data to bear on an analytical problem leads to better insights and better decisions. Big data analytics has already transformed several industries: web search (Google), shipping logistics (UPS) and social media (Facebook). At this point, nearly any online interaction is driven in some way by big data analytics, whether it’s searching, purchasing or interacting.
So why is big data stuck in what Gartner calls the “Trough of Disappointment?" It’s too complicated for the rest of us. After all, Google, UPS and Facebook are tough acts to follow. They’re enterprises with high degrees of software and analytics acumen, and their successes are the result of years (sometimes decades) of sustained investment. This expertise has been critical because of the relative immaturity of big data tools combined with the need for well-organized data. Deploying and maintaining big data platforms such as Hadoop is complicated, and in the end, Hadoop is very far removed from the actual business problems retailers and vendors seek to solve. (If you don’t know what Hadoop is, you’ve made my point.)