Big Data Is Only Half the Data Marketers Need

For marketers, truly valuable customer data comes in two forms: thick data and big data. Thick data is generated by ethnographers, anthropologists, and others adept at observing human behavior and its underlying motivations. Big data is generated by the millions of touchpoints companies have with customers. To date, thick data and big data have been promoted and employed by very different people. Thick data has been handled by companies grounded in the social sciences. Big data has been promoted by people with analytics degrees, often sitting in corporate IT functions. There has been very little dialogue between the two.

This is unfortunate. Combining the two approaches can solve many of the problems that each category of data faces on its own. Thick data’s strength comes from its ability to establish hypotheses about why people behave as they do. It cannot help answer questions of “how much,” only “why.” Big Data has the advantage of being largely unassailable because it is generated by the entire customer population rather than a smaller sample size. But it can only quantify human behavior, it cannot explain its motivations. That is to say, it cannot arrive at a “why.”

It’s only by combining the two forms of data that a complete picture emerges and real solutions to the strategic problems facing CMOs may be found. As companies start combining thick and big data, they will also stop relying on what has so far been a cornerstone of most customer insights programs—namely endless surveys and focus groups that purport to explain customers’ motivations and attitudes but in reality add very little strategic value.

Take the case of a large European supermarket chain that recently tried to arrest declining sales and eroding market share. The CMO could see all this in his company’s sales data, just as he could see that shoppers’ big weekend trips to the market—one of the key parts of his business—seemed to be disappearing. But he had no idea what was causing the change.

To understand what was going on, the CMO followed the traditional playbook: he launched a large survey. More than 6,000 shoppers in each market were asked more than 80 questions about everything from shopping decisions and price sensitivity to the importance of brands to occasions and emotions driving purchases.

The survey, however, didn’t surface any real insights. When asked, people noted that price was the most important factor, but 80% also said, “I always choose high quality over low quality, even though it will cost me more.” And 75% of the so-called foodies said they regularly shopped at discount stores. It was a common belief among the management team that they were losing customers to the discount stores, but if that was really the case, why would people say they would pay for quality?

Left even more uncertain than before the survey, the CMO decided to commission a study to come up with thicker data: he wanted insights from spending time with consumers in their homes and daily lives.

Over the course of two months, a team of social science researchers spent hours going shopping with customers, watching them as they shopped, planned, and made dinners with their families.

As the executives looked at the findings from the study, a major shift in consumers lives was apparent. Not only had their food habits changed, but people’s whole social lives were different. Stable family routines were dissolving, and predicting what next week would look like was increasingly difficult. One of the most telling pieces of data was the disappearance of the family meal on weekdays. Families simply were not eating together at the same time every day. Many families also now had three or four different diets to consider. The dinner table had started moonlighting as a work station, pushing the sit-down dinner into different rooms.

This fundamental shift had a severe impact on shopping behavior. People were shopping more than nine times a week on average; one respondent shopped three times a day. People were not loyal to specific supermarkets but chose the ones that fit their need for fast, convenient shopping. Exhausted from working all day, the last thing they wanted was to carefully consider different prices at different supermarkets.

The study also revealed that the traditional assumptions around price versus quality were superficial. People didn’t categorize supermarkets by discount or premium. Rather, they seemed to be guided by the mood and experience of the stores. Some stores projected a mood of efficiency. Others felt fresh and local, and others seemed practical and thrifty, offering good everyday value.

To meet consumer needs, the CMO realized, a supermarket had to deliver shopping experiences that were both convenient and distinctive—in other words, a mood.

To validate these insights, the marketing team cross-checked them against big data from its stores. They looked at the importance of convenience by correlating data on store location and shopping volume for individual stores. The data revealed a pattern: the most successful supermarket chains were located where the traffic was most dense. This was particularly true in suburban areas. They looked at the role different moods and experiences played in stores, by comparing sales and looking at store size and customer demographic data. The best-performing stores had a high degree of distinctness, calibrated to suit the demographics in the surrounding area. Again, the data revealed that the supermarket chain’s own stores were not set up for this reality.

The conclusion was clear: the experiences that the company’s stores offered were out of sync with the reality of the consumers. Instead of focusing on lowering prices, the supermarkets future strategy was built on a different idea: building distinctive shopping experiences that fit into customers’ fragmented lives.

As this example shows, CMOs need to familiarize themselves with the strengths and weaknesses of the two data types. The big data alarmed the CMO, prompting the exploration of why the numbers were changing. The thick data afforded the needed insights to understand what bigger shifts were behind the numbers, and provided the renewed take on what kind of business the retailer was in. This gave the CMO the direction he needed devise a strategy of how to get the retailers earnings back on track.

Armed with a robust strategic framework the CMO could now revisit the big data sets to quantify the findings of the qualitative studies—how many customers and purchases were we talking about? In which stores? This back and forth between what they knew was happening (big data) and why (thick data) was key to making a sound decision.

To start working successfully with both data types in concert, CMOs need to revisit their customer insights departments as well as establish close ties to CFOs. Many companies are well advised in weak data sources, but unskilled in obtaining and making sense of thick data. Similarly, many companies have no clear strategy for their big data collection, and they end up locking it into different silos or not cleaning it up enough for effective usage.

Melding big and thick data together isn’t easy. It requires changing practices, hiring new people, and allocating funds away from familiar ways of doing things. But once you’ve seen the power of real data, you’ll question the millions of dollars wasted on surveys and focus groups.