Intro

A long time ago (September 2011) I was working as a consumer insights director, reflecting on trends I was seeing in what used to be called “market research.” Or “marketing research.” Or “consumer research.” Or whatever. In short: surveys were spreading like cancer as low-cost software made it possible for every PTA, brand manager, IT department, high school yearbook committee, and amateurs of every sort to release the SurveyMonkey. The Monkeys reproduced like rabbits, and inboxes were increasingly cluttered (oh, and how much worse things are now).

This is not a rant about amateur involvement in collecting feedback; any feedback is better than no feedback! But it is similar to what happens if a person new to, say, political polling, hangs a shingle and starts publishing election predictions based on no knowledge or experience of the intricacies of measuring voter intentions. Web and marketing tech has made it possible for small teams to create a lot of noise, and it becomes difficult to distinguish between a poll from Pew and a Twitter survey from Breitbart masquerading as a scientific exercise.

Anywho – In short:

  • Response rates were plummeting by 10x.
  • What data we could collect was increasingly filled with smartphone-tapped obscenities.
  • Nobody not directly on the front line had much of a clue about how bad it was getting.

And so, frustrated, I wrote up 8 Things. (I know, every good listicle has 10 things, but I was short on free time.) This is a retrospective on those 8 Things, with a little bit of (1) self-assessment and (2) prognostication on where those trends are pointing at the end of 2018.

1. Get out of the survey business.

First, some quick history: up until the mid-90’s, there were no online surveys. We all had land lines and caller ID and physical mailboxes and went to the malls and were politely intercepted by interviewers from centralized focus group facilities. That was only 20 years ago! Once the Internet took over enough of our lives that we felt comfortable managing our money in virtual banks without humans, we started pushing surveys into email inboxes. Those were very profitable years for research consultants as there’s a big cost difference between sending out 10,000 emails and calling 10,000 phone numbers.

Hypothesis: By the end of the 00’s, cracks were forming. It had become much more affordable to in-house the logistics of running research studies, and the relative simplicity of the tools (compared to what was involved in phone banking and mass mailers) made it really appealing to marketing budgets to turn a $100k vendor project into a $100k employee that can run 10 projects a year. It was classic disintermediation, and time (I argued) to move on.

7 Years Later: A-: Obviously, you can still hire an agency to run your surveys. I know – I ended up running a research agency for about 5 years. But while the beginning of that cycle had us doing basic logistics of survey fieldwork and data analysis, within a couple of years it was basically impossible to sell any projects that stopped at “data collection.” Clients needed either unique capabilities (e.g., the MaxDiff gamification engine we built and is still used to this day) or deeper thinking and interpretation of what it all means. I can’t give this a perfect grade as it’s still possible to make a living doing this, but it’s really challenging unless it’s under the umbrella of a top-tier agency brand (Nielsen, Ipsos, etc.) or bargain basement pricing.

Looking Forward: Well, it ain’t gonna get better. We’re pretty immunized against unsolicited requests for feedback; there’s a profit motive assumption for every survey and we’re increasingly aware that our data (and time) have real-world value. I wouldn’t be surprised if 10 years from now, the notion of general market or public policy surveys via online data collection has faded completely and been replaced with data-scraping / modeling techniques based on measured behaviors.

2. Find a different better way besides surveys to collect data.

Hypothesis: Because surveys suck so much, if you want to understand people’s motivations and wants you’re going to need to find another way of gathering that info. Part of this trend was a simple response to our shift to mobile devices as our primary gateway to the Internet; taking a 30-minute survey on a phone is pretty damned miserable.

7 Years Later: A+: Man, has there been an explosion of new user feedback and data collection platforms! Pollfish (2013), Survata (2012), YouEye (2011, acquired by UserZoom in 2016), Affinio (2015), Lucid (2010), Gutcheck (2010). These are just from memory. Qualtrics sold to SAP in an $8 BILLION cash transaction, driven in part by their aggressive push beyond just being a survey engine and expanding into a more holistic set of integrated feedback technologies.

Looking Forward: We seem to be entering an interesting time, between GDPR and growing global sensitivities around data privacy. There’s a new awareness of how our data is used (or weaponized); with Facebook/Instagram pulling back on how much data they share with 3rd party partners, the groups that want our data must now invest effort in forming a direct relationship with us. That means “social media analytics” tools that vacuum up social firehoses will be facing challenges from technologies that directly integrate into the user experience…which is good for things like 1-question polling overlays and more explict data collection mechanisms. So I expect a more explicit quid pro quo in the organization-to-human data exchange.

3. Get out of the syndicated report business.

Hypothesis: With the exception of large syndicated datasets with an intrinsic reputation (e.g., Nielsen data) or well-established industry analysts (e.g., IDC, Gartner), most syndicated reports don’t contribute anything to an organization making a decision. At best, they support (via cherry-picked anecdotes) an existing or proposed decision.

7 Years Later: B: There are plenty of industry-specific niche players that do quite well here, e.g. Newzoo who recently was majority-acquired by Advance and Superdata, who was acquired by Nielsen. Through old-fashioned hustle and perseverance, these are two companies that became industry-known brands by showing up and doing the unglamorous but necessary work of talking about what’s going on in the data of an industry. But they succeeded in large part because it was a blue ocean that had been surrendered by NPD, who previously held that role but weren’t able to get their act together and start measuring digital. But I acknowledge you can build a solid $50M business doing this.

Looking Forward: I’ll confess to not being terribly excited about this space, either as a consumer or practitioner. But at the same time, whatever industry you’re currently occupying you can probably think of a handful of operators who seem to have the mindspace cornered for “they’ve got industry data.” I’m softening my view on syndicated to one of passive acceptance: whenever a new market forms, there will be an information vacuum and one or several parties will race to fill that void. (Quick tip: there’s gonna be a lot of money made on marijuana data as legalization expands.)

4. Become a data integration super-ninja.

Hypothesis: As research and data analysis shifts from external (agency) to internal (DIY), there’s a large skills gap when it comes to making sense of that data and integrating it with other relevant data.

7 Years Later: A: Today, “data science” and “data engineering” are two of the hottest employment sectors in tech. Why? Because we have shitloads of data and while there are throngs of young grads who are excited about building mobile apps and learning the latest front-end UI frameworks, data architecture and engineering is…well, it’s complicated and tedious. “Big Data” was an earlier effort to describe the complexity, but it’s so much bigger than most people realize. Some people have so much data, Amazon will drive a semi truck to their facility so they can plug in network cables to speed up the data transfer into the cloud. And all those fancy things you hear about machine learning and AI depend on integrating and processing large amounts of data from multiple data feeds and making sense of it.

Looking Forward: God, this is going to get so much bigger. 10x bigger. The infrastructure we operate on (Amazon, Google, Microsoft, whomever) will get bigger too, and will continue to expand in its domain-specific expertise, because there simply aren’t going to be enough data engineers to go around. Hold onto your data engineers like they’re made of gold. (Data analysts are going to have a more difficult time, unfortunately. Tools like Periscope make it really easy to build dashboards. My advice to a data analyst: learn SQL and Python.)

5. Sell impact, not methodology.

Hypothesis: Methodological rigor only matters to other PhDs and statisticians trained in the nuances of methodology. For everybody else, own your voice as the expert and translate the numbers into impact. In short: there is no methodologist on the buyer’s side of the table 99% of the time.

7 Years Later: B+/A-: It goes a bit too far to say that methodology doesn’t matter…because it does, for two reasons: (1) every once in a while, somebody will ask you to back up your claims; and (2) bad methodology will eventually be outperformed in the market as bad methods make bad recommendations. I still maintain, however, that the method matters much less than the presentation of the method. My obvious bias here is that I’m not a PhD, and so the world of academic research (and the debates that go on amongst that incredibly smart community) is not a world I’m qualified to describe. But it’s a rare polymath who can both speak stats and sell ice to eskimos, as the saying goes.

Looking Forward: I actually think methods and math are going to shrink in relevance even more over the coming decade. Commercially, the practice of predicting things has taken a severe beating between Brexit, GDPR, Trump, and various other highly visible forecasting failures. We’ve lost the de facto trust of our audience; instead of “trust, but verify” we are now in an era of “multiple truths.” Being effective at communicating context and persuading clients and other audiences that your measured assessment of the truth is better than alternate narratives is, unfortunately, more about salesmanship and marketing than methods.

6. Build or buy technology-based scalability.

Hypothesis: Organizations without digitally scalable technology (and accompanying business models) will lose out to those that do.

7 Years Later: A: Full service custom research is still an enormous, multi-billion dollar business, but it has plateaued. The past several years could be described as “early adopter” years for a wide range of alternative solutions that are better, faster, and cheaper than hiring a consulting team and keep all of the knowledge and skill within the client’s org. All of the innovation has emerged on the technology side of the equation: automation, scalable qualitative, data marketplaces, mobile-savvy data collection methods, DIY/self-service, automated analytics, etc. And those “smaller” players aren’t that small any more (e.g., ZappiStore).

Looking Forward: This trend will march along, faster and faster with each passing year, until eventually there will only be a handful of extremely large players on the full service agency side (think of how the Big 8 firms are now the Big 4) and a rich portfolio of tech-driven solution providers (think of the Internet transition from 1995 to 2005). Technology-based upstarts have already been nibbling away at this work.

7. Recruit technologists.

Hypothesis: If you believe #6 above – that you need technology in your insights business to thrive – then you need to staff accordingly.

7 Years Later: A: I don’t have enough line-of-sight into enough research and insights businesses to evaluate whether this change has begun. I would say, however, that companies you normally think of as technology companies (e.g., Amazon, Google, Facebook, Twitter, Microsoft) have all made significant investments into data analytics and consumer insights solutions. It may turn out to be a lot easier for tech companies to bring in research industry expertise than for research companies to fully understand how to leverage technology.

Looking Forward: In hindsight, this was a bit too obvious. The need for tech-savvy architects and product managers and designers rings true for every industry, particularly in data-centric ones like insights and analytics. The challenging aspect is finding talented people with both skills and domain knowledge, but in this particular case I think the skillset is the more difficult one to cultivate. Look to see agencies and research tech companies bringing people over from adjacent verticals.

8. Embrace multimodal interaction.

Hypothesis: No single channel of information or observation can tell an adequate story.

7 Years Later: B: I feel there is still far to go with multimodal. Data integration solutions exist within one broad sphere, e.g. combining survey feedback with social media data mining, but finding threads that connect between social and OTA and streaming and IOT and geotargeting is difficult on many levels. It’s difficult to integrate, it’s difficult to analyze and interpret, and it’s difficult to not feel like Creepy McCreeperson when you’re saturating yourself in the entirety of a person’s life.

Looking Forward: This will continue to get creepier.

TL;DR JSA. Where’s my infographic?

Original Hypothesis 7 Years Later Looking Forward
The basic tasks of fielding surveys have been automated to the point of agency disintermediation. A- Gonna get worse. Surveys will never die, but will fade away from their current overused condition.
Other non-survey forms of data collection will emerge and dominate. A This is already happening, and it’s beautiful. Look for this functionality to move closer to the core of product design, with brands wanting to integrate data collection functionality directly into their product experience.
Syndicated reports are a stagnant and fading business model. B Syndicated data at the right price is still valuable. Every industry has its own cartel of industry data brokers, but there are better ways to live than trying to break into a cartel.
Integrating data between multiple sources is still hard, and a lot of value is locked up here. A This will be a valuable skill for quite some time, and will be very fast-moving.
There’s more value in solving problems than selling tools. B+/A- Yes, but it’s non-trivial to evolve your workforce from selling tools and processes to selling consultative advice. After watching research agencies attempt this pivot for a decade, I think those who can make the transition are already on board.
You won’t remain competitive without some unique, scalable tech in your portfolio. A I’d go even further and propose that if tech isn’t at the core of your business model, you’re not going to win.
You need technologists on your roster to build tech. Hire some. A This is a difficult future for industries that don’t “sound” like tech companies. Demand for data-savvy engineers / designers / product people at FAANG is so extreme that it will be hard to compete in those compensation bidding wars.
Embrace multimodal. B Connecting digital and real-world data and behaviors is the holy grail of consumer insights, but serious societal friction about “who owns my data” and “what can you do with my data” is going to make this space chaotic and messy. I do believe in the end, however, the oligarchs will win and the data will be connected.

Where Next?

Overall, I think the broad trends held true:

  • Technology became the core of the insights industry’s growth engine
  • Technology also became the primary competitor and cannibalizer of decades-old, stable revenue streams
  • Established research companies have treaded water fine, grown through acquisition of tech, but not gone through a cultural realignment
  • The data quality and project feasibility concerns facing survey research have metastasized

As before, we still throw around a lot of buzzwords and trends that we think are going to impact the future of consumer research. A decade ago, it was bullshit like mobile → mobile compatible → mobile friendly → mobile first… but we’re still regularly distributing 30-minute surveys and expecting the 80% of people who answer surveys from their Androids and iPhones to put up with that. Studies still flat-out lie to their customers about the expectations of their survey experience, undervalue their time, or restrict survey qualification to desktop devices (?!?).

The current buzzwords are machine learning, AI, and blockchain; I’m personally most enthusiastic about machine learning in an abstract sense but that’s because ML is basically quantitative modeling at scale. In my current full-time role I spend quite a bit of time thinking about machine learning methods and how to apply them to my current domain, and also what’s involved in the transition from traditional analytics pipelines and techniques. It’s a non-trival change. For insights services, it’s as big a change in thinking as the shift to mobile devices so I’m pretty cynical that established players are going to master AI over the next decade.

That work, as before, will fall to the upstarts.