It is inevitable that generative AI applications, such as ChatGPT, will make an impact on the customer insights industry. The promise of this technology is too great to not find its place in our toolboxes: from instantly summarizing unstructured text across many data sources to querying data for specific answers using simple human commands.
The customer insights industry is, however, a conservative collective. As researchers, we are trained to be skeptical about new methods and to question those too good to be true solutions that offer easy answers to complex problems. One could argue, we can be sometimes too skeptical, and our skepticism can sometimes lead us to resist ideas we should adopt to keep up with the changing world around us. A case in point: online surveys were once a hotly contested and polarizing concept that drove traditional pen-and-paper researchers to the brink of insanity… and ultimately to their extinction.
So, what is it with generative AI… or AI in general? Should we start thinking about the world post-online surveys? Will AI simply sit atop the data universe, and will the art of customer insights boil down to being able to write the best Chat-GPT query? Where do human researchers fit?
Computers will always be faster at tabulating data… including unstructured language data. And that is an exciting thing.
While instantly summarizing language data was difficult to imagine 20 years ago, even then the concept of letting faster and cheaper computers add up numbers for us was already making our lives easier. The only difference between Big Data of twenty years ago and AI models of today is that we can now quantify relationships between natural language concepts, not just numbers. To me, that is exciting.
When I started GroupSolver almost nine years ago, the act of finding quantified answers to important, open-ended questions has always depended on the thesis that NLP models will continue to evolve at a rapid pace. That assumption would allow us to capitalize on the novel idea that people answering the same question in their own words could be quantified in the same way we may rate our satisfaction with a meal we just had. What AI– specifically NLP models– allow us to do is to not treat individual answers as uniquely stated and unrelated data points (verbatim answers), but rather as n-dimensional vectors that can be placed in precisely defined relation to each other.
Generative AI is just taking this concept to the next level in a natural evolution sort of way. As anyone who has learned a foreign language knows, you can start to understand written (or spoken) text much faster than you can speak or write it fluently. To write in a foreign language takes learning beyond memorized vocabulary – grammar and style are critical skills to adopt in order for you to sound like a native speaker. What has happened to AI in the last 5 years is simply progressing from a student who is barely able to read to one who has learned to write grammatically correctly and is now working on comprehension.
We, as humans, communicate by telling each other stories, and that is no different in market research. Our customers hire us to explain to them the story of the customer data in a language that can be comprehended in a managerial, marketing, engineering, or other business context. We deploy research to different data sources – be it social listening, individual interviews, quantitative surveys or ethnography – we digest it and then we tell a story from that data. The cynic in me says that in fact the research briefs we receive from our clients are conceptually just very long, ChatGPT-like prompts.
But at the end of the day I am not a cynic, and I appreciate the time generative AI can save in summarizing massive amounts of information into research-ready Cliff notes. I believe that generative AI can be a useful tool to professionals who are trained to ask questions and look at answers with a healthy level of skepticism. We just need to understand its limitations and not become lazy to expect that it will find the a-ha moments we seek for us (but more on that later).
Fakes and bots will be harder to detect. And that scares me.
Bots that pollute online panels are reasonably simple these days, but it is a matter of time before the cost of generative AI drops significantly enough that any online survey – quant or qual – will be at risk of being compromised by an instant army of smart fakers. They will recognize trap questions, knowledge quizzes, and they will never straight-line responses because they will be able to answer all the questions reasonably well within the context of who they pretend to be.
It is very conceivable that a command to answer a survey as a “middle aged man, 2nd generation immigrant father of three, working as night shift worker at Amazon fulfillment facility, in a small town in the Midwest” will be enough of a prompt for a bot to answer the questions convincingly correctly… not too fast and not too slow.
Dealing with this problem in the market research industry is no different than what our whole society has been dealing with in deep fakes. Today, we don’t even have great tools to navigate challenges with simplistic click farms and bots. And I am sad to say that even with the tools we do have, we often choose not to use them. What worries me is that unless we (as a society, as market research industry, etc…) find a solution to reliably detect data generated by AI from what a real person says, online surveys – or any online research – may face the most dire consequence of all: lack of trust in data and the insights they support.
I believe this is a very hard problem to solve, and what may help us is actually our human imperfection. One of the things we have learned at GroupSolver – where our bread and butter is to deal with “dirty” survey data – is that humans are not good writers. They make grammar mistakes. They are lazy spellers and punctuation appears to be optional. You can almost say that a genuine respondent’s fingerprint on surveys wouldn’t even pass a 2nd grade English class.
This offers a glimmer of hope for being able to discern fake respondents from the genuine ones: currently, asking generative AI to complete the same survey with the same (or very similar) prompt will yield the same (or very similar) responses… and that creates a detectable pattern. Ultimately, finding bots, fakes, and click farms has always been and will always remain a cat-and-mouse game. In the new reality, AI-generated fakes may be taking this game to an entirely new level, and we simply will have to adapt. While until now a little bit of statistics and thorough data review could detect a good chunk of fakes, this will now be a job for real data science.
Discovering insights, the a-ha moments, and interesting findings is a uniquely human skill (at least for now). And that gives me hope.
Having led an innovative market research tech company that harnesses NLP to produce some really neat and unique data has taught me one lesson: data by itself doesn’t really move the needle. What matters is the story the data tells. It is all about the a-ha’s, unexpected discoveries, and hearing that unfiltered customer’s voice. Yes, a lot of online survey research is confirmatory. But the more we venture into deeper, qualitative questions – asking the why and “tell me more about that” – the more we are opening ourselves to surprises that we can’t get by clicking through a matrix question.
And here lies an opportunity for a human researcher to add value above and beyond what a generative AI can do with a data set. Generative AI can tell you a story, and it can even sound like your grandma telling it. But the AI technology always centers in the middle (for now), summarizing rather than discovering. Explaining rather than wondering. Generative AI does not write a story questioning why this is interesting or important. AI only confirms that the story stays within the parameters of the prompt and the underlying data set.
Stay curious. Ask why.
Curiosity is something we may get AI to do one day, but it is not there now. Asking the probing questions and knitting together stories that satisfy that curiosity remains the most distinctive skill we as human researchers have to offer. I am excited to continue to embrace our curiosity and with it the tools, such as AI, that give us the ability to dive deeper, faster and with more confidence than ever before.
As has been the case with every technological innovation, generative AI will find its place in market research, and we will learn and understand its power and its limitations. Those of us who don’t love it will learn to live with it. Personally, I am curious to see the path it will take and the discoveries we will make along the way.
Rasto Ivanic
Founder & CEO
GroupSolver
About GroupSolver:
GroupSolver is an intelligent research platform that empowers companies to remain curious and continuously ask and answer the question of, “Why?” to inform important business questions. Backed by our AI Open-End™ technology, we humanize the survey experience and empower researchers to crowd-source and analyze qualitative data at scale. GroupSolver serves as your thought partner to help make informed business decisions by quickly and effectively uncovering the story your data is telling.
