ChatGPT and generative AI are shaking up the world of information. In this context, is there still a point in conducting market research? Will ChatGPT replace the work of a market research firm in the future? This question concerns me greatly, and I wanted to answer it objectively and honestly.
Contact IntoTheMinds to carry out your market research
If you only have 30 seconds
- Generative AIs such as ChatGPT provide amazing results for market research based on secondary data (desk research)
- A closer look at the answers provided by ChatGPT shows that the elements provided are not necessarily relevant to the analysis or are even wrong.
- The data available is not necessarily the most relevant, and the sources are also very limited
- For complex queries, Google often points to more specific pages. ChatGPT, on the other hand, states
- Working with generative AI for desk research is an interesting starting point but not an end.
- Market research is mainly carried out based on primary data. ChatGPT or other generative AI can’t be of any help.
Desk research: ChatGPT has some advantages
Desk research is the collection of secondary data. It is about collecting and analyzing data already available and provided by others.
I see 3 main challenges in desk research:
- finding relevant data
- synthesize them in an objective way
- Putting it into the context of the market research
ChatGPT allows you to quickly get a first framework and very global ideas of a market. Here is the result for the French market of electric bikes.
At first sight, it is quite impressive. There are figures, sources, contextualization, and organization of the information. This is what I find most impressive: the ability of this generative AI to structure the information and thus simplify its assimilation.
However, market research via desk research must continue beyond such a summary text, and a few statistics end to end. Understanding a market and predicting its evolution requires interpretation skills outside the scope of ChatGPT.
I am also concerned about the sources used. This summary uses only two of them, which need to be revised.
Finally, some figures leave me doubtful. The French state aid for purchasing electric bikes differs from the one announced in the article. So, we have to be extremely careful.
Ethan Mollick posted several screenshots of similar tests with the Bing conversational agent on Twitter. Here again, it is the structure of the information that is impressive. If Ethan Mollick sees the end of consultants, we must put it in perspective. Which (real) consultants do SWOT analyses, and how many rely on secondary data? (See the next paragraph for an in-depth discussion on the subject).
Advantages and disadvantages of ChatGPT for market research
Here is a summary of the advantages and disadvantages of desk research with ChatGPT
|Structure of the first ideas||Very superficial ideas that need to be developed|
|Potentially interesting perspectives||Perspectives and figures far from exhaustive|
|Data put in context||Veracity of some figures to be questioned|
In conclusion, ChatGPT can help you to get a first general idea of a market, but you will have to:
- complete it with other sources and other angles of analysis
- check the accuracy of the data
- interpret the data in the context of your market research
Primary data: ChatGPT is knocked out
ChatGPT can only retrieve data that it has already “ingested.” In other words, using ChatGPT for market research is limited to public data. It is, therefore, what we call secondary data. But market research without primary data is not research.
Primary data is collected on an ad hoc basis solely for your project. This data is unique and confidential. The market research we conduct for our customers has not been made public. The same is true for all firms. ChatGPT does not have access to this type of data. Yet it represents 95% of the added value of market research.
How to collect primary data?
Primary data can be collected mainly by 2 methods:
- qualitative (interviews, focus groups, …)
- quantitative (online surveys, telephone surveys)
Remember that without primary data, market research is worthless. Only primary data can allow you to provide precise answers to your questions. In this field, ChatGPT cannot do anything for you.
ChatGPT gives the illusion of analysis without it being one.
Data interpretation: ChatGPT can’t do anything
An inherent weakness of ChatGPT concerns the interpretation of data. I have already raised this point in another article. ChatGPT can only juxtapose elements, more or less true, in a standard structure. In doing so, ChatGPT gives the illusion of analysis without it being one.
The analysis (and the added value) only comes from pooling facts and their interpretation in a given context. However, this exercise involves elements such as:
- the analysis of weak, peripheral signals
- general conditions (e.g., macroeconomic) that influence the market
- primary data focused on the project itself
- the business expertise of the analyst
As an example, I submitted an analysis request based on public data. When data, even secondary data, is not readily available, ChatGPT makes it clear. You have to look for the information yourself.
Carrying out (real) market research is outside the reach of generative AI. Neither the collection of value-added data (primary data) nor the contextual analysis can be carried out by a machine. At this stage, chatGPT can only help students to carry out theoretical projects without financial stakes.
Human interaction with the analyst is as important as the data itself. Market research is not an end in itself. It’s a start. You must bring the data to life, illustrate it, and propose solutions. Fortunately, artificial intelligence will never be able to do this.
Posted in Innovation.