Sometimes you need a little bit of data and lots of right brain

At a recent market research conference, I learned about some insane qualitative datasets - insane in terms of the size or time given to collect and analyse the data. Collecting 25 interviews is a regular-sized sample for a qualitative study, but adding another four focus groups to validate the findings of the “in-depth” analysis of the interviews within a timeframe of four weeks – is insane.

If you are a market researcher, you probably think it is normal. This is just how qualitative market research is conducted in the industry. Your clients want results fast, and they are not willing to pay for a more extended analysis.

What your clients get is lots of qualitative data but NO analysis. Just going through some notes, reading through the transcripts and pulling out some quotes is not analysis. Of course, something can be learned from just conducting the interviews.

I always learn something in conversations with people or if I follow a webinar and take notes. I retain more information if I take notes and even more if I discuss those notes with someone afterwards. However, I don’t think that anyone would call this analysis, let alone an “in-depth analysis”.

Let’s take a look at what an in-depth analysis is. I asked chatGPT to define it:

“In-depth analysis of qualitative data refers to the systematic and detailed examination of non-numerical data to uncover patterns, themes, and meanings that are not immediately apparent.

The process of in-depth analysis of qualitative data typically involves several steps, including data coding, theme identification, and interpretation. In data coding, the researcher assigns labels or codes to different parts of the data that represent essential concepts or themes. In theme identification, the researcher looks for patterns and connections between different codes to identify broader themes that emerge from the data. Finally, in interpretation, the researcher uses their expertise and knowledge to make sense of the themes and draw conclusions about the meaning of the data.”

Given the timeframe that is allocated in market research, what can be extracted is what is immediately apparent. To uncover patterns, themes and meanings, you have to work with the data more systematically.

What I don’t understand is that so much money is spent on collecting qualitative data and very little money on actually learning something from the data that goes beyond the obvious.

The question any person who is commissioning a qualitative market research study should ask is: How much will it cost to implement the recommendations made based on such studies? How much can potentially be lost if it’s not working out?

Wouldn’t that be worth a bit more time and money spent on a proper analysis? A thematic analysis of twenty-five 1-hour interviews could be done in about four weeks if two people work on it. If you want an in-depth analysis, you must give the researchers more time.

If I listen to the jargon that is used in marketing research like IDI, my impression is that no one knows anymore what an actual in-depth interview is, and I doubt that many IDIs are indeed in-depth interviews. This is just the term used, and no one is questioning it. The same goes for ‘in-depth’ analysis. People think they do it because this is what their colleagues before them have done, and this is the jargon they grew up with in the marketing world.

Yes, I am calling out the elephant in the room, and I am not the only one who has noticed this. Here is a blog by Kevin Gray that already mentioned it a few years ago.

One of the first things you learn in methodology classes is that the choice of the method needs to be appropriate to the subject that you want to investigate. Was it necessary to conduct 25 interviews plus four focus groups to validate the findings? Maybe 10 to 12 interviews would have been enough and would have given more time for analysis.

A Radical Suggestion

There are techniques to analyse qualitative data that allow going beyond the surface and do not require lots of data. I demonstrate this below based on a data segment from the sample data set I used in my last article.

The data are comments on a blog about parenting. Below you see one of the comments and the coding that has been done by an AI tool – just to give you an idea of how “good” AI currently is at coding data. If you want to learn more, read my last newsletter.

Figure 1: The data segment coded by AI

Let’s take a closer look at what this person wrote. At the beginning of her post, she writes about motherhood; at the end, she uses the word ‘mom’. Motherhood can be defined as:

“… the state of being a mother. A person enters motherhood when they become a mother. This most commonly happens when their child is born, but it can also happen through adoption or by marrying or becoming a partner to someone with children. Motherhood is a gender-specific version of the term parenthood” (dictionary.com).

Motherhood, thus is objective, formal, and general. Using the words ‘motherhood’ and ‘mom’ points to an outside–inside perspective. The term 'motherhood' is distant and objective, whereas 'mom' is personal and emotional. In connection with the word ‘becoming’, it appears that the respondent describes a transitional process—the process of becoming a parent.

Let’s look at some other words she wrote: rough and forced. What does it mean? On the surface level, you might notice that she had a difficult time. What else can we learn if we take a closer look? What can you associate with the words rough and forced? For instance:

A rough surface, a rough sea, a rough person, a rough day, a rough breakup.Being forced to do something: doing something against your will or desire; abuse; manipulation, etc.

A rough surface, sea, or person is something outside one’s personal control. The consequences of encountering something rough, or being forced, are usually not pleasant. You might be able to do something about it, e.g. by wearing protective clothes, avoiding a person, or cheering yourself up after a rough day or break up (actions and strategies).

Circling back to the data segment, what can we learn from this that helps us better understand what the respondent wrote? Before I get to that, did you notice that she wrote: ‘I was only 25 and becoming a mom….” When you do such an analysis, pay attention to flag words like ‘always’, ‘never’, and ‘only’.

Given the analysis of the meaning of the words ‘rough’ and ‘forced’, in combination with the fact that she felt too young to become a mother, it is likely that the respondent did not plan her first child. It was out of her control; it just happened. Now she had to deal with the consequences in an adverse environment. 

Another word she uses is ‘glorifies’. The picture that emerges for me is that of a battle, a battle from which she emerges triumphant despite the difficulties the surrounding society presents.

Figure 2: Triumph after the battle

When going deeper into a text, you can ask the following questions: What can be associated with a word, what are the conditions for this to occur, the consequences, and the strategies and actions that can be taken? When doing this exercise, you want to think outside the box and leave the context. The aim is to develop abstract concepts that go beyond the data, and that can be applied more widely.

Within those few lines, the respondent wrote, there is an entire story, a story many people can relate to, a story that many can fill with their own content.

It doesn’t matter whether, for this woman, it is actually true or not that her first child wasn’t planned. It is more about the concepts that can be generated from this text passage, like role adaption and transition, planned or unplanned pregnancies, personal choice, preparedness, acceptance, etc.

What we have learned is not just about this one person. It is true for many others. There is no need to collect another 1000 similar statements to validate this experience.

Concepts are based on actual phenomena and are generalized ideas of something of meaning

 If you want to try out what other concepts you might come up with, take a closer look at the words ‘responsibility’ and ‘self-centeredness’. It is probably more fun if you do it in a group. Write down all associations, also wild ones. Note and describe the contexts. What are the conditions for these contexts? What actions occur? Which strategies are or can be used in these contexts, and what are the consequences? Then circle back and look at how this mental exercise can help you to understand the data.

If we were to develop a strategy for a product or service that facilitates the life of a parent, we need a bit more data than just this one paragraph. But usually, 10 to 12 interviews are sufficient to reach data saturation. See also the article by Dr. Graham Kenny. Not every paragraph needs to be subjected to such a detailed analysis. Once you go through some of the data in depth and feel that you have reached a saturation point, you use the rest of the data to validate the developed concepts.

Bonus Effect

Analysing data in this way may also solve another conundrum, which has been summarized by Roger Martin in a recent interview:

“The tricky thing about data is -- 100% of it is in the past. As of the time you analyse, it’s in the past. Otherwise, it wouldn’t exist yet, and we would have no data about the future. […] but you are determining your strategy for when? The past? No, that has already happened. So, you don’t need a strategy for that. You need a strategy for the future.”

Since the goal of the in-depth analysis outlined above is to develop abstract concepts with generalized meanings, these can also be used in the future. Would you have guessed that the data used here are from 2010? Topics like transitioning into the parenting role, personal growth, or personal choice are still relevant today, as they were over ten years ago.

In the interview, Roger Martin talked about statistics and data analytics, a method based on default thinking. This approach answers questions about what and how much; it is hypothesis-driven and based on hard, measurable evidence. And, as he points out, this type of research is on what is and has been.

A qualitative approach, as presented here, is exploratory and provides answers to why questions. If done correctly, this type of research is on what is to come.

Sometimes it can be simple – just use a little bit of data and lots of right brain.

P.S. The idea for the title comes from a the chapter "The Right Brain Strikes back" written by Michael Agar in the book "Using Computers in Qualitative Research" edited by Nigel G. Fielding and Raymond M. Lee (1991).

Get in touch

if you want to book a workshop for your team to learn how to conduct good qualitative interviews and how to do an in-depth analysis that will yield phenomenal insights:

Susanne.friese@qeludra.com