• Workshops
  • Rethinking Analysis
  • Community
  • Blog
  • About
Susanne Friese, August 3 2024

Ethical Aspects of AI in Qualitative Research - Part 2

Ethical and Responsible Use of AI for Qualitative Analysis

In this article, I discuss key considerations for using generative AI to analyse qualitative data in an ethical manner and I present an example. In Part 1 of this series, I outlined the ethical research criteria established by the Federal Ministry of Education, Science, and Research in Austria. For the context of data analysis, the following three criteria are most relevant:

Honesty and integrity: The presentation of research results must be sincere, transparent and free from any form of manipulation in order to strengthen credibility and trust in scientific research. To achieve this, the researcher needs to know how the research results came about.

Objectivity: Researchers are encouraged to conduct their work free of personal prejudices and to adopt a neutral and impartial attitude at all times. It is well known that subjectivity is a strength of qualitative research. What is important with regard to this criterion is the recognition and reflection of possible biases.

Careful and accurate documentation: It is essential that all research data is accurately recorded and conscientiously documented to ensure traceability and reproducibility of the research. In order to be accountable for this criterion, the researcher must know how research results were obtained.

When AI is Doing Everything for Us – Obtaining Results with a Click on a Button

The following two images created with DALL.E depict a human researcher collaborating with an AI assistant. These were the first results generated by DALL.E, which were different from what I had initially envisioned (as you'll learn below). However, these images seem to echo the common messages we encounter online:

AI is portrayed as doing all the work for us, with the researcher merely sitting on the sidelines, observing. It's quick, easy, and seemingly effortless. However, these promises often fall short in practice. In recent months, more and more articles are published where researchers show that generative AI cannot operate effectively on its own (Gamildien et al, 2023; Goyanes et al. 2024; Lee et al. 2023; Nguyen-Trung, 2024, Zhang et al. (2023). Human intervention, correction, and further instructions were necessary to achieve the desired results.

More importantly, by turning analysis over to AI important ethical codes of conduct are violated. It is unclear how results are obtained in autopilot mode, how bias is monitored, and whether the results are genuinely based on the data and not fabricated. Researchers often have to rely entirely on the outcomes generated by AI, with little control over the process.

Davidson et al. (2024), in their article on the ethics of using generative AI for qualitative data analysis, wrote:

"Automated qualitative coding can only examine the syntax, but it can't really capture the semantic and pragmatic aspects of the data. [...] An automated coding process could lead to a mundane and neutral analysis that fails to identify or disclose hidden aspects in the qualitative data. The output analysis then involves an incomplete and possibly superficial reading of the data. In addition, common (or neutral) chunking and coding methods could influence and limit our potential learning from data analysis."

Therefore, the question arises: How can qualitative data analysis be carried out to counteract these ethical issues?

My answer to this question consists of two components: 

(1) Use an interactive tool instead of expecting results at the click of a button. 

(2) Use a reflexive approach to analysis.

AI-Supported Qualitative Analysis as Interaction

Using an interactive approach allows you to collaborate with an AI assistant, directing the analysis rather than letting the AI take the lead. Instead of relying on simple buttons like “Summarize data,” “Extract themes,” or “Code the data,” you can enter prompts and initiate deeper exploration. This interaction enables you to reflect on the results, questioning and probing where necessary. If you suspect that an answer is biased or fabricated, you can highlight this concern to your AI assistant. You can request the assistant to reflect on its response and provide reasoning behind it. Additionally, you can challenge the AI to offer alternative perspectives. If you suspect that the AI has generated incorrect or hallucinated information, you can ask it to re-examine the data and provide supporting evidence.

Another important aspect is for the AI assistant to provide references, including links to the original source files. This transparency allows you to see exactly which parts of the data the AI's answers are based on, ensuring accountability and traceability in the analysis process. Below you see an example how this could look like. Thus, when evaluating an AI tool, this is a crucial feature to look for.


If you do all of this, then you are on a good path to adhering to the ethical guidelines listed above: Honesty and integrity; objectivity understood as recognition and reflection of possible biases, and careful and accurate documentation.

Below, I'll provide an example of how an interactive analysis with an AI assistant can look. However, to understand why I recommend starting with either a single case or a small subset of your data, you need to be aware of an important current technical limitation with LLMs.

A Current Technical Limitation of LLMs

Although there are many promises about what AI can do for us, there are also current limitations that are often not discussed. One such limitation is the so-called context window. This has been brilliantly explained by Jeff Lagana:

“Picture yourself engaged in a conversation with a friend. Initially, you discuss plans to have dinner together and agree to meet at 6pm. You then delve into your respective days, their family matters, and your weekend plans. As you're about to part ways, you ask again about the dinner plans. However, to your dismay, your friend has no recollection of ever discussing dinner or making any arrangements to meet. It can be frustrating to realize that someone has seemingly forgotten something that was previously discussed.

While this analogy is merely aggravating in real life, it highlights a fundamental limitation of today's large language models. This limitation becomes particularly significant when dealing with generative AI and processing extensive datasets.

Now, picture yourself asking an LLM to summarize a large amount of data. It eagerly provides you with the requested summary, which you subsequently present to the board without double checking. You later discover that it had omitted the first half of the data you provided; it essentially "forgot" about it. Your summary to the board was factually inaccurate.

Understanding the limitations of context is crucial to avoid these types of scenarios. An AI model will not explicitly notify you when its memory limits have been exceeded; instead, it will simply act as if the data was never received” (Lagana, October 2023, LinkedIn Newsletter).

-----

The context window is a limitation that prevents you from uploading all of your 20 interviews at once to ChatGPT or other AI chatbots. While some AI applications may allow you to upload more data, they often fail to inform you about the consequences of exceeding the context window. This can lead to superficial answers, leaving users puzzled. For example, a research team tried using ChatGPT to analyse responses to open-ended survey questions. Initially, they were pleased with the results, but upon closer inspection, they realized only 20% of the data had been analysed.

There are several strategies to manage the limitation of the context window in LLMs. The worst-case scenario is that the LLM only analyses data up to the context window limit, ignoring the rest. Another scenario involves the LLM analysing summaries of the documents, which often results in superficial answers with little depth. The preferred scenario is when an app is designed to perform a smart search as the first step, selecting only relevant parts of the documents for the LLM to analyse. This approach allows for the analysis of data segments from a larger set of documents.

A common context window for many of the popular LLMS at this moment is 128.000 tokens. 

A one-hour interview transcript typically contains about 9,000 to 10,000 words, assuming medium speaking speed, which averages around 150-170 words per minute.

Given that each word in English averages about 1.3 tokens (due to the way tokens are split by spaces, punctuation, and multi-character words in models like GPT), this would result in roughly 12,000 to 13,000 tokens for an hour-long interview. Thus, the most popular LLMs at the moment cannot process more than the full content of about 10 one-hour interviews at the same time.

When applications use smart search to filter and extract relevant portions of data before sending it to an LLM, they can effectively increase the amount of content that can be analysed. This process prioritizes key segments related to the user's query, thereby maximizing the use of the context window. While the exact number of one-hour interviews that can be processed depends on the specificity of the smart search and the density of relevant content, a rough estimate might be that such systems could manage content from around 30 to 40 one-hour interviews.

How to Handle the Context Window Limitation

My recommendation is to apply a case-based approach when analysing documents, interviews, or focus group data. By selecting a few cases that seem most relevant, most different, or best representing the dataset as a whole allows you to manage the data effectively within the context window limitations of LLMs.

Based on what you learn from these case analyses, you can develop more abstract concepts, explore potential relationships between them, and test these ideas against the rest of your data. This can be done either with the next set of cases within the context window limit or with the remaining data. The latter strategy should only be chosen if you know the LLM can process data beyond the context window without stopping. Additionally, ensure the quality of your analysis is sufficient if the LLM is basing its work on summaries of the data.

If you don’t know how the AI tool of your choice handles data processing limits, it's important to ask or push for a clear answer. Transparency in this area is often lacking, so gaining clarity on how the tool of your choice manages data beyond the context window is crucial for accurate analysis.

Conversational Analysis with AI: An Example

In the following section, I analyze five 20-minute interviews focusing on their content. The respondents were asked about their work-life balance. This example analysis concentrates on one key aspect: the separation of work and private life. Each interview varies in length between 8,000 and 9,000 characters. Thus, I am staying well within the context window limit.

For the example, I've chosen an inductive approach, starting from the data bottom-up rather than beginning with high-level concepts in a more deductive manner, as suggested by Morgan (2023). Morgan's QBA (query-based) approach recommends starting with theme extractions and then asking follow-up questions on each theme. 

As the full dialogue with the AI assistant spans over 20 pages, below I only show selected excerpts to give you an idea of how such an analysis can look. You can download the full dialogue [here]. To help you quickly scan my prompts and instructions, I have highlighted them.

Step 1

A recommended first step is to inform the AI assistant about your project. The more your AI assistant "knows" about your project, the more focused and relevant the answers will be.

Researcher: Hi, I am Susanne. I would like to analyse with you as my assistant interviews I conducted for a research project. The aim of the project was to explore the relationship between work-life balance and well-being. Ten people with various socio-demographic backgrounds from two major cities in the US were interviewed. Respondents were first asked about their daily routine before talking about the time they spend on work and other live activities. We were particularly interested in how happy or satisfied respondents are with the time they can currently devote to their personal lives and work.”

AI-Assistant: Thank you for providing the detailed project description and interview data, Researcher. How should we start the analysis?

Step 2

Begin by asking questions about your data. You could start by requesting your AI assistant to summarize each document. Some applications do this automatically, but this approach lets the AI assistant create its own narrative, resulting in summaries with different focuses each time. Instead, writing a prompt with clear instructions on what aspects to include in the summary will give you consistent and comparable summaries across interviews:

Researcher: I would like you to summarize each interview with a focus on the following questions: respondents' daily routines, their satisfaction with work-life balance, and well-being.


AI-Assistant: Summary of the interviews 

1. Paul

2. Claus

3. Mikel

4. Simone

5. Clara

“Let me know what specific aspect you would like to investigate next.


Researcher: Let's focus on work-life balance. Retrieve the information from the interviews regarding the following aspect: Separation of work and private life.

AI-Assistant: Work-life balance analysis in interviews


Researcher: It seems that for some it is important, but for others it is not so important. How important is this separation for the different respondents?


AI-Assistant: […]

Summary of the importance of the separation of work and private life

High importance: Claus, Mikel, Clara- These respondents consider the separation of work and private life to be crucial for their well-being and express a desire for clearer boundaries to make their private and professional lives more effective.

Medium meaning: Paul- Paul finds the separation of work and private life moderately important. While he's getting a handle on his current situation, he admits that a better balance would improve his happiness.

Minor importance: Claus- Claus does not give priority to the separation of work and private life and is content to integrate work into his free time, reflecting the low importance given to respecting clear boundaries.


Researcher: Let's talk more about some of the answers before moving on to the next step. For Claus, work seems to be a way of life. So, he doesn't want a clear difference. He also has children, how does that fit together in his way of thinking?


AI-Assistant: Claus' perspective on work and life reflects a unique mix where he sees no clear separation between the two. Here's a closer look at how that fits together, especially considering his family life. […] Claus'perspective on work-life integration reflects a high level of satisfaction with his work and a unique approach to merging work and personal life. He prioritizes family breakfasts to ensure daily connection, involves his wife in work events whenever possible, and focuses on family activities on weekends. Despite some regrets about missing out on family events, Claus finds fulfillment in his career, which he considers an integral part of his life. This blended approach allows him to maintain high levels of satisfaction in both work and family life, even if it sometimes means limited involvement in childcare and domestic tasks.

Do you want to know more about other respondents, or should we write a summary and interpretation based on this detailed analysis?

Researcher: Tell me more about Simone and her view on the separation of work and personal life.


AI-Assistant: […] Simone's perspective on the separation of work and personal life highlights the significant challenges she faces when it comes to maintaining the boundaries between her professional and personal life. Working from both home and the office offers some flexibility, but it also makes it difficult to separate work and family. Sharing childcare with her husband helps manage daily routines, but Simone often feels guilty about the limited time she can spend with her daughter. It recognises the importance of separating work and private life and expresses a desire for clearer boundaries, fewer working hours and more personal time. Simone'scexperience highlights the difficulties of achieving balanced work-life integration, especially for working parents with demanding jobs.

Do you want to know more details about other respondents?


Researcher: Tell me more about Clara, who is still focused on the separation of work and personal life.


AI-Assistant: […] Clara's perspective on the separation of work and private life underlines the great challenges she faces as a single mother and kindergarten teacher. Her work goes beyond the official opening hours and blurs the boundaries between professional and private life. The importance of separating work and private life is reflected in their desire for clearer boundaries and a more structured working day. Despite a supportive network, Clara feels the burden of managing childcare, household and professional responsibilities largely on her own. She has very little time for self-care and personal interests, highlighting the need for changes in her work structure to achieve better balance and improve her well-being.

Would you like to discuss more about other respondents or specific aspects of the data?


Researcher: Now, tell me more about Mikel.


AI-Assistant: […] Mikel's perspective on the separation of work and private life underlines his successful handling of the boundary between professional and private life. He appreciates and maintains a clear separation that helps him manage stress and enjoy his free time. His flexible 32-hour week allows him to pursue various activities and hobbies, which contributes to his overall satisfaction. Without childcare, Amadi can fully use his free time for personal interests and socializing. His approach to work-life balance, characterized by effective separation and satisfaction with his flexible schedule, shows how he achieves a fulfilling personal and professional life.

Would you like to know more details?


Researcher: Give me a quote from Mikel.


AI-Assistant: Certainly! Here is a relevant quote from Mikel on the separation of work and private life:

‘I'm pretty good at keeping work and personal life separate. It probably has something to do with personality, too. Over time, I learned—or more accurately, had to learn—to draw that line. When I leave the office, I switch off from work mode. I also make a point of keeping communication channels distinct; for instance, I don’t have my work Skype on my personal phone. Similarly, I don’t usually check work emails at home, unless there's something truly urgent. But overall, I think I manage the separation well, and it's something I find very important.’ This quote sums up Mikel's approach to maintaining a clear separation between work and personal life.


Researcher: Does he say more about why he had to learn it?


AI-Assistant: “Yes, Mikel explains why he had to learn to separate work and personal life: 

‘I spent some time working as a freelancer, which meant managing my own projects and deadlines. It was challenging, especially when important calls or tasks came up—like if someone called at 9 p.m., I’d feel I had to respond since the next assignment could depend on it. Eventually, I realized I needed to establish boundaries between work and personal life to avoid feeling constantly on edge.’

This quote explains that his experience as a freelancer, where work demands were unpredictable and often encroached on personal time, taught him the importance of maintaining a clear boundary between work and personal life in order to manage stress effectively.


Researcher: Did he deliberately stop working as a freelancer because of this?


AI-Assistant: Yes, Mikel made a conscious decision to stop his freelance work due to the challenges of separating work and personal life. Here is a relevant quote:

'I had to learn to create boundaries between work and personal life to reduce the constant stress. I'm definitely happier now. It’s not just about getting calls at all hours; it's also about the mental toll of being on the clock 24/7. Now, I can truly switch off at a certain time each day.’

This quote shows that the difficulty of maintaining a work-life balance while freelancing contributed to his decision to pursue a more structured work arrangement.

[End of follow-up questions]

-------

In this example, you can see that I engage with the AI assistant similarly to conducting an interview. My follow-up questions build on previous answers, allowing for a more detailed exploration of each respondent. I intentionally ask about each respondent one by one, rather than posing a single question across all respondents at once, because: a) It provides more detailed answers for each individual, and b) It allows me to ask more targeted follow-up questions.

Are there standard prompts for qualitative data analysis?

I am often asked whether I can provide a list of prompts to use for analysis. Above you have seen that each prompt included information about the data I was analysing, or was a follow-up question on an answer the AI-assistant has given. Therefore, I find it diffcult to give you a list of 10 prompts and tell you to just enter them one by one and this will be your analysis. 

However, I can provide some "procedural prompts" that can guide your analysis steps:

Procedural prompts

Procedural prompts provide a framework for guiding your analysis rather than offering specific wording. These prompts help you structure your interaction with the AI assistant, allowing you to customize the content based on your unique research needs. Here are the procedural prompts to implement a Conversational Analysis with AI:

Prompt 1: Provide your AI assistant with information about your project, including a short description, the aim of the project, research questions, and descriptions of the research participants.

Prompt 2: Request summaries but provide focus by listing specific aspects that should be included in the analysis.

Prompt 3: Ask a question about a topic of interest. Use the summaries for inspiration or base the first question on your research interest.

Prompt 4 [a, b,…x]: Continue the dialogue based on the information provided by your AI assistant. Stop asking follow-up questions once you have an exhaustive answer to your first question.

Review the answers and write a synthesis in your own words. Upload or copy your synthesis into the prompt field, then ask:

Prompt 5: Please check the accuracy of the data, the flow of reasoning, and ensure nothing is missing. Correct any spelling errors, improve sentence structure, and include a quote from each person if one is not already included.

Since writing the first version of this article, I have thought about this some more, and have developed some prompts with likely activities that you want an LLM to perform like extracting themes, comparing data sources, addressing gaps, clarifying ambiguities, etc. You can find the prompts on my community page:

Invite Link to join the Prompt Resources

Kai Dröge has recently added some reusable prompts to the latest version of QualCoder AI (3.6.1). Kai kindly gave me the permission to share the prompts with you. You can find them in full lenght in the appendix of this document and in the community space.

Step 3: Synthesize

Working with an AI assistant produces a significant amount of text for you to digest and reflect upon. The pages of text generated by the AI assistant are not yet the final results but the necessary ingredients for it. When teaching computer-assisted analysis in the past, I always emphasized that 50% of the analysis happens while you write. The same applies when using generative AI.

I recently came across some drawings that nicely emphasize this:

If you just let the AI assistant’s answers fly by, it is like water running through your hands. You need to catch them—save all answers about one topic as a report. Then read through all the answers again and think about them.

To understand the data and gain a clear perspective, the next step is to write it up in your own words, incorporating your reflections and interpretations. You can validate your synthesis with the help of your AI assistant. For instance, you can ask it to check for missing information, logical consistency, or supporting quote, if not already included. This might look something like this:

Researcher: Here is my synthesis for 'Separation of Work and Private Life.' Please check the accuracy of the data, the flow of reasoning, and ensure nothing is missing. Correct any spelling errors, improve sentence structure and include a quote from each person if one is not already included.

It seems that a clear separation between life and work contributes to well-being for most, but not all. David, for example, sees no reason to separate work and life. He enjoys his work very much and the thought of separating work and life would mean devaluing work in some way, as if only the "life" aspect was fun and therefore a separation was necessary. He says: "I just spend my time doing things that I enjoy, and at the moment: I like to work."

It's similar for Arne, because he also enjoys his job very much. In his case, however, the nature of that activity does not allow him to combine private and professional activities. Therefore, the frequent overtime shortens his free time, leaving little time for friends and leisure activities. In the future, he would like to become self-employed in order to combine work and life in a more similar way to David.

The situation changes when childcare comes into the picture. David has children, but his wife mainly takes care of the childcare and apart from the usual family routines such as breakfast in the morning, he can arrange his day as he wishes. Kazima shares childcare with her husband, and on the days, she works from home, she has to combine childcare with work. Therefore, she longs for more separation between work and life. For her, an optimal situation would be more flexibility when and where she wants to work and a four-day week so that there is enough time for work and her child, and she would not have to mix it up. In addition, she hopes, it would leave more time for herself. The same applies to the single mother Tatjana. She hopes for more structure in her working day and clear boundaries, in her case a fixed end to her working day, so that she can better plan her life outside of work. Similar to Kazima, she believes that working hours are too long and that the workweek should be reduced to 35 hours to reduce the stress of balancing work and life with a child.

Amadi seems to have what Kazumi and Tatjana still dream of: a four-day week. However, his perspective is still somewhat different. He clearly wants to separate work and life: "I also attach importance to practically separating my communication channels. Example: Skype: My Work-Skype is not installed on my mobile phone. Things like that." Due to his experiences as a freelancer, where it was difficult to separate work and life, he decided to work only a 30-hour week as an employee. He has no children, so he can use the time he is not working for himself and as he wishes. For him, the fact that he could not separate work and life led to stress, as Kazima and Tatjana also expressed. He attributes it to personality type, which could be an explanation – some people are more resistant to stress than others, but maybe it's also related to the type of work you do and the joy you get from it, so it doesn't feel like work at all."


AI-Assistant: Your analysis effectively captures the respondents' different perspectives on the separation of work and private life. Here are some improvements for accuracy, flow, and completeness:

1. Introduction: Start with a clear statement about the different importance of the separation of work and private life among the respondents.

2. Paul's perspective: Emphasize his lack of need for separation and enjoyment at work.

3. Claus' perspective: Highlight his desire for independence in order to better integrate work and life.

4. Childcare and work-life separation: Address the impact of childcare obligations on Simone and Clara more clearly.

5.Mikel's perspective: Clearly state his conscious decision to work 32 hours and maintain separation to relieve stress.

6. Conclusion: Summarize the different perspectives and note the factors that contribute to the need for separation.

Would you like me to incorporate these improvements into your text?


Researcher: Yes, please.


AI-Assistant: 

Separation of work and private life

It seems that a clear separation between life and work contributes to well-being for most respondents, but not for all. Paul, for example, sees no reason to separate work and life. He enjoys his work very much, and the thought of separating work from life would mean devaluing work in some way, as if only the "life" aspect was fun and therefore a separation was necessary. He says: "I just spend my time doing things that I enjoy, and at the moment: I like to work."

It's similar for Claus, as he also enjoys his job very much. In his case, however, the nature of his activity does not allow him to combine private and professional activities. Frequent overtime shortens his free time, leaving little time for friends and leisure activities. In the future, he would like to become self-employed in order to combine work and life more like Paul. He expresses dissatisfaction with the current separation, explaining: "I'm a little less satisfied with [the separation between work and leisure] at the moment because we have to work a lot of overtime."

The situation changes when childcare comes into the picture. Paul has children, but his wife mainly takes care of the childcare. Apart from regular family routines such as breakfast in the morning, he can arrange his day as he wishes. Simone, on the other hand, shares childcare with her husband. On the days when she works from home, she combines childcare with work. Therefore, she longs for more separation between work and life. An optimal situation for her would be more flexibility in when and where they work and a four-day week, so that there is enough time for both work and their child without having to mix the two. She also hopes that this will give her more time for herself. She says: "I'm generally satisfied—it’s a good setup from the employer’s perspective. However, I sometimes find it hard to make the most of it. I wish there was more flexibility, like being able to decide on short notice, “I’ll stay home tomorrow,” but that’s not always an option.”

The same applies to the single mother Clara. She hopes for more structure in her working day and clear boundaries, such as a fixed end to her working day, so that she can better plan her life outside of work. Similar to Simone, she believes that working hours are too long and that the workweek should be reduced to 30 hours to reduce the stress of balancing work and life with a child. Clara says: "I would prefer it if I officially came home from work, but unfortunately that is not always possible."

Paul seems to have what Simone and Clara dream of: a four-day week. However, his perspective is somewhat different. He clearly wants to separate work and life. He explains: "I also attach importance to practically separating my communication channels. Example: Zoom: My Work-Zoom is not installed on my mobile phone. Things like that."

Due to his experiences as a freelancer, where it was difficult to separate work and life, he decided to work only a 32-hour week as an employee. Without children, he can use his time outside of work for himself and as he pleases. For him, the fact that he could not separate work and life led to stress, as Simone and Clara expressed. He attributes it to personality type, which could be an explanation – some people are more resistant to stress than others. However, it can also refer to the type of work you do and the joy you get from it, so it doesn't feel like work at all.

Possible Step 3A: Elevating the Analysis to an Abstract Level

Depending on the goal of your study, you may want to move beyond a purely descriptive analysis to a more conceptual or theoretical level. This involves identifying broader patterns, themes, and relationships within your data, and developing abstract concepts that offer deeper insights into the subject matter. Given the above example, it appears that people’s approach to separating work and life can be classified into different types. So, I asked my AI-assistant about it:

Researcher: I realize that the above ways to approach the separation of work and life could be classified into different types. Please take a look, extract and describe the different types.


AI-Assistant: Here are the different types of work-life separation identified. […]

The different types of work-life separation highlight different strategies and desires among respondents:

1. Integrated approach: Some people, like Paul, prefer to combine work and life and find satisfaction in both without needing clear boundaries.

2. Limited integration: Others, like Mikel, strive to integrate work and life, but are limited by the nature of their work.

3. Necessary separation: Individuals like Simone need separation due to additional responsibilities such as childcare and are looking for more flexibility.

4. Strict separation to manage stress: People like Paul have learned the importance of strict separation to manage stress effectively.

5. Struggle for separation: Respondents like Clara struggle to maintain separation due to demanding working hours and responsibilities and would like to see more structured and shorter working hours.

These perspectives shed light on how various factors such as the type of workplace, personal responsibilities, and individual preferences influence the work-life separation approach.


I did not fully agree with this answer and wrote:

Researcher: I see Simone and Clara similarly; they both seek more separation to better manage their jobs, childcare, and self-care. There is some overlap with Mikel, particularly regarding reduced stress following a breakup. Paul and Claus share a different approach, as they love their work so much that they don't want strict separation since their job is their life. However, Claus lacks the flexibility that David has. Flexibility is a common theme among all five respondents, with each believing that more flexibility would lead to better balance. Based on this, create three types and highlight the overlaps.

Based on my question, the AI assistant listed three types and described them in more detail. From this output, I synthesized the following:

Type 1: Integrated work-life approach

People that belong to this type enjoy their work to the extent that they do not strive for a strict separation between work and life. They view their work as an integral part of their lives, bringing them satisfaction and fulfilment.

Type 2: Desire for separation to cope with stress and responsibility

People that belong to this type emphasize the need for a clear separation between work and life in order to manage stress and fulfil their responsibilities, especially when it comes to childcare and self-care.

Common denominator: Flexibility

The common denominator among all respondents is the realization that increased flexibility would enhance their ability to balance work and private life effectively. For some, this flexibility would help integrate the two spheres better, while for others, it would aid in keeping them separate.

-------

What you can see from the above is that you should always critically reflect on the answers given by your AI assistant. Never take for granted that they are correct. If you cannot remember the original sources well enough, make sure to validate the answers by going back to the source files and checking what your respondents actually said.

In case you're wondering, the entire analysis took me about one hour. Writing the synthesis took the most time, but it's well worth the effort, as it will be essential for your report.

Step 4

Repeat steps 2 and 3 for each research question you want to investigate with your AI assistant. If your data set is larger than what fits into the context window, continue exploring your first research question with the next set of data. You could start by asking the same initial question and build from there, or your entry point could be some abstract concepts you developed when working with the first set of data.

To give you an example, if we were to analyse the next set of five interviews in the work-life balance study, I could use the two types I have developed as entry points and ask the AI assistant to identify these types in the other interviews as well. I could then investigate the role of flexibility and explore further with the AI assistant whether other types beyond those identified in the first set of interviews are present in the data.

Summary

Above I have provided an example how a “Conversational Analysis with AI” can be conducted. The aim was to develop an approach that leverages the strengths of new technology rather than forcing it into processes that served us well in the past but are no longer a good fit. The proposed method aligns with the ethical criteria of conducting research mentioned in the introduction and helps avoid many typical AI-related issues such as bias and hallucination. However, avoiding these issues does not happen automatically. The researcher must be reflective and critically engage with the answers provided by the AI rather than accepting them without demur. Unlike automated processes, this method allows for such reflective engagement, making it possible to address potential issues as they arise. The type of analysis I hope to see develop in the future is depicted by the following two images:

If you are interested to learn the approach, I have written a custom GPT for it. When using this GPT, you will be guided by an AI assistant through the steps of a conversational analysis with AI. If you get stuck, simply ask the AI assistant for guidance. Check it out here. Have fun and let us know about your experience.


Appendix

Here are two prompts developed by Kai Dröger (QualCoder AI) that can be used either as additional instructions for an initial prompt or as follow-up prompts after having asked questions about a particular topic.

Prompt: Looking for the Unexpected

Take the given topic and description and briefly explain what empirical results would be commonly expected based on your own knowledge about the phenomenon in question. Then look at the actual empirical data given to you and pick out relevant aspects which are most surprising and unexpected given your previously outlined expectations.

Prompt: Analysing Differences

Use the given topic and description to analyse the given empirical data but looking especially at differences between cases or documents in the data. If you find relevant differences, point these out clearly. If you don't find real differences, that's also a valid result. Base your analysis firmly on the empirical data. Don't make any assumptions which are not backed up by the data.

References

Curtain, C. (2023). QualCoder (Version 3.5) [Computer software]. Updated by Kai Dröge (Version 3.6.1. July 2024).

Davison, R.M., Chughtai, H., Nielsen, P., Marabelli, M., Iannacci, F., van Offenbeek, M., Tarafdar, M., Trenz, M., Techatassanasoontorn, A.A., Díaz Andrade, A. and Panteli, N. (2024), The ethics of using generative AI for qualitative data analysis. Inf Syst J.

Gamieldien, Yasir and Case, Jennifer M. and Katz, Andrew, Advancing Qualitative Analysis: An Exploration of the Potential of Generative AI and NLP in Thematic Coding (June 21, 2023). Available at SSRN: https://ssrn.com/abstract=4487768 or http://dx.doi.org/10.2139/ssrn.4487768

Goyanes, M., Lopezosa, C., & Jordá, B. (2024, March 1). Thematic Analysis of Interview Data with ChatGPT: Designing and Testing a Reliable Research Protocol for Qualitative Research. https://doi.org/10.31235/osf.io/8mr2f

Lagana, J. (2023). Overcoming the Limitations of Context Size in Language Models: Strategies and Solutions. LinkedIn Newsletter.

Lee, V., Lubbe, P.S., Lay, P., Goh, H., & Valderas, P.J. (2023). Harnessing ChatGPT for Thematic Analysis: Are We Ready? Journal of Medical Internet Research, 26.

Nguyen-Trung, K. (2024, January 28). ChatGPT in Thematic Analysis: Can AI become a research assistant in qualitative research? https://doi.org/10.31219/osf.io/vefwc

Zhang, H., Wu, C., Xie, J., Lyu, Y., Cai, J., & Carroll, J.M. (2023). Redefining Qualitative Analysis in the AI Era: Utilizing ChatGPT for Efficient Thematic Analysis. ArXiv, abs/2309.10771.

Others:

Visuals on writing: https://www.milanicreative.com


*For the seasoned qualitative researcher: I am aware that there are many approaches to qualitative data analysis. What I am demonstrating here is a simple content analysis. Depending on how you prompt the AI assistant, other approaches can also be implemented. We are still at the beginning of figuring out how to best make use of generative AI for qualitative analysis and it will take some experimentation and experience to describe the use of a conversational approach to analysis for other methodologies like Grounded Theory.

Written by

Susanne Friese

Tags

Older Navigating Ethics in AI-Driven Qualitative Research - Part 1