Keywords

Qualitative Research, GenAI, Research Methods, Thematic Analysis, Large Language Models

Abstract

The integration of technology and technological advancements in qualitative research has transformed the research process over the past several decades. Various tools, applications, and devices have expanded opportunities for multimodal field sites and created possibilities for online observations, focus groups, and interviews. As a result of new technologies and innovations, methods of qualitative data collection and analysis have transformed, and methodological approaches have evolved. As new technologies emerge, it is important to understand their impacts on the research process. With the increasing accessibility of large language models, researchers and institutions must carefully assess the implications for qualitative analysis. In this qualitative methodological dissertation I explore the possibilities and limitations of utilizing generative artificial intelligence (GenAI) to analyze text data. I demonstrate the knowledge required to approach thematic analysis using Copilot Pro in Word (Copilot). I discuss the methodological decisions I encountered while exploring Copilot's features for qualitative analysis and explain my reasoning for choosing to utilize Copilot for this study. Further, I compare the results from a traditionally human-conducted thematic analysis to the codes, categories, and themes generated by Copilot. In the process, I developed the criteria by which I compared the outcomes while also evaluating Copilot’s output against the American Educational Research Association’s Standards for Reporting on Empirical Social Science Research. The findings provide insights into opportunities and limitations in leveraging GenAI tools in the qualitative research process. Through this methodological study I demonstrate Copilot’s capabilities generating codes inductively, grouping codes into categories, and developing themes. I argue that researchers need to balance the capabilities of the tool with an understanding of its limitations, particularly concerning time and efficiency, transparency regarding its analytic process, the reliability of its responses, the presentation of its outcomes, and the level of support provided to substantiate its claims.

Completion Date

2024

Semester

Summer

Committee Chair

Skukauskaitė, Audra

Degree

Doctor of Philosophy (Ph.D.)

College

College of Community Innovation and Education

Department

Learning Sciences and Educational Research

Degree Program

Methodology, Measurement, and Analysis

Format

application/pdf

Identifier

DP0028881

URL

https://stars.library.ucf.edu/cgi/viewcontent.cgi?article=1427&context=etd2023

Language

English

Rights

In copyright

Release Date

2-15-2028

Length of Campus-only Access

3 years

Access Status

Doctoral Dissertation (Campus-only Access)

Campus Location

Orlando (Main) Campus

Accessibility Status

Meets minimum standards for ETDs/HUTs

Download

Restricted to the UCF community until 2-15-2028; it will then be open access.

Share

COinS