Skip to content

Futures in Methods

AI-aided content analysis: opportunities, challenges, and strategies for validating results

JungHwan Yang
University of Illinois Urbana-Champaign

The integration of Artificial Intelligence (AI), particularly Large Language Models (LLMs) like ChatGPT, into content analysis presents both opportunities and challenges. Traditionally, content analysis is a labor-intensive, slow, and costly process reliant on human coding to systematically extract information from texts and multimodal media. The advent of AI offers potential to enhance efficiency, accuracy, and reliability, but also raises critical considerations regarding its role in the analytical process. This working group will explore the nuanced integration of AI in content analysis, focusing on the following key questions: 1) Should AI be viewed merely as a tool or as an autonomous agent within the analytical process? 2) How can we test intercoder reliability when one of the coders is an AI? 3) What roles should human coders and AI play when they collaborate? We will examine practical applications, such as using AI to develop codebooks, classify documents, and extract explicit and implicit data from texts.