A new study investigates whether AI can serve as an automated assistant in creative tasks, yielding mixed results. The study, conducted by researchers Anil Doshi and Oliver Hauser at University College London and the University of Exeter, respectively, found that AI helped less naturally creative individuals write more original short stories, but dampened the overall creativity of the group. This trade-off may become more common as AI tools become more prevalent in creative fields.
Published in Science Advances, the study highlights the dual nature of AI’s impact on creativity. Hauser told TechCrunch,
Our study represents an early view on a very big question on how large language models and generative AI more generally will affect human activities, including creativity. While there is huge potential (and, no doubt, huge hype) for this technology to have big impacts in media and creativity more generally, it will be important that AI is actually being evaluated rigorously — rather than just implemented widely, under the assumption that it will have positive outcomes.”
The experiment involved hundreds of participants writing very short stories, around eight sentences, on any topic suitable for a broad audience. One group wrote independently, a second group could consult GPT-4 for a single story idea, and a third group could access up to five AI-generated story starters. The stories were then evaluated by both their writers and an independent group unaware of the AI assistance. They rated the stories on novelty, usefulness (i.e., likelihood of publishing), and emotional enjoyment.
Prior to writing, participants completed a word-production task to approximate creativity, a complex concept that cannot be directly measured. Hauser noted, “Capturing something so rich and complex as creativity with any measure seems fraught with complications. There is, however, a rich set of research around human creativity and there is a live debate about how best to capture the idea of creativity in a measure.” Their approach, widely used and documented in academia, helped validate the findings.
The study revealed that participants with lower creativity metrics benefited the most from AI assistance. Stories by less creative individuals who wrote independently were rated lower in quality, enjoyability, and novelty. However, those who used AI-generated ideas scored higher on all metrics, with better results from five suggestions compared to one. This suggests AI can genuinely improve the work of those struggling with creativity.
Conversely, highly creative participants saw little to no benefit from AI assistance, with some even receiving slightly lower ratings. The data indicates that creative individuals produced their best work independently, without AI intervention. This finding suggests that AI may not enhance creativity for those who already possess high creative abilities.
Beyond subjective evaluations, the researchers used OpenAI’s embeddings API to analyze the similarity of stories within each category (human-only, one AI option, or five AI options). They found that access to generative AI led to more similar and less varied stories, with a 9% to 10% increase in similarity. While this does not indicate identical stories, it raises concerns about AI’s potential to homogenize creative output.
This finding warrants caution, as echoed in the researchers’ conclusions. It reflects fears in visual art and web content that AI might perpetuate a cycle of blandness if it continually trains on its own output. As generative AI becomes more integrated into various media, studies like this provide a counterbalance to claims of limitless creativity and new eras of AI-generated content.
Hauser and Doshi acknowledge the preliminary nature of their work, emphasizing that the field is in its infancy. “There are a number of paths that we expect future research to pick up on. For instance, implementation of generative AI ‘in the wild’ will look very different than our controlled setting,” Hauser wrote. “Ideally, our study helps guide both the technology and how we interact with it to ensure continued diversity of creative ideas, whether it is in writing, or art, or music.”
