A recent study by the Tow Center for Digital Journalism at Columbia University highlights significant concerns about how ChatGPT handles citations for publishers’ content. Despite licensing agreements between publishers and OpenAI, ChatGPT’s tendency to fabricate or misrepresent information underscores critical risks for journalistic integrity, transparency, and publisher reputation.
Key Findings of the Study
The study evaluated ChatGPT’s ability to accurately attribute 200 quotes from 20 publishers, including major outlets like The New York Times, The Washington Post, and The Financial Times. The findings revealed widespread inaccuracies:
- Unreliable Citations:
- Only a small number of citations were entirely correct, providing accurate publisher names, dates, and URLs.
- In 153 out of 200 cases, ChatGPT delivered partially or entirely incorrect citations, with fabricated sources or plagiarized content.
- Confabulated Responses:
- When unable to identify a source, ChatGPT rarely admitted its limitations, opting instead to generate inaccurate citations. It only acknowledged its inability to provide an accurate response in seven instances.
- Decontextualized Journalism:
- ChatGPT often treated journalistic work as isolated content, disregarding its original production context. This approach led to citations that failed to reflect the authenticity of the original sources.
Implications for Publishers
Risk of Misrepresentation
The study found ChatGPT occasionally attributed content to plagiarized sources rather than the original publishers. For example, the chatbot cited a website that had copied The New York Times content without attribution, raising ethical questions about OpenAI’s ability to filter and validate its data sources.
Impact on Publishers with Licensing Deals
Even publishers with licensing agreements, such as The Financial Times, experienced inaccurate citations, undermining the presumed benefits of granting OpenAI access to their content.
Variation in Responses
ChatGPT’s inconsistency in producing the same answers for repeated queries further complicates its reliability. This variation is especially problematic when accuracy is paramount, such as in attribution and sourcing.
Reputation and Traffic Concerns
Publishers allowing OpenAI’s crawlers may hope for increased visibility in ChatGPT’s summaries and citations. However, inaccuracies in these citations could harm their reputation or direct readers to competitors’ content instead.
Challenges of Blocking OpenAI Crawlers
Some publishers, like The New York Times, have chosen to block OpenAI’s crawlers to protect their intellectual property. However, the study revealed that even in these cases, ChatGPT still generated inaccurate citations for their content, raising questions about OpenAI’s broader data handling practices.
OpenAI’s Response
OpenAI defended its practices, claiming the study conducted an “atypical test” of its product. It emphasized ongoing efforts to improve citation accuracy through partnerships and tools like the OAI-SearchBot, which respects publisher preferences defined in their robots.txt files.
Conclusion
The Tow Center study underscores the challenges publishers face in managing their content in the era of generative AI. OpenAI’s citation inaccuracies and reliance on decontextualized content raise questions about the ethical and practical implications of these tools. For now, publishers have limited control over how their content is used and represented by ChatGPT, leaving them vulnerable to reputational and commercial risks.