The Impact of Large Language Models on Knowledge Sharing: A 25% Decline in Stack Overflow Activity
In recent years, the rapid adoption of large language models (LLMs), such as OpenAI’s ChatGPT, has transformed how people interact with digital knowledge platforms. These advanced AI systems are powerful tools, capable of answering complex queries and providing solutions across a wide range of fields. However, as a new study published in PNAS Nexus reveals, the increasing reliance on LLMs may be diminishing human contributions to public knowledge platforms, with significant consequences for both the future of AI and the open web.
The Decline of Stack Overflow Activity
The research, led by R. Maria del Rio-Chanona and colleagues, highlights the impact of ChatGPT on Stack Overflow, one of the largest question-and-answer platforms for software development. Within six months of ChatGPT’s release, Stack Overflow saw a 25% decline in activity compared to its Russian and Chinese counterparts, where access to ChatGPT is limited. This decline was particularly pronounced in discussions around popular programming languages like Python and JavaScript, where ChatGPT’s capabilities are strongest.
The researchers employed a difference-in-differences model, comparing Stack Overflow’s activity with that of platforms less affected by ChatGPT, such as mathematics-focused forums and platforms in regions with restricted access to the tool. The results showed that ChatGPT users were less likely to post questions and answers on Stack Overflow, instead turning to the AI tool for solutions.
A Threat to Public Knowledge?
One of the central concerns raised by this study is the potential threat that LLMs pose to the open web and the generation of new public knowledge. As more users rely on AI for information, fewer contributions are made to platforms like Stack Overflow. This could create a feedback loop where LLMs, which are trained on vast amounts of human-generated data, eventually run out of high-quality, up-to-date data to learn from. As the authors point out, training future models on AI-generated content is akin to “making a photocopy of a photocopy,” potentially degrading the performance of future models.
Moreover, the study found no significant difference in the quality of posts after the release of ChatGPT. Both high-quality and low-quality posts were displaced by AI-generated content, suggesting that the decline in human contributions affects not only simple or duplicate questions but also valuable, expert-level discussions.
Implications for AI Development and Digital Public Goods
The reduction in human-generated content presents a challenge for the sustainability of the AI ecosystem. While LLMs like ChatGPT are valuable tools, their long-term development depends on a steady flow of new, high-quality training data. Without continued contributions to public platforms like Stack Overflow, the ability to train future models effectively could be compromised.
The study also raises broader concerns about the future of digital public goods. Platforms like Stack Overflow have long served as vital repositories of open knowledge, accessible to anyone with an internet connection. As AI increasingly takes over knowledge-sharing tasks, there is a risk that the web will become a more closed, privatized space, where interactions with AI systems are not added to the public pool of knowledge.
Moving Forward: Balancing AI and Human Contributions
As we navigate this new era of AI-driven knowledge sharing, it is crucial to consider how to balance the benefits of LLMs with the need to maintain healthy, open knowledge ecosystems. Encouraging continued human participation on platforms like Stack Overflow will be essential for both the future of public knowledge and the development of more effective AI models.
The findings of this study serve as a reminder that while AI has the potential to revolutionize how we access and create knowledge, it is the collaborative efforts of human contributors that have built the foundation of the digital world. Ensuring that these contributions continue will be key to preserving the value of both public knowledge and the future of AI.
This blog post is based on the research paper: “Large language models reduce public knowledge sharing on online Q&A platforms,” published in PNAS Nexus in 2024. DOI: 10.1093/pnasnexus/pgae400.
