Have you ever wondered about the reliability of news from AI platforms? Recent research indicates that AI struggles significantly with accurately reporting news, which could have big implications for what you trust. Understanding these challenges is crucial, especially in a world where AI is becoming a major source of information.
The findings come from a comprehensive study by the European Broadcasting Union (EBU), involving 22 media organizations across 18 countries and 14 languages. A staggering 3,000 responses from popular AI chatbots like ChatGPT and Microsoft Copilot were examined for key aspects such as accuracy, sourcing, and the distinction between opinion and fact.
1. The Problem with AI-Powered News
The study revealed that 45% of AI-generated news responses contained at least one major error, while 81% showed minor issues. The lack of proper sourcing was identified as a key factor, with one-third of responses exhibiting significant problems, including misleading and incorrect attributions.
2. Accuracy Issues in AI Responses
Major accuracy issues affected 30% of responses, such as outdated information and fabricated details. For instance, ChatGPT inaccurately asserted that Pope Francis—who had passed away recently—was still the current Pope, only to be succeeded by Pope Leo XIV. Similarly, Microsoft Copilot misled users about a bird flu vaccine trial by citing a 2006 BBC article.
3. Which AI Model Performed the Worst?
Among the AIs tested, Google’s Gemini came in last, with problems in 76% of its responses, far exceeding the error rates of its competitors. Copilot and ChatGPT followed with 37% and 36% issues, respectively, while Perplexity had the lowest error rate at 30%.
4. Challenges with Fast-Paced News
Researchers found that AI struggled especially with rapidly changing stories and complex topics that require clarity between facts and opinions. For example, nearly half of the AI models faltered when asked if Trump was starting a trade war.
Jean Philip De Tender, EBU Media Director, summed it up well: “This research conclusively shows that these failings are not isolated incidents. They are systemic, cross-border, and multilingual.” This is critical because it affects how people engage with news and can compromise public trust.
5. The Growing Role of AI in Information Consumption
Despite these issues, AI assistants are gaining traction as a primary information source for users, competing fiercely with traditional search engines. As AI companies build on this trend, advancements like OpenAI’s ChatGPT Atlas are changing how we interact with the online world.
6. Does AI Provide Reliable News?
The latest report from the Reuters Institute and University of Oxford found that while using AI for news is increasing—doubling from last year—it remains a minority practice. Particularly, younger demographics, like 18–24-year-olds in Argentina and the U.S., are leading this trend. Interestingly, 48% of respondents in this age group utilized AI to simplify complex news stories.
However, if AI assistants are unreliable sources of news, yet many consumers trust them, there lies a significant concern. A recent report pointed out that this situation risks diluting the trust in established news outlets.
What are the key findings from the EBU study on AI news accuracy?
The EBU study highlighted that a staggering 45% of AI-generated responses contained major inaccuracies, mainly related to sourcing issues.
Why do AI assistants struggle with news?
AI assistants struggle with fast-paced stories, intricate timelines, and topics requiring clear distinctions between fact and opinion, leading to increased error rates.
Is AI news consumption rising?
Yes, using AI to access news has doubled in the past year, especially among younger individuals in regions like the U.S. and Argentina.
Are there any improvements in AI language models regarding news?
Yes, the EBU study noted some improvements in models like Gemini regarding accuracy, although issues with sourcing persist.
Ultimately, while AI is evolving, it still falls short in delivering reliable news. That said, the quest for more trustworthy AI continues as both creators and consumers navigate this transformative landscape. Dive deeper into AI and its implications at Moyens I/O.