google-site-verification: googlec7193c3de77668c9.html

ChatGPT’s search results for news are ‘unpredictable’ and frequently inaccurate

A rendition of OpenAI’s logo, which looks like a stylized whirlpool.
Illustration: The Verge

Based on testing done by Columbia’s Tow Center for Digital Journalism researchers, OpenAI’s ChatGPT search tool has some issues when it comes to responding with the truth.

Advertisements

OpenAI launched the tool for subscribers in October, saying it could give “fast, timely answers with links to relevant web sources.” Instead, Futurism points out that the researchers said ChatGPT search struggled to correctly identify quotes from articles, even when they came from publishers with arrangements to share data with OpenAI.

The authors asked ChatGPT to identify the source of “two hundred quotes from twenty publications.” Forty of those quotes were taken from publishers who’d disallowed OpenAI’s search crawler from accessing their site. Yet, the chatbot confidently replied with false information anyway, rarely admitting it was unsure about the details it gave:

In total, ChatGPT returned partially or entirely incorrect responses on a hundred and fifty-three occasions, though it only acknowledged an inability to accurately respond to a query seven times. Only in those seven outputs did the chatbot use qualifying words and phrases like “appears,” “it’s possible,” or “might,” or statements like “I couldn’t locate the exact article.”

A chart showing how often ChatGPT answered confidently or was unsure, with a breakdown of how often its confident replies were “Wrong,” (89) “Partially Correct,” (57) and “Correct” (47).
Image: Columbia Journalism Review
ChatGPT was fully or partially wrong more than right, but almost always confidently so.

The Tow Center test’s authors documented ChatGPT search results that misattributed a letter-to-the-editor quote from the Orlando Sentinel to a story published in Time. In another example, when asked to identify the source of a quote from a New York Times article about endangered whales, it returned a link to a different website that had wholly plagiarized the story.

“Misattribution is hard to address without the data and methodology that the Tow Center withheld,” OpenAI told the Columbia Journalism Review, “and the study represents an atypical test of our product.” The company went on to promise to “keep enhancing search results.”

Views: 0

See also  Chiefs vs. Bills: How to Watch NFL Week 11 Online Today

Check Also

The FBI reportedly won’t investigate ICE anymore

According to the New York Times, federal agents have been told that the FBI will …

1010Benja shows Suno can be more than an AI music slop engine

I would never go so far as to say there’s no place for AI in …

US Marshals arrest the Tate brothers in Miami

The manosphere influencers Andrew and Tristan Tate were arrested Saturday in Miami by US Marshals …

Leave a Reply

Available for Amazon Prime