The Narrative Gap: Can LLMs Help Us Navigate Diverse Narratives Across Languages?
THE QUICK READ
The authors propose six capabilities for multilingual information seeking across competing narratives and introduce NarrativeBench, built from 1,333 news articles in 23 publication languages across 23 contested topics and 460 Reddit-grounded queries. They report evaluating 14 LLMs across 24 query languages and conclude that current models struggle to present or preserve diverse narratives, even when users explicitly request diversity.
KEY POINTS
- The authors define six capabilities: narrative inclusion, narrative exclusion, missing narrative identification, presenting diverse narratives, preserving coverage under confirmatory queries, and cross-language information parity.
- NarrativeBench covers 23 publication languages, 23 contested topics, 1,333 news articles, and 460 Reddit-grounded queries.
- The authors report that 14 evaluated LLMs showed substantial gaps in exposing users to competing accounts across query framings and languages.
READING LIMITS
This summary relies only on the title and abstract; the benchmark design, evaluation methodology, model list, and reported results were not independently verified, and peer-review status is unknown.
Source abstract [EN]
Large language models (LLMs) increasingly mediate how people seek and consume information, including accounts that differ across languages and regions, especially on contested topics. Supporting diverse information seeking requires models to help users navigate competing narratives across languages and information streams. We define six capabilities necessary for diverse multilingual information seeking: three foundational capabilities for narrative inclusion, narrative exclusion, and missing narrative identification; and three higher-level capabilities for presenting diverse narratives, preserving narrative coverage under confirmatory queries, and maintaining information parity across languages. To guide the development of those information seeking capabilities, we introduce NarrativeBench, a benchmark comprising 1,333 news articles in 23 publication languages across 23 contested topics and 460 Reddit-grounded queries. Evaluating 14 LLMs across 24 query languages, we find that current models lack the capabilities needed to reliably navigate diverse narratives, limiting users' control over the perspectives they receive and leaving substantial gaps in exposure to competing accounts even when diversity is explicitly requested. Information access also differs across query framings and languages, risking echo chambers, linguistic filter bubbles, and information disparities that weaken the common ground needed for democratic discourse. NarrativeBench provides a foundation for developing information systems that give users across languages and regions agency to navigate diverse narratives.
AI summaries use public titles and abstracts only. Research claims belong to the authors; consult the original paper.