Abstract
Exponential growth in the quantity of digital news, social media, and other textual sources makes it difficult for humans to keep up with rapidly evolving narratives about world events. Various visualisation techniques have been touted to help people to understand such discourse by exposing relationships between texts (such as news articles) as topics and themes evolve over time. Arguably, the understandability of such visualisations hinges on the assumption that people will be able to easily interpret the relationships in such visual network structures. To test this assumption, we begin by defining an abstract model of time-dependent text visualisation based on directed graph structures. From this model we distill motifs that capture the set of possible ways that texts can be linked across changes in time. We also develop a controlled synthetic text generation methodology that leverages the power of modern LLMs to create fictional, yet structured sets of time-dependent texts that fit each of our patterns. Therefore, we create a clean user study environment (n=30) for participants to identify patterns that best represent a given set of synthetic articles. We find that it is a challenging task for the user to identify and recover the predefined motif. We analyse qualitative data to map an unexpectedly rich variety of user rationales when divergences from expected interpretation occur. A deeper analysis also points to unexpected complexities inherent in the formation of synthetic datasets with LLMs that undermine the study control in some cases. Furthermore, analysis of individual decision-making in our study hints at a future where text discourse visualisation may need to dispense with a one-size-fits-all approach and, instead, should be more adaptable to the specific user who is exploring the visualisation in front of them.
Abstract (translated)
数字新闻、社交媒体和其他文本来源的数量呈指数级增长，这使得人类难以跟上关于世界事件的快速演变叙述。各种可视化技术被提出，以帮助人们通过揭示不同时间点之间文章（如新闻报道）之间的关系来理解这样的讨论。然而，此类可视化的可理解性假设是人们能够轻松地解读这些网络结构中的关系。 为了检验这一假设，我们首先定义了一个基于有向图结构的时间依赖文本可视化的抽象模型，并从中提炼出一系列模式，这些模式捕捉了在时间变化中文章之间可能的链接方式。同时，我们开发了一种受现代大型语言模型（LLMs）启发的合成文本生成方法论，能够创建一套符合每一种模式的虚构但结构化的时间相关文本集。 因此，我们为参与者创造了一个清洁的用户研究环境（n=30），让他们识别一组合成文章中最佳代表特定模式的方式。我们的发现表明，用户很难准确地识别并恢复预定义的模式。通过分析定性数据，我们绘制了当用户的解读与预期解释出现偏差时所采用的各种丰富的用户逻辑。 此外，对生成合成数据集过程中使用LLMs的深入分析揭示了一些意外复杂性和挑战，这在某些情况下影响了研究的控制性。最后，我们在研究中关于个人决策过程的分析暗示了一个未来方向，在这个未来中，文本讨论可视化可能需要摒弃“一刀切”的方法，并且应该更加灵活地适应正在探索可视化的具体用户需求。
URL
https://arxiv.org/abs/2603.02422
