Studies Using reVISit

This paper assesses how visualization badges-small informative visual labels placed alongside visualizations-affect people's ability to read, interpret, and learn from visualizations. Similar to visual labels on product packaging, visualization badges highlight potentially relevant information to the reader, such as pointing to a 'Major Finding', cautioning a 'Truncated Axis', or informing about 'AI Involvement'. We scrutinize 10 qualities of badges through three complementary studies: (1) a controlled Prolific study with a general audience (N=24); (2) interviews with 12 domain experts experiencing badges as part of professional visualization applications; and (3) interviews with 5 visualization designers gauging the feasibility of badges for themselves and their audiences. We found clear evidence for badges attracting curiosity, aiding interpretation, fostering trust, and motivating broader adoption. We conclude by discussing the role of badges in data and visualization literacy and exploring their potential to become an accepted convention for visualization. By providing our study materials, reusable badge implementation, and additional badge resources at https://vis-badges.github.io, we invite the community to use, adapt, and further study visualization badges.

We conducted crowdsourced experiments on seven core visual channels (position, length, tilt, area, curvature, luminance, and saturation) using primitive visual stimuli, a set of visual marks without chart-specific scaffolding to isolate channel-level variation. We evaluated these channels across four perceptual tasks (accuracy, discriminability, separability, and pop-out) and found that channel effectiveness is fundamentally multi-dimensional, with rankings shifting substantially across tasks.

To close these gaps, we develop and investigate two web-based qualitative assessments for testing the critique and design aspects of visualization literacy through online think-aloud critique and sketching of visualization designs based on data and a prompt. We compare performance on our assessments to two established visualization literacy assessments, CALVI and Mini-VLAT, by administering them to three groups that represent three experience levels: crowdworkers, students who have taken a relevant course, and researchers.

We conducted two crowdsourced studies investigating the impact of incentives on (i) a low-level, perceptual task (perception of correlations in scatterplots or parallel coordinate plots), and (ii) a task involving reasoning (decision-making based on a weather forecast represented as intervals or density plots). In each of these studies, we manipulate both the visual representation and the presence of incentives as between-subject conditions.

We use reVISit to back an interview study with 12 BLV participants, in which we compare two learning formats: (1) a tactile chart, a textual explanation, and an LLM chatbot; and (2) a textual explanation and an LLM chatbot. The learning phase was followed by exploration of an unfamiliar dataset using alt text and an LLM.

Dimensionality Reduction (DR) techniques like t-SNE and UMAP, while powerful visualization tools, inherently introduce distortion during the high-to-low-dimensional mapping process. This distortion means the resulting layout often fails to represent the original high-dimensional distances and relationships faithfully. For instance, visually proximal clusters may be distant in the original space (a global distortion artifact), or densely grouped high-dimensional points may appear scattered (a local density artifact). Prior research consistently shows that practitioners working with DR layouts, especially those less experienced, are frequently unaware of these inherent visual pitfalls, leading them to perform faulty analyses and draw incorrect conclusions based on misleading layouts. To directly address this analytical misuse, this work shows that simple geometry-based layout enrichment mechanisms can substantially prevent misinterpretations. By enriching the structures that surface in the 2D projection with information about their true high-dimensional distances, our approach avoids the approximation errors and smoothness assumptions introduced by typical distortion metrics. We present a Delaunay-based layout enrichment visualization, demonstrating the corrective power of simple distance-based approaches to assist users in their analysis. Furthermore, a comprehensive user evaluation confirms our hypothesis that participants equipped with our enriched layouts exhibit a lower incidence of analytical errors compared to those using established baselines, validating the practical utility of our approach.

Annotation is among the most demanding visualization tasks to automate, as it simultaneously requires correctly navigating visual, semantic, and stylistic constraints. Failure to meet any of these conditions severely undermines the utility of an annotation, rendering it challenging to read, inaccurate, or visually discordant. Despite a growing body of annotation tools and automations, no existing benchmark or evaluation framework tests whether these conditions are met because of their scope and annotation not being the focus of their studies. We introduce AnnoBench, a benchmark for visualization annotation that materializes the inherent challenges of this domain in a structured and testable manner. AnnoBench pairs visualizations from professional data journalism and visualization galleries with annotation tasks, spanning four representation formats, five chart description conditions, and two prompt specification levels. The benchmark is executed via VLM-as-a-judge, using models aligned with manual human assessment. We evaluate the benchmark via four one-factor-at-a-time experiments, exploring the effects of input representation, semantic context, and prompt specificity, and model selection on annotation quality. This work provides a foundation for advancing annotation automation, tooling, and visualization-generation pipelines

Color discrimination is a fundamental aspect of visualization as it influences how people interpret visual encodings. Many visualization guidelines are informed by perceptual studies, yet relatively few have been replicated. Acknowledging that the interaction between human perception, visual tasks, and display technology can change over time, we replicate two crowdsourced color discrimination studies conducted 10 years earlier. Specifically, we replicated a visualization-focused color discrimination task (N = 144) and a more general perceptual discrimination task (N = 394). In both studies, our results reproduced the original perceptual effects. We further use the replication to investigate whether color-related practice influences color discrimination. Specifically, we extended our replication studies by adding questions about participants’ engagement with color practices. We then examined whether diverse color-related practices (e.g., artistic hobbies, knowledge of color theory, and cosmetic makeup use) influenced color discrimination. We found no significant difference between participants who reported engaging in color-related practices and those who did not, suggesting that design guidance regarding color discrimination may generalize across viewers regardless of their regular color practice.

Designing a color palette for data requires designers to balance multiple constraints, including accessibility and aesthetics. Color palette tools support this process through features icluding direct manipulation, automated palette generation and evaluation, previews, and so on.We conducted an exploratory think-aloud crowd work study with 40 self-identified designers. Each participant used one of four palette tools selected to span different interaction modalities to complete a series of accessibility- and aesthetics-oriented design tasks. We observed two preliminary patterns. First, tool differences were more pronounced in accessibility-constrained tasks. Second, even when accessibility was not explicitly required, some tools produced more accessibility-friendly palettes and prompted more accessibility-oriented thinking. In this tool genre, then, system design shapes outcomes both via built-in functionality, as well as by directing designers' attention toward particular constraints and design considerations.

Wrangling consumes a disproportionate share of the effort associated with any data project. While a variety of tools support it, relatively little is known about how their differing interface forms shape the way people actually wrangle. We conduct a between-subjects (N=40) observational study of data cleaning tasks performed in tools spanning distinct interface paradigms: Jupyter (notebook), Excel (spreadsheet), ChatGPT (conversational AI), and OpenRefine (visual wranglers). We situate our observations within the Technical Dimensions of Programming Systems framework, which we use as a conceptual scaffold for comparing across interface paradigms. Within the context of our study, the results suggest that tool affordances steer user strategies but do not determine outcomes. There is no consistent advantage of any single tool, nor convergence of results within tools observed across our outcome measures. Instead, we identify trade-offs and connect them with observed practice. For example, a key tension is between data- and abstraction-centered interfaces, where data-centered interfaces encourage opportunistic cleaning rather than systematic, planned transformations found in abstraction-focused tools (but come with a cognitive burden). Tool design, beyond mere functionality, plays a structuring role in how data work unfolds.

Textual data serialization formats, such as JSON or XML, are ubiquitous, supporting tasks like software configuration and data tabularization. Despite their prominence, little is known about their usability. What makes one good or bad? Is there a best one for cognitive efficiency? We explore these questions via a (N = 215) crowd work study and a (N = 9) semi- structured interview study. We find that format distinctions (like indentation versus curly braces) do not consistently translate into substantial usability differences. While HJSON and YAML performed better than other formats in certain modification tasks, these advantages disappeared in more realistic settings where task complexity was either trivial or highly demanding. Instead, usability appears driven by sociotechnical ecosystems: the tooling, documentation, and community practices surrounding a format matter more than syntax.

Prior work on perceptual effectiveness has decomposed visualizations into smaller common units (e.g., channels such as angle, position, and length) to establish rankings. While useful, these decompositions lack the computational structure to predict performance for new visualization x task combinations, requiring new experiments for each. We propose an alternative unit of analysis: operationalizing quantitative visualization interpretation as sequences of composable visual decoding operators. Using probability density function (PDF) and cumulative distribution function (CDF) charts, we examine how four chart-specific tasks can be decomposed into five reusable, chart-agnostic perceptual operations and characterize their error profiles through hierarchical Bayesian modeling. We then test generalizability by composing one kind of learned operators to predict performance on a structurally different task: Moritz et al.'s [37] scatterplot mean-estimation experiment, where the chart type, chart dimensions, and analytic goal all differ from the learning conditions. With a pre-registered analysis plan, we compose operators under six candidate strategies and evaluate each against empirical data with no parameters fit to the response data. One strategy captures both bias and variance of observed responses; five alternatives fail in distinguishable ways. We argue that this decoding-operator-oriented approach to empirical visualization research demonstrates the feasibility of a different way of doing empirical visualization research, one where findings compose, and predictions extend beyond the conditions in which they were measured. Free copy of this paper and supplemental materials: https://osf.io/prtfq.

This study investigates the ability of multimodal Large Language Models (LLMs) to identify and interpret misleading visualizations, and recognize these observations along with their underlying causes and potential intentionality. Our analysis leverages concepts from visualization rhetoric and a newly developed taxonomy of authorial intents as explanatory lenses. We formulated three research questions and addressed them experimentally using a dataset of 2,336 COVID-19-related tweets, half of which contain misleading visualizations, and supplemented it with real-world examples of perceptual, cognitive, and conceptual errors drawn from VisLies, the IEEE VIS community event dedicated to showcasing deceptive and misleading visualizations. To ensure broad coverage of the current LLM landscape, we evaluated 16 state-of-the-art models. Among them, 15 are open-weight models, spanning a wide range of model sizes, architectural families, and reasoning capabilities. The selection comprises small models, namely Nemotron-Nano-V2-VL (12B parameters), Mistral-Small-3.2 (24B), DeepSeek-VL2 (27B), Gemma3 (27B), and GTA1 (32B); medium-sized models, namely Qianfan-VL (70B), Molmo (72B), GLM-4.5V (108B), LLaVA-NeXT (110B), and Pixtral-Large (124B); and large models, namely Qwen3-VL (235B), InternVL3.5 (241B), Step3 (321B), Llama-4-Maverick (400B), and Kimi-K2.5 (1000B). In addition, we employed OpenAI GPT-5.4, a frontier proprietary model. To establish a human perspective on these tasks, we also conducted a user study with visualization experts to assess how people perceive rhetorical techniques and the authorial intentions behind the same misleading visualizations. This allows comparison between model and expert behavior, revealing similarities and differences that provide insights into where LLMs align with human judgment and where they diverge.

This study investigates how visual cues can support metacognitive sensitivity during different phases of reasoning in logical inference tasks. Drawing on deductive reasoning psychology and multimedia learning, we introduce a visual cueing framework that highlights premises and conclusions, represents conditional statements as diagrams, and significantly improves users’ metacognitive sensitivity in an empirical study with 164 participants.

This study examines guardrail selection in line charts to provide context for persuasive visualizations.

This paper evaluates visual encodings for representing bivariate edge attributes in adjacency matrices.

We report on a crowdsourced study to collect human ratings of perceived complexity for diverse visualizations using ReVISit. Using these ratings as ground truth, we then evaluated three methods to estimate this perceived complexity: image analysis metrics, multilinear regression using manually coded visualization features, and automated feature extraction using a large language model (LLM).

Based on data from a 235-participant user study covering three visualization tests (mini-VLAT, CALVI, and SGL), we show that distinct attention patterns in visual data exploration can correlate with participants' literacy levels: While experts (high-scorers) generally show a strong attentional focus, novices (low-scorers) focus less and explore more.

To demonstrate the capability of reVISit 2.0 we run three replication studies on just noticeable difference (perception), the role of search (interaction), and designing textures (provenance).

Stress in a graph drawing has been a popular layout principle for more than two decades. Low stress drawings exhibit the property that the geometric distances between all pairs of nodes correlate with the shortest paths between them. The assumption has always been that low stress drawings are "nicer" and better support human perception and comprehension than high stress drawings. In this paper, we put these assumptions to the test. We use a normalised scale-independent and rotation-independent metric for stress; this is necessary to ensure strict controls on our experimental stimuli. We report on three experiments, exploring human perception of stress, preference for stress, and the effect of stress on a graph performance task. We conclude that people can see stress in a graph drawing, that they prefer low stress drawings, and that their performance in a shortest path task improves as stress decreases - thus empirically confirming long-standing assumptions.

This paper explores the alignment of people's perceptions with fairness metrics through the use of a two-alternative forced-choice methodology, compared to prior methods that used ordinal rating scales such as Likert scales.

We introduce Grid Labeling - a novel annotation method for collecting task-specific importance data to enhance saliency prediction models. We conducted a human subject study comparing Grid Labeling with existing annotation methods, ImportAnnots, and BubbleView across multiple metrics.

This paper conducts an online survey to investigate the task load of administering different versions of the NASA-TLX in a meta-study.

This paper presents a system for automatically generating accessible text descriptions for UpSet plots, aiming to make complex set data visualizations understandable for blind and low-vision users and potentially benefiting sighted users as well.

This paper investigates the potential for large language models (LLMs) to automate the generation of multiple-choice visualization items. We develop the VILA (Visualization Items Generated by Large LAnguage Models) pipeline, for efficiently generating visualization items that measure people’s ability to accomplish visualization task.

We ran two studies with revisit. The first study showed participants a few interactive charts and had them answer questions about the data, and the second showed participants static visualizations with associated captions and asked them if they trusted the data.

This study explores the potential of visualization design to combat cherry-picking—a common tactic used to misrepresent data—by developing a design space of guardrails aimed at discouraging this practice. Through crowd-sourced experiments, the researchers evaluate the effectiveness of these guardrails.

This project investigates analysts' behaviors and goals when interacting with attributed summaries, utilizing interactive prototypes and collected data to understand how summary usage aligns with goals such as time-saving, highlighting key takeaways, or answering lookup questions.