CAREER: Manipulable Semantic Components in Data Visualization Design

Flexible manipulation of graphical objects is essential in tools like Microsoft PowerPoint and Adobe Illustrator for vector-based design. Users can design complex visuals by manipulating on-canvas objects to change their visual properties, apply grouping and layouts, and define spatial relationships. Ideally, this interaction would not only improve usability and reduce cognitive load, but also serve as a cornerstone for seamless workflows and design life cycles in a larger ecosystem. Multiple people could iterate on a design’s vector representation collectively, and the graphical components could be repurposed for new contexts with ease. Such an interaction paradigm, however, is largely missing in today’s design practices for interactive data visualization. Visualization creators mostly work with hardwired, inflexible templates, and must be proficient in coding or navigating complex interfaces to produce custom designs. Once a visualization is created, it is difficult to reuse it on a different dataset or repurpose it for new contexts. This project investigates how we can more effectively support data visualization design under a new interaction paradigm: manipulable semantic components (MSC). Similar to manipulable graphical objects in vector graphics design, manipulable semantic components in interactive data visualization are graphical objects and associated relationships that can be visually represented and manipulated at design time (instead of run time). The project has the potential to democratize visualization design and transform current practices in the authoring, reuse, and collaborative design of data visualization. The investigator will conduct research on the theoretical frameworks, computational techniques, and applications of manipulable semantic components (MSC). In the first phase, the investigator will formulate an MSC framework for interactive visualizations of multivariate tables, trees, and networks. The framework will describe both the types of semantic components and operations to manipulate the components. In the second phase, the project will investigate new algorithms and techniques that compute component positions jointly determined by multiple layout factors in real-time, and automatically deconstruct existing visualizations for analysis and reuse. In the third phase, the theoretical foundation and the computational techniques will power new interfaces and interactive tools for flexible editing and transfer of components, tool-agnostic example reuse, and synchronous collaborative design. To foster a tighter integration between research and education, the investigator will use MSC to innovate the teaching of design thinking with two education objectives: exploration of design alternatives, and collaborative design feedback and iterations. The investigator will focus on two activities, parallel prototyping, and critique by redesign, in various educational contexts including new design curricula in undergraduate and graduate classes, and outreach programs for industry practitioners and students from other disciplines with visualization needs. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.