David Meets Goliath: Smartwatch plus Large Display for Visual Data Exploration

A conceptual framework enabling analysts to explore data items, track interaction histories, and alter visualization configurations through mechanisms using both a smartwatch and a large display in combination.

Video Illustrating our Concepts

https://youtu.be/q5P16kFs-nA

In a hurry? We also have a 30-second preview video.

Table of Contents

1. Abstract
2. Presentation @ ACM CHI ’18
3. Combining David and Goliath: Fundamentals
4. Conceptual Framework
5. Mapping Task Classifications
6. Related Publications
7. Related Student Theses

Abstract

We explore the combination of smartwatches and a large interactive display to support visual data analysis. These two extremes of interactive surfaces are increasingly popular, but feature different characteristics—display and input modalities, personal/public use, performance, and portability.

In this paper, we first identify possible roles for both devices and the interplay between them through an example scenario. We then propose a conceptual framework to enable analysts to explore data items, track interaction histories, and alter visualization configurations through mechanisms using both devices in combination.

We validate an implementation of our framework through a formative evaluation and a user study. The results show that this device combination, compared to just a large display, allows users to develop complex insights more fluidly by leveraging the roles of the two devices. Finally, we report on the interaction patterns and interplay between the devices for visual exploration as observed during our study.

Presentation @ ACM CHI ’18

The David meets Goliath paper has been presented at the ACM CHI 2018 (Montréal, Quebec, Canada) in the Visualization Beyond the Desktop session.
The slides of the talk are available as a PDF.

Honorable Mention We are particularly pleased that we received the honorable mention award (top 5%) for our paper at the conference.

Additional Reads

• Article from the University of Maryland Institute for Advanced Computer Studies
• Blog post on Medium

Combining David and Goliath: Fundamentals

Conceptual Framework

Item Sets & Connective Areas

Sets The primary role of the smartwatch is to act as a personalized storage of sets. We define sets as a generalized term for a collection of multiple entities of a certain type. In our framework, we currently consider two different set types: data items and configuration properties (e.g., axis dimension, chart type). These sets can also be predefined; for instance, for each existing axis dimension, a corresponding set is generated. On the smartwatch, the stored sets are provided as a list. Consistent with the set notion, sets of the same type can be combined using set operations (i.e., union, intersection, complement).

Connective Areas We define four zones for each visualization—called connective areas (CA)—that will provide the context (what) of an interaction: the marks, canvas, axes, as well as a special element close to the origin. Connective areas define the set type and control the functionalities accessed on the two devices. To focus on a CA, the interaction comprises of tapping or circling marks (i.e., data points) for selection. For other CAs, user can set the focus in two ways: by performing a touch-and-hold (long touch), the focus is set onto the respective area underneath the touch point but stays only active for the duration of the hold; by performing a double tap, the focus is kept as long as not actively changed. Setting the focus activates suitable functionalities for the specific connective area on the watch. On focus, stored set content can also be previewed on the large display.

Creating & Managing Sets for Visual Exploration

Mapping Established Task Classifications to our Interaction Sequences

Our framework addresses a wide range of tasks, here illustrated by mapping two established task classifications by Brehmer and Munzner [13] and Yi et al. [58] onto interaction sequences that are enabled by our framework (examples given in italics). For some tasks, certain aspects are also still supported by the large display itself, e.g., zooming and panning from abstract/elaborate and explore [58]. Regarding the typology by Brehmer and Munzner [13], we focus on their how part. From this part, a few tasks (encode, annotate, import, derive) are not considered as they are going beyond the scope of this paper. CA: Connective Area.