In this project, the potential of novel interaction and visualization techniques for processing node-link diagrams (e.g. UML diagrams, business process models, etc.) was explored. The goal was to find the most promising approaches (e.g. multi-touch gestures) to facilitate the editing and exploration of node-link diagrams. The developed techniques were realized using iterative development processes and evaluated by laboratory studies. They are implemented in a system that can serve as a basis for further research in the field of diagram interaction. However, they can also be transferred to other application areas.
Creation and editing of node-link diagrams on interactive displays
Visual representations of graphs in the form of node-link diagrams play an important role in many application areas, for example in software modeling, biology or process management. Besides exploration and analysis, in many situations diagrams are created and edited by hand. For this purpose they are drawn on paper or whiteboards or edited in digital diagram editors. However, interaction with mouse-based editors is often cumbersome, and there is no seamless integration of hand-drawn sketches.
Therefore, this project explored the potential of novel technologies, such as interactive displays and digital pen and paper, for processing node-link diagrams. Especially the combination of multi-touch and pen input played an important role. It allows not only the simultaneous execution of several actions, but also fast mode switching with gestures, where finger and pen interaction can complement each other in a useful way.
Results
Within the scope of this project a prototype was developed, which offers the following functionalities:
- Interaction with finger and pen gestures: The realized gesture set is based on a collection of gestures suggested by users.
- Consideration of two predominant mental models: Structured editing (creating diagrams by selecting items from menus) and sketching diagrams (free drawing as on electronic whiteboards).
Multitouch layout techniques for interactive displays
The precise alignment of graphic objects and the creation of layouts are elementary processes when working with various software tools, such as graphics and presentation applications or diagram editors. The use of interactive displays seems promising for such applications. The goal of this project was the development and evaluation of interaction techniques as well as virtual tools that support the user in layout tasks in tabletop environments.
Results
As a result of this project the NEAT system (Natural and Effective Layout Techniques) was developed. It consists of three core components:
- Interactive Background Grid: This is an interactive grid that is displayed on the background of the drawing area. It allows precise positioning of graphical objects using snap dragging. The cell sizes of the grid can be interactively adjusted and changed by multi-touch input. Furthermore, it is possible to quickly and seamlessly transform an orthogonal grid into a radial grid.
- Multitouch Alignment Guides: These guides are created explicitly, for example by drawing them, and can be freely positioned with the help of a handle. The guideline can take any shape (from straight lines and curves to circles or rectangles) and allow graphical objects to be bound. These can then be positioned and aligned along the line using multitouch interaction.
- Layout Gestures: A set of multi-touch and pen gestures has been developed that allow graphic objects to be quickly duplicated and aligned with each other with pixel precision. The gestures are abstract in nature, so they have to be learned, and explicitly address the needs of modeling experts.
Off-Screen Visualization for Node-Link Diagrams
Node-link diagrams, such as software models, become so large in many situations that they are difficult to display in their entirety on standard displays. As a result, the contents of overview views are difficult to read and correlations are often difficult to recognize. This project deals with the use of modern visualization techniques to make large graphs easier to understand and navigate.
Results
A focus and context technique was developed to support the exploration of large diagrams. It allows to quickly navigate to currently invisible nodes of a diagram. When enlarging a diagram section, representative objects for the nodes outside the visible area are displayed at the edge of the display. These interactive proxy objects provide the user with context information. By clicking on them, they allow quick automatic navigation to the linked diagram element.
Puplications
@inbook{fixme,
author = {Mathias Frisch and Raimund Dachselt},
editor = {Thomas Schlegel},
title = {Kombinierte Multi-Touch und Stift-Interaktion: Ein Gesten-Set zum Editieren von Diagrammen},
booktitle = {Multi-Touch: Interaktion durch Ber\"{u}hrung},
series = {Xpert.press},
year = {2014},
month = {2},
isbn = {978-3-642-36112-8},
pages = {89--116},
doi = {10.1007/978-3-642-36113-5_5},
url = {http://dx.doi.org/10.1007/978-3-642-36113-5_5},
publisher = {Springer Berlin Heidelberg}
}List of additional material
@article{Frisch-2013-InfoVisOffScreen,
author = {Mathias Frisch and Raimund Dachselt},
title = {Visualizing Off-Screen Elements of Node-Link Diagrams},
booktitle = {Information Visualization},
volume = {12(2)},
year = {2013},
month = {4},
pages = {133--162},
numpages = {30},
publisher = {SAGE}
}List of additional material
Visualization and Interaction Techniques for Node-Link Diagram Editing and Exploration
@phdthesis{fixme,
author = {Mathias Frisch},
title = {Visualization and Interaction Techniques for Node-Link Diagram Editing and Exploration},
series = {Informatik},
year = {2012},
month = {6},
isbn = {978-3-8439-0563-3},
location = {Otto-von-Guericke-Universit\"{a}t Magdeburg},
numpages = {253},
url = {http://www.dr.hut-verlag.de/978-3-8439-0563-3.html},
publisher = {Verlag Dr. Hut},
address = {M\"{u}nchen}
}List of additional material
@inproceedings{frisch-2012-ITS,
author = {Mathias Frisch and Ricardo Langner and Raimund Dachselt},
title = {NEAT: A Set of Flexible Tools and Gestures for Layout Tasks on Interactive Displays},
booktitle = {Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces 2011},
series = {ITS '11},
year = {2011},
month = {11},
isbn = {978-1-4503-0871-7},
location = {Kobe, Japan},
pages = {1--10},
numpages = {10},
doi = {10.1145/2076354.2076356},
url = {http://doi.acm.org/10.1145/2076354.2076356},
acmid = {2076356},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {alignment, bimanual interaction, constraints, expert mode, gestures, multi-touch, pen input, tabletops}
}List of additional material
@inproceedings{Heydekorn-2011-MuC,
author = {Jens Heydekorn and Mathias Frisch and Raimund Dachselt},
title = {Evaluating a User-Elicited Gesture Set for Interactive Displays},
booktitle = {Proceedings of Mensch und Computer 2011},
series = {Mensch und Computer 2011},
year = {2011},
month = {9},
isbn = {978-3-486-71235-3},
location = {Chemnitz, Germany},
pages = {191----200},
numpages = {10},
publisher = {Oldenbourg Publishing},
address = {Munich, Germany}
}List of additional material
@inproceedings{frisch-2011-CHI,
author = {Mathias Frisch and Sebastian Kleinau and Ricardo Langner and Raimund Dachselt},
title = {Grids \& Guides: Multi-Touch Layout and Alignment Tools},
booktitle = {Proceedings of the Conference on Human Factors in Computing Systems},
series = {CHI '11},
year = {2011},
month = {5},
isbn = {978-1-4503-0228-9},
location = {Vancouver, BC, Canada},
pages = {1615--1618},
numpages = {4},
doi = {10.1145/1978942.1979177},
url = {http://doi.acm.org/10.1145/1978942.1979177},
acmid = {1979177},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {interactive grids, layout techniques, manual diagram layout, multi-touch alignment guides, snap-dragging}
}List of additional material
@inproceedings{fixme,
author = {Mathias Frisch and Ricardo Langner and Sebastian Kleinau and Raimund Dachselt},
title = {A Multi-Touch Alignment Guide for Interactive Displays},
booktitle = {Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces 2010},
series = {ITS '10},
year = {2010},
month = {11},
isbn = {978-1-4503-0399-6},
location = {Saarbr\"{u}cken, Germany},
pages = {255--256},
numpages = {2},
doi = {10.1145/1936652.1936703},
url = {http://doi.acm.org/10.1145/1936652.1936703},
acmid = {1936703},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {alignment, interactive surfaces, multi-touch}
}List of additional material
@inproceedings{fixme,
author = {Mathias Frisch and Jens Heydekorn and Sebastian Schmidt and Miguel Nacenta and Raimund Dachselt and Sheelagh Carpendale},
title = {Editing and Exploring Node-Link Diagrams on Pen- and Multi-Touch-operated Tabletops},
booktitle = {Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces 2010},
series = {ITS '10},
year = {2010},
month = {11},
isbn = {978-1-4503-0399-6},
location = {Saarbr\"{u}cken, Germany},
pages = {304},
numpages = {1},
doi = {10.1145/1936652.1936732},
url = {http://doi.acm.org/10.1145/1936652.1936732},
acmid = {1936732},
publisher = {ACM},
address = {New York, NY, USA}
}List of additional material
@inproceedings{fixme,
author = {Mathias Frisch and Raimund Dachselt},
title = {Off-Screen Visualization Techniques for Class Diagrams},
booktitle = {Proceedings of the ACM Symposium on Software Visualization 2010},
year = {2010},
month = {10},
isbn = {978-1-4503-0028-5},
location = {Salt Lake City, USA},
pages = {163----172},
numpages = {10},
doi = {10.1145/1879211.1879236},
url = {http://doi.acm.org/10.1145/1879211.1879236},
acmid = {1879236},
publisher = {ACM},
keywords = {UML, contextual view, interaction, navigation, node-link diagrams, off-screen visualization}
}List of additional material
@inproceedings{fixme,
author = {Jens Heydekorn and Mathias Frisch and Raimund Dachselt},
title = {Eine Architektur zum flexiblen Einsatz von gestischer Interaktion},
booktitle = {Proceedings of Mensch und Computer 2010},
year = {2010},
month = {9},
isbn = {978-3486704082},
location = {Duisburg, Germany},
pages = {321----330},
numpages = {10},
publisher = {Oldenbourg Publishing}
}List of additional material
@inproceedings{fixme,
author = {Mathias Frisch and Jens Heydekorn and Raimund Dachselt},
title = {Diagram Editing on Interactive Displays Using Multi-Touch and Pen Gestures},
booktitle = {Proceedings of the International Conference on the Theory and Application of Diagrams 2010},
year = {2010},
month = {8},
isbn = {978-3-642-14599-5},
location = {Portland, Oregon},
pages = {182----196},
numpages = {15},
url = {http://portal.acm.org/citation.cfm?id=1884734.1884759},
acmid = {1884759},
publisher = {Springer},
keywords = {digital pen, interaction techniques for diagrams, mental models, multi-touch and pen interaction, sketching, tabletops, user-centered approach}
}List of additional material
@article{fixme,
author = {Jens Heydekorn and Mathias Frisch and Raimund Dachselt},
title = {Prospects of user elicited gestural interaction techniques},
booktitle = {CHI 2010 Workshop on Natural User Interfaces: The Prospect and Challenge of Touch and Gestural Computing},
year = {2010},
month = {4},
address = {Atlanta, Georgia, USA}
}List of additional material
@inproceedings{fixme,
author = {Mathias Frisch and Jens Heydekorn and Raimund Dachselt},
title = {Investigating Multi-Touch and Pen Gestures for Diagram Editing on Interactive Surfaces},
booktitle = {Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces 2009},
series = {ITS '09},
year = {2009},
month = {11},
isbn = {978-1-60558-733-2},
location = {Banff, Alberta, Canada},
pages = {167--174},
numpages = {8},
doi = {10.1145/1731903.1731933},
url = {http://doi.acm.org/10.1145/1731903.1731933},
acmid = {1731933},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {bimanual input, diagram editing, hand gestures, multi-touch, node-link diagrams, pen interaction, tabletop}
}
@article{fixme,
author = {Mathias Frisch and Raimund Dachselt},
title = {Towards a Framework for Supporting Software Modeling Activities Through Novel Interaction and Visualization Techniques},
booktitle = {Software Engineering - Companion Volume, 2009. ICSE-Companion 2009. 31st International Conference on Software Engineering},
year = {2009},
month = {5},
location = {Vancouver, Canada},
pages = { 359----362},
numpages = {4},
doi = {10.1109/ICSE-COMPANION.2009.5071021},
url = {http://dx.doi.org/10.1109/ICSE-COMPANION.2009.5071021}
}List of additional material
@inproceedings{fixme,
author = {Mathias Frisch and Raimund Dachselt and Tobias Br\"{u}ckmann},
title = {Towards seamless semantic zooming techniques for UML diagrams},
booktitle = {Proceedings of the 4th ACM Symposium on Software Visualization (Ammersee, Germany, September 16-17, 2008)},
year = {2008},
month = {9},
isbn = {978-1-60558-112-5},
location = {Ammersee, Germany},
pages = {207 -- 208},
numpages = {2},
doi = {10.1145/1409720.1409758},
url = {http://doi.acm.org/10.1145/1409720.1409758},
acmid = {1409758},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {UML, diagrams, interaction techniques, models, semantic zooming, visualization, zoomable user interface}
}List of additional material
@inproceedings{fixme,
author = {Raimund Dachselt and Mathias Frisch and Eike Decker},
title = {Enhancing UML sketch tools with digital pens and paper},
booktitle = {Proceedings of the 4th ACM Symposium on Software Visualization (Ammersee, Germany, September 16-17, 2008).},
year = {2008},
month = {9},
isbn = {978-1-60558-112-5},
location = {Ammersee, Germany},
pages = {203--204},
numpages = {2},
doi = {10.1145/1409720.1409756},
url = {http://doi.acm.org/10.1145/1409720.1409756},
acmid = {1409756},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {Anoto, UML, sketching, software design}
}List of additional material
@article{frisch-2008-CHASE,
author = {Mathias Frisch and Raimund Dachselt},
title = {Benefits of interactive display environments in the software development process},
booktitle = {Proceedings of the International Workshop on Cooperative and Human Aspects of Software Engineering 2008},
year = {2008},
month = {5},
isbn = {978-1-60558-039-5},
location = {Leipzig, Germany},
pages = {53--56},
numpages = {4},
doi = {10.1145/1370114.1370128},
url = {http://doi.acm.org/10.1145/1370114.1370128},
acmid = {1370128},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {UML, diagrams, interaction techniques, models, multi touch, semantic zooming, software development process, visualization, zoomable user interface}
}List of additional material
