Bill Buxton

Computer Systems Research Institute

Toronto, Ontario

Canada M5S 1A4

Buxton@dgp.toronto.edu

June 1, 1994

Introduction

The IRG is a group of faculty, staff and students at the University of Toronto investigating human input to computer systems. The group was started in September 1989, and is primarily concerned with what has been called "haptic" input: input that involves physical contact, (such as by the hand or foot). As such, it is not involved in speech input, for example.

The prime objective of the group is to understand better the relationship between the motor-sensory and the cognitive aspects of interaction. An underlying hypothesis is that the way that information is articulated by the motor system can has a strong impact on the cognitive aspects of human performance. Hence, the intent of the work goes beyond device comparisons and time-motion studies. One fall-out of this is that the group could equally well be named, "The Appropriate Feedback Group."

The group follows a number of paradigms in its approach. Much of the work involves formal experimentation using human subjects. This has included studies investigating mode errors, and testing models to characterize different elemental transactions found in direct manipulation systems. However, the group also uses an "experimental programming" paradigm, in which prototype (often toy) systems are built and informally tested.

The rest of this document provides a set of titles and abstracts of recent papers, and introduces members of the group. Interested parties are encouraged to contact us for additional information or in order to pursue other sorts of scientific cooperation.

Primary support for the IRG has come from . Xerox PARC, the Information Technology Research Centre of Ontario, and Digital Equipment Corp. Additional support has been provided by the Natural Sciences and Engineering Research Council of Canada, Apple Computer Inc., IBM Canada"s Laboratory Centre for Advanced Studies and the Arnott Design Group of Toronto. This support is gratefully acknowledged.

Recent Papers and Publications

Bier, E., Buxton, W., Baudel, T. & Stone, M. (in press). A taxonomy of see-through tools. To appear in Proceedings of CHI '94, Boston, April 24-28.

The see-through interface is a new user interaction paradigm, based on a set of semi-transparent tools that are moved over an application with one hand while the other hand applies the tools with a traditional cursor. We describe some of the attributes of see-through tools that we have discovered to date. In addition, we describe a set of see-through tools that illustrate these attributes. The particular tools described here are all useful in the domais of graphical illustration and text editing.

Bier, E. A., Stone, M., Pier, K., Buxton, W. & DeRose. T. (1993) Toolglass and magic lenses: the see-Through interface. Proceedings of SIGGRAPH '93, 73-80.

Toolglass widgets are new user interface tools that can appear, as though on a transparent sheet of glass, between an application and a traditional cursor. They can be positioned with one hand while the other positions the cursor. The widgets provide a rich and concise vocabulary for operating on objects. These widgets may incorporate visual filters, called Magic Lens filters, that modify the presentation of application objects to reveal hidden information, to enhance data of interest, or to suppress distracting information. Together these tools for a see through interface, that offers many advantages over traditional controls. They provide a new style of interaction that better exploits the user's everyday skills. They can reduce steps, cursor motion, ad errors. Many widgets can be provided in a user interface, by designers and by users, without requiring dedicated screen space. In addition, lenses provide rich context-dependent feedback and the ability to view details and context simultaneously. Our widgets and lenses can be combined to form operation and viewing macros, and can be used over multiple applications.

Brown, E., Buxton, W. & Murtagh, K. (1990). Windows on tablets as a means of achieving virtual input devices. In D. Diaper et al. (Eds), Human-Computer Interaction - INTERACT '90, Elsevier Science Publishers B.V. (North-Holland), 675-681.

Users of computer systems are often constrained by the limited number of physical devices at their disposal. For displays, window systems have proven an effective way of addressing this problem. As commonly used, a window system partitions a single physical display into a number of different virtual displays. It is our objective to demonstrate that the model is also useful when applied to input.

We show how the surface of a single input device, a tablet, can be partitioned into a number of virtual input devices. The demonstration makes a number of important points. First, it demonstrates that such usage can improve the power and flexibility of the user interfaces that we can implement with a given set of resources. Second, it demonstrates a property of tablets that distinguishes them from other input devices, such as mice. Third, it shows how the technique can be particularly effective when implemented using a touch sensitive tablet. And finally, it describes the implementation of a prototype an "input window manager" that greatly facilitates our ability to develop user interfaces using the technique.

The research described has significant implications on direct manipulation interfaces, rapid prototyping, tailorability, and user interface management systems.

Buxton, W. (1990). Smoke and Mirrors. Byte,15(7), 205-210.

An approach to evaluating user interfaces is presented. It introduces a metaphor of the system as three mirrors, and argues that design quality is directly correlated to the fidelity of the reflection in each of the three mirrors. Each system mirror "reflects" a different aspect of the user: motor/sensory properties, cognitive and problem solving structures, and socio-political structures. The claim is made that unless the system accurately reflects the user in all three, the design is inadequate. By these criteria, it is concluded that the self-congratulatory attitude of purveyors of GUIs is premature, and that the standards of current design are woefully inadequate.

Buxton, W. (1990). A three state model of graphical input. In D. Diaper et al. (Eds), Human-Computer Interaction - INTERACT '90, Elsevier Science Publishers B.V. (North-Holland), 449-456.

A model to help characterize graphical input is presented. It is a refinement of a model first introduced by Buxton, Hill and Rowley (1985). The importance of the model is that it can characterize both many of the demands of interactive transactions, and many of the capabilities of input transducers. Hence, it provides a simple and usable means to aid finding a match between the two.

After an introduction, an overview of approaches to categorizing input is presented. The model is then described and discussed in terms of a number of different input technologies and techniques.

Buxton, W. (1990). The natural language of interaction: A perspective on non-verbal dialogues. In Laurel, B. (Ed.). The Art of Human-Computer Interface Design, Reading, MA: Addison-Wesley. 405-416.

The argument is made that the concept of "natural language understanding systems" should be extended to include non-verbal types of dialogue. The claim is made that such interfaces are, in many ways, more natural than those based on words. Furthermore, it is argued that the hopes for verbal natural language systems are out of proportion with the potential gains when compared to natural non-verbal interfaces. The benefits of non-verbal natural language systems can be delivered by technology available today, and in general, the benefits will most likely generally exceed those of verbal interfaces (if and when they ever become generally available).

Buxton, W. (1993). HCI and the inadequacies of Direct Manipulation systems. SIGCHI Bulletin, 25(1), 21-22.

The Direct Manipulation (DM) style of user interface made popular by the Macintosh is becoming a de facto standard. Rather than being taken as a point of departure, it appears to be taken more as a standard to achieve. Using the specification of scope as an example, DM interfaces are shown to be deficient in supporting a transaction fundamental to word processing, information retreival and CAD. This essay is a plea for designers to break out of the complacency that surrounds the DM approach. It also calls into question the methodologies of HCI for the very limited degree to which they have challenged the DM approach and their paucity of ideas for generating strong new alternatives.

Buxton, W. (1994). Human skills in interface design. In L.W. MacDonald & J. Vince (Eds.). Interacting with virtual environments. New York: Wiley, 1-12.

The thesis of this paper is that we should evaluate technology in terms of the fidelity with which it reflects human capabilities on three levels:

* physical: how we are built and what motor/sensory skills we possess;

* cognitive: how we think, learn, solve problems and what cognitive skills we possess;

* social: how we relate to our social milieu, including group structure and dynamics, power, politics, and what social skills we possess.

Our metaphor is one of three separate mirrors, each reflecting one of these levels. In order to be judged acceptable, designs must provide an acceptable degree of fidelity in how they reflect each of these three aspects of human makeup and activity. The benefit is in how the model can provide a simple but valuable test that can be used during the design process. We now look at each of these mirrors in turn.

Fitzmaurice, G.W. (1993) Situated Information Spaces and Spatially Aware Palmtop Computers, Communications of the ACM, Vol. 36, No. 7, 38-49.

No longer will we need to be tethered to a stationary computer workstation to browse electronic databases or synthetic 3-D information spaces transformed onto a 2-D display surface. Instead, we will browse, interact and manipulate the electronic information within the context and situation in which the information originated and holds the strongest meaning. A small, portable, high fidelity display and palmtop computer should be spatially aware and act as a window onto the 3-D situated information space -- providing a bridge between the computer synthesized data and physical objects. The Chameleon prototype explores some of the combined input controller and output display paradigm needed to visualize and manipulate the 3-D situated information spaces. A set of computer augmented applications are described to reveal research issues and illustrate the need for a device similar to the Chameleon.

Fitzmaurice, G.W., Zhai, S. and Chignell, M.H. (1993) Virtual Reality for Palmtop Computers, ACM Transactions on Information Systems, Vol. 11, No. 3, 197-218.

We are exploring how virtual reality theories can be applied toward palmtop computers. In our prototype, called the Chameleon, a small 4 inch hand-held monitor acts as a palmtop computer with the capabilities of a silicon graphics workstation. A 6D input device and a response button are attached to the small monitor to detect user gestures and input selections for issuing commands. An experiment was conducted to evaluate our design and to see how well depth could be perceived in the small screen compared to a large 21 inch screen, and the extent to which movement of the small display (in a palmtop virtual reality condition) could improve depth perception. Results show that with very little training, perception of depth in the palmtop virtual reality condition is about as good as corresponding depth perception in a large (but static) display. Variations to the initial design are also discussed, along with issues to be explored in future research. Our research suggests that palmtop virtual reality may support effective navigation, search and retrieval in rich, portable, information spaces.

Hamann, G.E., Shein, G.F., and Milner, M., (1990). Nod at your computer: Switchless selection techniques using a headpointing device, Proceedings of the 12th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Philadelphia, Pennsylvania, 2296-2297.

Current graphical user interfaces often require very fine hand control in order to manipulate a typical pointing device such as a mouse or a trackball. This can be a barrier to many people with physical disabilities who do not have such fine control with their hands. In this paper, the emulation of a mouse by a headpointing device is described. In particular, two switchless techniques are described in which head gestures are used to emulate the mouse buttons.

Hardock, G. (1993). A marking-based text editing system for collaborative writing. M.Sc. Thesis, Department of Computer Science, University of Toronto.

Markings have been used to communicate to people, in the form of markup annotations, and to communicate to computers, via a machine-understandable marking language, but these two uses have never been integrated. In order to investigate such an integration of these two uses of markings, we designed, implemented, and user tested a marking based collaborative text editing system termed MATE for Markup Annotator / Text Editor. This exploratory research can be applied to three areas: asynchronous collaborative writing, the visibility of markings, and interaction languages which can be understood by both people and computers. In addition to these research topics, we are also interested how each topic interacts with the others.

Through the design, implementation, and usability testing of MATE, many insights were made:

- The properties of markings, visibility in particular, have not been fully exploited and allow marks to be used in many new ways.

- Common interaction languages are very useful even when the computer can only understand a limited subset of the language.

- Asynchronous collaborative writing can benefit in many ways from the integration of annotations and editing commands.

Rather than studying each research area in isolation, we concentrated on a specific application which enabled us to find new insights in each area from our knowledge of the other areas.

Hardock, G. (1991). Design issues for line-driven text editing / annotation systems. Proceedings of Graphics Interface '91, 77-84.

Recent research on Line Driven interfaces indicates that there are many potential benefits and applications of such interfaces. But there are still many unknown and unsolved issues to be addressed. This paper examines some of these issues in the context of an asynchronous collaborative text editing system, termed MATE, currently under design and implementation.

Hardock, G., Kurtenbach, G., and Buxton, W. (1993). A Marking Based Interface For Collaborative Writing. Proccedings of UIST'93, 259-266.

We describe a system to support a particular model of document creation. In this model, the document flows from the primary author to one or more collaborators. They annotate it, then return it to the author who makes the final changes. Annotations are made using conventional marks, typically using a stylus. The intent is to match the flow of paper documents that we observed in the everyday world, and how they are marked up. The system is very much modeled on Wang Freestyle (Perkins, Blatt, Workman and Ehrlich, 1989; Francik and Akagi, 1989; & Levine and Ehrlich, in press). Our contribution is to incorporate mark recognition into the system and to explore some novel navigation tools that are enabled by the higher-level data structures that we use. The system is described and the results of our limited user-testing reported.

Kabbash, P., Buxton, W.& Sellen, A. (1994), Two-handed input in a compound task. To appear in Proceedings of CHI '94, 417-423.

Four techniques for performing a compound drawing/color selection task were studied: a unimanual technique, a bimanual technique where different hands controlled independent subtasks, and two other bimanual techniques in which the action of the right hand depended on that of the left hand. We call this latter class of two-handed technique "asymmetric dependent," and predict that because tasks of this sort most closely conform to bimanual tasks in the everyday world, they would give rise to the best performance. Results showed that one of the asymmetric bimanual techniques, called the Toolglass technique, did indeed give rise to the best overall performance. Reasons for the superiority of this technique are discussed in terms of their implications for design. These are contrasted with other kinds of two-handed techniques, and it is shown how, if designed inappropriately, two hands can be worse than one.

Kabbash, P., MacKenzie, I.S. & Buxton, W. (1993). Human performance using computer input devices in the preferred and non-preferred hands. Proceedings of InterCHI '93, 474-481.

Subjects' performance was compared in pointing and dragging tasks using the preferred and non-preferred hands. Tasks were tested using three different input devices: a mouse, a trackball and a tablet-with-stylus. The trackball had the least degradation across hands in performing the tasks, however it remained inferior to both the mouse and stylus. For small distances and small targets, the preferred hand was superior. However, for larger targets and larger distances, both hands performed about the same. The experiment shows that the non-preferred is more than a poor approximation of the preferred hand. The hands are complimentary, each having its own strength and weakness. One design implication is that the non-preferred hand is well suited for tasks that do not require precise action, such as scrolling.

Kurtenbach, G. (1993). The design and evaluation of marking menus. Ph.D. Thesis, Department of Computer Science, University of Toronto.

This research focuses on the use of hand drawn marks as a human-computer input technique. Drawing a mark is an efficient command input technique in many situations. However, marks are not intrinsically self-explanatory as are other interactive techniques such as buttons and menus. This research develops and evaluates an interaction technique called marking menus which integrates menus and marks such that both self-explanation and efficient interaction can be provided.

A marking menu allows a user to perform a menu selection by either popping up a radial menu and then selecting an item, or by drawing a straight mark in the direction of the desired menu item. Drawing a mark avoids popping up the menu. Marking menus can also be hierarchic. In this case, hierarchic radial menus and zig-zag marks are used. Marking menus are based on three design principles: self-revelation, guidance and rehearsal. Self-revelation means a marking menu reveals to a user what functions or items are available. Guidance means a marking menu guides a user in selecting an item. Rehearsal means that the guidance provided by the marking menu is a rehearsal of making the mark needed to select an item. Self-revelation helps a novice determine what functions are available, while guidance and rehearsal train a novice to use the marks like an expert. The intention is to allow a user to make a smooth and efficient transition from novice to expert behavior.

Kurtenbach, G. & Baudel, T. (1992). Hypermark: Issuing commands by drawing marks in Hypercard. CHI'92 Posters and Short Talks, p64

Pen-based interfaces that use markings to issue commands are becoming more popular every day. The advantages of markings as commands can also be used in traditional mouse-based interfaces. We have developed a system called "HyperMark" which allows markings to be used in Apple's HyperCard. For example, if HyperMarks are added to a screen button, not only does a button react to a mouse press, but marks can also be drawn on the button which trigger other actions. This results in fewer buttons and faster interactions in some cases. In effect, HyperMarks are similar to pop-menus where additional functions are "hidden" under a button until popped up. However, with HyperMarks, a user does not have to wait for menu pop-up, visually search the menu and point to an item. Instead, a mark triggers the item directly and quickly. Our intention is that ordinary HyperCard users/programmers can incorporate markings into their own HyperCard stacks.

Two types of marking recognition systems can be used in HyperMark. One system is a user trainable gesture recognizer developed by Rubine which we have ported to HyperCard. In Rubine's system a user can create their own vocabulary of markings and train the recognizer with several examples of each marking. The other recognition technique called "marking menus", developed by Kurtenbach, has a preset vocabulary of markings. This vocabulary of marks consists of straight stroke marks distinguished by the angle of the stroke. Although this marking set is very limited, pie menus are used in conjunction to help a user learn and remember the associations between marks and commands.

This poster presents the design issues concerning user programming and use of either of these systems in the context of HyperMark. Furthermore, we examine how these design issues apply to marking based interfaces in general. The major issues are: how much programming effort is required by a user to make use of one of these systems? How easy to use is each system? How self explanatory are they? How easily and successfully can marks be drawn in either system?

We have found that either system has its advantages and disadvantages and that the systems can successfully be used together. The advantage of the Rubine's system is that user can create one's own custom set of markings. However the disadvantage is that it is then the user's responsibility to design an unambiguous mark set and provide examples to train the system. Also, because markings are not self-revealing like buttons or menus, some sort of user built explanation must also be created. In contrast, with marking menus, because the marking set is preset, no marking set need be designed and ambiguity is not a problem. Furthermore, the pie menu aspect of marking menus provides built in help. Adding a marking menu is as simple as adding a pop-up menu. Thus for very little implementation overhead a user can obtain the benefits of using marks.

Kurtenbach, G. & Buxton, W. (1991). GEdit: a testbed for editing by contiguous gesture. SIGCHI Bulletin, 23(2), 22 - 26.

GEdit is a prototype graphical editor that permits you to create and manipulate three simple types of objects using shorthand and proof-reader's type gestures. Using hand-drawn symbols, the user can add, delete, move and copy objects. Objects can be manipulated individually or in groups. Groups are specified with hand-drawn circling symbols. GEdit provides a toy world that serves to demonstrate a number of concepts:

* The use of proof-reader's like symbols (rather than the more common alphanumeric character recognition);

* The use of where a symbol is drawn as well as what symbol is drawn to determine intent;

* The use of compound symbols, that is, symbols (such as move) that have more than one "token" embedded in a single continuous line, such as the command (move), what is to be moved (direct object), and where it is to be moved to (indirect object).

Kurtenbach, G. & Buxton, W. (1991). Integrating mark-up and direct manipulation techniques. Proceedings of the Fourth ACM SIGGRAPH Symposium on User Interface Technology (UIST), 137-144.

The direct manipulation paradigm has been effective in helping designers create easy to use mouse and keyboard based interfaces. The development of flat display surfaces and transparent tablets are now making interfaces possible where a user can write directly on the screen using a special stylus. Part of the intention of these interfaces is to exploit user's existing handwriting, mark-up and drawing skills. This paper reports on a test-bed program which we are using for exploring hand-marking types of interactions and their integration with direct manipulation interactions.

Kurtenbach, G. & Buxton, W. (1993). The limits of expert performance using hierarchic marking menus. Proceedings of InterCHI '93, 482-487.

A marking menu allows a user to perform a menu selection by either popping-up a radial (or pie) menu, or by making a straight mark in the direction of desired menu item without popping-up the menu. A hierarchic marking menu uses hierarchic radial menus and "zig-zag" marks to select from the hierarchy. This paper experimentally investigates the bounds on how many items can be in each level and how deep the hierarchy can be before using a marking to select an item becomes too slow or prone to errors.

Kurtenbach, G. & Buxton, W. (1994). User learning and performance with marking menus. Proceedings of CHI '94, 258-264.

A marking menu is designed to allow a user to perform a menu selection by either popping-up a radial (or pie) menu, or by making a straight mark in the direction of the desired menu item, without popping-up the menu. Previous evaluations in laboratory settings have shown the potential of marking menus. This paper reports on a case study of user behaviour with marking menus in a real work situation. This study demonstrates the following: First, marking menus are used as designed. When users become expert with the menus, marks are used extensively. However, the transition to using marks is not one-way. Expert users still switch back to menus to refresh their memory of menu layout. Second, marking is an extremely efficient interaction technique. Using a mark on average was 3.5 times faster than selection using the menu. Third, design principles can be followed that make menu item/mark associations easier to learn, and interaction efficient.

Kurtenbach, G., Moran, T. & Buxton, W. (in press). Contextural animation of gestural comands. To appear in Proceedings of Graphics Interface '94, 16-20 May, 1994, Banff Alberta.

Kurtenbach, G., Sellen, A. & Buxton, W. (1992). An empirical evaluation of some articulatory and cognitive aspects of "marking menus." To appear in Human Computer Interaction.

We describe "marking menus", an extension of "pie menus" particularly well-suited for stylus based interfaces. Pie menus are circular menus subdivided into sectors, each of which might correspond to a different command. Marking menus are pie menus in which the path of the cursor during selection leaves an ink trail. Thus, selecting a sector from a marking menu creates a visual mark similar to a pen stroke on paper. Marking menus are also unique in that they ease the transition from novice to expert user. Novices can "pop-up" a menu and make a selection, whereas experts can simply make the corresponding mark without waiting for the menu to appear.

This paper describes an experiment in which we explored both articulatory and cognitive aspects of marking menus, across three different kinds of input devices (mouse, trackball and stylus) and for different numbers of items per menu. By "articulatory aspects" we mean how well subjects could execute the physical actions necessary to select from marking menus. Articulatory aspects were investigated by presenting one group of subjects with the task of selecting from fully visible or "exposed" menus. Because one feature of marking menus is that users should be able to select from them without seeing the menus (or by "marking ahead"), we also ran two groups of subjects with invisible or "hidden" pie menus: one group with an ink trail, and one without. These subjects were therefore faced with the task of either mentally representing the menu, or by associating marks with the commands they invoke through practice. These then are the "cognitive aspects" to which we refer. We discuss the findings for all three groups of subjects with respect to number of menu items, input device, analysis of markings used, and learning.

MacKenzie, I. S. (1992). Fitts' law as a research and design tool in human-computer interaction. Human Computer Interaction, 7(1), 91-139.

According to Fitts' law, human movement can be modeled by analogy to the transmission of information. Fitts' popular model has been widely adopted in numerous research areas, including kinematics, human factors, and, recently, human- computer interaction. The present study provides an historical and theoretical context for the model, including an analysis of problems that have emerged through the systematic deviation of observations from predictions. Refinements to the model are described, including a formulation for the index of task difficulty that is claimed to be more theoretically sound than Fitts' original formulation. The model's utility in predicting the time to position a cursor and select a target is explored through a review of six Fitts' law studies employing devices such as the mouse, trackball, joystick, touchpad, helmet-mounted sight, and eye tracker. An analysis of the performance measures reveals tremendous inconsistencies, making across-study comparisons difficult. Sources of experimental variation are identified to reconcile these differences.

MacKenzie, I. S. (1992). Movement time prediction in human-computer interfaces. Proceedings of Graphics Interface '92, 140-150.

The prediction of movement time in human-computer interfaces as undertaken using Fitts' law is reviewed. Techniques for model building are summarized and three refinements to improve the theoretical and empirical accuracy of the law are presented. Refinements include (1) the Shannon formulation for the index of task difficulty, (2) new interpretations of "target width" for two- and three-dimensional tasks, and (3) a technique for normalizing error rates across experimental factors. Finally, a detailed application example is developed showing the potential of Fitts' law to predict and compare the performance of user interfaces before designs are finalized.

MacKenzie, I. S. (1989). A note on the information-theoretic basis for Fitts' law. Journal of Motor Behavior, 21, 323-330.

Fitts' law is an information-theoretic view of human motor behavior developed for Shannon's Theorem 17, a fundamental theorem of communications systems. Using data from Fitts' original experiments, we demonstrate that Fitts' choice of an equation that deviates slightly from the underlying principle is perhaps unfounded, and that the relationship is improved using an exact adaptation of Shannon's equation.

MacKenzie, I. S. & Buxton, W. (1992). Extending Fitts' law to two-dimensional tasks. Proceedings of CHI '92, 219-226.

Fitts' law, a one-dimensional model of human movement, is commonly applied to two-dimensional target acquisition tasks on interactive computer systems. For rectangular targets, such as words, it is demonstrated that the model can break down and yield unrealistically low (even negative!) ratings for a task's index of difficulty (ID). The Shannon formulation is shown to partially correct this problem, since ID is always >= 0 bits. As well, two alternative interpretations of "target width" are introduced that accommodate the two-dimensional nature of tasks. Results of an experiment are presented that show a significant improvement in the model's performance using the suggested changes.

MacKenzie, I.S. & Buxton, W. (1993). A tool for the rapid evaluation of input devices using Fitts' law models. SIGCHI Bulletin, 25(3), 58-63.

A tool for building Fitts' law models is described. model builder runs on the Apple Macintosh using any device which connects to the Apple Desktop Bus. After 16 blocks of trials taking about 4-5 minutes, the program provides an immediate (albeit tentative) statistical analysis, showing the coefficients in the prediction equation, the coefficient of correlation, and a regression line with scatter points. model builder can be retrieved anonymously by researchers, educators, developers, or anyone with access to internet through file-transfer-protocol (ftp).

MacKenzie, I. S., Sellen, A. & Buxton, W. (1991). A comparison of input devices in elemental pointing and dragging tasks. Proceedings of CHI '91, ACM Conference on Human Factors in Software, 161-166.

An experiment is described comparing three devices (a mouse, a trackball, and a stylus with tablet) in the performance of pointing and dragging tasks. The paper shows how Fitts' law can model both tasks, and illustrates that for all devices the index of performance is lower when dragging than when pointing. The results lay the foundation for extending models to better apply them to direct manipulation systems. The data will also assist designers in making better device selections.

McQueen J.C., MacKenzie I.S., Nonnecke B., Riddersma S., Meltz M., (1994). A comparison of four methods of numeric entry on pen-based computers. Proceeding of Graphics Interface '94, 75-82.

Four numeric entry methods for pen-based computers were comparedwith respect to accuracy and speed. Sixteen subjects enterednumbers on a digitizing display using four conditions: handwriting(with recognition software), keypad taping, pie pad, and movingpie menu. Keypad tapping was the most accurate (98.8%) andfastest (30.4 wpm) entry method. It was also most preferredby subjects. Handwriting was nearly as preferred as keypadtapping, even though it was substantially less accurate (89.6%)and slower (18.5 wpm). The pie menu conditions were least preferred,least accurate, and slowest. However, some subjects did achievesuperior performance on the pie menu conditions; thus, withsufficient practice, pie menu entry may be a valid alternativeto handwriting for numbers. These results call into question thepresumed superiority of handwriting as the preferred entry methodon pen-based computers.

McQueen J.C., MacKenzie I.S., Nonnecke B., Riddersma S., Meltz M. (1994). A comparison of three methods of character entry on pen-based computers. To appear in the Proccedings of the 1994 Conference of the Human Factors and Ergonomics Society, October 24-28, Nashville, Tennessee.

Methods for entering text on pen-based computers were compared withrespect to speed, accuracy, and user preference. Fifteen subjectsentered text on a digitizing display tablet using three methods:hand printing, QWERTY-tapping, and ABC-tapping. The tapping methodsused display-based keyboards, one with a QWERTY layout, the otherwith two rows of 13 characters in alphabetic order. ABC-tapping hadthe lowest error rate (0.6%) but was the slowest entry method(12.9 wpm). It was also the least preferred input method. TheQWERTY-tapping condition was the most preferred, the fastest(22.9 wpm), and had a low error rate (1.1%). Although subjects alsoliked hand printing, it was 41% slower than QWERTY-tapping and had avery high error rate (8.1%). The results suggest that characterrecognition on pen-based computers must improve to attract walk-upusers, and that alternatives such as tapping on a QWERTY image areeffective input methods.

Matias, Edgar, MacKenzie[, ]I. Scott & Buxton[, ] William (1993), Half-QWERTY: A one-handed keyboard facilitating skill transfer from QWERTY. Proceedings of InterCHI '93, 88-94.

Half-QWERTY is a new one-handed typing technique, designed to facilitate the transfer of two-handed typing skill to the one-handed condition. It is performed on a standard keyboard, or a special half keyboard (with full-sized keys). In an experiment using touch typists, hunt-and-peck typing speeds were surpassed after 3-4 hours of practice. Subjects reached 50% of their two-handed typing speed after about 8 hours. After 10 hours, all subjects typed between 43% and 76% of their two-handed speed, ranging from 23.8 to 42.8 wpm. These results are important in providing access to disabled users, and for the design of compact computers. They also bring into question previous research claiming finger actions of one hand map to the other via spatial congruence rather than mirror image.

Sellen, A., Kurtenbach, G. & Buxton, W. (1990). The role of visual and kinesthetic feedback in the prevention of mode errors. In D. Diaper et al. (Eds), Human-Computer Interaction - INTERACT '90, Elsevier Science Publishers B.V. (North-Holland), 667-673.

The use of visual and kinesthetic feedback in preventing mode errors was investigated. Mode errors were defined in the context of text editing as attempting to issue navigation commands while in insert mode, or attempting to insert text while in command mode. Twelve novices and twelve expert users of the Unix-based text editor vi performed a simple text editing task in conjunction with a distractor task in four different conditions. These conditions consisted of comparing the use of keyboard versus foot pedal for changing mode, crossed with the presence or absence of visual feedback to indicate mode. Both visual and kinesthetic feedback were effective in reducing mode errors, although for experts visual feedback was redundant given that they were using a foot pedal. Other measures of system usability indicate the superiority of the use of a foot pedal over visual feedback in delivering system state information for this type of task.

Sellen, A., Kurtenbach, G. & Buxton, W. (1992). The prevention of mode errors through sensory feedback. Human Computer Interaction, 7(2), 141-164.

The use of different kinds of feedback in preventing mode errors was investigated. Two experiments examined the frequency of mode errors in a text editing task where a mode error was defined as an attempt to issue navigation commands while in insert mode, or an attempt to insert text while in command mode. In Experiment 1 the effectiveness of visual versus kinesthetic feedback was compared in four different conditions: the use of keyboard versus foot pedal for changing mode, crossed with the presence or absence of visual feedback to indicate mode. The results showed kinesthetic feedback to be more effective than visual feedback in reducing the cognitive load associated with mode changes. Experiment 2 tested the hypothesis that the superiority of this kinesthetic feedback was due to the fact that the foot pedal required subjects actively to maintain insert mode. The results confirmed that the use of a non-latching foot pedal for switching modes provided a more salient source of information on mode state than the use of a latching pedal. We suggest that user-maintained mode states prevent mode errors more effectively than system-maintained mode states.

Zhai. S. (1993). Investigation of feel for 6DOF inputs: isometric and elastic rate control for manipulation in 3D environments, Proc. of the Human Factors and Ergonomics Society 37th Annual Meeting, Oct. 11-15.

An increasing need exists for both a theoretical basis and practical human factors guidelines for designing and selecting high degree-of-freedom (DOF) computer input devices for 3D interactive environments such as telerobotic and virtual reality systems. This study evaluates elastic versus isometric rate control devices, in a 3D object positioning task. An experiment was conducted with a stereoscopic virtual reality system. The results showed that the elastic rate controller facilitated faster task completion time in the first of four phases of the experiment. The results are discussed in light of psychomotor literature. While the richer proprioceptive feedback afforded by an elastic controller is necessary for achieving superior performance in the early stages of learning, subjects performed equally well with the isometric controller in later learning stages. The study provides evidence to support a theory of skill shift from closed-loop to open-loop behaviour as learning progresses.

Zhai, S., Buxton, W. & Milgram, P. (1994). The "Silk Cursor": Investigating transparency for 3D target acquisition. Proceedings of CHI '94, 459-464.

This study investigates dynamic 3D target acquisition. The focus is on the relative effect of specific perceptual cues. A novel technique is introduced and we report on an experiment that evaluates its effectiveness.

There are two aspects to the new technique. First, in contrast to normal practice, the tracking symbol is a volume rather than a point. Second, the surface of this volume is semi-transparent, thereby affording occlusion cues during target acquisition.

The experiment shows that the volume/occlusion cues were effective in both monocular and stereoscopic conditions. For some tasks where stereoscopic presentation is unavailable or infeasible, the new technique offers an effective alternative.

Zhai, S. & Milgram, P. (1993). Human performance evaluation of isometric and elastic rate controlleR in a 6 dof tracking task, Proc.SPIE vol. 2057: Telemanipulator Technology and Space Telerobotics, Boston, MA, USA, Sept. 7-10.

The control feel of the master controller in a telemanipulation system has a critical effect on human-machine system performance. Many theoretical and practical human factors issues on hand controller design and selection remain to be explored. In this paper, two types of hand controllers, elastic versus isometric rate controllers, were studied in a 6 degree-of-freedom pursuit tracking experiment. 26 subjects participated in the experiment. The results showed that the elastic rate controller facilitated significantly lower tracking error than the isometric rate controller, presumably due to richer proprioceptive feedback afforded by the elastic device. It was also found that, although the elastic controller was superior to the isometric controller in general, the magnitude of performance differences between the two controllers decreased as subjects gained more experience, supporting the theory that human manipulation shifts from more closed-loop behaviour towards open loop skills.

Zhai, S. & Milgram, P. (1993). Human Performance Evaluation of Manipulation Schemes in Virtual Environments, Proc VRAIS'93: the first IEEE Virtual Reality Annual International Symposium, Seattle, USA, Sept.

This paper presents the results of one of the first experiments in a research programme aimed at systematically investigating manipulation schemes for spatial input, from a human factors point of view. A three dimensional design space model is proposed as a framework for such investigations and four options within this model are tested in a 6 degree-of-freedom target acquisition task within a virtual environment. Experimental results indicate strong performance advantages for isometric sensing combined with rate control and for isotonic sensing combined with position control. A strong interaction between sensing mode and mapping function is found. The findings are discussed in relation to the literature on spatial manipulation.

Zhai, S., Milgram, P., & Drascic, D. (1993). An Evaluation of Four 6 Degree-Of-Freedom Input Techniques, Adjunct Proc. INTERCHI'93 (short paper), Amsterdam, The Netherlands, April .

An investigation was carried out to compare the efficiency of a variety of 6 Degree of freedom input techniques. Subjects performed a target acquisition task, in which they manipulated objects in 3D virtual environment. Results clearly indicated comparable superiority of both isotonic position control and isometric rate control; one is cautioned, however, that simply to compare sensing modes (isometric vs isotonic) or mapping functions (position vs rate control) is likely to be misleading.

IRG Team Members

Bill Buxton is an associate professor in the Department of Computer Science. Besides directing the IRG, he is scientific director of the Ontario Telepresence Project. He has had a long association with Xerox PARC, to which he currently consults half time. Buxton has a strong interest in the relationship between technology and the arts and culture, as well as research interests in human input, gesture, the use of nonspeech audio at the interface, telepresence and multimedia documents and systems.

George Fitzmaurice is a Ph.D student in the Department of Computer Science at the University of Toronto. He holds a B.Sc. in mathematics with computer science from M.I.T. and a M.Sc. in computer science from Brown University. He has worked as a researcher at Brown University and at Xerox PARC. His research interests include computer augmented environments, ubiquitous computing, and applying everyday skills towards novel 2D and 3D input devices and interaction techniques.

Gil Hamann is currently an M.A.Sc. student in the Institute of Biomedical Engineering at the University of Toronto. His research is carried out at the Hugh MacMillan Rehabilitation Centre. He received his B.A.Sc. degree in Electrical Engineering at the University of Calgary. His current research interests include human-computer interaction, particularly in providing accessible devices and interfaces for people with disabilities.

Gary Hardock is a recent M.Sc. graduate in the Department of Computer Science at the University of Toronto. He holds a B.A.Sc. in Systems Design Engineering from the University of Waterloo. His current research interests include human-computer interaction, in particular input devices, gestural interactions and stylus-driven input.

Paul Kabbash is an M.Sc. student in the Department of Comtputer Science ath the University of Toronto. He has a background in music (PhD., Yale 1983) and taught music theory and piano for several years. His current research involves two-handed input.

Gordon Kurtenbach is a recent Ph.D. graduate in computer science at the University of Toronto. He holds a B.Sc. from the University of Sask. and a M.Sc. from the University of Toronto in computer science. He has worked as a researcher at the University of Sask., in the Advanced Technology Group at Apple Computer and at Xerox PARC, where he is now a Post-Doc. His research interests are in human computer interaction, in particular, input techniques and the role of gesture and user expectations, multi-media interfaces and computer aided music.

Scott MacKenzie is on the faculty in the Department of Computing and Information Science University of Guelph where he continues to be an active member of the IRG. Previously he was a research associate at the University of Toronto, and taught computer engineering at Seneca College of Applied Arts and Technology. He recently completed his Ph.D. in Education at the Ontario Institute for Studies in Education at the University of Toronto. He holds a Bachelor of Music degree (Queen's University), diploma in Electronics Engineering Technology (Durham College), and a Master of Education degree (University of Toronto). Research interests include performance measurement, prediction, and modelling for human-computer interaction. In his spare time he runs marathons.

Craig McQueen is currently a Masters student in Computer Science atthe University of Toronto. He did his undergraduate degree inSystems Design Engineering at the University of Waterloo. Hisresearch interests are using sound in the interface and gesture-based input. Combining the two is the focus of his Masters thesisin that he is creating a gesture-based input mechanism thatuses sound feedback.

Edgar Matias is an undergraduate student at the University of Toronto. He is the inventor of the Half-QWERTY one-handed keyboard, and is currently researching its properties. He is also an entrepreneur, having appeared on the CBC busines show, Venture.

Abigail Sellen is a former Research Associate at the Computer Systems Research Institute of the University of Toronto and is currently a research scientist at Rank Xerox EuroPARC, Cambridge, England (where she continues as an active member of the IRG). She holds a B.Sc. in Psychology (Univ. of Toronto), M.A. in Psychology (UCSD), M.A.Sc. in Industrial Engineering (Univ. of Toronto), and a Ph.D. in Cognitive Science and Psychology (UCSD). She has worked for the Human Interface Group of Apple Computer and the Design Interpretive Group of Bell Northern Research. She has also consulted for Xerox PARC and Human Factors North. Current research interests are human error and reliability, human action systems, human-computer interaction, computer-supported collaborative work and mental workload.

Mark Tapia is a Ph.D. student in the Department of Computer Science at the University of Toronto. He holds a B.S. in mathematics from Stanford, and did his M. Sc. in the area of symbolic algebra at the University of Toronto. His Ph. D. research is in the area of shape grammars. His professional work experience includes software engineering, electronic mail systems, and user interface testing and documentation. His research interests are formal systems for specifying architectural/graphic design tools for exploring graphical design spaces.

Shumin Zhai is a Ph.D student in the Department of Industrial Engineering at the University of Toronto. He holds a BS and a MS degree in electrical engineering from XiDian University, Xian, China. From 1984 to 1989, he was on the faculty of XiDian University where he taught and conducted research in the area of control systems. His current research interest is human-machine interaction through 3D interfaces, including 6 degree-of-freedom input techniques, stereoscopic displays, tele-manipulation and virtual reality.