Social Science Models and Strategies for Telepresence User Studies

Gale Moore - February 1993

OTP-93-01

The Role of Social Sciences in the Design Process

A basic premise of the Ontario Telepresence Project is that information and communication technologies are inherently social in nature. Consequently, it is essential in the design of these technologies to take advantage of the social skills and the knowledge of social protocols that the future users already possess, building on existing skills rather than demanding or forcing entirely new behaviours. Similarly, we need to incorporate an understanding of work and workplace variables in the design. The specification and elaboration of the relationships among work variables is a primary goal of our research. One size is unlikely to fit all and the successful development, adoption and diffusion of the Telepresence system must take into account existing work practices and organizational culture.

The first year of Ontario Telepresence Project ended on Dec. 31, 1992. It is important, however, to note that the social sciences component only became institutionalized in the project in July 1992 when a full-time social scientist was appointed.

Previous Research in this Area

The Ontario Telepresence Project has in part grown out of work on media space carried out at the University of Toronto on the CAVECAT (Computer Audio Video Enhanced Collaboration and Telepresence) System. The CAVECAT System consisted of a number of enhanced workstations connected by a digital + audio+ video network. Each workstation consisted of a personal computer, a TV monitor, a camcorder, a pair of speakers and a microphone. A 4x1 video board allowed the display of composite images of up to four sites. The heart of the system is the switching network patterned after the iiif server developed at Rank Xerox Europarc ( Buxton, Moran, 1990/Milligan, 1989.). Two faculty offices, the system programmer's office and the graduate student work area were linked and constituted the first user group. As a result of both user studies and experimental research on video-mediated communication, the system was modified. The research published on this project should be consulted for further details (see for example, Buxton & Sellen, 1991[3 ], Sellen, 1991[4] ,Mantei et al, 1991[5] ).

CurrentFocus: From the Laboratory to the Workplace

The Ontario Telepresence Project builds on the work of the earlier CAVECAT Project taking advantage of research findings, particularly in the areas of cognitive psychology and human computer interface. The focus of the current project is on moving the design from the laboratory into the workplace. Central to this process is research into the social aspects of diffusion. We know, for example, that information technology can significantly alter the nature of human communication patterns, but exactly how, and under what conditions remains to be specified. As the result of the CAVECAT studies we also undertand that there are many communication variables that were not considered in the original design. For example, it is easy to take for granted aspects implicit in face-to-face communication such as the physical presence of someone in an office implying a desire to communicate, or the meaning of nonverbal gestures of individuals in a meeting. Metaphors for communciation and privacy protection that follow standard, already understood, social practice are being developed. Taking the project beyond the laboratory and the local workplace into external sites broadens our research plan, both substantively and methodologically. In year 2 of the Project we anticipate that we will carry out field studies in a number of sites external to the laboratory, but that we will also continue to do research internally. The rationale for this is as follows:

The Laboratory

There is still much to be learned within a research environment and from users familiar with the system and tolerant of change. In addition, this user group is willing to use and test application software and interface designs that are still under development. The Telepresence Project itself has been structured to encourage the team to work in ways permitted by the technology. For example, the group in Toronto is located in several buildings and in one building split between two floors. Both the engineering group and the social science group are divided between Ottawa and Toronto, and, in these locations, across institutions. One of the scientific directors of the Project works in the south of France, and the other, while residing in Toronto, spends at least one week per month in California. The sites of researchers in Ottawa, and France are now accessible through a codec which permits desktop attendance at meetings and collaborative work, as well as facilitating group meetings. Thus, the Project itself has been set up in such a way that in order to carry out our work we must collaborate and work with colleagues who are at a distance. It is the natural laboratory for testing new ideas, applications, and technology.

The Workplace

The deployment of the technology to sites distant from the research environment presents both opportunities and risks. There are numerous places where this technology could used. Our primary focus is on the application of Telepresence technology in the workplace. We are interested how this technology can be used to support and enhance work activities and to study the impact on work behaviour (e.g., communication patterns) and on the organizations that adopt the technology. We anticipate that our sites for field studies will include business, industry, government and university locations. We have a particular interest in shared work applications (e.g., cscw) and in teleworking. The models for the identification and selection of sites are discussed below. Research projects have begun in two external sites.

The Diffusion of Telepresence Technology

The overall goal of the Ontario Telepresence Project is to take Telepresence technology out of the laboratory to arms-length users and to study how this technology can support and enhance work activities. To achieve this goal in year 3 assumes both robust technology and a reliable set of research methods and instruments. It is towards the development of both the technology and instruments that years 1 and 2 are directed.

In the first year (6 months) of the Project the focus in the social sciences has been in the development of models and in defining a strategy for the user studies that are central to this project. This has included :

Metaphorical Model of Telepresence Technology

The Ontario Telepresence Project is a research project in which two sets of activities are carried out simultaneously. On the one hand there is continual research and development and study of the system in house - the R & D Strand in fig. 1 below. On the other hand, there are field studies that involve the installation and use of a version of the technology in sites remote from the laboratory - the Field Study Strand. Technological development is thus driven by the needs of the social science research team, the results of social science studies and the experiences of users.

The Project's objective is not to overtly develop products or services for the short-term market. Rather, the project seeks, whenever possible, to integrate and repackage existing components rather than re-invent them. It must be understood that what is tolerable, even desirable, in the research environment will not be acceptable in a business or commercial enterprise. Outside the laboratory the technology deployed must be robust and the applications mounted adequately supported. There is flow from the R&D Strand to the Field Study Strand and from the Field Studies Strand back to R & D.

Fig. 1

The next section describes the model developed for the identification of sites.

Model for the Identification and Selection of Sites

The development of the Telepresence system is ongoing and iterative, involving both local researchers as users and users at client sites. The identification and selection of sites cannot, however, be random or selected completely opportunistically for the following reasons. There are two areas of exposure in deploying the technology outside the laboratory - one technical, the other social. In the early stages of the project, social science methodologies are being developed and tested. On the technical side, the Telepresence system has been extended to link several offices across two floors in a buildig at the University of Toronto , but this is the extent of our experience in deploying the system. On both the social science and technical sides, our human resources are limited and factors such as organizational size, diversity, and physical constraints of the building and its location e.g., cable installation, must be taken into account when selecting a site.

Thus, it seems a logical and systemic approach to move carefully in the direction of transferring the technology first to our partners and then to more remote sites. In addition, we hypothesize that we can extend our research through the study of commercial teleconferencing technology. Research in these sites reduces our exposure on the technical side, while generating useful social science data to feed back into the design processes in Telepresence. A second strategy, therefore, is to seize opportunities that are available to study the use of commercial multi media technologies already in the workplace, or being introduced into workplaces to which we can gain access. This provides three opportunities:

Figure 2, below, illustrates how this has been conceptualized.

For example, we are currently working with two field sites. In the first site we are studying a commercial desktop product, VISIT, and in the second, we are installing and studying Telepresence technology.

Broadly, the user studies may be categorized into types:

In this model, in-house studies are conducted with the latest versions of software and hardware and are the least robust. Field Studies are done out-of-house in collaboration with Industrial Partners or other affiliated organizations. Field Trials are done with groups that are arms-length from the Project and require very robust implementations of the technological systems deployed, and the social science research instruments.

When we model these types of user studies as concentric circles as shown in Figure 3 below, we can visualize the various needs and constraints of each kind of study.

Fig. 3

As the radius "R" increases:

Therefore, it is the strategy of the Project to start studies locally and, as the underlying test-bed, applications and social science techniques mature, move toward sites that are more arms length.

Criteria for the Selection of Sites

We expect that our criteria for the selection of sites will change over time. The research will be cumulative, and as we learn more about the social processes around this innovation our requirements for sites will change. However, a general set of guidelines for site selection has been developed.

Research Agenda

In the Telepresence Project several disciplines are included under the umbrella of the social sciences. Our research program draws upon expertise in each of the following disciplines:

Computer science and industrial engineering are not traditional social science disciplines, but are included here because some Project members are also experts in these areas. The Project team powerfully links social science with computer science and engineering. In the case of industrial engineering, some of the human factors research is strongly social in the problems addressed, e.g., studies of proxemics, while in other cases it is technical.

Crucial Variables

The focus of the research in this project is on the use of Telepresence and telepresence-like technologies in the workplace. That is, our focus is on the ways in which these technologies can be used to support and enhance work in organizations, such as government agencies, businesses, etc.

The following variables are the key to comparing how Telepresence is used in different work settings. While all possible variations cannot be studied, the full list can guide site selection and provide an analytic perspective on those that will be investigated.

Distance

Range

Telepresence is clearly aimed at people working together at a distance. We need to investigate at what distance it becomes (a) used to any appreciable extent, and (b) it becomes the primary mode of communication. Research into communications networks, including the CAVECAT predecessor of Telepresence, suggests that the effects of distance are step-wise, rather than continuous. Moreover, at some point, use may decline with increasing distance. It is possible that there will be low intercontinental use because of different workdays and because of the paucity of the face-to-face contact that may be necessary to sustain telecommunications contact.

Examples of distance between work spaces include:

Bringing Branches into the Loop

We question to what extent the increased connectivity afforded by Telepresence technology will incorporate outlying branches into the main decision-making activity of the organization. Alternately, it may further reduce the need for outlying organizational managers, just as there is some evidence that increased communications and computer connectivity may already have led to a decrease in the number and role of middle managers.

Nature of the Labour Force and Work Modes

We expect that our primary focus will be on knowledge workers, that is, those whose occupations are currently categorized as managerial, professional, and technical in official statistics, e.g., the Canadian Census. Furthermore, we expect these knowledge workers will work in two different modes - as a focussed group or as a loosely coupled network (see below). We will evaluate the usefulness of Telepresence for each of these modes of work. We expect that the software, and possibly the hardware, implications of each may be different.

Focused group

A densely-knit workgroup, usually small, which gives all of their attention to solving a set of problems or carrying out a set of activities. In this situation, co-workers have frequent, virtually instant access to each other, and there is not much concern about maintaining privacy and confidentiality within the group. Metaphorically, the group is hard-wired together and is self-sufficient.

Loosely coupled network

Each individual works on a variety of projects, virtually simultaneously. To accomplish the work required for each project, the individual must have access to a continuously shifting set of others. When they are working on one project, they need to control their privacy and confidentiality so that non-relevant others do not have automatic rights to connect with them. To connect with others, each person needs to select from a large list of potential colleagues, and to be able to shift easily between different sets of colleagues.

Note that the same person may move between working in a focused group mode and in a loosely coupled network mode. These are styles of work, and may or may not be stable for any one person or organization.

Work Tasks

It is expected that different types of work tasks and work relationships cause people to select different types of communication media. We are interested in the choice of various communication channel technologies such as telephone, electronic mail, or telepresence technology over face-to-face communication. We are particularly interested in the situations in which the task or the work relationship causes a user to select one medium while another somewhat similar the tasks and the work relationships, we hope to extrapolate reasons and properties of either that caused the choice. For example, if more complex tasks such as computer-programming are done face-to-face whereas administrative tasks are done by electronic mail, then it would be appropriate to work on shared applications that support the complex tasks at a distance and not to work on ones which support administrative tasks.

Organizational/Workplace Culture

An organization or workplace can be thought of as comprising a set of ongoing social groups. These have a structure, a hierarchy and a formal agenda. They also develop a set of customs, and a shared history. They develop a method of doing their work, intermeshed roles and have a shared understanding around their work interaction. Often these groups and their members resist social change because they are buttressed by dense interlocking ways of acting and reciprocal understandings. They are conservative. When new technology is introduced it will have to be fit into the existing social setting, or it risks being rejected. It is thus important to understand the pre-existing culture and structure of the workplace before technology is introduced. Research into the diffusion of innovation shows that if members of a group are able to influence or shape how the technology is introduced, they are more likely to adopt it.

Research Design and Research Methods

A variety of research methods and research designs will be used. The design and methods selected for any particular study will vary according to the specific questions being addressed. They may be broadly characterized as follows:

Experimental Method

In the social sciences, this approach is most common in psychology, but experimental and quasi-experimental designs are, on occasion, used in sociology. This methods is frequently used in HCI: Human Computer Interaction research. In general, this type of research is carried out in the laboratory where conditions can be controlled. For example, in the CAVECAT Project, the predecessor to Telepresence, the social science research focus was in the area of cognitive psychology and the majority of studies used an experimental design.

Surveys

This is perhaps the most commonly used research methodology in the social sciences. Surveys are often the method of choice for gathering attitudinal and perceptual data. This methods involves the administration of a questionnaire or an interview to a sample of respondents. Surveys can be carried out in a variety of ways - in person, by telephone, by post, email, etc. Surveys may include poll-type questions. (i.e., the respondent is required to tick a box), open-ended questions where the respondent is invited to provide comment or a mixture of both types of questions. The results are tabulated and may be manipulated statistically.

User and usability studies

User studies have a long tradition in information science and have been applied to a number of design problems in the library and information science field. This approach has also been used extensively in the field of HCI; Human Computer Interaction. A variety of methods, many drawn from survey research, are used to elicit user requirements, and to understand user attitudes, behaviour and characteristics. In addition, specialized user evaluation methods such as mockups, cognitive walk throughs and usability studies, have been developed. Recently, there has been an attempt to involve the user more directly in the design process. (See Mantei report entitled Telepresence User Interface Design Issues and Solutions for more information on this area)

Field Research

This method has its origins in anthropology and involves going into the 'field', to observe and record what takes place. This approach to research is inductive as opposed to the deductive model of the experimental and survey methodologies. Social science researchers have developed effective techniques for observing, taking notes, and analysing the observations gathered. This approach includes techniques such as participant observation, and the analysis of logs and/or diaries kept by participants in the study, interaction analysis and the writing of ethnographies.

Network Analysis

Social networks are the focus of this research. To a great extent, social networks can be studied in the same way as telephone or computer networks. A social network, like other networks, is a set of nodes and a set of ties linking these nodes, with the particular specification in a social network that these nodes may be persons, groups, corporate entities, clusters of ties of other institutions. Ties, in effect, represent relations of control, cooperation, or flows of resources. Social network analysis has been widely used to study relations within and between organizations. Whole network studies describe the overall structure of relationships in a social system and provide powerful information on how different role relationships link system members. Egocentric network analysis, on the other hand, defines the network from the standpoint of the individual.

Research In Progress

Field Study I

Site: Research organization

Technology: Telepresence technology

Duration: Dec. 1992-Sept. 1993

Research Project:

This is the first installation of the Telepresence system outside the laboratory. A series of studies are being conducted pre and post installation of Telepresence. There are 5 strands in the study design and the foci of the research are on work practices and interface design. (For more information on the latter see the report entitled Telepresence User Interface Design Issues and Solutions in this report) This research provides an opportunity to study adoption and diffusion patterns of this technology, to explore the relationships between media choice and the nature of the work tasks and to further specify our understanding of the importance of context in the deployment of new technologies. Context includes factors such as workplace culture, organizational structure, and labour force characteristics.

Field Study II

Site: A corporation in the communications industry

Technology: VISIT Video (Desk-top Videoconferencing System)

Duration: March 1993-June 1993

Research Project:

This is a much more limited study. In this case, the participating organization has deployed approximately 20 VISIT Video kits across several units in the corporation, and has designated the first 90 days post installation as a field trial. The individuals and groups targeted to receive kits were identified by the corporation according to internal criteria. The members of the group are very loosely connected, and while they perform related activities for the corporation they do not necessarily interact with each other. Members of the group are scattered across Southern Ontario, but the majority are in located in Toronto. The social science research team was interested in how the participants would respond to and use this particular technology. With the constraints on design imposed by the nature of this project we are attempting a limited study. We have proposed the establishment of a User Group which would meet twice during the 90 day trial; once early in the trial and again toward the end. Participants in the trial have been instructed to keep a diary of their experiences with using the system for the trial period. The record of the User Group meetings and the diaries will constitute the database for analysis.

Research Studies Underway

Human Resources

Dr. Gale Moore , Head, Social Sciences Research, Ontario Telepresence Project

Professor Mark Chignell, Industrial Engineering (University of Toronto)

Professor Richard Dillon, Dept. of Psychology (Carleton University)

Professor Marilyn Mantei, Dept. of Computer Science (University of Toronto)

Faculty of Library & Information Science (University of Toronto)

Professor Janet Salaff, Dept. of Sociology (University of Toronto)

Professor Jo Tombaugh, Dept. of Psychology (Carleton Univerrsity)

Professor Barry Wellman, Dept. of Sociology (University of Toronto)

Graduate students from Sociology, Computer Science and Information Science

Back to Telepresence Papers
Back to Telepresence home page