Alex Mitchell, Ilona Posner and Ronald Baecker
This paper describes an ethnographic study of collaborative writing by two groups of 4 grade six students using synchronous collaborative writing software for one hour per week over a 12 week period. Despite initially having little appreciation of what it means to write together, and no experience in synchronous collaborative writing, both groups produced nearly one dozen short collaboratively conceived, written, and edited documents by the end of the study.
A careful analysis of video tape records, written documents, questionnaires, and interviews demonstrated the importance of concepts such as awareness, ownership, and control in the writing process, and highlighted many examples of strengths and weaknesses in the writing software.
KEYWORDS: CSCW, groupware, group work, collaborative writing, learning to write, novice writers, ethnography.
© Copyright 1995 ACM, Inc.
Figure 1: Cover of the magazine, by Ryan Fields, age 12.
The above research has provided insight into the collaborative writing process. A number of theories of collaborative writing have been developed [5, 20, 23] to characterize this process. There have also been a number of tools designed to support the collaborative writing process. Most notable among the many systems are GROVE [6], PREP [16], Quilt [10], SASSE [1], and ShrEdit [17].
Studies have been conducted into the use of several of these collaborative writing tools [12, 18]. These types of studies, while valuable, do not provide much insight into how collaborative writing tools would be used in real, extended scenarios.
Based on this previous research and our own experience with SASSE [1], we felt useful insight could be gained from a detailed study of the use of a collaborative writing tool by novice writers over an extended period of time. Although collaborative writing is a common practice, novice writers are uncertain as to how to proceed, and often have difficulty even understanding what is meant by collaborative writing. Studying the use of a collaborative writing system provides the opportunity to observe how novice writers learn to write together, and reveals much about the special needs of novice writers. We felt that the use of a collaborative writing tool would help with the process of learning to write together, and would give us insight into the strengths and weaknesses of the tools, providing clues as to the requirements for design of future systems.
Our study involves two groups of 4 grade six students working together to produce a magazine on prejudice (see Figure 1). Through the course of the twelve week study, they learned how to use Aspects{1}, a commercial synchronous collaborative editor, and developed the skills necessary to successfully write together. The students all developed different levels of expertise, and were able to share that expertise with each other to help the group accomplish its task. The study had several objectives. We wanted to observe extended use of a synchronous collaborative editor, in a situated context, with inexperienced writers, in the hope of both validating the concept of a synchronous shared editor and deriving useful insight into the design of future systems. We also wanted to see if the use of a synchronous collaborative editor would benefit the writing and learning processes.
This paper presents preliminary results of the study in the form of qualitative observations and suggestions for the design of future collaborative writing tools. We will begin by giving an overview of the study, the setup and activities involved, and the data collection and analysis methods. We will then briefly explain the way in which the students developed expertise at using Aspects and writing together. The rest of the paper will focus on the way in which the students used the collaborative writing tool, and the design recommendations that can be derived from these observations. A companion paper [21] will provide more details into the learning process, how the use of collaborative writing tools affected the product, and the impacts the experience had on the participants.
We conducted an ethnographic study of two groups of students preparing the written material for this magazine. Eight students were selected, with the assistance of their teachers, from 14 volunteers out of two grade 5/6 classes. The students were experienced with the Computer Supported Intentional Learning Environment (CSILE) shared knowledge building system [22], but were not familiar with synchronous collaborative work on a computer.
Through the course of the study the students learned to write together using Aspects on networked Macintosh computers. Aspects allows multiple users to work concurrently on shared documents. It uses a replicated architecture, provides various locking mechanisms, and minimal consistency control. It is a fairly conservative but stable system as opposed to research systems which may provide many useful features but may lack the robustness necessary for serious extended use.
Figure 2: Physical setup of the study
During the twelve-week period, the students were given training in the use of the Aspects system, introduced to the concepts and skills necessary for collaborative writing, and then given the freedom to use those skills as they saw fit.
The first five weeks were highly structured in order to cover various topics related to prejudice. The format of group work was guided by the instructor (Ilona) in an attempt to introduce a variety of ways of working collaboratively. The activities included writing a poem and a story as a group. Students were exposed to the various mechanisms provided by Aspects, giving them the tools necessary to perform their tasks. The assigned tasks were designed to expose the students to a variety of writing styles and approaches [19, 20, 22]. They started with scribe/consultant writing, where a scribe enters text in a document, and one or more consultants provide ideas but do not actually enter them in the document. They were also given tasks involving parallel writing, with writers individually entering text at the same time in the same document, but in different regions, and joint writing, with writers working closely on one section of the document.
In the remaining weeks the students were free to choose what they wanted to work on and how they would work together. Their work included doing research, writing articles, creating artwork, and editing some of the materials created in the first five weeks to be incorporated into the magazine.
Throughout the study, a balance had to be maintained between the amount of training given and our desire to see how the students would use the technology without guidance.
Exploratory analysis was conducted on the video records of the sessions by two people using Timelines. This analysis identified common problems, incidents and trends in the data. More in-depth discourse analysis and coding will be done in the future to explore in detail some of the areas discussed in this paper.
A number of other measures were used to provide a rich view of the sessions. The students' teachers were asked to evaluate the students' abilities, and to provide blind evaluations of a selection of work, both individual and group. The evaluations were intended to provide information which could be used to give insight into the effects of the use of groupware technology on the students' learning experience and the quality of the document. The results from this data were used in the companion paper [21].
We also conducted a halfway and a final questionnaire and a final individual interview with each student. A few weeks after the end of the project, we came back and conducted a group discussion about the project, which gave additional insight into the students' experience.
Having never worked on a group project that demanded such close collaboration, the students had to learn both what it means to write together and how to do that successfully. In addition to learning about the task of writing together, the students had to learn to use the tools provided. The students had no problem working with the computers. All had extensive experience with computers both in the classroom and at home. In fact, the students were so comfortable with the computers that they took a cruel pleasure in causing the software to crash.
The first major difference encountered by any user when moving from single-user to multi-user software is the notion of a shared space in which other people are working. This experience was illustrated in many ways. The students developed awareness of themselves and each other. Similarly they developed patterns for determining who would control the shared space. Finally, they negotiated access to that space.
When they found that entering duplicate text without consultation didn't work very well, the group switched to a consultant/scribe mode of writing. Despite the ability to access the document synchronously, one person would do all the typing with the others suggesting ideas. This style of writing leads to interesting control and ownership problems, as will be discussed later. This form of writing most closely resembles the way groups work together synchronously using traditional technology [20], and thus involved the least adaptation on the part of the students.
Later, as they became more comfortable with the task and the system, the students began to make full use of the synchronous editing capabilities of Aspects. The following is a brief example of how the students made effective use of the shared workspace. While composing a story during the fifth week of the study, both groups used a consultant/scribe writing style. One student "drove" the writing, eliciting ideas and entering them in the document, while the others contributed ideas and followed on their own screens. However, one student frequently moved around in the document, rereading the story from the start and suggesting changes. She also pointed out errors using a telepointer. Interestingly, though, she did not actually make changes. This notion of control over the document by the scribe will be discussed further in the section on control below.
Part of the problem with the notion of what it means to write together is the question of what it means for a piece of writing to be a group document. The students weren't sure whether a document written in parallel could ever be considered a unified piece of writing. They felt that such a document should be rewritten from scratch by one person. This came up when the first group was editing a poem with stanzas written in parallel by each group member{2}.
Sue I think we should write the whole thing over because it sounds like 3 poems stuck together. Hope Yeah, I think one person has to rewrite it. Liz Yeah.This same difficulty was encountered with several other documents that were written in parallel. It was only when a document was written jointly with one person acting as scribe that the group acknowledged that the document was coherent.
This problem faced by our novice writers highlights an ongoing question faced by researchers of collaborations: when should a document be considered to have been truly written together.
The feedback provided by the system was often obscure and confusing, leading the students to ignore it. When the messages were critical, such as when document consistency was lost, this led to later problems (b). The fact that documents are shared, yet replicated, caused continuous confusion to both the students and the experimenters. A number of documents were lost and had to be reentered because the various contributors all assumed that someone else had saved the document. The location of the document was not at all obvious from the interface (c).
However, as the students became more familiar with the concept of shared access to a common document, they developed working patterns that took advantage of the technology. As will be discussed below, the students came to realize how the technology could be used in different situations.
Where Am I?
There are several things that the students were asking when they asked "Where am I?" The simplest is location in the shared document. However, it is also important to provide some feedback as to whether the user is in a conference or working alone, and whether the current document is private or shared with others (d).
It was common for people to become confused as to whether their work would be seen by others or not, and whether they would be able to see others' work. This was especially true when someone stopped working closely with the group, and then later returned.
In one case, it turned out that the students who had been working together to edit a document had in fact been working in separate copies of the document (e).
Sue [moving her mouse and looking at Hope's screen] Why doesn't it [her pointer] show up? Ilona Why don't I have a copy of that? Dan [moves mouse, looks at Sue's]
This situation wasn't discovered until Sue happened to glance at Hope's screen and notice that her telepointer was not showing up.
What Am I Doing?
Similarly, in a multi-user conference with multiple documents it isn't always obvious what you are doing at a given moment. This can range from confusion as to whether you are telepointing or not, a simple interface problem, to more subtle concerns, such as whether you are interfering with someone else's attempts to edit text. The system needs to make the information about your relative location and influence on others readily available (e).
Where Are You?
Lack of any reminder of where others are makes it easy to forget that there is a shared workspace (f):
Group [they start to divide the task up by questions - each group member tells the others what they wrote as if the others can't see it] Ilona Everyone can see the same thing. [goes over and scrolls Sally's to show them]
What Are You Doing?
When working individually, it is easy to lose track of what others are doing. This problem is often overcome by resorting to physical pointing and glancing at each other's screens (g):
Sally I'm getting confused. Rob, what are we changing here? Rob [makes changes, points to screen to indicate what he is doing] Carol [watches what Rob is doing] Sally [doesn't notice Rob's gesture] Rob what are we doing here? Rob Its gonna look like a poem. Sally [sees gesture and looks over] OkayWho Did That?
With synchronous shared access to the document, it is possible to enter text or to delete someone else's text without that person's knowledge. This can lead to confusion:
Dan [deleting something] Liz No no don't erase it DON'T! who erased that? Hope Not me I just got in. Liz [looks over at Sue] Sue?!? Dan [looks around, says nothing]Tracking of where other people are and what they are doing can come in many forms. Aspects provides bars alone the side of the document indicating that someone has control of a region of text. This tells you that someone is there, but not who it is.
Sally Rob, can you show me what you're trying to do? Rob Take a look here. [points to her screen]Telepointers were too limiting because they were both unable to draw collaborators' attention and lacked information about the person who was pointing. All this information is available in a simple hand gesture (g).
The telepointers also tended to be rather distracting; the students often ended up chasing each others' pointers around the screen. However, one group did learn to use them effectively when proofreading and editing:
Liz Meant is spelled M-E-A-N-T [points at her screen] Dan Where is it? Liz Its, I'll mark it, there I've marked it. See? That's where it is [uses telepointer] where my little annoying thingy is.Having discovered this function, Liz explains it to the others:
Liz Say he spelt birth wrong [points with finger, Sue looks] you go to that [moves telepointer there] and go like that. [wiggles it] Sue OHHH.The shared workspace encourages this type of consulting and collaborative learning.
The physical placement of the machines also allowed people to notice when someone is looking at their work; this is useful for encouraging and facilitating collaboration and consultation:
Group [they start entering comments] Liz [glances at Hope's screen for confirmation she's doing it right] Hope [notices, gives her advice]However, the physical placement can also lead to formation of subgroups and exclusion of peripheral group members. To minimize this, we rearranged seating patterns each session.
Similarly, there was a connection between who typed in a section of text and who got credit for the ideas contained in the text. We observed that the scribe usually provided fewer ideas than the rest of the group. Despite this, the scribe occasionally took credit for the content of the document. For example, in the first group, the person who typed a story claimed the next day that it was his story:
Dan I wrote the story Sue No I did. Liz I did. Hope We all did. Dan The one about the... Hope I made up Tiger Lily. Ilona I thought everybody wrote it. Dan I wrote it most cause I typed everything.Although the system did not provide explicit ownership information, the students tended to continue to identify text with the person who typed it. This showed up most clearly when the second group was editing one of the documents. In this case, the sections of the document were all written in parallel, with each section easily identifiable as belonging to a different person. Two of the group members were suggesting changes, but refused to make the final alterations until the entire group gave permission.
However, when the group was working together on a document that had already been edited, the group members had no reservations about arbitrarily deleting someone else's text without telling them (h). This suggests that at this point the group members had come to regard the text as shared, rather than just owned by the person who typed it.
The students in the study used Aspects in paragraph locking mode. This allows each user to gain control of a paragraph of text and make changes within that paragraph. As long as the user doesn't move the selection point out of the paragraph, other users are locked out. This granularity of locking led to some interesting behaviour.
When working with physical documents, the students were able to gain control over a paper by grabbing it if necessary. In Aspects there was no way to force a shift in control. One student understood the technology to the extent that he deliberately kept an entire document as one paragraph to keep control of the changes being made to the text, even when encouraged to add paragraph breaks:
Ilona Can I suggest you put some more returns in there? [gets up and puts returns in so Dan isn't locking the whole document] Liz & [start typing like mad as Ilona Sue puts in spaces] Dan I didn't want to or else they'll start doing funny things with it.However, the group soon learned to overcome the limitations of the locking mechanism by simply using the other person's computer rather than trying to get control of the document from within the system.
Dan [takes Sue's mouse when she's not looking, moves her out of a paragraph] Sue [looks back, sees Dan] Dan One sec... stop, let me work on this part.Control over the text also had an effect on the roles taken on by the group members. When deciding who would be the scribe, the group would either take a vote or argue until someone managed to get control of the text. For example, when the first group was composing a story, Dan ended up gaining control of the text. The group went along with this, everyone dictating while he types; ideas were passed around, negotiated, and the final decision was made by the scribe.
Although the system gave the scribe explicit control of the document, the other members of the group were still able to make significant contributions and provide feedback which affected the contents of the document. In the above example, where Dan had control of the document, there were several occasions where other students tried to make changes. Kim would attempt to alter a sentence, and failing that would ask Dan to make the change, using the telepointer to indicate the change. So, in spite of the control mechanisms provided by the technology, the entire group was able to influence the document (i).
Working synchronously but on separate sections of the document worked well in a task that lends itself to division. On the tenth day, the two members of the first group were preparing the questions for an interview - they discussed the content, then split the task up, but talked back and forth while entering the text. Then, after the interview, they worked in a scribe fashion, one student dictating the answers while the other typed.
However, it was not always clear who had been given the role of scribe. In fact, the assignment of roles tended to change dynamically, since the technology lets anyone take control as long as the previous scribe is willing to relinquish control. In general the groups were able to adapt to this shifting of control. If someone's ideas were not being accepted or they were being ignored, that person would sometimes go off and start entering the ideas independently in a different section of the document.
The students were able to adapt their use of the system to suit their working patterns, and to take in to consideration the social and group interactions taking place as they worked. It was possible because the system did not attempt to impose strict roles and patterns of usage on the students (j).
Another important problem introduced by collaborative tools is the need to keep track of changes in a shared document. One way to provide the necessary information about changes in the document is through the display of differences in the document, often called "diffs", either as change bars [15], through the use of annotations [14], or through more active notification [13].
Ownership of and access to the shared workspace are also important considerations. As we have seen, the way in which both of these are handled by the system have an influence on the behaviour of the group. While some systems such as Grove [6] and PREP [16] assign roles to collaborators, most systems leave this up to the group. This, along with flexible access and floor control mechanisms, allows social interactions, not the system, to determine working patterns and group behaviour.
The above discussion suggests a number of design recommendations, which we will summarize below:
The students testified to having written the magazine "together", something that they did not even know the meaning of at the start of the study. Perhaps this in itself is the best definition of group writing - the perception that the results of your work are the result of the group's work, rather than the work of the individual members of the group. This achievement validates the concept of a synchronous shared text editor, and provides promise for the use of such technology in education and writing in general.
The group jointly learned about an important topic, prejudice. As one student said in the final group discussion:
Liz Everyone is different, we all have different beliefs, and we should respect that. this is what we learned.They had become comfortable with the idea of writing together, and confident with the technology. From not knowing what it means to write together, they had progressed to feeling that they were able to succeed at, and enjoy, group writing. During the final group discussion, one student volunteered the following:
Rob The best thing was learning how to work with everybody, we weren't too good at that before.By observing this learning process, we have been able to gain an insight into the nature of group writing, and identify some of the effects of the use of collaborative writing tools on this process.
We have seen that the technology has a distinct effect on the way in which novice writers approach the collaborative writing task. However, at the same time we have seen that writers, as they become familiar with both the task and the technology, are able to exploit the features of the system and use it to their advantage in creative ways. The observations we have made of the problems students have learning to write together are very similar to those experienced seen in adult writers. From these observations we have drawn a series of recommendations for future design.
Our study stretched over twelve weeks. With a task domain as complicated and unfamiliar as collaborative writing, it is important that any observations be made over an extended period, in situ, and in a situation where users are allowed to approach their tasks as freely as possible. This allows usage patterns to develop naturally, and provides the time needed to learn about the task and the technology. Although this type of ethnographic study is harder to run and much more time-consuming to analyze than a traditional lab study, the type of real usage that we have observed could never be seen in a usability laboratory.
{2}Note that the names of the students have been changed, but gender has been preserved.
{3} Throughout our discussion of the study we will use letters such as (a) to link our observations to the list of design recommendations appearing near the end of this paper.