This report is also available as an Acrobat file.

Contents

Chapter 5

The Laboratory Study

For our second study into the use of video conferencing systems we used the University of Derby Video Conferencing system to run a series of artificial video conferences. The conferences involved placing small groups of users at different workstations and asking them to use the video conferencing system, a shared drawing tool and a telephone audio link to work together on various tasks.

After each collaborative session the subjects were asked to complete a questionnaire in which they were asked to score various aspects of the collaborate environment. Questions were on a five point scale from Agree Strongly to Disagree Strongly . The questionnaire as designed with both positive and negative statements and contained questions relating to the following the categories 1) The Appropriateness of the Task; 2) Social Communication; 3) Performance of the System; 4) Audio; and 5) Group Working (the full questionnaire is given in Appendix 3).

Artificial Tasks

In developing the artificial tasks for the study we aimed to identify areas either had close similarities to the types of activities common in day-to-day work or would offer particular insights into the usability of the video conferencing system. Five such tasks were developed and one pair of collaborators completed each task:

Task 1 - Jointly Produce a Document, and Task 4 - Agree the Allocation of Access Funds. Two people were placed at different video conferencing workstations and asked to use a CUSeeMe link and a telephone link to work together on the task which was presented to them in the form of a written brief. For Task 1 the brief simply contained a number of document elements (text, pictures, headings etc.) and the two collaborators has to agree upon a layout. For Task 4 the brief consisted of a scenario in which they collaborators had to agree on the allocation of limited funds to a number of project proposals. No other collaboration tools were used.

Task 2 - Design of an Office Layout, was a Design task that involved asking a pair of collaborators to use a telephone link, a CUSeeMe link and a shared drawing tool called Aspects to work together on the design of an office layout (see Appendix 4).

Task 3 - University Scholarship Committee, each collaborator was given a different student and had to argue that their student should receive a University Scholarship. There we told that they had to come to an agreement by the end of the session so a degree of explaining and negotiation was involved. Only a telephone link and CUSeeMe link were used.

Task 5 - Charades, this task simply involved playing a game of Charades via the video conferencing link. Each collaborator was given a list of TV programmes and films to act out for the other. Clearly, the goal of this task was to see how successful CUSeeMe was at supporting gestural communication. Only CUSeeMe was available with no audio link (see Appendix 5).

The Experimental Procedure

The subjects were sat at their video conferencing workstations and told that they had up to one hour to work on the brief. At the end of the session they were both given a questionnaire. All tasks were completed over the 10Mbps Local Area Network, giving a video frame-rate of approximately 7-10 frames per second.

Results of the Questionnaire

The following results are divided into the categories of the questionnaire that were identified earlier. A total of nine responses were received (one questionnaire was not returned) to the questionnaires and although small a few key findings can be drawn from their responses.

The Appropriateness of the Task

100% Agreed or Agreed Strongly that the discussions held over the system went well. All but one respondent Agreed or Agreed Strongly that it was possible to fully express what they wanted over the system. 70% disagreed that it was hard to judge when to speak and when to wait. Of those it applied to, 2/3rds Disagreed that it was difficult to write as well as stay involved in the discussion.

Social Communication

Nearly 45% Disagreed that it was difficult to initiate eye contact whereas nearly 45% Agreed or Agreed Strongly that it was difficult. Just over 60% Disagreed that it was easy to maintain eye contact. 55% Disagreed or Disagreed Strongly that gestures by the other participants were unclear, whereas only 33% Agreed Gestures were unclear. All respondents Agreed or Agreed Strongly that the system did enable the group to build a rapport with one another.

Performance of the System

2/3rds Agreed that the update rate of the screen was sufficient for the discussions held. Nearly 90% Agreed or Agreed Strongly that the quality of the image in terms of resolution was adequate for the uses made of the system. Nearly 90% Disagreed or Disagreed Strongly that the display adversely affected the exchange of information between participants. Less conclusive was the issue of the position of the cameras, 45% disagreed with the statement that "the position of the cameras were not acceptable for the discussions we held. 33% did however Agree that the camera position was not acceptable, 11% actually Agreed Strongly.

Audio

Over 75% Agreed or Agreed Strongly that it was easy to identify who was speaking. This finding is not particularly startling because all the interaction only had only two participants! 1/3rd neither Agreed nor Disagreed that the discussion we held were only possible using the system set up rather than a normal phone call. 1/3rd did however Agree and 11% Agreed Strongly.

Group Working

2/3rds Agreed that it was clear what all the participants thought about the discussion at different times. Of those it applied to nearly 90% Agreed or Agreed Strongly that it was easy to reach a conclusion to their discussions. Over half Disagreed or Disagreed Strongly that the system set-up was not well suited to the type of tasks we did, whereas only 1/3rd Agreed.

Discussion and Breakdown According to Task

This study showed how video conferencing suited some tasks more than others. It became clear, for instance, that the Task 2 (Design an Office Layout) did not really need the video link, with the link and shared drawing surface providing sufficient means for collaboration to take place.

The Charades Task highlighted how powerful a video link can be in terms of communication. No audio link was provided and participants could only type their answers to each other. One the whole all items were guessed quickly. Although clearly, larger video windows would have made the process easier and a greater bandwidth allowance would have given a smoother image and allowed a clearer expression to occur. Once again the subjects reported that the position of the cameras was critical to success.

The discussion of who should gain what amount of money from an Access Fund committee (Task 4.), resulted in some useful findings. A few practical problems were reported. For one collaborator their video camera was to the side and the other partner felt that a head on display would have been preferred rather than the profile they received. Since the paper brief became the focus for both participants, rather than a shared view on screen, it was felt that an on-screen text document with the video window of the other participant next to it would have mirrored a true head to head discussion far more easily and naturally. One of the participants did feel the video enabled a sense of rapport to be developed and hence it became easier to know how to react to the other user and judge his sense of humour or the level of discussion.

Contents

Graphics Multimedia Virtual Environments Visualisation Contents