IN SYSTEM DEVELOPMENT:
TOWARDS A COOPERATIVE PROTOTYPING APPROACH
Kaj Grønbæk,
Computer Science Department,
Aarhus University
Ph.D. Thesis
January 1991
CONTENTS
0. PROTOTYPING AND ACTIVE USER INVOLVEMENT IN 1
SYSTEM DEVELOPMENT:
TOWARDS A COOPERATIVE
PROTOTYPING APPROACH (OVERVIEW PAPER)
1. RAPID PROTOTYPING WITH FOURTH GENERATION 91
SYSTEMS: AN EMPIRICAL
STUDY
2. COOPERATIVE PROTOTYPING STUDIES: USERS AND 115
DESIGNERS ENVISION A
DENTAL CASE RECORD SYSTEM
3. SUPPORTING ACTIVE USER INVOLVEMENT 137
IN PROTOTYPING
4. COOPERATIVE PROTOTYPING: 161
USERS AND DESIGNERS IN MUTUAL ACTIVITY
5. DISCOVERING NEW USER-DESIGNER PATTERNS 191
OF INTERACTION: A VIDEO-BASED
ANALYSIS
OF THE COOPERATIVE PROTOTYPING PROCESS
0. PROTOTYPING AND ACTIVE USER INVOLVEMENT IN SYSTEM DEVELOPMENT: TOWARDS A COOPERATIVE PROTOTYPING APPROACH (OVERVIEW PAPER)
Kaj Grønbæk
Målet med at kombinere ideer fra ovennævnte forskningsområder er at opnå en større forståelse af, hvordan systemudvikling kan udføres for at håndtere de centrale problemer bedre end det har været tilfældet hidtil. Udfordringen i består i at "afdække" brugernes og organisationernes behov og ønsker og kreativt at kombinere disse med edb-tekniske muligheder. Brugernes behov og ønsker kan imidlertid ikke afdækkes, nedskrives og formidles til udviklerne i starten af et udviklingsprojekt, som det har været antagelsen i mange traditionelle systemudviklingsmetoder. Dette skyldes mange faktorer: Brugere er ikke vant til at formulere ønsker til edb-systemer. Mange aspekter af brugeres arbejde er vanskelige at analysere, idet menneskelig aktivitet i høj grad er baseret på såkaldt tavs viden (tacit knowledge) eller uartikulerede operationer, der kun kommer i fokus i sammenbrudssituationer (breakdowns), når de faktiske betingelser for en operations gennemførelse ikke svarer til de forventede. Brugere har ikke tilstrækkelig viden om de tekniske muligheder, og de kan derfor ikke relatere disse til deres arbejde. Organisationer og teknologi forandrer sig parallelt med systemudviklingsprocessers forløb, og ønsker til edb-støtte kan således ikke holdes stabile.
Forskere med fokus på prototyping beskæftiger sig med et vidt spektrum af teknikker, metoder og tilhørende værktøjer, som har det til fælles, at der udvikles og afprøves en eller flere sædvanligvis ufuldstændige versioner, prototyper, af det fremtidige edb-system. Sådanne prototyper kan på konkret vis illustrere dele af de essentielle egenskaber, som det færdige edb-system forventes at have. I modsætning til systembeskrivelser kan prototyper gennem en håndgribelig afprøvning (hands-on experience) evalueres af brugere og/eller udviklere før det endelige edb-system implementeres. Forskningen i prototyping er typisk teknisk orienteret mod værktøjer, mod at øge produktiviteten i systemudvikling eller mod at opstille projektmodeller for gennemførelse af systemudvikling med prototyping. Kapitel 2 giver en oversigt over forskning i prototyping.
Forskere med fokus på brugerdeltagelse i systemudvikling beskæftiger sig med forskellige metoder og teknikker til at inddrage menneskelige forhold i bred forstand, d.v.s. organisatoriske, sociale, politiske og psykologiske forhold, i forbindelse med udvikling af edb-systemer. Der tages hensyn til disse forhold ved i forskellig grad at inddrage brugernes kompetencer i systemudviklingen. Det er vanskeligt at angive en fællesnævner for anvendelse af teknikker og værktøjer indenfor dette område, idet der hersker stor uenighed om fremgangsmåder. Et fælles træk er dog, at forskerne er enige om, at systemudvikling ikke kan ses som en snæver teknisk disciplin. Det anses som vigtigt for effektiviteten og kvaliteten af et edb-system, at det passer til brugerorganisationen og de enkelte (grupper af) brugeres behov. En sådan effektivitet og kvalitet kan ikke opnås uden at inddrage brugerne i udviklingen. Forskningen i brugerdeltagelse er typisk orienteret mod de menneskelige, organisatoriske og sociale forhold, og der gives kun i ringe omfang konkrete bud på den tekniske udvikling i form af design teknikker. Kapitel 3 giver en oversigt over forskning i brugerdeltagelse indenfor systemudvikling.
Der er kun lidt sammenfald mellem de to beskrevne forskningsaktiviteter, så vidt det kan vurderes fra litteraturen indenfor de to emneområder. En del af arbejdet med emnet for afhandlingen har således bestået i at kombinere ideer og erfaringer fra ovennævnte områder. Hovedindsatsen har været rettet mod at udvikle teknikker og komme med forslag til systemudviklingsværktøjer, der forbedrer mulighederne for at bringe systemudvikleres og brugeres kompetencer sammen i bestræbelserne mod at håndtere de centrale problemer i systemudvikling, som nævnt ovenfor.
Fra prototyping området er inspirationen hovedsageligt hentet fra forskere, der arbejder med udforskende (exploratory) prototyping til at undersøge nye og alternative ideer for et system design tidligt i et udviklingsforløb eller i forskningsøjemed. Fra brugerdeltagelsesområdet er inspirationen hovedsageligt hentet fra forskere med baggrund i den såkaldte "fagpolitiske tradition" indenfor skandinavisk forskning i systemudvikling, idet jeg selv har en aktiv tilknytning hertil. I den fagpolitiske tradition fokuseres på etablering af en udviklingsproces med aktiv brugerdeltagelse, der understøtter en gensidig læring (mutual learning) mellem brugere og udviklere. Den gensidige læring består i, dels at brugerne lærer om tekniske muligheder i relation til deres arbejde, og dels at udviklerne lærer om brugernes arbejde for at kunne designe relevant edb-støtte til dette. En sådan udviklingsproces, hvor udviklere og brugere deltager på lige fod betegnes i nyere litteratur som kooperativ design (cooperative design). Målene med kooperativ design er: at brugerne aktivt bidrager med kvalifikationer hørende til deres arbejde, at udviklerne bidrager med design og edb-tekniske kvalifikationer, at brugerne får demokratisk indflydelse på udviklingen af teknologi til deres arbejdsplads, og at brugerne øger kvaliteten af deres eget arbejde gennem edb-støtte.
Arbejdet med emnet har ledt til introduktionen af nye teknikker til prototyping med aktiv brugerdeltagelse, en såkaldt kooperativ prototyping fremgangsmåde (Cooperative Prototyping Approach). Denne prototyping fremgangsmåde er motiveret, udviklet, afprøvet og videreudviklet gennem feltstudier. Der har bl.a. været gennemført to korte projektforløb med grupper af henholdsvis tandklinikassistenter og kommunale sagsbehandlere (fra en teknisk forvaltning). De publicerede artikler dokumenterer dette arbejde. Kapitel 4 og 5 i denne oversigtsartikel giver en sammenfattende introduktion til kooperativ prototyping i form af en række udsagn, byggende på erfaringer fra feltstudier og litteraturstudier. Disse udsagn findes i en dansk oversigt i Appendix B.
Hovedelementerne i kooperativ prototyping - et kort resumé: Kooperative prototyping teknikker understøtter et aktivt samarbejde mellem brugere og systemudviklere under udvikling og modifikation af prototyper tidligt i et systemudviklingsforløb. De centrale teknikker er: 1) at anvende prototyper som "skitseblokke" til fælles udforskning af design ideer i relation til brugeres arbejde, 2) at etablere evalueringssessioner med brugslignende afprøvning af prototyper baseret på simuleret funktionalitet og eksempeldata. De basale teknikker understøtter en gensidig læring hos deltagerne. Teknik 1 understøtter således, at brugerne lærer om de teknologiske muligheder og mulighederne for at øve indflydelse på designbeslutninger, idet de deltager direkte i modifikation og udvikling af (eventuelt alternative) prototyper. Teknik 1 understøtter også, at udviklerne lærer om brugernes arbejde for eksempel ved at prototypen inspirerer eller provokerer brugerne til at fortælle anekdoter relateret til det arbejde, der designes edb-støtte til. Teknik 2 understøtter, at udviklerne lærer om de urekflekterede og tavse aspekter af brugernes arbejde, idet forekomsten af sammenbrud (breakdowns) i brugslignende afprøvninger af prototyper bringer sådanne aspekter frem. Der kan således tages højde for sådanne sammenbrud, når prototyperne redesignes. Endvidere understøtter teknik 2, at brugerne oplæres i brugen af prototyper, hvorved en senere oplæring i brugen af det færdige system lettes. De centrale teknikker må selvfølgelig understøttes af en række supplerende teknikker til f.eks. indledende analyse, forberedelse af prototypingsessioner, dokumentation af sessioner og behandling af resultater fra sessioner. Endelig forudsættes adgang til fleksible prototypingværktøjer. Disse aspekter er detaljeret behandlet i artiklerne samt kapitel 4 og 5.
I kapitel 6 diskuteres de anvendte forskningsmetoder. Her beskrives hvorledes forskningsmetoderne har været eksplorative og i høj grad bygger på feltstudier. Der argumenteres også for den forskningsmæssige fremgangsmåde, hvor styrkelse af relevansen af resultaterne har været prioriteret over muligheden for at kontrollere korrektheden.
Endelig beskrives i kapitel 7 to mulige typer af fortsættelsesprojekter, der udnytter resultaterne fra denne afhandling. Den første type projekter er rettet mod yderligere at udvikle kooperativ prototyping gennem anvendelse af fremgangsmåden i fuldskala systemudviklingsprojekter. Den anden type projekter er rettet mod at udvikle prototyping værktøjer, der understøtter anvendelsen af kooperativ prototyping bedre end hidtidige værktøjer. Der beskrives kort et værktøjsudviklingsprojekt, som jeg allerede arbejder på.
Afhandlingen er indleveret ved Datalogisk Afdeling, Aarhus Universitet, med henblik på erhvervelse af den naturvidenskabelige Ph.D.-grad (tidligere kaldet licentiatgraden). Lektor Morten Kyng, Datalogisk Afdeling, har været vejleder på projektet.
Before delving into the contents of the thesis I briefly describe the process that led to the completion of this thesis.
I finished my master's degree in computer science and mathematics in January 1988; afterwards I was offered grants, lasting two and a half years to work on a Ph.D. project on "development of prototyping tools and techniques that support the cooperation between the actors in system development" at the Computer Science Department, Aarhus University. This project was aimed at continuing the master thesis work carried out by me and two fellow students. In the same period a research program on "Computer Support for Cooperative Work and Communication" (COOP) (Bøgh Andersen et al., 1987) was initiated as a joint program between the Computer Science Department and the Institute of Information and Media Science. In addition, an ambitious program on "Object Oriented Office Organization" (O4) based on EEC funding was planned. The O4 program would bring research groups at Aarhus University together with European use and development organizations outside the University environment in joint research and design projects. Without getting into details about these programs, they both seemed good environments for me to do my thesis work. I needed to be in an environment where several people were engaged in work on similar issues to make my ambitions on contributing to development of both tools and techniques realistic.
The O4 preparation gave me the opportunity to get in contact with other researchers mainly at Xerox EuroPARC and Xerox PARC. Moreover, Aarhus University was granted a number of Xerox 1186 graphical workstations of which I was assigned one. Among other things the COOP group, Xerox EuroPARC, and Jutland Technological Institute planned to undertake a little pre-project to O4 in a Danish electronics company, Søren T. Lyngsø A/S. This pre-project was based on funding from The Technical Research Council in Denmark. The aim was to make analysis of the work environment of the technicians at Søren T. Lyngsø A/S and design an integrated computer supported work environment for the technicians. My goal for the participation in this pre-project was to get experiences in applying prototyping techniques in a realistic setting. Moreover, the design ideas moved towards use of graphical workstations; a movement which was a good opportunity for me to study prototyping tools for graphical workstations.
However, I realized the hard way that research projects depending on external funding are really too uncertain to rely on for a thesis study with a limited time frame: The O4 program proposal was turned down by the EEC. The Technical Research Council delayed decisions on the pre-project at Lyngsø several times, and finally they decided to give only reduced support for the project. The reduction of the budget for the Lyngsø pre-project also meant that the focus moved away from what was originally planned. Only the COOP program became a reality. These facts meant that many people including me spent (and wasted) a lot of time on proposal writing and planning meetings. In fact, such activities occupied much of my time the first year. Although I wasted time on involving myself in projects that never turned out the way I needed it for my thesis studies, the project start up activities had some positive side-effects. I made contacts to researchers with similar interests, I gained insight in the work environment of a large Danish electronics company, and I gained experience with the advanced Xerox Lisp programming environment including the NoteCards hypertext system. The latter experience made me interested in hypermedia/hypertext issues which led to a paper (Bannon & Grønbæk, 1989) which was outside the goals of my thesis. During the first year of my study I also wrote Paper 1 which was a theoretical elaboration of the prototyping aspects of my master thesis work.
Overcoming the frustrations of the failed engagement in the O4 and Lyngsø project plans, I was independently seeking other ways of reaching the goals of my thesis project. Within the frames of the COOP program I took the initiative to establish two smaller field studies on prototyping techniques featuring active user involvement by trying out some of the ideas from my master thesis studies and the work behind Paper 1. Both these field studies were carried out in cooperation with Susanne Bødker (SB); the first one in the fall of 1988 and the second one in the spring of 1989. These field studies combined with literature studies served as background for writing Papers 2-5. These papers introduce and describe a Cooperative Prototyping approach. In conjunction with the writing of the five papers, I co-authored a chapter (Bødker & Grønbæk, 1991c) for the book Design at Work: Cooperative Design of Computer Systems (Greenbaum & Kyng, 1991).
In conjunction with my research I have taught courses in system development at the Computer Science Department. In these courses prototyping was among the topics, and I have used both my own papers and the literature that I have studied for this teaching. I have also presented some of the results from my studies to practitioners by giving talks at two Danish one day conferences entitled "Ny viden om Systemudvikling" ("New knowledge about system development") arranged by a Danish consulting company called Metodica. These conferences gave me constructive and positive feedback on the possibilities for applying the cooperative prototyping approach in practice.
This work was supported economically by the Natural Science Faculty and the Computer Science Department at Aarhus University; and the Natural Science Council with travel funding and project support.
I owe several people grateful thanks for their constructive support to my work on this thesis. First of all I wish to thank Susanne Bødker both for her cooperation on the field studies and on the writing of Papers 2, 4, and 5, and for a lot of inspiring discussions. Secondly I wish to thank Randall H. Trigg for his cooperation on the analysis of the Municipality study and on the writing of Paper 5 plus his constructive comments on this overview paper. Thirdly, I wish to thank Morten Kyng, who was the adviser on this thesis project, for his encouragements and constructive critique of my work throughout the project. Next I wish to thank Søren Christensen and Tove Rolskov for their cooperation on the field studies behind our joint master thesis which provided the background for Paper 1. I also wish to thank Lars Mathiassen, who was the adviser on our master thesis project, for his encouragement to start this Ph.D. study, and for his on-going interest in discussing my work. I also wish to thank Liam Bannon and Jonathan Grudin for inspiring discussions and constructive comments on drafts of papers. I wish to acknowledge the dental assistants and municipal caseworkers who participated in the field studies which are central to this thesis. I also acknowledge the COOP group and the Computer Science Department, in particular Ole Lehrman Madsen and Brian Mayoh who were on my "Ph.D. Committee," for providing me with an environment to undertake this Ph.D. work. Finally, I wish to thank Iben and my one year old son Jens Emil for their patience, interruptions, and personal support.
This thesis was submitted to the Computer Science Department, Aarhus University, Denmark, in order to fulfil the requirements of the Ph.D. degree (earlier called the Lic. Scient. degree). Associate Professor Morten Kyng, Computer Science Department, is adviser on the Ph.D. project.
Kaj Grønbæk,
Århus, January 1991
In studying the system development literature it appears that approaches explicitly mentioning prototyping emerged in the late seventies. Prototyping has been used to mean everything from a bad excuse for iterating on a program without making a proper specification, to a deliberately chosen technique to involve users in the design process. The various prototyping approaches share a focus on developing and trying out sketches or incomplete versions of the computer system to be built before the final implementation.
Approaches supporting user involvement started to appear in the literature in the late sixties. The concepts of user involvement and user centered design also cover a rich variety of approaches spanning from designers taking the user and human factors into account when designing a system; to approaches involving users as full participants in the development process. This area involves political disagreements on who should be involved and to which level they should be involved. But the approaches have in common that system development cannot be seen as a purely technical development process. It is recognized that human, organizational, social, and political factors have a large impact on the quality and efficiency of new computer systems.
These two branches of system development literature meet only late in their short "history." The prototyping literature mainly has its focus on technical issues, prototyping tools, and descriptions of project models featuring prototyping; and the user involvement literature mainly has its focus on human and organizational issues.
Locating my thesis in this spectrum, I shall emphasize that the goal here is to increase the understanding of prototyping as a technique to involve users actively in design of computer-based systems. The thesis is an attempt to bring together ideas from the areas mentioned above. Hence, historical background and a discussion of both user involvement and prototyping approaches in system development is given in this overview paper in order to set the scene for a discussion of active user involvement in prototyping.
The main part of the thesis is the introduction of a Cooperative Prototyping approach which aims at making system design using prototyping into a cooperative activity among users and designers. The ideas are introduced and experiences from two field studies are presented and analyzed. The field studies are called the Dental Clinic study, and the Municipality study, throughout this overview paper. Two notes on terminology should be given here: I use the terms `dental assistant' and `caseworker' when referring to specific users from the Dental Clinic study and the Municipality study, respectively. I use the term `user' whenever I talk generally about what is usually called "end-users" of computer systems. I use the terms `developer' and `designer' interchangeably to cover the computer professionals who participate in the design project. I do not make distinctions between analysts, system designers, and programmers, since I do not assume a strict division of labour in the kind of development processes discussed in this thesis.
Much of the existing literature on prototyping and user involvement focuses on fairly abstract models for organizing system design activities. Compared to this literature my thesis contributes with a qualitative analysis of prototyping situations where users and designers cooperate. Concrete situations from the field studies are described and analyzed. The analyses lead to suggestions for improvements of techniques and tools to increase users' ability to participate in system design activities in close cooperation with designers. The overview paper gives a detailed discussion of the background and presents results combined from literature studies, field studies, and analyses. The published papers, constituting the main part of the thesis, report on detailed analysis of the field studies. The results described in the papers are presented in an example-driven descriptive way rather than as general guidelines, i.e. scenarios from the use of the cooperative prototyping approach are used frequently. This is a deliberate decision to emphasize the situatedness of cooperative design (Greenbaum & Kyng 1991).[1] This way of presenting results force the reader to consider similarities and differences between the described example and new projects where the approach may be used. This exercise shifts the attention towards how to fit the technique to the current setting, rather than sticking to a guide-line that probably doesn't fit the situation.
The theoretical analysis of situations and experiences from the field studies are based on qualitative analysis techniques. Since the subject area does not have established theories and methods it is necessary to do ad hoc analysis and draw on work from other areas. The main areas drawn on for analysis purposes are: Activity Theory based on the work of Bødker (1991) and Engeström & Engeström (1989) is applied for parts of the analysis. Interaction Analysis of video recordings based on the work of Suchman & Trigg (1991) is applied for another part of the analysis.
During my master's programme I was taking courses and doing projects both within traditional computer science subjects and within so-called System Work. The System Work group at Aarhus University is a group of researchers working with a spectrum of subjects related to development and use of computer-based systems. The group was established in the mid seventies when Kristen Nygaard, a key person in the Norwegian NJMF project (see Section 3.1) and in the development of the SIMULA and BETA programming languages, was a visiting professor at the department. The System Work group played a central role in a number of research projects, such as DUE , UTOPIA, and MARS. DUE and UTOPIA were both projects taking a trade union perspective on development and use of computer systems. Bansler (1989) characterize these projects as belonging to the "Critical tradition" of Scandinavian research on system development. The main academic publications coming from these projects are chapters in (Bjerknes, Ehn & Kyng, 1987) and the book Work-Oriented Design of Computer Artifacts by Ehn (1988). They led to formulation of the Collective Resource Approach, described briefly in Sections 3.1 and 3.2. The MARS project focused on methods for system development and project management, and the main academic publication from this project is the book Professional System Development by Andersen, Kensing, Lundin, Mathiassen, Munk-Madsen, Rasbech, & Sørgaard (1990).
My master thesis work applied theory and empirical research methods closely related to theories and methods applied in the MARS project. But due to my course and project activities I was also familiar with the theories and methods related to the Collective Resource Approach.
In conjunction with my educational background my perspective on system development of course also inherits from the fact that I am a Danish citizen possessing certain political points of view. But a perspective on a certain matter cannot be fully argued. In addition, ones perspective unconsciously direct the interpretations of the phenomena studied, thus objective conclusions cannot be reached within the kind of research area I am engaged in. It is always the reader or the user of the results who must judge where biases may be found. To make it easier to judge I can give an account of my perspective by listing three assumptions which I consider fundamental and which I believe to have a significant impact on how I work: Firstly, workers have fundamental democratic rights to influence the conditions under which they are working. This implies that users of technology, such as computer systems, have democratic rights to influence the development and use of technology for their workplace. Secondly, the goal of developing new technology for a workplace should be to increase the quality of both work processes and resulting products. This implies that computer systems should enhance both the individual and collective skills of workers rather than substitute human skills with automatized procedures. The metaphor of craftsmen developing and using tools can to some extent be used to describe the ideal for development and use of computer systems as discussed by Ehn (1988, Part IV). Thirdly, the system development process does not follow a rational ideal and thus cannot be accomplished according to given guidelines. Every development project is unique and development processes are contingent on the context in which they are situated (Andersen et al., 1990). Most development contexts are dynamic, complex, and possess potential conflicts, hence complete knowledge to pursue a development process can never be obtained. Some degree of experimentation is always necessary.
1. Rapid Prototyping with Fourth Generation Systems - an Empirical Study (Grønbæk, 1989)
2. Cooperative Prototyping Studies - users and designers envision a dental case record system (Bødker & Grønbæk, 1991a)
3. Supporting Active User Involvement in Prototyping (Grønbæk, 1990)
4. Cooperative Prototyping - Users and Designers in Mutual Activity (Bødker & Grønbæk, 1991b)
5. Discovering new User-Designer Patterns of Interaction: A Video-Based Analysis of the Cooperative Prototyping Process. This is a revision of the paper (Trigg, Bødker, & Grønbæk, 1990) which is in preparation for journal publication.
This overview paper consists of seven sections. Section 1 is this introduction. Section 2 provides a brief historical overview and a discussion of prototyping approaches based on literature studies. The section serves as a resource for comparing the cooperative prototyping approach with other research on prototyping. Section 3 gives a brief historical overview and discussion of different perspectives on user involvement in system design over the last three decades. The section serves as a resource for comparing the cooperative prototyping approach with other research on user involvement. Section 4 provides background and motivation for cooperative prototyping. Section 5 gives a discussion and a summary of the approach to cooperative prototyping as advocated in the published papers. The main results and conclusions on cooperative prototyping from Sections 4 and 5 are summarized in a set of concluding statements, which I denote tenets. Section 6 provides a brief discussion of the applied research methods including the limitations of the results which are heavily based on qualitative empirical studies. Section 7 points to possible continuations of the thesis work. Appendices A and B list the tenets from Sections 4 and 5 in English and Danish respectively. Following the overview paper the five published papers are reprinted in the sequence given above.
Paper 1 is a theoretical elaboration of the prototyping aspects of my master thesis work. A central part of my master thesis work was an empirical study of the use of 4'th Generation Systems in 9 Danish development organizations. Paper 1 provides an empirical motivation for the Cooperative Prototyping approach and gives recommendations on how to achieve a more active user involvement in prototyping.[2] Paper 1 was accepted for presentation in an International forum on Rapid Prototyping at the 22nd Hawaii International Conference on System Sciences (HICSS 22) January 1989, and it was later published in OFFICE: Technology and People.
Within the framework of the COOP program I initiated two field studies, the Dental Clinic study and the Municipality study, on prototyping techniques featuring active user involvement. Both these field studies were carried out in cooperation with Susanne Bødker (SB); the first field study took place in the fall of 1988 and the second in the spring of 1989.
The Dental Clinic study was undertaken in connection with a series of DUE courses that SB and I were giving for Dental Assistants.[3] This field study led to the writing of Paper 2 together with SB. This paper states the early ideas of the Cooperative Prototyping technique, describes situations from the prototyping process, and discusses problems and prospects of the approach. Paper 2 was presented at the first European Conference on Computer Supported Cooperative Work (EC-CSCW '89) 1989, and appears as a chapter in the book Studies in Computer Supported Cooperative Work: Theory, Practice and Design, edited by J. Bowers and S. Benford, Elsevier Science Publishers, 1991.
The Dental Clinic study and literature studies inspired me to survey potential sources of developing tools for Cooperative Prototyping. Paper 3 presents this survey in conjunction with an elaboration of the ideas behind Cooperative Prototyping. Paper 3 was presented at the 12'th Information Research seminar In Scandinavia (12'th IRIS) 1989, and later published in Scandinavian Journal of Information Systems.
Using prior contacts to a Municipality Office located in the surroundings of Århus, I succeeded in establishing a second field study, the Municipality study, together with SB. The study took place in the technical department of the Municipality Office, where we cooperated with the so-called caseworkers. This study was based on the experiences from the Dental Clinic study, and it was an extended study of the problems and prospects in applying the Cooperative Prototyping technique in a larger scale. Even though it was still exploratory in nature the Municipality study was more extensively documented. For instance, we used video recording for documenting the Cooperative Prototyping process in order to support later analysis. The Municipality study led to the writing of Paper 4 together with SB. Paper 4 was accepted for a special issue on Computer Supported Cooperative Work of the International Journal of Man-Machine Studies, where it is in press. A slightly different version of Paper 4 is in press in the ISCRAT `90 proceedings.
The Municipality study also led to the writing of Paper 5 together with SB and Randall Trigg (RT). RT had earlier at Xerox PARC been involved in making video based analysis of human interaction. Inspired by seeing our video recordings from prototyping sessions he strongly encouraged us to undertake a detailed video-based interaction analysis of a fragment of the recordings. These analyses contributed both to an improved understanding of cooperative prototyping situations and to an understanding of the potential of video analysis in the design process. The analyses are documented in Paper 5 which was presented at the 13'th Information Research seminar In Scandinavia (13'th IRIS) 1990, and is now under revision for publication in Scandinavian Journal of Information Systems.
It is widely recognized in electrical and mechanical engineering that it is hard to build any kind of technology purely from drawings and descriptions on paper. Ideas and descriptions were embodied in various kinds of models that illustrate aspects of the new technology, or even fully implements an instance of the new technology, to be tried out under different conditions. It is considered important for the engineers to get "hands-on" experience with such models prior to building the final product. This point was expressed in a radical way in an early book with essays on software engineering:
"Where a new system concept or new technology is used one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time.
The management question, is therefore not whether to build a pilot system and throw it away. You will do that. The only question is whether to plan in advance to build a throwaway, or to promise to deliver the throwaway to customers.......
Hence plan to throw one away; you will anyhow."
Brooks, 1975, p 116.
In engineering, the intermediate embodiments of ideas and descriptions are denoted with a variety of terms, such as: breadboard model, an experimental set-up to test the functionality of an electronic or a mechanical component; mock-up, full-size structural model, e.g. of an airplane cockpit, with little or no functionality; prototype, a first full-scale version of a product, such as an airplane; and pilot system, a small scale version of a larger system.
With respect to software development it was assumed for a long time that computer systems could be built from pure paper-based specifications without any intermediate embodiments of specifications by other means. This assumption is fundamental in the widespread "waterfall" models and specification-oriented "life cycle models" for software development processes, see e.g. (Boehm, 1981) and (Yourdon, 1982) for descriptions of such models. However, software developers faced a number of problems in making systems fit the requirements of users and their organizations. Hence, researchers and developers started adopting ideas from engineering on making and testing working models also of parts of software prior to delivering a system. Within software development the term `prototype' became the widespread label used for such working models. The use of the term is, however, different from the use in engineering, which is illustrated in the following.
Webster's Ninth New Collegiate Dictionary, 1987, p 947
The term 'prototype' originates from Greek language meaning "first of a type" and can be traced back to year 1552 according to the dictionary. The term is usually associated with electrical and mechanical engineering disciplines where it was used commonly for a long time. In engineering the typical use of the term 'prototype' is to denote the first complete full-scale version of a product, such as a car, an airplane, or a computer, which is developed and thoroughly tested before procedures for mass production are worked out. This makes a prototype for engineering both time consuming and expensive to produce compared to the individual product that a customer buys. In conjunction with clarifying the requirements of the technology being developed, the problems here are also to verify the design and to set up precise procedures for mass production, since introducing the same error in each of the individual products is extremely expensive for the producing company. Thus a complete full-scale prototype is in fact a cheap learning vehicle in this case, and the prototype itself is usually never used as a product to be sold - it is thrown-away or used for exhibition. In engineering the development of prototypes takes place late in the design process. A prototype is often preceded by development of formal mathematical, physical, or chemical models described on paper, followed by development of more or less complete experimental models, such as mock-ups or breadboard models, as mentioned above.
Even though system development is often considered an engineering discipline, illustrated by the widespread use of the term "software engineering," differs with respect to the characteristics of products and the use of the term prototype. In all non-trivial system development projects, it is a major problem to clarify system requirements, hence working models are also useful learning vehicles for clarifying requirements here. But it is almost never the case that setting up procedures for mass production is a major problem demanding a full-scale prototype.
Software products are typically one-off products for a particular user organization developed in-house or by a contracted supplier, standard packages to be sold to many organizations accompanied with some tailoring to the specific organization, software embedded in electronic and mechanical devices, or it is ready-to-use "off-the-shelf" products. Among these types of products only the embedded systems, share similarities with other engineering products which require a full-scale prototype. Such systems are usually downloaded to ROM[4] hardware components which are mass produced as parts of larger devices. Regarding the other types of products no real mass production takes place. In "one-off"-development a full-scale prototype requires nearly the same resources as the final system. This is true also for standard packages and off-the-shelf products, since the individual products to be sold can be copied to a storage medium at a low price. This implies that full-scale prototypes are only suitable as cheap learning vehicles in few cases of software development. Hence, the common use of the term 'prototype' in software development is to denote an early incomplete working model or version of the future system which exhibits essential features. Software prototypes are intentionally incomplete and open for expansion and modification (Naumann & Jenkins 1982). Software prototypes more often resemble the types of working models, such as breadboard models, mock-ups, and pilot systems, preceding a full-scale engineering prototype. The most important feature of a prototype in system development is that it is possible to develop it early, cheap, and fast compared to the development of a full-scale version of the system. Using this notion of the term prototype, it makes sense to develop software prototypes prior to the final implementation of all four types of software products mentioned above and not only the embedded software. It should be noted here that a release of a software product can be viewed as a full-scale prototype being tested in use prior to development of the next release, but this is a less common use of the term `prototype'. When applied deliberately in a development strategy this is usually called evolutionary development or "versioning" (Floyd, 1984).
The term 'prototyping' is commonly used to cover approaches or techniques based on some kind of software prototype development. But as the following sections shows, the term prototyping is used to cover a rather diverse set of approaches to system design and use of prototypes.
The literature on prototyping within system development can be sorted into three categories according to the research area in which the literature has its origin. The three categories I use are: Information System Development, Software Engineering, and User Interface Design. There is of course some overlap between the categories, but Sections 2.3-2.5 illustrate through examples from the literature that a mainstream of approaches can be identified in each of the categories. I use the framework of Floyd (1984) to describe and discuss these approaches, hence I will briefly introduce this framework before I discuss the literature in the three categories.
Floyd's high-level characterization includes the distinction between horizontal and vertical prototyping. A horizontal prototype is a prototype where none of the functionality is implemented in detail. Only the surface visible and tangible to the user is implemented. Underlying functionality may be simulated or provided as dummy computations. A vertical prototype is a prototype where selected parts are implemented in detail as they are expected to appear in the final system. Other parts of the intended system are neglected in the prototype. This distinction expresses contrasting selection of functional scope, but often in practice prototypes are combined from both horizontal and vertical prototypes.
Floyd's characterization of prototyping approaches is based on a distinction between exploratory prototyping, experimental prototyping and evolutionary prototyping. The goal of exploratory prototyping is to clarify ideas and requirements for a new system. Alternatives may be explored before a solution is selected and further specified. The goal of experimental prototyping is to determine the adequacy of a proposed solution before the full scale implementation begins. Floyd makes a further distinction for this category between full or partial functional simulation, human interface simulation, skeleton programming and base machine construction. This detailed distinction expresses differences in the functional selection for the prototypes and in the way prototypes are constructed. The goal of evolutionary prototyping is to gradually adapt a system to changing requirements, which cannot be anticipated before implementation begins. For this category Floyd also makes a further distinction between incremental development and evolutionary development. Incremental development is characterized as stepwise implementing of a pre-specified overall design. Whereas evolutionary development, also called "versioning," is characterized as iterative development where the overall design changes through a series of cycles of design, implementation, and evaluation.
These categories of approaches should be viewed as archetypes - in practical projects they may be combined.
The presented parts of the framework are applied in the following sections and thus explained further through the examples of use.
Some of the earliest papers on applying prototyping to software development are found in this area. One of the early papers in this category is (Bally, Brittan, & Wagner, 1977). In this paper it is observed that "linear" or phase-oriented approaches to IS development lead to un-intentional loopy processes where a project group jump between the phases and become confused. Reasons given, are that projects usually start from vague ideas about the expected results, i.e the users do not express their needs precisely. To deal with this kind of vaguely defined development tasks they discuss two prototyping approaches denoted the "Plug-in strategy" and the "Prototype strategy." The "Plug-in strategy" is an incremental development approach according to Floyd, because the overall design is decided prior to a division of the system into sub-systems that are developed stepwise and plugged into the collection of already implemented sub-systems. The "Prototype strategy" is evolutionary development where versions of a prototype is developed in cycles of analysis, design, implementation, and operation. Bally et al. (1977) notes that these two approaches are a first cut on introducing prototyping ideas in IS development, and they suggest a more fine grained understanding of prototyping being gained from studying other engineering disciplines.
The journal MIS Quarterly[5] has from the late 70s published several papers about prototyping in (Management)IS development (Berrisford & Wetherbe, 1979; Naumann & Jenkins, 1982; Kraushaar & Shirland, 1985; Janson & Smith, 1985). Most of these papers propose variants of evolutionary development approaches due to the recognition of serious problems with traditional phase-oriented, "waterfall" models for system development. The proposed approaches all draw on emerging technical facilities such as interactive tools, database systems, screen editors, report generators, and so-called Very High Level Languages (VHLL).
For instance, the paper by (Berrisford & Wetherbe, 1979) proposes a so-called "Heuristic Development"(HD) approach. The idea of HD is to substitute two of the traditional phases, "design" and "development/test" with three phases: "Heuristically design/development of output system," "Design of input system," "Development and test of input system." The aim of this is to be able to present a database and resulting output, e.g reports and screens for ad hoc queries, to the users in an early prototype and get the requirements for this part stable before the input system is added incrementally. The assumption is that the input system is much easier to design once the system output and thus the functionality has been clarified through a prototype. The database for the prototype of the output system should initially be designed and loaded with sample data by the designers/programmers.
The paper by Naumann & Jenkins (1982) presents a generic prototyping model for IS development, discusses problems and prospects of the prototyping model, and provides a survey of published examples of applying prototyping to IS development. Hence the paper is a good overview of the early perspectives on prototyping in IS development. The presented prototyping model is outlined in Figure 1.
Figure 1: The prototyping model by Naumann & Jenkins (1982)
It is stressed that the initial prototype should be developed in a very short time ("overnight"), and only need to be based on "half specified" requirements. The prototype should be provided for "hands-on" use by users. Recognized undesirable and missing features should lead to revision and enhancement of the current prototype to produce a new version with a short turn-around time. The steps 3 and 4 (cf. Figure 1) are repeated until they produce a system satisfying the users. Hence the generic prototyping model represents an evolutionary prototyping approach.
In the paper by Kraushaar & Shirland (1985) a "two-prototype" approach is presented, i.e. two prototype versions are made before the final system is implemented by reusing major parts of the second prototype. An interesting point in their description is that they point to the possibility of making modifications of certain parts of the initial prototype together with users: "These 'on-the-fly' changes further encouraged the experimental tone and user involvement." These observations resemble observations made when applying the cooperative prototyping approach as described in Section 5.
The journal EDP Analyzer also contribute to the discussion of prototyping in IS development (Canning, 1981, and 1985). These papers similarly advocate evolutionary approaches to prototyping by describing steps involved in prototyping and examples of applying such prototyping approaches. In these discussions the economic advantages of the approaches, such as shorter development processes, are emphasized, and the economic aspects seem even more important than the quality of developed systems.
The papers (Mason & Carey, 1983; Hsia & Yaung, 1988) propose slightly different approaches to IS prototyping by initially using so-called scenarios for prototyping. Scenario-based prototyping is characterized as "sketching the behaviour of a system (i.e., the external view of a system) without dealing with much of the programming detail of rapid prototyping" (Hsia & Yaung, 1988). This kind of prototyping where only screens, menus and their relationships are implemented correspond to horizontal prototyping according to the framework. In (Hsia & Yaung, 1988) a special purpose scenario-generator is described as a tool for this kind of prototyping. Mason & Carey (1983) describe a method and a tool to support starting with scenarios, but also enabling that "the ultimate prototype can be the product system itself."
Finally, a number of books and reports (Boar, 1984; Xephon Cosultancy, 1985; Lantz, 1986; Lie-Nielsen & Colliander, 1986) discuss prototyping within the IS area. The books (Lie-Nielsen & Colliander, 1986; Lantz, 1986) similar to the papers above argue for a variant of evolutionary prototyping. Lantz (1986) argues for starting with a thorough analysis leading to a description of the document flow of the use domain and an understanding of at least 80% of the data contents of the system before building the first prototype. The initial prototype should be defined on paper first. The initial prototype is thereafter built and incrementally developed to become the system. The possibility of building the first prototype on a micro computer to make it easily accessible to the users is mentioned. But it is an advantage that the first prototype is compatible with the target development environment such that a smooth transfer based on reuse of code can be made. Lie-Nielsen & Colliander (1986) advocate a similar approach. They focus closely on the tools for supporting prototyping in IS development. They in particular emphasize the importance of specifying data contents, validation, and processing rules in a data-dictionary as basis for developing prototypes. The data-dictionary should be integrated in the target development environment to support easy transfer of prototypes into production systems.
Similar to the earlier mentioned authors, Boar (1984) and Xephon Cosultancy (1985) advocate evolutionary prototyping approaches. They actually introduce prototyping life cycles to substitute traditional life cycle models. But they also describe examples of different styles of prototyping and hybrids of those. Among these are: "Screens Only" prototyping[6]; using software packages as prototypes; end-user prototyping; prototyping as part of request for proposal from vendors. Some of these styles may locally fit into certain phases in a traditional project hence prototyping can to some extent be integrated in a traditional life cycle project.
In general the IS prototyping approaches are closely coupled to the use of application oriented development tools such as Application Generators and 4th Generation Systems as described in (Horowitz, Kemper, & Narasimhan, 1985; Martin 1983; Martin & Leben, 1986; Martland, Holloway, & Bhabuta, 1986). The application of prototyping and 4th generation systems does, however, not automatically lead to active user involvement as argued in Paper 1, which discusses problems with prototyping based on 4th generation systems in IS development. It is often claimed to be a major advantage that prototypes can be developed rapidly, but also be reused in the production system. Based on this assumption about prototyping, several authors, e.g. (Naumann & Jenkins, 1982; Canning, 1985; Lantz, 1986), claims that prototyping reduces IS development costs dramatically. Actually some literature highlights this advantage of prototyping even more than its ability to clarify requirements and produce systems better fitting the users needs. This illustrates that the underlying perspective of the approaches typically represents the project managers' and the system designers' point of view on the design process. The perspective can be illustrated by quotes of the claims that prototyping: "Reduces development costs," "Lowers operation costs," "Slashes calendar time required," "Produces the right system the first time," and "Cuts manpower needed" (Lantz, 1986, p. 11).
The majority of papers in the collection from an ACM software engineering workshop on rapid prototyping (Squires, Zelkowitz, & Branstad, 1982) describes tools and techniques for such approaches to prototyping. The key idea is to undertake iterative design and verification of program functionality by means of formal specifications made in specification languages being executable and thus capable of simulating a later efficient implementation of the specification. For instance, in the papers by Balzer, Goldman, & Wile (1982), Feather (1982), and Cohen, Swartout, & Balzer (1982), a specification language called GIST is applied for specifying the prototypes. GIST specifications are purely textual and consists of three components: structural declaration, definition of possible states in the system; stimulus-response rules, definition of behaviour; constraints, definition of possible behaviour given by the structural declaration and the stimulus response rules. It is discussed how such specifications support functional simulation. The typical applications are: specification of detailed functionality, such as handling of complex data structures, flow and processing of data. The specifications are usually limited to cover only critical parts of a system and the evaluation of the prototypes seem to be undertaken by the designers themselves. Assessed from the small examples given, the specification of a real size system would explode into a huge specification, which will be at least the size of the code for a final implementation.
In the paper by Wasserman & Shewmake (1982) from the same collection and the later paper (Wasserman, Pircher, Shewmake, & Kersten, 1986), it is discussed how transition diagrams,[7] as a supplement to tools to specify functionality, can be used for making executable specifications of character-oriented terminal dialogs. Via a Transition Diagram Editor (TDE) the user interface can be specified as a diagram with nodes, connectors, inscriptions and some additional textual specification of screen layouts. A Transition Diagram Interpreter (TDI) can then generate a (horizontal) prototype which can execute the sequence of dialogs and request input from the terminal. All changes to dialogs are made indirectly through the TDE and a new prototype is generated with the TDI. Given the prototyping tool combining executable specifications of functionality with the transition diagram specification of user interfaces, they claim that the application domain for their approach is Interactive Information Systems. In (Wasserman, Pircher, Shewmake, & Kersten, 1986) they briefly discuss problems an prospects of having users test prototypes.
In the paper by Lamersdorf & Schmidt (1984) and the book by Hekmatpour & Ince (1988), it is suggested to use a variant of VDM as specification language for data models and functionality. In both cases interpreters executing the specifications are described. Hekmatpour & Ince (1988) also describes how executable VDM specifications of functionality can be combined with executable state-transition diagrams to specify user interface aspects in a prototyping system called EPROS. A full specification of a prototype of a small Library system is developed in detail in the book as an example. It claimed that this prototype specification can be used to clarify user requirements, but it is not discussed how users were or should have been involved in the development process.
In papers by Tavendale (1985) and Leibrandt & Schnupp (1984), it is suggested to use the logic based programming language PROLOG as specification language for specifying prototypes. In (Tavendale, 1985) PROLOG is used as a meta language to support developing other executable specification languages. The specifications are all made purely textual. The examples of prototypes given are of small pieces of programs dealing with, e.g. the use of a specific data-structure. Only abstract arguments are given concerning the possibility of involving users in tests of prototypes.
Recently, these functional simulation approaches have produced an offspring oriented towards IS development, the so-called CASE-tools (Computer Aided Software Engineering-tools). The goal of these tools is to be able to make formal specifications for large scale IS systems by means of data-flow diagrams, structure charts, state-transition diagrams and the like, which have been made executable. The ultimate vision is to be able to generate the whole production system including documentation for users and maintenance people from the specifications. Some authors talk about this idea as a "Software Factory," see several chapters in (Ng & Yeh, 1990). Another less ambitious vision of these tools is to have support for executable specification prototyping. The state of development of this kind of tools have not fulfilled the ambitions yet. The current CASE tools boils down to be tools for combining data flow descriptions, pseudo-code specifications of programs and a data dictionary.[8] The code generation is typically restricted to generation of (relational) database definitions from the data dictionary. Yourdon (1989) describe the widespread Structured Analysis and Design (SA/SD) method and how it is commonly supported with CASE-tools. It is premature to assess how the CASE-tools impact on the users ability to be involved in prototyping.
Recommendations covering other aspects of the system development process than the functional simulation are seldom given, e.g. there is little concern about proposing alternative life cycle models.
Generally spoken, the perspective of the approaches in this category is mainly an engineer's perspective focussing on developing a technically reliable (software) product. This can be illustrated with the statements by Hekmatpour & Ince (1988): "Apart from being intellectually appealing this technique [executable specification] ensures that a precise level of documentation is always available to the developer" (p. 20). "Used in conjunction with mathematical verification such prototyping [with the EPROS system] is capable of producing a system with a high degree of correctness..." (p. 81).
An early example, of a prototyping effort both leading to the design of a standard software package and stating a guiding example for development of graphical user interfaces, is the pioneering work on the Xerox Star User Interface. This work also contributed to a dramatical expansion of the research and design field focussing on user interfaces based on the development of graphical workstations in the early 80s. A paper by Smith, Kimball, Irby, Verplank, & Harslem (1982) describes the development of the Star User Interface which was the first existing graphical user interface in line with what is well-known today from, e.g. the Apple Macintosh. Smith et al. describe how the Alto machine, a predecessor to the Star, served as a prototyping environment:
"The Alto served as a valuable prototype for Star. Over a thousand Altos were eventually built. Alto users have had several thousands of work-years of experience with them over a period of eight years, making Alto perhaps the largest computer prototyping effort ever. Dozens of experimental programs were written for the Alto by members of the Xerox Palo Alto Research Center. Without the creative ideas of the authors of those systems Star in its present form would have been impossible. In addition, we ourselves programmed various aspects of the Star design on Alto, but all of it was 'throwaway' code"
Smith et al., 1982
The development of the Star user interface took place as a large exploratory prototyping effort, where alternative ideas and solutions to problems were tried out by means of prototypes on the Alto machine. When design ideas were clarified the Alto prototypes were thrown away, and a Star implementation was made. In this case the development of the user interface software was closely coupled to the development of the hardware, hence the notion of prototyping here resembles the traditional engineering use of the term. The development in other respects as well resemble engineering, for instance by the fact that the designers based the development of the user interface on the formulation of a model, in advance to development of prototypes. In this case it was not a mathematical or physical model, rather it was the designers' conception of "the user's conceptual model." With respect to the user involvement, the paper only report on the researchers' experimental use of the Alto as a prototyping environment, it does not report on involvement of, e.g. office workers, who were among the potential users of the Star, in the evaluation of the prototype user interface.
To further illustrate the approaches in the user interface design area Wilson & Rosenberg (1988) provide an introduction to and an overview of tools and techniques for rapid prototyping in user interface design. They list three classes of techniques: (1) storyboard/sketching and slide show techniques, (2) Wizard of Oz simulations, and (3) animated and testable simulations. The techniques are all exploratory approaches according to the framework due to the flexibility provided to experiment on alternatives, and due to the fact that the prototypes themselves are not the important result; because they only in few cases are parts of completed systems.
Techniques in class (1), storyboarding/sketching and slide show, have recently received much attention. Such techniques are described in, e.g. (Andriole, 1989; Vertelney, 1989; Booker & Vertelney, 1990). The idea of these techniques is to illustrate user interaction with a future system by means of cheap and simple tools. The techniques are inspired from industrial design and from film production, choreography and the like where directors envision the scenes of a play by means of scripts and storyboards prior to instruction of the players. Curtis & Vertelney (1990) in their CHI '90 tutorial provide a detailed introduction of storyboard prototyping and they list the following elements of the technique: scripts, drawings, screen shots, flipbooks, animatics, puppets, computer animations, special effects, and interactive software. These elements can be combined in many ways to envision a future interface. Scripts are narrative descriptions of the interaction between the user and the application, and storyboards are sequences of images serving as visual representations of the scripts. Animation of images on a computer or pieces of video can be used to illustrate the behaviour described in scripts and storyboards. These means are mainly useful for the designers to discuss the design themselves. Later in the process interactive software based on the use of tools, such as HyperCard and VideoWorks[9] may be used to implement flexible prototypes that can be tried out by users in so-called user testing (Salomon, 1990) or user observation (Gomoll, 1990). User testing and user observation is undertaken by bringing in more or less arbitrarily chosen people as "users" and asking them to perform certain tasks using the prototypes. The designers observe the users trying out the prototype and ask about their reactions to the user interface. Based on these observations the designers make decisions themselves on how to improve the prototype.
These emerging storyboard prototyping techniques to a large extent resemble the use of plywood, slide, and cardboard mock-up techniques applied in the UTOPIA project in the beginning of the 80s as discussed later in Sections 3.1 and 3.2. However, the perspective on user involvement in the UTOPIA project was much different. The system being designed was aimed at a certain profession and users, representatives for the profession, were in this case actively influencing the design.
Techniques in class (2), Wizard of Oz, are for instance described in the paper by Good, Whiteside, Wixon, & Jones (1984). This paper gives a good example of a controlled experiment based on prototyping aimed at producing general guidelines on command language user interfaces. Wizard of Oz is a special kind of prototyping based on human simulated functionality (this is discussed in further detail in Paper 3 Section 4.3). The aim of the prototyping experiment was to establish a controlled experiment which was set up as follows. A number of people (called users) with no previous experience with computers are asked to perform certain specified tasks concerning sending and receiving email. The users are placed in front of a terminal and told to issue whatever commands they find appropriate to solve their tasks. If the commands are known by the prototype's command interpreter it is interpreted immediately. If it is not known a human operator hidden to the user tries to make sense of the command and give the prototype a correct sequence of commands to perform the task and provide the user with a (slightly delayed) feedback. If that is not possible the operator responds to the user with a message about where the problem occurred in the interpretation.
The users' spontaneous commands were logged and the designers iteratively extended the prototype command interpreter to understand a larger set of the commands that novice users typically issue. The results of this prototyping effort were guidelines such as: provide synonyms for commands, and provide flexible parametrization for commands to make it easier for both novice and experienced users to learn and use a command language interface. It should be noted that the example given is prototypical for controlled experiment design processes where users are involved. The users are involved as test subjects to participate exactly as the designers prescribe. They were in this case even deceived to believe they were sitting in front of an extremely powerful command language interpreter. They were never told that the interpreter was faked by human simulated functionality.
Techniques in class (3), fully animated and testable simulations, may be hard to distinguish from interactive software prototypes mentioned in connection to the storyboard techniques, but here is some examples.
The Trillium design and simulation tool for "simple" machine interfaces is an example of a tool supporting such prototyping techniques (Henderson, 1986). Trillium is used to prototype control/display user interfaces for "simple" machines such as photo copiers and printers. It supports direct manipulation design of the layout of the user interface plus simulation of behaviour such that each simple item is associated a piece of (LISP) code specifying the behaviour, and the behaviour of a composite item is represented by the behaviour of its component items. Trillium is mainly used by photocopier designers internally at Xerox to prototype the interface with a flexible tool before building the actual hardware and software controls/displays for a new photocopier. A Trillium prototype cannot be reused for the final implementation, since it is running on a general purpose computer which is not compatible to the special purpose computer in, e.g. a photo copier. As an aside here it should be mentioned that Blomberg & Henderson (1990) analyze the development of Trillium itself, in particular how it was actually undertaken in cooperation with the photocopier designers. They, however, conclude that the development in several ways differ from ideals of participatory design, for instance, by the fact that the development was mainly technology driven rather than taking the copier designers work situation as the starting point.
So-called User Interface Management Systems (UIMSs) may also be used to provide prototypes to support this class of techniques, where prototypes can be reused for the final implementation. UIMSs are aimed at isolating the part of applications handling the user interaction into a separate environment which is integrated in the application software (Tanner & Buxton, 1983). Hence UIMSs can be used to modify the user interface experimentally without affecting the underlying functionality. UIMSs may also support building horizontal prototypes simulating interaction without having implemented underlying functionality yet (Lewis, Handloser, Bose, & Yang, 1989).
To put it a little different, the approaches in this area are having an experimental research perspective aimed at formulating generalisable theories. Hypothesis concerning feasibility of the technical aspects of user interfaces are tested by exploring alternative proposals experimentally. Prototypes are used for setting up controlled experiments to assess mainly the learnability and the user friendliness of a certain type of user interface. The "users" who are involved in the controlled experiments are viewed as test subjects producing data to be analyzed by the designers. The test subjects to be involved should share certain characteristics to enable generalisation of the results. For experiments, where learnability and user friendliness is assessed, it is often considered an advantage to pick novices, i.e. people who have little or none experience with computers. Based on more or less structured observations of the test subjects the designers draw their own conclusions on how to improve the user interface which guide-line to give on a certain type of user interface.
An improved prototype is not per se the important part of the approaches it is rather the prototype's ability to lead to general conclusions on the design of a certain product or guidelines for a family of user interfaces which is important. Hence the approaches are also viewed as process-oriented (Floyd, 1984) - the learning process of building and experimenting with prototypes is more important than the resulting prototype.
However, some literature on prototyping deals explicitly with a more active user involvement than described in the categories above. I mention this literature here even though it may belong to the IS development area. The first work is that of Pape & Thoresen (1987) and Thoresen (1990). These papers describe an iterative prototyping based development project aimed at creating new work organization and supporting computer systems for Town Planning departments in Norwegian municipalities. The key characteristics of the project are: system development takes its outset in an organizational change process initiated by workers and management in cooperation, and organizational changes take place in close interplay with development/evaluation of prototypes. In the described approach, prototypes of essential aspects of a new system are developed to a degree of stability that enables users to perform real work tasks with them. Thoresen (1990) reports on difficulties in evaluating prototypes in use this way, she states that the quality of prototypes cannot be evaluated from studying the prototype itself. Thus quality can neither be derived from the internal quality of the program nor from the "user friendliness." For instance, Thoresen (1990) observes that prototype demonstrations only deal with normal use, unusual and unexpected cases are never considered this way. The conclusion is that the quality of prototypes should be evaluated from the role they play in the workers use of them.
The second work to be mentioned is that of Reisin & Wegge (1989). They discuss experiences with prototyping in a project denoted PEtS (Participative Software Development for Transparency in Computer-Supported Work). The project uses the so-called STEPS approach (Floyd, Reisin & Schmidt, 1989) and it is aimed at supporting administration of Wage Agreements it the German Trade Union. Reisin & Wegge claims that "Prototyping is social interaction between developers and users," and they claim maintaining this interaction requires deliberate focussing on the process. For instance, they observe potential dangers, such as: the users may view the prototype as a bad provisional solution to their problems; and the developers can easily take over and run the joint sessions without getting the users really involved. To cope with such dangers Reisin & Wegge (1989) give suggestions on terminology to use and steps to take to prepare for prototyping. They emphasize the importance of clarifying the goals of the prototyping prior to the session. And they introduce a distinction between four types of experimental prototyping: exploratory experiments, users and developers jointly clarifying conceptual problems; decision-making experiments, users and developers jointly choosing among alternatives; evaluative experiments, users and developers jointly evaluating a particular component to be built into the target version; pilot experiments, users have the opportunity to use and evaluate a prototype over a longer period.
The works described here are among the few examples of literature outside my own research environment focussing on prototyping as a cooperative activity. The mainstream prototyping approaches follow the lines described in Sections 2.3-2.5, and take more strict software engineer, project manager, or experimental researcher perspectives on system development, i.e. the main goal is to develop "user friendly" and reliable programs at a low price. None of these approaches explicitly consider user perspectives.
In a user's perspective the focus when considering new technology is on work processes, control over own work processes, division of labour, qualifications required for new work tasks, how interaction with new systems like, etc. Hence, high quality of computer systems from users' points of view means high quality support for the users' specific work and the work of the organization as a whole. Taking these issues into account, a system of high quality can be characterized in several ways, for instance: A high quality system means a tool supporting users work and enhancing their skills similar to the ideal of a craftsman using and building his own tools (Ehn, 1988). A high quality system enable users to perform work with a minimum of breakdowns (Winograd & Flores, 1986; Bødker, 1991) caused by the use of the system, i.e. the system should be as much as possible a transparent tool for the user. Program correctness is of course a prerequisite to get few breakdowns, and low development costs are also important from the point of view of users who want access to new tools. But these issues are usually not the main concerns when users evaluate a new system design, the focus is rather on what future work with the system will be like. When it comes to development of high quality systems meeting the users' expectations current approaches to prototyping share a number of drawbacks similar to traditional system description approaches. The main reason for this is that the approaches still let the users have a passive role in the design process (Paper 1). Put in another way system design is not sufficiently based on the users practical understanding, which is crucial to design a high quality system according to the ideals mentioned above. The users are acting as evaluators of technical proposals, made by designers on their own, rather than active contributors to design of computer systems influencing their future work-place. The ideals of high quality computer systems and ways to overcome some of the deficiencies of traditional prototyping approaches are further discussed in Sections 4 and 5.
The notion of user involvement in system development is discussed in literature under several labels such as: Participative design, User-Centered Design, Collaborative design, Prototyping, and Cooperative Design. This section gives a brief historical overview of dominating perspectives on user involvement in system development. Hopefully it also provides bit of clarification on what is the difference between the perspectives. The overview is based on a mixture of secondary literature, such as (Kubicek, 1982), and (Bansler, 1987) dealing explicitly with the history, and original sources describing the perspectives.
An application of Socio-Technical approaches in conjunction with system design is described in (Mumford & Ward, 1968). The researchers illustrate how the approach is used on an example with a management in a bank who wants to introduce a new centralized account and customer information system. The main contribution from the approach in this case is an analysis of so-called attitude-variables among the bank personnel in order to help the management in choosing a strategy that minimizes personnel reluctance towards the new system. The main outcome of the application of the approach is a set of alternative management strategies which can be chosen dependent on how the personnel react to the starting development of the new computer system. From the example given the approach seem to be aimed more at seducing shop floor workers than at involving them in the change of their workplace.
But the socio-technical approach as developed by the Tavistock Institute was further developed and applied to several system development projects in Europe, but also in North America and also in the USSR. The approach was interpreted in several ways also making the approach more participative than the illustration given in (Mumford & Ward, 1968) mentioned above.
Lucas proposes to solve the organizational behaviour problems by creative system design, which include: involvement of users in planning the organizational changes being necessary in conjunction with the introduction of a new information system. To address both the technical problems and the organizational behaviour problems Lucas propose guidelines on designing the user interface to information systems. In addition, he emphasizes that the quality of the system should be assessed by the users and not from a technical point of view. He also proposes iterative design of system input/output such as screens and reports to let the users decide on the design. Thus he proposes a kind of prototyping without mentioning the term.
In the proposed creative system design approach Lucas is paying attention to users who are decision makers. He is aiming at going beyond decision support at an operational level. Thus his conception of a user is mainly decision makers, i.e. managers at a strategic level in the organization. In general the socio-technical focus on management level users is in contrast to approaches discussed later, where the focus is on shop floor workers.
With respect to tools and techniques the socio-technical approach only contributes with general recommendations. The approach mainly appeals to system developers and managers to change their attitudes towards paying attention to social factors, such as job satisfaction, when developing computer applications. According to Bansler (1987) socio-technical recommendations for system developers boils down to slight modifications of traditional phase-oriented system development models. One such modification is to add a phase or an activity focussing on "organizational system analysis" as a supplement to the traditional activities. Such a socio-technical phase model for system development is reprinted in (Bansler, 1987, p. 160.).
The work in these Scandinavian projects was carried out as action research by researchers working together with shop floor workers in companies where new technology was or was about to be introduced. This work was mainly undertaken at a company level. The researchers participated on the union and worker side of the table. Consequences of the introduction of new technology were analyzed in details by workers and researchers in cooperation. Among the consequences being analyzed were for instance: change of control of work, long-term job-security, de-skilling of job categories, changes in division of labour. Based on the consequence analysis, researchers and union representatives aimed at influencing the way new systems should be designed and used in that particular company. In few cases such work led to a change of the system being introduced (Bergo & Nygaard, 1974), but often the work was taking place after a new system was introduced by the management of the company. Thus, it was hard for the workers and researchers to change the design of the system. Rather they could work on influencing the intended use of the system. In few cases the workers' reluctance to use a new system stopped the final installation (DUE, 1981, pp. 25-32).
The results of these projects can be seen at two levels. On the one hand the actual workers in the companies improved their work situation with the new technology. On the other hand these early research activities resulted in development of action strategies for trade unions, educational material and programs (Nygaard & Bergo, 1972; DUE, 1981) and introduction of technology agreements (Mathiassen, Rolskov, & Vedel, 1983) between unions and employers' organizations. In particular the idea that trade unions should work on the introduction of technology agreements in organizations was getting a lot of attention outside Scandinavia as a result of these projects.
The trade unions at that time possessed only modest competences for actually engaging in the design of computer systems. Hence, the kind of user involvement that was experienced in this work was either at a level of strategic negotiations or at a level of influencing the use of particular systems. There were only few examples (Bergo & Nygaard, 1974) of users being involved in actual design activities influencing a system under development. But a lot of workers and in particular shop stewards were trained in negotiating technology issues within their organizations or their trade in general. In the next phase to be described, however, trade unions took steps themselves to engage in alternative design activities.
In the UTOPIA project it was experienced that traditional system specification techniques were insufficient to let workers participate in the actual design activities. Thus, several different techniques were tried out to involve graphical workers in design activities aimed at developing alternative technology. The UTOPIA project continued to the midst 80s and it has led to a number of suggestions on concrete techniques to support user involvement in system design, (Bødker, Ehn, Kammersgaard, Kyng, & Sundblad, 1987). Design techniques based on plywood and paper mock-ups and even prototyping were introduced in this project. It was also in relation to this project that the idea of viewing computer systems as tools emerged (Ehn & Kyng, 1984). When applying a tool perspective on computer systems, the users are seen as persons who possesses certain skills specific to the application domain that should be supported by the tool. The tool perspective is inspired by the way tools were developed and used within traditional craftsmanship. Here the use and development of tools is deeply rooted in a practical understanding which is hard to articulate, and thus considered as tacit knowledge (Polanyi, 1966).
Mumford (1987) gives a brief overview of the socio-technical techniques. These techniques are still aimed more at organizational design than at design of computer systems. The key technique is description, and the key aspects to describe are: information flow, variances in work, and the social system. Compared to other approaches to system description the advantage of these techniques with respect to user interests is that they focus also on the social system and on variances, i.e. exceptions to rules and standards.
With respect to the overall design strategy Mumford (1987) states a number of abstract principles to follow, among which one notable in this context: "9. Principle of incompletion. This principle states that the design is an interactive and continous process." This points to an evolutionary and an iterative design approach similar to ideas related to prototyping, but here mainly applied to organizational design.
In relation to the discussion of actual design techniques the following theses/assumptions are of particular interest: "labour processes cannot be reduced to information processes" and "Important aspects of labour processes - in relation to design of computer support - cannot be formally described." Ehn and Kyng (1987) claim that these theses are strongly supported by the experiences from the projects. In addition, these experiences also cohere with philosophical ideas of, e.g. Polanyi (1966) on tacit knowledge. These experiences stress the need to go beyond existing system description techniques when involving users in system design.
These assumptions lead to theses guiding the design process: "Design should be done with users, neither for them nor by them" and "mutual learning should be an important part of the work in a design group." The key point is that both experience and knowledge on the labour process of the users and experience and knowledge on computers are needed to design a valuable system for the users. Users and designers need to transcend their own experience and knowledge in order to be innovative and improve the conditions for the future work process by using computers.
A more concrete thesis on how to establish this mutual learning is given in the discussion of the thesis: "Design by Doing." Here it is argued that it is important to not only base design activities on descriptions, but also to simulate the future work with computers by means of mock-ups or prototypes. Although Ehn & Kyng (1987) mention prototypes and prototyping tools they are more in favour of using simple mock-ups made of paper and plywood etc. This is because mock-ups are claimed to be: cheap to create, easier to manipulate than computer-based prototypes, and not restricted by the limitations of current technology. The mock-up techniques allow users to be active in the design process, because they can participate in the manipulation of the mock-ups, and because they can to a limited extent simulate the use of systems by means of a mock-up.
Within this research area it is considered important to take the users into account when designing user interfaces. But the approaches to do this are typically limited to evaluation techniques such as, controlled experiments (Baecker & Buxton, 1987; Monk, 1987) "user testing" (Gould, 1988; Salomon, 1990) or "user observation" (Gomoll, 1990) techniques. Methods has been taken from experimental psychology to set up controlled experiments with "users" or rather test subjects trying out a user interface under well-specified conditions. The "users" participating in such experiments are often chosen arbitrarily among people "from the street" or among secretaries in the staff of the development organization. The test subjects are brought to a laboratory to test a user interface that is required to user friendly, i.e. easy to adopt (see also Section 2.5). The test subjects are asked to perform certain pre-specified tasks with the system. Their interaction with the user interface is logged and observed and sometimes the test subjects are told to "think aloud" about what they are doing and what they expect the system to do (Lewis, 1982). Sometimes the test subjects are also asked to answer a questionnaire after the experiment. The observed reactions, recordings of the number and types of errors made, and the answers to the questionnaire are analyzed by the designers and used to redesign the user interface or to state guidelines for new design activities. The test subjects never get the opportunity to directly influence the design. Users are considered as representative test subjects that from a psychological point of view could give an idea of how users in general would react to a certain interface design. These techniques are mainly used to design and evaluate user interfaces to systems that are supposed to be marketed to a broad group of users which should be able to start using the systems quickly. The focus in this research is thus not on development of systems for specific professions or user organizations. General user friendliness is given priority over supporting specific users' skills and enhancing their ability to increase the quality of their work.
Within this area of user interface design the ideas of prototyping are widely applied, see Section 2.5. Prototypes are however often developed by designers on their own and tested by users in the sense described above. In some cases, prototypes with human simulated functionality ("Wizard of Oz") are even used to deceive the test subjects to believe that they are working with a really advanced system which was not developed yet (Good et al., 1984).
But some researchers in the HCI field go beyond this restricted sense of user involvement. In the mid 80s a group of HCI researchers with relations to University of California at San Diego wrote a book entitled User Centered System Design edited by Norman & Draper (1986). This book is a kind of mosaic of both psychological and technical issues to encounter when designing User Interfaces. It is neither a book of methods nor does it explicitly point to tools and techniques supporting users in influencing system design. But it outlines a "pluralistic" set of perspectives on the notion of User-Centered design. Some of the chapters argue for a more active user involvement than the approaches mentioned above. for instance the chapter by Bannon (1986), points to a broader context of use than generalisable aspects of users and task domains. Bannon points to the importance of considering social and organizational issues and involving the people who are actually going to use the technology when designing user interfaces. Several chapters, e.g. (Norman, 1986; Miyata & Norman, 1986), in the book do, however, also focus on psychological characteristics which are considered general for humans, and thus are important to consider when designing user interfaces to computer systems. An underlying assumption in these chapters is that designers to large degree can achieve a general theory or a model of users (and task domains): "We need to know more about how people actually do things, which means a theory of action. There isn't any realistic hope of getting the theory of action, at least for a long time, but certainly we should be able to develop approximate theories. And that is what follows:..." (Norman, 1986, pp. 37-38).
It need also to be remarked that several authors within the HCI field have found it frustrating to be limited to the low level of user involvement which is often the case when working with design of user interface to systems (Grudin, 1990a; Bannon, 1991). The user interface designers have too weak a basis for designing a good interface when they do not have a picture of who the users might be. These authors describe cases from projects where they were prohibited in getting access to end-users of the products they were designing interfaces for. Reasons for this were deeply rooted in organizational constraints in the development organization. Human-factors people are not assigned to projects before the software engineers have developed the core functionality of the new system. It is the marketing department who maintains all contacts to customers and they are supposed to propagate user comments to the software engineers and human factors people. Both Grudin (1990a) and Bannon (1991) are strongly arguing for achieving a more active user involvement in user interface design.
The idea of this book is to take concrete professions, organizations, and workplaces as the starting point for design activities. People who are actually going to use the technology should be involved as full participants in a cooperative design activity to create the needed computer system and the workplace around it. The book lays out a variety of techniques to facilitate a cooperative design process among professional designers and professionals from other domains. Future Workshops, Mock-up techniques, Organizational Games and Cooperative Prototyping are among the techniques being suggested in the book[14]. A key issue of these techniques is the use of tools and materials having a family resemblance to tools and materials from the users' work enabling them to participate and take initiatives in the design activities.
Compared to the Collective Resource Approach, the ideas of this book goes beyond a pure trade union based design approach. The ideas are presented in a way that makes them applicable in workplaces and organizations without assuming a strong trade union relationship. However, skill enhancement, democratic influence and awareness of conflicts in organizations are fundamental goals and assumptions in the book. In the chapter by Bannon (1991) it is noted that the perspective of Design at Work, goes far beyond the limited involvement of users in test and evaluation activities after implementation of a system or a prototype as often the case within a HCI perspective. Rather the focus of the book is on involving users early in the design process to make them contribute also in analysis, idea generation and making design decisions.
Within the socio-technical approaches the designers always take part for management. But a change in approach can be seen. In the beginning these approaches were just paying attention to human and social factors through investigations where workers are supposed to supply the designers with information. Later the approaches focus more on setting up iterative design processes where designers experiment together with workers on different work organizations around new technology.
Within the approaches with a trade union perspective the level of user involvement has always been extensive and it still is. The designers originally took part purely on the side of the trade unions. But within these approaches there is a move in perspective. In the beginning the approaches was aimed at fighting for influence on the design of management developed systems. Next the focus was on developing alternative technology together with trade unions with no direct influence from employer organizations. Now these approaches are becoming open for application in projects within organizations where groups of managers and workers are involved as full participants in design activities.
Within the HCI field the dominating approach is to bring users or test subjects in to participate in tests of user interfaces arranged after a system (or a prototype) was developed. This is done to adjust the user interface design of already design products. But within this field there is also a growing concern for getting access to users earlier in the design process and to involve them more actively.
Hence the trends within all the areas concerned with user involvement seem to be going towards a more active user involvement than seen earlier. The approaches rooted in the trade union perspectives still have the most radical suggestion on active user involvement namely: full participation. This focus on full participation from both designers and future users also suggests a shift of terminology. Instead of the term "participatory design," having the connotation that users are allowed to take part in the designers' business; the term "cooperative design" is being used more frequently to bring about the message that both users' and designers' competences are equally important to develop a useful system. When this thesis emphasize active user involvement in system development it corresponds to an understanding of design as a cooperative activity. This is further developed in the next section on the cooperative prototyping approach.
Paper 1 presents an analysis of the prototyping issues from this field study. In the nine Danish projects analyzed, 4'th generation systems were applied to develop mainly horizontal prototypes and in two projects also vertical prototypes.[16] In the seven projects where only horizontal prototypes were developed, this was done within a traditional phase-oriented project model. Whereas the projects using also vertical prototypes were following a more iterative approach starting from a horizontal prototype and incrementally extending it with functionality (vertical prototypes) that were tried out by users. The horizontal prototypes were used to substitute parts of the requirements specification. They were demonstrated to user representatives at meetings and occasionally made accessible to the representatives at their terminals. This usage of the prototypes did, however, not lead to much feedback from the involved user representatives. Reasons were: the demonstrations did not let the users get hands-on experience as part of the evaluation, the evaluations of prototypes accessible on the users' terminals were not planned, and the users were expected to try out the prototypes on their own initiative in between other work. The developers in many cases felt disappointed that they did not get more reactions to the prototypes. However, in most cases the developers viewed this silence as an acceptance of the prototype and hence of the requirements specification. The requirements specification was then frozen and the system implemented. But in most of the projects subscribing to this strategy a lot of request for changes were raised by users when they had the opportunity to test or use the "final" system. Some of these requests were serious and led to unexpected iterations, budget overrun, and even in one case cancelling of the project.
Users need to be actively involved in prototyping - passive participation in demonstrations and unplanned evaluations of prototypes is insufficient to get benefits from prototyping.
In the two projects using an iterative strategy with on-going tests of vertical prototypes there were much less problems when the users were confronted with the final system. In these cases user representatives had been able to perform work-like tasks with the prototype before the requirements specification was frozen, thus many of the problems with the design had been found and fixed before the most extended version of the prototype was cleaned up and the final system implemented.
In both types of projects there were frustrations among the developers. In the first type of projects frustrations occurred at the end, because the prototyping approach did not lead to a better feedback from the users and a better design (less user complaints on the final system) than did the specification oriented approaches. In the second type of projects the developers were frustrated during the last half of the project as well. When the developers introduced the vertical prototypes step-by-step, they received much feedback on the system design from the users; because they now could relate them to their work, and they started performing work-like tasks with the prototypes. This feedback led to several time-consuming changes in the vertical prototypes. However, developers still considered the system modules as prototypes and many suggestions by the users were followed. Proposals not being implemented were recorded for possible later developments, hence the introduction of the final system went a lot smoother in these projects.
Prototypes need to be closely related to the users' work practice.
In Paper 1 these observations are explained by the fact that users need to have work-like experiences with prototypes to draw on the tacit aspects of their knowledge. These tacit aspects were not surfaced until the test phase in the projects freezing specifications based on horizontal prototypes. The observations also lead to suggestions for how to improve prototyping techniques in order to get better results in the context of the study. Here better results is seen from the developers point-of-view. On the one hand, the introduction of the final system should be smooth. On the other hand, the developers would like to have the feedback from the users early in order to avoid "expensive" changes to the modules of the final system. The suggestions given in Paper 1 were: (1) involve end-users in building initial prototypes, (2) utilize simulated functionality to support early work-like testing of prototypes, and (3) organize prototyping sessions carefully. Elaborations on these suggestions are key aspects of the cooperative prototyping approach as described below.
"... the development of any computer-based system will have to proceed in a cycle from design to experience and back again. It is impossible to anticipate all of the relevant breakdowns and their domains. They emerge gradually in practice. System development methodologies need to take this fundamental condition of generating the relevant domains, and to facilitate it through techniques such as building prototypes early in the design process and applying them in situations as close as possible to those in which they eventually be used."
Winograd & Flores, 1986, p. 171
According to Winograd & Flores (1986) a breakdown is a moment of interruption in our usual, unhampered, habitual acting, or 'being-in-the-world' in Heidegger's terms. Using Heidegger's terms this expresses that objects involved in the acting and conditions for the acting move from being 'ready-to-hand' to be 'present-at-hand'. In plain language it means that a breakdown makes us consciously aware of objects and conditions that in normal habits of acting are treated intuitively and unreflected. A third way of putting this is to say that the objects and conditions for our everyday acting are transparent to us until problems are encountered and force us to reflect on our situation. An example of a breakdown situation is the following: I am busy writing a paper using a word processor on one of our networked workstations. I hit command 'S' frequently to save my additions and changes to the fileserver. At some point I hit command 'S' and instead of a successful save, a message shows up on the screen saying: "Cannot perform Save operation - file system is full." This is a breakdown situation, where I suddenly become aware of the fact that I am sharing a fileserver with a number of other users, and that this server is limited in size. I am now facing the fact that I need to do something unusual to get my work saved to the server. I need to do one of the following things: contact the system administrator to ask him to clean up the fileserver for temporary files, traverse my own files to see whether there are some files that I don't need anymore, or find a way to save the file to another server or device. By the time when I have solved my 'save' problem I have probably forgotten all about the paragraph I was in the middle of writing.
Breakdowns in a domain of action such as writing papers using word processors on networked workstations cannot be fully avoided nor can they be fully anticipated. A domain of action also have a domain of breakdowns, i.e. recurrent breakdowns. A domain of breakdowns is the language we incrementally build up about possible actions to take to deal with the potential breakdowns in a domain. In the case of writing papers on networked workstations we build up a language and a set of actions to deal with the recurrent breakdowns related to limited file system resources, printers needing paper, cumbersome backup routines, slow network traffic etc.
"A breakdown reveals the nexus of relations necessary for us to accomplish our task. This creates a clear objective for design--to anticipate the forms of breakdowns and provide a space for action when they occur. It is impossible to completely avoid breakdowns by means of design. What can be designed are aids for those who live in a particular domain of breakdowns."
Winograd & Flores, 1986, p. 165
Thus uncovering recurrent patterns of breakdowns in a domain is the challenge when designing computer-based systems for the domain. The technique for establishing work-like use situations which is introduced in the next section is an attempt to meet this challenge.
".... we ought to talk about human operation of a computer application rather than human-computer interaction. The user interface can be defined as the software and hardware supporting human operation of the computer application in a specific type of use activity, constituting some of the material conditions for triggering specific operations in a specific use situation. This triggering is part of the non-articulable aspects of practice which play a special role in user interface design: we cannot deal with them without dealing with the specific use activities in which the artifact is to be applied.
Bødker, 1991, p. 140
Operations, in Activity Theory terms, according to Bødker, mean sensomotoric units performed unconsciously by humans when carrying out conscious actions to achieve the goals of the activity. By redefining human-computer interaction in terms of operations, and not in terms of actions, Bødker stresses the importance of considering the unconscious and tacit aspects of the use activity when evaluating user interfaces. Furthermore, she conclude the following on the design activity:
"Design means conceptualization of former operations and creation of new ones. Furthermore, design may mean automation of former human operations. Design deals with operations and conditions by which they are triggered. We design new conditions for the collective as well as the individual activity."
Bødker, 1991, p. 144
Bødker also introduce a breakdown to be a conflict between assumed conditions and actual conditions for operations. And successful design is defined as design leading to few breakdowns in the future use situation. Thus good design becomes the ability to consider the future use activity when designing a computer application in the present. Bødker argues that mock-ups and prototypes of applications being developed can be used to let users try out future actions and operations in real or simulated settings and thus experience parts of the future conditions. In a paper, Bødker (1987) discusses these issues further with respect to prototyping.
Bødker's novel application of Activity Theory to design of user interfaces for computer applications provides, similar to the work by Flores & Winograd, a strong theoretical motivation for developing the cooperative prototyping approach. In particular, the Activity Theory framework emphasizes the need for developing an approach enabling users to experience future use situations when cooperating with designers during the design activity.
The philosophical work by Winograd & Flores (1986) and by Bødker (1991), proposes an ideal of good computer systems as follows:
Good computer systems should cause few breakdowns in use. Good systems are also accompanied with a language of actions to handle recurrent breakdowns.
Similarly the authors point to important issues to consider for in order to develop good systems:
Unreflected and unarticulated aspects of users' work need to be considered to design good systems.
Wittgenstein's theories deals with human language, interaction and communication. Ehn draws on the later theories of Wittgenstein introducing the idea to view interaction and communication as language-games. Based on Ehn's (1988, ch. 4) interpretation the language-game concept stresses that communication goes beyond a simple exchange of communication units such as words and sentences with an explicit given semantics. Communication viewed as a language-game is a social activity where the participants need to share and master certain techniques and conventions inherent in a given practice, the form of life, to understand each other. Human practice is based on practical understanding rather than knowledge that is articulated through formal language. Sharing background hence becomes important for being able to communicate about human practice. However, humans with different backgrounds may understand each others language-games partly, because there is a family resemblance between the games. For instance, a system designer, may be able to understand parts of architects' professional language-games, because there is a family resemblance between designing houses and designing computer applications.
With the Wittgensteinian framework Ehn expresses the challenge of design as follows:
"To design new artifacts that are useful for people, designers have to understand the language-games of the use activity, or users have to understand the language game of design, or users must be able to give complete explicit descriptions of their demands."
Ehn 1988, p. 108
The latter alternative outlined is, according to Ehn, the dominating assumption in existing design methods, but following the line of Wittgensteinian philosophy it has to be excluded. Hence, the real challenge in design is to create a family resemblance or a constructive interplay between the language-games of design and the language-games of use in a particular domain. Ehn's suggestion on how to cope with this challenge is expressed in the following quote:
"What kind of design artifacts may be applied to support this interplay between language-games, and to make this mediation from one to another possible? I think that what we in the UTOPIA project called design-by-doing methods, e.g. the use of prototyping, mock-ups, scenarios etc. are good examples.
The language-games played in design-by-doing can be viewed both from the point of view of the users and of the designers."
Ehn 1988, pp. 117-118
Ehn notes that it is more accurate to talk about an interplay between the users and designers language-games than an understanding, because users and designers do not have to fully understand each others. But they need to establish a common language-game of design making sense to both groups. This common language-game may, however, also make different sense when interpreted by designers and when interpreted by users.
Ehn's use of the Wittgensteinian philosophy can be seen as a strong motivation for developing the cooperative prototyping approach to support communication and interaction between different professions, represented by designers and users, possessing different language-games. The conclusions with this respect is summarized as follows:
It is important to establish a design language-game which makes sense to both users and designers.
Figure 2: Typical process according to traditional Information System prototyping approaches. Few activities where users participate.
The work behind this thesis represents a deliberate attempt on bringing together ideas from research on prototyping and user involvement. An active user involvement is advocated and to stress this the proposed approach is denoted cooperative prototyping. Cooperative prototyping brings together qualifications of users and designers throughout system design processes. Ideally it will allow the designers and users in cooperation to transcend their current qualifications and to be creative in the process of developing a new computer system. As suggested in Figure 3 the degree of cooperation changes during a development process dependent on the kind of activities undertaken. For instance users are more active in analysis, design and evaluation activities and less active in implementation activities.
Figure 3: A typical prototyping process as suggested by the cooperative prototyping approach. More activities where users participate, and more interplay between activities.
Figures 2 and 3 serve as a simplified illustration of the difference between traditional prototyping approaches and the cooperative approach with respect to user's and designers' contributions. A more detailed discussion of the activities of cooperative prototyping is given in the following section.
1. Cooperative prototyping applies to design of computer systems aimed at providing interactive support and being integrated in users' work practice.
Compared to the activities in a development process as outlined by Andersen et al. (1990) cooperative prototyping has mainly been applied to early analysis and design activities. The approach supported mutual learning in these activities leading to embodiment of ideas as prototypes and notes constituting a basis for implementation. The techniques have not yet been developed in a field study of a full-scale project, but this lack of experience does not mean that the techniques cannot be applied to later activities. In particular, the techniques for establishing work-like evaluations (see below) can be applied throughout an evolutionary development process, as also suggested in Figure 3 above.
With respect to project management activities the experiences are on the one hand closely related to establishment of project working groups consisting of both users and designers, and on the other hand to the preparation of prototyping sessions. These issues are discussed in Section 5.5 on principles for organization.
The perspective on good computer systems and design processes was illustrated by the statements in Section 4. In addition, the perspective can be characterized as close to the perspectives behind the Collective Resource Approach and the book Design At Work (Greenbaum & Kyng, 1991) described in Section 3.2. and 3.3. This implies that also the following assumptions are fundamental.
The perspective on computer systems can be expressed as follows: Computer systems should enhance workplace skills rather than degrade and rationalize them. Computer systems are tools to be controlled by competent users. Introducing computer systems is closely coupled to organizational changes. Computer systems should improve quality of work and products as well as increase workplace productivity.
The perspective on design processes can be expressed as follows: Design processes take place in organizational contexts and are both political and social processes with potential conflicts. Computer systems should be designed with full participation from users and designers - both for democratic reasons and to ensure that relevant competencies are represented. Design processes should take the users' practice, i.e. their skills, current work tasks, and current problems, as the starting point for design. The techniques should enable also the users to be creative in the design process.
2. The users' work practice is the starting point for design activities and a sample of users should be fully involved in activities determining the system design.
With respect to the role of cooperative prototyping in organizations, neither a strict trade union nor a strict management perspective need be assumed to apply the approach. It is important to establish working groups with actual future users to participate in the design process independent of whether they are managers or shop floor workers. But it may require extra attention to get a cooperative design process going with a mixed group of users with different skills and positions in the organization. In specific cases potential conflicts may require a splitting into several working groups for selected activities. For instance, workers may feel uncomfortable about expressing their needs, proposals, or critique while their manager is in the room. Cooperative prototyping may uncover conflicts, but conflicts cannot usually be dealt with or resolved by experimental design. Dealing with conflicts requires actions at a different level. Such negotiation and decision making is not within the scope of cooperative prototyping techniques as outlined here.
3. Cooperative prototyping utilizes two main techniques: (1) using prototypes as "sketch pads" for joint exploration of design ideas and users' work practice, and (2) promoting evaluation sessions based on work-like use of prototypes.
A similar distinction between techniques (1) and (2) was introduced in Paper 3. In Paper 3 the emphasis of technique (1) was on exploration of design ideas. However, the experiences from the Municipality study Papers 4 and 5 show that sketchy prototypes may also trigger useful work practice focused exploration. In particular, Paper 5 illustrates how a caseworker triggered by a prototype starts telling relevant anecdotes from her work. These observations explain the elaborated formulation of the techniques. Both these techniques (1) and (2) support a mutual learning process between users and designers, but the emphasis is different in the two techniques. Technique (1) has its emphasis on learning about the multitude of technical possibilities and their relation to the users' current work as expressed by tenet 4 below. Using a sketch pad as a metaphor for the prototype is meant to emphasize properties such as incompleteness, flexibility in change, being open for change, and being open for throwing away. Technique (2) has its emphasis on learning about the future use situations and work organizations. The users get "hands-on" experience with prototypes embodying design ideas in ways that allow setting up evaluations with simulated work-like situations, as discussed below (tenet 5). Early in a development process work-like evaluations are based on simulated or dummy functionality and temporary entered sample data. Both techniques familiarize users with the technology being developed, and hence ease learning of the final system.
4. Exploration of design ideas and work practice through joint manipulation of prototypes stimulates users' active involvement and supports mutual learning about technology and the users' work practice.
The Dental Clinic study (Paper 2) and the Municipality study (Papers 4 and 5) both show examples of users' active engagement in discussing future technical possibilities mediated by prototypes as common sketch pads. Papers 4 and 5 also show examples of sketchy prototypes triggering shifts of focus towards deeper exploration of the users' current work practice.
The design ideas explored through prototyping originated from previous analysis activities focussing on the users' work practice. Both the activities with emphasis on the designers learning about the application domain, as described under tenet 18, and other analysis activities contributed with ideas. For instance, the Future Workshop technique (Jungk & Müllert, 1987) was used in the Municipality study as a brainstorming technique to uncover problems and formulate visions. From these activities sketchy prototypes were developed to embody central ideas for further exploration. But the prototyping activities per se also surface new ideas.
To utilize technique (1) it should be demonstrated to the users early in the process that both prototypes and the overall design are open for modification. When experiencing the opportunities for influencing the design users express their ideas and wishes freely. In both field studies the users contributed actively to exploration of alternatives and several of those were suggested by the users - a fact which also illustrates that users participate creatively in the process.
In the Dental Clinic study alternative ways of utilizing direct representation of teeth on the screen were considered. The dental assistants pointed to both problems and solutions triggered by the confrontation with a preliminary prototype illustrating the basic idea. Some of the suggestions were tried out immediately. For instance, a new search facility was provided by direct manipulation copying a button on the screen followed by a quick modification of the underlying code to search on a different criteria (Paper 2).
In the Municipality study the caseworkers were confronted with a preliminary prototype to support navigation through material such as maps and site information for urban planning. Triggered by this confrontation they requested automatic support for cross-referencing between sections of their long-term planning documents consisting of several hundred printed pages. This suggestion led the designers to introduce the basic ideas of hypertext to the caseworkers. After this introduction it was decided to design hypertext support for the structure of their large planning documents. This extension of the prototype was made in between two sessions with a distance of two weeks (Paper 4).
Examples of prototypes triggering exploration of current work practice are given in Paper 4 (Section 5.3), and in Paper 5 (Section 4.3) where a radical example of this is given: the caseworker, when confronted with a facility from the current prototype, suggested several alternative solutions on how to archive letters regarding streetnaming cases. Triggered by the prototype she maintained a discussion, for a couple of minutes mostly with herself , about what was really the problems to be addressed with computer support with respect to the streetnaming frame task. This discussion was concluded by her stating that she wanted to change her daily work procedures even before any computer system was introduced. The prototype somehow provoked the caseworker to reflect on her work practice and even change it. In fact, Mogensen (1990) proposes to use the term `provotyping' for the use of prototypes to provoke reflection and critique of aspects of current work practice.
When the focus is on exploration of ideas and work practice through manipulation of prototypes, the designers' learning about users' work is usually limited to aspects being articulated and illustrated without actually performing work tasks (Paper 4). In contrast to this the next tenet expresses how designers can learn about unarticulated aspects of the users' work.
5. Analysis of breakdowns in work-like use situations is an important resource for learning about unarticulated aspects of users' work and how these may affect the design of a future computer system.
It was argued above in the motivation for cooperative prototyping that it is important to have unreflected and unarticulated aspects of users' work contribute to the design of a new computer system. This cannot be facilitated by means of descriptions detached from the work activity. Consideration of breakdowns in users' work-like use of prototypes of the future system is needed to draw on such aspects (cf. Section 4.2).
Technique (2) of cooperative prototyping is to establish such work-like use situations with prototypes and actively monitor breakdowns occurring in the users work with the prototypes. The users and designers select frame-tasks from the work domain to be carried out with support from the prototype (see tenet 10 below). In the work-like use situations the designers often keep in the background only intervening when breakdowns occur (Paper 2). Breakdowns may be due to different causes which need to be analyzed in order to assess actions to take in the continuing design process. Typical causes of breakdowns in use of prototypes are: lack of user training, programming bugs, missing sample data, frame tasks only partially supported, inappropriate interaction style, insufficient functionality, conflicts between users' work practice and the prototype, (Paper 4 discusses examples in further detail). Breakdowns may lead to immediate actions such as: the designers teach the users about the facilities of the prototype if the breakdown is due to lack of training; the users start teaching the designers about their work, because there is a misconception on how the prototype should function; the prototype is modified immediately by users and designers in cooperation in order to reestablish the work-like evaluation (tenet 6 below); or the sessions has to stop or certain parts of the frame-task must be skipped, because of a serious problem with the prototype, which cannot be fixed immediately.
Not all breakdowns are equally productive with respect to the design of the future system. For instance, "everyday" programming bugs do usually not uncover important knowledge that needs to be encountered for the design. On the other hand, breakdowns caused by the users attempting to perform relevant work actions not being well supported often give important hints on how to improve the design. Examples of such breakdowns are seen in both field studies. For instance, Paper 4 describes an example of a caseworker who was using the prototype to search for important informations for handling a certain local area plan. He found the information on a map and examines data on a connected site, moved to another place in the computer system and a few minutes later he wanted to examine the same information again as if it was lying on his physical desk-top. This caused a breakdown where he had to enter the same procedure to find the information as first time he accessed it. This breakdown was discussed and it was assessed to be a general problem, a potential recurrent breakdown, in the future system. To get around this type of breakdowns we decided to design of a facility to support easy re-finding of recent material. In the session we took the decision to build it, but it was implemented by the designers between-sessions.
6. Breakdowns in work-like use situations may be handled by in-session modifications of prototypes to reestablish work-like use evaluation.
The example described in conjunction with tenet 5 above illustrates that a breakdown is analyzed and lead to the design of new facility which is build into the prototype after the session. But in an early stage in a project cooperative prototyping may be undertaken as a combination of the techniques (1) and (2) by allowing breakdowns in evaluation sessions with work-like use to lead to immediate, in-session, modifications of the prototype followed by reestablishing the work-like use evaluation. Paper 2 describes examples of such situations.
One example is a problem with the layout of the tooth representation. The designer had turned both mouth parts the same way with the front teeth at the top of the screen to be able to place the navigation buttons `previous page' and `next page' in the middle of the screen at the bottom. However, when a dental assistant was working with the prototype she could not match this representation with the mapping of the teeth she was using in her work examining the mouth of patient together with the dentist. This breakdown led to a modification of the prototype, where the designer quickly flipped the picture of the lower mouth part and moved the navigation buttons to the right side of the screen. After that the dental assistant continued using the prototype with the improved mouth representation.
Another example from the same study was a situation where a dental assistant needed to search on the social security number (the CPR number) for a certain patient. This caused a breakdown, since the only search facility currently in the prototype was using the patient name as the key. This breakdown also led to an immediate modification of the prototype. The name search facility had been implemented by a script behind a button.[19] A CPR search facility was quickly provided by the designer who copied the name search button and modified the script to search on the CPR field of the database instead. Again the work-like use of the prototype was reestablished and the dental assistant immediately used the new search facility.
However, not all modifications can be carried out in-session followed by continuation of a work-like evaluation. The designers need to carefully consider whether the needed modification of the prototype is feasible as an in-session modification or should be postponed for between-session implementation. The designers possess computer specific skills, thus they should have the competence to assess the feasibility of in-session modifications.
7. Establishing a common language game of prototyping requires that designers primarily draw on terminology, tools, and materials familiar to users.
As described in Section 4.2, Ehn (1988) argues for the need to establish a common language game of design based on design-by-doing techniques. Cooperative prototyping is a kind of design-by-doing, where users are involved in building, modifying, and evaluating prototypes. This requires initial learning on the part of the users: they need to understand the prototype's relation to their work and they need to understand (to some extent) what is going on and what is possible with respect to modifying the prototype. This learning would be a serious obstacle to get going if it required the users to understand the terminology of computers, system descriptions, and prototypes. Thus the challenge for designers is to stick to terms and types of modifications of prototypes being somehow familiar to the users. To help the users to understand prototypes and tools they need to be designed around concepts and objects from the users' work. A visual and tangible representation in prototypes of concepts and objects (if possible) helps the users to quickly learn how to interact with them. Tenets 14 and 15 below discuss in further detail how prototyping tools can support establishing a common language game of prototyping through the use of direct manipulation facilities.
System development of course has to deal with complexity and technical aspects requiring use of techniques and tools, such as formal descriptions techniques and conventional programming languages, being familiar only to the designers. The challenge for the designers is to apply these techniques and tools in ways that do not limit the users' ability to influence design decisions. For instance, designers may want to use formal descriptions of a system as internal documentation, but they should not use these descriptions as a means of communicating with users, since they most likely cannot understand such descriptions.
8. Cooperative prototyping progresses from sessions emphasizing exploration of design ideas and work practice, to sessions with point-wise work-like evaluation, and later to sessions with fluent work-like evaluation.
As suggested in Figure 3 cooperative prototyping processes iteratively progress towards system implementation and installation. The process usually starts with open situations where participants possess only vague and abstract ideas of how technology can improve work conditions. Cooperative prototyping aims at exploring such ideas and their relations to users' work practice through experimentation with one or more alternative preliminary prototypes. The first prototypes are typically just sketches to be means for discussions. Then they will be provided with enough simulated functionality to make them suitable for point-wise work-like evaluations (Paper 4). A point-wise work-like evaluation is an evaluation where fragments of frame tasks (tenet 10) can be tried out with the prototype. Such evaluations take place in interplay with discussions of ideas and work practice otherwise mediated by the prototype. Prototypes ideally grow more stable and become suitable for users' fluent work-like evaluations with only few serious breakdowns and interventions by the designers. In some cases early sketchy prototypes requiring little development effort may be driven relatively fluent by users as was the case with the prototypes described in Paper 2.
Bondgård, Degn & Jensen (1990) describe a field study where they applied a variant of cooperative prototyping to the design of a reservation system for one of the largest hairdresser's shops in Denmark. Late in their design process they aimed at establishing evaluation sessions where the prototype should function in real use for four days with a minimum of designer intervention. They report how setting up this session raised a lot of new requirements on the robustness and data content of the prototype. But they also found the effort paying off in an improved design by the fact that both the hairdressers and the designers learned a lot about the design. For instance, the sessions uncovered the hairdressers' needs to perform many tasks at the same time and to share data in the system. These aspects were not uncovered when the designers studied how the hairdressers used a plain paper calendar to maintain their reservations.
Even if early prototypes were evaluated during analysis and design activities, work-like evaluations should continue regularly throughout the system development process. An example of such an incremental development process with ongoing user evaluation of prototypes is given by Pape & Thoresen (1987) and Thoresen (1990). These papers describe how a system for municipal town planning starts being developed as a pilot system in one municipality through prototyping with users and is later installed in other municipalities through a shorter prototyping process. The first municipality is involved in developing three prototype versions of the system, the next municipality is involved in the development of the two last versions, and the third municipality is only involved in development of the last version. This way stable prototypes immediately ready for work-like evaluations were provided to the two later municipalities. The goal of this was to evaluate the differences in work processes in the municipalities and assess needs for tailoring for each of the municipalities.
9. Initial prototype(s) should be viewed as learn-and-throw-away. Viewing a prototype as part of the future system often hinders experimentation.
Early in development processes the emphasis is on analysis and exploration of design ideas. At this stage it is an advantage not to be too limited by current technology (Ehn & Kyng, 1987). Hence, prototypes should be viewed as sketch pads for embodiment of ideas. One way to avoid limitations is not to be concerned with, for instance, an optimal modularization of the code or thorough documentation of details in the code. Taking the message "..plan to throw one away; you will anyhow." from Brooks (1975, p 116) seriously, means that initial prototypes should only be learning vehicles. Being aware of the fact that initial prototypes will be thrown away and thus do not limit the final system, encourage users and designers to experiment more freely with prototypes.
Functionality being essential to connect the prototype to work tasks should be simulated as far as possible. This in order to make initial prototypes cheap to modify or throwaway compared to full-fledged implementations as the vertical prototypes used in the projects studied for Paper 1. Paper 3 discusses different approaches and tools to make cheap simulation of functionality for prototyping. At a stage where the working group decides that it is mature to move on to use the target environment, the ideas captured by the last version of the learn-and-throw-away should be carefully structured and transferred into a first prototype in the target development environment. The fact that the initial prototypes should be viewed as learn-and-throw-away does not imply that parts of them cannot be reused, if the initial prototyping environment and the target development environment are compatible. The group may even choose to start building initial prototypes in the target development environment and then decide to spend effort on restructuring the prototype when the ideas become sufficiently stable. But to achieve a situation were in particular the designers are open-minded towards trying out new and alternative ideas that might not at a first glance look as a likely candidate for implementation, it is important that choosing the target environment does not imply a reluctance against experimentation similar to the cases discussed in Paper 1.
10. Frame tasks for work-like evaluations should be rooted in the users' work practice and be selected by users and designers in cooperation.
Evaluation techniques, defining tasks for evaluations, and setting up controlled experiments receive much attention in Human-Computer Interaction literature as mentioned in Sections 2.5 and 3.2. But the selection of tasks to be performed in such sessions is usually made by the designers on their own, based either on the current features of the prototype or on a specific hypothesis concerning an interaction style that the designers want to evaluate. Many evaluations carried out this way are focussing on novices trying out a system they have never seen before. The users' specific work and their skills are not in focus in such evaluations. Thus these approaches for selecting tasks for evaluation are not suitable for cooperative prototyping.
In the studies described in Papers 4 and 5, a central element of setting up the prototyping sessions was selection of central work tasks to focus on in the evaluation sessions. These tasks frame the evaluations in an open-ended fashion, hence we called them frame tasks. The frame tasks in these studies were carefully chosen by the users and designers in cooperation before the prototyping activities started. The tasks were chosen among the tasks that the caseworkers wanted support for and they were formulated in the caseworkers terminology. Examples of frame tasks from the studies described in Papers 4 and 5 include: An engineer caseworker use the prototype to prepare an inspection visit at a certain company that potentially produced illegal pollution. A draftswoman use the prototype to perform the work involved in changing a streetname in the city. This strategy for choosing frame tasks for the work-like evaluations help focus on the relation between computer technology, the specific work of the users, and the skills possessed by the users.
The study described in Paper 5 shows an example of how analyzing a frame task this way stimulated the mutual learning. The choice of the streetnaming frame task led to a session being much different from what the designers expected: in the initial analysis activities this task had been described as one requiring computer support for storing and retrieving letters and other documents. But when starting exploration of the frame task with the prototype, it turned out that the task required a much different support than originally expected by the designers. When the caseworker was confronted with the prototype, she shifted focus and it turned out that the primary facility needed, was support for annotating streetmaps with information on changes of streetnames and numbers.
11. Illustrative sample data should be prepared and maintained for work-like evaluations.
It is important to pay attention to sample data early in a prototyping process. Examples from the Municipality study (Papers 4 and 5) showed that extensive sample data was equally important as the prototype in order to establish fluent work-like evaluation. In the first sessions where the caseworkers tried to start on their frame tasks, we recognized several breakdowns due to the lack of relevant sample data. The caseworkers' work was heavily dependent on combining information from a lot of different sources such as laws, maps and long-term plans. For instance, the long-term municipal plans in this case was a couple of hundred pages of text and figures, and it did not exist in computer-readable form. Hence it was hard to enter these plans as sample data to be comprehensive enough for non-trivial cases. The only realistic possibility for quickly providing sample data here was to scan parts of the material into the prototype. Such scanning would supply sample data for situations where the caseworkers were navigating in existing information. But it would not be sufficient in situations where the caseworkers needed to modify the material.
In the Dental Clinic study (Paper 2) the situation with respect to sample data was different. The dental assistants did not need much sample data to get started on a frame task, such as modifying a patient record according to a treatment, they only needed sample data for this particular patient. When modifying the patient record they also entered sample data. In fact, data entry was a considerable part of their work hence the prototype evaluation produced sample data in itself.
Finally, being able to reestablish work-like evaluations (tenet 6) requires sample data to be preserved across in-session modifications to prototypes. This determines a requirement for prototyping tools as discussed in relation to tenet 16. But it also creates an issue for the designers to be aware of when assessing the feasibility of making a modification in-session.
12. Cooperative prototyping sessions should be kept open-ended, users should be encouraged to take initiatives, and focus shifts should be viewed as openings for learning.
There is a tension between careful planning and keeping the sessions open-ended for the users' and designers' continous mutual learning. It is important to make the initial purpose of a session clear (tenet 20) and keep this purpose in mind in the session. But it is equally important not to impose too much control of the session. Earlier it was argued that breakdowns in work-like evaluations (which to a large degree are unpredictable) are useful resources for the learning process. Papers 4 and 5 also argue that an important feature of cooperative prototyping is the users' freedom to take initiatives, to contribute with knowledge, and to influence the design. Hence, other kinds of focus shifts triggered by users as well as designers can also stimulate the learning process. Examples of such focus shifts being potentially productive are: when users leave the prototype and bring material from their work into sessions to explain certain aspects of their work (Paper 4), and when users start telling anecdotes about their work triggered by a prototype (Paper 5). This kind of open-endedness is much different from the controlled experiment approaches traditionally recommended for evaluation in Human-Computer Interaction literature as discussed in Sections 2.5 and 3.2. In controlled experiments the users are not encouraged to take any initiatives beside of performing the specific tasks and answering questions from the designers.
Focus shifts as outlined here may also lead to activities where the prototype is not the optimal choice of medium. The designers who are the experienced facilitators should assess whether it is better to turn away from the prototype and use for instance a blackboard or a flip-over to mediate another kind of discussion, because it is important for the design. The working group might also realize that a particular focus shift is caused by an organizational problem rather than a problem which can be discussed closely coupled to a prototype, hence the group may take action to establish other kinds of meetings for instance using the Organizational Game technique (Ehn, Mölleryd, & Sjögren, 1990).
13. Video recordings are useful for documenting sessions. Analysis of recordings can give input for improving design proposals, and for improving prototyping techniques as well.
Papers 4 and 5 illustrate how video recordings were used for documenting sessions from the Municipality study to support retrospective analysis of aspects of the sessions that were not recognized by the participants during the session. This use of video was inspired by (Suchman & Trigg, 1991). The recordings were mainly used for analysis of situations from the sessions aimed at improving the understanding of techniques for cooperative prototyping. But the analysis also uncovered aspects of the use of the prototype, which could lead to improved design if the analysis had been carried out in conjunction with the on-going design process and not as a post hoc analysis. When a breakdown happens it is useful to be able to rediscover it on video and to be able to examine the operations carried out just before. The discussion of situations and focus shifts in Paper 4 is based on a rough analysis of the videos from four of the five sessions using an Activity Theory based framework for interpreting the situations.
Paper 5 describes a detailed interaction-based analysis of a 36 minutes video recording from the fifth session. Among the results of these analysis was the recognition of users telling anecdotes as being an important feature of cooperative prototyping. The latter kind of video-analysis is quite time-consuming, and it may not be feasible to undertake such detailed analysis when applying video for documenting prototyping sessions in conjunction with on-going design activities. But having tried to undertake a detailed video-analysis make the persons who participate in the analysis better at understanding videos from sessions without applying the detailed analysis techniques (Paper 5). Thus it can be recommended to spend effort on analyzing a piece of video recording that document situations which are hard to understand at a first glance. Difficulty in understanding the user's reaction to the prototyping situation was the motivation for the analyses described in Paper 5 from a research point of view. But if a breakdown in the use of a prototype is hard to understand similar analyses may give better ideas for redesign of the prototype than recalling the memory about the breakdown from the participants, hence the video analysis can lead to better design.
14. Cooperative prototyping requires powerful and interactive computer tools supporting direct manipulation.
The ideal of computer support for cooperative prototyping is to enable both users and designers to be creative in the design. This implies that users should be able to participate in the modification of prototypes equally as easy as to participate in the modification of plywood and paper mock-ups. Hence the tools should ideally be understandable, tangible, and transparent to both users and designers, as is the case with the use of glue, saws and scissors to make mock-ups (Bødker et al., 1987). The term tangible models was introduced in Paper 3 to describe computer-based and other means for design that can be manipulated similar direct and simple with accompanying tools. Various combinations of manual, computer-based, and even interactive video-based tools may be used as tangible models for cooperative prototyping. Users and designers can should be supported in sketching and exploring ideas with the tools. Evaluation in work-like settings, however, requires functional behaviour, such as navigation between facilities, support for entering and requesting data from the prototype. Hence, a computer-based simulation of system behaviour is always needed to allow hands-on testing of behavioural aspects. It may be hard to make the manipulation of such functionality understandable to users; for instance, a programming language is not easily understood by non-programmer users.
However, tools providing support for direct manipulation have emerged from research on user interfaces. Direct manipulation interfaces are interactive point-and-select interfaces for manipulation of objects in a computer system[20]. For cooperative prototyping, direct manipulation tools have a number of intriguing advantages compared to conventional programming tools. Object manipulation becomes visible, because of immediate feedback highlighting the objects being manipulated. Similarly, results of an operation are immediately made visible when possible. Operations on objects are often chosen from menus and dialogues. In such interaction, spelled out commands can be used, because it is not necessary to type in the commands to perform the operation as is the case with conventional programming and command oriented interaction. The interaction is characterized by selecting objects and tools, and by using selected tools on selected objects. Such interaction has a family resemblance to other aspects of human work and makes the design process easier to enter for users. Design tools providing such direct manipulation interaction can reduce the asymmetry between users' and designers' understanding of the design process. However, it it is still the designers who possess the skills needed to understand the design tool in detail. Thus, the designers possess the power when discussing limitations for technical possibilities and when assessing what kinds of modifications that can be made in-session.
Paper 3 surveys sources for developing computer support for cooperative prototyping. Five categories of emerging computer-based tools that may support cooperative prototyping are discussed. The tools are grouped according to the techniques supported, as follows: Animation, Simulation, Wizard of Oz, Design by Example, and Visual Programming.
Alternatives and supplements to direct manipulation based prototyping tools may also be considered. Hauerslev & Jakobsen (1988) describe an experiment where they develop and use a network based prototyping tool using conventional programming. The set-up for this experiment is a user and a designer evaluating a prototype on one workstation and a programmer making requested changes on a connected workstation in the same room. The idea is that the user and the designer communicate requests to the programmer who modifies the prototype. When the modification is complete, the user should re-enter the work-like use situation with the new version of the prototype installed by the programmer from the remote workstation. Hauerslev & Jakobsen (1988) discuss problems and prospects of this kind of prototyping support. Among the problems recognized are: that it was hard hard to keep the user engaged when requests are communicated between the designer and the programmer, and it was hard for the programmer to assess his changes to the prototype, because he did not share the user's view of the prototype.
For the field studies described in Papers 2, 4, and 5, HyperCard for the Apple Macintosh[21] was the main tool used, because of its support for direct manipulation of user interface objects, its support for linking program scripts closely to user interface objects and the rapid test possibilities provided with the interpreted programming language HyperTalk. HyperTalk is, however, not a "good" programming language for structuring and maintaining programs even for the short life time of a learn-and-throw-away prototype. To develop an even more powerful prototyping tool it is good idea to break the limitations of HyperCard by building similar tools embedded in a general purpose object-oriented programming language. Such a development effort is briefly described in Section 7 about future work.
15. Prototyping tools should support representing objects and concepts from the users' work in familiar ways. Object-oriented environments seem promising in this respect.
To establish a common language game of prototyping it is required to be able to facilitate users' understanding of prototypes. This requires that prototypes are designed around concepts and objects which to a large extent are familiar from the users' work. A visual and "tangible" representation of concepts and objects (if possible) helps the users to easily interact with them. For instance, the use of HyperCard to build a patient record for dental clinics (Paper 2) was superior to choosing a conventional database, because HyperCard immediately supported mapping a patient record, which consists of a collection of sheets in a folder, to cards and stacks that were visible and tangible in the sense that they could be browsed and manipulated similar to the real record. In addition, using pictures of the mouth parts for the teeth sheets made the interaction with the record natural for the dental assistants. Thus the dental assistants could continue using and talking about the patient record in their usual terms with the prototype as medium. This required learning about the computer, but not (re)learning their familiar tools and materials in a new terminology. Mapping their well-known objects and concepts into the prototype made a good starting point to experiment on how to take advantage of having the patient records on a computer.
In-session modifications of prototypes need to be supported in ways that enable the users to follow them without loosing their patience. Keeping a repertoire of application-oriented building blocks which are naturally related to the users' work supports their understanding of prototype extensions and modifications.
For the Municipality studies, we developed a several application-oriented building blocks which could be used for quick extensions of the prototype in cooperation with the caseworkers. An example of such a building block was an object we called an "index-field" which was used to represent new text documents to be represented in the hypertext-like system. The "index-field" was actually a complex aggregation of fields and buttons which was initiated with the title of the document at the top and a table of contents where each entry was linked to the corresponding section of the document. This building block was used in several situations to extend the prototype. The caseworkers, however, rarely used the term "index-field" to denote the class of objects, even though they denoted the table of contents for a document for an "index." They rather refered to the "index-field" by pointing to an instance of it or by talking about representing a new document "X" the same way as the Local Area Plan. Thus both the visibility of instances of object on the screen and the fact that instances of it had familiar names made the caseworkers able to refer to the building blocks on their own. Other examples of building blocks for this field study were: a template to specify specific search facilities by filling in familiar names of attributes and objects; and a cluster of buttons supporting navigation in all compass directions on a card representing an area map or a street map for urban planning.
Object-oriented environments which support specialization and aggregation of existing objects seem promising with respect to support such building block based modification of prototypes. HyperCard which was used in the field studies is object-oriented to some extent, but it does not provide good support for generalization, specialization and modularization hence larger HyperCard prototypes become difficult to understand and maintain. This is problematic also for throw-away prototypes, hence having flexibility similar to HyperCard built on top of a powerful object-oriented programming environment seem as a good basis for a tool for cooperative prototyping. Such a tool should integrate direct manipulation support as mentioned above, application-oriented building blocks, and facilities for preserving sample data as argued below, to enable both in-session modifications of prototypes and larger between-session modifications.
16. Prototyping tools should support preserving sample data across in-session modifications of prototypes.
As noted above (tenet 11) it is important to preserve sample data across in-session modifications. Paper 2 describes an in-session modification shown to be problematic with respect to preservation of example data. The aim of the modification was to split a data entry/browse screen ("card") into two screens. The structure of the prototype could easily be modified this way by direct manipulation in-session, but the sample data stored in the fields to be moved to the secondary screen could not be preserved with the current tool, HyperCard, which collapses database and user interface in the notion of "cards". Hence it was hard to reestablish a work-like situation in the same session after this modification.
Making data persistent with respect to modifications of prototypes is a hard problem to address. In the widespread 4th generation systems relational database systems are being used, because they are known to be flexible to modify. In fact, it is usually not difficult to change the definition of a relational table in a relational database systems. But many DBMS's requires the table being modified to be empty,[22] implying that a modification requires the following steps: sample data in the old table being copied to a temporary table, the old table being redefined (and deleted), and the sample data from the temporary table being converted to the new table. Some database systems does this kind of conversion semi-automatic,[23] here a modification of a database table must be followed by the invocation of a `reorganize data' procedure. These kinds of conversions are usually not suitable for in-session modifications, since they are not interesting for the users and are usually also time-consuming. The semi-automatic conversion may work for smaller sets of sample data.
HyperCard,[24] which was used for both field studies supports persistency of data well when work is restricted to one stack of cards, which are the basic building blocks for both database and user interface objects in HyperCard. In HyperCard data is stored in fields on cards, and no explicit save operations are needed to store data, it is automatically stored on the disk. Similarly new fields can be added to cards and stacks without making explicit save operations. But as mentioned in the example above moving background fields across stacks implies loss of data.
Recent object-oriented environments provide persistent objects, i.e. objects that are automatically kept in a database and thus survives between program executions. Prograph[25] is an example of such an environment, where objects under certain conditions can be declared as persistents. Conditions are for instance, that all non-simple objects need to be stored in list objects, and all persistents need to be global.
An advantage of providing persistent objects integrated within object-oriented programming language/environments is that the database and the programming language become closely integrated. Hence more powerful and flexible support for manipulation of data and linking of data elements to user interface representations may be developed. But it is premature to assess such facilities in connection to cooperative prototyping since no practical experiences exist yet.
17. Documentation of breakdowns, new ideas, work practice related issues, and decisions may require supplementary tools, such as video equipment, integrated hypertext, and flip-overs.
It was argued (tenet 13) that video recordings can be useful for documenting what happens in the intensive prototyping sessions. This however, requires access to video equipment. A single video camera on a tripod placed such that it covers the I/O devices and the people working with it is sufficient in many cases. However, if both gestural activity, interaction among people, and the precise look of the screen(s) is considered important, more advanced set-ups are needed (Tang & Leifer, 1988). But there might be serious trade-offs in extensive use of video equipment in prototyping sessions: some people feel awkward being recorded and others refuse to appear on video at all. These issues need to be considered in advance, because the use of video can potentially spoil the whole session. In most cases it is advantageous to place the equipment as discretely as possible, but all participants must be informed and agree on being recorded.
Both field studies showed that it was sometimes hard for the participants to remember the rationale behind certain prototype modifications, in particular in-session modifications, when examining the prototype at a later stage. To document decisions closely related to the prototype, an annotation facility integrated with the prototyping tool would be useful. Such needs were also recognized by Hauerslev & Jakobsen (1988) in similar studies. We used simple hypertext facilities[26] to document our application-oriented building blocks in the Municipality study. But since our prototyping tool, HyperCard, at that time did not provide multiple windows, we did not plan to use annotation facilities to document the in-sessions modifications. Making annotations would require us to switch between annotation facilities and the actual prototype. Having hypertext facilities integrated with the prototyping tool could provide an opportunity for making annotations to prototypes. But experiences in using such facilities are needed to assess how the interplay between annotation activities and other prototyping activities will turn out.
In Paper 4, focus shifts turning the action from design to analysis were discussed. An instance of that was when prototyping activities changed towards the caseworkers teaching about their work practice and the designers posing interview-like inquiries. This focus shift took place without moving to more proper tools and techniques to accomplish and document the activities. In order to be better prepared the designers could set up the sessions in a room equipped with pens and a flip-over (or just white cardboard pieces). From our experience with future workshops, it seems to be a good idea to have simple means like white cardboard to write on, because it can easily be carried away as documentation to be used after sessions. To designers a computer-based tools such as the hypertext facilities mentioned above might be appropriate. But in our experience users generally find it more comfortable to use simpler means such as paper and pens when they need to express things in drawings or written text.
18. Designers need to be acquainted with the users' work domain prior to cooperative prototyping activities.
Prototyping activities are not suited to support the designers' initial learning about the users' practice. The designers need to be acquainted with the work domain in advance to building the first prototypes. Ehn & Kyng (1987) claim that a long period (a couple of years) of designers getting acquainted with the work domain of the users' is necessary before turning into design activities. I agree that this initial learning is important, but I claim that less effort is sufficient. In the Dental Clinic study (Paper 2) we, the designers, had been involved in assessing current computer technology for dental clinics and in developing educational material on new technology for dental assistants.[27] The dominating technique used for these activities were informal interviewing, trying out existing systems, and reading relevant material. Although these activities were spread over a longer period of time they represented an initial learning effort on a couple of months. In the Municipality study (Papers 4 and 5) I, one of the designers, had prior to the project start been involved in advising students who were doing an analysis project in the municipality office as part of their master's programme. When the actual project started we, the designers, visited the Office for several days and made informal interviews with all the caseworkers from the department that we were working with. In addition, we studied forms, maps, municipality plans, and other sorts of material used in the department. We also applied the future workshop technique (Jungk & Müllert, 1987; Kensing & Madsen, 1991) to help the caseworkers express critique and visions on the development of their workplace. Again these initial activities boiled down to a few months effort before starting the prototyping activities.
In both cases the initial learning activities of the designers were sufficient to start on cooperative prototyping, and gain further knowledge through the mutual learning taking place in these activities. It does, however, not imply that these initial studies were sufficient as analysis to accomplish the projects, but after the initial learning there was a close coupling between analysis and design activities.
More formal description-oriented analysis/design techniques such as Structured Analysis (SA) (Yourdon, 1989) or Object Oriented Analysis (OOA) (Coad & Yourdon, 1990) may precede or take place in parallel to prototyping activities. But a well-recognized danger of such techniques is that they draw away the focus from learning. Rather the focus moves towards getting a formalism right on the paper and signed-off by the users or their managers to be able to hand them over to a group of programmers to implement the description (Lantz, 1986). The goal of the initial analysis for prototyping is, however, that the designers who are going to continue with prototyping learn enough about the users' domain be able to cooperate with them on designing computer support.
19. Cooperative prototyping takes place primarily in joint sessions. But users and designers often need to prepare both separately and jointly for sessions.
Users and designers take different responsibilities during a cooperative prototyping process and hence need to prepare separately. The designers are still the ones who mainly have the technical competences and not all technical issues can be made familiar to users who have different backgrounds. Thus the designers need to accomplish activities between sessions such as: preparing for larger prototype modifications, and making parts of the prototype into application-oriented building blocks that can be reused for rapid in-session modifications later. In the Municipality studies, a large effort, compared to the actual sessions, was put into such activities and an additional programmer was involved in making changes enabling the designers to become ready for the next session as soon as possible. The designers might also discover serious holes in their understanding of the users' work and thus need to undertake more analysis and/or examine video recordings of sessions.
The users should prepare for the closely work related issues. The users typically undertake activities between the sessions, such as: selecting and preparing for frame tasks to be accomplished with the prototype, suggesting sample data that should be entered in the prototype prior to sessions, and selecting materials from their work that should be used during work-like evaluations of the prototype.
Finally users and designers also need to arrange joint meetings not being prototyping sessions. For instance, planning of a series of sessions and analyzing what happened in a prototyping session documented on video might be undertaken at joint meetings. Usually it is the designers as facilitators of the process who take the initiative to set up such meetings.
20. Cooperative prototyping sessions need to be carefully prepared, in particular it is important to clarify the purpose of sessions.
One of the problems identified in the empirical studies described in Paper 1 was that the developers in most of the projects studied did not sufficiently clarify the purposes of developing and using prototypes. Evaluations were done arbitrarily and the prototypes hence did not improve the developers interaction with the users enough to avoid serious misconceptions of the users' needs (see Section 4.1). Paper 1 propose a set of issues and questions to consider when preparing prototyping sessions. This and the following five statements represent an elaboration of these issues and questions, based on experiences from the later field studies, mainly the Municipality study. A reminder about the importance of keeping cooperative prototyping sessions open-ended (tenet 12) is also needed here. Attention should be payed to the tension between careful planning and keeping sessions open-ended. The tenets about organizing sessions should help in establishing a process in progress and help avoid sessions from being arbitrarily focused. The tenets are not meant as arguments for a tight control of the sessions.
The importance of careful preparation of prototyping sessions (experiments) was also recognized by Reisin & Wegge (1989). They as well emphasize the importance of clarifying the purpose of sessions and gear them to given project situations (see also Section 2.6).
Paper 1 proposes a distinction concerning the purpose of a session between creating visions about desirable features and evaluating the adequacy of a proposed solution. The priority of these contrasting purposes need to be clarified, when organizing a session or a series of sessions. This distinction on purpose determines to what level of detail prototypes should be developed and how the emphasis should be on the techniques (1) and (2).
To summarize, at least the following questions concerning purposes should be on the working group's agenda when organizing sessions: What is the purpose of the session? To what extent should the attention be on creating visions about the new system? To what extent should the attention be on evaluating the adequacy of an existing design proposal? Which frame tasks should be supported? To what extent should modification of prototypes be allowed during the session? To what extent should work-like evaluations be kept fluent? How should the designers intervene in the users work with the prototype?
21. The extent of prototypes and sample data to be provided for sessions should be planned according to the purpose. The needs for setting up alternative prototypes should be assessed.
Decisions on the purpose of sessions determines the needs for prototype development and entering of sample data prior to sessions. For instance, when the purpose is to establish fluent work-like evaluation, the prototype has to be fairly stable and sample data should fit the frame tasks being supported.
The choice of tools for building prototypes is also dependent on the purpose of the session. For instance, if the emphasis is decided to be on technique (1), exploration of design ideas and users' work practice, it is important to use flexible tools allowing rapid manipulation of prototypes. For early prototypes, functionality may be provided by means of simulations or animations (Paper 3). Later in a development process decisions have to be made about moving prototypes to the target development environment and continue work-like evaluations with more stable prototypes here.
It may be needed to develop alternative prototypes to explore different aspects of the users' work practice or to assess the feasibility of alternative design ideas. For instance, in the Municipality study, alternative prototypes accessing the same databases were set up to explore the differences between the urban planning caseworkers' and the environmental caseworkers' usage of these databases (Paper 4).
The following questions outline some of the options that may be considered with respect to the extent of prototype development: To what extent should the user interface be implemented? To what extent should functionality be simulated or implemented? To what extent should interaction with other systems be implemented or simulated? Which and how much sample data should be entered? Should the target development environment be used at this stage? Which alternatives should be provided?
22. Small working groups should be established with designers, and future users released from parts of their daily work responsibilities. Having more than one designer present in sessions is always desirable.
To get going with cooperative prototyping it is necessary to establish a group of users and designers that can cooperate efficiently in prototyping sessions (and meetings). This means that a group have to be reasonable small (3 - 7 people) even though the user population of the system to be designed is big. This may be achieved by splitting the project into a number of smaller cooperating projects.
The participating users should primarily come from the group of actual future users of the system and secondarily a few people who share responsibility without actually working with the system may participate. It is important that the users are experienced in their work, for instance they should have experience in handling also exceptions to the usual procedures. To establish good conditions for the users' participation they also need to be released from parts of their daily work responsibilities, for instance by providing substitutes. The studies discussed in Paper 1 showed that it was not enough just to make prototypes available on terminals in the user organization to establish a proper evaluation. While the users were not freed from their daily work they didn't have time and motivation to start up and spend time on evaluating a prototype not providing any functionality. Users only took such initiatives to work with a prototype a case where a vertically developed prototype actually could produce useful reports for them.
The participating designers should be experienced in analysis and design activities plus the use of the prototyping tools being applied. They should be motivated for working with users. Some of the designers should participate throughout the project since experiences from cooperative prototyping are hard to propagate to others.
At least two designers should participate in the joint sessions in order to keep track of what happens and to conduct the session (Paper 4). The designers may have different roles in the sessions. For instance, in the Municipality studies, the designers decided prior to each session, that one of them had the role as primary designer, i.e. he or she was sitting close to the caseworker(s) and the machine taking responsibility for the interaction with both caseworker(s) and prototype. The secondary designer stayed in the background observing, taking notes, maintaining the video recording, and intervening in the design activity when appropriate. This division of work was useful for instance, in situations where users kept talking when the primary designer was making a modification to the prototype. Most of this talking would be missed if a single designer had to concentrate on both modifying the prototype and listening at the same time.
It may be necessary to temporary involve other people mainly users to carry out more extensive evaluations. There may also be problems in mixing people arbitrarily in working groups. In Paper 5 an example is given on how individual sessions were established to avoid women draftspeople being dominated in sessions by engineers and architects. Finally, selection of participants may be controversial, for instance managers may want to appoint middle managers as users for the group, whereas designers need real (end-)users to accomplish cooperative prototyping. Hence establishing working groups may require negotiation between the user organization and the designers.
To summarize, at least the following questions concerning selection of participants should be considered when organizing sessions: Which skills should be represented? Which end-users should participate? How can it be ensured that end-users get resources to participate in the sessions? Which managers should participate? Which designers should participate? To what extent should they participate? Will potential conflicts require splitting into subgroups? How are specific roles assigned?
23. The participants need to be prepared for sessions. They all need to understand the purpose of sessions, and users may need specific training.
It is important that both designers and users understand the purpose of the session they are involved in. The designers need to have a sufficient understanding of the frame task to be able to understand the purpose of a session. On the other hand the users also need to understand the basic principles of prototyping. There are several reasons for this: to avoid creating unrealistic expectations among the users on the distance between the prototype and the future implementation (Paper 2); and to be able to engage the users in a play-like interaction with an incomplete prototype. Making users understand the purpose of session and engage them in play-like evaluations may require some effort. In both fields studies this understanding was achieved through joint discussions of the purpose, the frame task, and the status of the prototypes. But Hauerslev & Jakobsen (1988) suggest that participants are "trained" in playing with models in advance to prototyping sessions.
Papers 4 and 5 illustrate that users may employ different styles of interaction in prototyping session. Some might easily engage in a play-like evaluation of a prototype (like caseworker A described in Paper 4), others (like caseworker E described in Paper 5) may withdraw from playing with prototypes and take initiatives and influence design in different ways. Different user styles also generates different challenges for preparation of sessions.
Finally, the users may not have experience in advance with the equipment used for the prototyping. To avoid too many breakdowns due to lack of training with the equipment, such training may be provided in advance to sessions.
To summarize, at least the following questions concerning preparation of participants should be on the working group agenda when organizing sessions: Which information concerning the process is necessary for the users to participate in the sessions? How can the purpose and status of the sessions be made clear for the participants? Which information concerning the prototype(s) is needed for the users to participate in the sessions? How can possible training be provided?
24. The setting of sessions should be designed according to the purpose. Among the options to consider is whether sessions should take place in a laboratory setting or in the users' workplace.
Choosing the setting for the sessions is also dependent on the purpose of the session. If the session is aimed at exploring design ideas based on a preliminary prototype it might be reasonable to set up the session in a laboratory remote from the users workplace. Whereas the evaluation of a more stable prototype in work-like situations is better undertaken in the users workplace.
In the Dental clinic study (Paper 2) the prototyping activities took place at a kind of residential course where the Dental Assistants were remote from their workplace. An advantage of this was that they could concentrate on the activities going on. A disadvantage, however, was that they did not have immediate access to the materials from their daily work. In the Municipality study (Papers 4 and 5) the prototyping activities were taking place in a meeting room at the same floor in the building where the caseworkers had their offices. The caseworkers then attended the sessions in between their daily work activities. A big advantage of this was that the caseworkers could walk to their offices, pick up specific materials from their work, and return to the session whenever such additional material was needed. A disadvantage, however, was that a couple of phone messages were handed over to the caseworkers in the meeting room, and this caused inconvenient disturbance of the prototyping activities.
Also the number of participants at a time in the session is important to consider when setting up a session. Paper 4 compares the pros and cons of the choices in the Dental clinic study and the Municipality study. To summarize, at least the following questions concerning setting up sessions should be on the agenda for the working group when organizing sessions: How can a work-like situation be established? Which sample data is provided? Which frame tasks is used for the evaluation? Where shall the session take place? How many users should be active with the prototype at the time? Which additional material should be available? To what extent should video equipment be used?
25. Stop and success criteria should be planned in advance to sessions. Such criteria should be formulated in terms of the frame tasks for the sessions.
Prototyping processes are often criticized for being hard to manage (Alavi, 1984). This may also be a potential critique of cooperative prototyping processes, since they are exploratory, iterative and creative in nature, implying that both product and process to some degree are unpredictable. But to avoid starting an infinite loop when initiating a cooperative prototyping process, evaluation criteria should be discussed and planned in advance (Paper 1) to be able to assess the progress of the process. According to Andersen et al. (1990) such criteria should be formulated in terms of contents of activities rather than in terms of time and phases. Due to the open-endedness of cooperative prototyping (tenet 12) it seem to be a good idea to discuss both stop and success criteria. A stop criterion is an outline of conditions under which a series of sessions have to be stopped and the purposes need to be reassessed independent of whether it has been fulfilled or not. A success criterion is an outline of a satisfactory outcome of a series of sessions or a single session that will allow the working group to move to new activities. Hence the open-endedness of cooperative prototyping may lead to continuation of a series of sessions after the success criteria has been met, because new relevant issues has been surfaced. The stop criterion may then determine when the situation need to be assessed. It may be noted here that resource considerations often will determine the stop criteria in development projects.
If cooperative prototyping is taking place as part of an incremental development project, for instance, stop and success criteria for the overall design could be considered to decide the extent to which the overall design need to be explored and evaluated in a temporary prototyping environment, before the group starts using the target environment to develop and deliver sub-systems to the users.
But also smaller portions of a cooperative prototyping process may be planned in terms of stop and success criteria. For sessions with emphasis on technique (1), exploration of design ideas and users' work practice, stop and success criteria can be formulated as questions to be explored regarding specific frame tasks and the overall design. The working group then regularly turns to the questions and assesses whether it is satisfied with the current results. For sessions with emphasis on technique (2), work-like evaluations, the stop and success criteria are related to assessments of the prototype's ability to support users' performance of the frame task with few breakdowns due to misfits between prototype and work practice.
Assessing the results of sessions is hard to do in retrospect if no documentation was made during the sessions. The need for documentation vary depending on the purpose of the session. To summarize, at least the following questions concerning evaluation criteria should be considered when organizing sessions: What determines when exploration and evaluation is sufficient to let follow up activities continue? What determines when the process need to be reassessed? Will the result of the prototyping determine a choice between several alternative continuations of the project, and how? What are the criteria to be used for the evaluation of prototypes? Which data should be collected during the evaluation, and how? How should the results be reported?
Pointing to the preparation issues and questions in statements 20-25 does not mean that the working group is required to formally use the check list for each two hour session they are arranging. But it is a good idea for the group to keep them in mind and to consider all these issues in advance to a series of sessions with a common purpose; and for each of the sessions consider whether the purpose has changed and hence other issues need reconsideration.
This thesis is multi-disciplinary in nature, and the applied research methods were inspired from computer science, social science, psychology, and anthropology. It should also be noted that the subject, or area, which was studied is not covered by an established set of theories and methods nor is there an established research community working with the area. Rather the area is fuzzy and part of the work was to combine ideas from different areas of research as it was outlined in Sections 2 and 3.
The methods that can be applied to research in such an area are necessarily exploratory in nature as well as the attempt to apply existing methods is exploratory. I were working both theoretical, empirical, and experimental to accomplish the research described in this thesis. The results of the research is theoretical in nature. The results are presented in the papers as discussions of literature, descriptive analysis of field studies and existing prototyping tools, and recommendations on techniques. In addition, the fields studies were undertaken as a kind of action research, hence the research had a direct, although modest, impact on the practice of the dental assistants and the caseworkers who participated.
Yes No
Do the researchers control Experiment Observation
the situation ?
Is it a real situation ? Field Laboratory
Is the theoretical Structured Unstructured
framework decided upon in (Quantitative) (Qualitative)
advance ?
Table
1: Characterization of empirical research approachesMunk-Madsen (1986) also claims a fundamental relationship between correctness and relevance of empirical research results: researchers achieve correctness on the expense of relevance and vice versa when they plan and undertake a research process. This corresponds to the distinction made by Mason (Manuscript) between "Tightness of Control" and "Richness of Worldly Realism." Mason describes the relationship between these as illustrated in Figure 4.
Paper 1 represents a theoretical analysis of results from a field study (Christensen, Grønbæk, & Rolskov, 1987) based on open-ended qualitative interviews (Patton, 1984, ch. 7) with system developers. Compared to table 1 this study was based on observation through interviews concerning situations from real projects. We used the same open-ended interview guide in each of the projects studied. The study was qualitative in nature and illuminating new research questions was at least as important as answering questions we had posed in advance. The research is characterized as unstructured qualitative, since we had not decided on a theoretical framework in advance to the study. The study was planned to give relevance priority over correctness. This means that we cannot support our general conclusions with extensive quantitative data material. The relevance of the conclusions can only be supported by observing practitioners', in this case system developers', reactions to the conclusions. This research contributed with ideas and research questions that were further explored in the Dental Clinic study and the Municipality study which are discussed in Papers 2, 4, and 5.
Figure 4: Relationship between "Tightness of Control" and "Richness of Worldly Realism" according to Mason (Manuscript).
The Dental Clinic study, discussed in Paper 2, can be characterized according to the framework as follows. The degree of control was somewhere in between that of an experiment and that of an observation. The researchers (SB and myself) were participating one at the time as the designer in the project being analyzed. Thus we could maintain some degree of control with the situations studied. This degree of control was, however, far from being comparable to a controlled experiment in the strict sense. We were controlling such aspects as the choice of tools, techniques, how much time were spent in the sessions and the like. But we did not set up rules for the Dental Assistants' course of action in the sessions. We did not either prepare a set of questions for the Dental Assistants to answer in conjunction to the study, rather we were observing and taking notes on what happened during the sessions. The studies took place in a kind of laboratory setting, a residential course taking place remote from the Dental Assistants' workplaces. In addition, Dental Assistants who usually do not work together was brought together in the sessions. This in fact turned out to be an advantage for studying their reactions, since they were reflecting on their experiences by talking to each other during the sessions. With respect to the theoretical framework, we again followed an unstructured qualitative approach and let the observations direct the following analysis. This implies that the relevance again is given higher priority than correctness. The main contribution of the study was a proof of concept of the ability to establish a close interplay between in-session modifications of prototypes and work-like evaluations. The descriptive analysis of situations illustrating this was met with interest by both researchers (Paper 2 was accepted for publication) and practitioners (results were presented to practitioners at conferences cf. the Preface).
The Municipality study, discussed in Papers 4 and 5, may be characterized according to the framework as follows. Again the degree of control was somewhere in between that of an experiment and that of an observation. The researchers (SB and myself) were participating at the same time as designers in the design project being analyzed. We could maintain some degree of control, but again the study was far from being a controlled experiment in the strict sense. We were controlling such aspects as the choice of tools, techniques, and the division of work between designers. In this case we could hardly control the amount of time to be spent in the sessions, because we were interleaving the sessions with the daily work going on in the municipality office. We did not set up rules for the caseworkers' course of action during the sessions, but we asked them to prepare for the sessions. For instance, by asking them to collect materials to be used for the frame tasks. Again we did not prepare a set of questions for the caseworkers to answer in conjunction to the study. But in conjunction to observation and note taking during the sessions we made video recordings of the later sessions in order to analyze the situations post hoc. The study was taking place in the field, i.e. close the daily work situations of the caseworkes: the prototyping sessions took place in a room next to the caseworkers offices, which made it possible for them to go and pick up material from their desk, or they could ask us to follow them to their office to see something relevant. The prototyping sessions were interleaved with the caseworkers daily work, but they were not part of the daily work. Hence the study still share characteristics with a laboratory setting, but in weaker sense than in the Dental Clinic study. With respect to the theoretical framework, we again followed an unstructured qualitative approach and let the observations direct the following analysis. In this case we, however, decided in advance to make a more detailed descriptive analysis of situations where users and designers engage in cooperative prototyping activities. This purpose directed the decision on making video recordings of the sessions. Moreover, we also had in our minds that the Activity Theory based analysis techniques described by (Engeström & Engeström, 1989; Engeström et al., 1988) could be applied, but it did not have any impact on the planning of the study except for making video recordings. To summarize, this study also gives higher priority to relevance than to correctness, i.e. general conclusions cannot be supported with extensive data, but with respect to the specific conclusions the video recordings serve as original documentation that can be assessed. The main contribution of the study is the descriptive analysis and categorisation of the variety of situations and focus shifts that may take place in cooperative prototyping sessions. Again the relevance can only be supported by the fact that the results were met with interest by both researchers (Papers 4 and 5 were accepted for publication) and practitioners (results were presented to practitioners at conferences cf. the Preface).
The first kind of theoretical work served as background for the introductory sections of this overview where examples of literature from various areas of research on prototyping and user involvement is categorized and discussed. Also Paper 3 is based on this kind of theoretical work, for Paper 3 it is mainly literature on prototyping tools and programming environments which is discussed. Experiences from using some of the tools contributed to the discussion in Paper 3 as well.
The second kind of theoretical work served as background for the discussions in Papers 1 and 3. In both cases it was general knowledge about computer systems, system development, and related problem areas that contributed to the analysis of the field studies. Of course this kind of theoretical work apply to the Municipality study as well, but for Papers 1 and 3 the theoretical analysis is purely based on this background. In Paper 1 philosophical literature like (Polanyi, 1966) is also referenced, but this kind of literature is not unusual as part of a curriculum in system development inheriting perspectives from the Collective Resource Approach. Thus it was familiar to me from studying system development.
The third kind of theoretical work was applied for the analysis related to the municipality studies. The results of this kind of work is described in Paper 4. We were inspired by the studies of health center physicians' consultations with patients described in (Engeström & Engeström, 1989; Engeström et al., 1988). In Paper 4 an Activity Theory based framework is applied for analyzing situations and focus shifts occurring in prototyping sessions. This framework helped us to identify subjects, objects, and instruments and contradictions between these, when studying videos from cooperative prototyping sessions. This identification again helped us in categorizing situations and focus shifts that resulted from contradictions, e.g. between the instruments of the designers and the instruments of the users.
The fourth kind of theoretical work is the basis for Paper 5, and it is strongly inspired by the work of Suchman & Trigg (1991) on applying interaction based video analysis to study situations of computer use as well as system design. SB and I had made approximately 6 hours of what Suchman & Trigg call setting-oriented recordings from the room where we undertook our cooperative prototyping sessions in the Municipality study. By setting-oriented recording is meant a stationary camera which covers a fixed part of the physical space where the activity to study takes place. When showing pieces of these video recordings to Randall Trigg he encouraged us to participate in an interaction based analysis of a piece of this video that SB and I found difficult to understand and use for our discussions of cooperative prototyping. SB and I had in conjunction with the Activity Theory based analysis for Paper 4 made a content log of the selected piece of video as our annotation of the video. But to accomplish the interaction analysis we delved into detailed analysis where we ended up transcribing a major part of the selected 36 minutes of video. As described in Paper 5 this detailed analysis changed SB and my understanding and view of the situation. This experience made us also give recommendations to designers on the prospects of using video analysis for studying and developing their own techniques.
The type (1) continuation is hard to establish within a research environment. To be ideal it requires extensive participation from the researchers in a full development project. But less ideal continuations in this direction can and will be set up. A current project at the Work Environment Inspection office in Aarhus utilizes the cooperative prototyping approach in conjunction with other cooperative design techniques as outlined in (Bødker, Grønbæk, & Kyng, 1990).
An EEC founded Esprit II research project, EuroCoOp, (EuroCoOp, 1990) where Aarhus University, Computer Science Department is a partner, starts in the spring of 1991. Among other things this project shall produce a requirements specification for potential use of CSCW technology at the Danish Great Belt company. This company is employed in building the Great Belt bridge and tunnel connection between Zealand and Fuen. For making the requirements specification we expect among other techniques to apply a cooperative prototyping approach, and in conjunction to this also study problems and prospects of the approach.
A type (2) continuation was already initiated within the DEVISE research project (DEVISE, 1991). We are working on developing an object-oriented application generator, APPLBUILDER, which is aimed at supporting both prototyping and final development of applications (Grønbæk, Hviid & Trigg, 1990). APPLBUILDER includes the following features: (1) A direct manipulation graphical interface which is important for quickly laying out complex application interfaces. The graphical editors used to build the application's user interface objects (UIOs) automatically generate code. (2) An object-oriented programming language which was argued (tenet 15) to be useful for creating prototypes by assembling and specializing objects from existing prototypes and libraries of UIO implementations. Moreover, the ability to represent objects familiar to users directly in the user interface and thus in programs supports the end-users' understanding of prototypes and systems. (3) Support for fine-grained modularization of code and separate compilation is essential in order that the work of modification be localized at appropriate points in the prototype (for example, in the form of scripts behind UIOs) and that minimal re-compilation overhead be incurred. This ensures a reasonable short turn-around time for modifications.
APPLBUILDER is being built on top of the object-oriented BETA programming language and makes use of Mjølner BETA System's Meta Programming and Fragment Systems (Kristensen, Madsen, Møller-Pedersen, & Nygaard, 1987, Kristensen, Madsen, Møller-Pedersen, & Nygaard, 1990, Madsen & Nørgaard 1988). The current prototype implementation of APPLBUILDER is for the Apple Macintosh. APPLBUILDER is based on code generation and management using the Meta Programming System as proposed by Bjerregaard & Hviid (1990).
APPLBUILDER is a tool meant to be driven by system designers when it comes to programming of scripts and other code modules. But the graphical and direct manipulation facilities support applying it in cooperative prototyping sessions as well. APPLBUILDER provides a freedom in choice of approach to application prototyping and development. In fact, APPLBUILDER supports at least the following approaches: (1) starting with design of UIOs and building up a horizontal prototype having little functionality; (2) starting from a "model", i.e. an implemented object-oriented description of the system, but with no UIOs implemented; (3) vertical prototyping, i.e. implementing the functionality behind a subset of a horizontal prototype or just starting with a full implementation of a small subset of a system prepared for incremental extension; (4) simulation of functionality, i.e. adding temporary short cut or dummy computations to a horizontal prototype to make it operate on sample data; and finally (5) full application development. In addition, these different ways of using APPLBUILDER may be combined differently.
We are planning to develop APPLBUILDER into an industrial prototype during the next two years within the DEVISE project. A long term plan is to also integrate APPLBUILDER with a distributed hypermedia based cooperative design tool being developed in the EuroCoOp (EuroCoOp, 1990) project. This integration should support making both process and product oriented documentation in conjunction with the development of prototypes and systems (cf. tenet 17 above).
Andersen, N. E., Kensing, F., Lundin, J., Mathiassen, L., Munk-Madsen, A., Rasbech, M., & Sørgaard, P. Professional system development - Experience ideas and action. Englewood Cliffs, NJ: Prentice Hall, 1990.
Andriole S. J. Storyboard Prototyping - A New Approach to User Requirements Analysis. QED Information Sciences, Inc., Wellesley, MA, USA, 1989.
Baecker, R. & Buxton, W. Empirical Evaluation of User Interfaces. In Baecker, R. M. & Buxton, W. A. S. (eds.), Readings in Human Computer Interaction: A multidisciplinary Approach. Los Altos, CA: Morgan Kaufman, 1987, pp. 131-134.
Bally, L., Brittan, J. & Wagner, K. H. A Prototype Approach to Information System Design and Development. Information & Management 1, 1977, pp. 21-26.
Balzer, R., Goldman, N. M., & Wile, D. S. Operational Specification as the Basis for Rapid Prototyping. In Software Engineering Notes 7(5) Special Issue on Rapid Prototyping, Squires, S. L., Branstad, M., & Zelkowitz, M. (eds.), ACM, New York, December 1982, pp. 3-16.
Bannon, L. & Grønbæk, K. Hypermedia: Support for a more natural information organization. In H. Clausen (ed) Proceedings of the 7th Nordic Conference for Information and Documentation, DK-2800 Lyngby, Denmark, Dansk Teknisk Litteraturselskab, 1989. Also available as DAIMI-PB 287.
Bannon, L. From Human Factors to Human Actors: The Role of Psychology and Human-Computer Interaction Studies in Systems Design. In J. Greenbaum & M. Kyng (eds.), Design at Work: Approaches to Collaborative Design. Hillsdale, NJ: Lawrence Erlbaum Associates, 1991. In press.
Bannon, L. Issues in Design: Some notes. In Norman, D. A. & Draper, S. W. (eds.) User Centered System Design - New Perspectives on Human-Computer Interaction, Lawrence Erlbaum Associates, Publishers, Hillsdale, New Jersey, 1986.
Bansler, J. Systems Development Research in Scandinavia: Three Theoretical Schools. Scandinavian Journal of Information Systems 1, August 1989, pp. 3-20.
Bansler, J. Systemudvikling - teori og historie i skandinavisk perspektiv (in English : System Development - Theory and History in a Scandinavian Perspective). Lund: Studentlitteratur, 1987.
Bergo, O. T. & Nygaard, K. (eds.) En vurdering av styrings- og informationssystemet KVPOL. Oslo: Tiden Norsk Forlag, 1974.
Berrisford, T. & Wetherbe, J. C. Heuristic Development: A Redesign of Systems Design. MIS Quarterly, 3(1) March 1979, pp. 11-19.
Bjerknes, G., Ehn, P., & Kyng, M. (eds.) Computers and Democracy - A Scandinavian Challenge, Avebury, Aldershot, England, 1987.
Bjerregaard, B. S, & Hviid, A, The Development of a Graphical Object-Oriented Prototyping System - GrOOPS. DAIMI IR 93 Computer Science Department, Aarhus University, Århus, Master Thesis, May 1990.
Blomberg, J. & Henderson, D. A. Reflections on Participatory Design: Lessons from the Trillium Experience. In J. C. Chew & J. Whiteside (eds.), Empowering People CHI '90 Conference Proceedings, ACM, New York, 1990, pp. 353-359.
Boar, B. H. Application Prototyping - A requirements definition strategy for the 80s, John Wiley and Sons, Inc., New York, 1984.
Boehm, B. W. Software Engineering Economics, Prentice-Hall, Englewood Cliffs, New Jersey , 1981.
Bondgård, P., Degn, E. M., & Jensen, K. V. Idealer og realiteter i eksperimentel systemudvikling. [Ideals and Realities in Experimental System Development]. Unpublished report. Institute of Electronic Systems, Department of Computer Science. University of Ålborg, Denmark, 1990.
Booker, S. & Vertelney, L. Designing the Whole-Product user Interface. In Laurel, B. (ed.) The Art of Human Computer Interface Design. San Francisco CA: Addison Wesley, 1990.
Brooks, F. P., Jr. The Mythical Man-Month - Essays on Software Engineering, Addison-Wesley Publishing Company, Reading, Mass, 1975.
Bødker, S. & Grønbæk, K. Cooperative Prototyping Studies - Users and Designers Envision a Dental Case Record System. In J. Bowers & S. Benford (eds.), Studies in Computer Supported Cooperative Work: Theory, Practice and Design, Amsterdam: Elsevier Science Publishers/North Holland. 1991a.
Bødker, S. & Grønbæk, K. Cooperative Prototyping: Users and Designers in Mutual Activity. International Journal of Man-Machine Studies, 34, Special Issue on CSCW, 1991b.
Bødker, S. & Grønbæk, K. Design in Action: From Prototyping by Demonstration to Cooperative Prototyping. In Greenbaum, J. & Kyng, M. (eds.). Design at Work: Cooperative Design of Computer Systems. Hillsdale, NJ: Lawrence Erlbaum Associates, 1991c. In press.
Bødker, S.: Prototyping revisited - design with users in a cooperative setting. In P. Järvinen, (ed.) Report of the 10th IRIS, Vaskievesi, Finland, 1987, pp. 71-92.
Bødker, S. Through the Interface - a Human Activity Approach to User Interface Design. Hillsdale, NJ: Lawrence Erlbaum Associates, 1991.
Bødker, S., Ehn, P., Kammersgaard, J., Kyng, M., & Sundblad, Y. A UTOPIAN Experience: On design of powerful computer-based tools for skilled graphic workers. In Computers and Democracy. Bjerknes, G., Ehn, P., & Kyng, M. (eds.), Avebury, Aldershot, England, 1987, pp. 251-278.
Bødker, S., Grønbæk, K. & Kyng, M. Cooperative Design: Techniques and Experiences from the Scandinavian Scene. Draft submitted for publication, November 1990.
Bøgh Andersen, P. et al: Research Programme on Computer Support in Cooperative Design and Communication, DAIMI-IR 70, University of Aarhus, 1987.
Canning, R. G. Developing systems by prototyping. Edp Analyzer 19(9), September 1981.
Canning, R. G. Speeding up application development. Edp analyzer 23(4), April 1985.
Christensen, S., Grønbæk, K., &, Rolskov, T. Arbejdsformer under anvendelse af 4. generationsværktøjer. DAIMI-IR 69, Computer Science Department, Aarhus University, Masterthesis. (Title in English: Use of 4th Generation Systems in System Development), May 1987.
Coad, P. & Yourdon, E. Object-Oriented Analysis. Englewood Cliffs, NJ: Prentice Hall, 1990.
Cohen, D., Swartout, W., & Balzer, R. Using Symbolic Execution to Characterize Behaviour. In Software Engineering Notes 7(5) Special Issue on Rapid Prototyping, Squires, S. L., Branstad, M., & Zelkowitz, M. (eds.), ACM, New York, December 1982, pp. 25-32.
Conklin, J. Hypertext: An Introduction and Survey. IEEE Computer, 20(9), 1987, pp. 17-41.
Curtis, G. & Vertelney, L. Storyboards and Sketch Prototypes for Rapid Interface Visualization. Tutorial notes no. 33 from the CHI `90 conference, 1990.
DEVISE. Center for Experimental Systems Development. Funding application sent to the Coordination Committee for Informatics research in Denmark. January, 1991.
DUE. Klubarbejde og EDB. [Local Union Activities and EDP], Denmark: Fremad 1981.
Ehn, P. & Kyng, M.: The Collective Resource Approach to Systems Design, in Bjerknes, G. et al. (eds.): Computers and Democracy - a Scandinavian Challenge, (pp. 17-58) Avebury, 1987
Ehn, P. & Sandberg, Å. Local Union Influence on Technology and Work Organization. In Briefs, U., Ciborra, C. & Schneider, L. (eds.) Systems Design For, With and By the Users. Amsterdam: North-Holland Publishing Company, 1983. Pages 427-437.
Ehn, P. Work-oriented design of computer artifacts. Falköping: Arbetslivscentrum/Almqvist & Wiksell International, 1988.
Ehn, P., & Kyng, M. A tool perspective on design of interactive computer support for skilled workers. In M. Sääksjärvi (Ed.), Proceedings from the Seventh Scandinavian Research Seminar on Systemeering. Helsinki: Helsinki Business School, 1984, pp. 211-242.
Ehn, P., Kyng, M. & Sundblad, Y. The UTOPIA project. In Briefs, U., Ciborra, C. & Schneider, L. (eds.) Systems Design For, With and By the Users. Amsterdam: North-Holland Publishing Company, 1983. Pages 439-449.
Ehn, P., Mölleryd, B. & Sjögren, D. Playing in Reality: A paradigm case. Scandinavian Journal of Information Systems 2, August 1990, (pp. 101-120).
Engeström, Y., & Engeström, R. Constructing the object in the work activity of primary care physicians. Unpublished manuscript, 1989.
Engeström, Y., Engeström, R. & Saarelma, O. Computerized Medical Records, Production Pressure and Compartimentalization in the Work Activity of Health Center Physicians. In Proceedings of Conference on CSCW, Portland, Oregon, September 1988 (pp. 65-84) New York: ACM.
EuroCoOp. ESPRIT project 5303: EuroCoOp - IT Support for Distributed Cooperative Work. Technical Annex. November 1990.
Feather, M. S. Mappings for Rapid Prototyping. In Software Engineering Notes 7(5) Special Issue on Rapid Prototyping, Squires, S. L., Branstad, M., & Zelkowitz, M. (eds.), ACM, New York, December 1982, pp. 17-24.
Floyd, C. A Systematic Look of Prototyping. In R. Budde, K. Kuhlenkamp, L. Mathiassen, & H. Züllighoven (eds.), Approaches to Prototyping. Berlin: Springer Verlag, 1984, pp. 1-18.
Floyd, C., Reisin, F.-M., & Schmidt, G. STEPS to Software Development with Users. In C. Ghezzi & J. M. McDermid (eds.) proceedings from ESEC `89. Lecture Notes in Computer Science 387, Berlin: Springer-Verlag, 1989, pp. 48-64.
Gomoll, K. Some Techniques for Observing Users, In B. Laurel (ed.) The Art of Human Computer Interface Design, Reading, MA: Addison Wesley, 1990, pp. 85-90.
Good, M. D., Whiteside, J. A., Wixon, D. R., & Jones, S. J. Building a User-Derived Interface. Communications of the ACM 27, 10 (October 1984), 1032-1043.
Goodman, D. The Complete HyperCard Handbook, Bantam Books, New York, 1987.
Gould, J. D. How To Design Usable Systems, In Helander, M. (ed.) Handbook of Human -Computer Interaction, Amsterdam: Elsevier (North-Holland) 1988. Pages 757-789.
Greenbaum, J. & Kyng, M. (eds.). Design at Work: Cooperative Design of Computer Systems. Hillsdale, NJ: Lawrence Erlbaum Associates, 1991. In press.
Grudin, J. Obstacles to Participatory Design in Large Product Development Organizations. In Namioka, A. & Schuler, D. (eds.) Proceedings of the Participatory Design Conference (PDC' 90) held in Seattle, WA, March 31- April 1, 1990. Palo Alto, CA: Computer Professionals for Social Responsibility, Workplace Project, 1990a.
Grudin, J. The Computer Reaches Out: The Historical Continuity of Interface Design. In Chew, J.C. & Whiteside, J. (eds.), Empowering People CHI '90 Conference Proceedings (pp. 261-268), New York: ACM, 1990b.
Grønbæk, K. Rapid Prototyping with Fourth Generation Systems - an Empirical Study. OFFICE:Technology and People, 5(2), 1989, pp. 105-125.
Grønbæk, K. Supporting Active User Involvement in Prototyping. Scandinavian Journal of Information Systems 2, 1990. pp. 3-24.
Grønbæk, K., Hviid, A., & Trigg, R. H. APPLBUILDER - an Object-Oriented Application Generator Supporting Rapid Prototyping. Draft submitted for publication. November 1990.
Hauerslev, J. & Jakobsen, N. SOKRATES: Sprogspil og kvalifikationer - en rapport om arbejdet med en teori for erkendelsesprocesserne i systemudvikling. [SOKRATES: Language games and qualifications - a report on the work with a theory of learning processes in system development] Master thesis - Unpublished report. Computer Science Dept., Aarhus University, November 1988.
Hekmatpour, S. & Ince, D. Software Prototyping, Formal Methods and VDM, Addison-Wesley, Wokingham, England, 1988.
Henderson, D. A. The Trillium User Interface Design Environment. In CHI '86 Proceedings, ACM, 1986.
Horowitz, E., Kemper, A., & Narasimhan, B. A Survey of Application Generators. IEEE Software, January 1985, pp. 40-54.
Hsia, P. & Yaung, A. T. Screen-based Scenario Generator: A Tool for Scenario-based Prototyping. In Proceedings of the Twenty-First Annual Hawaii International Conference on System Sciences, Vol. II, Shriver, B. D., Computer Society Press of the IEEE, Massachusetts, Washington D.C., 1988.
Janson, M. A. & Smith, L. D. Prototyping for Systems Development: A Critical Appraisal. MIS Quarterly, 9(4) December 1985, pp. 305-316.
Johnson, J. (moderator). Panel on Participatory Design of Computer Systems. In Chew, J. C. & Whiteside, J. (eds.), Empowering People CHI '90 Conference Proceedings New York: ACM, 1990, pp. 261-268,
Jungk, R. & Müllert, N. Future Workshops: How to create desirable futures. London: Institute for Social Inventions, 1987.
Kensing, F. & Madsen, K. H. Generating Visions: Future Workshops and Metaphorical Design. In Greenbaum, J. & Kyng, M. (eds), Design at Work: Cooperative Design of Computer Systems. New Jersey, USA: Lawrence Earlbaum Associates, 1991. In press.
Kraushaar, J. M. & Shirland, L. E. A Prototyping Method for Applications Development by End Users and Information Systems Specialists. MIS Quarterly, 9(3) September 1985, pp. 189-197.
Kristensen, B. B., Madsen, O. L., Møller-Pedersen, B., & Nygaard, K: Object-Oriented Programming in the Beta Programming Language. Book in preparation, January 1990.
Kristensen, B. B., Madsen, O. L., Møller-Pedersen, B., & Nygaard, K: The BETA Programming Language. In: B.D. Shriver, P.Wegner(eds.), Research Directions in Object Oriented Programming, MIT Press, 1987.
Kubicek, H. User Participation in System Design - Some Questions. In Briefs, U., Ciborra, C. & Schneider, L. (eds.) Systems Design For, With and By the Users. Amsterdam: North-Holland Publishing Company, 1983. Pages 3-18.
Kyng, M. & L. Mathiassen: Systems Development and trade union activities, in Bjørn-Andersen, N. (ed.): Information Society, for richer, for poorer, North-Holland, Amsterdam 1982
Lamersdorf, W., & Schmidt, J. W. Specification and prototyping of data model semantics. In R. Budde, K. Kuhlenkamp, L. Mathiassen, & H. Züllighoven (eds.), Approaches to Prototyping. Berlin: Springer Verlag, 1984, pp. 214-231.
Lantz, K. E. The Prototyping Methodology, Prentice Hall, Englewood Cliffs, 1986.
Leibrandt, U., & Schnupp, P. An evaluation of Prolog as a prototyping system. In R. Budde, K. Kuhlenkamp, L. Mathiassen, & H. Züllighoven (eds.), Approaches to Prototyping. Berlin: Springer Verlag, 1984, pp. 424-433.
Lewis, C. Using the "Thinking-aloud" Method in Cognitive Interface Design. Research Report. IBM Research Division, Yorktown 1982.
Lewis, T. G., Handloser, F., Bose, S., & Yang, S. Prototypes From Standard User Interface Management Systems. In Proceedings of the 22nd annual Hawaii International Conference on System Sciences, Vol II, Shriver, B. D., Computer Society of the IEEE, Washington D.C., January 1989, pp. 397-406.
Lie-Nielsen, S. & Colliander, L. Principer för en ny generation systemutvecklingsverktyg. RDFs Rapportserie no. 16, Riksdataförbundet, Lund, 1986.
Lucas, H. C. Toward Creative Systems Design. New York: Columbia University Press, 1974.
Madsen, O.L., Nørgaard, C.: An Object-Oriented Metaprogramming System. Hawaii International Conference on System Sciences - 21, pp 406-415, January 5-8, 1988.
Martin, J. & Leben, J. Fourth generation languages. Vol. II: Representative 4GLS, Prentice-Hall, Englewood Cliffs, N.J., 1986.
Martin, J. Fourth generation languages. Vol. I , SAVANT, 1983.
Martland, D., Holloway, S., & Bhabuta, L. Fourth generation Languages and Application Generators, The Technical Press - Unicom Applied Information Technology Report Series, 1986.
Mason, R. E. A. & Carey, T. T. Prototyping Interactive Information systems. Communications of the ACM 26(5), May 1983, pp. 347-354.
Mason, R. O. Experimentation and Knowledge - A Pragmatic Perspective. Manuscript. Edwin L. Cox School of Business, SMU, Dallas, TX. Manuscript.
Mathiassen, L. Systemudvikling og systemudviklingsmetode [Systems development and systems development method] DAIMI PB-136. Århus: University of Aarhus, 1981.
Mathiassen, L., Rolskov, B., & Vedel., E. Regulating the Use of Edp by Law and Agreements. In U. Briefs et al. Systems Design For, With and By Users. North Holland, 1983.
Miyata, Y. & Norman, D. A. Psychological Issues in Support of Multiple Activities. In Norman, D. A. & Draper, S. W. (eds.) User Centered System Design - New Perspectives on Human-Computer Interaction, Lawrence Erlbaum Associates, Publishers, Hillsdale, New Jersey, 1986.
Mogensen, P. Provotyping? In Hellman, R., Ruohonen, M., & Sørgaard, P. (eds.): Preceedings of the 13th IRIS conference, Reports on Computer Science & Mathematics no. 108, Åbo Akademi University, 1990, pp 299-312.
Monk, A. How and When to Collect Behavioural Data. In Baecker, R. & Buxton, W. (eds.) Readings in Human Computer Interaction: A multidisciplinary approach. Morgan Kaufman, Los Altos, CA, 1987. Pages 138-142.
Mumford, E. & Ward, T. B. EDB, system og menneske (Danish translation of "Computers: Planning for People," 1968). Copenhagen: Hasselbachs forlag, 1970.
Mumford, E. Sociotechnical Systems Design: Evolving Theory and Practice. In Computers and Democracy - A Scandinavian Challenge. Avebury, Bjerknes, G., Ehn, P., & Kyng, M., pp. 59-76, Aldershot, England, 1987.
Munk-Madsen, A. Viden om systemudvikling. [Knowledge about System Development] DAIMI PB 213, Computer Science Department, Aarhus University, 1986.
Namioka, A. & Schuler, D. (eds.) Proceedings of the Participatory Design Conference (PDC' 90) held in Seattle, WA, March 31- April 1, 1990. Palo Alto, CA: Computer Professionals for Social Responsibility, Workplace Project, 1990.
Naumann, J. D. & Jenkins, A. M. Prototyping: The New Paradigm for Systems Development. MIS Quarterly, 6(3). September 1982, pp. 29-44.
Ng, P. A. & Yeh, R. T. Modern Software Engineering - Foundations and Current Perspectives. New York: Van Norstrand Reinhold, 1990.
Norman, D. A. & Draper, S. W. (eds.) User Centered System Design - New Perspectives on Human-Computer Interaction, Lawrence Erlbaum Associates, Publishers, Hillsdale, New Jersey, 1986.
Norman, D. A. Cognitive Engineering. In Norman, D. A. & Draper, S. W. (eds.) User Centered System Design. Lawrence Erlbaum Associates, Publishers, Hillsdale, New Jersey, 1986.
Nygaard, K. & Bergo, O. T. Planlegging, styring og databehandling. Grunnbok for fagbevegelsen, Tiden, Oslo, 1972.
Nygaard, K. & Bergo, O. T. Trade Unions New Users of Research. Personnel Review, 4(2), 1975.
Pape, T. C., & Thoresen, K. Development of Common Systems by Prototyping. In Bjerknes, G., Ehn, P., &, Kyng, M. (eds) Computers and Democracy - A Scandinavian Challenge. Aldershot, England: Avebury, 1987.
Patton, M. Q. Qualitative Evaluation Methods, Beverly Hills: SAGE Publications, 1984.
Polanyi, M. The Tacit Dimension. Rutledge & Kegan Paul ltd., 1966.
Reisin, F.-M. & Wegge, D. Prototyping-based experiments in user-oriented system development. In S. Bødker (ed.), Proceedings of the 12th IRIS conference, DAIMI-PB 296-II, Aarhus University, December 1989, pp. 517-532.
Salomon, G. B. Designing Casual-User Hypertext: The CHI '90 Info Booth. In Chew, J. C. & Whiteside, J. (eds.), Empowering People CHI '90 Conference Proceedings, New York: ACM, pp. 451-458.
Shneiderman, B.: Direct manipulation: A step beyond programming languages, IEEE Computer, August 1983, pp. 57-69.
Smith, D. C., Kimball, R., Irby, C., Verplank, W., & Harslem, E. Designing the STAR User Interface. Byte Magazine, 7(4), April 1982. Also available in Baecker, R. & Buxton, W. (eds.) Readings in Human Computer Interaction: A multidisciplinary approach. Morgan Kaufman, Los Altos, CA, 1987. Pages 653-661.
Squires, S. L., Branstad, M., & Zelkowitz, M. (eds.) Special Issue on Rapid Prototyping. Software Engineering Notes, Vol 7., ACM SIGSOFT, Baltimore, Working Papers from ACM SIGSOFT Rapid Prototyping Workshop, 1982.
Suchman, L. & Trigg, R. Understanding practice: Video as a Medium for Reflection and Design. In Greenbaum, J. & Kyng, M. (eds), Design at Work: Cooperative Design of Computer Systems. Lawrence Earlbaum Associates, New Jersey, USA, 1991. In press.
Suchman, L. Plans and Situated Actions, Cambridge UK: Cambridge University Press, 1987.
Tang J. C. & Leifer, L. J. A Framework for Understanding the Workspace Activity of Design Teams. In Tatar, D. (ed) Proceedings of Conference on CSCW, Portland, Oregon, September 1988, New York: ACM. (pp. 244-249)
Tanner, P. P. & Buxton, W. A. S. Some Issues in Future User Interface Management Systems (UIMS) development. In Pfaff, G. E., (ed.) User Interface Management Systems, Springer-Verlag, Berlin, 1985.
Tatar, D. (ed) Proceedings of Conference on CSCW, Portland, Oregon, September 1988, New York: ACM.
Tavendale, T. D. A technique for Prototyping Directly from a Specification. In proceedings of the IEEE 8th International Conference on Software Engineering, 1985.
Thoresen, K. Prototyping organizational changes. In Namioka, A. & Schuler, D. (eds.) Proceedings of the Participatory Design Conference (PDC' 90) held in Seattle, WA, March 31- April 1, 1990. Palo Alto, CA: Computer Professionals for Social Responsibility, Workplace Project, 1990.
Trigg, R. H., Bødker, S., & Grønbæk, K. A Video-based Analysis of the Cooperative Prototyping Process. In Hellman, R., Ruohonen, M., & Sørgaard, P. (eds.): Preceedings of the 13th IRIS conference, Reports on Computer Science & Mathematics no. 108, Åbo Akademi University, 1990, pp 453-474.
Vertelney, L. Using video to prototype user interfaces. SIGCHI Bulletin, 21(2), 1989, pp. 57-61.
Wasserman, A. I. & Shewmake, D. T. Rapid Prototyping of Interactive Information Systems. In Software Engineering Notes 7(5) Special Issue on Rapid Prototyping, Squires, S. L., Branstad, M., & Zelkowitz, M. (eds.), ACM, New York, December 1982, pp. 171-180.
Wasserman, A. I., Pircher, P. A., Shewmake, D. T., & Kersten, M. L. Developing Interactive Information Systems with the User Software Engineering Methodology. IEEE Transactions on Software Engineering. SE-12(2), February 1986, pp. 326-345.
Wilson, J. & Rosenberg, D. Rapid Prototyping for User Interface Design. In Helander, M. (ed.) Handbook of Human-Computer Interaction. North-Holland, Amsterdam, 1988.
Winograd, T. & Flores, F. Understanding Computers and Cognition, Ablex Publishing Corp., Norwood, New Jersey, 1986.
Xephon Cosultancy. Prototyping for IBM users. Xephon Technology Transfer Ltd., 1985,
Yourdon, E. Managing the System Life Cycle, Yourdon Press, New York, 1982.
Yourdon, E. Modern Structured Analysis. Englewood Cliffs, NJ: Prentice Hall, 1989.
Prototypes need to be closely related to the users' work practice.
Good computer systems should cause few breakdowns in use. Good systems are also accompanied with a language of actions to handle recurrent breakdowns.
Unreflected and unarticulated aspects of users' work need to be considered to design good systems.
It is important to establish a design language-game which makes sense to both users and designers.
4. Exploration of design ideas and work practice through joint manipulation of prototypes stimulates users' active involvement and supports mutual learning about technology and the users' work practice.
5. Analysis of breakdowns in work-like use situations is an important resource for learning about unarticulated aspects of users' work and how these may affect the design of a future computer system.
6. Breakdowns in work-like use situations may be handled by in-session modifications of prototypes to reestablish work-like use evaluation.
7. Establishing a common language game of prototyping requires that designers primarily draw on terminology, tools, and materials familiar to users.
8. Cooperative prototyping progresses from sessions emphasizing exploration of design ideas and work practice, to sessions with point-wise work-like evaluation, and later to sessions with fluent work-like evaluation.
9. Initial prototype(s) should be viewed as learn-and-throw-away. Viewing a prototype as part of the future system often hinders experimentation.
10. Frame tasks for work-like evaluations should be rooted in the users' work practice and be selected by users and designers in cooperation.
11. Illustrative sample data should be prepared and maintained for work-like evaluations.
12. Cooperative prototyping sessions should be kept open-ended, users should be encouraged to take initiatives, and focus shifts should be viewed as openings for learning.
13. Video recordings are useful for documenting sessions. Analysis of recordings can give input for improving design proposals, and for improving prototyping techniques as well.
15. Prototyping tools should support representing objects and concepts from the users' work in familiar ways. Object-oriented environments seem promising in this respect.
16. Prototyping tools should support preserving sample data across in-session modifications of prototypes.
17. Documentation of breakdowns, new ideas, work practice related issues, and decisions may require supplementary tools, such as video equipment, integrated hypertext, and flip-overs.
19. Cooperative prototyping takes place primarily in joint sessions. But users and designers often need to prepare both separately and jointly for sessions.
20. Cooperative prototyping sessions need to be carefully prepared, in particular it is important to clarify the purpose of sessions.
21. The extent of prototypes and sample data to be provided for sessions should be planned according to the purpose. The needs for setting up alternative prototypes should be assessed.
22. Small working groups should be established with designers, and future users released from parts of their daily work responsibilities. Having more than one designer present in sessions is always desirable.
23. The participants need to be prepared for sessions. They all need to understand the purpose of sessions, and users may need specific training.
24. The setting of sessions should be designed according to the purpose. Among the options to consider is whether sessions should take place in a laboratory setting or in the users' workplace.
25. Stop and success criteria should be planned in advance to sessions. Such criteria should be formulated in terms of the frame tasks for the sessions.
Prototyper skal relateres til brugernes arbejde.
Gode edb-systemer giver anledning til få sammenbrud i brug. Til gode edb-systemer er der knyttet et sæt af mulige handlinger overfor tilbagevendende sammenbrud.
Ureflekterede og uartikulerede aspekter af brugeres arbejde skal inddrages for at designe gode edb-systemer.
Det er vigtigt at etablere et design sprogspil, der giver mening for både brugere og designere..
4. Udforskning af designideer og arbejdssituationer gennem fælles manipulering af prototyper stimulerer brugeres aktive deltagelse og understøtter en fælles læring om teknologi og brugernes arbejde.
5. Analyse af sammenbrud i brugslignende arbejdssituationer er en væsentlig kilde til at lære om uartikulerede aspekter af brugeres arbejde, og hvordan disse har indflydelse på udformningen af et fremtidigt edb-system.
6. Sammenbrud i brugslignende afprøvning kan i nogle tilfælde håndteres med umiddelbare modifikationer af prototypen for at genetablere en brugslignende afprøvning.
7. Etablering af et fælles sprogspil for prototyping forudsætter at designerne primært udnytter terminologi, værktøjer og materialer, som brugerne er fortrolige med.
8. Hovedbevægelsesretningen i kooperativ prototyping er fra sessioner med fokus på udforskning af ideer og arbejdssituationer, over sessioner med punktvis brugslignende evaluering, til sessioner med flydende brugslignende evaluering.
9. Tidlige prototyper bør opfattes som lær-og-smid-væk prototyper. Hvis prototyper opfattes som en del af det fremtidige system, lægger det ofte en dæmper på viljen til at eksperimentere.
10. Rammeopgaver for brugslignende afprøvninger bør tage udgangspunkt i brugernes arbejde, og de bør vælges af brugere og designere i fællesskab.
11. Illustrative eksempeldata skal forberedes for og vedligeholdes under brugslignende evalueringer.
12. Kooperative prototyping sessioner bør være åbne, brugerne skal have mulighed for at tage initiativer, og fokusskift bør opfattes som åbninger for øget erkendelse.
13. Videooptagelser er værdifulde som dokumentation af sessioner. Analyse af optagelser kan give ideer til forbedring af det konkrete designforslag samt ideer til forbedring prototyping teknikker.
15. Prototypingværktøjer bør repræsentere objekter og begreber fra brugernes arbejde på en form, der virker fortrolig for brugerne. Objekt-orienterede omgivelser er specielt anvendelige til dette formål.
16. Prototypingværktøjerne bør understøtte bevarelse af eksempeldata under manipulation af prototypen.
17. Dokumentation af sammenbrud, nye ideer, arbejdsrelaterede forhold og beslutninger kan kræve supplerende værktøjer, som for eksempel videoudstyr, integreret hypertekst og "flip-overs".
19. Kooperativ prototyping foregår primært i fælles sessioner. Men brugere og designere er ofte nødt til at forberede sig både hver for sig og sammen til sessionerne.
20. Kooperative prototyping sessioner bør forberedes omhyggeligt, især er det vigtigt at afklare formålet med sessioner.
21. Niveauet for udvikling af prototyper og eksempeldata før sessioner bør tilrettelægges i forhold til formålet med sessionen. Behovene for at udvikle alternative prototyper bør vurderes.
22. Små arbejdsgrupper etableres med designere og fremtidige brugere, der er frigjort fra dele af deres daglige arbejde. Det er altid en fordel at være mere end én designer i sessionerne.
23. Deltagerne skal forberedes til sessioner. Alle skal forstå formålet med en session, og nogle brugere har behov for specifik oplæring.
24. En sessions omgivelser skal etableres i forhold til formålet med sessionen. Valgene står bl.a. mellem at arrangere sessionen i et "laboratorium" eller på brugernes arbejdsplads.
25. Stop- og succeskriterier bør planlægges forud for sessioner. Kriterierne bør formuleres i termer af rammeopgaverne for sessionerne.