5. Intentions for future work

There are three main areas of consideration: doing further fieldwork, enhancing the usefulness of the methods considered, and exploring various theoretical questions. Fieldwork is useful partly for its own sake - to ground my thoughts on method in real experience of evaluations - but also to allow experimentation with the developing method to continue. Various options for future fieldwork present themselves:

continuing the health centre studies.
evaluation of other PhD students' systems, as an exploration of 'generalised' evaluation (see below).
other contacts that may arise.

One methodological question to be pursued is the difference in the evaluation of systems in use in a particular situation (the programme evaluation model) and the evaluation of a system in a more generalised setting (the HCI model). Whether the latter is usefully achievable is one question, but it seems clear that it is necessary for CSCW designers who are building systems without a specific organisational context in mind. For this reason I have mentioned above the evaluation of other PhD students' work, I suppose primarily as 'laboratory' evaluation (whatever the methods).

The obvious route in the development of methods is to use further, and look further into, the PETRA method. While multiplicity seems important, it might also be worth considering other ways of harnessing multiplicity of perspective and of method than in PETRA. An example of this is Multiview; another is Soft Systems Methodology, an interesting combination of interpretive approaches. The question of different stakeholders' perspectives needs to be considered further, and particuarly their integration if they are not the same: should one set be privileged above another, or should some attempt at reconciling conflicting views be made?

On theory, the deeper integration of the wealth of material in programme evaluation into my ideas would be very useful. I would particular like to explore further the dialectical approach of Guba and Lincoln (1989), which integrates different stakeholders' perspectives during the interview process. The argument against specific methods as keys to everything, developed in philosophy by Hans-Georg Gadamer (1975) and in the philosophy of science by Paul Feyerberand (1975) is of great interest. While these may appear to be side issues - or at least overly technical - they are helpful in the deeper development of a global view on CSCW evaluation.

The ultimate product of the work is still a little hazy. I am not interested in producing a step-by-step evaluation tool along the lines of the MEDA tool produced by Murray Saunders and Joan Machell, good though that tool is for its purposes. Likewise, a set of software engineering metrics are not at all likely.

However, I am increasingly convinced of the value of the cookbook approach: a set of principles to assist designers in their evaluation procedures. Wadsworth (1991) called her book Everyday Evaluation on the Run, on the grounds that evaluation should be integrated into everyday work. I certainly don't want a step-by-step guide to evaluation, which is likely to be so general that it is useless. Rather, a set of ideas for the creative cook (evaluator!) seems better as a model of a cookbook. Whether this will be part of my thesis or will be developed alongside/after it with others remains to be seen, but hopefully I can set some of the groundwork here. And of course the development of methods that illustrate these ideas will continue.

Appendix A / Previous Chapter (3) / Contents / References

Go to the Evaluation of Cooperative Systems home page

Cooperative Systems Engineering Group | Computing Department | Lancaster University

Magnus Ramage 10 October 1995