Patton (1980:22) has written: "Qualitative measurement has to do with the kinds of data or information that are collected. Qualitative data consist of detailed descriptions of situations, events, people, interactions, and observed behaviours; direct quotations from people about their experiences, attitudes, beliefs, and thoughts; and excerpts or entire passages from documents, correspondence, records, and case histories. The detailed descriptions, direct quotations, and case documentation of qualitative measurement are data from the empirical world. The data are collected as open-ended without attempting to fit programme activities or peoples' experiences into predetermined, standardised categories such as the response choices that comprise typical questionnaires or tests."
Patton also stresses the centrality in qualitative evaluation of a holistic, inductive and a naturalistic approach. That is, the researcher "strives to understand the gestalt, the totality, and the unifying nature of particular settings" (p.41), "begin[s] with specific observations and build[s] towards general patterns" (p.41) and "does not attempt to manipulate the research setting" (p.42). This last point refers specifically to situatedness of qualitative methods: they tend to be based in the 'real world' rather than in the laboratory, although as I have written in Plowman et al (1995), the real-world/laboratory axis is quite distinct in workplace studies from the qualitative/quantitative axis, and several excellent qualitative studies have been conducted in research centres. This idea of not designing an artificial setting seems to me quite different from the notion of not affecting the situation under study, which also seems to me useful at times - see appendix A for a discussion of this relating to the specific question emancipation.
Stern (1993) describes formative evaluation of a kind he calls "real-time". This has the characteristics that: it takes place along with the programme; the evaluation is formative and intrusive, in that it presents its findings during the lifetime of the programme; and the evolution and implementation of the programme are of particular interest. This derives from three particular concerns of the Tavistock Institute Evaluation, Development and Review Unit (EDRU): the use of evaluation to support learning by participants and organisations, the increasing of mutual understanding; participants are actively involved in the planning and implementation of the evaluation; the evaluation is grounded in the wider context beyond the programme - in the organisation and in society. Real-time evaluation works well for long-term programmes with complex dynamics, difficult processes of implementation and uncertain objectives.
Patton (1981) identifies the need for creativity in evaluation, suggesting many participatory techniques for handling the evaluation process. These include: the use of metaphor; flow-charting; matrix thinking (e.g. 2x2s); experiential methods (games, simulation, role-play, demonstrations); use of acronyms and word-play to stimulate thought; story-telling; drawing pictures; the use of humour. He emphasises the importance of making evaluations useful and the many different roles evaluators play ("information broker, data manager, storyteller, artist, humorist or scientist", p.277).
A useful work not discussed further here (for the pragmatic reason that I don't have it currently to hand) is that of Guba and Lincoln (1989) on Fourth-Generation Evaluation, which suggests that not only qualitative but explicitly interpretive methods are necessary for evaluation. They are particularly interested in the management and integration of different stakeholders' perspectives, something that is most useful to this work. This book has much to offer as an approach to evaluation.
Go to the Evaluation of Cooperative Systems home page