2.3.4. Taxonomy of approaches

• systems analysis: highly positivist. Uses a few quantitative output variables (e.g. cost), measures these (often) in an experimental fashion, and produces statistical results. Very popular for summative evaluation, and useful for simple situations, but tends to be very managerial in its focus, and to over-simplify complex situations. Note that this term is not related to the computing methods under the same name, or to the systemic perspective advocated in my introduction.
• behavioural objectives (goal-based): looks at the programme goals and determines whether these have been met. Useful and much-used in that it answers the question, "has the programme done what it set to do?" (and is hence relatively easy). However, fails to take into account exactly who set the goals and whether they were appropriate.
• decision-making: the important question is held to be the needs of the decision-makers concerning the programme (managers etc.) in the changing of it, the goal being to increase utilisation. Interviews and questionnaires are common. Useful in that it concentrates on getting maximal use of the evaluation; problematic in that it makes the viewpoint of decision makers central.
• goal-free: the suggestion here is that goal-based evaluation tends to bias the evaluator, leading them to a set of particular concerns. Rather, they should enter a situation with no preconceptions. The consumer evaluations carried out for publications like "Which?" are an example.
• art criticism: evaluators are held to be experts on the area under evaluation, and hence able to draw informed although subjective judgements, the model being art or literary criticism rather than science.
• professional review: programmes are evaluated not by outsiders but by professional peers. Tends to be somewhat introverted and inconsiderate of the public interest.
• quasi-legal: it is possible to run an evaluation like a legal hearing, either on the model of a panel hearing or a court trial. In the former case, a group of experts listen to evidence and adjudicate; in the latter, advocates put opposing sides of an issue and the decision is made by a panel, jury or judge.
• case study: presents a wide range of qualitative views on the programme. Tends to use interviews to appreciate the whole of the participants' understandings. Cf. ethnography.

Go to the Evaluation of Cooperative Systems home page

Cooperative Systems Engineering Group | Computing Department | Lancaster University