2.3.2. The nature of evaluation
Some general issues in evaluation are worth considering from the literature. The Guide to Local Evaluation written by Elizabeth Sommerlad (1992) and others at the Tavistock Institute summarises many such issues. In general (pp.3-5), she says there are three kinds of issues in evaluation:
- Conceptual ("how the people involved think about the evaluation"): what is the evaluation for; who are the stakeholders; who is interested in the findings; what approach/model/framework will guide the evaluation; what are the main evaluation questions?
- Technical: what methods are appropriate? what will be the unit of analysis? what steps will be taken to ensure quality data? what kind of data will be collected, from whom and using what instruments? what evaluation products for which audiences? what mechanisms/opportunities are there for reflection and review? how should evaluation feed into decision-making?
- Operational: who will be responsible for the evaluation? who will do the evaluation? how involved in planning the evaluation are the people responsible for gathering or providing information? how will the evaluation be managed?
Why evaluation is conducted is an important question. Sommerlad (ibid., pp.7-9) identifies three main purposes:
- To demonstrate accountability (i.e. to funders - cf. summative evaluation).
- As a means to improving programme implementation (cf. formative).
- Learning - "contributing to the professional self-development of participants through critical self-reflection; with encouraging systematic learning about complex problems and concepts and with learning about the process of managing change" (p.8). Cf organisational learning etc.
Stern (1991) lists several more reasons why evaluation is performed: to make a case for funds, to prove to decision-makers that promises made have been kept, to show what benefits have been gained from the programme, even as a way to delay decisions until "all the facts are known". A common phenomenon in public policy is the "justificatory' evaluation": that which is conducted as a public relations exercise, to show funding bodies that the programme was worth doing.
Finally, Sommerlad (1992:11-13) lists the following stakeholders:
- those who care about the programme and its effectiveness (e.g. staff)
- those who influence programme decisions & environment (e.g. funders, administrators, management)
- those who are the intended beneficiaries
- users of the evaluation findings (e.g. academics)
Important points on these are that "stakeholders have different and possibly competing views about what is important, what constitutes success and how success might be measured". Also, "evaluation is 'contested terrain' and so the evaluator must address [stakeholders'] different interests, document the plurality of notions of 'success' and negotiate those issues that are points of contention." (p.11)
Next Section (2.3.3) / Next Chapter (3) / Previous Section (2.3.1) / Previous Chapter (1) / Contents / References
Go to the Evaluation of Cooperative Systems home page
Magnus Ramage 10 October 1995