Patton (1981:23) has written, "Like the shift in perspective among modern philosophers from a search for universal laws to a concern with situational ethics, the scientific revolution manifested by the emergence of the new standards of evaluation represents a situational approach to the conduct of social science. This stands in stark contrast to the tradition of logical-positivism, with its search for universal truths, and a carefully prescribed set of scientific rules, and operating procedures, to be followed in every legitimate scientific inquiry. The driving force that gave rise to this new evaluation ideology is that there is no one best way to conduct an evaluation. This insight is critical. On this belief rests all the new principles, standards, and premises of the evaluation research discipline. Every evaluation situation is unique. A successful evaluation (one that is practical, ethical, useful, and accurate) emerges from the special characteristics and conditions of a particular situation - a mixture of people, politics, history, context, resources, constraints, values, needs, interests, and chance. Despite the rather obvious, almost trite and basically commonsense nature of these observations, there are a host of subtleties and nuances implicit in the shift in perspective from evaluation judged by a single, standard, and universal set of criteria (methodological rigor as defined by the dominant hypothetico-deductive paradigm) to situational evaluation where decision criteria are multiple, flexible and diverse. This shift in perspective, this revolution in standards of excellence, places new demands on evaluators." Patton also writes (1980, p.20) of a "paradigm of choices" that "recognises that different methods are appropriate for different situations".
Similarly, Sommerlad (1992) argues that methods need to be chosen according to the particular situation, based on criteria such as stakeholders, resources, expertise, organisational politics, and the nature of the evidence required. "Our view [is] that the choice of method should be shaped by what the evaluation is for, the kind of questions being asked, and who will be making use of the evaluation findings. All too commonly, it would seem evaluation begins inappropriately with methodology ('let's do a survey') before clarifying the other questions. ... What matters is choosing the right method for the particular situation." (ibid., p.31; italics in orig.)
One solution is to select methods in toto according to the needs of the situation - the other is to mix together various methods. Patton (1980) discusses two forms of methodological mixes: triangulation, where a series of different methods are used in the evaluation of a particular programme, and the combination of parts from different methodologies. Quoting Norman Denzin, he gives four forms of triangulation - "(1) data triangulation - the use of a variety of data sources in a study; (2) investigator triangulation - the use of several different researchers or evaluators; (3) theory triangulation - the use of multiple perspectives to interpret a single set of data; and (4) methodological triangulation - the use of multiple methods to study a single problem or program" (p.108-9). However, as he points out, triangulation can be very expensive, and may even mean "a series of poorly implemented methods rather than one approach well executed" (p.109). He also discusses the mixture of different elements from different methodologies (as his consideration is qualitative v quantitative, specifically the combination of data types, analysis and research design from these approaches).
Go to the Evaluation of Cooperative Systems home page