Internal Evaluation: A Synthesis of Traditional Methods and Industrial Engineering

Morell, J.A. American Journal of Evaluation #1 2000 Pages 41-52

Summary

In evaluation as we have been practicing it there is a tension between the rigor of method and the extent of involvement in evaluation by a wide variety of program staff. The reason for this relationship is that evaluation often employs methodologies whose use requires specialized training. Thus to insure the use of their results, evaluators must devote considerable resources to the task of knowledge transfer. The knowledge-use challenge is greatly lessened when evaluation is planned and carried out by the staff who work in the programs being evaluated. Such evaluation however, will be done by people who are not schooled in evaluation methodology, and thus will be relatively weak from a methodological point of view. There is a way to remove the tension between “rigor” and “use” by applying certain industrial engineering (IE) tools which, though sophisticated and powerful, are packaged in a manner that allows use by people with relatively little specialized training. There are two levels of use for IE tools in evaluation. The first level is additive. It simply requires that IE tools be added to the evaluator’s toolbox. The second level employs the methodological abilities of evaluators to adapt and improve IE tools to make them even more useful. Much effort is required to effect a workable integration of traditional evaluation and IE. Problems to be overcome include inserting the new evaluation technology into program settings, training and education, and developing a different kind of relationship between professional evaluators and those who work in settings where evaluation is takes place.