Choosing Priorities for Evaluation
While reading over the Caserta Conference papers, I got to thinking about the problem of picking evaluation targets, given the ever shifting nature of public programs. I had the idea that it might be useful to categorize programs as in Figure 1 to help set priorities. I find it useful personally, and I'd appreciate comments to help me develop it further. My notion is that any given program can be rated on the likelihood of shifts in program goals over time and program structure over time. By placing these ratings on the same graph we get a sense of where evaluation makes sense. "A" will show results in a short period of time and not change much, so its an ideal candidate for evaluation. "E" is probably a good choice as well. "B" is more a policy evaluation than a program evaluation because while the "program" (in terms of structure) will change, and it will take a long time to observe impact, the goals remain constant. Its probably worth looking at, but the methodologies have to change because analysis of long term trends becomes critical. I suppose "D" might be worth evaluating, but it has too much variation (even with short term impact) for my tastes. "C" should not only not be evaluated, but should probably be junked as a program. If I really wanted to get fancy, I'd then take the good evaluation candidates and rank them in terms of their importance to the public good, and then set evaluation priorities. As far as I am concerned the real value of this system is to get rid of the candidates where evaluation will be a waste. If that alone could be done, all other value to the system is gravy.
|