This book presents a plan for developing evaluation into a new form of applied social research which will be methodologically sound, relevant to the problems of society, and built on a technological (as opposed to a scientific) model. It is a plan which I believe can be achieved, will have general utility and will insure the place of evaluation in the world of social research. Many efforts have already been made in that direction. This book is an effort to assess where those efforts have brought us, to specify the options which are open, and to suggest directions which should be chosen. I firmly believe that the model presented here will have wide utility; but above all, this book is a personal statement of what evaluation means to me.
I have tried to make that statement clear, coherent, and consistent. In that effort, I know that I have failed. As I developed this book, I found strains that I could not separate one from the other, and elements which, despite my best efforts, defied clear categorization or explanation. Further, my beliefs and attitudes about many aspects of evaluation were often ambivalent, and try as I might, that ambivalence could not be eliminated from my consciousness. Thus much of that ambivalence is incorporated in the text, with the result that at times some parts of the work may seem contradictory to others. Still, writing this book has helped me immensely in understanding what evaluation is, what it might be, and how it should performed. I hope that it will help others as well.
The work was a developmental process. When I started to write, I was sure that the entire structure and the main points of the work were firmly ensconced in my mind. How wrong I was. How different this book is from the one I thought I was going to write. The book is an attempt to conceptualize evaluation as something special-as something different-from other aspects of social research. In order to develop that conceptualization, I had to bring to bear many diverse fields. Research methodology. Epistemology. Sociology. Psychology. Ethics. And many more. The number of pieces needed to construct the model were considerably greater than what I had originally anticipated, and I found myself in constant debate with myself and with others concerning what was important, how elements fit together, and what lessons could be drawn from other fields about evaluation. Each time a new element or dimension was drawn into consideration, I had to consider its influence on the entire line of reasoning in the book. Each new element involved a consideration of what went before, and what was to come. Small wonder that the final product was different from what I had originally intended. The sections on the ethics of evaluation, the professionalization of evaluation and the differences between science and technology were unforeseen when I first conceptualized the book, and all these areas had profound influence on my thinking about what evaluation is and what it might become.
I have tried to organize the book in a way that clearly presents the various themes and foci that exist in evaluation. In order to do this, I had to draw distinctions in a sharper light than is actually the case. Many distinctions and categories were developed for the sake of explanation and better understanding. I hope the exaggerations which were made for the sake of explanation will be useful, and that the distortions introduced by the exaggerations will be forgiven. I also hope that the redundancy which is sometimes apparent will be forgiven. I have tried to cut the repetition to a minimum, but by making some forced categorizations, I found it necessary on occasion to repeat myself.
The book could have been organized in an infinite variety of ways, and I had no lack of suggestions as to how it should be done. Each method of organization and presentation made some arguments easier to follow, and some more difficult. By juxtaposing certain elements, I had to separate others, and the choice was largely a matter of preference, style, and a sense of what I thought were the most important aspects of the work. Many disagreed with those choices, and argued that other elements should have been emphasized or that a different organization should have been followed. As an example, I could have taken a particular aspect of methodology, and shown how it operated across each type of evaluation. In so doing, major section headings would have dealt with methodology, and minor headings with each of the three types of evaluation. I chose not to do this, as I wanted to make it clear that there are three distinct types of evaluation and that although they share some methodological difficulties, the emphasis of methodological concern does in fact differ for each. I am grateful to those who advanced different opinions about the book's organization as those suggestions helped me to understand what I really wanted to do with the book.
I believe that program evaluation is merely the latest fad in social research, and that as with all popular ideas, it will fade from prominence. But I also believe that as a distinct form of social research, evaluation has something enduring to offer, and that in one guise or another, it is likely to be important for some time. More accurately, I believe that we can make evaluation into a pursuit which will have enduring interest and value, but that it is entirely possible to develop the field in ways which will insure its quick oblivion. Evaluation is a unique type of social research which is applied, relevant and technological. If these attributes are developed and exploited, evaluation will become an important strategy for helping to solve social problems. If these attributes are not developed, evaluation will be swallowed up by the trends to come, reabsorbed by its parent disciplines, and fade from existence as a distinct field or discipline. This would be a shame, as evaluation does in fact have something to contribute to our search for a better life for all.
This is a short book. Deliberately so. I tried not to repeat information which was well developed by others. Rather, I settled for as cogent a summary as I could write about those already developed ideas, and added as complete a reference list as I could. I did this for several reasons. First, others can explain various concepts far better than I. Second, the book draws on so many areas that a complete treatment of even a few of them would have made this book unwieldy. Third, I wanted this book to be about evaluation, and not about survey research, quasi-experimental designs, the dynamics of organizational change, or any one of the myriad concepts and traditions which bear on the conduct of program evaluation. Finally, I prefer to read short books. As a consequence of making this book short, it will not do as a self-contained text or handbook on the field of program evaluation. Readers will have to draw on numerous other works when they actually attempt to do an evaluation. This is not necessarily a loss, as the need for multiple sources is probably a necessity in any case. I do hope that the book will help people understand how evaluation is different from its historical antecedents, and how to bring specialized knowledge from many areas to bear on the problem of intelligently evaluating social programs.
This book is influenced by my interests and background, and readers are entitled to know what those influences are. In thinking about what I have written, I have been able to identify nine relatively distinct intellectual elements which have gone into the making of this book. First, I was trained within a psychology department as a social psychologist. Hence the psychological nature of examples used in this book. Second, I am involved in evaluation training, both as Associate Director of the Evaluation Training Program at Hahnemann, and through my work with the Training Committee of the Evaluation Research Society. Third, I have always been intrigued with the types of explanations which can be derived from sociological levels of analysis. Fourth, I feel strongly that I could not understand methodology or research unless I grasped a basic philosophical sense of what it means to discover and to know. Hence my interest in the philosophy of science. Fifth, I believe that a special responsibility is implied in accepting the role of a social researcher. That responsibility derives from the power of the researcher's special expertise, and from the societal implications of conducting social research. Sixth, my experience as editor of Evaluation and Program Planning has given me an opportunity to know what is going on in the field of evaluation, to understand the trends which are developing, and to form opinions about the implications of those trends. Those opinions, however, are formed by the type of articles which are submitted to Evaluation and Program Planning and may not be representative of all aspects of evaluation. Seventh, I have conducted a lot of evaluation and research in the areas of mental health, drug abuse, and education. My experiences in those efforts have profoundly shaped my sense of what can be done and what should be done. Eighth, I have always had a passion for categorizing events and trying to see the implications of the interactions among a wide variety of diverse ideas. Hence the emphasis in this book is on fitting a wide diversity on concepts into neat categories. I hope my zealousness in that regard has not been overbearing. Finally, my most basic interest is in the methodology of social research. I hope this book adequately conveys my strong belief that if evaluation does not have methodological credibility, it has nothing. I recite these details in order to convey a sense of what this book is about, what it is trying to do, and what biases the reader may expect. I have been assured by many critics of earlier drafts that the book has wide utility, but it is most certainly biased, and for that I make no apologies.
| 193 |