Evaluation Models

New Directions for Evaluation - Tập 2001 Số 89 - Trang 7-98 - 2001
Daniel L. Stufflebeam1
1Western Michigan University

Tóm tắt

AbstractIn entering a new millennium, it is a good time for evaluators to critically appraise their program evaluation approaches and decide which ones are most worthy of continued application and further development. It is equally important to decide which approaches are best abandoned. In this spirit, this monograph identifies and assesses twenty‐two approaches often employed to evaluate programs. These approaches, in varying degrees, are unique and cover most program evaluation efforts. Two of the approaches, reflecting the political realities of evaluation, are often used illegitimately to falsely characterize a program's value and are labeled pseudo‐evaluations. The remaining twenty approaches are typically used legitimately to judge programs and are divided into questions/methods‐oriented approaches, improvement/accountability approaches, and social agenda/advocacy approaches. The best and most applicable of the program evaluation approaches appear to be Client‐Centered/Responsive, Utilization‐Focused, Decision/Accountability, Consumer‐Oriented, Constructivist, Case Study, Outcome/Value‐Added Assessment, and Accreditation, with the new Deliberative Democratic approach showing promise. The approaches judged indefensible or least useful were Politically Controlled, Public Relations, Accountability (especially payment by results), Clarification Hearing, and Program Theory‐Based. The rest including Objectives‐Based, Experimental Studies, Management Information Systems, Criticism and Connoisseurship, Mixed Methods, Benefit‐Cost analysis, Performance Testing, and Objective Testing Programs were judged to have restricted though beneficial use in program evaluation. All legitimate approaches are enhanced when keyed to and assessed against professional standards for evaluations.

Từ khóa


Tài liệu tham khảo

Aguaro R., 1990, R. Deming: The American who taught the Japanese about quality

Alkin M. C., 1969, Evaluation theory development, Evaluation Comment, 2, 2

Alkin M. C.(1995 November).Lessons learned about evaluation use. Panel presentation at the International Evaluation Conference American Evaluation Association Vancouver British Columbia.

10.1037/0003-066X.48.12.1210

Bandura A., 1977, Social learning theory

Bayless D., 1992, Quality improvement in education today and the future: Adapting W. Edwards Deming's quality improvement principles and methods to education

10.1177/109019817400200407

10.1016/S0191-491X(98)00019-4

Bickman L., 1990, Advances in Program Theory. New Directions in Program Evaluation

Bloom B. S., 1956, Taxonomy of educational objectives: Handbook I: Cognitive domain

10.1016/0886-1633(94)90021-3

10.1177/001041407500800204

Campbell D. T., 1988, Methodology and epistemology for social science: Selected papers

Campbell D. T., 1963, Handbook of research on training

Chelimsky E., 1987, What have we learned about the politics of evaluation?, Evaluation Practice, 5, 8

Chen H., 1990, Theory driven evaluations

Clancy T., 1999, Every man a tiger

1966 U.S. Office of Education Washington DC D. L. Cook Program evaluation and review techniques applications in education.

Cook T. D., 1979, Qualitative and quantitative methods in evaluation research

Cousins J. B., 1992, The case for participatory evaluation, Educational Evaluation and Policy Analysis, 14

Cronbach L. J., 1963, Course improvement through evaluation, Teachers College Record, 64

Cronbach L. J., 1982, Designing evaluations of educational and social programs.

Cronbach L. J., 1980, Toward reform of program evaluation

Cronbach L. J., 1969, Individual differences in learning ability as a function of instructional variables

Davis H. R., 1975, Handbook of evaluation research, Vol. 1.

Deming W. E., 1986, Out of the crisis

Denny T., 1978, Story telling and educational understanding. Occasional Paper No. 12

Ebel R. L., 1965, Measuring educational achievement

Eisner E. W.(1975 March).The perceptive eye: Toward a reformation of educational evaluation. Invited address Division B Curriculum and Objectives American Educational Research Association Washington DC.

Eisner E. W., 1983, Evaluation models

Ferguson R., 1999, Ideological marketing, The Education Industry Report

Fetterman D. M., 1984, Ethnography in educational evaluation

Fetterman D., 1994, Empowerment evaluation, Evaluation Practice, 15

Fisher R.A., 1951, The design of experiments (6th ed.)

10.1037/h0053800

10.1126/science.32.810.41

Flinders D. J., 2000, Evaluation models

Glaser B. G., 1967, The discovery of grounded theory

10.3102/0013189X004003009

Glass G. V., 1968, Analysis of time‐series quasi‐experiments. (U.S. Office of Education Report No. 6‐8329.)

Greene J. C., 1988, Communication of results and utilization in participatory program evaluation, Evaluation and Program Planning, 11

Green L. W., 1991, Health promotion planning: An educational and environmental approach, (2nd ed.), 22

Guba E. G., 1969, The failure of educational evaluation, Educational Technology, 9, 29

Guba E. G.(1977).Educational evaluation: The state of the art. Keynote address at the annual meeting of the Evaluation Network St. Louis.

Guba E. G., 1978, CSE Monograph Series in Evaluation

Guba E. G., 1990, The paradigm dialog

Guba E. G., 1981, Effective evaluation

Guba E. G., 1989, Fourth generation evaluation

10.1007/978-94-017-1988-9

Hammond R. L., 1972, Evaluation at the local level

Hastings T.(1976).A portrayal of the changing evaluation scene. Keynote speech at the annual meeting of the Evaluation Network St. Louis.

10.1207/s15326977ea0103_2

House E. R., 1973, School evaluation: The politics and process

House E. R., 1980, Evaluating with validity

House E. R., 1983, Evaluation models

House E. R., 1998, Deliberative democratic evaluation in practice

10.1002/ev.1157

House E. R., 2000, Evaluation models

10.1177/109019818401100101

Joint Committee on Standards for Educational Evaluation, 1981, Standards for evaluations of educational programs, projects, and materials

Joint Committee on Standards for Educational Evaluation, 1988, The personnel evaluation standards: How to assess systems for evaluating educators

Joint Committee on Standards for Educational Evaluation, 1994, The program evaluation standards: How to assess evaluations of educational programs

Kaplan A., 1964, The conduct of inquiry

Karlsson O., 1998, Scandinavian perspectives on the evaluator's role in informing social policy. New Directions for Evaluation, 77, 21

Kaufman R. A., 1969, Toward educational system planning: Alice in educationland, Audiovisual Instructor, 14, 47

Kee J. E., 1995, Handbook of practical program evaluation, 456

Kentucky Department of Education, 1993, Kentucky results information system, 1991–92 technical report

Kidder L., 1987, Qualitative and quantitative methods: When stories converge. Multiple methods in program evaluation, New Directions for Program Evaluation, 35

Kirst M. W., 1990, Accountability: Implications for state and local policymakers. U.S. Department of Education ‐ Policy Perspectives Series

Koretz D., 1996, Improving the performance of America's schools, 171

Koretz D. M., 1998, The validity of gains in scores on the Kentucky Instructional Results Information System (KIRIS)

Lessinger L. M., 1970, Every kid a winner: Accountability in education

Levin H. M., 1983, Cost‐effectiveness: A primer. New Perspectives in Evaluation 4

Levine M., 1974, Scientific method and the adversary model, American Psychologist, 666

10.1016/0147-1767(85)90062-8

Lindquist E. F., 1951, Educational measurement

Lindquist E. F., 1953, Design and analysis of experiments in psychology and education

10.3102/0013189X020008015

Lord F. M., 1968, Statistical theories of mental test scores

MacDonald B., 1975, Evaluation: The state of the art

10.1007/978-94-009-6675-8

10.1007/978-94-009-2679-0

10.1111/j.1745-3992.1992.tb00220.x

10.3102/0013189X023002013

Metfessel N. S., 1967, A paradigm involving multiple criterion measures for the evaluation of the effectiveness of school programs, Educational and Psychological Measurement, 27, 931, 10.1177/001316446702700441

Miron G., 1998, Education reform in the south in the 1990s

10.1016/0277-9536(87)90291-7

National Science Foundation, 1997, User‐friendly handbook for mixed method evaluations. NSF 97‐153

Nave B., 2000, Evaluation models

10.1016/0886-1633(93)90033-L

Owens T., 1973, School evaluation: The politics and process

Parlett M., 1972, Evaluation as illumination: A new approach to the study of innovatory programs

Patton M. Q., 1980, Qualitative evaluation methods

Patton M. Q., 1982, Practical evaluation

Patton M. Q., 1990, Qualitative evaluation and research methods, (2nd ed.)

10.1016/0886-1633(94)90026-4

Patton M. Q., 1997, Utilization‐focused evaluation: The new century text (3rd ed.)

Patton M. Q., 2000, Evaluation models

Peters T. J., 1982, In search of excellence

10.1177/001139292040001004

Popham W. J., 1969, Instructional objectives.

Popham W. J., 1983, Evaluation models

Prochaska J. O., 1992, Progress in behavior modification, 28

Provus M. N., 1969, Discrepancy evaluation model

Provus M. N., 1971, Discrepancy evaluation

Rippey R. M., 1973, Studies in transactional evaluation

Rogers P. R., 2000, Evaluation models

Rossi P. H., 1993, Evaluation: A systematic approach (5th ed.)

Sanders J. R., 1992, Evaluating school programs

Sanders W. L., 1989, Using customized standardized tests. ERIC Digest No. ED 314429(Contract No. R‐88‐062003)

10.1007/BF00973726

Schatzman L., 1973, Field research

Schwandt T. A.(1984).An examination of alternative models for socio‐behavioral inquiry. Unpublished Ph.D. dissertation Indiana University.

10.3102/0013189X018008011

Scriven M. S., 1967, Curriculum evaluation. AERA Monograph Series on Curriculum Evaluation (Vol. 1)

Scriven M., 1974, Evaluation in education: Current applications

Scriven M., 1991, Evaluation thesaurus

Scriven M., 1993, Hard‐won lessons in program evaluation, New Directions for Program Evaluation, 58

10.1016/S0191-491X(00)80010-3

10.1016/0886-1633(94)90031-0

10.1016/0886-1633(94)90059-0

Shadish W. R., 1991, Foundations of program evaluation

Smith L. M., 1974, Four examples: Economic, anthropological, narrative, and portrayal

10.1002/ev.1425

10.1007/978-94-015-7827-1

10.1016/0149-7189(87)90002-4

Stake R. E., 1967, The countenance of educational evaluation, Teachers College Record, 68, 523, 10.1177/016146816706800707

Stake R. E., 1971, Measuring what learners learn (mimeograph)

10.1111/j.1745-3984.1966.tb00857.x

Tyler R. W., 1932, Service studies in higher education

Tymms P.(1995).Setting up a national "value‐added" system for primary education in England: Problems and possibilities. Paper presented at the National Evaluation Institute Kalamazoo MI.

Vallance E.(1973).Aesthetic criticism and curriculum description. Ph.D. dissertation Stanford University.

Webster W. J.(1975 March). The organization and functions of research evaluation in a large urban school district. Paper presented at the annual meeting of the American Educational Research Association Washington DC. (ERIC Clearinghouse on Tests Measurements and Evaluation. ED 106 345)

10.1016/0191-491X(95)00012-J

10.1016/S0191-491X(00)80009-7

Weiss C. H., 1972, Evaluation

Weiss C. H., 1995, New approaches to evaluating community initiatives

Whitmore E., 1998, Understanding and practicing participatory evaluation, New Directions for Evaluation, 80

Wholey J. S., 1995, Handbook of practical program evaluation, 15

Wiggins G., 1989, A true test: Toward more authentic and equitable assessment, Phi Delta Kappan, 70, 703

10.1086/442819

Wolf R. L., 1975, Trial by jury: A new evaluation method, Phi Delta Kappan, 57, 185

Worthen B. R., 1987, Educational evaluation: Alternative approaches and practical guidelines

Worthen B. R., 1997, Program evaluation (2nd Ed.)

10.1177/001139292040001009