International peacekeeping is at the top of the agenda of the United Nations and many national governments, with the end of the Soviet-U.S. rivalry apparently ushering in a "new world order." Yet, despite the increasing resort to peacekeeping, there is little systematic understanding of its appropriate application. It is particularly important to understand what these missions are designed to accomplish and to determine the extent to which they have in fact accomplished these goals. Progress depends on advances in conceptualization and methodology as well as the relations among theory, policy, and practice. Yet, analysts disagree about how to think about peacekeeping and how its impacts should be evaluated (see Johansen, 1994).
As a contribution to the debate on evaluating peacekeeping missions, the National Research Council's Committee on International Conflict Resolution asked five analysts to address a common set of questions. Each analyst was chosen for his or her published contributions to the topic. They consist of Paul F. Diehl of the Department of Political Science at the University of Illinois (see Diehl, 1993), William J. Durch of the Henry L. Stimson Center in Washington, D.C. (see Durch, 1996), A.B. Fetherston of the Department of Peace Studies at the University of Bradford (see Fetherston, 1994), Robert C. Johansen of the Joan B, Kroc Institute for International Peace Studies at the University of Notre Dame( see Johansen, 1994), and Steven R. Ratner of the School of Law at the University of Texas (see Ratner, 1995).
The result of the exercise was a lively exchange of views about criteria for defining success, indicators, time frames for evaluating impacts, and the way that academic work can be used in the policy process. These views are summarized in the form of responses to five questions asked by us. Following the exchange, we highlight the key themes and issues on which both agreements and disagreements occurred. We conclude with some implications of these themes for further research and analysis.
Diehl offers a number of approaches for evaluating a mission's success. One is whether the purpose, as stated in the mandate, was fulfilled. This criterion is useful to the extent that the mandate is clear but clarity itself is often used also as a criterion of success. The specific accomplishments of the mission such as the number of people fed, disasters avoided, and cease-fires achieved is useful but leaves open the question of just how many accomplishments are needed to qualify as successful. A third criterion is the impact of the operation on the local population. Important as this criterion is, it begs the question "compared to what," and ignores the impacts on broader political processes that may have larger impacts on the lives of the local population. Rather than focus on what has been accomplished, another approach examines the manner in which it has been achieved. While placing a premium on efficiency, this criterion may favor the interests of particular constituencies or organizations over the broader goals as defined, for example, by the United Nations. More broadly, an analyst can examine the extent to which an operation contributes to conflict avoidance or resolution. Although these criteria provide a basis for comparison across operations, they may place too large a burden on the peacekeepers, obscuring other positive benefits provided by the operation.
Distinguishing between general and mission-specific criteria, Durch agrees with three of Diehl's criteria. Most generally, he argues for examining missions in terms of whether they contribute to containing conflict and its underlying causes. More specifically, he asks whether the operation fulfilled its tasks as set forth in the mandate and whether it was effectively planned and carried out. The former is of particular concern to those constituencies that sponsor the mission. The latter is of interest to those organizations who must carry out the tasks.
While agreeing with most of the criteria set forth by Diehl and Durch, Ratner distinguishes among them in terms of their usefulness. He promotes evaluating impacts on the host states and on the implementing organization such as the UN. For the host states, it is important to take into account the time horizon for impacts. For the implementing organization, it is important to consider opportunity costs. Evaluating missions in terms of their mandates or in terms of accomplishments of other operations has drawbacks. The former tends to ignore the influence of exogenous (external) factors on the ability of a mission to achieve its goals. The latter ignores the variety of mandates assigned to different operations. For example, first-generation missions aimed at truce supervision or impasse management have simpler goals that the second-generation missions designed to implement political settlements.
These commentators also highlight the difficulties of using "counter-factuals" as a basis for comparison. Since we cannot know what would have happened if the operation were not carried out, it is problematic to rely on such "what..if" evaluations. Although thought-experiments of this type may stimulate productive discussions among analysts and policy-makers, they should not be used as a basis for judging success or for generating policy options.
Johansen and Fetherston view the issue differently. They stress the contribution of peacekeeping to larger values rather than to self-serving gains. For Johansen these values are world peace, justice, and the reduction of human suffering. He regards such instrumental criteria as efficiency and task accomplishments in terms of their contribution to reducing conflict and achieving peace. For Fetherston these values refer to the needs of people who live in the war-torn societies. She is critical of criteria that serve the interests of national governments and their militaries, rather than the needs and interests of the societies in conflict. Both of these commentators favor criteria that assess peacekeeping in terms of its contribution to positive peace or to combating cultures of violence and repressive civil or state structures.
The commentators are divided on this issue. Diehl identifies two differences between the way that scholars and practitioners view success. Whereas many practitioners prefer criteria that are unique to specific missions, scholars tend to prefer broad criteria that apply to a wide variety of missions in order to develop generalizations. Another difference is one of focus. Many practitioners examine outcomes at a micro-level in terms of the military operation itself. Scholars, on the other hand, tend to focus on broader political goals and "overall" success at a macro level of analysis. Ratner also discerns differences between practitioners and scholars. The former emphasize the extent to which a mission accomplished its mandate as well as its short-term impact upon the country. While being concerned also with casualties, practitioners are less preoccupied with them than are their political leaders. Scholars are more inclined to emphasize factors out of the peacekeepers' control, exogenous to the mission, and be more concerned with long-term consequences.
The other commentators view the issue differently. Although he comments on divisions between practitioner and scholar perspectives, Durch is more concerned with possible contributions that scholars can make to practice. He offers help to practitioners by asking scholars to identify the various conditions that may influence mission effectiveness, as a kind of check list of factors that should be considered before committing resources to a mission. Johansen and Fetherston question the usefulness of distinguishing between scholars and practitioners. Taking a more prescriptive approach, they encourage a more collaborative approach in which agreed definitions emerge that serve common goals of more-peaceful societies. Fetherston is particularly concerning with developing conceptualizations that unite theory and practice (see also Fetherston, 1994). Johansen is concerned with developing a process that results in consensual understandings of criteria that serve the larger, less self-serving, goal of world peace.
Ratner argues that there is a need for a comprehensive empirical study of all types of operations to yield insights into the relative importance of external versus internal factors as influences on success. Before such a study is done, however, Diehl and Johansen call for the inclusion and definition of certain variables largely overlooked by researchers to date as well as clarification for some research design issues. For Diehl these include clarification about whose perspective peacekeeping success is to be judged, the relation between indicators of macro-level conflict resolution and the performance of the peacekeeping forces, opportunity and other costs of missions, a standard or baseline against which to judge mission success, and a clearer definition of the contextual factors that define the environment in which peacekeepers operate. Johansen seeks clarification for three issues that he suggests should guide the design of a study: What are the social consequences of peacekeeping?, What are its economic consequences?, and What are the ethical considerations of peacekeeping? While these questions suggest a need for reconceptualization, they do not go as far as Fetherston's suggestion that we develop frameworks that enable us to think about peacekeeping in the larger context of peacebuilding and conflict transformation.
The commentators agreed that both short and long-term considerations are important. They also agreed that while long-term effects may be more important, they are also more difficult to evaluate. Diehl argued that short-term concerns should be subordinated to long-term concerns. A difficulty, of course, with long-term criteria is that analysts must delay for a considerable period of time making a judgment, especially with regard to recent or ongoing missions. But, time frame is not the only problem with long-term evaluations. Johansen noted that long-term considerations can be elusive. For example, it is difficult to develop criteria for knowing whether the mission contributed to enhancing the legitimacy of the peacekeepers or to strengthening the international community's norms of peace. Fetherston carried this theme further. She argued that long-term perspectives on peacekeeping must deal with issues of conflict transformation. The difficulty involved in defining what is meant by "transformation" is due to the realization that we are part of the power structures that help to perpetuate those cultures of violence.
Durch and Ratner raised interesting issues about the relation between short and long-term considerations. Durch noted that while reductions in violence or famine may be short-term goals, they may also entail long-term commitments: for example, limiting hunger may create dependency, especially when dealing with displaced populations. Such long-term goals as post-war reconstruction and reconciliation may not be met because nations (or the UN) insist on short timelines, both to limit their costs and to limit international responsibility for rebuilding societies. Ratner made the interesting observation that apparent long-term failures should not be used as an indictment of the mission's success. A return to chaos after a long period of time does not make the earlier mission a failure. For example, the UN's 1960-64 Congo mission accomplished its goal of permitting the Congo to become independent and avoided the secession of Katanga even though we witness today the desperate state of Zaire. Thus, accomplishing short term goals may not portend a peaceful future either because of some implications of the earlier achievements or because of new developments that occur despite the successful earlier interventions.
A difference of opinion among the commentators concerns their preference for research designs that examine relationships among many variables across a large number of cases and approaches that provide a basis for new research designs that may contribute more directly to improved practice, particularly with regard to the people most immediately affected by the violence. A related difference of opinion is between those who aspire to developing models based on rigorous empirical research and those more concerned with the consequences of any research on the lives of people in the conflict zones. Diehl and Ratner were quite explicit about the value of systematic quantitative work. Diehl's research strategy consists of precise evaluations of a variety of factors thought to contribute to success and determining through sensitivity analysis the effects of altering those factors. To the extent that the factors can also be manipulated by decision-makers, the results would contribute to policy assessments and to choices among alternative options. Ratner was less explicit about the details of designing and implementing research but appreciated the value of systematic comparative research for identifying factors that could be manipulated in order to increase a mission's effectiveness.
Johansen offered a large number of research questions that could guide analysis of diverse missions. His questions focus primarily on indicators of success and provides the criteria for evaluating the impacts of peacekeeping operations. He emphasized the impacts of such factors as government officials' obligations to uphold norms of peace, prosecution of war crimes, surveillance technologies, and the level of professionalism among UN forces. Similarly, Fetherston suggested that research should be guided by such questions as: How is peacekeeping related to peacebuilding or conflict transformation?, How, specifically, does peacekeeping facilitate or inhibit long term transformative possibilities? She expressed skepticism about research designed to identify manipulable factors shown to have impacts on mission success. Rather, she stressed the importance of reconceptualizing the meaning of peacekeeping in terms of implications for social change and issues of power and cultural violence. Above all, research is useful to the extent that it is grounded in the experiences and work of the people most immediately affected by continuing violence.
For Diehl and Ratner, the goal is to use the results of empirical research to construct models that can be used by policy-makers to improve the practice. For Johansen and Fetherston, the goal is to conduct research that contributes directly to an improvement in peoples'lives, and this is most likely to take the form of thinking through the implications of alternative social structures consonant with strengthening the norms of peace. Although both sets of commentators agree that research, to be useful, must contribute to policy, they disagree about how policy is defined, who implements it, and how it is carried out.
Differences of opinion among the commentators are found on a number of evaluation issues. However, we also find agreement among them on several themes. In this section, we highlight the key themes and issues.
1. There are many types of peacekeeping activities. While some of the commentators argue in favor of developing universal criteria for comparing missions, they also agree that there are different criteria for success depending on the goal of the mission. The issue is how to compare different missions while taking into account the unique characteristics of each of the missions being compared: Is it possible to develop a general theory of peacekeeping without overlooking those features of missions which make them unique?
2. It is important to define reasonable expectations for peacekeeping. Different opinions were expressed on the larger objectives for peacekeeping. The more ambitious the goal -- such as contributing to norms of positive peace -- the more likely the mission will be regarded as a failure. It is important to recognize that missions are influenced by factors outside the control of the peacekeepers. Included among these factors are international events, rules imposed on the peacekeepers' behavior, resources available for the operation, and limits determined by national policies.
3. Different actors and constituencies have different objectives and different criteria for evaluating success. Military peacekeepers concerned with the efficiency of an operation have different interests than the local citizens, the NGOs involved in the area, or the larger international community. The question of whose objectives should prevail is a matter of values about which the commentators disagree. It should be possible, however, to treat this as an empirical issue where a mission can be judged against any objectives independently of whether an analyst values that objective.
4. Both short and long-term objectives are important, although the latter are more difficult to assess. While differing on the relative importance of short and long-term impacts, the commentators agree that these are related. For example, reduced violence may enable the combatants to discuss the deeper issues that divide them leading eventually to a resolution of the conflict. On the other hand, while producing desirable short-term effects, prolonged peacekeeping may lead to the militarization of a society which would prevent the parties from addressing the underlying sources of their conflict. Further understanding of these connections would contribute to the way we think about success.
5. A major analytical challenge is to devise a baseline against which to judge a mission. Both qualitative and quantitative approaches confront the question, "success compared to what?" It is difficult to decide on a proper standard, and some present serious methodological problems. Possibilities include judging a mission against the conditions that existed before the intervention occurred, against an ideal state that the sponsors hope to achieve, against other similar missions (including earlier interventions in the same region), and against an estimate of what would have happened with no mission or with an alternative type of intervention. Before-after longitudinal comparisons without control groups prevent analysts from attributing any changes to the intervention per se. Cross-sectional comparisons of different missions prevent analysts from isolating the specific conditions responsible for the outcomes; there are too many dimensions on which the missions differ. And, "what-if" counterfactual analyses suffer from hindsight biases that distort causal attributions. Until progress is made on overcoming these problems, evaluations will not be regarded as being definitive.
6. The difficulties in evaluating peacekeeping missions are both conceptual and methodological. Although the commentators disagree on the emphasis to be placed on developing indicators versus re-thinking the meaning of peacekeeping, most agree that both analysis and conceptualizing need to be done. In addition to the design and measurement problems discussed above, more thought should be given to relationships between micro and macro-level variables, to the connections between short and long-term impacts, to the contributions of academic research to policy, and to the differences between normative approaches in a peace studies tradition and descriptive-empirical approaches to evaluation that reflect a positive tradition of scholarship.
The many issues discussed above makes evident the complexity of the task. Conceptual issues include the considerable variety of types of missions and constituencies for those missions as well as the extent to which normative-policy concerns must be part of an evaluation design. Methodological issues include the development of appropriate baselines for comparison and of both short and long-term indicators as well as the way that endogenous (under the control of peacekeepers) and exogenous (out of the peacekeepers' control) influences are taken into account in an analysis. Further progress depends on developing a broad conceptual framework that can guide evaluations. It is unlikely, however, that any framework can be constructed without reflecting the orientations of its developers. These orientations concern approaches taken to the following issues:
Given the diversity of opinions expressed by the commentators on these issues, it is unlikely that all evaluations will be guided by the same analytical framework. It should be possible, however, to use the different approaches to evaluation in a complementary fashion. By emphasizing different aspects of the process, and different time horizons for evaluating success, analysts can contribute to an improved understanding of the diverse challenges and opportunities of peacekeeping.
Bercovitch, J., J. T. Anagnoson, and D. J. Willie. 1991. “Some Conceptual Issues and Empirical Trends in the Study of International Mediation in International Relations.” Journal of Peace Research, Vol. 28, pp. 7-17.
Diehl, P.F. 1993. International Peacekeeping. Baltimore: The Johns Hopkins University Press.
Diehl, P. F., D. Druckman, and J. Wall. 1998. “International Peacekeeping and Conflict Resolution: A Taxonomic Analysis with Implications.” Journal of Conflict Resolution, Vol. 42, pp. 33-55.
Durch, W. J. 1996. UN Peacekeeping, American Politics, and the Uncivil Wars of the 1990s. New York: St. Martin's Press.
Evans, G. 1993. Cooperating for Peace: The Global Agenda for the 1990s and Beyond. St. -Leonards, New South Wales: Allen and Unwin.
Fetherston, A. B. 1995. “Habitus UN Cooperating for Peace: A Critique of Peacekeeping.” In S. Lawson, ed., The New Agenda for Global Security: Cooperating for Peace and Beyond. St. Leonards, New South Wales: Allen and Unwin.
Fetherston, A.B. 1994. Towards a Theory of United Nations Peacekeeping. New York: St. Martin's Press.
Fetherston, A. B. and C. Nordstrom. 1995. “Overcoming Habitus in Conflict Management: UN Peacekeeping and Warzone Ethnograpy.” Peace and Change, Vol. 20, pp. 96-119.
Gurr, T. R. 1993. Minorities at Risk: A Global View of Ethnopolitical Conflict. Washington, DC: United States Institute of Peace.
Johansen, R.C. 1994. "U.N. peacekeeping: How should we measure success?" Mershon International Studies Review 38: 307-310.
Lederach, J. P. 1995. Preparing for Peace: Conflict Transformation Across Cultures. Syracuse, NY: Syracuse University Press.
Mack, A. 1975. “Numbers Are Not Enough: A Critique of Internal/External Conflict Behavior Research. Comparative Politics, Vol. 7, pp. 597-618.
Neack, L. 1995. “UN peacekeeping: In the Interest of Community or Self?” Journal of Peace Research, Vol. 32, pp. 181-196.
Ratner, S.R. 1995. The New UN Peacekeeping: Building Peace in Lands of Conflict After the Civil War. New York: St. Martin's Press.
Rupesinghe, K. 1995. Conflict Transformation. London: Macmillan.
Urquhart, B. 1987. “United Nations Peacekeeping Operations and How Their Role Might Be Enhanced.” In UNITAR, ed., The United Nations and the Maintenance of International Peace and Security. Lancaster: Martinus Nijhoff.
Urquhart, B. 1990. “Beyond the Sheriff’s Posse.”
Survival, Vol. 32, pp. 196-205.