Participatory assessment : tools for empowering , learning and legitimating ?

Since the 1960s, a large number of participatory assessment tools and methods have been developed for use in a wide variety of policy venues and fields. There are many opinions on what participatory tools are about. As will be explained, these relate in large part to ongoing debates about the goals of participation. Hence, there is no shared authoritative definition of participatory tools and this chapter has no intention of developing one. Rather pragmatically, it distinguishes between participatory methods, which refer to procedures, and participatory tools, which relate to steps in a procedure. Just as an authoritative definition of participatory assessment tools and methods is lacking, so too is consensus over the outcome they aim at. What they have in common and what makes them distinct from other (social) science methods and tools is that they assist in bringing people together at a specific location (which could include the Internet) and facilitate some sort of joint assessment (Hisschemöller 2005). Hence, the distinctive features of participatory methods and tools are that they facilitate dialogue as a way to come to grips with complex (unstructured) decision problems that cannot be addressed by scientific expertise alone. Given this definition, participatory tools overlap with some of the other policy formulation tools that also employ stakeholder involvement (for example, participatory modelling or participatory multi-criteria analysis (MCA)). Participatory assessment needs to be distinguished from legal procedures for political participation that are mandatory in many countries and sometimes also prescribed by international law. Its use is broadly recommended and facilitated by international organizations, for example the World Bank (1996), UNHCR (2006) and the World Food Programme (2001). Participatory assessment tools and methods are used to assist mandatory


INTRODUCTION
Since the 1960s, a large number of participatory assessment tools and methods have been developed for use in a wide variety of policy venues and fields.There are many opinions on what participatory tools are about.As will be explained, these relate in large part to ongoing debates about the goals of participation.Hence, there is no shared authoritative definition of participatory tools and this chapter has no intention of developing one.Rather pragmatically, it distinguishes between participatory methods, which refer to procedures, and participatory tools, which relate to steps in a procedure.Just as an authoritative definition of participatory assessment tools and methods is lacking, so too is consensus over the outcome they aim at.What they have in common and what makes them distinct from other (social) science methods and tools is that they assist in bringing people together at a specific location (which could include the Internet) and facilitate some sort of joint assessment (Hisschemöller 2005).Hence, the distinctive features of participatory methods and tools are that they facilitate dialogue as a way to come to grips with complex (unstructured) decision problems that cannot be addressed by scientific expertise alone.Given this definition, participatory tools overlap with some of the other policy formulation tools that also employ stakeholder involvement (for example, participatory modelling or participatory multi-criteria analysis (MCA)).
Participatory assessment needs to be distinguished from legal procedures for political participation that are mandatory in many countries and sometimes also prescribed by international law.Its use is broadly recommended and facilitated by international organizations, for example the World Bank (1996), UNHCR (2006) and the World Food Programme (2001).
Participatory assessment tools and methods are used to assist mandatory fact-finding procedures, such as social or environmental impact assessment, which inform decision makers and the public at large as to the consequences of policy choices.Much EU legislation, including for example the Water Framework Directive, assigns a key role to European citizens in the preparation of policy plans.However, participatory tools themselves are normally (see section 3) not prescribed by law.This chapter cannot provide a complete overview of all participatory assessment tools and methods.It focuses on tools designed for facilitation of actual dialogue in a face-to-face setting.This means that the huge range of computer tools currently available for stakeholder participation is beyond its scope (but on this, see Chapter 5, this volume).This chapter is also unable to cover all venues where participatory tools are applied.Examples reflect the authors' expertise in environmental studies, but our discussion of participatory tools does have relevance well beyond this field.
Section 2 traces the various origins of participatory assessment tools and methods and discusses the basic rationales for participation.Section 3 presents a four-stage model of policy formulation and shows where participatory tools fit in.Section 4 then goes into more detail on methods and tools that are relevant for the four stages of the policy formulation process.Section 5 addresses the practice of participatory assessment.Section 6 then wraps up and concludes.

ORIGINS AND RATIONALES OF PARTICIPATORY ASSESSMENT TOOLS
The emergence and popularity of participatory assessment tools and methods can be related to the rise of social movements since the 1960s, which aimed at democratizing decision making at all levels of society.Participation has been intrinsically linked to the idea of empowering groups who are less able to make themselves heard, enabling them to effectively defend their interests against the powerful.Criticizing mainstream political theories that legitimized distance between the governors and the governed, social scientists increasingly abandoned Schumpeter's (1942Schumpeter's ( , 1976) ) radical notion that citizens are incapable of rational involvement in the political process.Focusing instead on structural disempowerment of the poor, non-white and women, critics took exception to the normativity of the 'pluralist' conception of the policy arena as a market place that, as Berelson et al. put it, 'makes for enough consensus to hold the system together and enough cleavage to make it move' (Berelson et al. 1954, p. 318).Political science witnessed a revival of 'classical' political ideas, of which   Pateman's (1979) discussion of Rousseau's social contract remains an eloquent example to this day.The classical democratic ideal, expressed by the likes of Rousseau and J.S. Mill, sketches a polity where people open-mindedly engage in an enriching process of learning (see Held 1987).Learning is central to Habermas' (1984) famous notion of the ideal speech situation, where persons with different views interact without obstruction by differences in power and influence.Policy scientists inspired by Habermas criticized mainstream 'technocratic' practices in policymaking and policy analysis (see for example , Fischer 1990).What counts in the end for these policy scientists, or at least what should prevail in the context of good governance, is the quality of policy argument (Dunn 1982;Fischer and Forester 1993).
The participatory wave provided fertile ground not only to analyse and theorise, but also to develop and apply tools to facilitate participation.The notion of participation as empowerment, as put forward by Freire (2004), inspired scholars in the field of development studies to create tools known as Participatory Action Research (Fals Borda and Rahman 1991;Hall 2005) and Participatory Rural Appraisal (Chambers 2008; for an overview see Tufte and Mefalopulos 2009).Urban planning experimented with deliberative tools such as, in Germany, Planungszelle and Citizens' Fora (Renn 2004).The 1980s witnessed harsh controversies related to environmental and technological risk, such as the worldwide concerns over nuclear power, hazardous waste and (transboundary) water pollution.The rise of tools such as citizens' and science courts (Kantrowitz 1967;Seley 1983), citizens' juries (Huitema et al. 2007), scientific mediation (Abrams and Primack 1980) and consensus conferences (see for example, Einsiedel et al. 2001) corresponded with this period.The invention and application of participatory tools to help policy officials in 'dealing with an angry public' (Susskind and Field 1996) was also witnessed.Apart from new tools, existing tools were reinvented and/or adjusted, such as focus groups (Merton and Kendall 1946) and brainstorming (Osborn 1963).
Although the conceptual link between participation and learning is echoed among a wide group of policy scientists, it would be incorrect to trace the origins of participatory assessment tools and methods to the participatory ideology exclusively.Before the Cold War, the US Defense establishment recognized the critical importance of avoiding tunnel vision and 'group-think' among decision makers in situations characterized by stress and uncertainty.Tools for simulation and gaming (see Chapter 3, this volume), originally developed in the military and international relations studies, have found wide use in participatory settings (Mayer 1997).Critical notions developed in decision science found their way through many science disciplines, especially that of the 'wicked problem' (Rittel and Webber 1973) and related notions such as 'type 3 error' -solving the 'wrong problem' (Raiffa 1968) -and bounded rationality (Simon 1973).A wicked, ill-structured or unstructured problem is defined in terms of uncertainty or conflict with respect to the (relevance of) knowledge and values at stake (Hisschemöller and Hoppe 2001).Interestingly, for Rittel and his followers the participatory wave was not the starting point but the necessary consequence of so-called second generation design.The appearance of so many 'wicked' problems he considered a good reason to transfer the ways in which the large-scale NASA and military-type technological problems had been approached into civilian or other design areas (Bayazit 2004).Management science also delivered its own contribution to participatory tool development, inspired by notions from decision science and philosophers like Ackoff (1978), Churchman (1967) and Dewey (1932).
At this point we may understand why assessing the specific qualities of participatory tools for policy formulation is far from easy.This is because there is persistent ambiguity in political thought with respect to the moral and practical benefits of participation.Participation, as scholars tend to agree, can serve three purposes: empowerment, learning and legitimization or, in the terminology introduced by Fiorino (1990), normative, substantive and instrumental.The normative view relates to the very concept of democracy, which means rule by the people (the demos) and the idea that every citizen has the right to speak and be heard.Learning relates to the substantive rationale for participation.In this view, participation is a method for knowledge production.The connection between the normative and the substantive has become reflected in statements that 'lay people are experts with respect to their own problems' (Mitroff et al. 1983) or that 'citizens are the best judges of their own interests' (Fiorino 1990, p. 228).For participatory assessment tools and methods this implies that they must be able to incorporate a maximum of diversity (Stirling 2008).Diversity enhances learning, because it helps articulate marginal viewpoints that have more probative value than mainstream thinking (Dunn 1997).Participation as knowledge production is the focus of transdisciplinary research (Funtowicz and Ravetz 1993;Gibbons et al. 1994).Third, legitimization relates to implementation, which in Fiorino's terminology is the instrumental rationale for participation.A decision is likely to be accepted if the process is considered fair, even by those who have lost the struggle over the outcome.
Notwithstanding an inclination among advocates of participation to tie these three features neatly together, they are not by definition compatible.The notion of diversity appears especially problematic.From the instrumental perspective, too much diversity endangers effective and legitimate decisions.Yet, from the perspective of empowerment, (too much) diversity would undermine the unity needed to effectively oppose the powersthat-be.A closer look into the history of political thought reveals that diversity has not consistently or exclusively been an ingredient of democratic theories.Instead, the necessity of (managing a certain amount of) diversity can (also) be traced to political thought of Machiavelli (1970), who is not usually considered a democrat at all.Machiavelli argues that diversity and social conflict are conditions allowing states to adapt to changing realities, safeguarding their people from war and disaster.In a mild way, this argument has been adopted by pluralist theorists as manifest in the 'intelligence of democracy' (Lindblom 1965).

PARTICIPATORY ASSESSMENT AND THE POLICY FORMULATION PROCESS
In typologies of participatory tools (for example, van Asselt and Rijkens-Klomp 2002; Rowe and Frewer 2005), one theme has returned over time, which can probably be best labelled as 'opening-up' versus 'closing down' (Stirling 2008).This theme relates to the critical features of political rationality: differentiation and unification (Diesing 1962).Differentiation relates to problem structuring, that is, collecting as much (contradictory) information as possible on the issue at stake, and therefore requires a maximum degree of participation.Unification relates to choosing an intervention perspective based on at least part of the information available.Policy formulation heuristics normally echo this distinction in that they identify distinct stages.The first stages, namely agenda setting and problem conceptualization, normally show a degree of differentiation, whereas later in the process of policy formulation unification becomes prominent, especially through the ranking of policy alternatives and the final decision.However, neither in reality nor for Diesing (1962) is policy formulation a linear process, because differentiation and unification are in constant tension.Table 2.1 presents a simple four-stage model of policy formulation leading to decision making in the left-hand column and in the right-hand column a four-step model of participatory methodology.We pragmatically assume that both models are compatible, and that each step of the participatory methodology precedes, or provides input to, the related stage of the policy formulation process.
The decision heuristic works toward a final decision, narrowing down step-by-step the scope and focus of the issue under consideration (moving from differentiation to unification).As Table 2.1 also shows, each step in policy formulation allows for differentiation but is simultaneously aimed at reaching some form of unification.Stage 1 concludes with a conceptualization of the problem, stage 2 with a choice of policy objectives and stage 3 with a ranking of alternatives.Hence, Table 2.1 emphasizes the persistent tension between the two basic features of political rationality.The same is true for the steps in participatory assessment.In the course of the policy process, participatory assessment tools and methods are capable of opening-up to the extent allowed for by the constraints set in previous stages.The range of alternatives to be explored in stage 3 is highly dependent on the range of policy objectives specified in stage 2.And the range of policy objectives considered is constrained by the problem conceptualization in stage 1.The problem conceptualization, in turn, is largely dependent on the variety of stakeholders and perspectives identified.
The extent to which each step is covered by participatory methods and tools varies.A wide range of methods and tools is available that supports the articulation of perspectives (step 2) and the appraisal of alternative policy options (step 3).Notably few tools, however, focus on synthesis and follow-up, which suggests that participatory tools and methods are used mainly to open up policy appraisal (Stirling 2008) or, in the words of Diesing (1962), aim at differentiation rather than unification.

PARTICIPATORY METHODS AND TOOLS
This section discusses participatory assessment tools and methods with specific relevance for different stages in the policy formulation process.

Stage 1: Agenda Setting and Problem Conceptualization
Ultimately, it is the identification of stakeholders and the range of divergent views they represent which, apart from the organization of the dialogue itself, shapes the contents of the participatory assessment.Remarkably, participatory assessments often identify stakeholders in a rather intuitive way, according to their (assumed) position with respect to a certain issue (Hisschemöller 2005).They may use techniques such as random, stratified or snowball sampling.While these techniques may be helpful to assure representative, large sample sizes in quantitative research, their use for participatory assessment is disputable (Cuppen 2010;2012a).From a learning perspective, representation implies the balanced inclusion of the variety of perspectives.Such 'discursive representation' (Dryzek and Niemeyer 2008, p. 281) asks for tools that enable a selection based on measured rather than assumed stakeholder perspectives.Examples of such tools are Q Methodology (for example, Cuppen et al. 2010) and the Repertory Grid Technique (van de Kerkhof et al. 2009), which allow for both qualitative and quantitative analysis.As for Q Methodology, a limited sample of respondents sort a set of subjective statements on a policy issue, according to a bell-shaped distribution that represents salience to the individual ('most agree' versus 'most disagree').A subsequent quantitative analysis results in a number of (usually two to six) factors which can be interpreted as perspectives.The quantitative analysis enables the identification of respondents who can 'represent' each of the perspectives in a dialogue (Cuppen et al. 2010).The combined use of qualitative and quantitative research techniques via Q Methodology or Repertory Grid Technique reveals that perspectives cannot simply be 'read off' from stakeholders' affiliations with business, environmental NGOs or other stakeholders.Actor types have been found to be heterogeneous with regard to perspectives (Cuppen et al. 2010;Vasileiadou et al. 2014).

Stage 2: Specification of Policy Objectives
In this stage, participatory assessment tools and methods must be geared towards the articulation of diverse stakeholder perspectives in order to share and explore these.A wide range of tools exists for this step, some of which were already addressed for step 1 (for example, Repertory Grid Technique, Q Methodology).Others are dealt with under stage 3.
A widely applied tool that deserves mentioning is Focus Group methodology.Originally developed for marketing, this tool has been adjusted to the context of environmental policy formulation (Greenbaum 1998;Wilkinson 2004;Gerger Swartling 2006).Its basic idea is to arrange for a conversation in a (small) group on a topic presented in a more or less detailed way in order to find out about peoples' impressions and opinions.Focus Group methodology in fact covers a diversity of approaches, ranging from more to less structured, low to high diversity in the group or from little to much information presented.Climate change modelling has used focus groups for receiving feed-back on scientific models and scenarios (see also Chapter 5, this volume).

Stage 3: Identification and Appraisal of Potential Policy Options
Much experience has been gained with tools to support participatory technology assessments on controversial issues such as genetic modification.Examples include Consensus Conferences, Planning Cells and Citizens' Juries.These tools aim at facilitating a dialogue between experts and laypersons in homogeneous (either experts or laypersons) and heterogeneous (experts and laypersons together) groups.As a common feature, these tools often a priori allocate 'knowledge' to the expert domain and 'values' to the domain of laypeople.Separating 'facts' and 'values' in such a way is at odds, however, with the findings of studies into risk perception indicating that both are intertwined (Cuppen et al. 2009).These tools and methods tend to be based on the assumption that differences in judgement mainly exist between experts and laypersons.However, there may be as many differences within layperson and expert groups as there are between them.
Another 'family' of participatory assessment tools and methods originating from management science aims at improving the quality of (business) plans by assessing conflicting stakeholder assumptions.Well-known examples are Devil's Advocate (Schwenk and Cosier 1980;Schweiger et al. 1986), Policy Delphi (Turoff 1975) and Dialectical Methodology (Mason and Mitroff 1981).The underlying idea of these tools and methods is that the appraisal of competing alternatives benefits from the articulation of stakeholders' contradictory (but hidden) assumptions rather than from invoking 'objectified' expert judgement.These participatory assessment tools and methods differ from Consensus Conferences, for example, in that they recognize different realms of stakeholder expertise, including practical knowledge, alongside scientific (academic) expertise, and treat them as equally valuable.Thus they do not separate stakeholders into an expert and lay group.However, they also make assumptions that have an impact on the structuring of the dialogue process.One assumption is that stakeholder debate can be negatively influenced by differences in power and authority of those involved.Therefore, participatory assessment tools and methods may structure stakeholder interaction in such a way that participants do not know each others' identity (for example, in classical delphi), a practice in line with Habermas' notion of the ideal speech situation.Sometimes, the Devil's Advocate technique is organized as a game: the advocate pretends to be against the proposed plan but actually plays a role.Evaluation research suggests that such role playing (artificial conflict) contributes little to learning, while so-called authentic conflict contributes more (for example, Nemeth et al. 2004).
Yet another approach for appraisal of policy options is backcasting, developed as an alternative to forecasting (see Chapter 3, this volume).Participatory Backcasting first identifies a particular (desirable) future end-point and then works backward from it to the present, as if it were already realized.Backcasting can be a powerful tool to assess the feasibility of a (desired) future state and the interventions needed to reach that point (Robinson 2003).It is able to avoid the conservatism inherent in forecasting, as it encourages reflection on the breaking of dominant trends through 'out-of-the-box' thinking (Dreborg 1996).

Stage 4: Decision Making
Some participatory assessment tools and methods are specifically aimed at reaching a decision, such as joint fact finding (McCreary et al. 2001) and the decision seminar (Lasswell 1960).A well-known example of such methodology is Consensus Building, which aims to 'forge agreements that satisfy everyone's primary interests and concerns' (Susskind et al. 1999, p. xvii).Grounded in the theory and practice of interest-based negotiation and mediation (Innes 2004), this problem-solving approach is essentially different from those participatory assessment tools and methods that focus on problem structuring.The notion of consensus is critical in many participatory assessment tools.In essence, consensus tends to be regarded as preferable to dissent, if only because disagreement might cause troublesome personal relationships, which people working in a (small) group would like to avoid.In case of irreconcilable values, consensus may be artificial or symbolic (Kupper 2006).Such consensus obstructs learning and may lead to the adoption of invalid assumptions or inferior choices (Janis 1972;Gregory et al. 2001;Stasser and Titus 1985;Coglianese 1999).It should be noted that artificial or symbolic consensus is not necessarily negative, as it keeps the process moving and enables parties to develop trust (Hisschemöller and Hoppe 2001).
Despite many criticisms, consensus building is, to our knowledge, the only participatory assessment tool and method with an institutionalized sibling.The US Negotiated Rulemaking Act prescribes the negotiation of the terms of a particular proposed rule, hence the name.

PARTICIPATORY ASSESSMENT IN PRACTICE
Unfortunately there is a lack of systematic evaluation of the policy impact and effectiveness of participatory assessment tools and methods or participatory assessment in general.One of the main reasons for this probably relates to the conflicting (and hidden) aims of participation noted above.There is often a discrepancy between how particular tools and methods are applied in practice and how they are prescribed by theory.While this chapter focuses on participatory assessment tools and methods for policy formulation, in practice they are often used to legitimize already decided policy.This has repercussions, for example when different expectations exist with regard to the role and intended impact of a participatory assessment.The policy evaluation literature shows numerous examples of disappointments among participants who have invested energy in participatory assessments only to find out in the end that policymakers did not use -or Consensus Conferences (CCs) have been documented for Denmark (below), New Zealand (Goven 2003), the UK (Joss 2005;Irwin et al. 2012), Norway (Oughton and Strand 2004), Belgium (Vandenabeele and Goorden 2004), Canada and Australia (Einsiedel et al. 2001), and the Netherlands (Jelsma 2001).Most CCs have been commissioned by government and organized by an external institute, either affiliated to parliament (e.g. the Danish Board of Technology, an NGO since late 2011) or independent.Evaluations of specific CCs show that practice may deviate from theory due to particular contextual and venue-specific factors.For example, limited interaction between citizens and experts was reported for a Belgian and Austrian CC (Vandenabeele andGoorden 2004, Joss andBellucci 2002).Too little time for public debate, lack of transparency and overall mistrust were reported for CCs in the UK and the Netherlands (Joss 2005, Jelsma 2001).
In Denmark, where parliament has recognized CC as an important policymaking tool, CCs have provided a base for policy directions (Grundahl 1995), but have not had an immediate policy impact (Einsiedel et al. 2001;Vandenabeele and Goorden 2004;Joss and Bellucci 2002).At best, evaluations report learning among the participating citizens and experts.
Example 2: Participatory Backcasting (PB) Participatory Backcasting (PB) has been widely applied in cases ranging from: the future of natural areas in Canada (Tansey et al. 2002;VanWynsberghe et al. 2003); energy futures for the Netherlands (Hisschemöller and Bode 2011;Breukers et al. 2013), for the UK (combined with multi-criteria appraisal; Eames and McDowall 2010) and for Belgium (Keune and Goorden 2002); sustainable households in five countries (Green and Vergragt 2002); and long-term changes of Swedish city life (Carlsson-Kanyama et al. 2003).Participants are usually stakeholders, representing different sectors and groups.There is no evidence for immediate policy impact of PB.Yet, as Quist (2007) shows, it may encourage higher order learning among participants as well as follow-up programmes.Venuespecific factors shape how PB works out in practice.An example is provided by the Dutch Hydrogen Dialogue (2004Dialogue ( -2008)), funded by the Dutch Organization for Scientific Research.This addressed the question of how hydrogen can contribute to a future sustainable energy system (Hisschemöller and Bode 2011).About 60 stakeholders (from the Netherlands and abroad) participated, including energy companies, small innovative firms, knowledge institutes, vehicle lease and transport companies, NGOs and one association of home owners considering the establishment of a hydrogen-based energy system in their neighbourhood.Since participants valued the utilization of policy-relevant results highly, the project team committed three former Dutch MPs as independent chairs of three dialogue groups.Participants were invited based on the outcome of a Repertory Grid exercise (van de Kerkhof et al. 2009), which unfolded three perspectives on a 'hydrogen economy'.PB was then used for developing (competing) hydrogen pathways.

Matthijs Hisschemöller and Eefje Cuppen -9781783477043
Downloaded from Elgar Online at 12/16/2018 09:49:26PM via free access even abused -their contribution.Processes considered unfair, biased or as pseudo-participation generally do not contribute to public acceptance.A related explanation for the lack of systematic evaluation may be that (hidden) conflicts between instrumental, substantive and normative aims of participation also undermine the evaluation itself, especially as the (main) goal, of gaining acceptance, remains implicit or is covered under the veil of substantive or normative aims.Some authors observe resistance among policymakers and their techno-scientific advisers to participatory exercises (for example, Irwin et al. 2012).Participatory assessments and public participation in general increase uncertainty among policymakers with respect to the timing and actual outcome of a policy formulation process (Hisschemöller and Hoppe 2001).Whereas policymakers may like the idea that participatory assessments contribute to the public acceptance of policies, they dislike the idea that successful participatory processes may diminish their control.Hence, they may not have an interest to know about the impact of participatory assessment tools and methods.
Another explanation for the lack of systematic evaluation relates to difficulties in measuring the impact of such tools and methods.First of all, policy learning is a slow process, as is generally the case with the utilization of research.Evaluating effectiveness is difficult, as policy change is a complex process that takes place over periods of at least a decade (Sabatier 1999).Participatory assessments may therefore have an impact only in the longer term, which may be difficult to measure.Difficulties in measuring the impact of participatory assessments also relate to the fact that the impact may not be restricted to changes in governmental policy, but At a 'Confrontation Workshop', the pathways were reviewed by international keynote speakers, a national Advisory Board including experts and policymakers, and the participants themselves.In this application of PB, creative conflict was a central design issue, intended to stimulate learning through interaction between stakeholders from different (inter)national networks.However, the anticipated learning effect was hampered because the conflict on substance turned into a conflict of interests.Eventually, the participants from the national Energy Research Institute distanced themselves from the entire dialogue report, because in their view the dialogue facilitators did not sufficiently distinguish energy experts' views from non-expert opinions.
The dialogue did not have an immediate impact on policy.However, a few years later the Dutch National company, Gasunie, started implementing the option most controversial throughout the dialogue, adding large quantities of H 2 into a local gas infrastructure.The actors taking most advantage of the dialogue were small innovative entrepreneurs, seeking like-minded stakeholders to start up transition experiments.

Matthijs Hisschemöller and Eefje Cuppen -9781783477043
Downloaded from Elgar Online at 12/16/2018 09:49:26PM via free access may affect other domains and actors as well.There is evidence that stakeholders learned, especially about the different perspectives on the topic (Cuppen 2012b).Academics, companies, innovative entrepreneurs, NGOs and (local) government officials then initiate follow-up activities beyond the level of (national) government (Quist 2007).Surprisingly, the authors' own participatory assessments on climate and energy have led to technological inventions and initiatives for collaboration among stakeholders.This may also confirm that the impacts of participatory assessments may be especially significant in the longer term.
Interestingly, this suggests that participatory assessment tools and methods are not primarily used in the venues where they were initiated.It suggests that participatory assessments, like participatory processes in general, can themselves create venues as well.Participatory assessment tools and methods are a vehicle for bringing together different actors, exchanging ideas and viewpoints and mobilizing resources.In other words, they create new networks, most of these starting as informal and at some distance from state policy venues.However, over time these venues may expand into new institutions for deliberating on policy objectives, options and strategies, as for example, Sabatier (1999) shows.
A last point to be mentioned here is that there are many participatory assessments of varying quality, which makes it hard to systematically evaluate their impacts.For some examples, it is even questionable whether they may legitimately be described as 'participatory'.In an evaluation of the Austrian trans-disciplinary programme, Felt et al. (2012) find that the researchers on the one hand strongly convey the participatory discourse, but simultaneously tend to protect the privileged position of the researcher vis-à-vis societal stakeholders.
In conclusion, there is still much work to do in evaluating the real impact and quality of participatory assessment tools and methods, starting with developing methodologies for categorizing and measuring these impacts.This may support quality and usefulness of future tools and methods.

CONCLUSIONS
This chapter has highlighted the great variety of participatory assessment tools and methods applied in many policy sectors and venues across the world.These tools and methods have in common that they facilitate some sort of dialogue between people with different views on a specific topic: participatory methods arrange for a procedure along the various stages of the policy formulation process, whereas tools can be applied in only one or few stages.Participatory assessment tools and methods can easily be integrated into other policy formulation tools that require feedback from stakeholders, such as environmental modelling (see Chapter 5, this volume) or multi-criteria appraisal (see Chapter 6, this volume).
We find that only a few participatory assessment tools and methods seriously address the issue of stakeholder identification and selection, despite the fact that this first step determines the process and outcome to a high degree.Most participatory assessment tools and methods can be used for identifying objectives or for exploring alternative courses of action.
In assessing the potential of participatory assessment tools and methods, two issues are of critical importance.First, we find that different (sometimes irreconcilable) views on participation have immediate consequences for their design and application.Some focus on reaching consensus, in order to facilitate decisions on controversial issues, while others focus on articulating conflicting perspectives to enhance learning with respect to developing new policy approaches and options.Second, the practice of applying participatory assessment tools and methods often suffers from contradictory objectives among participants and disappointments that policymakers are more interested in legitimizing already decided measures than in gaining new ideas for addressing intractable issues.These two observations may also explain the observed lack of systematic evaluation of participatory assessment tools and methods in practice.
The critical evaluation this chapter offers is meant to present a state of the art with a fair assessment of the challenges in the field.We do not intend it to discourage readers from studying and employing participatory approaches.After all, despite much scepticism and resistance in policy venues, openness to new insights is and must remain a major feature of good policy formulation and governance.

Table 2 .
1 A comparison of the different stages in the policy formulation process and the main steps in participatory methodology