Multi-criteria analysis : a tool for going beyond monetization ?

Multicriteria analysis (MCA) has emerged from the field of operational research and management science as an appraisal tool able to handle complex multifactorial decision problems that affect several stakeholders and where an equitable, inclusive and transparent decision process is sought. According to the International MultiCriteria Decision Society (IMCDM 2013), multicriteria analysis dates back to the 1950s when analysts started to consider multiple objectives for optimality conditions in nonlinear programming – socalled ‘Goal Programming’. Since then, a multitude of MCA methods have been developed (some of which will be discussed below) and their use has gone far beyond the realm of operational and business research, as we will demonstrate later in the chapter. To assess the worth of different policy options, MCA aggregates the results on multiple evaluation criteria into indicators of the overall performance of options without enforcing the transformation of criteria and their results to a common – what is in many other tools a monetary – scale. In its role as a decision aiding, rather than a decision making tool, MCA seeks to render the evaluation of policy options transparent to the decision maker and other stakeholders, instead of ‘replacing the decision maker with a mathematical model’ (Roy and Vincke 1981, p. 208). MCA thereby seeks to promote ‘good decision making’ (Keeney and Raiffa 1972, p. 65) by offering a clearer illustration of the different inputs that typically go into a policy formulation process, and by dealing in a structured way with multiple, conflicting objectives and value systems. In particular, the problemstructuring phase of the policy formulation process – during which the goals of policy, the options to be evaluated and the criteria according to which this is to be done are defined – is recognized as a useful learning opportunity to which MCA can contribute (Marttunen and Hämäläinen 1995). In this phase, MCA stimulates discussion between the various stakeholders (French et al. 1993) and helps decision makers to


INTRODUCTION
Multi-criteria analysis (MCA) has emerged from the field of operational research and management science as an appraisal tool able to handle complex multi-factorial decision problems that affect several stakeholders and where an equitable, inclusive and transparent decision process is sought.According to the International Multi-Criteria Decision Society (IMCDM 2013), multi-criteria analysis dates back to the 1950s when analysts started to consider multiple objectives for optimality conditions in non-linear programming -so-called 'Goal Programming'.Since then, a multitude of MCA methods have been developed (some of which will be discussed below) and their use has gone far beyond the realm of operational and business research, as we will demonstrate later in the chapter.To assess the worth of different policy options, MCA aggregates the results on multiple evaluation criteria into indicators of the overall performance of options without enforcing the transformation of criteria and their results to a common -what is in many other tools a monetaryscale.In its role as a decision aiding, rather than a decision making tool, MCA seeks to render the evaluation of policy options transparent to the decision maker and other stakeholders, instead of 'replacing the decision maker with a mathematical model' (Roy and Vincke 1981, p. 208).MCA thereby seeks to promote 'good decision making' (Keeney and Raiffa 1972, p. 65) by offering a clearer illustration of the different inputs that typically go into a policy formulation process, and by dealing in a structured way with multiple, conflicting objectives and value systems.In particular, the problem-structuring phase of the policy formulation process -during which the goals of policy, the options to be evaluated and the criteria according to which this is to be done are defined -is recognized as a useful learning opportunity to which MCA can contribute (Marttunen and Hämäläinen 1995).In this phase, MCA stimulates discussion between the various stakeholders (French et al. 1993) and helps decision makers to better comprehend the decision problem, as well as the values and priorities involved (Belton and Stewart 2002).
Numerous case studies in the literature suggest that MCA has seen widespread application across different policy venues, spanning many policy areas concerning the environment, public transport and health, analysis of vulnerability to natural and man-made hazards, and many others.Indeed, MCA has been recognized by a number of governments, NGOs and international organizations as the preferred way to analyze complex decisions.It has even been legally prescribed in some cases.The tool's ability to open up to different value systems by directly representing stakeholders' preferences (through participation in the evaluation process) has particularly appealed to critics of other evaluation tools that can integrate preferences only in indirect ways, such as through monetary evaluation in CBA (see Chapter 7, this volume).
To evaluate the tool's merits, we will first take a close look at the main methodological aspects of MCA, before analyzing its application across different policy venues.This will lead us to an evaluation of the tool's added value and caveats which policymakers and analysts have to bear in mind when applying MCA in policy formulation processes.We will also provide insight into the venues that are most favourable to its application.This should hopefully inform the future application and development of MCA across different policy venues and sectors.

MAIN METHODOLOGICAL ASPECTS OF MULTI-CRITERIA ANALYSIS
In common with some other policy formulation tools, for instance CBA (cost-benefit analysis) or CEA (cost-effectiveness analysis) -covered in Chapter 7 -MCA provides an integrative decision making methodology, from problem and objectives definition, through evaluation of policy options, to ranking/comparing options.The underlying methodology, however, is different.
Multi-criteria analysis may be structured in several steps (see for example, Keeney 1992;Roy 1996;Dodgson et al. 2000;Munda 2004): characterization of the decision context (for example, individual or group decision making, need for participation, and so on) and the type of recommendation needed (for example, ranking, choice of best option, and so on); definition of options; elaboration of evaluation criteria; assessment of options' impact with respect to these criteria; preference modelling and aggregation of preferences; sensitivity and/or robustness analysis.Figure 6.1 shows the similarities and differences between MCA and CBA, which is also frequently applied to inform policy formulation processes.
CBA, though following similar steps in the policy formulation process, identifies positive and negative impacts of policy options quite differently, as it uses a single evaluation criterion and requires the valuation of all impacts in monetary terms.Such a simplified logic makes input elements very straightforward to compare and has therefore attracted widespread application across diverse policy venues and sectors, from public health Source: Gamper et al. (2006, p. 294).

Figure 6.1 Multi-criteria analysis and cost-benefit analysis: a comparison of the different steps in the process
and transport to the environment.The monetization process demands a high degree of methodological rigour to avoid biases and maintain internal validity.This has raised issues in the practical application of this approach.
For instance, putting a price tag on the marginal costs of a loss of biodiversity is not only technically elaborate, but may prove unacceptable to those who believe that the intrinsic existence value of nature is unmeasurable in money terms.In practice, when rigorous monetary evaluation proves too time-and resource-intensive, CBA often leaves some values out of the equation altogether, rendering its results technically invalid (see for example Joubert et al. 1997;Brouwer and van Ek 2004).MCA offers scope for resolving some of these issues by accepting the multiplicity of impact dimensions and hence evaluation criteria for complex policy issues, such as the environment or health.This proves especially useful in the case of 'soft' or intangible factors, such as ethical, social, cultural or ecological ones, for which monetization of impacts may be exceedingly difficult and/or contentious (Gamper and Turcanu 2007).
MCA allows for consideration of several value systems and for participation to take into account the preferences of different stakeholders.Gamper et al. (2006) and Lebret et al. (2005) argue that MCA should be the preferred method if consensual solutions to resolve conflicts need to be found.For a detailed discussion of the methodological differences see Tietenberg (2001), Edwards-Jones et al. (2000), Munda et al. (2004), Gamper et al. (2006).

Identifying Objectives and Criteria
Decision makers' and stakeholders' values or preferences may be explicitly included in a MCA model through a set of criteria against which the impact of the potential policy options is evaluated.This may include environmental criteria, such as the protection of natural habitats for certain species, or economic criteria, such as the job and economic development opportunities or costs arising from an infrastructure development policy.Evaluation criteria can be built in two ways.The top-down approach starts from a main objective and builds a hierarchical tree structure of fundamental objectives (Keeney 1992) or key concerns (Bana e Costa and Beinat 2005).The bottom-up approach starts from the impacts of policy options and builds a consistent family of evaluation criteria (Roy 1996) by partial synthesis of related and non-conflicting items.In practice, a combination of the two approaches may prove the most efficient (Bana e Costa and Beinat 2005).
Numerous studies in the MCA literature have addressed the desirable properties of a good set of criteria: the most important include (1) exhaustiveness (the criteria selected characterize completely the evaluation of any policy option); (2) cohesiveness (partial preferences with respect to each individual criterion have to be consistent with the global preference); and (3) non-redundancy (elimination of any criterion from the chosen set of criteria leads to violation of at least one of the previous properties) (Roy 1996).The second in this list means for instance that an improvement of an option with respect to some criteria should not lead to a worse 'global' evaluation.Roy and Bouyssou (1993) give an example involving a 'reliability' criterion, proving that in some cases this property might not be valid: two cost values might be indifferent if they both have a low reliability, but the lower cost could be strictly preferred if the higher cost is obtained with a higher reliability.Keeney (1992) argues that the evaluation criteria should also be operational, that is, allowing impact assessment for the available policy options within reason given available time and resources.Related to the latter, one should note that in the presence of uncertainty, the evaluation of an option with respect to a criterion might not be a unique element, but rather an interval, a distribution or a fuzzy set.

Identifying Policy Options to Achieve the Set of Objectives
The set of possible policy options taken into consideration in the policy appraisal process should include those considered realistic by at least one of the actors, or assumed as such by the analyst (Roy and Bouyssou 1993), and that contribute to the achievement of objectives.The definition of policy options depends both on the problem itself and the actors involved, and strongly influences the subsequent methodological steps.The set of options can be comprehensive, if every option is exclusive of any other, or fragmented, if certain combinations of individual options have to be considered.
Since MCA can and should ideally be based on an iterative process, the initial set of options might be modified during the policy formulation process, but should always include a comparison with the option of remaining with the status quo.

Evaluating Policy Options
In the MCA literature (Vincke 1992;Roy 1996;Belton and Stewart 2002), three main categories of approach are highlighted.However, it is recognized that these categories do not have sharp boundaries and that combinations of two such approaches can be envisaged (Roy 1996).The categories are: (1) multi-attribute utility (MAUT) methods, seeking to aggregate all points of view into a unique function which is to be optimized; (2) outranking methods, which construct and exploit a synthesizing relation based on the decision maker's preferences; and (3) interactive/trial-error methods, which explore the space of feasible options through a dialogue with the decision maker(s).Some guidelines exist on choosing a specific MCA approach (for example, Guitouni and Martel 1998), as well as recommendations on the desired properties it should have (for example, Munda 2004).In the following, the main MCA methods are summarized.For more details on the underlying assumptions and the related theoretical aspects, the interested reader may consult the references provided in the text.

Multi-attribute Utility/Value Methods
Multi-attribute utility methods (MAUT) (Keeney and Raiffa 1976) are based on the assumption that the decision maker's preferences are coherent with some increasing real function U called utility, which (s)he attempts to maximize.In other words, an option a is preferred over another option b, if and only if U(a).U(b).
In the additive model, which is most commonly used, the utility of an option a is expressed as a sum of the partial utilities: where U i are single-attribute utility functions corresponding to the evaluation criteria g i and n is the number of criteria.An extensive literature has been dedicated to building the additive model, for example Fishburn (1967) and Jacquet-Lagrèze and Siskos (1982).Fishburn (1967) has formulated sufficient and necessary conditions for the existence of an additive utility function.A necessary condition for the validity of the additive model is, for instance, that any subset of criteria is preferentially independent of the remaining criteria. 1Keeney (1992) provides other examples of relatively simple (for example, multi-linear) utility functions and the conditions for the validity of the corresponding models.
When uncertainties are not taken into account, the model becomes a multi-attribute value model.The additive value model can be formulated as the maximization of a value function V given by: where the weights w i are scaling constants that indicate value trade-offs between criteria.These weights can be determined by various techniques, as illustrated for example in Hämäläinen (2002).
The uncertainty and imprecision in MAUT models can be modelled by means of probability theory.It is interesting to note that the shape of the utility function has a direct relation with the attitude to risk of the decision makers.A concave utility function corresponds to risk aversion, a linear function to risk neutrality and a convex function to risk proneness (Keeney 1992).

Outranking Methods
Outranking methods (Roy 1985) were developed to address some difficulties experienced with the MAUT approach in dealing with practical problems.Bouyssou (2001) notes that outranking methods do not require establishing trade-offs between criteria in order to derive overall preferences and that they are mostly non-compensatory.This implies for instance that a very weak performance on an important criterion (say, health effects) cannot be compensated by better performances on a number of less important criteria, as it could be in the case for MAUT methods.
Outranking methods may involve the use of a more general criterion model, called pseudo-criterion, which is characterized by two thresholds describing the concepts of indifference and strong preference.These thresholds are related in some cases to the uncertainty inherent in the evaluation of certain criteria.The analysis of options in outranking methods entails pairwise comparison of options on each criterion, and subsequently building an overall preference relation (also called outranking relation) aggregating these partial preferences.The underlying principle is 'democratic majority, without strong minority'.Accordingly, an option a outranks option b, or in other words a is at least as good as b, if a majority (or more important set) of criteria supports this assertion and if the opposition of the other criteria (their number or their importance) is 'not too strong' (Bouyssou 2001, pp. 249-250).The outranking relation can be further exploited to derive the best option(s) and issue a recommendation.Some of the outranking methods, such as ELECTRE I-III (Roy and Bouyssou 1993) and PROMETHEE (Brans et al. 1984), also require assigning weights to criteria.However, for such methods weights represent the intrinsic importance of the evaluation criteria, instead of value trade-offs, as in the case of MAUT.Some outranking methods such as MELCHIOR (Leclercq 1984) or ELECTRE IV (Roy 1996) can also cope with situations when criteria weights cannot be assessed.

Interactive Methods
Interactive methods (Steuer 1986;Vanderpooten and Vincke 1989;Vincke 1992;Lee and Olson 1999) alternate the computation steps with interaction steps in which the analyst gradually specifies or revises preference information, in accordance to the decision maker's or other stakeholders' requests.
In the early stages of investigation, the set of decision options may be itself an outcome of this interaction.
The underlying principle of this MCA approach is inspired by Simon's theory of satisficing (Simon 1976), the goal being to find a satisfactory compromise solution.This is especially appropriate (Belton and Stewart 2002) for the case when the participants in the decision process have some good a priori ideas about the realistically achievable levels for the evaluation criteria.
Interactive methods can be seen to function in a search-oriented or learning-oriented framework.In the latter setting, the set of nondominated solutions is freely explored, the current solution found being compared with the most preferred up to that stage.Therefore, a solution discarded at some step might be reconsidered at a later stage.

Sensitivity and Robustness Analysis
Data uncertainty and imprecision are inherent characteristics of real-life applications and equally affect MCA models.A classical way to deal with this is to undertake sensitivity analysis.This seeks to determine the parameters which contribute most to the variance in the MCA results or how much the model parameters (for example, criteria weights) may vary such that the conclusion of interest (for example, that a policy option achieves the best rank) still holds.
An alternative way to address uncertainty and imprecision in MCA is robustness analysis.The notion of robustness may have different interpretations (Dias 2006).Roy and Bouyssou (1993, p. 315) use the term 'robust' for a result or conclusion that is not 'clearly invalidated' for any possible instance of the decision model parameters (for example, weights or thresholds).Connected to that, the robustness analysis is the process of elaborating recommendations founded on robust conclusions.Dias and Clímaco (1999) identify two types: For instance, if the range of a criterion's weight is estimated as [0.3, 0.5], sensitivity analysis may point out for example that the ranking of the different policy options changes if this weight becomes larger than 0.4.Robustness analysis can indicate instead that a given option will always outperform another, no matter what the particular value of the weight in the given interval.

MULTI-CRITERIA ANALYSIS IN POLICY PRACTICE
In the following we will look at aspects of usage, the policy venues where it occurs, the quality of implementation, and the legitimacy and policy relevance of MCA.We first ask why and under what circumstances the application of MCA was initiated and by whom it is used, whether its application has been driven by decision analysts or whether it has been more formally adopted in legal frameworks prescribing it as a tool to evaluate policy options.We then assess the quality of policy formulating processes with MCA based on: (1) their transparency and legitimacy (for example, integration of stakeholders in the process) and ( 2) the degree to which the results of the policy formulating process through MCA were relevant for actual policymaking.

Multi-criteria Analysis: Its Venues
In this section we discuss venues favourable to the application of MCA.As will be seen, MCA is applied at different levels of governance, with different triggering factors.To get a better insight into the real potential and use of MCA, we go beyond the definition of venues as introduced in Chapter 1. Accordingly, we look not only at the institutional environments where MCA has been applied, but also the processes by which it became relevant to policy appraisal (for example, formal requirement versus experts' decision), the predominant application fields and the policy context.

The users
MCA has been used by various actors to inform policy formulation processes.While the use of the tool is often initiated by analysts as support for local, regional or even national policy formulation processes (for example, Bana e Costa and Oliveira 2002; Petras 1997), governments themselves have also used and recommended its application (Del Rio Vilas et al. 2013;Munda 2004; van Gennip et al. 1997).International organizations have also applied MCA, as shown for example in the UN Environment Programme's use of the method for the evaluation of emissions abatement options (Borges and Villavicencio 2004).In still further cases, a MCA-based research study initiated by analysts without governmental involvement or participation later informed a policymaking process, as was the case in a French study where MCA results became part of the government's strategic programme for flood prevention (Azibi and Vanderpooten 2003).

The main policy areas
MCA studies have been applied as policy formulation tools across a number of policy areas (see examples in Table 6.1), most notably environment, Other public policy areas where MCA has been quite frequently applied are public health and infrastructure.For example, Bana e Costa et al. ( 2001) have analyzed the case of a Portuguese railway line, van Gennip et al. (1997) used MCA to rank the most common diseases in the Netherlands in order to come up with a prioritization strategy for the government's financing of public health, while Del Rio Vilas et al. ( 2013) applied MCA as a decision-support tool for the Veterinary Risk Group in the UK.

Formal Requirements Prescribing MCA
In contrast to other tools (such as CBA) which are frequently legally prescribed (for example in large infrastructure projects in the Netherlands or the United Kingdom especially) and for natural hazard management (for example, in Austria, Switzerland), MCA has not received such widespread legal backing.Gamper and Turcanu (2007) identify some of the difficulties linked to the application of MCA at governmental level, including the variety of MCA tools which makes standardization problematic; the difficult inter-comparison of case studies (different methods may yield different results) and the technical complexity of MCA modelling.A recent survey by Adelle et al. (2011) among 124 specialists in charge of Impact Assessment showed that only 6 per cent thought that the use of MCA is encouraged in their country, while this rose to 27 per cent for monetary evaluation tools such as CBA and CEA (see also Chapter 9, this volume).
Nevertheless, some notable examples of legal requirement for MCA do exist.For example, the Italian law for public works (ANAC 2011) stipulates that project selection should be done using a multi-attribute value method, ELECTRE, or any other MCA method recognized in the scientific literature (AHP, TOPSIS, and so on).Another example can be found in Spain where the acquisition of data-processing equipment by the central public administration offices has to be conducted based on MCA (Barba-Romero 2001).According to Joubert et al. (1997), MCA is also implicitly required by law in the USA for water resource planning.The increased focus on MCA is currently reflected by the inclusion of the tool in policy guidance documents elaborated at governmental level (Dodgson et al. 2000;Brooks et al. 2009).For instance, Brooks et al. (2009, p. 46) recommend MCA as 'particularly suitable for participatory decision making'.
A number of European Union and United Nations documents currently recommend the use of MCA.The UN Framework Convention on Climate Change (UNFCCC), for example, recommends MCA if evaluation criteria cannot (easily) be accommodated in monetary evaluation (UNFCCC 2002) and FAO (the UN's Food and Agriculture Organization) proposes cost-utility analysis based on an additive utility model for the evaluation of food quality systems as an alternative to other tools such as CBA (Krieger et al. 2007).

The Reasons for Choosing MCA
The results reported in the literature show that a general motivation for choosing MCA over other tools is to gain a better insight into the complexity of decisions on public policies and their consequences when these are felt not only on one, but on multiple dimensions, including economic, social, environmental or institutional.The tool is applied for systematic comparison and ranking of policy options, sometimes in combination with other tools such as CEA (Wu et al. 2012) or Life Cycle Analysis (Tjandraatmadja et al. 2013).MCA is also applied to lay out the basis for future policies, for instance by evaluating and prioritizing emerging threats and vulnerabilities (Del Rio Vilas et al. 2013).
MCA is chosen when trading between different objectives (for example, sustainability objectives and economic objectives) is difficult (Dichmont et al. 2013, p. 130), where its appeal comes from its 'attention on impacts related to specific objectives, thus reducing potential bias'.Similarly, for Hobbs and Horn (1997, p. 357), MCA was chosen in an energy planning application because it makes 'choices more explicit, rational and efficient', which is accomplished, among other ways, by displaying trade-offs among criteria so that 'planners, regulators and the public can understand the advantages and disadvantages of alternatives'.
However, the potential of MCA is most evident in situations involving a complex policy context, with multiple, potentially competing objectives and value systems, which cannot be easily quantified (for example, environmental issues) let alone translated into monetary terms, due to their intangible nature (for example, social, cultural or psychological issues).This applies particularly to resolving conflicts around public policy decisions that spread over jurisdictional borders, where no established decision making procedures are in place, and conflict potential may arise, as demonstrated for example by the case of managing water users' interests on the river Spree in Germany (Messner et al. 2006).
But MCA's capabilities go further than that: it can structure and facilitate stakeholders' involvement in decision processes.This is a key aspect, since through participation it contributes to the democratization of the policy formulation process and to its enhanced fairness and overall efficiency (Stirling 2008), potentially increasing the quality of decisions (Beierle 2002) and resulting in more widely accepted policy options (for an example see Linkov et al. 2006).The European Commission's Evaluation of Socio-Economic Development Guide (EC 2013, p. 135) suggests that MCA provides a framework facilitating the participation of all actors in decision making and in problem solving, which may help in 'reaching a compromise or defining a coalition of views, without dictating the individual or collective judgement of the partners'.Some case studies in the literature mention the participatory framework provided by MCA as among the main reason for choosing the tool.For instance, Haque et al. (2012) have used MCA following the recommendations of the UNFCCC (2002), to develop a 'participatory integrated assessment' of adaptation options for flood protection in Bangladesh.

The Quality and Legitimacy of Multi-criteria Analysis in Practice
Opening up the policy formulation process can increase or undermine the legitimacy of a MCA study Among the advantages of policy formulation with MCA is its opening to different value systems, as mentioned above, which is particularly fostered by a transparent and inclusive participatory process (see Chapter 2, this volume).Although a generic MCA cannot be considered as a participatory tool in itself, stakeholders can participate in some or all stages of the policy formulation process when MCA is used.Some MCA tools specifically designed for opening up the decision making process to participation, such as multi-criteria mapping (Stirling and Mayer 2001), are listed among the current tools for participatory policy assessment (see Chapter 2).
In practice, the extent to which stakeholder inclusion in MCA takes place largely depends on the people steering the process and brings with it challenges inherent to any participatory assessment exercise.In reported case studies (see Table 6.1) its inclusion has sometimes involved fairly narrow approaches, such as when only experts and/or authorities are included (Bana e Costa and Oliveira 2002; van Gennip et al. 1997;Petras 1997;Brouwer and van Ek 2004).Broader participation entails sharing the involvement and responsibilities in the policy formulation process more widely, as shown by Borges and Villavicencio (2004) or Marttunen and Hämäläinen (1995).In these cases, experts built the scenarios, as well as the criteria, while the evaluation and the weighting process were performed by wider stakeholder groups.Or, as in the case described by Bombaerts et al. (2007), the options can be identified in a participatory manner, such as was done for low-level radioactive waste disposal options elaborated in a dialogue between a radioactive waste agency, local communities and local individuals.Examples where stakeholders participated in all phases of the policy formulation analysis, including the definition of alternatives and criteria, are provided for example by Joubert et al. (1997) and Messner et al. (2006).
Opening up the appraisal processes to a wider stakeholder group has other clear benefits, not only in making the outcome of a policy formulation process with MCA more legitimate, but also in terms of clarifying the problem, both between the public and experts, as well as between experts of different fields (Kontić et al. 2006).In a larger participative context, including the general public, Renn et al. (1993) propose a three step procedure relying on MCA, but making a division of decision making tasks between three levels of society: evaluation criteria are to be constructed by involvement of all relevant stakeholders; identification and impact assessment for the decision options are mainly carried out by experts; and weighting should be done by citizens' panels.
Early involvement of stakeholders (Banville et al. 1998) can give a more pragmatic dimension to MCA and contribute to an increased acceptance of the final result.Stakeholder processes are, however, costly and time consuming and, in terms of legitimacy, may on some occasions not contribute in the manner suggested by advocates (see Chapter 2, this volume).In some cases they may even lead to a stalling of the decision process.Therefore, difficult questions remain over which stakeholders should get involved, at what time and through which processes.
In addition, politics may sometimes constrain wide stakeholder involvement.Political actors may not wish to openly express their priorities, or may have their own hidden agendas.Bana e Costa et al. ( 2001) describe a case where direct participation was replaced -at the request of one of the actors -by an analyst simulating the viewpoints of all relevant stakeholders.Similarly, Brouwer and van Ek (2004) report on a stakeholder analysis, where experts judged the effect different strategies might have on the elicited stakeholder groups, but the MCA was then performed without them.
Another challenge relates to knowledge and information sharing, which means on the one hand making technical information understandable to all stakeholders, and on the other making technical specialists aware of the social and political dimensions of the problem they face (Bardos et al. 2002, p. 19).This brings with it a necessary reduction of complexity, but carries the risk that participants consider that the evaluation criteria employed in the final analysis oversimplify the underlying problem (Marttunen and Hämäläinen 1995).
MCA is sometimes regarded as challenging the expert's prerogatives since it may be interpreted as making specialist knowledge subject to non-expert evaluation.In a study on the use of decision aid tools including MCA in environmental management, Joliveau et al. (2000) show that experts may oppose such tools due to several factors, including inter alia hesitation in changing the usual procedures, fear that the model will collide with their recommendations or reluctance to share their power of decision.By contrast, Belton and Stewart (2002, p. 160) emphasize that an important role of MCA is to provide a 'sounding board' against which experts and decision makers can test their intuitions.They illustrate a good number of MCAs for which analysis and intuition were 'successfully reconciled' (Belton and Stewart 2002, p. 288), for example due to creation of new decision alternatives, or to the reconsideration of preferences.

MCA results can be directly and indirectly policy-relevant
Even though the essential contribution of MCA in a policy formulation process lies in decision aiding, rather than making, it is still relevant to understand whether it actually informs policymaking or is employed instead as a symbolic planning tool.The latter can render participatory processes ineffective, once participants sense that the decision has already been taken regardless of their inclusion.In practice, the final decision may or may not comply with the recommendations derived from the MCA.However, the analysis itself, the questions raised and the type of reasoning promoted (Bouyssou et al. 2000) can have a positive impact on the decision process, in that preferences are revealed and can thus be considered by the final decision maker (thereby avoiding the interests of some being favoured automatically over those of other affected stakeholders).
The usefulness and integration of the MCA outcomes in policy formulation are not easily observed through a desk review.This information is rarely tangible and seldom reported in the case study results, partly because it may take time after the process until the actual decision by policymakers is taken.A more in-depth understanding would require research among analysts, stakeholders and policymakers to understand the actual translation of outcomes in the policymaking process.Some indicative information can nevertheless be found.For example, in the Dutch case reported by van Gennip et al. (1997), the results of the MCA were directly discussed in the Dutch parliament to formulate health policy options.Also in the study of Bana e Costa and Oliveira (2002, p. 390) the results are 'informally used for deciding which requests for building works should be given priority in each year'.The case reported by Borges and Villavicencio (2004) presents another example of a MCA study the outcome of which subsequently formed the basis for the policy options presented by the Peruvian government in its National Communication to the UN Framework on Climate Change (Borges and Villavicencio 2004).Finally, Fletcher et al. (2010) present a case where the priorities derived by the application of MCA for the evaluation of ecological assets in the West Coast Bioregion of Australia by the Department of Fisheries process form now the basis for the Department's budget planning process.

CONCLUSION
This chapter has shown that MCA's policy relevance is observed beyond its legal prescription in national laws or directives, being frequently encountered -in one approach or another -in situations requiring a transparent, well-structured and inclusive policy formulation process.As we have shown, one has to look deeper into the structure, initiation and the participating partners to assess its potential and impact on policy appraisal decisions.The actual relevance of MCA as a policy formulation tool is twofold.First, a number of successful applications in various policy domains show that MCA can cope with multi-factor, multi-stakeholder decision problems, its outcome being supported by stakeholders and decision makers in a good number of cases.Second, the mutual learning among experts and stakeholders promoted by MCA means that the findings or difficulties encountered can be used for improving the policy formulation process in various ways.This may involve developing new policy options (for example, Bana e Costa et al. 2001), broadening the group of stakeholders participating in the policy formulation process or incorporating the results of the procedure in new regulations (for example, van Gennip et al. 1997).Therefore MCA could be of use not only when embedded as a means of appraisal in new regulations, but also indirectly, in order to support the evaluation of current policies, while orientating future ones (Stirling and Mayer 2001).
However, MCA results can be seen as highly subjective, due to the emphasis on 'the judgement of the decision making team, in establishing objectives and criteria, estimating relative importance weights and, to some extent, in judging the contribution of each option to each performance criterion' (Dodgson et al. 2000, p. 20).For this reason, some authors (for example, Bardos et al. 2002;Lebret et al. 2005) advocate a need for international standardization and harmonization in the use of tools like MCA in order to increase their applicability.While weighting the different criteria is certainly dependent on the societal context, technical (legal) guidelines could be drafted for specific application domains, in terms of criteria and indicators to be considered, MCA methods to be used or stakeholder processes to be developed.This would facilitate the application of MCA by providing a generic comprehensive framework based on which policymakers and analysts could customize the method to their particular appraisal needs.NOTE 1.A subset of criteria S is called preferentially independent, if the preference between options differing only on criteria from S does not depend on their evaluation on the remaining criteria.For instance, comfort and fuel consumption might not be preferentially independent from price because the importance of comfort may increase with price (Marichal and Roubens 2000).

••••
Absolute robust conclusion, in other words, a statement referring to one option only, which is valid for all admissible instances of the MCA model parameters, for example, 'option a has the utility U(a).0.5';Relative robust conclusion, in other words, a statement referring to one option in relation to others, and which is valid for all admissible instances of the MCA parameters, for example, 'option a has a better rank than option b' or 'option a has the best rank'.