Indicators: tools for informing, monitoring or controlling?

Today, indicators are produced and used worldwide; across all levels and sectors of society; by public, private and civil society actors; for a variety of purposes, ranging from knowledge-provision to administrative control. While the use of quantitative data as policy support, including policy formulation, has a long history, recent decades have seen the rise of what some have called an ‘indicator industry’ (for example, Hezri and Hasan 2004), focused especially on the production of environmental and sustainability indicators, within a framework variously called ‘governance by numbers' (Miller 2001; Lascoumes and Le Gales 2005; Jackson 2011), ‘management by numbers’ in public service (for example, Hood 2007) or ‘numbers discourse’ (Jackson 2011, p. 23). Indicators are generally expected to enhance the rationality of policymaking and public debate by providing a supposedly more objective, robust, and reliable information base. Indicators can operate as ‘boundary objects’ (for example, Turnhout 2009; Star 2010), catering to both technocratic and deliberative ideals, by combining ‘hard facts’ and modelling with collective reasoning and ‘speculation’. 
 
Research and development work in the area has hitherto overwhelmingly concentrated on improving the technical quality of indicators, while the fate of indicators in policymaking and the associated sociopolitical aspects have attracted little attention. This chapter focuses on this neglected area of indicator research, by providing an overview of the multiple types of existing indicators, as well as their use and influence in various venues of policymaking. Empirical examples are drawn mainly from the fields of environmental and sustainability indicators.

p. 8), science, technology and innovation (STI) indicators in the 1950s (Godin 2003), and since the 1980s, performance management indicators -as an essential element of New Public Management and evidence-based policy -today most widely applied in the UK through sectoral performance indicator systems, league tables and rankings at various governance levels (Hood 2007, p. 100;Le Galès 2011;Jackson 2011, p. 17).Since the 1970s, national statistics offices and international organizations (especially the OECD) have pioneered the development of environmental and natural resource indicators, intended to support 'state of the environment' reporting, various types of assessment, multilateral environmental agreements (MEAs) and the development of environmental policy instruments (OECD 1991;Pintér et al. 2005, p. 2;Hezri 2006, p. 161).Most recent developments include the evolution of environmental indictors towards interdisciplinary and cross-sectoral approaches (Hezri 2006, p. 162), the introduction of sustainable development indicators at various levels of governance, and the proliferation of various composite indicators of sustainability, societal progress and wellbeing (for example, Stiglitz et al. 2010;Sébastien and Bauler 2013;Seaford 2013).
Research and development work in the area has hitherto overwhelmingly concentrated on improving the technical quality of indicators, while the fate of indicators in policymaking and the associated sociopolitical aspects have attracted little attention.This chapter focuses on this neglected area of indicator research, by providing an overview of the multiple types of existing indicators, as well as their use and influence in various venues of policymaking.Empirical examples are drawn mainly from the fields of environmental and sustainability indicators.
The remainder of this chapter is structured as follows.Section 2 outlines the different types of indicators and their intended functions, with particular emphasis on their role in policy formulation, and distinguishing between the concepts of use and influence.Section 3 looks at the actual practice, that is, the empirical evidence concerning the roles that indicators actually play in various policy venues.The section first examines the extent to which indicators fulfil their intended functions, and then turns to the broader, unintended consequences that indicator work has in society.Section 4 concludes.

TYPES AND PURPOSES OF INDICATORS
Indicators constitute a heterogeneous policy tool, with a range of purposes, functions, disciplinary backgrounds, application areas and levels, and theoretical and normative underpinnings.An often-cited definition perceives indicators as 'variables that summarize or otherwise simplify relevant information, make visible or perceptible phenomena of interest, and quantify, measure, and communicate relevant information' (Gallopin 1996, p. 108).Jackson (2011, p. 15), in defining a performance indicator as an 'unbiased estimate of true performance which cannot be measured directly', captures two essential features of indicators, namely that of 'indication', entailing the idea that an entity that is not directly measurable can nevertheless be 'assessed using a limited set of measurable parameters' (Turnhout 2009, p. 403), and that of 'signalling' -an indicator needs to be interpreted and given meaning (Jackson 2011, p. 15).According to Gudmundsson (2003, p. 4), the existence of an underlying conceptual framework distinguishes indicators from data or statistics.Such a framework determines the criteria and logic for the choice of specific indicators, anchors indicator systems in theory and ensures comparability and communicability (Gudmundsson 2003, p. 4;Pintér et al. 2005, p. 16).Godin (2003, p. 681) highlights the early warning trend-observation functions, while Jackson (2011, p. 24) underlines the imprecision inherent in indicators.Views diverge on whether indicators should necessarily be underpinned by a causal model (Godin 2003, p. 681;Cobb and Rixford 1998), or whether indeed indicators differ from evaluations in that only the latter necessarily seek to establish cause-effect relationships (Gudmundsson 2003, p. 2).Finally, Gudmundsson (2003, p. 4) evokes the objective of utilization as a defining characteristic of indicators, and distinguishes three alternative 'utilization frameworks', which classify indicators according to their function as providing information, monitoring or control.Information frameworks entail descriptive indicators, monitoring frameworks are designed to provide regular feedback through a combination of descriptive and performance indicators (for example, OECD Environmental Performance Indicators and the EU Lisbon Process competitiveness indicators), while control frameworks entail a stronger link to action through, for example, resource allocation and the associated sanctions.The question of indicator functions will be addressed in more detail later in this chapter.The next section will suggest a typology of different indicators.

Types of Indicators: Descriptive, Performance and Composites
A distinction can be made between descriptive, performance and composite indicators.Descriptive indicators 'can be dichotomous, number, grade, time series, or ratios or other derived functions', and indicate the state of a system (for example, the environment), while leaving specific policy interpretations aside (Gudmundsson 2003, p. 3).The absence of explicit interpretations obviously does not imply neutrality or objectivity.Performance indicators compare indicator values against a standard, target value or benchmark, measuring how well 'someone' is performing, thereby implying that this 'someone' has agency, that is, capacity to influence the course of events.Performance indicators can concern policy inputs, processes, outputs, outcomes, effectiveness or efficiency (Carter et al. 1993).By their very nature, performance indicators therefore already entail a specific type of intended use.Hood (2007, pp. 100-101) further distinguishes target systems, designed to measure performance against an aspirational standard and to help raise levels of performance; ranking systems that compare performance of a given unit with that of another, similar unit; and intelligence systems that do not rank or compare to a standard, but aim merely to build a knowledge base.
Finally, the production of composite indicators 1 that aggregate a series of individual indicators into one or a few numbers, on the basis of an underlying model of the multidimensional concept that is being measured (for example, Grupp and Schubert 2010), especially when single indicators cannot capture the richness of a multidimensional concept, has greatly expanded in the past years.While GDP remains the hegemonic composite indicator, recent efforts have concentrated on developing alternative indicators of sustainability, progress and wellbeing as well as an increasing variety of league tables and rankings of countries, public services, and so on (for example, Pintér et al. 2005).The constitution of composite indicators presents methodological challenges relating to choice, weighting and aggregation.A 'milder' variant of composite indicators are 'headline indicators' -a selection of key indicators in a given policy domain, designed to communicate in a concise manner to high-level policymakers and the general public the essence of progress towards main policy objectives.

Intended Functions of Indicators
As a specific means of operationalizing the concept of evidence-based policy, indicators can serve multiple functions, in particular those of communication and awareness raising (Rosenström and Lyytimäki 2006, p. 33), monitoring and evaluation of performance, supporting policy evaluation, early warning, political advocacy, control and accountability, transparency, and improving the quality of decisions.Further functions attributed to indicators include guidance to policy analysis and formation, improvement of government effectiveness (Moldan and Billharz 1997), setting targets and establishment of standards, promotion of the idea of integrated action, and focusing of policy discussion (Briguglio 2003).Indicators can serve as 'signals' that enable or prescribe an action or management function, and condense information in situations characterized by complexity (Gudmundsson 2003).Seen from such an instrumental The generic function of performance indicators is to strengthen accountability, in particular by helping to ensure an efficient and appropriate use of public money in the pursuit of commonly agreed societal goals.In practice, performance indicators are also expected to serve functions typically attributed to policy evaluation, such as learning, improvement, and 'symbolic' functions (Table 4.1).
Composite indicators are expected to focus attention on important policy issues, offer more rounded assessments of performance, and present the 'big picture' in a manner accessible to a range of audiences -in contrast with the potentially contradictory information provided by indicator sets that examine a phenomenon from multiple perspectives.Rankings and league tables can be used to signal quality of service and inform choice; for performance benchmarking, accountability and resource allocation; or the attribution of rewards (Jackson 2011, p. 20).Precisely because of the simplification inherent in their construction, composites cannot identify causal relationships and alone provide a sufficient knowledge basis for specific policy decisions (for example, Grupp and Schubert 2010, p. 77).The composites therefore can influence policy indirectly, by informing the public and the political debate about specific social objectives and policy trade-offs, making explicit the underlying assumptions, challenging the dominant models of measurement, helping the public to hold politicians to account, and so on (Seaford 2013).Illner's (1984) typology of indicator types and functions at different stages of a policy cycle is one among the many attempts to determine the expected and potential roles of indicators in policymaking (Table 4.2).Seaford (2013) has identified the potential roles that composite indicators of subjective wellbeing could play at different phases of a policy cycle (Figure 4.1).
Seaford emphasizes the largely indirect political and conceptual functions such as public accountability, agenda-setting and assessment of policy objectives, while Illner's account stresses more the direct and instrumental functions of indicators.
The level of governance (a key aspect of policy venue -see Chapter 1, this volume) decisively shapes the appropriate underlying framework, type (descriptive, performance, composite), and expected functions of the indicators in question.For instance, the various national-level composite indicators of sustainable development, environment and wellbeing have their counterparts at the sub-national and community levels, yet the functions of these indicators differ.While both seek to raise awareness, the community-level indicators aim at empowering communities and citizens.Performance measurement indicators, in turn, are typically designed to facilitate control of the civil service by decision makers at the relevant governance level -and to enable the civil service to improve its day-to-day performance.

Indicators and Policy Formulation
Many of the expected functions of indicators fall outside of the scope of policy formulation as defined in Chapter 1: in particular composite and headline indicators are designed to influence phases preceding policy formulation, notably agenda-setting and problem identification.In policy  Indicators also help to shape perceptions of which policies are deemed viable and relevant, that is, the identification of policy options.For instance, the choice of the parameters to constitute an air pollution index, or the choices of the appropriate biodiversity or climate change indicators shape the range and viability of alternative policy responses.Indicators are frequently used in assessing and comparing potential policy options.Finally, while indicators are not designed to recommend and/or propose a specific policy design, in reality they are frequently used to justify a given (often a pre-existing) policy design.

Instrumental, Conceptual and Political Functions of Indicators
The types of intended use of indicators often tend to focus on what the knowledge-utilization literature terms instrumental use (for example, Weiss 1999), entailing in our case the use of indicators as direct input to specific decisions, in line with the linear rational-positivist model of policymaking, typically involving 'single-loop' learning concerning the consequences of specific actions or policy options (Argyris and Schön 1978).Expectations concerning performance management indicators typically fall within this category.Yet many of the intended functions mentioned above can be considered as conceptual, as indicators are expected to constitute a part of a broad information base for decisions, shape conceptual frameworks and mental models of actors, and ultimately generate 'enlightenment' (Weiss 1999).Hence, indicators should foster especially the more complex types of social learning in the spirit of Habermasian 'communicative rationality'.Finally, some of the expected functions are political, especially when indicators are expected to influence agenda-setting and problem definition, highlight neglected issues, or (de)stabilize and (de)legitimize prevailing frameworks of thought.
Often the political use of indicators is overlooked and portrayed in a negative light, as misuse, abuse, attempts to conceal, cheat, delay and manipulate (for example, Hezri 2006).Alternatively, the absence of proof that indicators have influenced policy is often taken as a proof of failure.'Indicator advocates' hence often regret the fact that indicators are either ignored or used selectively, or 'misused' for strategic (in other words, illegitimate) purposes, without due regard for their inherent technical limitations.Numerous measures are then often suggested to minimize 'misuse', such as informing and educating the users or the introduction of 'statistical health warnings' (for example, Jackson 2011, p. 20;Pintér et al. 2005, p. 7;Hood 2007, p. 100;Grupp and Schubert 2010, p. 76).But the 'political' use of indicators can involve more than outright legitimization of decisions already made and 'symbolic' use.It can also entail necessary efforts to strengthen the legitimacy of democratic decision making, or advocacy for socially progressive objectives (Parris and Kates 2003).Regarding the former, Stirling (2006) qualifies this potentially constructive type of legitimization as 'weak' justification, targeted at legitimizing processes and/or institutions, whereas 'strong' justification would focus on justifying substantive policy outcomes (in other words, ex post legitimization of decisions that have been made on other grounds than those explicitly mentioned).Political use and functions of indicators can involve double-or triple-loop learning, including 'political learning', which concerns the 'political feasibility of a given idea or prospects for advancing a given problem through manoeuvring within and manipulation of policy processes' (Hezri 2006, p. 101).
As Table 4.1 (Behn 2003) suggests, the roles of indicators extend well beyond the direct policy formulation tasks, covering in particular crucial feedback functions of evaluation, control, learning, motivation and even various 'symbolic' functions.Such functions are, however, de facto rather than intended consequences of indicators.The following sections will review the empirical experience concerning the actually observed and potential unintended consequences of indicators.

INDICATORS 'IN PRACTICE': ARE THEY USED AND DOES 'USE' IMPLY INFLUENCE?
The discussion above focused merely on the intended, desired objectives of indicators -their 'legitimate' functions.However, the actual use, influence and broader consequences of indicators in practice often prove to be quite different from those foreseen by their designers and advocates.These well-intentioned and as such necessary recommendations reflect a limited perception of the political role of indicators, and hence often turn out ineffective.This section will set aside the issue of the 'correct' use of indicators, and adopt a less normative perspective, by examining the various ways in which indicator work in reality influences policy formulation.

Indicator Use and Indicator Influence
The discussion above focused on the use and intended functions of indicators, yet a first step towards analysing the broader roles of indicators in policymaking is to distinguish between (1) the use of indicators, that is, their handling (for example, receiving, processing, communicating and reporting) in a variety of policy venues, and (2) the influence on policy formulation processes stemming from the indicators or indicator sets, or from the processes through which indicators are developed or applied.
Indicator influence can concern the targeted policy or broader processes in society, such as administrative structures or the operation of democratic institutions.It can entail new or reconfirmed decisions and actions, shared understandings, and networking among, or changes in, the legitimacy of policy or policy actors (Valovirta 2002;Hezri and Dovers 2006;Pollitt 2006;Zittoun 2006;Lehtonen 2013).Indicators are not always used as intended, and the resulting influence may conflict with the objectives sought, or produce negative unintended effects (for example, Perrin 1998;2002;Jackson 2011).Use is therefore not always a 'good thing', nor is the learning entailed in indicator work automatically desirable or undesirable per se.

The (Lack of) Intended Use of Indicators -and Ways of Enhancing Use
The degree to which indicators are used for their intended purposes varies greatly across indicator types and policy areas.In particular, there seems to be a rather strong dichotomy between the wide use of established economic (for example, GDP, unemployment rate, levels of government debt and budget deficits) and performance management indicators on one hand, and the far more infrequent use of the various sectoral, cross-sectoral and sustainability indicators on the other.Government performance measures are certainly used for their intended control purposes.This can take place in internal venues, and draw on official sources of knowledge, when central government departments and agencies use performance indicators to allocate resources, or public service managers to motivate employees to improve performance; to trigger corrective action; to compare, identify and encourage 'best practice'; to plan and budget.The use of indicators by auditors and regulators to evaluate the provision of value for money by public sector organizations can, in turn, be defined as use in external venues, drawing on a variety of official and unofficial knowledge sources.
The degree to which the various sectoral indicators (including for example, indicators for monitoring national sustainable development strategies) are used varies widely.Perhaps most frequent is the use of these indicators in mandatory reporting exercises by government departments, which may take place in either internal (for example, annual sectoral reporting, or public sector performance measurement) or external venues (for example, obligatory EU policy assessments, OECD country reviews).These exercises draw mainly on official sources of knowledge, including those produced by international organizations.
The 'alternative' composite indicators of progress, wellbeing and sustainable development, in turn, are actively used in particular by their producers and policy advocates in order to promote their preferred worldviews, in other words, in venues external to the government, drawing on unofficial data sources.The uptake of such indicators by national and EU-level administrations in their daily work and decision making is far less widespread, probably largely due to the 'unofficial' status of the data underpinning the indicators.Some composite environmental and sustainability indicators, in particular the ecological footprint, have found a certain echo in the media and to a limited extent in public debate (for example, Morse 2011).The recent and ongoing effort by various governments -including collaboration with national statistics officesto develop 'official' alternative indicators of progress and wellbeing (for example, Seaford 2013; Sébastien et al. 2014) marks a shift in this indicator work towards the internal-official quadrant of the scheme in Chapter 1.However, the main expectation is that these indicators operate in the external venues, through public debate, and subsequent uptake by policymakers (for example, Seaford 2013).The extent of actual use and influence of these indicators still remains uncertain, not least because of the frequent doubts about their scientific credibility and technical robustness (Sébastien et al. 2014).Finally, in many cases indicators are not used, simply because the potential users are not even aware of their existence -a phenomenon that also obtains for indicators within the internal-official quadrant (for example, Lehtonen 2013).
The hope that users would consider for instance sustainability or environmental indicator sets in their totality, reflecting upon the trade-offs between the various indicators, has proven largely illusory.Especially in external venues, indicators are used selectively, interpreted out of their context, used as political ammunition rather than as a rational input to policy, or simply ignored.This is often a combined result of attributes relating to the indicators themselves, the actor 'repertoires' -'stabilized ways of thinking and acting (on the individual level) or stabilized codes, operations and technology (on other levels)' (van der Meer 1999, p. 390)and the broader policy context.Relevant factors may include excessively loose linking between reporting schemes and policymaking; lack of trust of potential users in the indicators (government actors may be institutionally  Cash et al. 2002).The relationships and determinants of salience, credibility and legitimacy are complex, and there are obvious trade-offs between the three criteria.For example, the frequent debates and disputes concerning the validity of rankings conducted by international organizations illustrate the vagueness and fluidity of the distinction between 'official' and 'unofficial' sources of knowledge.
The temporal aspects are also vital in determining indicator use.Frequent complaints by potential users include the lack of timely, upto-date indicator information (for example, Rosenström and Lyytimäki 2006) and the claim that backward-looking indicators are not useful in policy formulation -hence the greater appeal of forward-looking policy formulation tools such as cost-benefit analyses (see Chapter 7, this volume) and scenarios (see Chapter 3, this volume; Lehtonen 2013).
These perceived indicator qualities, in turn, are strongly shaped by the process of indicator production -the extent to which the actors participating in indicator processes are seen as legitimate and credible.Collaborative processes of indicator development may foster agreement on problem definitions, policy objectives and policy measures (Bell et al. 2011).In line with findings from evaluation research, the process of indicator production -through social learning, networking, problem framing, focus and motivation -is often equally or even more influential than the 'use' of the final, published indicator (for example, Mickwitz and Melanen 2009;Lehtonen 2013; see also Chapter 2, this volume).
Among the factors relating to the policy setting, those that shape indicator use include the existence (or absence) of an 'indicator culture', the weight of the policy area in question among policy priorities (for example, Sébastien et al. 2014), and the degree of agreement among key actors on problem definitions, policy objectives and policy measures (for example, Turnhout et al. 2007;Bell et al. 2011, p. 108).Use tends to be enhanced when the policy agenda in question has remained stable over time (Bell et al. 2011, p. 10), yet situations of crisis can open 'windows of opportunity' for enhanced indicator use, as the prevailing institutions and frameworks of thought are called into question (Hezri 2006, p. 172).
Table 4.3 presents a number of selected examples of indicators and their intended and actual use, classified according to the distinctions between internal and external venues, and between official and unofficial sources of knowledge.

Institutionalization, Codification and Mandatory Use
Institutionalization through the integration of new indicators into mainstream policy mechanisms and existing statistical, measurement and reporting systems is frequently seen as a key objective and success criterion and as a means of enhancing indicator influence (for example, Pintér et al. 2005, p. 3).Typically, institutionalization involves processes whereby the credibility of hitherto unofficial indicator information is strengthened by giving it an official 'seal' of credibility.Mandatory application of indicators can foster such institutionalization.Mandatory use in internal venues may also trigger use in external venues, through public debate generated for example as a result of the publication of government reviews.
In practice, such spillover effects have been rare.Indicator production and use have been institutionalized through the establishment of guidelines, mandates and designated venues for their production and use, with international organizations (for example, the OECD, Eurostat and the various UN organs) and processes (for example, Local Agenda 21), and national statistics offices in leading roles (Srebotnjak 2007;Stiglitz et al. 2010).Forms of institutionalization have ranged from the establishment of academic journals (for example, Social Indicators Research, Ecological Indicators), and regular international expert collaboration (Hezri 2006, p. 158), codification -and hence 'officialization' -of indicators through frameworks such as the System of National Accounts (SNA) and the System of Integrated Environmental and Economic Accounts (SEEA) (for example, Pintér et al. 2005, pp. 22-23), and manuals for indicator production (Godin 2003, p. 687).

Indicator Constituencies?
The creation of an 'indicator industry', and the associated codification and institutionalization of indicators, has been decisively fostered by groups advocating the use of their favourite indicators.These groups can alternatively be perceived as 'instrument constituencies' (Voß and Simons 2014), epistemic communities (Haas 1992) or advocacy coalitions (Sabatier 1988).
Statisticians, especially at national statistics offices and international organizations, still play a central role in such groups, yet the more recent processes such as the development of community-level and composite sustainability indicators and alternative indicators of progress have seen an increasing involvement of actors outside the government, for example think tanks, NGOs and grassroots community groups (for example, Sébastien and Bauler 2013;Sébastien et al. 2014).Indicator development processes tend to be highly sector-specific, variously led either by users or producers, and often highly international in nature.Depending on their status in policymaking hierarchies, such constituencies of like-minded experts and policy actors not only foster institutionalization of the indicator systems, but also shape the extent to which a sector or an organization develops an 'indicator culture'.Furthermore, the use of especially sustainable development indicators is often confined to an 'inner circle' of indicator producers and the obligated users of the indicators (Rinne et al. 2013), hence breaking the clear-cut distinction between the 'users' and 'producers' of indicators introduced above (for example, Bell et al. 2011;cf. Pintér et al. 2005, p. 18).

Beyond Intended Use: Unanticipated Consequences of Indicators
A central lesson from research on the role of indicators in policymaking is that their policy influence mostly stems not from direct use in policymaking to guide decisions, but instead from the multiple forms of indirect and largely unintended and uncontrollable pathways, best categorized as 'conceptual' and 'political' influence.The following brief survey will focus on three themes that have emerged as central in research concerning the influence of indicators: the theoretical approaches applied for examining influence, the debate on the broader societal impacts of performance indicators, and the 'paradoxes' concerning the use and influence of indicators.

Theoretical Approaches
A number of theoretical approaches have been suggested for the analysis of the broader, indirect roles of indicators.These include scholarship on governmentality (for example, Rydin 2007), 'government/management by numbers' (Lascoumes and Le Galès 2005;Hood 2007), and indicators as boundary objects capable of connecting science, policy and society (Turnhout et al. 2007;Star 2010;Sébastien et al. 2014).A primary criticism brought forward by these strands of literature is the tendency of indicators to 'depoliticize', that is, to reduce value conflicts and normative debates to supposedly neutral and commonly agreed numbers perceived as incontestable facts (Jany-Catrice 2010, p. 95).Urban studies (sociology, geography and urban planning) have called into question the presumed ability of indicators to foster socially desirable objectives, and highlighted the inseparability of indicator systems from the broader dynamics and trends in policymaking (for example, Hezri 2006, pp. 159-160;Rydin et al. 2003;also Rydin 2007).Le Galès (2011) highlights the 'revolutionary' consequences from indicator systems, which engender behaviours that conform to the demands of market society.Zittoun (2006) hence refers to the processes of 'instrumentation', whereby indicators embody power, by virtue of their participation in the processes of problem formulation and the design of problem solutions (see also Lascoumes and Le Galès (2005, p. 12), who describe instrumentation of public policy as the whole of the problems generated by and involved in the choice and use of instruments -techniques, operational modes, policy instruments -that make it possible to materialize and operationalize government action).Through simplification, indicators make problems accessible to non-experts, while at the same time legitimizing the power of experts as the only ones capable of truly 'mastering the numbers'.
The distinction between rationalist-technocratic and constructivistinterpretive models of policymaking is arguably even more pronounced in the case of indicators than in, for instance, evaluation and assessment practice.This is due to the quantitative and presumably accurate nature of indicators on one hand, and their ambition towards policy relevance on the other (for example, Rametsteiner et al. 2011).Hence, indicators are expected to 'close down', enabling better management and control by providing robust, accurate, quantitative and unambiguous information for the purposes of political advocacy and day-to-day policymaking, but they are increasingly also seen as a means of 'opening up' via the highlighting of uncertainties, trade-offs and neglected issues in policymaking (for example, Stirling 2008;Rafols et al. 2012; see also Chapter 2, this volume).Such a 'challenge function' is inherent in alternative indicators of progress, for example.The 'science-driven' statisticians who often drive indicator development are typically reluctant to abandon what they consider a 'nonpolitical', objective and science-based position (for example, Srebotnjak 2007), while the indicator users call for policy-relevant, rough-and-tumble indicators.At least implicitly, policy formulators often tend to adhere to a 'science-driven' and 'apolitical' perception of indicators (Rametsteiner et al. 2011), seeing the involvement of politics in indicator work as undesirable (for example, Lehtonen 2013), while the central role of statisticians in indicator development further accentuates the dominance of the rationalist -technocratic perspective.

THE INFLUENCE OF PERFORMANCE MEASUREMENT: PARADOXES AND DILEMMAS
The evaluation literature has provided plenty of useful lessons concerning the broader impacts of performance measurement -of which indicators constitute an essential element.For example, lessons from the literature concerning the impact of public sector performance measurement on decision making are far from conclusive (Hezri 2006, pp. 156-157).Norman (2002) characterizes the debate as a battle between three groups: the 'true believers' who highlight benefits such as new investment in data capture, harmonization of measurement methods across institutions, and behavioural changes; the 'critics and doubters' who stress problems in the use, interpretation and societal relevance, lack of political will, bureaucratic inertia, and use of indicators for propaganda purposes; and the 'pragmatic sceptics' (for example, van der Knaap 2006) who see active contestation as a sign of an evolution towards better theory and practice.As a counterpoint to the promise that indicators would provide greater accountability, efficiency and citizen control over policymakers, there is a considerable body of literature highlighting numerous negative features of performance measurement.These can be summarized as follows (for example , Perrin 1998;2002;Blalock 1999;Davies 1999;van der Knaap 2006;Hood 2007;Jackson 2011; Le Galès 2011):         • potential systemic effects: loss of public trust, risk of a system collapse (Hood 2007, p. 102).
The problems of performance measurement can be seen as a subset of the more generic paradoxes and dilemmas involved in indicator work.Hence, it is precisely the widespread use and institutionalization of performance indicators -policy 'success' -that accentuates their risks and downsides.
The absence of a linear connection between use and influence represents an example of the many paradoxes, dilemmas and trade-offs involved in indicator work.These include tensions between:   • international comparability and national/regional/local relevance;  • academic and practitioner emphasis, in other words, whether the quality of an indicator should be defined by the scientific quality or practical usefulness of the indicator (Cobb and Rixford 1998, pp. 3-4;Rosenström and Lyytimäki 2006).
Four further tensions merit particular attention:    The intended functions of indicators cover most of the policy formulation tasks identified in Chapter 1, but extend beyond policy formulation in the strict sense, from agenda-setting and problem formulation in the 'upstream' stages to ex post evaluation and monitoring in the 'downstream'.Indicators constitute an 'auxiliary' policy formulation tool typically applied in conjunction with other tools: both ex ante assessment and ex post policy evaluation make wide use of indicators; scenario-building draws increasingly upon 'forward-looking' indicators to characterize and assess the impacts of alternative scenarios (see Chapter 3); and participation of the relevant stakeholders (see Chapter 2) in indicator development has been repeatedly pinpointed as crucial if indicators are to be relevant for their intended users.
In practice, the high hopes concerning the ability of indicators to rationalize policymaking and change policy have often remained unfulfilled.Two contrasting experiences can be identified: the performance management indicators have clearly been directly used for control and management, often as part of mandatory monitoring, reporting, assessment, evaluation and performance measurement frameworks, many of which are internal to the government (or intergovernmental processes) and draw upon 'government-certified' information sources.By contrast, Markku Lehtonen -9781783477043 Downloaded from PubFactory at 09/15/2023 10:23:01PM via free access the voluntary use and media uptake of various sectoral and 'alternative' indicators of progress (sustainability, wellbeing, and so on), in venues outside of the government, have been far more rare and unpredictable.The lack of adoption and use cannot be attributed solely to the lack of credibility of 'unofficial' indicators and data sources, because the sectoral indicators typically carry a government 'label' of authority.Furthermore, indicator use often remains within a small circle of 'insiders' and specialists, in venues outside or at the margin of policy formulation in the strict sense.With the exception of performance management frameworks, indicators seldom directly influence policy.By contrast, the true power of indicators as policy formulation tools lies in their indirect, unintended, and partly intractable long-term impacts through learning, political advocacy and systemic effects.Indeed, at times, the greater the use of indicators, especially in the policy venues internal to the government, the weaker the potential of the indicators to challenge the prevailing frameworks of thought and institutional structures.The effects from the use of indicators in such situations are by no means negligible, even though the desirability of impacts such as routinization, conservatism and entrenchment of power structures embodied in the indicator systems may be called into question.
Policymakers and potential indicator users frequently criticize the poor policy relevance of indicators, yet the bulk of the attention in indicator research and development (and especially within the government) focuses on ensuring their scientific credibility.The production of 'alternative' indicators is certainly more driven by concerns for their political usability and relevance, but the debate around these -and the criticism against themmostly addresses questions of technical quality.What is often at stake in controversies concerning 'governance by numbers' are the trust, credibility and reputation of the different organizations producing indicatorsultimately the public trust in science and 'official' expertise.The processes of indicator production usually receive little attention beyond the call for broad participation of stakeholders, as do the potential systemic effects from the application of indicator schemes.While key questions for indicator work concern the most appropriate theoretical frameworks for examining the broader unintended and systemic impacts of indicators, the future of indicators in policy formulation practices remains as uncertain as ever.Indicators will certainly survive as a major type of policy formulation tool, yet uncertainty prevails particularly over the shape and even the survival of performance management frameworks and the increasingly numerous composite indicators, as well as over the persistence of the arguably 'revolutionary' impacts that especially the former have on public policy and organizational culture.Some commentators indeed predict a rather radical transformation of performance management frameworks, arguing that 'techniques such as league tables will probably be abandoned and consigned to the history of policy failures' (Jackson 2011, p. 24). 3 NOTES 1. See also the OECD glossary of statistical terms: http://stats.oecd.org/glossary/detail. asp?ID56278.2. I am grateful to Henrik Gudmundsson for having coined this term.3. The abandonment by the UK's new coalition government in summer 2010 of the highly elaborate government performance management framework developed over the past three decades may represent a test case for the strength of institutionalization and the resilience of the 'indicator culture' created in UK public sector management.

Figure 4 . 1
Figure 4.1 The potential roles of subjective wellbeing indicators at different stages of the policy cycle

Table 4 .
1 Performance indicator functions: different types

Table 4 .
3 Examples of indicators and their use in different policy venues use of indicators as inputs for the design and implementation of public policies versus as tools for monitoring and evaluation; • • The partly overlapping waves of indicator development have closely followed the political and societal agendas of their time.The Great Depression and the needs to manage the war economy stimulated the development of national economic accounting systems, the 'civilization critique' of the 1960s gave rise to the 'social indicators movement', environmental indicators developed together with environmental concern in the 1970s, while neo-liberalism brought along the performance management movement in the 1980s.The sustainable development indicator work in the wake of the Rio Conference in 1992 has been followed by a new kind of growth criticism, in the form of alternative indicators of progress and wellbeing.While the different indicators vary both in their form (descriptive, performance or composite) and specific purpose (monitoring, control, awareness-raising, advocacy, knowledge-production), they share the objective of providing a better, simpler and less ambiguous yet scientifically robust knowledge base for decision making.Largely due to its double ambition of policy relevance and scientific robustness, indicator work is typically characterized by a range of tensions and ambiguities, notably between the attempt to 'close down' by reducing ambiguity, and 'open up' via highlighting uncertainties, and challenging of established frameworks of thought and power structures.