The tools of policy formulation : an introduction

What techniques or means do public policymakers use in their attempts to achieve policy goals? The roles of what may be termed policy instruments, tools and methods (Howlett 2011, p. 22) have attracted a great deal of attention. It is generally accepted that policy tools and instruments exist at all stages of the policy process (Howlett 2011, p. 22), ranging from policy formulation through to ex post evaluation (Dunn 2004). But in the public policy literature, much of the debate has focused on instruments for implementing agreed policy objectives, such as regulations, subsidies, taxes and voluntary agreements (Hood 1983; Hood and Margetts 2007; Salamon 2002). Recently, a second category of implementing instruments has been identified: procedural tools (Howlett 2000). These include education, training, provision of information and public hearings. These are procedural in the sense that they seek to affect outcomes indirectly through manipulating policy processes. The manner in which both types of instruments are selected and deployed aims to change the substance, effects and outcomes of policy, by sending signals about what is to be achieved and how government is likely to respond to target groups. Understanding these processes is critical to a better understanding of governing activities. Adopting an ‘instruments perspective’ on these activities has arguably contributed significantly to the study of public policy and governance in general (Lascoumes and Le Galés 2007). There is, however, also a third category of policy tools and instruments which has largely remained outside the mainstream of policy research.1 These tools have typically been developed by researchers and policy practitioners with the aim of performing a rather different set of tasks to the implementing instruments described above. They are variously referred to as ‘analytical tools’ (Radin 2013, p. viii), ‘policyanalytic methods’ (Dunn


INTRODUCTION
What techniques or means do public policymakers use in their attempts to achieve policy goals?The roles of what may be termed policy instruments, tools and methods (Howlett 2011, p. 22) have attracted a great deal of attention.It is generally accepted that policy tools and instruments exist at all stages of the policy process (Howlett 2011, p. 22), ranging from policy formulation through to ex post evaluation (Dunn 2004).But in the public policy literature, much of the debate has focused on instruments for implementing agreed policy objectives, such as regulations, subsidies, taxes and voluntary agreements (Hood 1983;Hood and Margetts 2007;Salamon 2002).Recently, a second category of implementing instruments has been identified: procedural tools (Howlett 2000).These include education, training, provision of information and public hearings.These are procedural in the sense that they seek to affect outcomes indirectly through manipulating policy processes.The manner in which both types of instruments are selected and deployed aims to change the substance, effects and outcomes of policy, by sending signals about what is to be achieved and how government is likely to respond to target groups.Understanding these processes is critical to a better understanding of governing activities.Adopting an 'instruments perspective' on these activities has arguably contributed significantly to the study of public policy and governance in general (Lascoumes and Le Galés 2007).
There is, however, also a third category of policy tools and instruments which has largely remained outside the mainstream of policy research. 1These tools have typically been developed by researchers and policy practitioners with the aim of performing a rather different set of tasks to the implementing instruments described above.They are variously referred to as 'analytical tools' (Radin 2013, p. viii), 'policy-analytic methods' (Dunn 2004, p. 6), decision support tools or 'analycentric' tools (Schick 1977).Radin rightly devotes a whole chapter of her book charting the development of the field of policy analysis to telling their story -on the grounds that they constitute the 'tools of the [policy analysis] trade' (Radin 2013, p. 143).
From Radin's and others' accounts it soon becomes clear that what we shall term policy formulation tools 2 come in many different shapes and sizes.Initially, they were designed to support a very specific task, namely the 'collection of as much information and data as were available to help decision makers address the substantive aspects of the problem at hand' (Radin 2013, p. 23).Nowadays, these tools are regarded as a means to address many other policy formulation tasks, for example understanding the nature of policy problems, estimating how they might change over time and clarifying or even eliminating some of the many possible policy response options.In fact, to understand these tools fully, we argue that policy researchers must view them in the context of the broader activities and processes of policy formulation.
Policy formulation is a very different activity to policy implementation.It is an important phase devoted to 'generating options about what to do about a public problem' (Howlett 2011, p. 29), and is inherent to most, if not all, forms of policymaking.If the agenda-setting stage in the well-known policy cycle is essentially concerned with identifying where to go, the policy formulation stage is all about how to get there (Hill 2009, p. 171).If policy formulation is 'a process of identifying and addressing possible solutions to policy problems or, to put it another way, exploring the various options or alternatives available for addressing a problem', then developing and/or using policy formulation tools is a vital part of that process (Howlett 2011, p. 30).We suggest that, much more than for other policy stages, it is very hard to conceive of policy formulation -let alone properly study it -without thinking in terms of tools.Based on Dunn (2004), these include tools for forecasting and exploring future problems through the use of scenarios, tools for identifying and recommending policy options (for example, cost-benefit, cost-effectiveness and multi-criteria analyses) and tools for exploring problem structuring or framing (for example, brainstorming, boundary analysis and argumentation mapping).
In recent years, the number of potentially deployable policy formulation tools has expanded massively (for an indication of what is currently in the toolbox, see Dunn (2004) and Radin (2013, p. 146)).They include types that may be considered to fall into both positivist and post-positivist categories, with the latter inspired by critiques of the role of technocratic analysis and a concern to address subtle influences that act to condition the content of policy, such as material forces, discourses and ideologies (Fischer 1995).Yet, the policy tools and instruments literature remains stubbornly fixated on implementation instruments.And while there are many individual literatures that seek to promote and/or inform the use of specific policy formulation tools, the policy analysis literature is relatively silent on how, why, when, by whom, in what settings and with what effects, the various tools are used in practice.To the extent that they devote attention to formulation as a specific stage in the policy process, most textbooks frame it around understandings of processes, interests and expertise.In many ways, the limited academic treatment that policy formulation tools have received in the period following the Second World War is symptomatic of a wider division in policy analysis between those doing policy research and those engaged in policy practice.For reasons explored more fully below, when it comes to policy formulation tools, practice has arguably run well ahead of research.In this book, we seek to bring these two wings of the policy analysis community into a closer dialogue.
More specifically, in this book we investigate -for the first time -what might be gained by bringing the study of policy formulation tools back into the mainstream of public policy research.The policy instruments literature might lead us to expect each policy formulation tool to impart a specific 'spin' (Salamon 2002) on ensuing policy dynamics.Certain other literatures, such as science and technology studies (Stirling 2008) or planning (Owens and Cowell 2002), also suggest that certain tools serve to influence policy outputs in a variety of ways.For example, use of costbenefit analysis to develop policy has the potential to marginalize concern for equity in some sectors, in favour of outputs perceived as the most efficient use of scarce resources.But does this actually happen in practice, and if so how?At present, the various literatures are too fragmented and too detached from public policy theory to tell us.There has, of course, been a huge amount written on individual formulation tools, often by scholars who have invested a great deal in developing them and advocating their use.They are understandably eager to see them being taken up and used by policymakers.Yet we will show that many tool developers and promoters are often vexed -and sometimes deeply disappointed -by their apparent lack of use, or even outright misuse by practitioners (Shulock 1999).We feel that this is another topic which would benefit from greater interaction between those who (to employ another well-known distinction) analyse for policy, and those who conduct analysis of policy.
We believe that now is a particularly opportune moment to look afresh at policy formulation tools.Policy researchers and analysts are becoming more interested in policy formulation -arguably one of the most poorly understood of all the policy process stages; indeed, there is a growing belief that it may constitute the final, 'missing link' (Hargrove 1975) in policy analysis.Interest in policy design is also re-awakening, partly because of the rise to prominence of ever more complex problems such as energy insecurity and climate change that defy standard policy remedies (Howlett et al. 2014).And having invested heavily in tools in the past, tool promoters and policy practitioners are eager to understand how -and indeed if -they perform in practice.
The remainder of this chapter is divided as follows.The second section takes a step back by examining the main actors, processes and venues of policy formulation in a very general sense.The third section scours the various existing literatures to explore in more detail the development of the various policy formulation tools that could in principle be used in these venues.It also charts the subsequent turn away from these tools in mainstream public policy research, and explores some of the reasons why interest in policy formulation has recently undergone a renaissance.Section 4 explores the analytical steps that will be needed to re-assemble the various literatures into a more coherent sub-field of policy research, revolving around a series of common foci.To that end, we propose a new definition and typology of tools, and offer a means of re-assembling the field around an analytical framework focused on actors, venues, capacities and effects.We conclude by introducing the rest of the book, including our final, concluding chapter.

Actors: Who are the Policy Formulators?
The literature on policy formulation has expanded significantly in the last three decades (Wolman 1981;Thomas 2001;Wu et al. 2010;Howlett 2011).According to Howlett (2011, p. 29), it is the stage of the policy process 'in which options that might help resolve issues and problems recognized at the agenda-setting stage are identified, refined, appraised and formalized'.The process of identifying and comparing alternative actions is said to shape the subsequent stage -that of decision making (Linder and Peters 1990).During the formulation stage, policy analysts will typically have to confront trade-offs between legitimate public demands for action, and the political, technical and financial capabilities to address them.For many scholars, policy formulation is the very essence of public policy analysis, which Wildavsky (1987, pp. 15-16) characterized as how to understand the relationship between 'manipulable means and obtainable objectives'.
But who formulates public policies?It is generally recognized that policy formulation is a critically important but relatively inscrutable stage of the policy process (Wu et al. 2010, p. 47), with many different actors interacting, often under intense and focused political pressure from special advisers, lobbyists and interest groups.There is also a widespread assumption that unlike the agenda-setting stage (in which the media, politicians and the public may be more transparently involved), policy formulation is much more of a political netherworld, dominated by those with specialist knowledge, preferred access to decision makers or a paid position in a particular government agency or department (Howlett and Geist 2012, p. 19).Even though their precise role may be hard to fathom, in principle all may use or seek to use formulation tools.As we shall see, this creates a distinct set of challenges for those (like us) who want to study the use of the tools, or those who wish to design and/or promote them.
In many ways, policy formulation is the stage which the policy analysis community was originally established to understand and inform (Radin 2013, p. 5).Meltsner's (1976) pioneering study of the still relatively inchoate policy analysis community distinguished between analysts with political skills and those with more technical skills.As we shall see, it was the latter that took the lead in developing and applying the first policy formulation tools.The more general literatures have focused on the role of politicians and bureaucrats (Craft and Howlett 2012, p. 80).Pioneering accounts of policymaking (such as Page and Jenkins (2005) and Fleischer (2009)) have, for example, focused on the 'policy process generalists' who rarely, if ever, deal with policy tools in a substantive way and have very little training in formal policy analysis.
More specific studies of policy formulation have sought to offer a more detailed stocktake of the different policy analysts who are typically involved (Howlett 2011, p. 31).Together, these actors are often said to constitute a policy advisory system, comprising: decision makers (chiefly politicians); knowledge producers and/or providers; and knowledge brokers (Howlett 2011, pp. 31-33).Other typologies have differentiated the main participants in relation to their location (in other words, core actors -professional policy analysts, central agency officials and others); and level of influence (in other words, public sector insiders; private sector insiders; and outsiders) (Howlett 2011, p. 33).Precisely who formulates policy is ultimately an empirical question.The point which we wish to make is that it is important to appreciate the variety of actors who might be involved in policy formulation activities, as they might well have rather different motives and capabilities for using particular tools -a matter to which we now turn.

Policy Formulation Processes and Tasks
One of the most common ways to comprehend the process of policy formulation is to break it down into constituent steps or tasks.For Wolman (1981), policy formulation comprises several 'components', each impacting heavily on overall policy performance.In his view, the 'formulating process' starts with the 'conceptualization of the problem' by policymakers (Wolman 1981, p. 435).Like Wolman, Thomas (2001, pp. 216-217) also identifies an initial '[a]ppraisal phase' of data collection where 'critical issues . . .[are] identified' by stakeholders.However, as many commentators have observed, 'problems' themselves are not self-evident or neutral, with Wolman (1981, p. 437) arguing that they may be contested, subjective or socially constructed and may change through time in response to societal values.Problem characterization could therefore be considered to be an extension of the agenda-setting process.Policymakers may select certain forms of evidence to support action on specific issues, or issues themselves may be productive of certain types of evidence (see for example, Kingdon 2010;Baumgartner and Jones 1991).
Having established the existence of a policy problem (or problems) through some form of data collection, the various policy-relevant dimensions of the problem are then evaluated to determine their causes and extent, chiefly as a basis for identifying potential policy solutions.Inadequate understanding at this stage creates a need for what Wolman (1981, p. 437) terms '[t]heory evaluation and selection'.While the point is often made that causation tends to be difficult to precisely establish, Wolman observes that 'the better the understanding is of the causal process . . . the more likely . . .we will be able to devise public policy to deal with it successfully' (Wolman 1981, p. 437).Understanding causation, as Wolman puts it, is also reliant on the generation of adequate theoretical propositions in addition to relevant data on which to support them.For Wu et al. (2010, p. 40) '[u]nderstanding the source of the problem' is an unavoidable part of formulation.They also make the point that rarely is there 'full agreement over . . .underlying causes' (Wu et al. 2010, p. 40).Like initial problem characterization, evaluation of the causes of a problem may thus involve political conflict as different actors seek to apportion blame, reduce their perceived complicity or shape subsequent policy responses in line with their interests.These characteristics strongly condition the type of tools used.
Once a broad consensus has been reached on the nature and extent of the problem(s), policymakers turn to consider appropriate responses.From the initial information gathering and analysis of causes, formulators engage in the '[s]pecification of objectives' (Wolman 1981, p. 438) or '[c]larifying policy objectives' (Wu et al. 2010, p. 40) stage.Initially, this third step of objective specification can involve the determination of the objectives to be met and the timescales for action (Wu et al. 2010).Again, disagreements over objectives can quickly ensue but once they are established, as a fourth step, specific policy options can be assessed and recommendations made on policy design(s).Because any particular problem may have multiple potential solutions, each with differing costs and benefits, these options require comparative assessment to guide decision making.As Howlett (2011, p. 31) puts it, this part of the formulation process 'sees public officials weighing the evidence on various policy options and drafting some form of proposal that identifies which of these options will be advanced to the ratification stage'.
Prior to the adoption of the final policy, it undergoes a fifth stepdesign.Having determined objectives, various means are available for selection from the tool box (for example Howlett 2011;Jordan et al. 2012;Jordan et al. 2013b).Determining the preferred policy mix is central to design considerations.While typologies also abound in the instruments literature, four main categories are evident: regulations; market-based instruments; voluntary approaches; and informational measures (Jordan et al. 2013b).In addition, the instrument of public spending or budgeting may also be identified (see for example, Russel and Jordan 2014).Policymakers select from these instruments according to a range of considerations that are both internal and external to the instrument.This stage of formulation could, according to Wolman (1981, pp. 440-446), consequently involve the weighing-up of several factors: the 'causal efficacy' of the policy; 'political feasibility'; 'technical feasibility'; any 'secondary consequences' resulting from the design; instrument type (regulations or incentives); and the capacity of implementation structures.
As above, all the steps including this one may become deeply contested.After all, the final architecture of the policy could, once implemented, create winners and losers via processes of positive and negative feedback (Jordan and Matt 2014).One means of dissipating distributional conflict throughout the entire formulation process is to engage in what Thomas (2001, p. 218) terms consensus building or 'consolidation', whereby agreement is sought between the various policy formulators and their client groupings.We shall show that a number of tools have been developed specifically for this purpose.But while '[a]nticipating and addressing the . . .concerns of the various powerful social groups is essential', consultation may create associated transaction costs such as the slowing down of policy adoption (Wu et al. 2010, p. 41).A decision can be taken -the subsequent stage of the policy process -once agreement has been reached on the chosen course of action.
These five tasks constitute the standard steps or tasks of policy formulation.During the 1960s and 1970s, when the policy analysis movement was still in its infancy, policy formulation was depicted as though it were both analytically and in practice separate from agenda setting and decision making.It was the stage where policy analysts 'would explore alternative approaches to "solve" a policy problem that had gained the attention of decision makers and had reached the policy agenda' (Radin 2013, p. 23).In doing so, policy formulation could be 'politically deodorized' (Heclo 1972, p. 15) in a way that allowed policy specialists to draw on the state of the art in policy tools and planning philosophies, to ensure that policy remained on as rationally determined a track as possible (Self 1981, p. 222).
As we saw above, and shall explain more fully below, it soon became apparent that the politics could not be so easily squeezed out of policy formulation by using tools or indeed any other devices.It also became clear that some of the formulation tasks could overlap or be missed out entirely.Indeed, policy formulation may not culminate in the adoption of a discrete and hence settled 'policy': on the contrary, policies may continue to be (re) formulated throughout their implementation as tool-informed learning takes place in relation to their operational effectiveness and associated outcomes (Jordan et al. 2013a).As we shall show, many policy analysts responded to these discomforting discoveries by offering ever more strident recommendations on how policy formulation should be conducted (Vining and Weimer 2010;Dunn 2004); notably fewer have studied how it is actually practiced (Colebatch and Radin 2006;Noordegraaf 2011).In the following section we shall explore what a perspective focusing on tools and venues offers by way of greater insight into the steps and the venues of policy formulation.

The Venues of Policy Formulation
Policy formulation -like policymaking more generally -occurs in particular venues.Baumgartner andJones (1991, p. 1045) have termed these 'venues of policy action', going on to define them as 'institutional locations where authoritative decisions are made concerning a given issue' (Baumgartner and Jones 1993, p. 32).More specifically, Timmermans andScholten (2006, p. 1105) suggest that the venues 'are locations where policies originate, obtain support, and are adopted as binding decisions'.
To date, this notion has been explored in most depth within the 'venue shopping' literature on agenda setting; a particular sub-field of policy analysis that examines how interest groups strategically shift their demands for realizing political goals between different venues in multilevel systems of governance (Pralle 2003).Several types of venue have been detected, including, inter alia, within federal, state and local governments plus within international organizations (Pralle 2003), European Union institutions and national governments (Beyers and Kerremans 2012), and various trans-governmental co-operation mechanisms (Guiraudon 2002).Venues can include 'formal political arenas such as legislatures, executives and the judiciary, but also the media and the stock market' and so-called 'scientific venues such as research institutes, think-tanks and expert committees' (Timmermans andScholten 2006, p. 1105).A particular role is also ascribed to the use of scientific evidence by actors to achieve agendasetting demands in venue shopping strategies (Timmermans and Scholten 2006).
On this basis, any attempt to categorize venues for policy formulation should be cognizant of the institutional space itself and, significantly, the type of evidence used.With respect to the former, when examining formulation we can more neatly divide venues by functional power rather than institutional level or actor group.Here, in terms of relative power, it is national government executives that are still arguably dominant globally, despite increasing shifts towards multi-level governance (Jordan and Huitema 2014).To give greater analytical purchase to our conceptualizations we therefore build on Peters and Barker (1993), Baumgartner and Jones (1993) and Timmermans and Scholten (2006), and define policy formulation venues as institutional locations, both within and outside governments, in which certain policy formulation tasks are performed, with the aim of informing the design, content and effects of policymaking activities.
Policy formulation venues can in principle exist at different levels of governance (nation state versus supra/sub-national); and within or outside the structures of the state.There has been much work (see for example Barker 1993;Parsons 1995;Halligan 1995) on classifying policy advice systems, and two dimensions identified therein are particularly important for understanding policy formulation venues more generally.First, are the policy formulation tasks conducted externally or internally to the executive; in other words, where is the task undertaken?For example, internal venues may be populated wholly or mainly by serving officials or ministers and may include departmental inquiries, government committees and policy analysis units (for examples of the latter, see Page 2003).External venues may encompass legislative, governmental or public inquiries and involve non-executive actors such as elected parliamentarians, scientific advisors, think tanks, industry representatives and non-governmental organizations.
Second, are official (executive) or non-official sources of knowledge employed, that is, what knowledge sources do policy formulators draw upon?We distinguish between executive-sanctioned or derived knowledge, and unofficial sources that may include surveys, research which appears as non-formal reports, and the outputs of research networks and public intellectuals.Rather closed processes of policy formulation can occur within internal venues using officially derived evidence, in contrast to more open external venues that draw upon non-official forms of knowledge.
Neither of these two dimensions -well known to scholars of policy advisory systems (Craft and Howlett 2012, p. 87) -are binary.For example, there are varying degrees to which the entirety of a policy formulation task is undertaken internally or externally, and varying degrees to which different types of evidence are employed at different times or for different purposes.We therefore propose to represent them by means of a 232 matrix (Figure 1.1).

The Analycentric Turn in Policy Analysis
As noted above, tools have always had a special place in the history of policy analysis.Modern policy analysis is often held to have developed in earnest from the 1940s onwards (DeLeon 2006).Harold Lasswell's (1971)  'policy sciences of democracy' provided a vision of analysis that drew together different academic disciplines as well as different actors in the policy formulation process -academic, bureaucrat and the person in the street -to address public problems.This was a multidisciplinary endeavour that sought to solve problems in an applied fashion (Dunn 2004, p. 41).While departments of public administration and politics were supposed to supply an understanding of how political and administrative systems operated, the assumption was that the tools of analysis would be produced by technical experts in economics, operations and systems analysis (Dunn 2004, p. 41).
The 1950s and 1960s saw the rise of the professional policy analyst, providing specialist input to policy, and institutions for formalizing such input like the Systems Analysis Unit in the US Defense Department (Radin 2013, p. 14) and, later in the UK, the Central Policy Review Staff, both staffed by experts in the latest tools and methods.The Systems Analysis Unit was charged with implementing one of the very first (and most controversial) systematic policy formulation tools, known as the Programme Planning and Budgeting System (PPBS) (Schultze 1970).The PPBS sought to integrate budgeting and policy development in the quest for greater efficiency and hence more rational decisions.
These tool-driven or 'analycentric' approaches (Schick 1977) initially developed in the fields of defence and budgeting, but from the late 1960s, as the reach of governmental action spread further into fields such as education, health and social care, the scope of analytical activities also expanded (Parsons 1995;Radin 2013, pp. 17-22;DeLeon 2006) almost as a corollary.As Schick (1977, p. 258) observed: 'whenever positive government action has been extended to a new sphere, analytic activity has been sure to follow'.Crucially, the increasingly forceful turn towards analycentric tools and methods embedded a linear-rational approach to analysis of policy problems, in which -to put it simplistically -problems were to be identified and then 'solved' using analytical tools.In his manifesto for the new policy analysis community, Dror (1971, p. 232) famously declared that the 'aim of policy analysis is to permit improvements in decision making and policymaking by allowing a fuller consideration of a broader set of alternatives, with a wider context, with the help of more systematic tools'.
Tools, in other words, were absolutely central to the rapidly emerging field of policy analysis, and were to be taken forward by a new cadre of policy analysts, who operated in small policy analysis units like the Central Policy Review Staff based at the very apex of government.A direct consequence of these developments was a major effort to integrate analytical tools into policy formulation, an activity which until then had, as noted above, been dominated by generalists and those with a legal background (Radin 2013, p. 14).These tools initially drew on techniques from operational research and economic analysis, including methods for assessing the costs and benefits of different policy alternatives, and analysis of interacting parts of complex systems.Tools such as cost-benefit analysis (CBA) and computer models were to be found in the analycentric 'backroom' (Self 1981, p. 222), where political 'irrationalities' could be tempered and policy made more 'rational'.These tools and tool-utilizing skills had originally been developed and honed during the Second World War, but as Radin (2013, p. 14) puts it rather nicely, 'the energy of Americans that had been concentrated on making war in a more rational manner now sought new directions'.The tool specialists found a willing audience amongst politicians and policymakers who were anxious to embark upon new endeavours.

The Turn Away from Policy Formulation Tools
In the Lasswellian perspective, tools were seen as having a central role in the development of an integrated approach that united policy researchers with policy practitioners.But for a number of reasons, things did not quite match up to his vision, and policy formulation tools were gradually marginalized in public policy research and some fell out of favour with policymakers.
First, when used, CBA and integrated forms of planning and budgeting such as the PPBS fell some way short of initial expectations.When the academic backlash came it pushed the study of policy formulation tools back in the direction of the 'cloistered' (Radin 2013, p. 166) backroom of policy research.Tools such as computer modelling and CBA seemed to stand for everything that was bad about positivist and 'technocratic' forms of policy analysis (Goodin et al. 2006, p. 4).Tool specialists were derided as 'econocrats' (Self 1985) and 'whizzkids' (Mintrom and Williams 2013, p. 9).Wildavsky (1987, p. xxvi), never keen on tools even when they were in vogue, viewed policy analysis more as an art and a craft than an exercise in applying 'macro-macho' policy tools such as the PPBS and CBA to solve problems.'The technical base of policy analysis is weak', he continued.'Its strengths lie in the ability to make a little knowledge go a long way by combining and understanding of the constraints of a situation with the ability to explore the environment constructively' (Wildavsky 1987, p. 16).Others critiqued the assumption that using tools would take the politics out of policymaking; in practice, politics all too readily intervened (DeLeon and Martell 2006, p. 33).Why, to put it bluntly, should a bureaucrat perform a sophisticated policy assessment employing state-of-the-art tools, when critical policy decisions had already effectively been made?(Shulock 1999, p. 241).Politics could also intervene more insidiously, through the values embodied and reproduced by particular, ostensibly neutral tools.CBA in particular lost legitimacy in certain policy sectors as a result (Owens et al. 2004), though hung on quite tenaciously thereafter.The very idea that policy analysis should seek to provide analytical solutions for 'elites' was challenged; rather, claims were made that analysts should concentrate on understanding the multiple actors that are involved in policy formulation (Hajer and Wagenaar 2003), and uncover the many meanings that they bring to the process and the framings they employ (Radin 2013, p. 162).So while the academic critique of tools and methods were mostly centred on the most positivist, rational variants (in other words, the PPBS and CBA) (Self 1985), its effect was eventually much more wide ranging and long lasting.
Second, policymakers also began to turn away from centralized, tooldriven forms of policy planning.The abolition of PPBS in the 1970s and of the CPRS in the early 1980s, coupled with the rise of a much more explicitly ideological approach to policymaking in the 1980s, led not to the removal of analysis altogether, but changes in the type and tools of analysis demanded.Thus, the rise of private sector management techniques in running public services (in other words, the New Public Management agenda), coupled with desire to reduce the power and scope of bureaucracy, nurtured a demand for a new set of accounting tools for contracting out public services (Mintrom and Williams 2013).
Third, the mainstream of public policy research had long before turned to other research questions.These focused more on attempts (of which Lindblom (1959) is a classic early example) to better understand the policy process itself, not as a series of stages in which rational analysis could/ should be applied, but as a much more complex, negotiated and above all deeply political process.Others built on the claim that policy formulation was actually not especially influential -that policy implementation, not formulation, was the missing link -and devoted their energies to postdecisional policymaking processes.Meanwhile, after Salamon's (1989) influential intervention, policy instrument scholars increasingly focused on the selection and effects of the implementing instruments.
Finally, the tool designers and developers became ever more divided into 'clusters of functional interest' (Schick 1977, p. 260).The idea of an integrated policy analysis for democracy was quietly forgotten in the rush to design ever more sophisticated tools.Indeed, some have devoted their entire careers to this task, only later to discover that relatively few policymakers routinely use the tools they had designed (Pearce 1998;Hanley et al. 1990).As Schick (1977, p. 262) had earlier predicted, they believed that the route to usefulness was via ever greater precision and rigour -but it wasn't.

The Turn Back to Policy Formulation Tools
Nowadays, interest in policy formulation tools appears to be growing strongly once again, for several reasons.First, new tasks other than knowledge creation are being found for tools such as CBA and indicators.As noted above, they are seen as a means to implement the New Public Management agenda, for example.According to Boswell et al. (Chapter 11,this volume), they seek to incentivize improvements in performance, monitor progress and ensure political accountability.In many OECD countries, tool use has been institutionalized through systems of Regulatory Impact Assessment (Turnpenny et al. 2009;Nilsson et al. 2008).In developing countries (Chapter 10, this volume), tools are being used to rationalize policymaking in situations where the public sphere is still relatively weak, vis-à-vis traditional forms of politics based on patronage.
Second, the emergence of ever more complex policy problems has generated a fresh wave of interest in more sophisticated policy formulation tools such as scenarios and computer-based forms of modelling.There is a growing appreciation amongst practitioners and academics that policies in these areas will not 'design themselves' (Howlett and Lejano 2013, p. 14); according to Lindquist (1992, pp. 128-129), they: need new analytical tools that will help them to diagnose and map the external environments of the public agencies, to recognize the inherent tensions and dynamics in these environments as they pertain to policy development and consensus building, and to develop new strategies for 'working' in these environments in the interests both of their political masters and those of the broader communities they serve.
Tools, in other words, are no longer the preserve of technocrats operating in cloistered backrooms, well away from the public gaze.Unfortunately, there remains a lack of understanding of which tools are being used and how well they are performing in relation to this considerably longer list of tasks and purposes.In the UK, the Cabinet Office was sufficiently concerned to institute a wide-ranging review, which called for 'a fundamental change in culture to place good analysis at the heart of policymaking' (Cabinet Office 2000, p. 5).It asserted that 'the use of analysis and modelling in the US is more extensive . . .and of much better overall quality' (Cabinet Office 2000, p. 99), but acknowledged that there was no systematic audit of use across jurisdictions which could be used to identify best practices.Following a major failure in the use of models in UK government, a wide-ranging review was eventually undertaken in 2013 which reported that around 500 computerized models were being used, influencing many billions of pounds of government expenditure (HM Treasury 2013, p. 33).Yet this transformation in the tools of policy formulation being used seems to have escaped the attention of most policy scholars.
Third, the growing interest in policy formulation tools could also be seen as one symptom of the gradual re-discovery of policy design as both a policy goal (in other words, through state-led policymaking) and a research topic (Howlett et al. 2014).Far from reducing the need for state involvement, the emergence of a more complex, networked society and austerity pressures, makes it more important for interventions to be carefully targeted and legitimated (Howlett and Lejano 2013, p. 12).One way the pressure upon the state to discharge these functions manifests itself is in the perceived need for tools to formulate 'better' policies.Several of the chapters in this book (for example, Chapters 3, 9 and 12) make repeated references to tools that seek to engage with complex policy problems that are uniquely interconnected and cross-jurisdictional in their scale and scope, and have a very strong public interest dimension.
Finally, the number of policy formulation tool types has grown significantly in recent years.And as they have emerged from the analycentric 'backroom' (Self 1981, p. 222), the expectation has grown that they will respond more sensitively to changing contextual conditions and public expectations, somewhat addressing Wildavsky's (1987, p. vi) call for policy to be seen as an art and a craft rather than a technocratic exercise in selecting and employing tools to 'solve' problems.In the next section we attempt to bring a greater sense of analytical order to the expanding list of tools, methods, tasks and expectations.

FORMULATION TOOLS: TOWARDS A NEW SUB-FIELD OF POLICY ANALYSIS?
The Literatures on Policy Formulation Tools: Taking Stock In attempting to move the study of policy formulation tools back into the mainstream of public policy research, we immediately confront a problemthe relative absence of common definitions and typologies.Without these, it is difficult to believe that the literatures discussed above can be telescoped into a new sub-field.We believe that four literatures provide an especially important source of common terms and concepts, which we now briefly summarize.
The first literature describes the internal characteristics and functions of each tool, and/or offers tool kits which seek to assist policy formulators in selecting 'the right tool for the job'.On closer inspection, there are in fact many sub-literatures for all of a vast array of different tools; numerous classic texts like Dunn (2004) and Rossi et al. (2004) introduce some of the main ones.Generally speaking, rather fragmented into the main tool subtypes, and rather rationalistic in its framing, this literature nonetheless remains crucial because it outlines the intrinsic features of each tool.However (as repeatedly noted above), it does not have a great deal to say about where, how, why and by whom (in other words, by which actors and in which venues) they are used, and what effects they (do not) produce.
The second is dominated by typologies.Tools can be typologized in a number of different ways, for example: by the resources or capacities they require; by the activity they mainly support (for example, agenda setting, options appraisal); by the task they perform; and by their spatial resolution.Radin (2013, p. 145) opts for a more parsimonious framing, distinguishing between two main types: the more economic tools such as cost-benefit analysis (CBA) and what she terms the more 'systematic approaches' such as criteria analysis and political mapping.The problem is that dividing the field into two does not really offer much typological variation.In an earlier analysis, we elected to subdivide the main tools into three main types based on their level of technical complexity (Nilsson et al. 2008): • • Simple tools such as checklists, questionnaires, impact tables or similar techniques for assisting expert judgement.

•
• More formal tools, such as scenario techniques, CBA, risk assessment and multi-criteria analysis, which entail several analytical steps corresponding to predefined rules, methods and procedures.

•
• Advanced tools which attempt to capture the more dynamic and complex aspects of societal or economic development by performing computer-based simulation exercises.
At the time, we noted that there was no normative ranking implied in this typology.We also noted the basic difference between tools (such as scenarios and public participation) with more open procedures and purposes, and those like CBA that follow a set of standard procedural steps.But we did not relate these to the policy formulation tasks that tools could or should perform.We return to the matter of typologies below.
The third literature adopts a more critical perspective (Wildavsky 1987;Shulock 1999;Self 1981), offering words of caution about expecting too much from tools.It appears to have left a deep impression on a sufficient number of policy analysts, perhaps sufficient to militate against the development of a new sub-field.However, it is clear that despite these cautionary words, many tools have been developed and are very heavily applied in certain venues to routinely produce effects that are not currently understood.Hence, questions about precisely where, how, why and by whom they are used remain.
A fourth and final literature is more strongly focused on the main venues and processes of policy formulation rather than the tools.In attempting to better understand and explain how policy is made and what influences it, this literature encompasses studies of crucial factors such as the utilization of knowledge in policymaking (Radaelli 1995), and the role of power and institutions (for an excellent summary, see Sabatier 2005).The manner in which power and particular analytical practices are bound up with one another has been explored in planning/geography (see for example, Owens and Cowell 2002) and science and technology studies (Stirling 2008).Other aspects focus on the political demand for evidence-based policymaking (Sanderson 2002;Shine and Bartley 2011).Much of this literature adopts a macro-or a meso-level focus and draws on or develops theory.To the extent that it considers policy formulation tools at all, there is, however, a tendency (although by no means universal) to assume that tools are epiphenomenal and hence not warranting detailed analysis.But we shall argue that without more detailed research, these remain no more than untested assumptions.

Re-assembling the Field: A Definition and a Typology
To move forwards, we draw upon Jenkins-Smith (1990, p. 11) by defining a policy formulation tool as: a technique, scheme, device or operation (including -but not limited to -those developed in the fields of economics, mathematics, statistics, computing, operations research and systems dynamics), which can be used to collect, condense and make sense of different kinds of policy relevant knowledge to perform some or all of the various inter-linked tasks of policy formulation.
But what are the main tools of policy formulation and which of the interlinked formulation tasks mentioned in this definition do they seek to address?Today, the range of policy formulation tools is considerably wider and more 'eclectic' (Radin 2013, p. 159) than it was in Lasswell's time.While keenly aware that typologizing can very easily become an end in itself, developing some kind of workable taxonomy nonetheless remains a crucial next step towards enhancing a shared understanding of how policy formulation tools are used in contemporary public policymaking.
We propose that the five policy formulation tasks outlined aboveproblem characterization, problem evaluation, specification of objectives, policy options assessment and policy design -may be used to structure a typology of policy formulation tools, based on what might be termed the 'textbook' characteristics of what they may be capable of.We also draw on Dunn's (2004, pp. 6-7) schema of three types of tasks associated with policy formulation tools (problem structuring, forecasting and recommending), and de Ridder et al.'s (2007) typology of assessment tools (see Table 1.1).In Table 1.1, the first two tasks of 'problem characterization' and 'problem evaluation' broadly correspond to Dunn's (2004) problem structuring -that is, tools that produce information about what problem to solve.The remaining three tasks correspond to Dunn's forecastinghence tools that produce information about the expected outcomes of policies -and also recommending -hence tools that produce information about preferred policies.
Following Thomas (2001, p. 218), the consensus building or 'consolidation' that can occur throughout the formulation process may draw on feedback or consolidation tools for communicating findings back to policy actors.These can include many of the same sorts of tools presented under 'problem characterization', such as stakeholder meetings, the elicitation of public perceptions and/or expert opinions.

An Analytical Framework
In the rest of this book, a number of experts in policy formulation tools and venues seek to shed new light on the interaction between four key aspects of these tools, which together constitute our analytical framework: actors, capacities, venues and effects.

Actors
First, we seek to elucidate those actors who participate in policy formulation, particularly those that develop and/or promote particular policy formulation tools.The tools literature has often lacked a sense of human agency and, as noted above, the policy formulation literature tended to ignore the tools being used.These two aspects need to be brought together.In this book we therefore seek to know who the actors are and why they develop and/or promote particular tools.Why were particular tools developed, when and by whom?And what values do the tools embody?Venues Second, we want to know more about by whom and in which policy formulation venues such tools are used, and for what purposes.What factors shape the selection and deployment of particular tools?Again the broader question of agency seems to be largely unaddressed in the four existing literatures summarized above.Tool selection is treated largely as a 'given'; indeed many studies seem to ignore entirely the reasons why policymakers utilize them (or do not).Finally, relatively little is known about how the various tools and venues intersect, both in theory and, as importantly, in practice.

Capacities
Third, we wish to examine the relationship between policy capacity and policy formulation tools.Policy capacity is one of a number of subdimensions of state capacity, which together include the ability to create and maintain social order and exercise democratic authority (Matthews 2012).Broadly, it is the ability that governments have to identify and pursue policy goals and achieve certain policy outcomes in a more or less instrumental fashion, that is, 'to marshal the necessary resources to make intelligent collective choices about and set strategic directions for the allocation of scarce resources to public ends' (Painter and Pierre 2005, p. 2).
It is known to vary between policy systems and even between governance levels in the same policy system.Policy instruments and tools have long been assumed to have an important influence on policy capacity -if they did not, why use them (Howlett et al. 2014, p. 4)?The fact that they are unevenly used over time, for example, could explain why the policy capacity to get things done also varies across space and time (Bähr 2010;Wurzel et al. 2013).
The chapters of this book seek to examine the relationship between policy capacity and tools in three main ways.First, they conceive of the policy formulation or policy analytic capacities that inhere within each tool (in other words, Table 1.1).For example, scenarios and foresight exercises provide policymakers with the capacity to address the problem characterization and problem evaluation tasks, particularly in situations of high scientific uncertainty.By contrast, tools such as CBA and multicriteria analysis (MCA) provide a means to complete the policy assessment of option and policy design stages of the policy formulation process.
Second, the chapters also tackle the question of what policy capacities are in turn required by policymakers to employ -and perhaps even more fundamentally to select -certain policy formulation tools.For example, relatively heavily procedural tools such as MCA and CBA arguably require specialist staff and specific oversight systems.When these are weak or absent, the use made of tools may tend towards the symbolic.Thus, several questions may be posed.What capacities do actors have -or needto employ specific policy formulation tools?And what factors enable and/ or constrain these capacities?
Finally, the chapters open up the potentially very broad -but equally important -question of what factors might conceivably enable or constrain the availability of these capacities.The fact that critical supporting capacities may not be available in every policy system is something which is raised in several of the chapters.

Effects
Finally, what effects, both intended and actual, do the various tools generate when they are employed?As we explained above, our original expectation was that the tools would produce some quite specific epistemic and political effects.But while some evidence is available on their wider effects, much more is required.The policy instruments literature has been struggling to address this question, at least for implementation tools, ever since Salamon (2002, p. 2) speculated that each tool imparts its own distinctive spin or twist on policy dynamics.Substantive effects include learning in relation to new means to achieve given policy goals (a feature which is predominant amongst the more structured procedural tools such as CBA, but also computer modelling tools) through to the heuristic-conceptual effects on problem understandings (see for example Chapters 2 and 3, this volume).The procedural effects could be similarly wide ranging including (re-)channelling political attention, opening up new opportunities for outsiders to exert influence and uncovering political power relationships.The chapters examine whether or not these and other effects occurred, and whether they were, or were not, originally intended.

Plan of this Book
The chapters are grouped into two main parts.Those in Part II provide -in some cases, for the very first time -a systematic review of the literature on particular tools.They are written by tool experts according to a common template and draw upon examples from across the globe.Given space constraints, we elected to focus on six of the most widely known and commonly advocated tools, which broadly reflect the range of tool types and policy formulation tasks summarized in Table 1.1.Thus, Matthijs Hisschemöller and Eefje Cuppen begin by examining participatory tools (Chapter 2), Marta Pérez-Soba and Rob Maas cover scenarios (Chapter 3) and Markku Lehtonen reviews indicators (Chapter 4).Then, Martin van Ittersum and Barbara Sterk summarize what is currently known about computerized models (Chapter 5), Catherine Gamper and Catrinel Turcanu explore forms of multi-criteria analysis (Chapter 6) and Giles Atkinson concludes by reviewing the literature on cost-benefit analysis (Chapter 7).
The chapters in Part II explore the relationship between actors, venues, capacities and effects from the perspective of each tool.By contrast, the authors in Part III cut across and re-assemble these four categories by looking at tool-venue relationships in Europe, North America and Asia.Some (for example, Chapters 8 and 9) turn the analytical telescope right around and examine the use made of multiple tools in one venue.Each chapter employs different theories to interpret freshly collected empirical information to test explanations and identify pertinent new research questions.In broad terms, the first two chapters in Part III examine the use of multiple tools in one or more venues, whereas those that follow focus on the application of specific tools in one or more venues.Thus in their chapter, Michael Howlett and colleagues explore the distribution of all tools across many venues in Canada (Chapter 8), whereas John Turnpenny and colleagues explore the use of all the tools in the single venue of policy-level appraisal within Europe (Chapter 9).Sachin Warghade examines the use of two tools in a number of different venues in India (Chapter 10), and Christina Boswell et al. investigate the use of indicators in the UK (Chapter 11).Finally, Paul Upham and colleagues explore the application of a particular type of computerized model in a range of different policy formulation venues in the UK (Chapter 12).In the final Chapter (13), we draw together the main findings of the book and identify pertinent new policy and analytical research challenges.Conscious that this still has the look and feel of a sub-field of policy analysis 'in the making' we attempt to draw on these findings to critically reflect back on our typology, our definition of formulation tools and our analytical framework.
More generally, in Chapter 13 we seek to explore what a renewed focus on policy formulation tools adds to our understanding of three important matters.First, what stands to be gained in respect of our collective understanding of the tools themselves, which as we have repeatedly noted have often been studied in a rather isolated, static and descriptive manner?Second, what does it reveal in relation to policy formulation and policymaking more generally?Policy formulation is arguably the most difficult policy 'stage' of all to study since it is often 'out of the public eye . . .[and] in the realm of the experts' (Sidney 2007, p. 79).Howlett has argued that it is a 'highly diffuse and often disjointed process whose workings and results are often very difficult to discern and whose nuances in particular instances can be fully understood only through careful empirical case study' (Howlett 2011, p. 32).Aware of the challenges, in this book we seek to investigate what a renewed focus on tools is able to add to the current stock of knowledge.In doing so, we seek to directly challenge the conventional wisdom about tools as epiphenomenal, that is, wholly secondary to ideas, interests, power and knowledge.Finally, what does it add to our collective understanding of the politics of policymaking?This is an extremely pertinent question because many of the tools were originally conceived as a means to take the political heat out of policymaking.
Rationalism no longer holds the same grip on policy analysis as it once did, but the perceived need to 'design' policy interventions as effectively and as legitimately as possible remains as strong as ever.Therefore, whether or not the tools succeed in these tasks is something we believe will interest mainstream political scientists, as much as policy analysts and experts in the tools.NOTES 1. Hood and Margetts' (2007) concept of 'detector' tools for harvesting policy relevant information corresponds only to one of a number of different policy formulation tasks.2. Although we regard the terms tool and instrument as being broadly synonymous, henceforth we use the term 'tools' mainly to differentiate policy formulation tools from policy implementation instruments.
Figure 1.1 The main venues of policy formulation

Table 1 .
1 A typology of policy formulation tools, linking tools to their potential use in different policy formulation tasks • risk-benefit analysis; Source: Based on Dunn (2004); de Ridder et al. (2007).John R. Turnpenny, Andrew J. Jordan, David Benson and Tim Rayner -9781783477043 Downloaded from Elgar Online at 03/02/2019 01:30:19AM via free access