11. The effects of targets and indicators on policy formulation: narrowing down, crowding out and locking in

Targets have become an increasingly important component of governance and public sector management in the last two decades, especially across OECD countries. Such targets often involve the use of performance indicators, a policy tool introduced to measure and vouchsafe how far specific targets have been met. Indeed, the possibility of reliably measuring the achievement of targets through performance indicators (PIs) is generally a precondition for the selection of targets (Bevan and Hood 2006; Audit Commission 2000a). The two policy instruments are thus closely interconnected. The received wisdom among policymakers is that PIs and targets are management tools, adopted to improve the quality and valueformoney of public services. By introducing clear and transparent targets, and subjecting these to regular monitoring through measuring them against PIs, governments incentivize improvements in the performance of those involved in service delivery, and increase public accountability (HM Treasury 1998; Audit Commission 2000a; 2000b). Ostensibly, then, targets and related PIs might be best characterized as instruments for ensuring the effective delivery, or implementation, of policies and programmes that have already been adopted. However, we argue in this chapter that targets and PIs can also have an important role in policy formulation. They serve to shape and delimit the range of options open to policymakers. As scholars of public administration have noted, targets and PIs can have a number of unintended effects, encouraging forms of gaming or creating perverse incentive structures


INTRODUCTION
Targets have become an increasingly important component of governance and public sector management in the last two decades, especially across OECD countries.Such targets often involve the use of performance indicators, a policy tool introduced to measure and vouchsafe how far specific targets have been met.Indeed, the possibility of reliably measuring the achievement of targets through performance indicators (PIs) is generally a precondition for the selection of targets (Bevan and Hood 2006;Audit Commission 2000a).The two policy instruments are thus closely interconnected.
The received wisdom among policymakers is that PIs and targets are management tools, adopted to improve the quality and value-for-money of public services.By introducing clear and transparent targets, and subjecting these to regular monitoring through measuring them against PIs, governments incentivize improvements in the performance of those involved in service delivery, and increase public accountability (HM Treasury 1998;Audit Commission 2000a;2000b).Ostensibly, then, targets and related PIs might be best characterized as instruments for ensuring the effective delivery, or implementation, of policies and programmes that have already been adopted.
However, we argue in this chapter that targets and PIs can also have an important role in policy formulation.They serve to shape and delimit the range of options open to policymakers.As scholars of public administration have noted, targets and PIs can have a number of unintended effects, encouraging forms of gaming or creating perverse incentive structures Christina Boswell, Steve Yearley, Colin Fleming, Eugénia Rodrigues, and Graham Spinardi -9781783477043 Downloaded from https://www.elgaronline.com/at 10/14/2023 01:56:20PM via free access (Smith 1990;1995;James 2004;Pidd 2005;Bevan and Hood 2006).
Building on these contributions, we identify three main ways in which targets and PIs might potentially constrain policy formulation.First, in order to be deployable as targets, policy goals need to meet a number of managerial, political and technical criteria.This implies that only a subset of policy objectives may end up being codified as targets, narrowing down the range of policy options or objectives that receive the target treatment.This 'narrowing down' effect may be exacerbated by a second effect, which we call the 'crowding out' effect.Once adopted, targets can become the (sometimes exclusive) focus of political discussion and organizational action.This can 'crowd out' other objectives and considerations in policy formulation processes, with the target in some cases even supplanting the original underlying objective.What was initially a means becomes an end in itself.Third, targets and PIs may also have a 'locking in' effect over time.Once adopted, they can commit governments -and their critics -to overly specified courses of action, which are not responsive to changing conditions.Taken together, these three effects imply that the introduction of targets and PIs can have a significant effect on policy formulation, and not always in ways intended by those originally introducing them.This chapter explores how far these three constraining effects have influenced policy formulation in the case of targets and PIs developed as part of the Public Service Agreements (PSAs) rolled out by the UK government between 2000 and 2010.We compare three different policy areas: immigration control, climate change and defence procurement.These cases offer scope for comparing policy areas with quite distinct 'audiences'.Immigration is a highly politicized area, which is the object of ongoing media and political attention, and there is strong pressure on incumbents to demonstrate their capacity to manage the problem.Climate change is a more technocratic area, relatively protected from popular media and political attention, but subject to more specialized scrutiny from NGOs and bound by international treaty obligations.Defence procurement remains largely sequestered from popular, political or media attention, despite continued problems of overspend and poor performance -its main form of scrutiny is through parliamentary committees, the National Audit Office (NAO), and the controller of its purse strings, the Treasury.We expect these variations in audience to produce different types of pressures in selecting targets and PIs, in turn generating different patterns of constraint in policy formulation.
The chapter starts by setting out the main features of the PSAs introduced by the post-1997 Labour administration.It suggests the ways in which PSA targets and PIs may have had a constraining effect on policy formulation through processes of narrowing down, crowding out and  (Smith 1990).This approach was reinforced under the Labour administrations of 1997-2010.In 1998, the government conducted a Comprehensive Spending Review, which introduced performance requirements across government.Each department was instructed to undertake a series of improvements to the way they delivered their services, in order to justify funding allocations.These targets were updated in 2000 with a more comprehensive set of PSAs.The new PSAs set out for each major government department 'its aim, objectives and the targets against which success will be measured' (HM Treasury 2000).A key component was the measurement and monitoring of delivery of these targets, through annual departmental reports.Each objective was required to have at least one target which was 'SMART': specific, measurable, achievable, relevant and timed.PSAs were accompanied by Service Delivery Agreements (SDAs), concluded between the Treasury and each department, which set out more specific, lower-level targets and milestones to support delivery of the PSA targets.While government rhetoric on targets and PIs focused on performance and delivery, implying that they were a tool for implementing policy, in many cases the selection of targets could be better characterized as an instrument of policy formulation.The selection of targets involved translating broad policy aims and objectives into specific and practically achievable goals.It therefore implied a process of identifying and assessing different options for addressing policy problems -akin to Howlett's definition of policy formulation (Howlett 2011, p. 30; see also Chapter 1, this volume).This raises important questions about the process for, and rationale behind, selecting targets.For example, what sorts of consideration underpinned the selection of targets?
From the outset, it was clear that targets and PIs had a dual function.The Treasury characterized the PSAs as 'a major agenda to deliver and demonstrate change in the commissioning, management and delivery of public services' (HM Treasury 2002a, p. 13; emphasis added).'Departments were given a real incentive to drive up standards in public services and the public was given the opportunity to judge their performance' (ibid, p. 12).This dual purpose of steering performance while also demonstrating improvement conforms to insights which suggest that targets and indicators function as 'boundary objects' (Turnhout 2009, p. 405; see also Chapters 4 and 12, this volume).On the one hand, targets and PIs are adopted to enhance public sector performance, through what might be termed their 'disciplining' function: they provide incentives for actors involved in formulating and implementing policy to improve their performance and ensure 'value for money'.But at the same time, targets and PIs clearly have a range of other, more political, functions.They may be developed for symbolic reasons, to signal commitment to, and underscore achievement of, a range of political or organizational goals.Targets and PIs thus need to operate as management tools, providing relevant and practical guidance for steering policy; but at the same time, they need to resonate with -and often mitigate -public concerns about public service performance; and in some cases, they also need to signal to other audiences such as lobby groups, foreign governments or international organizations that the government is committed to a particular course of action (Boswell 2014).
Aside from this dual function of delivering and demonstrating improved public services, there were a number of formal and technical criteria that guided the selection of targets.First, targets needed to be monitored, and thus linked to indicators.The potential to measure and monitor targets through PIs was built into the very definition of targets.Second, targets increasingly became focused on outcomes.The House of Commons Treasury Select Committee, which monitors Treasury policy, calculated in 2000 that most of the targets under the 1998 PSA had been 'process' (51 per cent) or 'output' (27 per cent), with only 11 per cent comprising outcome targets (House of Commons Treasury Select Committee 2000).It recommended that the new PSAs established in 2000 focus more on outcomes; and indeed the National Audit Office classified 68 per cent of the targets adopted in 2000 as outcome targets (NAO 2001, p. 1).
In short, the selection of targets and PIs was guided by three sets of considerations: the managerial goal of disciplining behaviour to improve performance; the political goal of signalling to key audiences that key objectives were being met; and formal requirements linked to measurement, with a focus on outcomes.
What sorts of constraining effects did PSAs have on policy formulation?For a start, we can expect that the various managerial, political and technical requirements for selecting targets might have the effect of narrowing down the range of policy objectives that were to be considered.It was certainly a tall order for targets to meet all of the formal and substantive criteria set out.PSAs are therefore likely to have fostered a reliance on a narrow set of indicators as proxies for meeting a set of broader organizational and policy objectives.They represent what Bevan and Hood (2006, p. 521) have identified as a form of synecdoche: treating a part to stand for the whole.Second, and related to this, we might expect targets and PIs to have a crowding out effect, re-orienting both political debate and organizational action to focus on performance against the selected targets.There is limited scope in this chapter for examining how far PSAs influenced political debate on policy objectives.However, we can identify processes of crowding out by examining the type of scrutiny exercised by peer organizations.Notably, the NAO and parliamentary select committees had a formal role in monitoring performance, and assessing the effectiveness of PSAs across policy sectors.Thus one important indicator of crowding out is to explore how far these bodies bought into, or challenged, the selection and scope of targets and PIs.
Third, PSAs might be predicted to have a locking-in effect, tying government departments to a particular course of action, even in the event of a change in circumstances or policy priorities.Indeed, from early on, targets and PIs were criticized for being overly rigid and centralized, and allowing insufficient flexibility for local government and other actors involved in service delivery (NAO 2001;House of Commons Public Administration Select Committee 2003).The 2004 PSAs responded to these criticisms by claiming to reduce the number of targets, allowing more flexibility.Many targets also became more 'directional', with performance measured in terms of improvement or deterioration, rather than meeting a specific numerical target.Yet clearly the scope for such adjustments was constrained by the setting of earlier commitments and the fact that such commitments were transparent and had a built-in system of monitoring and scrutiny.
In order to investigate these constraining effects, we examine three areas: immigration control, climate change and defence procurement.One of our expectations is that these dynamics will vary depending on the audience being targeted by policymakers.Are policymakers involved in the selection of targets trying to meet the expectations of public opinion/ the media, parliamentary or other organizations involved in oversight, or the specialized policy community (practitioners, NGOs, researchers)?We expect this to have an influence on which targets and indicators they select; and, as a result, how pronounced the three effects on policy formulation are.For each case, we therefore start by examining the rationale for the selection of targets.We then explore how far this selection was associated with a crowding out effect.And finally, we analyse how far policymakers became locked in to a given course of action in the face of changed circumstances.

Immigration Control
Immigration control covers a range of measures to control the entry, residence and employment of immigrants and refugees.It has long been part of the remit of the Home Office, and more specifically its Border Agency.The UK Border Agency (UKBA -originally named the Border and Immigration Agency) was set up in 2007, as successor to the Immigration and Nationality Directorate.
A striking feature of the targets adopted on immigration control in the 2000s was the gap between very broad strategic objectives, and the very narrow scope of the targets adopted.The 2000 PSA and the 2002 Service Delivery Agreement both set out a broad objective for the Home Office in this area, covering the areas of meeting economic and skills requirements through work permits/entry policies, facilitating international travel, more efficient asylum systems, and -in the case of the 2002 SDA -effective programmes for dealing with citizenship and long-term immigration applications.Yet the targets set under this objective all related to asylum applications, removals and detention.Thus the 2000 targets were (1) to ensure that by 2004, 75 per cent of asylum applications are decided within two months, and (2) to remove a greater proportion of failed asylum seekers.The 2002 SDA further refined these two targets, and added a third target of increasing detention capacity.So despite a very broad set of objectives, the targets adopted focused on one very narrow area.What explains this disparity?One possible explanation is technical: many areas of performance relevant to the broader objectives would be difficult to measure.There is a high degree of uncertainty in measuring, for example, the successful integration of refugees, or the social and economic impact of immigration, or the scale of irregular migration.By contrast, asylum statistics are regular, reliable and based on a well-established registration system (Boswell 2012).Yet the same would apply to other aspects of the targetsuch as work permits, or citizenship applications and acquisition.It would also have been quite feasible to measure performance on, for example, the quality of first decisions on asylum applications.So while technical criteria may have partly explained the focus on asylum, it certainly was not the only aspect of policy meeting these conditions.
A far more plausible explanation is the political saliency of asylum at that time.Asylum applications had been rising in recent years, and there was intense mass media coverage of the issue.So while asylum is just one aspect of immigration policy -and arguably not as critical for socio-economic welfare as others, such as the impact of immigration on Britain's economy or society -it was the most politicized, and the one on which the government was receiving most criticism from the media.In the case of the Home Office's immigration targets, then, the criterion of selection seems to have been very much geared to meeting political objectives, notably addressing public concerns as articulated in mass media reporting.
The importance of public opinion becomes even clearer if we consider the audiences that were not being addressed through this selection of targets.It was certainly not responding to concerns about administrative inefficiency within the Home Office, which had been articulated in parliamentary debate and mass media reporting.Nor was there a concern to address the business/employers audience, who would be more concerned about ensuring an efficient and swift process for processing permits, and a flexible approach to policy on entry.The focus on removals and detention, as well as the emphasis on speeding up asylum decision making (rather than improving its quality) was also likely to be the object of criticism by NGOs and human rights groups.Indeed, given that the Labour government's immigration policy was in many ways emerging as quite liberal and progressive -at least in the area of labour migration -it is striking that they should have adopted a set of targets exclusively emphasizing the restrictive and potentially human-rights-violating aspects of Home Office policy.It represents a very pronounced case of narrowing down.
The highly politicized nature of the asylum-related targets was illustrated by Prime Minister Tony Blair's appearance on the BBC's flagship television current affairs programme, Newsnight, in early 2003.Blair unexpectedly pledged to halve asylum seeker numbers within a year, although this had not been part of either the 2000 or 2002 targets, and was not the object of prior consultation with the Home Office (Boswell 2009, p. 140).A target reflecting this new pledge was introduced in the Home Office Departmental Report, 2004-2005, and thereafter the goal of reducing the number of new asylum applications became one of the targets (incorporated into the PSAs covering 2004 and 2007).
If targets had a narrowing down effect, how far did they crowd out a focus on other policy objectives?The focus on asylum targets, and especially Blair's high profile announcement on Newsnight, was the object of wide media coverage.The target also became one of the top priorities for the Prime Minister's influential Delivery Unit, implying intense scrutiny and pressure from No. 10.This crowding out effect was reinforced by the system for monitoring performance.In principle, one might expect the bodies responsible for scrutinizing PSAs to have questioned the selection of targets as being overly narrow.Yet the bodies most closely involved in monitoring Home Office PSAs -the National Audit Office, and the House of Commons Home Affairs and Public Administration Select Committees -largely bought into the selection of targets.NAO reports on Home Office targets and PIs over this period focused almost exclusively on technical aspects of the PSAs.The Home Affairs Committee did raise some issues around the selection and potential effects of targets.Yet their focus was on problems of feasibility, whether they were sufficiently ambitious and whether there were too many targets.Rather than challenging the narrowing down and crowding out effects, these bodies arguably contributed to them by urging a focus on even narrower and more ambitious 'stretch' targets.
Finally, how far did these targets have a lock-in effect, restricting the flexibility of the Home Office in responding to changing circumstances?Here the evidence suggests that the lock-in effect was relatively limited.The Home Office was able to shift its targets and objectives several times, in response either to challenges in meeting the targets, or changed political objectives.For example, the target on removals saw a number of shifts over the decade.The first shift was towards a more precise target.While the 2000 PSA simply talked of 'removing a greater proportion of failed asylum seekers', the 2002 SDA aimed to increase the number of removals to 30,000 by March 2003.The Home Office was subsequently forced to admit this target was too ambitious.In a scathing critique, the House of Commons Home Affairs Select Committee (2003, p. 23) noted that: We are at a loss to understand the basis for the belief that a target of 30,000 removals a year was achievable, and ministerial pronouncements on the subject are obscure.It is surely not too much to expect that, if it is thought necessary to set targets for removals, they should be rational and achievable.
In the new 2004 PSA, this target was adjusted from a specific numerical target back to a 'directional' target, that is, to remove a greater proportion of failed asylum seekers in 2005-2006 compared with 2002-2003.This represents a clear case of 'gaming' through an attempt to manage the presentation of performance (James 2004, p. 409).Even this more modest target was not achieved.Despite criticism by the Home Affairs Select Committee, the Home Office retained its directional target.
The Home Office was also able to adjust its substantive targets over the decade.By the time of the 2007 PSA, the Home Office's objective for immigration had become more narrowly focused on control: 'Securing our borders, preventing abuse of our immigration laws and managing migration to boost the UK' (Home Office 2007, p. 54).With declining numbers of asylum applicants, the focus was also shifting to border control.In line with the government's approach of reducing the number of targets, the Home Office claimed to have just one target: reduce the time taken to process asylum applications.
To summarize, the setting of targets in the area of immigration policy appears to have been strongly driven by political considerations, notably the perceived need to signal to the public that the government was acting to reduce asylum applications and detain or deport those who were not considered to be genuine refugees.This led to a significant narrowing down of policy priorities, and a focus on scrutinizing performance against those targets.However, the Home Office found ways to avoid being locked in to these targets when they appeared either unfeasible, or no longer relevant to its core strategic objectives.

Climate Change
Before 2001, environmental commitments including climate change were dealt with by a large, portmanteau Department of the Environment, Transport and the Regions (DETR).From June 2001, the DETR was reorganized with the principal environmental responsibilities shifting to DEFRA (the Department for Environment, Food and Rural Affairs), resulting in the objectives and targets for climate change being located in a more conventionally environmental framework.Then, as a consequence of the new climate policy architecture defined by the 2008 Climate Change Act, in October of that year a new Ministry was established -the Department of Energy and Climate Change (DECC).While DEFRA retained some responsibilities for climate change, these were essentially restricted to adaptation to impacts -it was clear that climate change was now being framed as an energy (and thus industrial) issue as much as an environmental one (see Yearley 2002, p. 277-279).Thus, within a decade the political and organizational location -and, to some degree, the framingof the climate issue moved around a good deal and this in itself impacted the context for relevant PSAs.
Policies for and action around climate change have featured as objectives in all three sets of PSAs and the government's plans and obligations Though the 1998 Comprehensive Spending Review made no mention of climate change nor the greenhouse effect in its targets for DETR -the only possible link was to the overall aim of promoting sustainable development (HM Treasury 1998, p. 13) -by the time DEFRA's PSAs were spelled out in 2002, performance target 2 was to '[i]mprove the environment and the sustainable use of natural resources, including through the use of energy saving technologies, to help reduce greenhouse gas emissions by 12.5% from 1990 levels and moving towards a 20% reduction in carbon dioxide emissions by 2010' (HM Treasury 2002b, p. 27).The specific target here was precisely that adopted in the Kyoto Protocol of the United Nations Framework Convention on Climate Change (agreed in outline in 1997), to which the UK and the EU were signatories.
The Kyoto Protocol stipulated that signatory countries had to reduce emissions of six greenhouse gasses by set amounts.The UK commitment was to achieve a 12.5 per cent reduction by 2008-2012.As is clear in the quote above, in the 2002 PSA, this goal was supplemented by the more vaguely expressed idea of 'moving towards' a bigger reduction in CO 2 alone, though it was unclear whether the idea was to achieve this larger cut by 2010 or merely to be moving towards it by that date.The Treasury's document (2002b, p. 28) noted that the Secretary of State for Trade and Industry was jointly responsible for delivering these goals, though no mechanism was identified for ensuring joint action (see the subsequent probing comments in the Fourth Report of the House of Commons Select Committee on Environment, Food and Rural Affairs [2005, section 5, para 35]).In its Autumn Performance Report 2002, DEFRA (2002, p. 33) gave more detail in separate chapters on the Spending Review 2000, the 1998 Comprehensive Spending Review, and the Spending Review 2002: PSA and the Future, noting in each case a commitment to meeting the Kyoto targets and noting explicitly that DEFRA took on the environmental PSAs that formerly related to DETR.It also looked forward to the PSA for 2003-2006, for which it expressed the greenhouse gas target in exactly the same manner.
When DEFRA published its 2004 PSAs for the Comprehensive Spending Review, the same target 2 was in place, this time with the international treaty dimension made even clearer: To reduce greenhouse gas emissions to 12.5% below 1990 levels in line with our Kyoto commitment and move towards a 20% reduction in carbon dioxide emissions below 1990 levels by 2010, through measures including energy efficiency and renewables.This target was still in place at the time of the Autumn Performance Report 2006 (see p. 22ff) though in this document much greater detail was given about trends in performance of emissions and about new initiatives such as the Office of Climate Change (OCC) which was created 'to work across Government to provide a shared resource for analysis and development of climate change policy and strategy' (DEFRA 2006, p. 25).Details of how emissions are gauged were also available in the Technical Note to the PSAs (HM Treasury 2004b).In a summary table (DEFRA 2006, p. 73) listing 'progress against 2004 Spending Review Public Service Agreement targets', the climate action was said to be 'on course'.
The alignment of targets with international treaty obligations is in marked contrast to the immigration control case, where the selection of targets was dominated by more populist domestic political considerations.It suggests a quite different rationale for PSAs on climate change: that of seeking to meet international obligations through disciplining the behaviour of organizations and actors involved in delivering emissions reductions.The choice of such a transparent and public tool for setting out this target is also likely to have had a symbolic function, designed to signal to the specialized climate change community that the government was fully committed to meeting its obligations -a signal backed up in the detail of the Technical Note.
However, as with the immigration case, the focus on a very restricted range of targets is interesting in itself.In this policy area the focus on reductions in aggregate greenhouse gas emissions and in total releases of CO 2 , at least for the first two rounds of PSAs, did serve to narrow down the question of what climate policies are fundamentally about.The focus fell exclusively on emissions ascribed to the UK according to the conventions of Kyoto and thus, for example, reporting did not address emissions from the (then fast-growing) airline business.Equally, though it is clear that the British Government was keen to have a 'stretch' target for CO 2 beyond mere compliance with Kyoto, there was also a narrowing down in that the PSA targets highlighted emissions reductions, as opposed to adaptation to impacts for example.The concentration on the Kyoto targets narrowed down the scope for questioning whether those targets were adequate or rapid enough and, like the whole Kyoto process, tended to emphasize territorial emissions made directly from the UK rather than consumption-based ones embedded in products imported to the UK.
Such narrowing down was also accompanied by crowding out.As mentioned above, the rapid rise in emissions from innovative (often low-cost) airlines was not factored in.Also crowded out was the question of whether emissions reductions are being achieved simply through de-industrialization or switching to less polluting energy sources such as gas.On this latter point, it is noteworthy that the goods and services that British people consume could continue to have a rising 'carbon footprint' even while the UK's officially attributable emissions fell.There is at least one further crowding out effect which is that attention -even within the broad environmental gaze of DEFRA -was focused on emissions and much less on adapting to the unfolding impacts of climate change.
There is less evidence of a strong locking-in effect.If the 2002 and 2004 targets closely matched Kyoto commitments, the 2007 commitment adopted more ambitious goals, as adumbrated by the Royal Commission on Environmental Pollution (HM Treasury 2004b, para 2.6).The new PSA Delivery Agreement 27 of October 2007 proclaimed in its title the objective of leading the 'global effort to avoid dangerous climate change' (HM Treasury 2007).This was to be assessed through six 'key indicators' ranging from the UK's own emissions to international emissions trends, a proxy measure used to assess climate impacts (access to sustainable water abstraction) and an assessment of the size of the world carbon market (HM Treasury 2007, p. 5-6).This document also referred to the draft Climate Change Bill and its aim of setting CO 2 emissions for 2050 at least 60 per cent lower than the reference year, 1990.
This PSA was distinctive in several ways.For one thing, it introduced dramatically more demanding emissions-reductions targets for the UK.Second, it introduced a specific discussion of the issue of adaptation to climate change.It promised to set out an integrated adaptation framework, dealing with issues such as flooding arising from changing rainfall patterns, and potential impacts on biodiversity and agriculture.Finally, it had a focus on the UK's role not just in combating climate-changing emissions at home but in 'leading' the global effort and, in particular, 'demonstrating to other parties the practical, economic, environmental and social benefits that tackling climate change in a cost-effective way can deliver' (HM Treasury 2007, p. 3).Recognizing that climate problems cannot be addressed by any one country in isolation -and indeed that a country that is a lone pioneer could incur costs and accrue few benefits -the objectives shifted.The UK set itself a very demanding headline target but also put an emphasis on promoting international action; at the same time it has a clear notion of the shape that the international action should take: it should be a solution based on carbon markets.
This more ambitious goal reflected a changing political context in which no successor to the Kyoto agreement was in sight and where China and other fast-developing economies were highly significant CO 2 emitters but not required to take any action under the Kyoto Protocol.The government's domestic achievements could be seen to have been in vain if no steps were taken to address these aspects of climate change.In this sense, the signalling function was important.In its 2009 report, the National Audit Office picked up on this sensitivity, noting that: Under the HM Treasury performance rating system, the Department could have assessed its performance as 'strong progress' because more than half of the indicators were demonstrating improvement or meeting the success requirement.However, given that forecasts of global CO 2 emissions in 2050 have continued to rise, the Department considers that it has made only 'some progress' in 2008-09. (NAO 2009, p. 21) , p. 21) In its Annual Report (DECC, Annual Report and Resource Accounts 2008-2009, p. 51, cited in NAO 2009), DECC scored itself lower than it could have in order to signal that the key ambition of helping to contain global emissions had not been attained.It was apparently keen to forestall criticism through humble demeanour.

Defence Procurement
Defence procurement covers the commissioning and purchase of equipment for the British armed forces, and falls under the remit of the Ministry of Defence (MoD).A series of organizational changes and initiatives took place over the post-1997 period, including the establishment of the Defence Procurement Agency (DPA) and the Defence Logistics Organisation (DLO) in April 1999, a new Smart Procurement Agency in 2000, and eventually reorganization of the DPA and DLO into Defence Equipment & Support (DE&S) in 2007.As with DECC in the last section, these changes shaped the evolution of the MoD's PSA targets over the period 2000-2010.
The overall objective stated for equipment in the first Ministry of Defence PSA was 'to procure equipment which most cost-effectively meets agreed military requirements' (HM Treasury 1998, p. 69).The focus was on two key defence procurement concerns: cost and schedule overruns.There were three specific performance targets for procurement: 'on average, no in-year increase in major project costs'; 'on average, in-year slippage of In-Service Date of new major projects of less than 10 days'; and 'on average, in-year slippage of In-Service Date of existing major projects of less than 4 weeks' (HM Treasury 1998, p. 72).The targets for 2003-2006 included a further PI: '97% of customers' key requirements attained and maintained through the PSA period' (MoD 2004, p. 12), but did not detail how this was to be measured (MoD 2002, p. 58).
The focus on issues of cost and overrun is not surprising given longstanding concerns about efficiency and value-for-money in defence procurement practices.Defence procurement in the UK, as elsewhere, has long suffered from difficulties with delivering suitable equipment on Christina Boswell, Steve Yearley, Colin Fleming, Eugénia Rodrigues, and Graham Spinardi -9781783477043 Downloaded from https://www.elgaronline.com/at 10/14/2023 01:56:20PM via free access time and on budget (Gansler 1980;Page 2006).In 2009 an authoritative independent report carried out for the Ministry of Defence noted that on average equipment programmes 'cost 40% more than they were originally expected to, and are delivered 80% later than first estimates predicted' (Gray 2009, p. 16).A series of critical reports and attempts at reform over many decades has done little to improve what is an intractable problem.
By their nature, state-of-the-art weapons systems are likely to be expensive and take longer than planned.In this sense, the development of targets and PIs focused on these procedural aspects of procurement may not in themselves have narrowed down the MoD's focus: rather, they were an accurate reflection of organizational priorities over this period.Yet the preoccupation with procedural aspects of procurement, as codified in the targets, had the effect of decoupling narrow performance goals from considerations about the effectiveness of equipment.In this sense, the targets appear to have reinforced a crowding out tendency within the MoD.This can be illustrated most clearly by the gap between MoD claims about meeting PSA targets on the one hand, and real-world military performance on the other.
Between 1998 and 2010, the MoD consistently claimed to be meeting the targets and PIs set out in the PSAs.The MoD's 1998-1999 Performance Report, for example, claimed that 'cost and technical performance targets were met' (though the latter was not a target specified in the PSA), and provided combined data for project slippage dates for both new and existing projects which exceeded the target of four weeks by about 50 per cent (MoD 1999, para 38).The following year, the MoD's 1999-2000Performance Report (MoD 2000, p. 11) matched performance data more explicitly to the targets, with the cost target reported as achieved, the in-service slippage of existing projects reported as achieved for a revised interim target and date, and the in-service slippage of existing projects not achieved, again for an interim target.This report also claimed that there were no projects with 'unsatisfactory technical performance' (MoD 2000, p. 34) (though technical performance was still not a PSA target at the time).The cost target was again met the following year, but slippage of in-service dates exceeded the targets, with the verdict that performance was 'on course' rather than 'met' (MoD 2001, p. 10).Subsequent years showed similar performance, with cost again met in 2001-2002, but schedule targets not met.Over the rest of the decade some targets would be met and some not, typically with apparently small slippages in schedule more common than cost overruns.
From 2002 onwards, however, it was becoming increasingly apparent that these PSA reports' messages that 'key requirements' were being attained contrasted strongly with real-world military performance.With British forces involved in Afghanistan and Iraq after 2002, deficiencies in defence procurement processes were becoming increasingly apparent.Providing combat troops with suitable equipment in a timely fashion 'relied on a separate stream of fast-tracked acquisition to meet "urgent operational requirements" (UORs)' (Gray 2009, p. 22).As the House of Commons Defence Select Committee noted (2009, p. 18) 'the extent of UORs represents at least a partial failure by the MoD to equip adequately its forces for expeditionary operations'.This gap between PSA targets and operational performance was largely overlooked by bodies scrutinizing MoD performance, such as the Select Committee and the NAO, which retained a focus on problems of overspend and over-run.Neither body fundamentally questioned whether the procurement PSAs were fulfilling their supposed purpose.Taking one example, in its report, 'Defence Procurement 2006' (House of Commons Defence Select Committee 2006, para 17), the Select Committee cautioned that 'while cost growth on defence equipment projects in 2005-2006 was below target, we have concerns that the main reason for this was reduction in the quantity of equipment ordered'.Despite highlighting its concern that '[m]eeting key targets should not be given priority over meeting the requirements of our Armed Forces' (ibid, para 17), there is no discussion of whether targets are appropriate or relate to the wider strategic objectives of the department.Indeed, suggestive of a crowding out effect, the same report underscores its desire that the MoD 'continue to monitor its performance at procuring equipment to time, cost and quality . . .Otherwise, there is a risk that poor procurement performance could be buried in long-term project management data' (House of Commons Defence Select Committee 2006, para 26).
The overall judgement of UK defence procurement at the end of the 2000s can be seen in the critical report by Gray (2009, p. 28), which notes that every year the procurement agency would begin 'with plans to conduct activity some 10 per cent greater than the available, and known, budget for that year', and this shortfall could only be resolved by 'reprofiling' -in effect delaying programmes to delay their costs, but at the expense of schedule slippage, greater eventual costs, and 'projects more likely to experience problems'.For example, in 2008 the National Audit Office criticized the MoD for 'failing to forecast aggregate costs', resulting in an 'additional 96 month slippage rate' despite the exclusion of the Typhoon aircraft project on the grounds that it was 'commercially sensitive' (NAO 2008, summary paragraph 1).One target (cost) is prioritized because it is the most pressing as regards MoD budgets, but both schedule and performance thus suffer.Arguably, this crowding out was already endemic within the MoD, a response to long-standing pressures on the organization to focus on narrow procedural aspects of performance.But the codification of these goals within PSAs, and the reporting and scrutiny processes put in place as part of the PSAs, undoubtedly reinforced this tendency.The targets and PIs developed failed to provide an adequate measure of operational performance, with the result that this most important feature of performance was under-emphasized.
As with the two other cases, the defence procurement PSA is unlikely to have had a strong locking-in effect, except in the superficial sense of creating a reporting requirement.The MoD's annual Performance Reports during the decade bear witness to a tendency to pay lip service to targets, while the minor failures to meet some targets in some years led to a cumulative 'black hole' in the procurement budget.The consequences of trying to squeeze too much procurement into a constrained annual budget meant that UK defence procurement was certainly locked in to a vicious cycle, but the PSA targets were not the cause of this.

CONCLUSION
This chapter has explored the ways in which targets and PIs can influence policy formulation, focusing on three types of constraining effect: narrowing down the range of policy options considered; crowding out attention to broader policy objectives; and locking policymakers into a particular course of action regardless of changing conditions.We also examined how far these effects varied across policy areas characterized by rather different sets of pressures: immigration control; climate change; and defence procurement.
Our analysis suggested that variations in the organizations' audiences did indeed influence how targets and PIs were selected and deployed.In the case of the Home Office, the selection of targets appeared to be geared towards mollifying public opinion and the media, through focusing on a very limited set of goals associated with populist approaches to immigration.The focus on reducing asylum applications and increasing removals suggested that targets had a strong signalling function, implying a symbolic, rather than disciplining, function.In the case of climate change, DECC and its predecessors' choice of targets was oriented towards meeting international treaty obligations, again leading to a significant narrowing down of broader objectives to a very specific goal of reducing emissions.The selection of targets and PIs can be interpreted as having a dual function of disciplining those actors responsible for delivering emission reductions and signalling commitment to Kyoto.The subsequent shift to more ambitious targets was also designed to signal UK leadership in the international process of reducing greenhouse gas emissions.In the case of defence procurement, the MoD's selection of targets was again very narrowly focused, this time on addressing problems of poor management and financial oversight.The focus on these aspects of organizational practice suggests that the choice of indicators was a response to pressure exerted by organizations such as the Treasury, the NAO and the Commons Defence Committee.In all three cases, then, we saw a significant narrowing down of policy objectives, though for different reasons.And in all three cases, the implication was that targets and PIs covered only a small part of the broader strategic objectives identified by the respective department.We also showed that in each of the three cases, the structures put in place to monitor targets and PIs appeared to reinforce this narrowing down effect.In the terminology developed in this paper, they contributed to a crowding out of other types of objectives.In the case of immigration control, the NAO and Parliamentary Committee scrutiny of targets and PIs focused disproportionately on technical questions, as well as demands for more ambitious and specific targets.Questions of whether the targets were the right ones to select, whether they adequately balanced different priorities, or whether they did justice to the broader strategic objectives of the Home Office, were not raised.Similarly, in the case of targets on climate change, the narrowing down of goals to focus on emissions reduction was not a major object of scrutiny, with oversight instead focusing on more technical questions of distance to target.In the case of defence procurement, the focus of targets and PIs on narrow managerial criteria implied that these tools became decoupled from broader objectives related to the performance of equipment in contemporary conflict situations.Paradoxically, then, a set of tools designed to shift the political focus onto outcomes was deployed in a way that resulted in a preoccupation with process.
Our cases showed less evidence of lock-in effects.To be sure, problems of lock-in to inflexible, centralized objectives were an object of general concern in discussion of PSAs from the early 1990s onwards.But the fact that the organizations we examined could and frequently did adjust, reinterpret, evade, demote or abandon targets implied a high degree of flexibility in implementing targets and PIs.The three organizations all found ways of circumventing the limitations imposed by targets.In the case of immigration control, targets were watered down when they proved unfeasible (removals) or demoted when they were no longer a political priority (asylum numbers).In the case of climate change, by contrast, targets were made more ambitious (emissions reduction).And in the case of defence procurement, targets were added to (customer satisfaction) but also repeatedly unmet.So in line with the literature, our analysis suggests that the disciplining function of targets may be less effective than their authors might claim (Bevan and Hood 2006;Smith 1990).
Our analysis has important implications for the design of systems to monitor targets and PIs.The NAO and House of Commons committees tended to focus on technical features of these tools.This is not surprising in the case of the NAO, whose very existence is premised on the ethical and managerial virtues of accountability and audit.These bodies are committed to the idea that good practice in targets and PIs necessarily increases transparency, accountability and performance.It is perhaps less obvious that parliamentary scrutiny would focus on a rather narrow set of criticisms.Once in place, targets and PIs may well provide a useful short-cut for assessing performance in some areas, relieving overloaded committees of the task of defining which aspects of organizational performance to scrutinize, or on what basis to do so.This may create a temptation either to judge departments based on the targets and PIs they have created, or -where the targets and PIs themselves are criticized -to question them on the basis of whether they are sufficiently ambitious, precise, and so on.There appears to be very limited or no provision for pointing out flaws related to narrowing down and crowding out effects.Once these bodies have bought into the notion of accountability and performance monitoring -principles which are difficult to reject per sethen it may become difficult to find a basis for a broader critique of the targets selected.
Yet given the influence of such targets and PIs on policy formulation, and the potential for narrowing down and crowding out effects, we suggest it would be useful to find a mid-level critique: one that does not reject the value of monitoring per se; but one which at the same time does not focus too narrowly on technicalities.Such scrutiny should involve deliberation on how far the selection and implementation of targets and PIs does justice to broader policy objectives.In effect, then, this implies a process of deliberation that recognizes and constantly scrutinizes the link between monitoring and policy formulation.
In part two, we explore to what extent these effects operated in our three cases.In conclusion, we suggest the need for targets and PIs to build in procedures of scrutiny that help avoid the narrowing down and crowding out effects we observe.Our analysis draws on a range of policy documents: departmental annual reports and annual performance reviews; NAO reports on performance; and scrutiny of targets and PIs by relevant parliamentary committees.
area have consistently occupied a prominent position among the PSAs, being typically included high up in the list of environmental topics.