'The Provision of Out-of-Hours Care in England', Report by the Comptroller and Auditor General

The Provision of Out-of-Hours Care in England

REPORT BY THE COMPTROLLER AND AUDITOR GENERAL | HC 1041 Session 2005-2006 | 5 May 2006

The National Audit Office scrutinises public spending on behalf of Parliament. The Comptroller and Auditor General, Sir John Bourn, is an Officer of the House of Commons. He is the head of the National Audit Office, which employs some 800 staff. He, and the National Audit Office, are totally independent of Government. He certifies the accounts of all Government departments and a wide range of other public sector bodies; and he has statutory authority to report to Parliament on the economy, efficiency and effectiveness with which departments and other bodies have used their resources. Our work saves the taxpayer millions of pounds every year. At least £8 for every

£1 spent running the Office.

Layout and production by NAO Information Centre

DG Ref: 6662VC

Printed on Consort Renew Satin containing 50% recycled fibre

Published by TSO (The Stationery Office) and available from:

Online www.tso.co.uk/bookshop

Mail, Telephone, Fax & E-mail

TSO

PO Box 29, Norwich NR3 1GN

Telephone orders/General enquiries 0870 600 5522 Fax orders 0870 600 5533

Order through the Parliamentary Hotline

Lo-call 0845 7 023474

E-mail book.orders@tso.co.uk

Textphone 0870 240 3701

TSO Shops

123 Kingsway, London WC2B 6PQ

020 7242 6393 Fax 020 7242 6394 68-69 Bull Street, Birmingham B4 6AD 0121 236 9696 Fax 0121 236 9699

9-21 Princess Street, Manchester M60 8AS 0161 834 7201 Fax 0161 833 0634

16 Arthur Street, Belfast BT1 4GD

028 9023 8451 Fax 028 9023 5401 18-19 High Street, Cardiff CF10 1PT

029 2039 5548 Fax 029 2038 4347 71 Lothian Road, Edinburgh EH3 9AZ 0870 606 5566 Fax 0870 606 5588

The Parliamentary Bookshop £10.75 12 Bridge Street, Parliament Square,

London SW1A 2JX  ISBN 0-10-293758-3 Telephone orders/General enquiries 020 7219 3890

Fax orders 020 7219 3866

TSO Accredited Agents (see Yellow Pages)

and through good booksellers. 9 780102 937589

The Provision of Out-of-Hours Care in England

Ordered by the LONDON: The Stationery Office  House of Commons £10.75 to be printed on 2 May 2006

REPORT BY THE COMPTROLLER AND AUDITOR GENERAL | HC 1041 Session 2005-2006 | 5 May 2006

CONTENTS

SUMMARY  4 PART 1

This report has been prepared under  The previous out-of-hours service  8 Section 6 of the National Audit Act 1983  was unsustainable

for presentation to the House of Commons

in accordance with Section 9 of the Act. The Carson Review and other strategic documents  9

set out a vision for change

John Bourn  A new system of out-of-hours arrangements was  10 Comptroller and Auditor General  implemented during 2004-05

National Audit Office

21 April 2006

PART 2

The National Audit Office  There were some shortcomings in the  12 study team consisted of: commissioning process

Tom McDonald and Shamail Ahmad,  A lack of time and expertise resulted in  13 with assistance from Rebecca Lynch,  patchy preparations

Naomi Lock and Kajan Nithiananthan,

under the direction of Chris Shapcott There were widespread problems with contracts  14

Various factors undermined the rigours of  15 This report can be found on the National  competitive tendering processes

Audit Office web site at www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office

Press Office

157-197 Buckingham Palace Road Victoria

London

SW1W 9SP

Tel: 020 7798 7400

Email: enquiries@nao.gsi.gov.uk ' National Audit Office 2006

PART 3 PART 5

Out-of-hours providers are beginning to deliver a satisfactory standard

of service

Access problems do not seem to have compromised safety

Providers are not meeting all the Quality Requirements yet, especially those relating to speed of response

Patient experience is generally good, but one in five are dissatisfied

Management information is still poor

There is limited progress toward integration with other services

PART 4

Costs are higher than anticipated

There were misunderstandings about funding, in particular the £6,000 foregone by GPs

The Department made provision of £322m to reflect the known costs of the existing service, but the costs of the new service were higher

Costs are 22 per cent higher overall

There is scope to reduce costs in future by up to £134 million

GP pay rates remain the key driver of costs

18  Commissioners are entering into  34

contracts with multiple providers

and the market is maturing

19  The provider market is diversifying and maturing  35

Case studies and visits reveal much good practice  37 20

Commissioners are beginning to make  38 tough decisions

22  England performs well against UK and  38

international comparisons

24

APPENDICES

25

1  Methodology  39

2  Case Examples  41

3  Out-of-hours arrangements in  43 26 Scotland, Wales and Northern Ireland

27  4  Out-of-hours arrangements in Denmark,  45

Netherlands, USA and Australia

28  5  Quality Requirements  47

29 30

32

Cover courtesy of Mary Evans/Roger Mayne Photographs, inside photographs courtesy of Alamy.com and www.justinedesmondphotography.co.uk

SUMMARY

1  Most primary medical care takes place during the working day, but patients sometimes need care at other times as well. Such care is known as out-of-hours care, the term currently being used to describe the period from 6:30pm until 8:00am on weekdays, and all weekends, bank holidays and public holidays.

2  Out-of-hours services have undergone significant change since 2000, when the Department of Health (the Department) commissioned a review of these services in order to consider issues such as quality of care and links with wider NHS services. This review, known as the Carson Review1, made recommendations which, combined with the NHS Plan, set the foundations for current out-of-hours services.

3  Many General Practitioners (GPs) had already used powers granted in the mid-1990s to delegate out-of-hours provision to a third party. However, a new General Medical Services (GMS) contract came into force in 2004, which allowed GPs to opt out of the responsibility of organising out-of-hours care entirely from 1 April 2004. Where GPs opted out, they gave up an average of

£6,000 per annum and passed on responsibility to

their Primary Care Trust (PCT).

4  This report examines whether the Department is on the right track towards providing high-quality out-of-hours services. Appendix 1 sets out our methodology. Our work has found that:

n There were some shortcomings in the initial

commissioning process because PCTs lacked experience, time and reliable management data. There is also confusion over whether out-of-hours services should be restricted to urgent care.

n Out-of-hours providers are beginning to deliver a

satisfactory standard of service but most are not yet meeting all the national Quality Requirements, particularly on speed of response.

n In a survey of PCTs we found that the actual

costs of providing out-of-hours are £392 million, considerably more than the £322 million allocated by the Department.

n Commissioners are entering into contracts with

multiple providers and the market is maturing.

1 Named after the review panel s Chairman, Dr David Carson, a GP, who was head of primary care strategy and performance at East London & the City Health

Authority at the time of his appointment, and who had already assisted the Department of Health with its reviews of GP pay and GP postgraduate education.

summary

5  Our more detailed findings are as follows.

There were some shortcomings in the commissioning process

6  In 2005, many PCTs had not previously managed

or delivered out-of-hours services and so lacked both knowledge and experience in this area. There was

little, if any, reliable management information, for example on demand, activity and cost. This shortfall in information made it very difficult for PCTs to write service specifications and commission effectively. Some PCTs were, and remain, confused as to whether the out-of-hours service should be restricted to urgent care, or should respond to any request for medical care from members of the public. In addition, many PCTs allowed insufficient time for commissioning out-of-hours services, reducing the quality of the process.

7  Many contracts were signed late or not at all, with significant legal implications. This was due to poor service specifications, disagreements between commissioners and providers over risk-sharing, and the inability to reconcile PCTs limited budgets with providers estimated costs of meeting all the Quality Requirements. Our survey of PCTs, carried out jointly with the Audit Commission, found

that, where external providers deliver services, signed contracts were in place in only nine per cent of cases by the time the service began. This increased to 34 per cent

by 30 September 2005. Whilst services continued to be provided despite the lack of signed contracts, several providers told us that the lack of a formal contractual agreement forced them to carry extremely high

operational and legal risks.

8  Our survey found that 39 per cent of PCTs ran a competitive tendering process to award a contract.

Those that did not often cited departmental guidance, which stated that there was no requirement to undertake a formal tender exercise. Our survey found that the more rural a PCT was, the less likely it was to have undertaken formal tenders. The lack of competition from commercial providers in rural areas stems from the difficulty of achieving economies of scale. However, overall

we found that services which had been subject to a tendering process were no cheaper or better than those which had not.

Out-of-hours providers are beginning to deliver a satisfactory standard of service

9  Anecdotal evidence suggests that patients suffered longer waits in at least 50 per cent of England during

the first few days of the new service, but there is no indication that safety was compromised. Providers are

not yet meeting all the Quality Requirements, particularly on Saturday mornings when demand peaks. Patient

surveys run by PCTs show extremely high levels of satisfaction with the service provided. However, our survey of patients views of out-of-hours and other urgent care services found that they had had broadly good experiences, but one in five were dissatisfied. This suggests that there may be shortcomings in patient experiences that are currently not being captured by PCTs.

10  Despite upgrades and improvements to IT systems, management information is still poor, as demonstrated by the difficulties PCTs experienced in obtaining management data to complete our survey. This is not helped by inadequate call management technology in some areas or difficulties in using the Department s reporting template. There is also some confusion over the definition of compliance with the Quality Requirements, despite clear explanations in the accompanying commentary.

11  Limited progress has been made towards integration with other parts of the NHS, such as local Accident & Emergency Departments and ambulance services, but there are some individual examples of strong efforts to join up services. Further planning and commissioning of integrated services should reduce duplication and improve value for money.

Costs of providing out-of-hours services are higher than anticipated

12  Prior to the conclusion of the new GMS contract negotiations, the Department conducted an economic analysis of GP co-operatives to estimate the cost to GPs of providing the service and arrived at an approximate average figure of £9,500 per GP. The outcome of the negotiations for the new GMS contract was an agreed average opt-out figure of £6,000, although the precise amount for individual GPs varied depending on list size. The Department increased its out-of-hours development funding to around £3,500 per GP to help establish the new service, giving an average total of £9,500 for every GP opting out. Some 90 per cent of GPs decided to opt out, in line with what the Department told us were

their expectations.

13  The Department established a programme for PCTs to support the implementation of the new out-of-hours arrangements. The programme set out the expected average cost - of £9,500 per GP to provide out-of-hours services using the analysis that was completed in advance of the new GMS contract negotiations. The Department also explained their resource support package, which totalled an average of £9,500 per GP. Despite this, some PCTs failed to understand that the £6,000 opt-out sum was not intended to represent the true cost of the service, which led to many underestimating their costs. Our survey found that the actual costs of providing out-of-hours for 2005-06, the first full year of the new arrangements, were £392 million, 22 per cent more than the £322 million allocated by the Department, and an average of

£13,000 per whole-time equivalent GP.

14  The above funding gap may impact on investment

in out-of-hours infrastructure and staff training in the short term, but there is significant scope to reduce costs in future. Our analysis identified the best performing PCTs for each rural/urban classification in terms of quality levels and cost per head. Benchmarking PCTs in each category against the best suggested that if all PCTs matched the best, a saving of £134 million could be achieved without compromising quality. PCTs could make savings through a number of actions, including benchmarking themselves against similar PCTs and analysing local demand patterns to help patients access the service more appropriately.

Commissioners are entering into contracts with multiple providers and the market is maturing

15  There is now a wide array of new out-of-hours providers, including GP co-operatives, NHS Direct, PCTs themselves and private sector companies, and it is common for commissioners to enter into contracts with multiple providers to provide different elements of the service.

16  Whilst the GP-led model still predominates in both PCT-provided and commissioned services, we have seen evidence of various different models of skill mix, i.e. the employment of health professionals other than doctors in out-of-hours primary care. Providers tell us that changing skill mix increases the cost of the service in the short term, for example due to training costs, but savings are expected to materialise in the longer term. Many of the emerging staffing models are still quite small-scale and it is not yet clear how successful they will be in providing cost-effective performance. Our survey analysis found that the most cost-effective models varied depending on the rural classification of the PCT.

17  Providers are becoming more responsive to commissioners, who now have better management information and are taking decisions to penalise poor providers. England compares well both within the UK and internationally in terms of service structure and quality monitoring.

RECOMMENDATIONS

18  The Department should:

n Although PCTs have the primary responsibility

for out-of-hours services, the Department should nonetheless use all the levers at its disposal to encourage PCTs to improve the cost-effectiveness of the service through benchmarking of costs, improvements to local commissioning processes, and making available training and best practice.

n Ensure that commissioners and providers understand

the Quality Requirements and that they are aware that full compliance is an average performance of 95 per cent rather than 100 per cent, as set out in

the Department s guidance. The Department should also clarify the term definitive clinical assessment and consider how to focus the Quality Requirements further on quality and patient experience.

n Provide adequate training to ensure that providers

can use its reporting template effectively, and work in partnership with Adastra[2] to improve the management information which its various systems are producing to support performance management.

19  PCTs should:

n Benchmark their costs against those of other

geographically comparable PCTs to identify areas for improvement.

n Improve commissioners capacity in terms of writing

service specifications and market management in preparation for subsequent rounds of commissioning.

n Ensure that they understand local drivers of demand

to see if they can help patients to access the service more appropriately. They should conduct a thorough analysis of patient flows into all unscheduled care services in order to see the detail of case-mix and socio-economic groups using the different services.

n Ensure that they, or their providers, improve the

quality of their patient questionnaires and make the most of best practice from pilot and academic work to ensure realistic patient feedback.

n Use all the contractual and performance

management levers at their disposal to ensure that they or their providers meet the access requirements within the national Quality Requirements.

PART ONE

The previous out-of-hours service was unsustainable

Before 2004, GPs were responsible for ensuring the provision of out-of-hours services, although this was mostly delegated to GP co-operatives or the private sector. In response to public and Ombudsman concerns, and concern at the impact of this responsibility on the recruitment and retention of GPs, the Department commissioned a review of out-of-hours services in 2000. The review, known

as the Carson Review, and the NHS Plan, helped set the foundations for current out-of-hours services, for which Primary Care Trusts are now usually responsible.

The Carson Review and other strategic documents set out a vision for change

Before 2004 GPs were responsible for ensuring the provision of out-of-hours services for their patients

1. Most primary medical care takes place during the day, but prior to April 2004, General Practitioners (GPs) also had a responsibility to provide urgent medical

care during the out-of-hours period, which is now defined as from 6:30 pm until 8:00 am on weekdays, and all weekends, bank holidays and public holidays. Approximately nine million patients annually receive urgent primary care out-of-hours in England.

2. In recent decades, the responsibility for out-of-hours has become increasingly unpopular and GPs have sought ways to reduce the burden of out-of-hours cover, while

still discharging their duty of care. Under arrangements introduced in 1995, the Department encouraged GPs to co-operate more and to focus on premises-based care

in providing out-of-hours services. Apart from providing the service themselves, GPs could join a practice rota

or area co-operative, under which they could pool their responsibility through a rota system. In addition, GPs could employ a commercial deputising service. As a result, by the beginning of 2004, approximately 70 per cent of GPs had delegated the responsibility to a GP co-operative, and around 25 per cent to a commercial provider.

3. These trends helped reduce the scale of GPs personal involvement but, nonetheless, personal responsibility for the service remained unpopular among GPs, particularly among the growing numbers of female GPs.

The Department responded to public, Ombudsman & media concerns

4. Under the historic pattern of provision, neither the public nor the Department of Health (the Department) knew very much about the overall quality, costs or outcomes of out-of-hours provision. However, there

is anecdotal evidence that the quality of care varied considerably between different provider types and different geographical areas, and in early 2000, a rising number of complaints and negative reports in the media led the Health Service Commissioner (Ombudsman) to raise concerns about out-of-hours services with the Department.

5. This evidence led the Department to conclude that the existing model of out-of-hours was not sustainable. As a result, in March 2000 the Department announced a review of the arrangements for out-of-hours cover across England. The aim of the review was to identify ways of assuring quality and to make recommendations to improve services. The review was chaired by Dr. David Carson and the Carson Review 3 was published in October 2000.
6. The Carson Review made 22 recommendations, which were all accepted in full by the Department.

It identified a future model of out-of-hours care in which Primary Care Trusts (PCTs) would develop an integrated network of unscheduled care provision, bringing together providers of out-of-hours services to work collaboratively with other health and social care providers such as Accident and Emergency departments and ambulance services. The review also identified some core quality standards, to which all out-of-hours services should

be delivered in the future. Under a new accreditation scheme, the Department requested that all providers

of out-of-hours should achieve compliance with these standards by March 2004. The standards were reviewed in 2004 and then re-cast as the National Quality Requirements from 1 January 2005.

7. The NHS Plan4 was published mid-way through the work for the Carson Review, and built on its preliminary findings, whilst the 2001 report on Reforming Emergency Care5 helped clarify the Department s policy objectives for emergency services operating in the out-of-hours period. These three documents formed the background to current out-of-hours provision.

A new system of out-of-hours arrangements was implemented during 2004-05

GPs were relieved of the obligation to ensure provision of out-of-hours services

8. The new General Medical Services contract was negotiated between the NHS Confederation (the NHS employer s organisation) and the General Practitioners Committee (GPC) of the British Medical Association during 2002 and 2003. The Department acted as an observer. The new contract allowed GPs to opt out of responsibility for out-of-hours from 1 April 2004, at an average cost of £6,000 a year. Where GPs opted out, the responsibility for out-of-hours passed to the PCT with immediate effect, although PCTs who were unable to accept the responsibility could defer the transfer. The final deadline for deferral was 1 January 2005.

PCTs were given the opportunity to re-design services under Shifting the Balance of Power

9. In taking over responsibility for out-of-hours, PCTs were given the task of developing integrated networks of urgent care services. The Department had been working with PCTs since the publication of the Carson Review, with the aim of encouraging them to use their increased autonomy under Shifting the Balance of Power6 to make the Carson Review s aspirations of integrated care a reality. The Carson Review also developed a support programme which included work with the National Association of

GP Co-operatives to identify best practice, support for wider commissioning and an Exemplar scheme which

was designed to help sites covering around 20 per cent

of the population of England integrate their services

with NHS Direct.

3  Raising Standards for Patients: New Partnerships in Out-of-Hours Care. An Independent Review of GP Out-of-Hours Services in England, 2000. 4  The NHS Plan, Department of Health, 2000.

5  Reforming Emergency Care, Department of Health, 2001.

6  Shifting the Balance of Power within the NHS, Department of Health, 2001.

10. Following publication of the new GMS contract, the Department issued guidance to PCTs to help them in commissioning out-of-hours services. The guidance provided for out-of-hours services to be delivered under one of four contractual frameworks and stipulated that:

n either GPs or PCTs would have responsibility for

out-of-hours, depending on whether the GP opted out;

n in both cases, the service would either be delivered

in-house or contracted out to an external provider; and n responsibility for compliance with the Quality

Requirements[2] would remain with whoever was

responsible for the out-of-hours services i.e. either

the GP or the PCT.

11. The final date for the transfer of responsibility for out-of-hours services from GPs to PCTs was 1 January 2005. By this point the 90 per cent of GP practices which wished to transfer had done so. As at April 2005, the Department s understanding, based on data gathered from Strategic Health Authorities, was that some 75 per cent of service provision was PCT-organised or contracted through co-operatives of various types, with the remaining 25 per cent provided by commercial providers, ambulance trusts and others, with NHS Direct supplying initial call handling and triage for many providers.

PART TWO

There were some shortcomings in the commissioning process

PCTs were new to the responsibility of providing out-of-hours services. This lack of experience, combined with a busy agenda and poor management information on which to base commissioning decisions, reduced the quality of the process.

Poor service specifications, budget constraints

and disagreements between commissioners and providers over risk-sharing often resulted in services being commissioned and provided without a signed contract in place. Commissioners did not always run competitive tendering processes and competition was often undermined by immature markets or commissioners preferences for particular providers.

A lack of time and expertise resulted in patchy preparations

prior to handover and that 78 per cent conducted some form of needs assessment prior to specifying a new service. However, whilst many PCTs made efforts to gather what management information they could, when we spoke to those who did, two thirds said that this information was of poor quality or was simply not available. In some cases, the lack of information was due to providers reluctance to provide too much detail in case they were challenged to make savings as a result.

2.2  Examples of where good data were unavailable include: records of demand data, such as case and morbidity mix; records of activity levels, including for peak periods; and operating costs. Although the lack of good data was not as problematic for PCTs who were taking an existing service in-house or commissioning from existing providers, it did make it difficult for new providers and commissioners to plan new services.

Many PCTs took on a service for which there was little reliable data, which undermined specifications

2.1  Because out-of-hours services were not previously delivered or managed by PCTs, they tended to know relatively little about it. In a survey we conducted of PCTs and from visits to PCTs, we found that 91 per cent of PCTs did conduct some analysis of how the service was provided

There was confusion over whether the out-of-hours service should be restricted to urgent care

3. Many providers and commissioners told us that there

[2] was ongoing confusion about whether out-of-hours was

supposed to be an urgent or unscheduled care service,

and that the difference was not merely linguistic.

4. They explained the difference as follows. A truly urgent primary care service would likely treat patients classified as either emergency or urgent and all

others would be asked to make an appointment to see

a GP in-hours the next working day. Demand would

be cut and providers could focus on meeting Quality Requirements for patients requiring advice or care within short time-frames. An unscheduled care service, however, would be more responsive to patients and would not

seek to restrict access, no matter how minor the injury or illness. Since access would be unrestricted, this service could be more costly, but might provide more flexibility for patients and could interact better with existing daytime primary care services by allowing continuity of care.

5. Commissioners and providers would like the Department to decide which kind of service they should provide, as they feel that currently they are providing a hybrid model, with resulting confusion for commissioners, providers and patients. For example, we found anecdotal evidence of patients using out-of-hours services for non-urgent purposes and providers being unwilling to turn them away in case they complained. The recent primary care White Paper9 does not clarify whether out-of-hours services should be urgent or unscheduled. It does, however, put forward the possibility that out-of-hours providers could run evening surgeries and take on booked appointments and registered patients, suggesting a move away from strictly urgent care provision.

Some PCTs did not leave enough time for the process

6. During 2004 PCTs had an extremely busy agenda. Alongside their core work of commissioning and running a wide range of care through primary and secondary providers, they were also dealing with the new General Medical Services contract, the Agenda for Change programme, preparations for Payment by Results and Patient Choice, the introduction of increasing private sector provision and other major initiatives.

7. Many PCTs told us that these issues took priority over out-of-hours in terms of management time and attention. As a result, some PCTs did not leave enough time to plan and commission or provide what was for many a new and unknown service. This led to the range of problems set out below, many of which reduced the quality of the commissioning process.

The few PCTs who engaged early with GPs reaped benefits

8. It is clear from the evidence we gathered from providers and commissioners that those few PCTs who had the foresight to engage early with their GP communities reaped benefits in several ways. Hereford PCT (see Case Example 1, Appendix 2) was an example of a commissioner who saw the possibilities of improving their out-of-hours service and who engaged with GPs and other providers early to good effect. Early engagement with GPs often gave commissioners and providers a better chance of keeping them involved in the provision of the service.

There were widespread problems with contracts

Many contracts were signed late or not at all, with legal implications

9. Responding to concerns raised by the National Association of GP Co-operatives in January 2005 that many contracts had not been signed, the Department stated that this was indeed true for some PCTs. Concerned that, where this was the case, PCTs would have no contractual levers to ensure Quality Requirements were being met or to reconcile difficulties with providers where no enforceable dispute procedures were in place, the Department therefore used its established performance management route through the Recovery and Support Unit and Strategic Health Authorities to try and ensure that contracts were in place.

9  Our Health, Our care, Our Say: a new direction for community services, Department of Health, 2006.

10. However, our evidence below shows that this was not wholly successful. Responses to our survey suggest that, where external providers deliver services, signed contracts were in place in only nine per cent of cases in time for the service to begin. This figure increased to 34 per cent by 30 September 2005, the date on which they submitted their returns to us.
11. According to commissioners and providers that we interviewed, the two most common reasons for the lack of signed contracts were negotiations over cost and quality, and situations where PCTs were jointly commissioning in consortia. The negotiations over cost and quality mostly reflected an inability to reconcile a PCT s limited out-of-hours budget with the provider s estimates of running a service that would meet all the Quality Requirements. In the other scenario, the main difficulty was obtaining timely and co-ordinated decisions from all the relevant committees in the consortium.
12. Several providers told us that the lack of contract signatures meant that they had to carry extremely high operational and legal risks, even if there was a heads of agreement or some other temporary arrangement in place. They also stated that they felt compelled to run the service, even where contracts had not been signed, in order not to go bankrupt or fall out of favour with their commissioners.

Various factors undermined the rigours of competitive tendering processes

Markets were not mature in some places so competition was difficult

13. The legacy of previous out-of-hours services and the geographical differences between various PCT areas meant that it was more difficult to hold a competitive tendering process in some areas than in others. For example, rural areas may in the past have had a GP co-operative made up of local doctors which might not be easily replaced with a new provider, since the latter might find it harder to cover a wide area at a similar cost.

14. Even in urban or semi-urban areas, it was not necessarily possible to hold competitive tendering processes, since many providers were amalgamating, dissolving or re-considering their market position and might not have wanted to bid for a particular contract.
15. Departmental guidance on implementing the

new GMS contract stated that undertaking a formal

tender exercise was not compulsory under European public procurement rules. The guidance also left the

final decision to PCTs as to what the best process might be. Our survey found that 39 per cent of PCTs ran

a competitive tendering process in order to award a contract. The remaining 61 per cent did not for a variety of reasons. Some did not tender because they did not

have sufficient time to do so; others because they had a preferred provider in mind; and the remainder because they wished to provide the service in-house. Since the majority of PCTs did not invite other providers to bid for a contract, they did not know if they struck the best deal.

16. On the basis of responses to our survey, for each PCT we calculated the number of quality points achieved per pound spent. Quality points were awarded for each quality requirement, or part thereof, that the PCT stated

it was meeting. A maximum of 23 points were available. Figure 1 overleaf shows PCTs quality scores against their cost per head for services which were put out to tender

and Figure 2 overleaf shows PCTs quality scores against their cost per head which were not put out to tender.

They suggest that there is no correlation between cost and quality in either case.

17. Our survey found that the more rural the PCT, the fewer contracts were put out to tender. Of all the contracts entered into by PCTs classed as Major Urban (see paragraph 4.18), 70 per cent were awarded following a formal tender exercise, compared with only eight per cent of PCTs classed as the most rural. This can be attributed to the lack of competition in rural areas due to the relative difficulty

in providing a cheaper service than the one (usually a GP co-operative) previously in place. Rural PCTs had a much higher cost per head. However this was not due to the lack of tendering. Looking at all PCTs who tendered for the service, their average cost was £8.65 per head. This was only £0.29 per head cheaper than those who were selected without a tender.

1 Relationship of quality scores against cost per head for out-of-hours services which were put out to tender. The graph shows no correlation between

cost and quality.

Actual cost per head (£) 20

15

10

5

0

0 5 10 15 20

Quality score

Source: National Audit Office

18. In terms of average quality scores, there was no variance at all, with both types of provider averaging a score of 10.9 (out of a maximum 23 points10).

This demonstrates that a competitive tendering process would not automatically have helped commissioners strike a better deal with providers in this first round

of commissioning.

Some PCTs chose preferred providers or took existing services in-house to deliver quickly but there were some possible conflicts

of interest

19. Although a competitive tendering process was

not always necessary to select the best provider, its absence, or ineffectiveness, sometimes undermined the new service. Where PCTs did run competitive tendering processes, we found some evidence of possible conflicts of interest which may have undermined those processes. For example, in response to our survey, 16 per cent of PCTs recognised there had been a conflict of interest between people responsible for organising the tendering process and those providing the services.

20. In many cases PCTs knew little about the service they were commissioning and so naturally sought help from those who did. Unfortunately this meant that in some cases the proper separation which should have existed between commissioning and providing functions was breached. We found isolated examples of:

n contract specifications and prices being drawn up

by providers, who were then awarded the contracts virtually unchallenged; in 64 per cent of responses to our PCT survey, the provider gave assistance in drawing up the service specification; and

n provider staff also sitting on PCT boards or

Professional Executive Committees, which awarded contracts to their own organisations.

21. While it was understandable for commissioners dealing with a new service to seek advice from those with the expertise, they will have to ensure that stricter and more transparent processes are followed in future in order to avoid legal challenges from unsuccessful bidders and to protect value for money.

10  What looks like a low score for average performance can be partly explained by PCTs mistaken understanding that compliance must be 100 per cent.

If judged against the Department s actual compliance level of 95 per cent, some of the scores would be higher.

PART THREE

Out-of-hours providers are beginning to deliver a satisfactory standard of service

Serious access problems occurred during the handover of out-of-hours services, but there is no evidence that safety was compromised. Service has improved, but the vast majority of providers are not yet meeting all the new Quality Requirements. Patient experience has generally been good, but one in five are dissatisfied.

The main commercial IT supplier for out-of-hours, Adastra, has made upgrades and improvements to IT systems, but managers ability to produce high quality information is still poor, as demonstrated

by the difficulties PCTs experienced in obtaining data to complete our survey. There has been limited progress towards integration with other services, but there are some individual examples of strong efforts to join up services.

Access problems do not seem to have compromised safety

Patients in many areas suffered long waits during the handover period

1. Although some services were handed over to new providers in the autumn of 2004, many new services began at the beginning of 2005. This had the disadvantage of testing new services during a period of high demand which included public holidays (when normal in-hours GP cover would be limited) and cold weather. The Department told us that it had shared the concern of a number of PCTs that delays and confusion during the handover period might lead to large numbers of adverse patient incidents.

2. Our discussions with providers and commissioners revealed that some patients did in fact suffer long waits during the first few days of the new service. At least

50 per cent of providers we spoke to about the handover struggled to meet the Quality Requirements on access times. However, their view was that there was no evidence that these delays resulted in adverse patient incidents. Furthermore, they did not subsequently receive significant numbers of patient complaints, although they did acknowledge that patients who were unable to access their services might have gone elsewhere in the NHS for treatment and therefore remained unrecorded.

3. Specific difficulties reported during the handover period included:

n delays for patients accessing Saturday morning

services, because a lack of data or lack of planning had left providers poorly prepared for those sessions. The Department s view was that a large number of PCTs had failed to inform the public adequately about the change in Saturday morning provision;

n mixed results for public engagement. In some cases,

local publicity campaigns had successfully reassured and educated patients about how to access local services. In other cases, similar kinds of campaigns simply increased demand by raising awareness of the service;

n delays in responding to telephone calls led to

patients making repeat calls and further blocking the system; and

n widespread difficulties in meeting access targets

set out by the Quality Requirements. This was even more challenging for those providers who had not yet installed call management equipment.

Providers are not meeting all the Quality Requirements yet, especially those relating to speed of response

The situation is a great improvement from prior arrangements

4. Since 1 January 2005, out-of-hours providers have had to meet the national Quality Requirements as a contractual obligation (See Appendix 5 for a full list of the Quality Requirements). Virtually all providers and commissioners that we interviewed agreed that the current set of Quality Requirements was better than its predecessors, helped them to assess performance over time and was focused on the right elements of the service.
5. However, many also said that real improvements could, and should, be made to the Quality Requirements. There was general agreement from our meetings and from our expert panel that further work should be undertaken

to measure the quality of service delivered to the

patient. Our interviewees recognised that this would be challenging and would require a mixture of numerical and softer, qualitative analysis, but that this should be the focus for future improvements to the Quality Requirements.

6. There is, however, some uncertainty as to what

is currently required, despite clear definitions in the Department s guidance on the Quality Requirements

and their performance management. In this guidance,

the Department states that providers should aim to be

100 per cent compliant with the Quality Requirements. However, the guidance also says that average performance of 95 per cent and above would in fact represent

full compliance and that average performance of

between 90 per cent and 94.9 per cent would represent partial compliance.

7. Our meetings with providers and commissioners revealed some confusion in this area. Around 80 per cent of those we spoke to about the Quality Requirements believed that they should be aiming for 100 per cent compliance with all the Requirements and they did not seem to be aware of the Department s more nuanced view of performance. This has in turn led to arguments about acceptable levels of performance, particularly where the service is the subject of a formal contract. Some providers have noted that if they meet a particular standard

98 per cent of the time for a given period, they will

be deemed as not having met it and can be penalised, especially if the PCT is taking a rigid approach to performance management. The provider s view in this kind of case seems closer to the Department s guidance in allowing for variation with reasonable bounds. Whatever the merits of individual situations, it is clear that there is some confusion here which requires clarification.

8. Another area of confusion which emerged during our meetings was the definition of definitive clinical assessment . We found that some providers recorded their performance against Quality Requirements differently to others, meaning that some patients are being given lower standards of care than others. For example, where one provider might regard the decision to refer a patient on to a doctor as definitive clinical assessment, another would regard the subsequent conversation between doctor and patient as being the definitive assessment point.
9. The Royal College of General Practitioners told us that it was concerned about the quality of out-of-hours care, particularly where a service is delivered by a number of providers and also in relation to the training and accreditation of health care professionals other than GPs. The College was concerned that Quality Requirements were not adequately monitored or enforced. One area of particular concern was Quality Requirement 11, where the College felt it was particularly important for patients to have access to a GP where clinically appropriate.

The College also noted that training for GP Registrars

in urgent care was essential and that all GPs should be competent in this area of medicine.

Performance against the Quality Requirements is patchy

10. Our detailed analysis of PCT responses to our survey reveals some good performance, but also some interesting shortfalls in performance. Figure 3 sets out PCTs overall performance against 100 per cent compliance with the Quality Requirements. Clearly, Quality Requirements 8, 9, 10 and 12, which set out a series of targets for telephone and face-to-face access, are the most challenging to meet. This data is supported by findings from our interviews and expert panel.

11. Performance against some of the more important Quality Requirements are as follows. 94 per cent of respondents said they ensured that their provider reported to them regularly on performance, i.e. weekly, monthly or quarterly (Quality Requirement 1). However, 39 per cent of respondents stated that they were able to send details of all consultations to the patients registered practices by 8am the following day (Quality Requirement 2).

12. Eighty-two per cent of respondents reported that their provider regularly audited a random sample of patient contacts and took appropriate action on the results, however only 60 per cent of respondents had these reports made available to them (Quality Requirement 4). Eighty-two per cent of respondents confirmed that their provider regularly audited a random sample of patient experiences and took appropriate action on the results (Quality Requirement 5), whilst 99 per cent confirmed that they have complaints procedures in place (Quality Requirement 6).
13. Fifty per cent of respondents met the Quality Requirement for engaged calls (less than 0.1 per cent of calls should be engaged) and 27 per cent of respondents met the Quality Requirement for abandoned calls

(less than five per cent of calls should be abandoned).

Two per cent of respondents answered calls within

60 seconds of the introductory message finishing, if they had a recorded message. This rose to five per cent for

calls which had to be answered within 30 seconds (in the absence of a recorded message). These low scores against the telephone-related targets of Quality Requirement 8 may be explained partly by the demanding nature of the targets and partly by some instances of providers simply lacking the call management technology with which to measure it.

14. Just under 10 per cent of respondents fully met

the targets for telephone clinical assessment (Quality Requirement 9), whilst just over 20 per cent of respondents met the targets for face to face clinical assessment (Quality Requirement 10). The responses

for face to face consultations were marginally better for home visits (an average of 19 per cent across the three classifications) than for primary care centres (an average of 18 per cent).

Saturday mornings are particularly difficult

15. All providers agreed that demand was particularly strong on Saturday mornings and that it was frequently difficult to meet the access targets. Many thought that Saturday mornings should not have been included in the out-of-hours period as defined by the new GMS contract and that GPs should still conduct their own Saturday morning surgeries. However, some PCTs have been making imaginative use of their commissioning options to put alternative Saturday morning arrangements in place, as is illustrated by Case Example 2 (in Appendix 2).

16. Quality Requirement 7 relates to the provider s ability to meet service peaks such as Saturday mornings. It also mentions Sunday mornings and Bank Holidays

as other demand peaks and many providers and commissioners confirmed to us that it was also difficult to meet the access targets on these days. However, providers also told us that they were already planning for the Christmas holidays (in August and September) and felt much better placed to cope having been through one Christmas/New Year period already.

Patient experience is generally good, but one in five are dissatisfied

Patients had broadly good experiences across a representative NAO survey sample

17. The National Audit Office commissioned MORI to undertake a survey of the public to ascertain their views and experiences of out-of-hours services. The objectives of the survey were to ascertain awareness and usage of out-of-hours services and to measure satisfaction with various aspects of the service. Further detail on the survey scope and methods can be found in Appendix 1 and the complete MORI report can be found at www.nao.org. uk. When surveying respondents, MORI did not clarify what was meant by the term out-of-hours services and it is therefore likely that, in at least some of their answers, respondents are referring to services other than their GP-led out-of-hours service. Other services might include walk-in centres or Accident and Emergency departments.
18. The survey found that 81 per cent of respondents had not tried to obtain out-of-hours medical care in the last six months, either for themselves or someone they were caring for. Of those that had tried, 12 per cent had done so once, four per cent had done so twice and the remaining three per cent three or more times.
19. For the six-month period covered by the survey, usage of out-of-hours care was more common amongst some groups than others, in particular:

n women - 22 per cent of those questioned have used

the service once or more, compared to 19 per cent of respondents as a whole;

n those aged 35-54 - 23 per cent have used the service

once or more; and

n those with a child aged under 16 in their household

- 26 per cent had used the service. This increased to 33 per cent for those with two or more children.

20. Those respondents who had not required out-of-hours care were asked how they would go about trying to obtain it if they did need it. Just under half (47 per cent) said that they would call their local GP surgery and a further 13 per cent said that they would call NHS Direct. Other suggestions were: going to Accident and Emergency departments (ten per cent), calling a number given by a GP surgery (seven per cent) and calling 999 or calling the local hospital (four per cent each). These figures suggest that, even people who have not tried to access the service recently, and who may not know much about it, will normally try and telephone before travelling somewhere.
21. Of the 19 per cent of respondents who had accessed some form of out-of-hours service, 44 per cent had in fact travelled somewhere to see a doctor or nurse. 23 per cent had had a telephone conversation with a doctor, 18 per cent had had a telephone conversation with a nurse, 13 per cent had received a home visit from a doctor and two per cent had received a home visit from a nurse.
22. Quality Requirement 12 states that following a definitive clinical assessment, face to face consultation should commence, either in a centre or the patient s place of residence within one hour for emergencies, two hours for urgent cases and six hours for less urgent cases. Those respondents receiving home visits reported a wide range of waiting times, with the mean length of wait being five hours and 39 minutes. 61 per cent of respondents visited by a doctor or nurse waited less than two hours and 40 per cent waited less than one hour (Figure 4). A small number of respondents waited an extremely long time for their visits, although for some there were extenuating circumstances, such as patients themselves requesting a delay.
23. Service users who had telephone contact only with a doctor or nurse (including NHS Direct) were asked how long it took before the healthcare professional called them back. Quality Requirement 9 states that definitive clinical assessment must start within 20 minutes of an urgent call being answered and 60 minutes for all other calls. Where calls cannot be safely and effectively prioritised, the provider s target must be 20 minutes. Overall, our survey found that two thirds (65 per cent) of respondents were called back within 60 minutes and 30 per cent within

20 minutes (Figure 5). This leaves 35 per cent of all call-backs failing the quality requirement, even if the call is classed as non-urgent.

24. In terms of satisfaction with the service, 63 per cent of users rate the quality of care as good or excellent. However, 19 per cent think the quality of care is quite poor or very poor (Figure 6 overleaf).

4. Waiting times to see a doctor or nurse

Percentage of respondents 45

40

35

30

25

20

15

10 5 0

Less than 1 to 2 2 to 3 3 to 4 4 or more 1 hour hours hours hours hours

Time Source: MORI poll for National Audit Office

5. Waiting times for a doctor or nurse to call back

Percentage of respondents 30

20 10 0

Less 10 to 20 to 30 1 to 2 2 or than10 20 30 minutes hours more minutes minutes minutes to an hours

hour

Time Source: MORI poll for National Audit Office

6. Quality of out-of-hours care

Percentage of respondents 40

30 20 10 0

Excellent Good Fair Quite Very poor poor

Quality Source: MORI poll for National Audit Office

Management information is still poor

Systems are gradually being upgraded but it can still be difficult to extract the required information

3.28 Although IT systems in use at providers are being upgraded and improved, there are still a number of specific areas that we identified through our visits and in discussion with Adastra, the main commercial IT provider for out-of-hours, which make it difficult to extract the information required for the purposes of effective management. They are as follows:

n some providers report to a number of different

commissioners, with different degrees of detail or frequency. This can mean that, although they may not possess advanced IT or analytical functions, they have to spend large amounts of time trying to cut their data in different formats;

n

25. Most users appear satisfied with the advice they have received, with 72 per cent saying that the advice was fairly

good or excellent. Fifteen per cent said the advice made

no difference whilst fewer than one in ten think the advice

they received was wrong to some extent or totally wrong. n

Shortcomings in patient experiences from the

NAO survey are not mirrored by PCT views n

26. Whilst our survey of patient experiences paints

a largely positive picture of out-of-hours services,

it does reveal some dissatisfaction. However, PCTs

views of patient experiences are slightly different. Both commissioners and providers patient experience data

show extremely high satisfaction ratings in the various

patient surveys they have undertaken. There seems

therefore to be a discrepancy between what patients have  n told us and the messages service providers have received, suggesting that providers are currently not capturing

negative feedback.

27. One reason for this discrepancy might be that providers have been continuing to record satisfaction levels (as required under the old Quality Standard), rather than the actual patient experience, as the new Quality Requirement demands. Another reason might be that,

for some questions at least, patients in our survey were giving their views of both out-of-hours providers and other services, as noted above.

while Adastra software is capable of producing good management information, it does depend on the provider understanding how to configure their database to enter relevant information and then doing so in a consistent fashion;

there are some difficulties in matching local nomenclature (activity types, priorities etc.) to commissioners reporting requirements;

where a call is first taken by NHS Direct (and the quality requirement clock has started), but then handed on to a dedicated out-of-hours provider,

the latter does not necessarily know how long the patient has already been waiting. Not knowing when the patient first entered the system makes it difficult to tell whether or not Quality Requirements have been met;

reasonable and routine exceptions to response time standards are currently penalised. This would cover situations where there is no-one at home when a call is returned, or where a caller elects to delay an appointment because they are waiting for childcare arrangements. Currently these situations are seen as shortfalls in provider performance;

n many providers complained that the Department s

own reporting template, an electronic spreadsheet designed to simplify the reporting process, was difficult to use. The view of our expert panel was that, although technical improvements to the template could be made, the Department needed to consider further training for users;

n it is not clear how robust local protocols for

categorising emergency, urgent and other calls are. Work done by the Healthcare Foundation[2] has shown considerable variance in both initial categorisation and eventual outcomes which may have serious implications for patient safety; and

There is limited progress toward integration with other services

There are many individual examples of efforts to join up unscheduled care

3.31 As a result of the Carson Review s recommendations and Technical Links programmes run by the Department, there is now a network of local communications hubs across out-of-hours providers in England, which is helping to integrate providers systems and enable data sharing. In addition, providers in various parts of England have been developing more integrated arrangements with other members of their emergency care networks.

n a number of providers have inadequate call

management technology. This means that they  3.32 We found from our visits and survey that one model are simply unable to report on those Quality  some providers use is to introduce some form of physical

Requirements relating to telephone access. integration with the Accident and Emergency department. This could take the form simply of a GP sitting in an

Many PCTs had difficulty in sourcing basic  Accident and Emergency department and treating those data for our survey patients who are not acute or emergency cases in order to

reduce the flow into the trust. A more developed version

29. It is revealing of the difficulties commissioners and  of this is for the out-of-hours provider to build or lease providers have with management information that many  consulting rooms adjacent to Accident and Emergency PCTs found it difficult to gather meaningful data for our  departments in order to triage patients requiring primary joint survey. or non-emergency care.
30. Respondents told us that they had particular

difficulties where they commissioned from several  These efforts do not add up to full integration different providers. For example, where their initial  from the patient s perspective yet

call handling and their consultations were provided by  3.33 However, the numbers of PCTs involved in such two completely separate organisations, commissioners  arrangements are small. Most of these arrangements are struggled to amalgamate the two sets of data. In addition,  still in their early stages and do not yet represent full where PCTs commission in consortia, they told us that it  integration from the patient s perspective. This means was often impossible to split out activity data or quality  that the Carson Review s vision of a seamless care requirement performance by PCT. The implication of this  pathway initiated by a single telephone call has not been is that some PCTs may not actually know what level of  realised in most places and that the pace of integration service their own patients are receiving.  is slower than the Department would have liked. As well

as improving the patient s experience, further planning and commissioning of integrated services should reduce duplication and improve value for money.

PART FOUR

Costs are higher than anticipated

The inclusion of the out-of-hours opt-out in the new GMS contract contributed to the success of the negotiations. The £6,000 foregone by GPs opting out of out-of-hours was close to the average cost to GPs from their income, before accounting for PCT out-of-hours development funds. Using economic analysis, the Department calculated the average cost of the existing service at £9,500 per GP, from

a combination of out-of-hours and GP funds, and ensured that this resource was available to the service. However, many PCTs did not understand this and by basing the cost of out-of-hours on the £6,000 opt-out sum, significantly underestimated their costs.

Our survey of PCTs found that the cost of providing new out-of-hours services is 22 per cent greater than the combination of the funds allocated by

the Department and those funds given up by GPs. This may be due to a number of factors, including the failure to generate efficiencies, underestimates of costs and increases in GP pay rates. However, there is significant scope to reduce costs in future commissioning rounds.

There were misunderstandings about funding, in particular the £6,000 foregone by GPs

The Department wanted to make general practice more attractive to doctors

1. Twenty-four-hour patient responsibility was deeply unpopular amongst GPs and the Department told us that it believed that the opportunity to opt out was a key lever in the negotiation of the new contract. It also considered the widening of the definition of the out-of-hours period to include Saturday mornings to be an attractive feature of the negotiations for GPs.
2. The Royal College of General Practitioners told us that they did not collect data on the effect of the opt-out on recruitment and retention but that there was a widespread feeling of relief among GPs that they no longer had a 24-hour contractual responsibility. The College thought that this had undoubtedly had a positive effect on morale and believes that it will make a career in general practice more attractive. The College was also keen to stress that the opt-out was only from the contractual responsibility and that, in many areas, GPs continue to support and develop GP co-operatives and participate in other PCT arrangements.

The £6,000 figure was the outcome of the new General Medical Services contract negotiations

3. Out-of-hours services had been funded from a combination of GP contributions and central development funds since 1995. In deciding on an annual sum for

GPs to forego if they wanted to opt out of out-of-hours, the Department s primary aim was to encourage GPs to accept the new GMS contract. The cost of the opt-out was therefore a negotiating sum for this purpose and

was not intended to be a precise reflection of the cost of providing out-of-hours services. As noted in Part 1, the Department was an observer at these negotiations, which were conducted between the NHS Confederation and the British Medical Association.

4. Prior to the negotiations, the Department conducted some economic analysis of GP co-operatives, analysing figures for urban, rural and mixed areas, to estimate the cost to GPs of providing the service. The costs, including funds specifically allocated to PCTs for investment in GP practices, varied from £7,000 per year to £14,000 per year per GP, with a mean of approximately £9,500. This figure informed the contract negotiations with the outcome

of those negotiations being agreement of an amount of about £6,000 to be foregone by GPs from their income, but which did not include out-of-hours funding from PCTs. However, the precise sum varied between GPs, depending on list size and other factors. In line with what the Department told us were their expectations, around 90 per cent of GPs were prepared to forego this amount and opted out.

5. In addition to the £6,000 sum refunded by GPs, the Department provided development funding to PCTs of some £3,500 per GP to help establish the new service. The Department told us that this package of development funds and the funds given up by GPs, giving an average allocation of £9,500 per GP who opted out, would be sufficient on average to deliver services. Their expectation was that they would make additional funding available for areas in which this was not the case. However, the Department also hoped that increased integration with other services would drive costs down in due course.

The breakdown of funding available is set out below.

The £6,000 opt-out figure led many PCTs to underestimate their own costs

6. The Department is clear that it funded the service based on the average costs per GP of £9,500. It also set up a programme to support PCTs in implementing the new out-of-hours arrangements. This included providing information on the anticipated costs and the additional resources available to meet the cost. Despite this, some PCTs did not understand that the £6,000 opt-out sum was not the full cost of the service. The view of the National Association of GP Co-operatives and anecdotal evidence from our visits to providers and commissioners suggests that there was a widespread misunderstanding that the sum would be enough to cover the costs of out-of-hours.

The Department made provision of £322m to reflect the known costs of the existing service, but the costs of the new service were higher

Extra money was made available through out-of-hours development and rural funds

7. The Department established the following set

of funding arrangements, in order to ensure that commissioners would be able to provide out-of-hours once GPs had opted out. For the financial year 2004-05, the Department provided the following funds:

n the opt-out monies which would total £180 million

if all 30,000 GPs opted out;

n a ring-fenced development fund of £92 million;

n £14 million to support PCTs facing the biggest

challenges in developing out-of-hours services, such as those covering highly rural or highly urban areas; and

n £30 million (£100,000 for every PCT) in capital

incentives to reward PCTs for having robust arrangements in place for the handover of out-of-hours and, subsequently for providing high-quality, sustainable services. The Department told us that all of the capital incentive funds were disbursed to PCTs on the basis of Strategic Health

Authority judgements that their PCTs complied with the conditions. Given our findings that providers are not yet meeting all the Quality Requirements, we believe that this money was not spent on the basis of proven performance.

8. Taken together, up to £316 million was available for the provision of out-of-hours services in 2004-05. In addition, a centrally held fund of £4 million was available to support the Technical Links programme.
9. However, in addition to this, the Department was keen for PCTs to use their unified budgets to commission care in a new, integrated fashion. These unified budgets amounted to a total of £49.3 billion and the Department wanted PCTs to use some of this money to establish integrated networks of high quality out-of-hours and urgent care provision.
10. For the financial year 2005-06, similar funds were available with the following changes:

n there were some changes to the allocation formula

for the £92 million development fund. However, the Department stated that no PCTs would receive less than they did in 2004-05;

n the £30 million capital incentives were

non-recurrent and were therefore not available for this period;

n an additional £33.4 million was available for PCTs for out-of-hours and urgent care development;

n £3 million was made available to the 53 PCTs

involved in the Exemplar Programme for integrated out-of-hours services; and

n the Department made additional out-of-hours

funding available to PCTs in 2005-06, following new arrangements for commissioning NHS Direct. PCTs were allocated funding for NHS Direct enhanced services, including call handling and triaging out-of-hours GP calls. However, these services were contestable, i.e. commissioners could choose not

to commission these services from NHS Direct and commission a different provider instead.

In total, up to £322 million was made available to PCTs for out-of-hours services in 2005-06.

Costs are 22 per cent higher overall

Survey analysis shows a contracted cost of £380 million and likely actual cost of £392 million

11. The Department has not sought to calculate the actual costs of out-of-hours services. We therefore asked PCTs a number of questions about their costs through our survey. We asked PCTs what their contractual cost (or budgeted cost in the case of in-house services) was for the current financial year. We also asked them for the actual cost of the most recent quarter for which they had information.[2] For around 95 per cent of respondents, this was April-June 2005. We then extrapolated the costs of this quarter to derive an estimated cost for the whole year. 95 per cent of PCTs responded to our survey.
12. Our survey analysis shows that the actual costs of providing out-of-hours services are considerably more than the £322m allocated by the Department. PCTs responding to our survey reported that the contractual cost of providing out-of-hours services for 2005-06 was

£369 million and the actual cost was £380 million.

When this was extrapolated over the entire PCT population, contract and actual costs of £380 million and £392 million respectively were derived, giving financial commitments of 18 and 22 per cent over and above the funds provided by the Department. The Department s view was that increases in GP pay rates may have contributed to cost pressures in many areas. Anecdotal evidence from our interviews suggests that this is correct, although we were unable to quantify the scale of any rises.

There are considerable financial implications for PCTs

13. Even for those PCTs who foresaw the need to top

up the Department s allocations, the financial impact of this increase in cost has been considerable. All PCTs with whom we discussed costs have been forced to look for additional funding from other budgets or have entered into negotiations with their providers about how to reduce costs.

There is a likelihood of a funding gap while the market matures

14. For some PCTs, therefore, it will not be possible to reduce actual costs to the allocated funding level in the short term. We describe below how PCTs can reduce their costs in future, but for some, the combined effects of immature markets, high demand and high costs will mean that they will have to draw on their unified budgets to top up their out-of-hours funding before the impact of improvements kicks in.

n analysing case-mix to see if particular patient groups

can be targeted by specialist primary or secondary care teams in order to reduce those patients reliance on the out-of-hours service;

n commissioning integrated urgent and unscheduled

care services in order to reduce duplication; and

n providers making further operational improvements

to deliver more effective utilisation of infrastructure and staff.

4.17 In addition, analysis of data from our survey suggests

Contracts based on cost alone are not  that further savings could be made across the entire PCT allowing investment in training, facilities  community. We split PCTs into six categories by rural

or innovation classification13 and then assessed their performance in

each category. Using the PCT survey data, we calculated

4.15 Many providers complained that these shortfalls  the cost per head of the service, based on the total in funding were leading PCTs to let contracts that were  actual costs and the opt-out population of the PCT, and largely driven by cost. This was particularly true for  an overall quality score (see paragraph 2.16). Cost and small co-operatives and mutual organisations. Providers  quality were then combined by calculating the number of told us that this focus on cost was not allowing them  quality points achieved for each pound spent per head.

to spend money on any spend-to-save measures, such  Finally, PCTs were ranked within their urban/rural class

as experimenting with skill mix, upgrading facilities or  on the basis of their score, with the highest score being innovative integration pilots. PCTs desire to let short-term  the best (i.e. most cost-effective) out-of-hours service in contracts may have also mitigated against investments for  a given class. We then identified the best-performing PCTs the medium term in some instances. in each category. These are shown in Figure 7.

4.18 As Figure 8 shows, the cost of out-of-hours increases

There is scope to reduce costs in  the more rural a PCT is, whilst quality scores remain future by up to £134 million broadly the same. This correlation between rurality

and cost means that a model which works well in a Benchmarking suggests that the cost- significantly urban area may not necessarily be a suitable effectiveness of particular services is  model for a significantly rural area. It is also worth noting

that, while costs and scores have been calculated for influenced by rural classification individual PCTs, both factors can be affected where PCTs

4.16 Evidence from our visits to PCTs and the views of  commission services jointly. our expert panellists shows that there is scope for PCTs to

reduce the costs of out-of-hours in future by: Benchmarking against the best services could n driving value for money from future tendering  generate savings of up to £134 million

processes based on real competition; 4.19 Using the services identified as the best for each

n continuing to test the cost-effective use of other health  PCT classification, we calculated the savings that could be professionals alongside GPs in out-of-hours teams; made if each PCT in that classification provided its service

at the same cost as the best. The potential savings totalled n developing activity and cost data so as to improve  £134 million, and the breakdown of these savings can be

provider performance;

seen in Figure 9.

13  DEFRA Classification of Primary Care Trusts in England, Department for the Environment, Food and Rural Affairs, 2005.

7. Most cost-effective out-of-hours services in England

Classification  PCT  Actual cost  Quality score  Quality points

per head (£)  per £ spent

Major urban  Bexley Care Trust  3.64  15  4.13 Large urban  South Gloucestershire PCT  7.95  20  2.52 Other urban  Milton Keynes PCT  7.79  18  2.31 Significant rural  Bath and North East Somerset PCT  3.23  11  3.40 Rural-50 (i.e. largely rural)  Cotswold and Vale PCT  4.37  10  2.29 Rural-80 (i.e. major rural)  Central Suffolk PCT  8.96  14  1.56

Source: National Audit Office

8. Average cost and quality scores for each rural classification

Classification  Average  Average  Average actual cost  quality  quality points

per head (£)  score  per £ spent

Major urban  7.19  11.16  1.70 Large urban  8.43  10.97  1.37 Other urban  8.61  10.90  1.37 Significant rural  9.85  10.69  1.34 Rural-50  10.76  10.87  1.13 Rural-80  12.49  10.75  0.92

Source: National Audit Office

20. It is unrealistic to expect all PCTs to achieve as good a performance as the best in each category, so we also examined what savings might be made if the most expensive 50 per cent of PCTs could reach the average performance level in each category. After ranking all PCTs within the same category by their combined cost/quality score, the median-ranking PCT was selected. We then calculated the savings that could be made if all PCTs with a cost per head greater than the median were to reduce their costs down to the median cost. The potential savings totalled £53 million (Figure 10 overleaf).
21. Our analysis suggests that it is possible for out-of-hours services to be provided at lower cost without compromising quality. However, of our interviewees

who compared costs before and after the handover,

100 per cent stated that that the costs of providing the service had gone up. The opt-out of many GPs means the supply of GPs is now restricted, increasing their cost.

In addition, costs that were once absorbed, or not accurately identified in relation to out-of-hours, are now more transparent, which also makes the service appear more expensive. Quality monitoring is more rigorous now than it was previously, but this also has an inflationary effect on costs.

10 Savings (to the nearest £m) achievable using the median service in each classification of PCT

Classification  Total actual  Total cost at  Saving cost (£m)  median rate (£m)  (£m)

Major urban  86  83  3 Large urban  51  45  7 Other urban  52  47  5 Significant rural  45  30  15 Rural-50  80  71  9 Rural-80  57  43  14 Total  371  318  53

Source: National Audit Office

22. There is no single model which will work best for

all PCTs. Commissioners are at an experimentation stage at the moment and should continue to experiment with different arrangements to identify a model which works well for them. However, the models identified below should serve as a basic set of cost-effective benchmarks for providers operating in an urban, mixed or rural area:

n Central Suffolk PCT (largely rural PCT): out-of-hours

care is provided by GPs and nurses. Call handlers perform telephone triage and good integration with local minor injury units allows many calls to be diverted away. Patients requiring GP attention can be treated at a base or a home visit is arranged; and

n Bath and North East Somerset PCT (rural-urban

mix): out-of-hours care is provided by a mix of GPs and nurses. Telephone triage is contracted out to a larger, neighbouring PCT, since it is not cost-effective to perform in-house. There are two primary care centres, each located within a hospital and each staffed by one GP and a team of nurses with an additional GP on stand-by. Primary care centre consultations are performed by the GP or nurses

and home visits completed by GPs. If the GP is not due to return from a home visit for some time and patients present at the primary care centre requiring GP attention (following nurse assessment), the stand- by doctor is called.

23. Significantly, all of these providers are integrated with other services such as community nursing, minor injuries units, walk-in centres or Accident and Emergency departments and divert patients as appropriate.

n Bexley Care Trust (largely urban PCT): the out-of-hours

service is provided by GPs with support from nurse

practitioners in call handling and consultations.  GP pay rates remain the key driver There is one primary care centre at the north-west end  of costs

of the borough. It is manned by administrative staff

as the operational base all week, including during  PCT survey shows wide distribution of pay the day. An additional outreach service operates  rates - not solely on the basis of rurality

at weekends in a large GP practice. GPs complete

home visits during the evenings while other GPs  4.24 Our PCT survey shows a wide distribution of pay man the primary care centre; during the night shift  rates for GPs. The distribution of rural, urban and mixed administrative staff man the base, while the GP is  PCTs among the pay rates scale shows that rurality is not out. The service also stations GPs in the Accident and  the only driver behind how expensive the rates are. Emergency departments of two local hospitals during  Figure 11 shows that an average weekday evening rate periods of peak demand. The out-of-hours provider  of £58.36 can be nearly doubled during a bank holiday, has achieved a good level of integration with other  when the average rises to £102.54.

services, such as community nursing, minor injury

unit, Accident and Emergency departments, and

a primary care nursing team at one Accident and

Emergency unit, and diverts patients as appropriate;

11 GP pay rates

GP pay rates (£)  Weekday evening  Weekday overnight  Weekend  Weekend overnight  Public holiday Average rate  58.36  74.30  68.95  79.60  102.54 Maximum  107.00  114.00  141.00  141.00  165.00 Minimum  41.29  19.35  48.68  19.35  48.68

Source: National Audit Office

There is some evidence that good working relationships keep pay down

25. Many PCTs told us that they felt their finances were at the mercy of whatever pay rates GPs demanded. However, we also found evidence of providers and commissioners taking practical steps to help keep GP pay rates down, whilst keeping GPs themselves committed to providing a good service.
26. The Devon Doctors Co-operative is a particularly good example of effective planning and management which allows them to pay their GPs £50 per hour on weekday evenings and £70 per hour overnight and at weekends. This provider is over-subscribed with GPs willing to work out-of-hours shifts, allowing it to keep

overall costs down. The rich supply of local GP principals (who cover 70 per cent of shifts) can be attributed,

in part, to good preparation. Devon Doctors reduced uncertainty by implementing aspects of the new contract such as Saturday morning cover, well in advance of the new contract s introduction. Devon Doctors fills its rotas three months in advance using a preference system to ensure it has sufficient cover. Filling the rota is assisted by a pioneering website where shifts can be booked electronically. The advance planning also allows doctors greater freedom to plan their schedules. An additional incentive for GPs is speed of payment and reduced superannuation administration Devon Doctors pays its GPs on a weekly basis.

PART FIVE

Commissioners are entering into contracts with multiple providers and the market is maturing

There is a new array of providers and commissioners are contracting with increasing numbers of them. Providers are beginning to introduce mixed teams

of doctors, nurses and others into their services

and many different models exist. Mixed teams increase the cost of the service in the short term

and savings are expected to materialise in the longer term. Providers are becoming more responsive to commissioners who have better information and who are starting to make tough decisions to penalise poor providers. England compares well both within the UK and internationally.

The provider market is diversifying and maturing

There is a new array of providers

1. When GPs were responsible for the provision of out-of-hours care for their patients, the vast majority of practices participated in GP co-operatives. Since GPs have been able to opt out of this responsibility, there has been a significant change in the make-up of the provider market. It is now common for commissioners to have contracts with several providers, each providing a different element of the service. Our survey of PCTs found that the provider market is split as Figures 12 and 13 overleaf show.

2. Those providers offered a large number of contracts do not necessarily have a large share of the market.

For example, NHS Direct was awarded 18 per cent of

all out-of-hours contracts, but these only represented four per cent of the market by revenue.

Some providers are diversifying business to drive down cost and sweat assets

3. Anecdotal evidence from our interviews suggests that many providers are providing out-of-hours services either on very tight profit margins or at cost, therefore making no profit at all. In order to decrease costs or increase their margins, these providers are increasingly taking on other contracts. For example, Dorset Emergency Care Services has a contract with the Army, Essex Ambulance Service provides out-of-hours services for two local prisons and Derby Medical Services has a contract with a local police force. More and more providers are now also offering in-hours cover to maximise use of their assets which would otherwise be idle during the day.

Many smaller providers merged to find efficiencies

4. Some providers have realised that they must be larger if they are to benefit from economies of scale and are either expanding outwards into neighbouring regions, or merging with other providers to seek efficiencies in their operations (see Case Examples 5 and 6).

12 Provider market by share of revenue

Provider type

PCT (in-house)

Co-operative

Mutual organisation/CBS

Commercial provider

NHS Direct

Other

0 5 10 15 20 25 30

Percentage share of market revenue

Source: National Audit Office NOTE

1  Mutual organisations (also known as community benefit societies CBS) are large GP co-operatives which remain non-profit, but which are based on community ownership.

13 Provider market by contracts awarded

Provider type

NHS Direct

Co-operative

PCT (in-house)

Ambulance/A&E

Commercial provider

Other

Mutual organisation

Community Benefit Society

0 5 10 15 20

Percentage share of all contracts

Source: National Audit Office

Case studies and visits reveal much good practice

There are examples of the realistic and cost- effective use of skill mix

5. Skill mix, meaning the use of nurses and other health professionals alongside GPs to deliver out-of-hours care, was integral to the model of out-of-hours care set out in the Carson Review. The Department endorsed

this view, particularly by encouraging the employment

of NHS Direct in out-of-hours. Out-of-hours providers are increasingly introducing small numbers of nurses, emergency care practitioners and paramedics into their service for specific tasks, such as call handling, telephone triage or home visits. A small number have attempted much more radical solutions, such as trying to replace large numbers of GPs with other staff. There is no single template for doing this and providers have tested a wide range of operational models.

6. Providers who are experimenting with staffing in this way told us that there are challenges:

n changing skill mix can actually increase the cost of

out-of-hours services in the short term, whilst nurses and emergency care practitioners are being trained. There may also be additional costs to the wider NHS, as new staff are more likely to refer patients to other parts of the NHS. In addition, short-term contracts with limited financial flexibility may not incentivise providers to innovate in this way;

n the blanket introduction of large numbers of new staff

rarely has the desired effect in terms of affordability and productivity. The introduction of skill mix needs to be an iterative process14. However, as their knowledge and experience grows, new staff are able to take on more of the work presently performed by GPs, reducing the number of GP hours needed and, consequently, the cost of the service; and

n no single professional can fully replace a GP,

because of the wide range of competencies and skills that GPs possess. For example, neither a nurse nor an emergency care practitioner can prescribe the same array of drugs, sign a death certificate or complete the same range of diagnostic procedures. However, inter-disciplinary teams can provide a way around this challenge.

7. Most of those experimenting with skill mix are still in the early stages of their schemes. In addition, most are using only very small numbers of new staff. It is therefore difficult to say at this stage that there is conclusive evidence about cost savings. However, as our survey of PCTs found, not all of the best models identified used skill mix, even though many were moving towards doing so.
8. The Department s policy of encouraging skill mix was based mainly on evidence gathered at the time of the out- of-hours review from NHS Direct and those providers who had used nurses previously. There was much operational evidence and also some published papers on the effectiveness of nurses in triage and call assessment roles. There was much less evidence, operational or published,

to support the role of emergency care practitioners due to the newness of the role. However, one recent evaluation suggests that patients eligible to be seen by an emergency care practitioner benefit from fewer investigations and

are less likely to be referred on to other services.15 The Department should consider commissioning further research once schemes involving skill mix have been in place for long enough to have generated meaningful data.

9. There is a need for GPs to remain at the heart of

the service. Quality Requirement 11 states that, where it

is clinically appropriate, patients must be able to have a face-to-face consultation with a GP. Evidence from our visits to providers also showed that in services where there is a GP at the heart of the operation, they can act as a focal point for training and mentoring other staff.

10. A joint review of out-of-hours services in Northumberland16 highlighted another problem regarding the use of GPs. The review concluded that rural areas with community hospitals, such as Northumberland, require doctors with different skills to those in urban areas and that doctors with the necessary skills are not always available. The review recommended that the local organisations concerned should introduce a joint training strategy to develop a team of highly skilled doctors to provide a specialist service between primary care and acute hospitals.

Providers are beginning to reduce overlap with other services

11. There is currently considerable overlap of services for patients requiring out-of-hours care. In many areas of England, a housebound elderly patient requiring care at

14 This also applies to foreign doctors, whether salaried or used on a locum basis. They tend to be cheaper than UK doctors, but take time to adjust to a new

working environment.

15  A National Evaluation of the Clinical and Cost Effectiveness of Emergency Care Practitioners, 2005, Sheffield University Medical Care Research Unit.

16  A Joint Review of the New Out-of-hours Services in Northumberland, 2005, Northumberland Care Trust, Northumbria Healthcare NHS Trust, Northern

Doctors Urgent Care et al.

night could be visited by representatives of a large number of different services, including an out-of-hours doctor, nurse or emergency care practitioner, an intermediate care team, a district nurse or a paramedic team from the local ambulance service. Whilst there may be good reasons for the separate existence of these teams, there is clear scope for more joint working, especially in rural areas where resources for home visits are stretched thinly.

12. Our survey found that a growing minority of providers are already trying to work more closely with other urgent or unscheduled care teams. Although these arrangements are usually relatively informal, they are already providing

a better service for patients and have the potential to save money for the local health economy as a whole.

Some areas have good links with local services such as pharmacy

13. Providers have started to forge links with other local services such as pharmacies. For example, providers

in Doncaster have a well-established system for faxing repeat prescriptions through to a local pharmacy which is open until 10pm. Patients who have a prescription faxed to a pharmacy in this way do not require a face-to-face consultation, avoiding the need to visit the primary care centre or arrange a home visit, but will have received a telephone assessment carried out by a GP.

Commissioners are beginning to make tough decisions

De-commissioning of poor providers

14. The array of new providers has opened up the out-of-hours market to increased competition and commissioners are now able to penalise those providers whose service is less than satisfactory. Although this is not yet true for the whole of England, more and more commissioners are in a better position to take decisions about ongoing contracts for their providers.
15. Several commissioners told us that they

had decommissioned, or were in the process of decommissioning, their providers. NHS Direct was one

of a number of providers which lost contracts. The most common complaints were that NHS Direct was unable to cope with demand and that the service was too expensive. In addition, a number of commissioners stated they

found NHS Direct nurses to be greatly risk averse and, consequently, they referred an excessive proportion of calls

to 999 or on to a GP, rather than close the call with advice to the patient. In their view, this represented poor value for money and increased costs in other parts of the system. Despite these problems, NHS Direct told us that they had also won a number of contracts during the period, thanks to the introduction of more flexible pricing schemes and innovative service models tailored to local need.

16. These views echo the findings of the Joint Northumberland Review17, which found that, NHS Direct call-handling performance was falling below the expected standard. Concerns have also been raised in Scotland in relation to NHS 24, where call-backs rapidly became accepted as an integral part of the day-to-day service which created problems, not least because nurses were called away from live incoming calls to make return calls. The NHS 24 Review recommended a number of significant changes to remove call-backs as the main mechanism of handling demand.

England performs well against UK and international comparisons

England is at the forefront of thinking internationally

17. The closest direct comparisons to the funding and structure of primary care in England are to be found in

the northern European countries, which have a number of developments in common with out-of-hours here. It is rare to find a model of GPs undertaking their own care out-of-hours unsupported anywhere. In all the European systems we looked at, we found that formal standards are being introduced. These vary in style, content and application, but they follow the trend in England. See Appendix 4 for a full analysis of how England compares against other countries.

England compares well on cost and quality against the rest of the UK

18. Scotland, Wales and Northern Ireland have adopted the same basic out-of-hours system as England, with telephone assessment followed by mainly GP care.

A comparison of quality standards shows that all the administrations have broadly adopted the same approach. All have a significant focus on access. The approaches

in the three devolved administrations show significant influence arising from the pre-contract introduction of quality standards in England. Appendix 3 provides further information on the rest of the United Kingdom.

17  A Joint Review of the New Out-of-Hours Services in Northumberland, 2005, Northumberland Care Trust, Northumbria Healthcare NHS Trust, Northern

Doctors Urgent Care et al., p14.

APPENDIX ONE Methodology

Patient and Public Survey

1  We commissioned MORI to conduct a survey of the public to ascertain their awareness and experience of out-of-hours services. They conducted telephone interviews with a representative sample of the English population by age, gender, socio-economic group and region, totalling 3,447 people. Full results of the survey are available on our website (www.nao.org.uk).

PCT Survey

2  We conducted a survey of all Primary Care Trusts

in September and October 2005 in conjunction with the Audit Commission. In the survey we asked questions in

5 areas relating to the provision of out-of-hours services: general information, contracts, staffing, activity and quality. We had a final response rate of 95 per cent. The remaining five per cent did not respond because they

were in the process of merging, their GPs had retained responsibility for out-of-hours services or they failed to do so in time for our analysis.

3  The Audit Commission have used the survey data

to create an out-of-hours benchmarking tool. This gives individual PCTs electronic access to the complete set of data returns, enabling them to benchmark their own costs and performance against similar PCTs. Any PCT wishing to access the tool should contact their local appointed auditor.

Further interviews

5  We conducted a series of further interviews with other parties, including representatives from Adastra, NHS Direct, the Royal College of General Practitioners, the Royal Pharmaceutical Society, the National Association of GP Co-operatives and Bristol and Southampton Universities.

Expert panel

6  We tested our emerging findings with an expert panel of practitioners, academics and officials. Expert panel members included:

n Dr David Carson (The Healthcare Foundation)

n Nicholas Reeves (Department of Health)

n Nicola Bell (Department of Health)

n Edmund Jahn (Harmoni Clinical Process Outsourcing) n Norma Lane (Dorset Emergency Care Services)

n Michelle Preston (Birkenhead and Wallasey Primary

Care Trust)

n Logie Kelman (National Association of GP

Co-operatives)

n Pam Bradbury (NHS Direct)

Case study visits

4  In order to validate our PCT survey and ensure that we understood the operational challenges of delivering out-of-hours services, we conducted a number of visits to providers and commissioners. Choosing a representative selection of geographical areas and service types, we interviewed a total of 25 PCTs (some of whom were commissioners of the service and some of whom were both commissioners and providers) and 14 separate providers. We also interviewed staff from three Strategic Health Authorities.

Department of Health interviews and file review

7  We examined Department of Health papers and held a series of meetings with officials, including Gary Belfield (Head of Primary Care), Dr Dean Johnson (Head of Urgent Care), Sir George Alberti (National Clinical Director for Emergency Access) and David Colin-ThomØ (National Clinical Director for Primary Care).

Benchmarking of English out-of-hours services against national and international comparisons

8  We commissioned Dr David Carson, author of

the 2000 Carson Review, to assist us in benchmarking out-of-hours services in England against similar services in Scotland, Wales and Northern Ireland, as well as

in a selection of countries overseas. The aim of the benchmarking exercise was to assess England s progress against other countries and to determine whether there was any best practice abroad which could be usefully implemented in England. Dr Carson also assisted us in designing and carrying out our programme of visits to providers and commissioners.

APPENDIX TWO Case Examples

CASE EXAMPLE 2

Bassetlaw PCT

Bassetlaw PCT recognised the need to consult with and retain the support of its GPs for the new out-of-hours service and that Saturday mornings would be a significant problem within the new arrangements.

The service development and design process involved the local GPs at each stage. During the design and consultation process the GPs were asked how many would consider offering services on a Saturday morning. Approximately half the practices volunteered. The PCT already had a core of nurses and paramedics, supported by the PCT nursing lead who

were skilled in handling GP work. The PCT then expanded this team as it recognised that it would need to supplement

the GP workforce. This flexibility ensured adequate access to General Practice on Saturday mornings and avoided placing

an unmanageable demand on the call centre.

The engagement and inclusion of local GPs has meant they supported the new service and have continued to work shifts. The GPs recognise the service is of high quality and meets the needs of their patients.

APPENDIX THREE

Out-of-hours arrangements in Scotland, Wales and Northern Ireland

Introduction

All three countries have broadly adopted the same basic system of care as England with a telephone assessment (triage) followed by mainly GP care. The telephone assessment aspect is most heavily biased towards nurses in Scotland where all calls are triaged by nurses employed by NHS24.

The administrations have adopted similar published quality standards. Their approaches have been influenced by the pre-contract quality standards introduced in England, but are less detailed.

Scotland

Prior to the change in GP contracts in April 2004, out-of-hours services in rural Scotland were delivered mainly by individual GPs and practices. In urban areas local GP co-operatives were common, with limited use made of commercial deputising services. Around 3,500 of the 4,000 GPs in Scotland have opted out of providing 24-hour patient care and rural areas have seen the highest levels of opt-out.

A national telephone service (NHS24) covers all NHS Board areas as the public s main point of access to out-of-hours care. Clinical assessment and triage are carried out by nurses who can provide home care advice to the caller or refer them to their local out-of-hours service, A&E department or Scott ish Ambulance Service as appropriate.

Set up and introduced on a phased basis across Scotland between 2002 and 2004, NHS24 existed prior to the requirement for out-of-hours services under the new GMS contract. There was no requirement for competitive tendering for new out-of-hours services and most existing GP co-operatives were absorbed into the local NHS Boards, utilising NHS24 as the main point of access

to local out-of-hours care. In many areas, out-of-hours centres and minor injury and illness units, staffed by

GPs or other doctors, nurses and paramedical staff, have

opened in existing hospitals or community health centres. However, as in England, there is no evidence that the new arrangements have led to a drop in attendance at A&E departments.

As NHS24 was rolled out across Scotland during 2004, peak demand was higher than expected, resulting in

the introduction of a call-back arrangement in around a third of cases. However, by February 2006 this figure had dropped to around 17 per cent. To reduce the percentage of call-backs, NHS24 took a three-pronged approach. This focused on educating the public about appropriate use of the service, improving efficiency via the establishment of five new satellite centres and the recruitment of 100 new nurses and closer working relationships with the rest of NHS Scotland to ensure capacity met demand. NHS24 currently employs 527 nurses, 101 senior nurses and

419 call handlers. Four of the new satellite centres have additional nurses who are employed directly by local NHS Boards.

Formal quality standards cover three main areas: accessibility and availability at first point of contact; safe and effective care; and audit monitoring and reporting. Standards are less detailed than in England with, for example, the definition of accessibility and availability left to the local NHS board.

Wales

Most out-of-hours services in Wales are run internally

as Local Health Board departments, but a number of commercial providers deliver out-of-hours services under contract. In two areas the local NHS acute trust is the service provider a practice not seen elsewhere in the UK. Some areas use nurses to triage some of the calls received. Most are doctor-led with the initial telephone assessment followed by oral advice, reference to A&E or a home visit. Due to the wide variations in system and processes in use in Wales, there is no consensus on the best model or mix of staff.

NHS Direct Wales was launched in June 2000, following the roll-out of the NHS Direct service in England. Operating on the same 0845 46 47 telephone number, callers from landlines within Wales are directed to NHS Direct Wales, whilst callers on mobile phones are asked to select England or Wales before being connected

to the appropriate service. NHS Direct Wales holds location-specific information on health providers in Wales and is able to provide information and advice to callers in either English or Welsh.

As well as providing twenty-four hour access to health advice and information on the 0845 46 47 number, NHS Direct Wales provides a call handling and nurse triage service for out-of-hours providers in three Local Health Board areas. The service employs over 100 nurse advisers, 50 call handlers, 40 health information advisers and over 60 other professionals, handling an annual call volume of approximately 300,000. The service is hosted by Swansea NHS Trust and is based at three sites across Wales.

The headline quality standards for out-of-hours in Wales are similar to those introduced in England, but the breadth and depth of the standards are less. In a 2003 survey for the Commission for Health Improvement, 99 per cent of callers reported that they were satisfied or very satisfied18. However, the Commission felt that many healthcare professionals remained sceptical about the effectiveness of the service.

Northern Ireland

The four Health and Social Services Boards in Northern Ireland have taken slightly different but related approaches to providing the out-of-hours service. All four boards have either transferred existing GP cooperatives to a closely related non-profit organisation or set up an in-house team using salaried doctors. All practices have opted out, but are co-operating with the new arrangements.

There is no equivalent of NHS Direct or NHS 24 and each of the four Board s call handling and triage operations utilises a different mix of doctors, nurses and call handlers. All out-of-hours providers are required to undergo an annual inspection. Service standards similar to those in use in England have been introduced and all providers are now either meeting or working towards those standards.

The Department for Health, Social Security and Public Safety is considering closer links between the GP service with other parts of the out-of-hours network, such as ambulance services and A&E. A province-wide call handling service, along the lines of NHS Direct, is also being considered.

National Cost Comparisons

There are significant differences in the cost of out-of-hours provision per head of population between England and

the devolved administrations (Figure 14). Population densities and rural issues may have influenced the extra costs. Evening and night-time contacts per head of population vary significantly between individual trusts and between countries. Studies prior to the introduction of the new GP contracts suggested that out-of-hours demand

per head of population in Wales were significantly higher than in Scotland, with England lowest. GP costs per hour also vary significantly and may not be under the control

of the local trust. The higher cost of telephone handling

in Scotland may be explained by the higher rate of nurse assessment: less than 10 percent in England and Wales.

14 Comparison of annual costs of UK out-of-hours

provision in 2004-05

Total  Annual cost  Cost per head

£ million  £ per head

England  316  6 Scotland  N/A  N/A Northern Ireland  21  12 Wales  29  10

Source: National Audit Office

18  Commission for Health Improvement, Clinical governance review NHS Direct Wales, 2003.

APPENDIX FOUR

Out-of-hours arrangements in Denmark, Netherlands, USA and Australia

Introduction

All the developed western-style healthcare systems

that we examined have some form of telephone-based out-of-hours medical advice. However, many developed countries do not have a comprehensive general practitioner (family doctor) system. Instead, patients

may self-refer to hospitals or to specialists using either compulsory national insurance or commercial health insurance. Detailed data on out-of-hours provision is not readily available for most countries. We selected four countries where some limited information on the structure and performance of the out-of-hours provision was available through literature review. A comparison of costs was not possible.

Overall, the English system of GP out-of-hours care is as well developed as the European systems. England was one of the first countries to use the reform of out-of-hours care to reduce the anti-social working hours of GPs and improve quality of care. The English system of quality standards is the most structured and monitored of any

we examined. In most countries, integration with other healthcare services operating in the out-of-hours period, such as ambulance services and A&E, is at a similar stage to the UK.

Denmark

Consultation with GPs is free for most Danes, including out-of-hours care. Until 1993, all GPs carried out

their own out-of-hours service, participated in a local co-operative or were covered by a deputising service. In 1993, a national system of telephone assessment by

fully qualified GPs was introduced. Callers receive oral advice, are visited at home by a local GP or are seen at

an out-of-hours clinic. The reforms coincided with an increase in the remuneration of doctors for daytime work. The overall cost of out-of-hours provision remained stable.

Doctors fees were designed to encourage telephone handling rather than home visits. Five years after this reform, the percentage of calls handled by oral advice alone had almost doubled, to 48 per cent, mainly due

to a reduction in home visits now less than one in

five. As a result, the proportion of GPs who worked

10 hours or more per week out-of-hours dropped from about 65 per cent to 10 per cent. Patient satisfaction with the new arrangements initially fell but partly recovered to 72 per cent after 18 months[2]. The main cause of dissatisfaction was that some patients who expected a home visit were given telephone advice only.

The out-of-hours service is funded by the local commissioning organisation, a local state organisation which also funds the other parts of the GP system. There is little or no integration with A&E and ambulance services. There are telephone access standards set by the local commissioning organisation and the other standards are clinical best practice standards which also pertain to

the in-hours service. Demand has been rising since the system was introduced, but has been met by an increase in telephone consultations. On average, GPs should undertake one night duty every 35 nights and one evening duty every 15 evenings. However, younger GPs typically carry out additional out-of-hours duties and around a third of GPs carry out none.

The Netherlands

The Netherlands has a system of general practice funded by compulsory, state-backed insurance organisations and private insurance. Higher charges are paid by patients for house calls than for doctor s appointments or phone consultation fees can be reclaimed from the insurance schemes. The GP contract requires 24-hour cover, although over the last 10 years, 90 per cent of the population have switched to one of the 120 large-scale after-hours co-operatives (45 to 120 GPs). Most large towns now have out-of-hours clinics situated near to,

or co-located with, an A&E Department20. The service also co-operates well with community nursing and other out-of-hours services.

Most of the out-of-hours clinics require patients to call ahead to speak with triage nurses, who assess the urgency of patients problems. A study found that more than half of the patients believe that the reorganisation had improved out-of-hours care.21 Sixty-seven percent of patients who received telephone advice said they were satisfied with the out-of-hours care provided, compared to 80 per cent per who attended the clinic or received a home visit.

The patient s opinion on the doctor s assistant s attitude was the strongest predictor of overall satisfaction with

the new service. There are some standards around access and telephony; however standards are achieved mainly through professionally led self-regulation.

The USA

Most individuals and their families in the USA are covered by employers health care schemes, private medical insurance or government schemes (Medicare/ Medicaid) - although some 40 million people have no cover. There are family physicians but patients may seek specialist care directly.

There are large variations in the types of after-hours service available and by whom it is provided. Family practices generally provide out-of-hours cover using either a call handling service or combined with triage by nurses.

90 per cent of calls are forwarded to doctors,22 but house calls are rare. Many of the Health Plans and Managed

Care Organisations have introduced telephone advice services. Over 100 million people are estimated to have access to telephone triage. The main function of these schemes is to ensure patients access the most appropriate level and type of care, including self-care. There are also urgent treatment centres to supplement A&E services: insurers may refuse to cover the cost of self-referral to A&E typically over $200.

Australia

Each consultation with a GP attracts a fee, up to

85 per cent of which is refunded by the national Medicare system. Although patients may seek care from any GP, in practice most patients prefer a local doctor or practice. Out-of-hours services are very variable in style and content due to geographical differences. In order to maintain registration with the Royal Australian College of General Practitioners, doctors must ensure patients have access

to medical advice at all times. There are 15 companies specialising in out-of-hours and locum services, mainly

in urban areas. All home visits are paid for at the time of consultation, although a portion of the fee is rebated by Medicare. By contrast, hospital visits are free.

The national government has supported a number of pilot schemes to identify best practice in delivering out-of-hours care. As in most of the other developed healthcare systems, there is increasing use of nurse call centres and GP telephone triage. There is little, if any, integration between the GP out-of-hours service and other services working in the out-of-hours period. Some states are similar to the UK. For example, the Department of Health in Western Australia launched HealthDirect in 1999. This triage and health information service is provided by a private company operating from a dedicated call centre with 33 full-time-equivalent nurses. It is available free to the State s 1.7 million inhabitants on a 24-hour basis. The call centre costs £1.9 million a year, i.e. £1.07 per head or £8.53 per call.23 The service is managed through a wide range of performance indicators.

20  Taking the Pulse of Health Care Systems: Experiences of Patients with Health Problems in Six Countries, Cathy Schoen et al, Health Affairs Web Exclusive,

November 3, 2005.

21  Patient satisfaction with out-of-hours primary care in the Netherlands, CJT van Uden et al, BMC Health Services Research 2005.

22  After-hours telephone triage affects patient safety - Original Research, D E Hildebrandt et al, Journal of Family Practice, March, 2003.

23 Telephone triage in Western Australia, V F Turner et al, Medical Journal of Australia, 2002.

APPENDIX FIVE Quality Requirements

1  Providers24 must report regularly to PCTs on their compliance with the Quality Requirements.

2  Providers must send details of all out-of-hours consultations (including appropriate clinical information) to the practice where the patient is registered by 8.00 am the next working day. Where more than one organisation is involved in the provision of OOH services, there

must be clearly agreed responsibilities in respect of the transmission of patient data.

3  Providers must have systems in place to support and encourage the regular exchange of up-to-date and comprehensive information (including, where appropriate, an anticipatory care plan) between all those who may

be providing care to patients with predefined needs (including, for example, patients with terminal illness).

4  Providers must regularly audit a random sample

of patient contacts and appropriate action will be taken on the results of those audits. Regular reports of these audits will be made available to the contracting PCT. The sample must be defined in such a way that it will provide sufficient data to review the clinical performance of each individual working within the service. This audit must be led by a clinician with suitable experience in providing OOH care and, where appropriate, results will be shared with the multi-disciplinary team that delivers the service. Providers must cooperate fully with PCTs in ensuring that these audits include clinical consultations for those patients whose episode of care involved more than one provider organisation.

5  Providers must regularly audit a random sample

of patients experiences of the service (for example 1% per quarter) and appropriate action must be taken on the results of those audits. Regular reports of these audits must be made available to the contracting PCT. Providers must cooperate fully with PCTs in ensuring that these audits include the experiences of patients whose episode of care involved more than one provider organisation.

6  Providers must operate a complaints procedure that is consistent with the principles of the NHS complaints procedure. They will report anonymised details of each complaint, and the manner in which it has been dealt with, to the contracting PCT. All complaints must be audited in relation to individual staff so that, where necessary, appropriate action can be taken.

7  Providers must demonstrate their ability to match their capacity to meet predictable fluctuations in demand for their contracted service, especially at periods of peak demand, such as Saturday and Sunday mornings, and the third day of a Bank Holiday weekend. They must also have robust contingency policies for those circumstances in which they may be unable to meet unexpected demand.

8  Initial Telephone Call:

Engaged and abandoned calls:

n No more than 0.1% of calls engaged n No more than 5% calls abandoned.

Time taken for the call to be answered by a person:

n All calls must be answered within 60 seconds of

the end of the introductory message, which should normally be no more than 30 seconds long.

n Where there is no introductory message, all calls

must be answered within 30 seconds.

24 A provider is any organisation providing OOH services under any of the four primary care arrangements (General Medical Services, Personal Medical

Services, Alternative Provider Medical Services or PCT Medical Services).

9  Telephone Clinical Assessment

Identification of immediate life threatening conditions

Providers must have a robust system for identifying all immediate life threatening conditions and, once identified, those calls must be passed to the ambulance service within 3 minutes.

Definitive Clinical Assessment

Providers that can demonstrate that they have a clinically safe and effective system for prioritising calls, must meet the following standards:

n Start definitive clinical assessment for urgent calls

within 20 minutes of the call being answered by

a person

n Start definitive clinical assessment for all other

calls within 60 minutes of the call being answered by a person

Providers that do not have such a system, must start definitive clinical assessment for all calls within

20 minutes of the call being answered by a person.

Outcome

At the end of the assessment, the patient must be clear of the outcome, including (where appropriate) the timescale within which further action will be taken and the location of any face-to-face consultation.

10  Face to Face Clinical Assessment

Identification of immediate life threatening conditions

Providers must have a robust system for identifying all immediate life threatening conditions and, once identified, those patients must be passed to the most appropriate acute response (including the ambulance service) within three minutes.

Definitive Clinical Assessment

Providers that can demonstrate that they have a clinically safe and effective system for prioritising patients, must meet the following standards:

n Start definitive clinical assessment for patients

with urgent needs within 20 minutes of the patient arriving in the centre

n Start definitive clinical assessment for all other

patients within 60 minutes of the patient arriving in the centre

Providers that do not have such a system, must start definitive clinical assessment for all patients within 20 minutes of the patients arriving in the centre.

Outcome

At the end of the assessment, the patient must be clear about the outcome, including (where appropriate) the timescale within which further action will be taken and the location of any face-to-face consultation.

11  Providers must ensure that patients are treated by the clinician best equipped to meet their needs, (especially

at periods of peak demand such as Saturday mornings),

in the most appropriate location. Where it is clinically appropriate, patients must be able to have a face-to-face consultation with a GP, including where necessary, at the patient s place of residence

12  Face-to-face consultations (whether in a centre or in the patient s place of residence) must be started within the following timescales, after the definitive clinical assessment has been completed:

n Emergency: Within 1 hour n Urgent: Within 2 hours

n Less urgent: Within 6 hours

13  Patients unable to communicate effectively in English will be provided with an interpretation service within

15 minutes of initial contact. Providers must also make appropriate provision for patients with impaired hearing or impaired sight.

REPORTS BY THE COMPTROLLER AND AUDITOR GENERAL, SESSION 2005-2006

The Comptroller and Auditor General has to date, in Session 2005-2006, presented to the House of Commons the following reports under Section 9 of the National Audit Act, 1983. The reports are listed by subject category.

Publication date

Cross-Government

Home Office: Working with the Third Sector  HC 75  29 June 2005 Joint Targets  HC 453  14 October 2005 Progress in improving government efficiency  HC 802-I/II  17 February 2006 Second Validation Compendium Report: 2003-06 PSA data systems  HC 985  23 March 2006 Improving the efficiency of postal services procurement in the public sector  HC 946-I/II/III  24 March 2006 Smarter food procurement in the public sector  HC 963-I/II/III  30 March 2006 Update on PFI debt refinancing and the PFI equity market  HC 1040  21 April 2006

Culture Media and Sport

Procurement in the Culture, Media and Sport sector  HC 596  30 November 2005 The office accommodation of the Department for Culture,  HC 942  16 March 2006

Media and Sport and its sponsored bodies

Defence

Driving the Successful Delivery of Major Defence Projects:  HC 30  20 May 2005

Effective Project Control is a Key Factor in Successful Projects

Managing the Defence Estate  HC 25  25 May 2005 Assessing and Reporting Military Readiness  HC 72  15 June 2005 Major Projects Report 2005  HC 595  25 November 2005 Progress in Combat Identification  HC 936  3 March 2006 Reserve Forces  HC 964  31 March 2006

Education

Securing strategic leadership for the learning and skills sector in England  HC 29  18 May 2005 Extending access to learning through technology:  HC 460  4 November 2005

Ufi and the learndirect service

Employers perspectives on improving skills for employment  HC 461  14 December 2005 Improving poorly performing schools in England  HC 679  11 January 2006

Environment, Food and Rural Affairs

Lost in Translation? Responding to the challenges of European law  HC 26  26 May 2005 Environment Agency: Efficiency in water resource management  HC 73  17 June 2005

Europe

Financial management in the European Union  HC 999  29 March 2006

reports listing

Publication date Law, Order and Central

Public Guardianship Office:  HC 27  8 June 2005

Protecting and promoting the financial affairs of people who lose

mental capacity

Home Office: National Asylum Support Service: The provision of  HC 130  7 July 2005

accommodation for asylum seekers

Returning failed asylum applicants  HC 76  14 July 2005 National Offender Management Service:  HC 458  27 October 2005

Dealing with increased numbers in custody

The Electronic Monitoring of Adult Offenders  HC 800  1 February 2006 Crown Prosecution Service:  HC 798  15 February 2006

Effective use of magistrates courts hearings

Serving Time: Prisoner Diet and Exercise  HC 939  9 March 2006 The Management of Staff Sickness Absence in the National Probation Service HC 1042  26 April 2006

National Health Service

Innovation in the NHS: Local Improvement Finance Trusts  HC 28  19 May 2005 The Refinancing of the Norfolk and Norwich PFI Hospital:  HC 78  10 June 2005

how the deal can be viewed in the light of the refinancing

A Safer Place for Patients: Learning to improve patient safety  HC 456  3 November 2005 Reducing Brain Damage: Faster access to better stroke care  HC 452  16 November 2005 The Provision of Out-of-Hours Care in England  HC 1041  5 May 2006

Overseas Affairs

The Foreign and Commonwealth Office:  HC 594

Consular Services to British Nationals

Department for International Development:  HC 803

Tsunami: Provision of Financial Support for Humanitarian Assistance

Public Private Partnership

Progress on the Channel Tunnel Rail Link  HC 77 The Wider Markets Initiative  HC 799

24 November 2005

1 March 2006

21 July 2005 27 January 2006

Regions and Regeneration

Office of the Deputy Prime Minister: Enhancing Urban Green Space

Regulation

The Office of Fair Trading: Enforcing competition in markets

The Office of Gas and Electricity Markets: Sale of gas networks by National Grid Re-opening the post: Postcomm and the quality of mail services

Revenue departments

Filing of Income Tax Self Assessment Returns

Corporation Tax: companies managed by HM Revenue and Customs Area offices

HC 935  2 March 2006

HC 593  17 November 2005 HC 804  10 February 2006 HC 944  22 March 2006

HC 74  22 June 2005 HC 678  13 January 2006

Publication date

Trade and Industry

The closure of MG Rover  HC 961  10 March 2006 The restructuring of British Energy  HC 943  17 March 2006

Transport

Maintaining and improving Britain s railway stations  HC 132  20 July 2005 The South Eastern Passenger Rail Franchise  HC 457  2 December 2005 A5 Queue Relocation in Dunstable Wider Lessons  HC 1043  28 April 2006

Work and Pensions

Gaining and retaining a job: the Department for Work and Pensions'

support for disabled people

Department for Work and Pensions:

Dealing with the complexity of the benefits system

Department for Work and Pensions:

Using leaflets to communicate with the public about services and entitlements

Department for Work and Pensions:

Delivering effective services through contact centres

HC 455  13 October 2005 HC 592  18 November 2005 HC 797  25 January 2006 HC 941  15 March 2006

Printed in the UK for the Stationery Office Limited on behalf of the Controller of Her Majesty s Stationery Office 186892 05/06 7333