Use of Management Information in the Health and Social Services Department – Operating Theatres.

STATES OF JERSEY

USE OF MANAGEMENT INFORMATION IN THE HEALTH AND SOCIAL SERVICES DEPARTMENT – OPERATING THEATRES

Presented to the States on 11th July 2014 by the Comptroller and Auditor General

STATES GREFFE

2014   Price code: C  R.94

Comptroller & Auditor General

Use of Management Information in the Health and Social Services Department - Operating Theatres

10 July 2014

Use of Management Information in the Health and Social Services Department - Operating Theatres

Introduction

1. Access to relevant and high quality management information allows organisations to make strategic and operational decisions efficiently and effectively.
2. The Health and Social Services Department (HSSD) is planning significant changes in the way it provides services. The White Paper Caring for each other, caring for ourselves' sets out an ambitious programme of change, designed to address current and future challenges. It can be found on the States of Jersey Website[1]. HSSDs aim is to ensure that future services are not only safe, sustainable and affordable but that they meet the needs of Islanders, supporting their health and wellbeing. As part of this, changes to, and significant investment in, acute hospital facilities are planned.
3. HSSD has recognised the need for management information to underpin health reforms. The HSSD Informatics Strategy 2013-2018 aims to ensure that health informatics (management information) is an enabler of wider change, underpinning service reforms contained in the White Paper'. The Strategy includes a commitment to develop integrated reporting of finance, human resources and quality information. Delivery of the strategy has been delayed due to difficulties recruiting key staff.
4. This review focuses on the use of management information in one area of acute hospital performance, the use of operating theatres. Operating theatres are relatively expensive to build, equip, and run: in 2014 the revenue budget for the Theatres and Anaesthesia Division is nearly £12m, 10 percent of the General Hospital's budget. The Division recognises that, to plan and to manage these resources effectively and efficiently, it needs high quality information, including on how its operating theatres are utilised.

Objectives and scope of this review

1. The report covers the extent to which HSSD:

• Identifies relevant information needed to manage operating theatre utilisation;
• Ensures the quality of management information for operating theatre utilisation; and
• Uses management information about operating theatre utilisation effectively.

2. The review considers routine operating sessions, covering both private and States of Jersey funded patients in the four main theatres, two day-case theatres and the endoscopy suite.
3. The review does not extend to the theatre dedicated to maternity services. Nor does it assess actual performance of operating theatres and whether that compares well with good practice.

Identifying relevant management information needs

1. Useful information needs to relate to the objectives of the organisation. Failure to collect information that relates to organisational objectives impedes the ability to determine whether those objectives are being achieved and increases the risk of poor value for money.
2. The strategic objectives for HSSD as a whole are set out in its Business Plan. The improvement of health and social well-being of the population is to be secured by:

1. Redesign of the health and social care system to deliver safe, sustainable and affordable services;
2. Improved health outcomes by reducing the incidence of mortality, disease and injury;
3. Improved customer experience;
4. Promotion of an open culture based on good clinical and corporate governance and an emphasis on safety; and
5. Effective management of HSSD budget in accordance with the Medium Term Financial Plan.

3. The objectives for the Theatres and Anesthesia Division, set out in its divisional Business Plan, do not link directly to HSSD objectives: the language and the headings used are different. This creates a somewhat confused starting point for identifying the management information that the Division needs in order to demonstrate its contribution to HSSD's overall objectives.
4. Furthermore, the Theatres and Anesthesia Division objectives are inadequately outcome-focused and there is some confusion between tasks, targets, deliverables and success measures. As a result there is a risk that management information does not adequately support the delivery of desired outcomes.
5. The Theatres and Anesthesia Division's Business Plan includes some Key Performance Indicators (KPIs) which, when appropriate objectives are agreed, can be built upon. However, the range of KPIs is currently limited.
6. KPIs should encompass:
   1. What the Division is seeking to achieve (its objectives); and
   2. The things it wants to avoid happening (driven by risk logs).

8. KPIs can cover:
   1. Economy e.g. elements of cost of theatre provision;
   2. Efficiency e.g. cost per case, staff productivity; and
   3. Effectiveness e.g. clinical outcomes, patient experience.

3.8  KPIs work best when targets and tolerances are set providing a focus for performance to support agreed objectives.

Recommendations

R1 Update the Theatres and Anesthesia Division Business Plan to:

• Align objectives clearly to HSSD Business Plan objectives; and
• Include appropriate outcome-focused objectives.

R2 Identify appropriate KPIs measuring economy, efficiency and

effectiveness.

R3 For each KPI set appropriate targets and tolerances to support delivery

of objectives.

Ensuring data quality

1. Information for decision-making is most useful when derived from high quality data. Where data is of a low quality there is an increased risk that decisions are made which do not promote organisational objectives. There is also a risk that information derived from the data is ignored in decision-making.
2. Data quality has a number of attributes (see Exhibit 1).Exhibit 1: Attributes of data quality

Source: Developed from Improving information to support decision making: standards for better data quality Audit Commission (2007).

3. Data on the utilisation of operating theatres is drawn from key operational computer systems:

• In June 2012 a new Patient Administration System (PAS), TRAKcare, was implemented across the hospital. This did not contain a theatre utilisation module but HSSD's Clinical Lead for Informatics, who is an anesthetist, worked with theatre nurses to develop one that was implemented in October 2012. This module is used for the main operating theatres, day surgery unit, minor operations room and maternity theatre; and

• In early 2014, management of the endoscopy suite was switched from TRAKcare to the Endobase Olympus system, securing alignment with the system for equipment tracking and sterilisation. This system has the capability to provide endoscopy suite utilisation reports but this function is not yet fully implemented to meet Divisional information needs.

4. There are significant weaknesses in the arrangements for securing data quality for theatre utilisation in TRAKcare (See Exhibit 2).

Exhibit 2: Weaknesses in arrangements for securing data quality for theatre utilisation

5. As a result TRAKcare is providing inadequate management information about operating theatre utilisation to support decision-making.

6. HSSD uses the Datix' system to capture information about, and to manage, patient safety risks. The Division uses this to report on, for example, cancelled operations, returns to the theatre and missing equipment. All staff have access to the Datix system. However, there are no arrangements for routinely collecting other softer' information, for example from every-day observations of how operating theatres are used, from all staff groups. As a result management information might be incomplete.
7. I understand that the hospital is in the process of appointing a data lead who will be responsible for improving data quality, including within the Theatres and Anaesthesia Division.

Recommendations

R4 Develop and utilise management information reporting capability for the

Endobase Olympus system used in the endoscopy suite.

R5  Establish and reinforce clear accountabilities for the completeness and

accuracy of data entry.

R6  Further develop and utilise TRAKcare's management information

reporting by:

• Establishing and implementing a minimum data set';
• Adopting clear data definitions in all areas; and
• Establishing mandatory data and information fields.

R7 Implement arrangements for promoting and testing the quality of data. R8 Establish and implement arrangements to actively collect softer

information from all relevant staff groups about operating theatre utilisation in order to feed management information.

Using management information about operating theatre utilisation effectively

1. Good quality data is most valuable when it is compiled, reported and used appropriately to provide management information to support evidence-based decision-making.

Compiling management information

2. The hospital's Information Systems Manager, in response to requests from the Division, calculates operating theatre utilisation rates:

Utilised time in normal working hours

In hours' utilisation rate  =  ____________________________________________

Available time in normal working hours

3. There are weaknesses in the current derivation of both components of this indicator which reduce the value of the indicator to decision makers (see Exhibit 3).

Exhibit 3: Weaknesses in derivation of In hours' utilisation rate

4. The equivalent indicator for Theatre 4 (the emergency theatre), and for other emergency activity undertaken, is not routinely calculated, meaning that such activity is not routinely monitored or managed.

Recommendations

R9 Collect information on gaps between patients during lists and amend

the calculation of the utilisation rate performance indicator to account for this lost' time.

R10 Amend the calculation of the utilisation rate performance indicator to reflect actual available time in each theatre for each calendar month.

R11 Agree performance information required for Theatre 4 and other

emergency activity, implement systems to collect it and routinely prepare an appropriate performance indicator.

Reporting management information

5. The Theatres Management Group (TMG), that comprises senior clinicians, divisional managers and information managers, considers utilisation information quarterly. There are no mechanisms for including all theatre users in discussions about utilisation rates: as a result the potential use of the information to drive change in practices and improve utilisation is reduced.
6. Information is currently presented graphically, tracking utilised time and available time for each calendar month on a rolling 12-month basis, disaggregated to individual theatres and grouped for the main theatres and the day surgery theatres. The graphs are supported by spreadsheets that, for the main operating theatres and day surgery, provide a summary of under- and over-running sessions with reasons (where available).
7. Performance against the Division's Business Plan KPIs is presented in a dashboard'. This includes recording utilisation performance against KPIs such as unused lists, late starts and late finishes, with targets and tolerances set for each indicator. However:

• there is no KPI for early finishes or short lists, although these are an indication of potential under-utilisation of resources and are frequently discussed at TMG meetings;
• some of the KPIs are not adequately linked to underlying drivers of performance. For example, Did Not Attends' (where the patient does not arrive on the day) and cancelled patients' (including where the clinician cancels the patient from the list, for clinical or other reasons) are reported as one figure, although the drivers of Did Not Attends' and cancellations may be very different;
• KPIs are not measured at specialty level e.g. orthopaedics which reduces the value of the information to drive change.

8. The Hospital's Operational Management Group is currently working to improve the usefulness of all its Divisional dashboards.

Recommendations

R12 Disseminate relevant management information more widely and

provide training on its use.

R13 Review KPIs to align with business needs, including considering:

• Introducing a KPI for early finishes and short lists;
• Amending KPIs to help monitor and manage specific issues e.g. by reporting Did Not Attends' and cancelled patients' separately; and
• Disaggregating some KPIs to specialty level.

Using management information on a day to day basis

9. There are some specific uses of data and management information from the operating theatre utilisation module of TRAKcare that are working well. For example, data is used by individual clinicians to demonstrate their experience of certain procedures to support their appraisal and professional development.
10. However, the potential of TRAKcare data and management information are not consistently exploited to best effect. Understanding how long a patient procedure is likely to take is important in planning and scheduling theatre time. If data is properly completed, TRAKcare can show the typical time each consultant surgeon takes for a procedure. Where this is possible, profiling' of individual consultants' use of time enables a better fit' of patients on lists and reduces under- or over- booking. However, even where the necessary information is available, this is not yet routinely used.

Recommendations

R14 Use the full potential of data from TRAKcare, including to calculate

consultants' typical procedure times, to inform scheduling of patients for theatre.

Using management information to inform longer-term planning

11. Increases to operating theatre capacity are planned: two semi- permanent operating theatres are to be built, enabling one of the existing theatres to be dedicated to maternity; and the draft Medium- Term Financial Plan for 2016-19 includes consideration of a move towards a 12 hour surgery day. A six-day working week for operating theatres is also under discussion. But making informed decisions on the requirements for and use of operating theatre capacity requires good quality information on operating theatre utilisation that is not currently available.

12. Deciding on whether clinical procedures are undertaken on or off the Island is predominantly driven by clinical safety and then by affordability. Information about the cost of individual patient procedures is important in supporting decisions on affordability. Cost information is not routinely captured or matched to activity information to inform decisions. Work to model the impact of different interventions and changes in service delivery on the costs of patient pathways is being piloted in cardiology. The current modeling is relatively crude, relying as it does on poor quality data on theatre utilisation. HSSD is considering a number of opportunities to improve costing information and has recently appointed a full-time accountant to focus on the cost of individual patient procedures.

Recommendations

R15 As data quality improves, expand the use of data and management

information to inform longer-term decision-making.

R16 Prioritise development of the capacity to use cost information alongside

activity information to inform decision-making.

R17 Ensure decisions on the future requirements for, and use of, operating

theatre capacity are made - and where necessary revisited - using high quality theatre utilisation information.

Conclusion

1. The Theatres and Anesthesia Division recognises that there are weaknesses in its management information, including in respect of theatre utilisation. The HSSD Informatics Strategy sets out the Department's ambition for high quality management information and the Action Plan seeks to address known weaknesses.
2. Improvements to management information should be seen as a priority. Operating theatres are an expensive resource and evidence-based decision-making requires identification of relevant information needs, as well as improvements in the arrangements for securing data quality and effective use of the resulting management information.

Appendix 1 Summary of Recommendations

R1 Update the Theatres and Anesthesia Division Business Plan to:

• Align objectives clearly to HSSD Business Plan objectives; and
• Include appropriate outcome-focused objectives.

R2 Identify appropriate KPIs measuring economy, efficiency and

effectiveness.

R3 For each KPI set appropriate targets and tolerances to support delivery

of objectives.

R4 Develop and utilise management information reporting capability for the

Endobase Olympus system used in the endoscopy suite.

R5  Establish and reinforce clear accountabilities for the completeness and

accuracy of data entry.

R6  Further develop and utilise TRAKcare's management information

reporting by:

• Establishing and implementing a minimum data set';
• Adopting clear data definitions in all areas; and
• Establishing mandatory data and information fields.

R7 Implement arrangements for promoting and testing the quality of data. R8 Establish and implement arrangements to actively collect softer

information from all relevant staff groups about operating theatre utilisation in order to feed management information.

R9 Collect information on gaps between patients during lists and amend

the calculation of the utilisation rate performance indicator to account for this lost' time.

R10 Amend the calculation of the utilisation rate performance indicator to

reflect actual available time in each theatre for each calendar month.

R11 Agree performance information required for Theatre 4 and other

emergency activity, implement systems to collect it and routinely prepare an appropriate performance indicator.

R12 Disseminate relevant management information more widely and

provide training on its use.

R13 Review KPIs to align with business needs, including considering:

• Introducing a KPI for early finishes and short lists;
• Amending KPIs to help monitor and manage specific issues e.g. by reporting Did Not Attends' and cancelled patients' separately; and
• Disaggregating some KPIs to specialty level.

R14 Use the full potential of data from TRAKcare, including to calculate

consultants' typical procedure times, to inform scheduling of patients for theatre.

R15 As data quality improves, expand the use of data and management

information to inform longer-term decision-making.

R16 Prioritise development of the capacity to use cost information alongside

activity information to inform decision-making.

R17 Ensure decisions on the future requirements for, and use of, operating

theatre capacity are made - and where necessary revisited - using high quality theatre utilisation information.

KAREN McConnell COMPTROLLER & AUDITOR GENERAL

LINCOLN CHAMBERS (1ST FLOOR), 31 BROAD STREET, ST HELIER, JE2 3RR

T: + 44 1534 716800 E: enquiries@jerseyauditoffice.je W: www.jerseyauditoffice.je