States Assembly | R.21/2024

This content has been automatically generated from the original PDF and some formatting may have been lost. Let us know if you find any major problems.

Text in this format is not official and should not be relied upon to extract citations or propose amendments. Please see the PDF for the official version of the document.

PDF version

Electronic Patient Record

8 February 2024

jerseyauditoffice.je R.21/2024

The purpose of the Comptroller and Auditor General (C&AG), fulfilled through the Jersey Audit Office (JAO), is to provide independent assurance to the people of Jersey on the extent to which public money is spent economically, efficiently and effectively and on whether the controls and governance arrangements in place within public bodies demonstrate value for money. The C&AG's remit includes the audit of financial statements and wider consideration of public funds, including internal financial control, value for money and corporate governance.

This report can be found on the Jersey Audit Office website at https://www.jerseyauditoffice.je/

If you need a version of this report in an alternative format for accessibility reasons, or any of the figures in a different format, please contact enquiries@jerseyauditoffice.je with details of your request.

All information contained in this report is current at the date of publication.

The Comptroller and Auditor General and Jersey Audit Office are not responsible for the future validity of external links contained within the report.

All information contained in this report is © Copyright Office of the Comptroller and Auditor General and the Jersey Audit Office, with the exception or extracts included from external sources, which are © Copyright to those external sources.

The information contained in this report is for non-commercial purposes only and may not be copied, reproduced, or published without proper reference to its source. If you require the material contained in the report for any other purpose, you are required to contact enquiries@jerseyauditoffice.je with full details of your request.

Report by the Comptroller and Auditor General: 8 February 2024

This report has been prepared in accordance with Article 20 of the Comptroller and Auditor General (Jersey) Law 2014

Contents

Summary 4 Objectives and scope of the review 9 Detailed findings 11

Purpose 11 Value 14 Programme set up 20

Delivery variation and management 31 Appendix One – Audit Approach 35

Appendix Two – Summary of Recommendations, Work planned that

should be prioritised and Areas for consideration 39

Summary

Introduction

The Electronic Patient Record (EPR) system is a business process-based software solution that brings together key clinical and administrative information involved in the care and management of patients.
The Government of Jersey's Digital Care Strategy includes a four-year programme to implement a new EPR system. The stated aim of the EPR programme is to provide a single source of patient information available at the time and place where care is being delivered.
The November 2020 Business Case states that the outcomes to be delivered by the EPR programme include:

• a sustainable and continued improvement in the quality and safety of acute care within Jersey

• improvements in acute care patient outcomes

• reduction in unwarranted variation away from clinical pathways and protocols, with associated cost savings and improvements in care; and

• to continue to build and contribute to a Jersey Care Record, a unified digital care record for Jersey patients across all aspects of care.

The EPR programme has been designated as a Major Project under the Public Finances Manual. A total of £16.2 million has been allocated to the Digital Care Strategy in various Government Plans including the Government Plan 2023-2026.
The total estimated programme costs in the EPR November 2020 Business Case were £29.3 million between 2021 and 2031 including both capital and revenue expenditure.
The EPR implementation was planned to take place in a series of releases as shown in the EPR Roadmap in Exhibit 1.

Exhibit 1: EPR Roadmap

Source: Government of Jersey EPR Roadmap

At the time of my review Releases 1 and 2 had gone live, with the remaining three phases planned to be implemented with an estimated completion date of July of 2024, or possibly the autumn of 2024.

Key Findings

As part of the Digital Care Strategy, a Strategic Outline Case for a new EPR was documented in 2017. Historically, the Health and Community Services (HCS) Department had worked with a hybrid system of both paper and an electronic health record. This resulted in data fragmentation leading to difficulties in accessing a comprehensive view of a patient's medical history. The hybrid system did not provide a complete data analysis platform for HCS.
The development of the Business Case for the EPR commenced in February 2020. The Business Case was finalised in November 2020 and clearly outlines the reasons for the replacement of the previous patient record system.
Over the last three years there have been several significant IT implementations that have run in parallel and have required resource from the Government Modernisation and Digital (M&D) team. These implementations include the EPR programme as well as the Integrated Technology Solution programme (ITS). The parallel running of major IT implementation programmes has placed particular pressure on the M&D team and there has been a need to bring in additional external resources to support the ongoing implementations.

The HCS Digital Team transferred to M&D in March 2021. This transfer effectively meant that HCS subsequently lost the opportunity to have a dedicated Chief Information Officer (CIO) to work alongside the Chief Clinical Information Officer (CCIO) and project team on the EPR implementation and to scope and lead on major health and community services related IT projects.
There is evidence that the level of engagement with key stakeholders has varied over the lifetime of the EPR programme from January 2020 to May 2023. Since November 2020 there have been two changes in CCIO as well as changes in the leadership team of the HCS. To have this level of senior change in a complex and complicated long running programme, particularly one impacted by the COVID - 19 pandemic, may well have had a negative impact on the wider engagement with key stakeholders, particularly clinical stakeholders. There is evidence that despite significant attempts made by the EPR programme team, levels of engagement from senior clinicians and managers fell below what was expected and hoped for.
The Procurement Strategy for the EPR implementation did not provide a full examination of how the procurement could be packaged to minimise risk to the Government, meaning that the options for how risk could best be transferred to suppliers were not articulated. In practice, the procurement approach adopted differed to the Procurement Strategy in respect of: EPR implementation; and transformation support and data migration.
A new CCIO took up post in July 2021 and expressed concerns as to how the project had progressed to that point. The new CCIO provided a catalyst for a reset of the project at that time. This reset included the decisions to halt two of the initial procurements and appoint suppliers directly under procurement exemptions so that the project could proceed towards a more effective implementation. If there had been a better original understanding of the market for these services during the initial procurement earlier in 2021, this reset and the associated procurement exemptions and breaches would not have been required.
Prior to the new CCIO taking up post, the negotiations on contractual terms had been led by an external consultancy firm. The new CCIO identified significant risks to the Government with the contracts that had been proposed. The decision was made to seek to reduce the contractual risks to Government through further contractual negotiations that took place between July and October 2021. The final contract that was signed did not reflect the terms and conditions notified to potential bidders at the procurement stage. While the final terms and conditions were more favourable for the Government, it is not best practice to allow post - award contract negotiations to take place.
The way in which financial information was included in the November 2020 Business Case does not enable a transparent audit trail to the funding for the

programme included in the Government Plan 2021-2024. In addition, no Full Business Case (FBC) was produced at the end of the procurement stage. The financial information in the November 2020 Business Case did not reflect the value of the agreed contract entered into after the procurement stage.

The EPR programme has a long implementation period from October 2021 to the third quarter of 2024 (three years). During that time the EPR capital funding is not visible within the EPR programme budget because the EPR capital funding is drawn from a wider HCS digital transformation programme. One of the consequences of this is that it is hard to track whether the EPR programme is over or under spending.
The November 2020 Business Case had a clear and realistic view on what good looks like' and how benefits would be measured. In my view, the financial benefits outlined are not overly ambitious. The realisation of them does however rely on HCS adopting standardised clinical and business processes, which the new EPR would facilitate. There is a benefits tracker in existence that has been derived from the November 2020 Business Case. This benefits tracker was not however being actively used at the time of my audit as the emphasis of the programme was still on the short term operational delivery of Releases 1 and 2. While I note that a role of Change and Benefits Manager has been introduced there is a risk that between now and the final system Release 5 in the third quarter of 2024 the programme will be stuck in short term operational delivery mode and that benefits realisation may be pushed back to the fourth quarter of 2024, or even forgotten.
There is a clear Governance Framework that supports a Digital Health and Care Implementation Plan. This includes a Digital Health Portfolio Board, chaired by a Sponsoring Senior Responsible Officer with representatives including a Senior User, Project Management Officer and Supplying Senior Responsible Officer. The EPR Programme Board papers improved during the programme, particularly from 2022 onwards when additional Programme Management Office (PMO) support was provided. The EPR Programme Board has considered the issues and risks that I would expect.
The Go Live date of Releases 1 and 2 of the programme was put back on two occasions. The third Go Live date was 27 May 2023 and was seven months after the first. The reasons for the first delay were mainly issues with overall programme management, delays in engagement with staff and delays in obtaining access to TrakCare (the old' patient information system) data for data migration. The reasons for the second delay included specific programme risks that had not been identified and managed effectively.
Releases 1 and 2 went live on 27 May 2023. These Releases were not without their operational challenges although there was a structured approach to identifying

and addressing operational issues (including issues logs and risk registers). Evidence shows that a significant number of these issues were successfully addressed. There is however concern around the number of issues that did present and the length of time it is taking to resolve all of them. At September 2023 (three months after the Go Live date) seven key risks remained on the programme risk register that required urgent attention by the programme team.

There has been a detailed lessons learnt' review of Releases 1 and 2 in May 2023. The areas identified for change centred around six key themes. Going forward, each of these themes has nominated action owners.
I have not seen any evidence of a formal transition plan to Business As Usual (BAU) which should take place once implementation is complete and the legacy systems are decommissioned or converted to read-only. Although the autumn of 2024 is several months away, it would be prudent to consider how the complete implementation will be transitioned to BAU.

Conclusions

There is a large degree of similarity between the findings of this review and those of my previous reviews of the Integrated Technology Solution (October 2021 and April 2023) and Major and Strategic Projects, including Capital Projects (November 2023).
While good practice frameworks have been established for projects such as the EPR implementation, in order to drive value for money from significant investments there needs to be a much greater focus on effective stakeholder and user engagement as well as more effective discipline around the identification, monitoring and delivery of benefits.

Objectives and scope of the review

The review has evaluated the design and operation of the EPR programme against the best practice framework developed by the UK National Audit Office, as shown in Exhibit 2.

Exhibit 2: Framework to review programmes

Delivery Programme variation and

Purpose Value set up management

Is there a Does the Is the Are

strategic need programme programme mechanisms in for the provide set up in place to deliver programme value for accordance the intended and is this the money? with good outcomes and right practice respond to programme to and are risks change, and is meet the being well the programme business managed? progressing need? according to

plan?

Source: National Audit Office: Framework to review programmes Updated April 2021

In particular the review has evaluated whether:

• clear criteria for success have been articulated such that the programme has properly evaluated the options available to deliver the benefits identified

• the procurement approach has been appropriate and follows best practice

• strong governance, programme management and project management approaches and plans are in place, including robust assessment and management of programme and project risks

• robust organisational and digital strategies have been developed with a clear view of technological requirements specific to health

• the programme and project teams have a clear understanding of the operational realities, supported by operational experts committing time to help develop the strategy

• the complexity of legacy system issues is really understood including the challenges involved in data migration and systems configuration

• best practice is being followed in respect of information governance, security and data protection risks

• robust plans are in place in respect of operational readiness for the implementation of each phase of the programme, in line with the overarching agreed timetable

• there is sufficient challenge and review of existing operational and clinical business processes in preparation for EPR implementation; and

• systems users and other key stakeholders are being engaged with effectively with clear communications plans in place.

In April 2023 I published my report Integrated Technology Solution – Follow Up which considered the operation of the ITS programme to November 2022. The EPR programme is running concurrently with the ITS programme. Some of the recommendations made in my April 2023 report are also relevant to the EPR implementation.

Detailed findings

Purpose

I considered the purpose of the EPR programme against three criteria:

• need for the programme – is it clear what objective the programme is intended to achieve?

• portfolio management and dependencies – does the programme make sense in relation to the Government's strategic priorities?

• stakeholder engagement – have the right people bought into the need for the programme?

Need for the programme

As part of the Digital Care Strategy, a Strategic Outline Case for a new EPR was documented in 2017. Historically, HCS had worked with a hybrid system of both paper and an electronic health record. This resulted in data fragmentation leading to difficulties in accessing a comprehensive view of a patient's medical history. The hybrid system did not provide a complete data analysis platform for HCS.
The development of the Business Case for the EPR commenced in February 2020. The Business Case was finalised in November 2020 and clearly outlines the reasons for the replacement of the previous patient information system (TrakCare). The November 2020 Business Case also sets out the benefits of a new EPR system.
The November 2020 Business Case demonstrates a sound understanding of the need for the programme and it sets out the high-level risks in terms of their impact and probability. The highest identified risk relates to the need to complete the clinical transformation that is required to deliver the expected benefits. The manifestation of this risk is now the key issue to the delivery of benefits and the effective operational use of the EPR system.

Portfolio management and dependencies

There has been a Digital Strategy for Health and Care since 2017. Relevant strategies were updated in November 2020 in the Digital Health and Care Implementation Plan (Summary) 2021 to 2024. The EPR programme is included in this Implementation Plan.

The November 2020 Business Casefocusses on the health-related outcomes of the EPR programme and does not consider wider pan-Government strategies and dependencies. In my review ICT Cloud Implementation – Integrated Technology Solution (October 2021) I recommended that Government document an overarching IT strategy. This recommendation has not however yet been implemented.
The November 2020 Business Case included two wider perspective long listed options (out of the eight options included).

• Procure a commercial EPR across all healthcare sectors.

• Procure a commercial acute EPR for Jersey and Guernsey.

Neither of these two options made it onto the three-option short list. Instead it was decided that the key focus of the EPR programme would be to ensure that the existing acute hospital-based patient information system was replaced.
There are other health and care IT projects that are ongoing. These include:

• a review of the future of the mental health Care Partner system; and

• a review/procurement of the future of the General Practice IT system.

There is still a need to determine whether Jersey wishes to develop an integrated health and care IT clinical record solution for the Island. This was originally part of the Jersey Care Model concept but has not been taken forward in any of the current ongoing IT projects. The decision to implement an acute EPR that is cloud based' and open' means that the future architecture of a complete Island-wide solution can still potentially be obtained in the future.
All Government IT projects are supported by and require resources from the M&D team. Over the last three years, several significant IT implementations have run in parallel and have required M&D resource. These implementations include the EPR programme as well as the Integrated Technology Solution programme (ITS). The parallel running of major IT implementation programmes has placed particular pressure on the M&D team and there has been a need to bring in additional external resources to support the ongoing implementations.
During 2021 there were points in the EPR programme when there was internal pressure to delay or stop the EPR implementation. It is essential that all Government Departments co-operate fully to support each other when planning and implementing large programmes, particularly ones that run over many years.

Stakeholder engagement

There is evidence that the level of engagement with key stakeholders has varied over the lifetime of the EPR programme from January 2020 to May 2023. The CCIO who chaired the first EPR Programme Board meeting in January 2020 was also in post when the Business Case was approved in November 2020. Since that time however there have been two changes in CCIO as well as changes in the leadership team of HCS.
To have this level of senior change in a complex and complicated long running programme, particularly one impacted by the COVID-19 pandemic may well have had a negative impact on the wider engagement with key stakeholders, particularly clinical stakeholders.
The HCS Digital Team transferred to M&D in March 2021. This transfer effectively meant that HCS subsequently lost the opportunity to have a dedicated Chief Information Officer (CIO) to work alongside the CCIO and project team on the EPR implementation and to scope and lead on major health and community services related IT projects.
The timescales identified in the Business Case approved in November 2020 were not specific enough for the impact on resource needs to be assessed in an appropriate level of detail.

Recommendation

R1 Undertake a high-level stock take of all major digital change programmes planned

over the next four years and map out these programmes against the capacity and capability of the teams within Government to support these changes.

Work planned that should be prioritised

P1 Finalise the development, approve and adopt an overarching technology strategy

for Government.

Area for consideration

A1 Consider increasing the capacity of technical experts within Government to

support on complex IT and change management programmes.

Value

I have considered whether the EPR programme has been established to deliver value using the following criteria:

• option appraisal – does the option chosen meet the programme's objective and provide long-term value?

• business case – does the business case demonstrate value for money over the lifetime of the programme?

• cost and schedule – has the programme built up robust estimates of cost and schedule, including all programme components?

• benefits – does the programme: have a baseline; know what measurable change it is going to make; and actually measure it? Are benefits being achieved?

Option appraisal

The November 2020 Business Case, the procurement documentation and the EPR Programme Board papers make it clear that a wide range of options was considered for the delivery of the EPR programme. A long list of eight options was included in the November 2020 Business Case as follows (the shortlisted options are marked with an asterisk and highlighted in bold):

Do nothing*
Do minimum and continue with TrakCare and defer an upgrade to the UK edition until the new hospital is in place
Direct upgrade of TrakCare to the UK edition
Develop own in-house EPR
Procure a commercial hybrid' EPR*
Procure a commercial best of breed' acute EPR*
Procure a commercial EPR across all healthcare sectors
Procure a commercial acute EPR for Jersey and Guernsey

Procure a commercial hybrid EPR was the recommended option.

A Procurement Strategy for the programme was documented in September 2020. The November 2020 Business Case stated that three separate procurements (EPR software managed service, EPR implementation and transformation support and data migration) would be run in accordance with the Procurement Strategy.
I understand that the decision to package the programme into three separate procurements was made on the basis that smaller-scale providers would not be able to meet the full set of requirements for all three procurements. However this rationale is not explained in the Procurement Strategy or November 2020 Business Case. The Procurement Strategy did not provide a full examination of how the procurement could be packaged to minimise risk to the Government, meaning that the options for how risk could best be transferred to suppliers were not articulated.
If best practice had been followed there would have been a formal options appraisal which examined the risks and benefits of the potential packaging options. Even though smaller scale providers may not be able to meet the full set of requirements, it would have been possible to have placed the responsibility on a provider to enter into appropriate sub-contracting arrangements so that a prime contractor could have been appointed across some or all of the packages.
In practice, the procurement approach adopted differed to the Procurement Strategy in respect of EPR implementation and transformation support and data migration. The EPR implementation procurement package comprised:

• programme management and leadership

• clinical change management

• training; and

• system configuration and testing.

A decision was made during the procurement (at the EPR Board Meeting on 22 June 2021) to halt the procurement due to the scale of the costs submitted by the bidders. It was decided that the implementation work would instead be undertaken using internal resources. A specialist external company was to be asked, using an existing draw-down arrangement with that supplier, to undertake an EPR Readiness Assessment' to determine how the implementation and transformation package could be delivered internally. The EPR Board Meeting documentation from 16 July 2021 states that an internal organisational structure had been created to 'fill the gap' as a result of the decision to halt the EPR implementation and transformation procurement and to deliver it internally. As the programme progressed, an external supplier was appointed to support the

internal team in respect of testing. This procurement was later subject to the declaration of a retrospective procurement breach with a stated value of £180,000.

The procurement for data migration was also halted during the planned procurement process. Subsequently a decision was made to appoint a single supplier using a procurement exemption. The rationale for the exemption and selection was that the supplier had direct experience of recent migration from TrakCare (the old acute patient information system) to IMS MAXIMS (the new EPR system being implemented).
A new CCIO took up post in July 2021 and expressed concerns as to how the project had progressed to that point. The new CCIO provided a catalyst for a reset of the project at that time. This reset included the decisions to halt two of the initial procurements and appoint suppliers directly under procurement exemptions so that the project could proceed towards a more effective implementation. If there had been a better original understanding of the market for these services during the initial procurement earlier in 2021, this reset and the associated procurement exemptions and breaches would not have been required.

Business case

Since 2017 there has been a Digital Strategy for Health and Care in Jersey. Relevant strategies were updated in November 2020 in the Digital Health and Care Implementation Plan (Summary) 2021 to 2024'. This Implementation Plan included the Acute EPR programme.
The development of the November 2020 Business Case commenced in February 2020 and was supported by external expertise. There are risks associated with significant reliance on external expertise including continuity of staffing and lack of knowledge transfer.
The broad equivalent of a Strategic Outline Case (SOC) had been produced in 2017 and the November 2020 Business Case contained the content expected from an Outline Business Case (OBC).
In May 2021, the Project Delivery Framework was approved which represents a controlled framework for consistent delivery of projects across the Government of Jersey. For projects which were in-flight' prior to the launch of the Framework (such as the EPR programme), the Framework was applicable at the next stage gate' on the project. For the EPR programme this meant compliance from May 2021 onwards.
As the procurement stage of the EPR programme completed in October 2021, a FBC should have then been completed if the criteria requiring a FBC were met. No FBC was produced after the procurement stage of the EPR programme or after

the contract negotiations with the preferred bidder that took place in the summer and autumn of 2021. It is therefore difficult to reconcile the costs that have been approved (and subsequently incurred) in respect of this programme to a business case that demonstrates how value is expected to be driven from these costs.

A high-level implementation plan is included in the Business Case. However there is no detail which justifies the proposed plan and there is no consideration of contingency arrangements should the plan slip.

Cost and schedule

The November 2020 Business Case underpinned the inclusion of the scheme in the Government Plan 2021-2024. This Government Plan had total approved costs of £14.9 million (£9.4 million capital expenditure and £5.5 million revenue expenditure). The executive summary in the November 2020 Business Case has a high-level breakdown between revenue and capital totalling £29.2 million over the 11-year period from 2021 to 2031. In the detailed sections of the Business Case, the costs of the viable options considered are explored. There is however no correlation between these costs and the overall figure in the executive summary.
The November 2020 Business Case states that the total four-year approved EPR capital budget is £9.4 million. However the Business Case also goes on to say the projected capital spend for this four-year period is forecast to be £7.8 million.
The same Business Case states the approved four-year EPR revenue budget is £5.5 million,while the forecast revenue spend is £7.0million. The Business Case shows a combined capital and revenue forecast underspend of £168,000 against the total approved Government budget of £14.9 million. However, these costs include both expenditure on the existing TrakCare system and the new EPR.
The way in which financial information was included in the November 2020 Business Case does not enable a transparent audit trail to the funding for the programme included in the Government Plan 2021-2024. In addition, no FBC was produced at the end of the procurement stage. The financial information in the November 2020 Business Case did not reflect the agreed contract entered into after the procurement stage. There is though an audit trail of EPR financial information from the 2021-2024 Government Plan onwards.
The EPR programme is annually funded from the wider HCS digital transformational programme. While four-year programme funding was outlined in the November 2020 Business Case (and included in the Government Plan 2021 – 2024) the EPR funding, annually agreed, differs from this. This makes it difficult to understand and evidence whether this programme has over or underspent across its lifetime specifically from the November 2020 Business Case.

The EPR programme budget presented to the EPR Programme Board on 27 September 2023 shows a total cumulative budget up to the end of 2023 of £5.8 million against a forecast spend of £6.6 million (an overspend of more than £700,000).
However, the EPR programme budget does not include costs that were included in the original November 2020 Business Case (for example the previous TrakCare system ongoing costs, the costs of support staff from the M&D team and facilities costs).
The EPR programme has a long implementation period from October 2021 to the third quarter of 2024 (three years). During that time the EPR capital funding is not visible within the EPR programme budget because the EPR capital funding is drawn from a wider HCS digital transformation programme. One of the consequences of this is that it is hard to track whether the EPR programme is overspending. In practical terms, if the EPR programme does overspend, the rest of the HCS digital transformation programme will need to be reprioritised. The EPR programme revenue funding is even less visible as this is embedded within large departmental budgets.
At the time of my review there was an ongoing contractual dispute relating to the impact of inflation on the costs caused by delays to the programme. It is clear that there have been programme delays that have resulted in additional costs in respect of both the new EPR system and the TrakCare system. For example, due to the programme missing dates on data migration the second planned Go Live date was pushed back. One of the potential consequences of this was that there could have been significant additional costs in extending contracts for TrakCare. Initially an additional six months of contract costs (in excess of £380,000) were due to be incurred on TrakCare due to a failure to give three months' notice to terminate the contract at the end of the initial extension period. These costs were only avoided after the intervention of the Government Chief Digital Officer and negotiation with the supplier.
More generally, digital investments present challenges when it comes to the correct accounting treatment as revenue or as capital expenditure. The final accounting treatment will be determined at the end of the programme to ensure consistency with the original sources of Government funding.

Benefits

The benefits are identified in the November 2020 Business Case, including performance measures. The benefits outlined were:

• cash releasing savings over a ten-year period of £60,000, due to reduction in paper costs

• non–cash releasing savings of £12.7 million due to reduced length of inpatient stay and reduced administrative workload due to automation; and

• other qualitative benefits, such as improved data, quality of data, improved confidence of patients and reduced litigation.

The November 2020 Business Case had a clear and realistic view on what good looks like' and how benefits would be measured. In my view, the financial benefits outlined are not overly ambitious. The realisation of them does however rely on HCS adopting standardised clinical and business processes, which the new EPR would facilitate. This change in clinical working practices is not a technical change, it is an adaptive behaviour change. For such adaptive change to be effective and to realise benefits there needs to be excellent, skilled leadership and high degrees of engagement from those affected by the change.
There is a benefits tracker in existence that has been derived from the November 2020 Business Case. This benefits tracker was not however being actively used at the time of my audit as the emphasis of the programme was still on the short term operational delivery of Releases 1 and 2. While I note that a role of Change and Benefits Manager has been introduced there is a risk that between now and the final system Release 5 in the third quarter of 2024 the programme will be stuck in short term operational delivery mode and that benefits realisation may be pushed back to the fourth quarter of 2024, or even forgotten.
In summary, the programme is not in a position to validate any benefits achieved at this date and there is a risk that benefits may never be quantified and validated.

Recommendations

R2 Produce an ongoing full cost summary for all long running Major and Strategic

programmes, particularly those funded through wider Government or Departmental programmes or where funding is allocated to multiple Government Departments. This summary should be reconciled annually, to ensure whole life programme cost control is visible.

R3 Introduce requirements that ensure that procurement strategies document the

options for packaging the procurement in ways that lower the level of risk to the Government and detail the likely costs of each option under consideration.

R4 Ensure that programme benefits are identified and tracked as an integral part of

programme delivery during the planning and delivery phases and not left until after programme closure and the move to business as usual.

Programme set up

A pre-condition for successfully starting a programme and running an effective competition for commercial partners is that everyone involved in delivering the programme clearly understands what must be delivered, and when. Immature or incomplete specifications lead to scope creep and confusion across the supply chain and make it difficult to incentivise commercial partners to deliver effectively and to hold them to account for any subsequent shortcomings.
I evaluated the following elements of programme set up:

• governance and assurance – are there structures (internal and external) which provide strong and effective oversight, challenge and direction?

• leadership and culture – does the programme have strong leadership with the necessary authority and influence?

• resources – has the organisation allocated the resources (staffing, skills, equipment and so on) required to deliver the programme?

• putting the programme into practice – are scope and business requirements realistic, understood, clearly articulated and capable of being put into practice?

• risk management – are key risks identified, understood and addressed?

Governance and assurance

There is a clear governance framework that supports the Digital Health and Care Implementation Plan. This includes a Digital Health Portfolio Board, chaired by a Sponsoring Senior Responsible Officer with representatives including a Senior User, Project Management Officer and Supplying Senior Responsible Officer.
The initial EPR programme governance structure was created in January 2020 with the then CCIO as the Programme Sponsor. This governance structure included an EPR Programme Board with an agreed terms of reference.
EPR Programme Board meetings have taken place monthly throughout the programme to date. Exhibit 3 contains some of the key milestones in decision making through the EPR Programme Board meetings.

Exhibit 3: EPR Programme Board milestones

Date		Miles tone
30 January 2020		EPR Programme Board formed
28 February 2020		Commencement of Business Case development
29 September 2020		Procurement Strategy approved
20 November 2020		Business Case produced
27 January 2021		Procurement process update and outline supplier
		evaluation process
22 June 2021		Procurement outcome meeting
5 July 2021		Extraordinary meeting of the Digital Health Portfolio Board
		with the aim of the M&D team providing assurance to the
		HCS client team on the programme due to concerns being
		raised that M&D are considering delaying or stopping the
		programme
5 October 2021		Confirmation of readiness to proceed to contract award
14 October 2021		Draft Project Initiation Document (PID) created by IMS
		MAXIMS
3 March 2022		PID formally signed off
26 May 2022		Agreement to delay the planned October 2022 Go Live to
		February 2023
27 October 2022		The planned Go Live date of February 2023 for Release 1
		was stated to be at risk
30 November 2022		Agreement to delay the February 2023 planned Go Live for
		Release 1 to May 2023 and to combine Releases 1 and 2 to
		the same date
27 May 2023		Go Live of Releases 1 and 2

Source: Jersey Audit Office analysis of EPR Programme Board papers and minutes

The EPR Programme Board papers improved during the programme, particularly from 2022 onwards when additional PMO support was provided. The EPR Programme Board has considered the issues and risks that I would expect.
There have been a number of changes in leadership throughout the programme. The PID was not updated however to reflect these changes.

Leadership and culture

The business case process for the EPR replacement started early in 2020 (before the COVID-19 pandemic) and came at a challenging time for HCS. The procurement process ran between November 2020 and July 2021, again atan extremely challenging time. There is evidence to suggest the clinical engagement in this procurement phase was not as strong as it should have been and this may have contributed to a lack of ownership of the preferred solution among some clinicians.
The system mapping, build, data migration, testing, training and Go Live, ran from the autumn of 2021 to the spring of 2023 (a period of 18 months). During this time there were other significant challenges affecting HCS and the M&D team including:

• increased operational pressures arising from the clinical consequences of post - COVID-19 pandemic health and social care on the Island

• significant problems in recruiting and retaining staff in HCS, leading to high volumes of agency and locum staff

• several internal management reorganisations within HCS as detailed in my report Deployment of Staff Resources in Health and Community Services (January 2023). These included the transfer of the HCS Digital team into the M&D team in the first quarter of 2021

• dealing with the recommendations contained in the report into clinical governance arrangements from Professor Hugo Mascie- Taylor

• the implementation of ITS that went live in January 2023; and

• changes in senior leadership within HCS including changes in the Chief Clinical Information Officer (CCIO) role.

While the EPR programme was about implementing a new IT system, it is more fundamentally a huge change management project, impacting on the working lives of the majority of HCS employees. The implementation came at a time when there was disruption in HCS leadership. Exhibit 4 illustrates this disruption.

Exhibit 4: Illustration of disruption in leadership

Dates Activities Comments about leadership

February to Business Case HCS leadership focus on the COVID-19 November development pandemic

2020

March 2021 HCS Digital Team Chief Clinical Information Officer and

transfer to the Sponsoring EPR programme SRO leaves his role M&D team

March – June Procurement Interim CCIO in place 2021 process ongoing

June 2021 Decision made to While the outcome of the procurement process choose a would have been the same, there was no senior

preferred bidder HCS operational executive at the EPR Programme Board for this decision

HCS was represented by an Associate Director of Innovation and Improvement, a senior nurse, an interim CCIO and a deputy Head of Health Informatics

July 2021 Procurement ends New CCIO takes up post and becomes joint' (preferred bidder Sponsoring SRO for the EPR programme

identified) HCS Group Managing Director who is also joint' Sponsoring SRO leaves HCS

July – October Contract Chair of Digital Health Portfolio Board assumes 2021 negotiations take joint' Sponsoring SRO for the EPR programme

place alongside CCIO

July 2022 EPR PMO set up Professor Hugo Mascie- Taylor report into

Clinical governance arrangements within HCS' published in August 2022

October 2022 First EPR Go Live

date missed

January 2023 EPR programme HCS Chief Officer (and Accountable Officer

ongoing (AO) for the EPR programme) leaves

Chair of Digital Health Portfolio Board becomes interim AO and steps back from chairing the EPR Programme Board. Director of Clinical Services becomes HCS operational lead for EPR and joint' Sponsoring SRO

HCS turnaround team appointed

Dates Activities Comments about leadership

Chief Operating Officer responsible for the M&D team leaves

February 2023 Second EPR Go

Live date missed

April 2023 New interim HCS Chief Officer and AO

appointed (from the HCS turnaround team)

24 May 2023 Last EPR There was strong senior clinical representation

Programme Board at this meeting including the Medical Director, before the third the Chief of Service Surgical Group, Chief of Go Live date Service Primary Care and CCIO

There was however no HCS senior operational leadership present at the meeting. Those not present included the Interim Director of Clinical Services, the Interim Director of Nursing, the Associate Director of Innovation and Improvement and the Head of Health Informatics

27 May 2023 Third EPR Go Live

date achieved

Source: Jersey Audit Office analysis

These wider leadership changes did not derail the EPR implementation but did result in some delay to the programme (particularly the October 2022 delay). The leadership changes did however have a negative impact on the quality of clinical engagement with the programme. The consequence of this is that the quality of the EPR implementation suffered. Effective clinical engagement will need further work in the future if the benefits envisaged in the November 2020 Business Case are to be realised.
More emphasis needs to be placed on the vital importance of programme leadership and the continuity of leadership throughout longer running programmes, particularly the key role of Sponsoring SRO. From a HCS perspective this programme appears to have been driven by six key senior leaders from February 2020 to the time of my fieldwork.

• Original CCIO - February 2020 to March 2021

• Interim CCIO - March 2021 to July 2021

• New CCIO - July 2021 to present

• Group Managing Director HCS – February 2021 until July 2021

• Associate Director of Innovation and Improvement (HCS) - July 2021 to January 2023

• Director of Clinical Services (HCS) – January 2023 to present

Resources

The EPR programme has taken place over a period of time when both HCS staff and the M&D team have been heavily committed to other change programmes. For example, during 2020 and 2021 HCS digital teams and M&D resources were re-deployed into COVID-19 Track and Trace and Vaccination IT programmes.
The HCS Digital Team transferred to M&D in March 2021. At this time the M&D Chief Officer was keen to stop or delay the EPR implementation because it was happening at the same time as the implementation of ITS. The decision to proceed with the EPR implementation was however confirmed at a meeting of the Digital Health Portfolio Board on 5 July 2021. It was acknowledged that the version of TrakCare being used by HCS was an older version and there was a risk that it would not be supported in the future.
The decision to halt the implementation and transformation procurement and deliver these activities internally placed an increased burden on the internal team. The EPR Programme Board documentation of 16 July 2021 states that an internal team had been created in response to that decision. As the programme progressed, further decisions were made to bring in additional external resources to support the programme in relation to data migration and testing. The data migration contract required a procurement exemption and the award of the testing contract was a procurement breach.
This transfer of the HCS Digital Team to M&D effectively meant that HCS lost the opportunity to have a dedicated Chief Information Officer (CIO) to work alongside the CCIO and project team on the EPR implementation. There was no senior HCS operational lead named in the updated PID (dated 3 March 2022) aside from the CCIO. The lack of a dedicated HCS CIO meant that this role could not be included in the PID and neither was a communication or engagement officer named in the PID.

Putting the programme into practice

The decision to halt the procurement of implementation support and undertake the work internally also placed significant pressures on operational HCS staff. A significant number of clinical leads (mainly nursing staff) were seconded to the EPR programme for a period of 18 months to assist in the mapping of business processes for the new EPR system. While these staff understood clinical processes they did not have previous specific skills and experience in business process

mapping. Additionally, the internal team did not have sufficient experience of testing and an external supplier was appointed (under a procurement breach to the value of £180,000).

The Go Live date of the programme was put back on two occasions. The third Go Live date was 27 May 2023 and was seven months after the first. The reasons for the first delay were mainly issues with overall programme management, delays in engagement with staff and delays in obtaining access to TrakCare (the old' patient information system) data for data migration.
The reasons for the second delay included specific programme risks that had not been identified and managed effectively. For example, an interim maternity module had to be built into the new system during late 2022 and early 2023 due to issues experienced with the intended supplier for a separate maternity module.
Releases 1 and 2 went live on 27 May 2023. These Releases were not without their operational challenges although there was a structured approach to identifying and addressing operational issues (including issues logs and risk registers). Evidence shows that a significant number of these issues were successfully addressed. There is however concern around the number of issues that did present and the length of time it is taking to resolve all of them.
One of the principles behind the implementation of the new system was that it was intended to be a like for like' systems replacement based on existing functionality and existing business processes. In other words it would be based on how clinical and administrative staff performed their work rather than introducing an optimal new way of working.
In practice however the new system works very differently to the old system with the consequence that the user experience of the two systems is very different. While all of the business (clinical) process mapping was approved by Clinical Directors it is not clear how well the implications of the new processes were understood, particularly given the variable engagement from senior clinicians in the programme.
The old system (TrakCare)was a highly tailored and permissive system, based on a number of discrete tasks (for example booking an outpatient appointment, admitting an inpatient, creating a theatre list). Many people could undertake these tasks in TrakCare and in any order. As an illustration of the impact of this, during data migration to the new system, it was found there were over 400,000 open patient consultations in the system, which meant a patient could see a consultant again at a later date, without needing to be referred by a GP.
The new system (IMS MAXIMS) is a modern referral-based, standardised workflow system and pre-agreed tasks are allocated to roles in that process. These roles

have pre-agreed and authorised permissions. As a consequence, if an individual does not undertake the tasks allocated then that patient cannot be referred onto the next role in the process. In addition, a person cannot undertake the tasks assigned to another person unless they have authorised permission to do so.

The extent of change between the two systems is therefore significant, particularly for Consultants. The impact for staff is that for the new system to work effectively, all of the following must be in place:

• the current business process must have been correctly mapped

• the current business process must have been correctly built into the new system

• the correct permissions to undertake specific roles must be in place

• all staff must be trained on how to use the system and must be competent to use it

• staff must have access to appropriate digital devices to undertake the tasks allocated to them

• staff must have the time in their working day to undertake the tasks assigned to them; and

• staff must be culturally happy and committed to working within a standardised process and agree that certain tasks can only be performed by specific roles and individuals.

100. While the technical aspects of the EPR implementation have been done reasonably

well, the challenges experienced in practice in the implementation have related to where the system is dependent on staff to understand and perform the system roles allocated to them.

Risk management

101. The November 2020 Business Case demonstrates a sound understanding of the

need for the programme and it sets out the high-level risks in terms of their impact and likelihood. The highest identified risk relates to the need to complete the clinical transformation that is required to deliver the expected benefits. The manifestation of this risk is now the key issue to the delivery of benefits and the operational use of the EPR system.

102. The November 2020 Business Case states that there would be three separate

procurements (EPR software managed service, EPR implementation and transformation support and data migration). This is based on the Procurement Strategy that was documented in September 2020. Neither the Procurement Strategy nor the November 2020 Business Case identify the overall risk of this procurement approach instead of having a prime contractor for end-to-end programme delivery. This is a major gap in the risk analysis.

103. The management of programme risks has evolved over the duration of the

programme. From the summer of 2022, the management of programme risks improved significantly and was fit for purpose thereafter. Releases 1 and 2 went live on 27 May 2023. There was a structured approach to identifying and addressing operational issues, including issues logs and risk registers.

104. Three key areas of risk that I examined as part of my review related to information

governance, information security and data protection. While at the outset of the implementation programme there were robust standards in place for information security, the implemented processes for information governance were immature, and a formal Data Protection Impact Assessment (DPIA) had not been developed. This is now under development aided by a dedicated resource in M&D and is being enriched as the implementation programme progresses through its five phases.

105. One key role in relation to information governance and medical ethics is the

Caldicott Guardian. This is usually a senior clinician with a broad understanding of the whole spectrum of health services, and who has a detailed understanding of how health-related data should be controlled so that ethical standards are met. I have been provided with evidence that the Caldicott Guardian for HCS possesses these qualities and has a sufficiently senior level of responsibility in HCS.

106. The Government of Jersey Data Protection Office (DPO) also has an important role

in reviewing data protection arrangements as the EPR programme is implemented. I was provided with evidence that the DPO will be engaged at appropriate times during the implementation and that a strong relationship exists between the information governance function in HCS and the DPO.

107. In the section below that examines the delivery strategy we note that the selected

supplier (IMS MAXIMS) did not have formal ISO27001 accreditation at the time the pre-qualification questionnaire was evaluated as part of the procurement process. In October 2021 IMS MAXIMS achieved this standard, and that should provide the Government with comfort that its key supplier has robust information security standards in place.

108. At September 2023 (three months after the Go Live date) there were seven key

risks on the programme risk register that required urgent attention by the programme team. These were listed as:

• CareFlow Medicines Management, MilkTrac Module – the live set-up of handheld devices for this module was delayed due to configuration issues with the device

• clinical notes – the risk that patient records become fragmented as clinicians are updating records on paper and digitally which results in inconsistency

• planned surveillance of waiting lists – the risk of planned surveillance legacy records which need to have the planned date on waiting list have not been corrected in the system (with approximately 1,000 records to be corrected)

• additional budget requirements that may be required if contractual agreement settlements are not reached

• clinical coding – the risk that the clinical coding team may not be able to extract information required by multiple sources (EPR, case notes and other integrated systems) to assign codes

• cardiology – the risk that new EPR digital pathway is not fit for purpose preventing correct referrals to be completed by the team; and

• specific systems compliance issues relating to To Come In (TCI)', theatre builds and endoscopy.

109. These very granular and specific risks on the risk register evidence the fact that

clinicians and managers can engage in the programme and escalate issues (bottom up) as well as the EPR programme governance structure identifying risks (top down). However, the current risks highlight that too high a proportion of the engagement between clinicians and managers is retrospective. There is evidence that in practice despite significant attempts made by the EPR programme team, levels of engagement from senior clinicians and managers fell below what was expected and hoped for.

Recommendations

R5 Introduce a requirement for Project Boards to formally review and approve an

updated Project Initiation Document when there are changes (such as leadership changes) that materially impact a Major or Strategic Project.

R6 Ensure that a formal Data Protection Impact Assessment (DPIA) is developed for

the EPR programme.

Area for consideration

A2 Consider introducing a dedicated Chief Information Officer role for health and

community services within Government, either within HCS or within M&D. This role should focus on the broader spectrum of health and community services related information technology, encompassing clinical, administrative, and operational aspects, aligning technology with overall strategy and objectives and overseeing procurement and implementation.

Delivery variation and management

110. I have considered the arrangements within the EPR programme for delivery

variation and management. In doing so, I have considered specifically:

• delivery strategy – are there appropriate incentives for all parties to deliver (contractual, performance management or other)?

• change control – is there an effective mechanism to control programme alterations?

• responding to external change – is the programme sufficiently flexible to deal with setbacks and changes in the operating context?

• performance management – is progress being measured and assessed, including consideration that the programme is still the right thing to do?

• lessons learned – is the programme learning from experience on the current programme and previous relevant programmes?

• transition to business as usual – does the programme have a clear plan for transfer to operations/business as usual?

Delivery strategy

111. Following the appointment of a new CCIO in July 2021 there was a period of

significant contract negotiation before the final contract was signed in October 2021. Prior to the new CCIO taking up post, the negotiations on contractual terms had been led by an external consultancy firm. The new CCIO identified significant risks to the Government with the contracts that had been proposed. The decision was made to seek to reduce the contractual risks to Government through further contractual negotiations that took place between July and October 2021. These negotiations resulted in:

• the inclusion of significant penalties into the contract

• the inclusion of a requirement for the digital keys' of the system to be put into ESCROW in the event that the supplier goes into administration; and

• the de-coupling of secondary supplies to minimise risk and increase flexibility.

112. The final contract that was signed did not reflect the terms and conditions notified

to potential bidders at the procurement stage. While the final terms and conditions were more favourable for the Government, it is not best practice to allow post award contract negotiations to take place.

113. The procurement of the EPR software was run under a Restricted' procedure

which is a two-stage procurement. Stage one is the Pre-Qualification Questionnaire (PQQ) which assesses the technical and economic strengths of each bidder. I have the following observations on the PQQ response for the successful applicant:

• the applicant had, at that time, a less than favourable financial track record and had made significant losses for the trading years submitted in its PQQ response

• the applicant, at that time, was asked whether it had a formal IT security accreditation such as ISO27001 – the applicant stated that it did not, but was working towards one; and

• the PQQ has a pass/fail question in relation to the annual value of the contract in relation to the applicant's turnover, with specified limits on what would be classified as too small' or 'too large' so that an applicant could be assessed as being an appropriate and meaningful supplier. I have seen no evidence that this was evaluated as part of the PQQ assessment.

114. The next stage of the Restricted' procedure is that the technical requirements

specified in the Invitation To Tender (ITT) are then assessed for applicants that were successful at the PQQ stage.

115. In my view the PQQ assessment was insufficiently detailed and key questions as to

supplier viability were not considered fully. At the contract award stage, further questions were asked of the successful supplier, which were in effect post-contract negotiations. This approach is not consistent with best-practice procurement processes.

116. Some elements of the eventual contracts awarded required procurement

exemptions (for example the data migration contract) or were a procurement breach (for example the testing contract). There is an inherent risk that the use of procurement exemptions and breaches could stifle competition for the provision of services to Government and result in a risk of poor value for money.

117. In late December 2021, two months after the main contract was signed, an EPR

Programme Roles Charter was approved. Embedded in this structure were key roles for individuals (Clinical Leads) and Groups (Clinical Reference Groups). Two- way communication was a central part of these roles.

118. While the main contract was signed in October 2021, the ongoing COVID-19

pandemic contributed to delays in proceeding with the contract in 2021 and the early part of 2022. This led to an ongoing dispute regarding tender prices.

119. The effectiveness of the delivery strategy processes can be separated into two

parts:

• the programme from October 2021 to July 2022; and

• the programme from July 2022 to the present day.

120. Key to the effectiveness was the establishment of a PMO in July 2022 supported by

external resources. The establishment of the PMO contributed to increased effectiveness of the delivery strategy in practice.

Change control

121. Where major changes have been required such as extensions to Go Live dates,

decisions to halt certain procurements and instead resource internally, and to re - think the programme budget, there is clear evidence that these have been approved at the ERP Programme Board. In some instances there is a lack of evidence of discussion as to the available options at the time a decision is made. For example the decision to halt the procurement of a transformation partner and resource internally was labelled in the Programme Board documentation as No asks – Board information only'. I would however expect that such a major decision would be a point for discussion rather than being for information only.

Responding to external change

122. The programme has been largely successful in responding to external change. By

far the biggest example is the fact that the programme had to deal with the impact of the COVID-19 pandemic which had profound impacts on, for example, clinical engagement and the progress of contractual negotiations for the EPR package. As noted above, the establishment on a PMO in July 2022 has strengthened the Programme's ability to deal with externally imposed change.

Performance management

123. An effective performance management regime should assess at regular intervals

the progress made against the programme's key delivery milestones with regular assessments as to the need to continue to operate the programme. Since the establishment of the PMO there has been an improved level of focus on programme delivery for the EPR programme. In addition, the risk assessments as to the ongoing EPR programme implementation are now visible and well managed.

Lessons learned

124. There was a detailed lessons learnt' review of Releases 1 and 2 in May 2023. The

areas identified for change centred around six key themes:

• communications

• training

• plan visibility

• team structure

• change management (super users); and

• supplier management.

125. Going forward, each of these themes has nominated action owners. Transition to business as usual (BAU)

126. Releases 3 to 5 of the EPR programme are planned to conclude in autumn 2024.

There is however the potential for slippage against this plan because of the nature of the programme and the need to engender further clinical engagement as implementation progresses.

127. I have not seen any evidence of a formal transition plan to BAU which should take

place once implementation is complete and the legacy systems are decommissioned or converted to read-only. Although the autumn of 2024 is several months away, it would be prudent to consider how the complete implementation will be transitioned to BAU.

Recommendations

R7 Review procurement processes to ensure that all potential suppliers have the

opportunity to submit bids which can be evaluated equally, that terms and conditions of the contract are defined at the appropriate stage and that post - award negotiations are avoided.

R8 Ensure that Programme Boards are required to document their consideration of

options and rationale for key change decisions.

R9 Commence planning for Business as Usual (BAU) for the complete implementation

of the programme so that effective service management processes can be established in advance of the final phases of the ERP programme implementation.

Appendix One Audit Approach

This audit used a results-orientated approach focussing on:

• What results have been achieved?

• Have the States of Jersey met their objectives?

The audit used the following criteria

Source: National Audit Office: Framework to review programmes Updated April 2021

In particular the review evaluated whether: