Thinkpiece - Learning from previous IT implementations

Learning from previous IT

implementations:

A Thinkpiece

14 February 2025

jerseyauditoffice.je  R.19/2025

The purpose of the Comptroller and Auditor General (C&AG), fulfilled through the Jersey Audit Office (JAO), is to provide independent assurance to the people of Jersey on the extent to which public money is spent economically, efficiently and effectively and on whether the controls and governance arrangements in place within public bodies demonstrate value for money. The C&AG's remit includes the audit of financial statements and wider consideration of public funds, including internal financial control, value for money and corporate governance.

This Thinkpiece can be found on the Jersey Audit Office website at https://www.jerseyauditoffice.je/

If you need a version of this Thinkpiece in an alternative format for accessibility reasons, or any of the exhibits in a different format, please contact enquiries@jerseyauditoffice.je with details of your request.

All information contained in this Thinkpiece is current at the date of publication.

The Comptroller and Auditor General and Jersey Audit Office are not responsible for the future validity of external links contained within the report.

All information contained in this Thinkpiece is © Copyright Office of the Comptroller and Auditor General and the Jersey Audit Office, with the exception of extracts included from external sources, which are © Copyright to those external sources.

The information contained in this Thinkpiece is for non-commercial purposes only and may not be copied, reproduced, or published without proper reference to its source. If you require the material contained in the Thinkpiece for any other purpose, you are required to contact enquiries@jerseyauditoffice.je with full details of your request.

Thinkpiece by the Comptroller and Auditor General: 14 February 2025

This Thinkpiece has been prepared in accordance with Article 20 of the Comptroller and Auditor General (Jersey) Law 2014.

Contents

Introduction .................................................................................................................................... 4 Setting a strategic ambition .......................................................................................................... 6 Progress on developing an overarching strategy ...................................................................... 6 Risks associated with the current position .................................................................................. 7 Articulating the business case ...................................................................................................... 9 Effective procurement .................................................................................................................12 Stakeholder engagement ...........................................................................................................17 Project and programme management ......................................................................................24 Cost and financial controls .........................................................................................................30 Capacity and management of contractors ...............................................................................33 Appendix One - Approach adopted in preparing this Thinkpiece........................................41 Appendix Two - Framework to review programmes ...............................................................43 Appendix Three - Summary of Learning for Future IT implementations...............................46

Introduction

1. The States of Jersey have committed over £240 million in recent years to Information Technology (IT) programmes (see Exhibit 1). Of this commitment, almost £142 million had been spent to the end of 2023 with further expenditure committed of almost £100 million between 2024 and 2028.

Exhibit 1: States of Jersey actual and committed expenditure on IT programmes

Source: States of Jersey Group Annual Report and Accounts 2020, 2021, 2022 and 2023, Government Plan 2024-2027 and Budget 2025-2028

2. Many of these programmes require broad business changes of which the IT implementation is one element. In common with many other organisations, not all of the States' IT programmes have run to initial time and initial budget. In addition,

there has been expenditure on projects that have not been implemented, for example an automatic electoral registration system.

3. As the Government embarks on its next phase of investment it is important that it uses lessons learned to maximise the opportunities and minimise the risks inherent in IT implementation projects.
4. In developing this Thinkpiece, I have considered relevant reports from the C&AG, Scrutiny and the Public Accounts Committee (PAC), States' internal reports and reports and best practice from other jurisdictions.
5. A list of the documents reviewed is included at Appendix One. I have also considered the key elements of the framework I have previously used to review programmes as outlined in Appendix Two.
6. I have structured this Thinkpiece around eight key areas of learning I have identified as relevant to the States. These areas shown in Exhibit 2.

Exhibit 2: Key areas of learning  

Setting a  Articulating the  Effective procurement strategic vision business case

Stakeholder  Programme and  Cost and financial engagement project management controls

Capacity and

Tracking and realising management of

benefits contractors

7. I will continue to undertake audits of significant IT investment. I plan to undertake an audit of cyber security arrangements during 2025 and an audit of the Transform project during 2026.

Setting a strategic vision

Progress on developing an overarching strategy

8. Over recent years the Government of Jersey has embarked on a significant programme of digital modernisation. It has done so however without an underpinning vision for digital services to citizens and without a supporting overarching IT strategy.
9. A digital strategy determines how the way that services are delivered should evolve. In other words, it sets out a vision for the future of digital services to citizens. An IT strategy should drive the implementation of the digital strategy.
10. The Corporate Services Scrutiny Panel report on the Government Plan 2021-24, published in December 2020, found that 'there is no published strategy covering all IT spending in the Government Plan although this was mentioned as an action by Government following the recommendations put forward by the Panel in the previous Government Plan 2020-23'.
11. In its response to the Corporate Services Scrutiny Panel report of December 2020, the Government stated that it would commence work in Quarter 1 of 2021 to prepare a Technology Investment Strategy for the coming years and that it would be available to the public. A draft document had been prepared in 2019 that set out an overview of how the elements of technology investment that were planned at that time linked to one another and to an overall approach. This document however was not finalised or approved.
12. In my report ICT Cloud Implementation – Integrated Technology Solution (October 2021) I recommended that Government document an overarching IT strategy. This recommendation was accepted with a planned publication date of 31 March 2022 for an approved IT Strategy.
13. My Report Integrated Technology Solution – Follow up (April 2023) noted that an IT Strategy had been drafted but not finalised. A new target date of 30 September 2023 was agreed for the finalisation and publication of the Technology Strategy.
14. Since my 2023 Report there have been further delays in the development and finalisation of a Technology Strategy. At the end of 2024, the Government remains in a position of not having a clearly articulated vision for digital services to citizens and not having a supporting overarching IT Strategy.

Risks associated with the current position

15. Having a clear vision for digital services to citizens underpinned by a supporting IT strategy would enable the States of Jersey to:

• ensure there is a clear and shared view of what digital transformation the States wish to achieve and why

• ensure that the cultural implications associated with the proposed digital transformation are understood fully

• set clear boundaries to help prioritise investment and ensure that individual programmes have clear objectives, avoiding scope creep

• articulate a clear risk appetite for the changes associated with digital transformation

• manage stakeholder expectations

• set out the transformation in sequenced stages, providing clarity on the interconnectedness of individual IT programmes; and

• confirm that the right levels of resources, capabilities and skills are in place to deliver the vision and strategy.

16. I recognise that the Government has undertaken an exercise to prioritise the IT projects for investment as part of the proposed Budget 2025 to 2028. The prioritisation and re-prioritisation of projects and programmes in the absence of a coherent overarching vision and strategy however creates increased risks that projects and programmes will be implemented in a sub-optimal way and may not deliver intended benefits to citizens.
17. Exhibit 3 depicts the areas that should be considered in developing a digital vision supported by an underpinning technology strategy.

Exhibit 3: Digital and IT transformation strategy development process

Digital and IT strategy development process

Key business  Current state  Gap  Action and  Delivering the drivers assessment analysis investment planning  transformation

• Government  • Technical  • Evaulation of current  • Development of

objectives and  infrastructure and  state against current  business cases • Development of aspirations portfolio assessment and future business  • Development of  external procurement

• Internal business  • Supply chain  requirements prioritised and costed  strategy

factors (e.g. funding) requirements and  action plans • Confirmation and

capabilities • aUgndilitey rstandof curreingnt the  • Identification of  allocation of internal

• eEnvxteironmernal ntal factors infrastructure and  critical success factors  resources

• Organisational and  • aRessseourcssmeingnt  applications by which the strategy  • Staff training and can be implemented development

business process  • Budget history  • Capability and  • Project governance transformation  capacity of internal

and implementation opportunities and external resource

disciplines

providers

• Benefits identified, tracked and realised

Source: Jersey Audit Office identified best practice

18. I reiterate the recommendation made in previous reviews that the States of Jersey should finalise the development, approval and adoption of an overarching technology strategy for Government.

Learning for future IT implementations

L1  Finalise the development, approval and adoption of a Digital Strategy for services

to citizens supported by an overarching technology strategy for the States of Jersey.

Articulating the business case

States of Jersey requirements

19. The Corporate Portfolio Management Office (CPMO) Frameworks for Major, Strategic and other projects includes a Delivery Framework. This sets out the need for business cases as follows:

Major and Strategic Projects

• Strategic Outline Case (SOC) – detailed as mandatory; and

• Outline Business Case (OBC) and Full Business Case (FBC) – detailed as conditional following consultation with the Treasury and Exchequer Department's Investment Appraisal Team.

Programmes

• SOC and OBC – detailed as mandatory; and

• FBC – detailed as conditional following consultation with the Treasury and Exchequer Department's Investment Appraisal Team.

20. The Public Finances Manual (PFM) includes the following principles relevant to business cases:

• All projects should be outcome based and be supported by a business case, in line with Treasury and Exchequer guidance. This should set out tangible deliverables and measurable benefits with a well-considered plan for the realisation of these benefits (to include the impact on Jersey Performance Framework outcome measures) which considers both funding and other resources required to deliver.

• Internal and external dependencies should be identified, assessed, and managed throughout the lifecycle of a project.

• Those responsible for delivering projects should build a clear understanding of user needs, the business case and delivery models, consider the whole supply chain (in terms of market appetite, capacity and capability). This should include a comparison of all available commercial and contractual models to deliver the project, considering the optimal value-for-money solution and the management of all risks.

• Lessons learned from other similar projects should be sought at the outset of a project and used to inform the planning and delivery of the project.

Learning from previous work

21. In practice, the business cases relating to previous IT implementation programmes and projects have not adhered to these principles consistently.

Weaknesses in evaluating interdependencies

22. The C&AG report on eGovernment published in May 2016 found that eGovernment projects were being undertaken before the Government had established its target business and operating model. It also found:

• weaknesses in the arrangements for aligning the objectives of individual projects with those for the programme as a whole, increasing the risk that projects that only loosely related to the programme's objectives were adopted and funded; and

• fundamental issues with the way the programme supported the specific needs of business transformation.

23. In 2017 the PAC review of eGovernment reported a lack of a single strategy document serving as a focal point of reference for eGovernment stakeholders. Various vision' documents were provided but nothing that enabled a clear and common understanding of the core purpose of the programme.
24. My report ICT Cloud Implementation – Integrated Technology Solution (October 2021) found that neither the OBC nor the FBC articulated clearly the link between the ITS programme and other active programmes aimed at modernising Government services. The approach to managing interdependencies between different programmes was not therefore clear from these key documents.
25. My report Cyber Security Arrangements (May 2022) recommended that the OBC should document linkages to wider organisational strategies and initiatives.
26. In February 2024 my audit of Electronic Patient Record reported there was a clear Governance Framework that supported a Digital Health and Care Implementation Plan but the November 2020 Business Case focussed on the health-related outcomes of the programme and did not take a wider pan-Government view.

Inconsistencies in benefits and cost estimates

27. My report ICT Cloud Implementation – Integrated Technology Solution (October 2021) found that the OBC included an articulation of anticipated benefits and costs of the programme as well as the case for change. However while the case for change was consistent between the OBC and the FBC, the estimates of costs and benefits varied: the FBC showed the estimated costs increasing and the estimated financial benefits reducing.

28. I consider costs and financial controls and benefits realisation in more detail in later sections of this Thinkpiece.

Risks associated with the learning identified

29. Risks remain that programmes and projects are being implemented in a sub - optimal way due to interdependencies not being identified, assessed, and managed.
30. Risks also remain that the full costs and benefits of IT implementations are not being identified and evaluated at the business case stage.

Learning for future IT implementations

L2  Ensure that pan Government interdependencies are documented fully at the OBC

and FBC stages.

Effective procurement

States of Jersey requirements

31. The PFM includes the following principle relevant to procurement of major and strategic projects:

• Project commercial management: Those responsible for delivering projects should build a clear understanding of user needs, the business case and delivery models, consider the whole supply chain (in terms of market appetite, capacity and capability). This should include a comparison of all available commercial and contractual models to deliver the project, considering the optimal value-for-money solution and the management of all risks.

32. The Government's Procurement Best Practice & Procedures: User Guide & Toolkit includes that:  

To achieve the optimum value procurement should be a business partner and change agent and not purely a contract maker.'

and

If you are seeking to purchase any IT-related equipment or services then you should contact the Category Manager for Information Services / Modernisation and Digital before commencing any Procurement.'

33. It also sets out the need to consider Make vs. Buy' - the strategic decision whether to deliver a requirement through in-house (internal) delivery, or through an agreement with an external partner, or a mixture of both.
34. The CPMO Frameworks set out that for both Major and Strategic Projects and for Programmes, the need for a Procurement Strategy is conditional on the outcome of consultation with Commercial Services.

Learning from previous work

35. My work has confirmed that procurement strategies have been documented for the projects and programmes I have audited. However both my work and internal lessons learned' exercises undertaken by Government have identified some weaknesses in the strategies documented and applied in practice.

Revenue Transformation Programme

36. The internal lessons learned' review undertaken by Government on the Revenue Transformation Programme identified the following key learning in respect of procurement:

• there is a need to ensure clear delineation between any software and services elements of the solution

• the maturity of requirements needs to be considered carefully when identifying the appropriate commercial mechanism; and

• a considerable level of detail is required in statement of works for fixed price contracts.

Integrated Technology Solution

37. Procurement strategies were developed for different stages of the programme. A number of decisions taken within the procurement strategies had a significant impact on programme delivery and subsequent programme costs. These decisions were not however revisited and reassessed as the programme progressed. In particular, two decisions made at the procurement strategy stage had a significant impact on delivery and cost and were not revisited or reassessed as the programme progressed. These were as follows:

• the decision to remove payroll from the scope of the programme has resulted in significant costs associated with reconciling payroll data to Connect People. The benefits envisaged of having a single source of data on headcount has not been realised as a consequence; and

• despite a phased approach to the ITS programme as a whole, the decision to adopt a big bang' approach to the Go Live' for Connect Finance - where changes were implemented across a wide range of functional areas at the same time - was taken as part of the procurement stage of the project, after discussion of the relevant benefits and risks of various implementation strategies with the prospective delivery partners. However the risks associated with the continued adoption of a big bang' approach to implementation for the whole of the States of Jersey rather than adopting a phased implementation by functional area or department were not continually assessed and considered as the programme progressed. While a big bang' approach simplifies data migration to the new systems, a phased implementation would have allowed teething problems' to be identified and resolved without affecting the whole of the States of Jersey.

Electronic Patient Record

38. My audit of the EPR programme identified learning in respect of:

• documentation of the initial Procurement Strategy

• the procurement approach applied being different to the documented Procurement Strategy; and

• weaknesses in the way in which procedures were applied in practice.

Initial Procurement Strategy

39. The initial EPR Procurement Strategy did not consider or document:

• the rationale for the packaging of the programme into three separate procurements

• a full examination of how the procurement could be packaged to minimise risk to the Government, meaning that the options for how risk could best be transferred to suppliers were not articulated. If best practice had been followed there would have been a formal options appraisal which examined the risks and benefits of the potential packaging options; and

• the analysis of the procurement approach proposed instead of having a prime contractor for end-to-end programme delivery. This was a major gap in the risk analysis.

Procurement approach applied

40. In practice, the procurement approach adopted differed to the Procurement Strategy in respect of: EPR implementation; and transformation support and data migration.
41. In July 2021 there was a reset of the project. This reset included the decisions to halt two of the initial procurements and appoint suppliers directly under procurement exemptions. If there had been a better original understanding of the market for these services during the initial procurement earlier in 2021, this reset and the associated procurement exemptions and breaches would not have been required.
42. Negotiations on contractual terms had initially been led by an external consultancy firm. In July 2021 however Government officers identified significant risks to the Government with the contracts that had been proposed. The decision was made to seek to reduce the contractual risks to Government through further contractual negotiations that took place between July and October 2021. The final contract

that was signed did not reflect the terms and conditions notified to potential bidders at the procurement stage. While the final terms and conditions were more favourable for the Government, it is not best practice to allow post award contract negotiations to take place.

Weaknesses in the approach applied

43. I identified further weaknesses in the approach applied in practice in respect of:

• over-optimism as to the amount of work that could be undertaken internally resulting in additional costs incurred under procurement breaches and exemptions. There is an inherent risk that the use of procurement exemptions and breaches could stifle competition for the provision of services to Government and result in a risk of poor value for money

• weak clinical engagement during the procurement process that may have contributed to a lack of ownership of the preferred solution among some clinicians; and

• weaknesses in the Pre-Qualification Questionnaire (PQQ) stage of the process which assesses the technical and economic strengths of each bidder. In my view the PQQ assessment was insufficiently detailed and key questions as to supplier viability were not considered fully. At the contract award stage, further questions were asked of the successful supplier, which were in effect post - contract negotiations. This approach is not consistent with best-practice procurement processes.

Risks associated with the learning identified

44. Risks remain that programmes and projects are being procured in a sub-optimal way that may not result in value for money being achieved.

Learning for future IT implementations

L3  Ensure that procurement strategies and decisions are revisited where appropriate

during implementation if doing so could result in better value for money being achieved in practice.

L4  Ensure procurement strategies document the options for packaging the

procurement in ways that lower the level of risk to the Government and detail the likely costs of each option under consideration.

L5  Review procurement processes to ensure that all potential suppliers have the

opportunity to submit bids which can be evaluated equally, that terms and conditions of the contract are defined at the appropriate stage and that post award negotiations are avoided.

L6  Ensure that sufficient challenge mechanisms are in place when decisions are

proposed that change the initial procurement approach and result in more work to be undertaken using internal resources.

L7  Ensure that procurement evaluation processes are applied equally and

consistently to all bidders.

L8  Ensure that terms and conditions of the contract are defined at the appropriate

stage and that post award negotiations are avoided.

Stakeholder engagement

States of Jersey requirements

45. The PFM includes the following principle in respect of stakeholder engagement for Major and Strategic projects:

Stakeholder Engagement and Communication: Appropriate stakeholder engagement, consultation and communication should take place throughout the development of a project to ensure that it meets the end users' needs.

46. The Government's Frameworks for Major, Strategic and other projects sets out that:

• the Sponsoring Senior Responsible Officer:

• engages key stakeholders
• agrees appropriate governance structures and reporting protocols; and
• provides regular steer, support, and guidance in ensuring that Senior stakeholders are kept abreast of project progress; and

• the Supplying Senior Responsible Officer:

• manages contracts and relationships with external suppliers (as agreed with the Sponsoring Senior Responsible Officer)
• acts as the lead liaison on behalf of the supplier(s); and
• verifies the quality of the products delivered by suppliers.

47. For a Major and Strategic or Key/Local' Project to be considered compliant with the Government's Delivery Framework, requirements include a Project Brief and later a Project Initiation Document (PID). The PID should detail the Project's:

• organisational governance arrangement; and

• an initial Communications Strategy identifying stakeholders, to be followed by a more comprehensive Stakeholder Analysis Map.

48. For a Programme to be considered compliant with the Government's Delivery Framework, the following are needed:

• Programme Communication Plan

• Programme Responsible, Accountable, Consulted and Informed (RACI) matrix; and

• Programme Stakeholder Engagement Plan.

49. There is currently no note within the CPMO's Frameworks for Major, Strategic and other projects documentation about the specific needs of IT implementation projects and so the role of Digital Services is not set out. While I understand the Government approved an IT Project Delivery Framework in November 2022 and that this is available on the CPMO hub, it is not yet integrated as a requirement in the PFM or in the CPMO's Frameworks for Major, Strategic and other projects.
50. The Government's Digital Services (DS) delivers technology, change management and information services across the Government of Jersey. The function provides the expertise to enable the future digital strategy of the organisation' as well as interaction with the end customer'. As part of this, DS is responsible for the provision of a Design Authority' function, to ensure systems link together, that common capabilities are identified and that initiatives are aligned to common standards.
51. However, while there are limited requirements set out in the Public Finances (Jersey) Law 2019 and the supporting PFM, including that:

any business project with a technology element must consult with the Government of Jersey Design Authority to seek direction and approval'

in practice it remains the case that the DS is not involved early enough when considering IT implementation or when considering the IT aspects of other major projects, such as building or refurbishment projects.

52. While the prioritisation of IT projects for inclusion in the Budget 2025 to 2028 has been led by DS, there remains a need to ensure that there is adequate engagement of DS on all projects with IT implications.

Learning from previous work

53. Developing, keeping up to date and implementing a communications plan, so that all key stakeholders are properly engaged, can be crucial to delivery of IT projects. Good practice in this area includes:

• thinking widely about who the stakeholders are, not overlooking those that will be key to acceptance of change (some might be informal' influencers)

• involving stakeholders early enough in the process to enable them to have an influence and so be more likely to have a personal attachment to the projects success

• being open about the project's benefits and risks, to set expectations; and

• avoiding lip service' consultation and involvement.

eGovernment

54. In 2016 my predecessor's report emphasised the importance of securing buy-in' to the success of the eGovernment programme and that establishing strong links with stakeholders was a priority.
55. A communications plan had not been established at the beginning of the work and the project risk register identified that that more needed to be done to counter disagreement about the underlying approach to eGovernment and the lack of a common understanding across senior management.
56. The PAC's 2017 review of eGovernment made two recommendations covering internal roles and relationships. It stated that the Chief Executive should:

• use his authority to drive the necessary changes through the CMB [Corporate Management Board, now the Executive Leadership Team] and [ ] each Chief Officer should have eGovernment objectives in their performance appraisals; and

• ensure that the core vision, principles, values and skills are understood and embodied by staff and encourage and support those staff to work across departmental boundaries.

Cyber Security Arrangements

57. My 2022 review found that the Government's Cyber Security Programme had implemented communication pathways with business units across Government. However, it also reported that the prioritisation of transformation programmes and projects and of the interdependencies between concurrent transformation programmes and projects could be further improved.
58. My 2022 report also identified one of the factors that impacted on delivery of tranche one of the programme was the inexperience of some key stakeholders.
59. I recommended that:

• workshops are held routinely with programme stakeholders to identify and prioritise requirements for major programmes; and

• structured briefings are held for stakeholders at the commencement of their involvement in a programme so that they have a clear understanding of their role.

Integrated Technology Solution

60. My report ICT Implementation – Integrated Technology Solution (October 2021) identified that a Communication Plan and strategies were prepared that identified all key stakeholders and the arrangement for managing communications with them. I identified however that the needs and consideration of third parties, such as the States Assembly, had not been identified at a detailed level. I recommended that the Communication Plan for the ITS programme documented more fully the communication needs of the States Assembly and Scrutiny Panels and how these needs would be met.
61. In my 2023 follow up I found elements of good practice were in place in terms of working with departmental staff and other stakeholders. An example of this was the user experience venue in a dedicated facility which allowed end-users to be walked through the new systems and business processes.
62. However, my review also found that the full implications of releasing major functionality, in particular in relation to the impact on the end user community, should have been more fully assessed. I noted that in practice, several problems were experienced following Go Live'.
63. Feedback from the October 2021 Government survey of lessons learned' by staff and contractors engaged in the ITS Releases One and Two included that:

• relationships with procurement were still work in progress'

• business and user needs were not always being listened to, and the programme was not visible enough at a level where people know how things are done' – there were senior level briefings but not enough to operational staff; and

• communications was sometimes tell' rather than engage'.

Electronic Patient Record

64. In my 2024 report I noted that the level of engagement with key stakeholders had varied over the lifetime of the EPR programme from January 2020 to May 2023. Since November 2020 there had been two changes in Chief Clinical Information Officer as well as changes in the leadership team of the (then) Health and Community Services department. To have this level of senior change in a complex and complicated long running programme may well have had a negative impact on the wider engagement with key stakeholders, particularly clinical stakeholders.

65. At the time of my review of the EPR project (February 2024) the project risk register highlighted that too high a proportion of the engagement between clinicians and managers was happening retrospectively. Overall, the levels of engagement from senior clinicians and managers fell below what was expected and hoped for. These issues increased the risk to delivery of this ambitious programme.
66. I concluded that in order to drive value for money from significant investments, there needs to be a much greater focus on effective stakeholder and user engagement

Automatic Electoral Registration

67. The project closure report for the aborted automatic electoral registration project (October 2021) identified project communications as one of the key issues with the project.
68. The report identified that there appeared to have been a degree of miscommunication within the project between the Project Manager. Project Team and Project Board, which may have contributed to the project's status being perceived by the Project Board as being in a healthier state than it was in reality.
69. Three key areas of learning were identified in respect of communications:

• Any informal discussions and/or project team meetings should be followed-up with an email message confirming what was understood and agreed from the discussion/meetings. This would provide an opportunity for clarifications and subsequently ensure that all parties had the same understanding of the information shared during the discussion and any agreed actions.

• The Stakeholder Analysis and Communications Plan should be used to enable regular communications to all stakeholders to be embedded as standard practice across the project. In particular - as a minimum - the Project Manager should meet regularly with the Project Team as a whole, in addition to Project Board meetings.

• There should be regular sense checks' across the project regarding everyone's understanding of the project's status, especially following separate communications between individual members of the project team. In particular, it should be clear from meetings what are suggested options as opposed to formally agreed delivery approaches.

Revenue Jersey implementation

70. One of the success factors identified by Revenue Jersey was the adoption of an approach of investing significant time in exploring issues and challenges with all stakeholders involved in decision-making, to identify and address concerns in preparation for recommendations to the Board. The team identified that this not only ensured thorough exploration and evaluation of options and identified potential un-intended impacts but ensured timely actions and decisions at Board level.

71. The team identified however that there was a drop off of regular attendance from senior representatives at project Board meetings. The team reflected that consistent input of other departments' perspectives, knowledge and insight, while it may pose more challenges in reaching agreement, increases the quality of decisions made and helps manage interdependencies.

Risk associated with the learning identified

72. Risks remain that IT implementations will take longer than planned and will cost more than planned due to ineffective engagement and lack of buy in' from key relevant stakeholders.

Learning for future IT implementations

L9  Ensure advice or input is sought from Digital Services when considering IT

implementation or when considering the IT aspects of other major projects such as building projects.

L10  Ensure that sufficient time is spent to enhance the skills of key stakeholders

through activities such as:

• workshops with programme stakeholders to identify and prioritise requirements for major programmes; and

• structured briefings at the commencement of stakeholders' involvement in a programme so that they have a clear understanding of their role.

L11  Ensure that communication plans include the communication needs of the States

Assembly and Scrutiny Panels and how these needs will be met.

L12  Consider the use of a user experience venue in a dedicated facility to allow end -

users to be walked through new systems and business processes.

L13  Ensure the full implications of releasing major functionality, in particular in relation

to the impact on the end user community, are fully assessed prior to Go Live'.

L14  Ensure the project and programme have sufficient visibility at an operational level

as well as a senior level during all phases.

L15  Ensure business and user needs are listened to at an operational level as well as a

senior level and, in doing so, ensure communications are focussed on engagement' rather than tell'.

L16  Ensure that there is sufficient and effective proactive engagement with senior end

users. Where senior end users are not sufficiently engaged, consider proactively the implications for the programme and project delivery decisions.

L17  Build in regular sense checks' across a project regarding the understanding of all

key stakeholders of the project's status.

L18  Ensure that informal project meetings are followed up with formal documentation

to ensure consistency of understanding of decisions made and actions to be taken.

Project and programme management

73. Previous audits and reviews have identified particular learnings in respect of:

• setting realistic delivery ambitions

• understanding and managing programme risk and uncertainty; and  

• assessing data issues in a planned and incremental way.

Risks associated with the learning identified

74. There remain risks that future IT implementations will fail to deliver on time and to budget and will have more post Go Live' issues to resolve if the areas of learning outlined in this section are not addressed.

Learning on setting realistic delivery ambitions from previous work

75. Successful IT implementations are underpinned by setting realistic delivery ambitions that take into account the required investment in technology, people and skills.
76. This includes the need for realistic timetables to be set that are not unduly driven by near-term contract expiry dates.
77. In setting realistic delivery ambitions that underpin programme timescales and management there is a need to work on business analysis, new data needs and legacy dependencies in a timely manner. This reduces the risk that complexities are only exposed after contracts have been signed and delivery begins.

eGovernment

78. The 2016 C&AG audit found a lack of a roadmap for achieving objectives, including a lack of planned timescales for delivery of prioritised action. It also highlighted the need to establish effective arrangements to identify, assess, scale and manage threats to delivery, which among other things would help with planning realistic project timescales.
79. The 2017 PAC review of eGovernment reported a lack of meaningful and measurable outcomes backed by tangible project plans with realistic timescales and clear budgeting.

Cyber Security Arrangements

80. My report on Cyber Security Arrangements (May 2022) found that the implementation of Government's Cyber Security Programme was delayed and hampered by shortages in internal resources, the inexperience of some key stakeholders and weaknesses in management of interfaces and dependencies.
81. It was also difficult to see how changes to programme targets, including deferrals, had been risk assessed, agreed and communicated.

Integrated Technology Solution

82. My 2021 report ICT Cloud Implementation – Integrated Technology Solution included a recommendation to reassess the level of internal resources dedicated to the ITS programme and ensure that the programme timescales set were realistic in light of the resources actually available.
83. Feedback from the October 2021 Government survey of lessons learned' by staff and contractors engaged in the ITS Releases One and Two identified a lack of:

• simple, measurable outcomes to understand progress

• clarity on what's happening and why'

• co-ordination and centralisation – for example a number of offline spreadsheets were created; and

• clear and consistent lines of accountability.

84. Feedback by staff and contractors also noted that more time was needed for discovery and design phases.
85. At the time of my 2022 follow up fieldwork, Release One (Connect Finance, Connect Inventory and Connect Suppliers) and Release Two (Connect People) of the ITS programme were planned to Go Live' at the beginning of January 2023. This was already nine months later than had been forecast in the original programme plan.

Electronic Patient Record

86. My report on EPR (February 2024) identified the need to undertake a high-level stock take of all major digital change programmes planned over a four year period and map out these programmes against the capacity and capability of the teams within Government to support these changes.

Other IT implementation

87. A report presented to the States of Jersey in October 2021 by the Privileges and Procedures Committee (r.164-2021.pdf (gov.je)) set out issues with the planned introduction of Automatic Electoral Registration. A proof of concept seemed to indicate that this could be achieved in time for the 2022 elections. However, delays in establishing sound project management at the start of the work resulted in the recognition that the project had unrealistic ambitions, timeframe and budget.

Learning for future IT implementations

L19  For major programmes, adopt a set of success measures that can be used to

evaluate the impact of a programme in a clear and straightforward way.

L20  For major programmes, set overall milestones for delivery at programme level and

monitor against those milestones.

L21  Ensure that effective project management arrangements are established at the

start of each project.

L22  Before the start of each project ensure that there is sufficient capacity and

capability within all relevant departments to support the changes being proposed.

Learning on understanding and managing programme uncertainty and risk from previous work

Managing strategic risks

88. My previous audits have found good Governance Frameworks in place to support technology implementations. I have also found that Project and Programme Boards have considered the issues and risks that I would expect.
89. However, my audits have also found that while project level risk management often works effectively at a detailed level, the key strategic risks in relation to programmes are not always evident and, as a consequence, have not always been managed effectively.
90. As noted earlier, the Government chose to adopt a big bang' approach when implementing the Connect Finance modules of ITS. While such an approach simplifies data migration to the new systems, a phased implementation would have allowed teething problems' to be identified and resolved without affecting the whole of the States of Jersey.

91. The decision to adopt a big bang' approach was taken as part of the procurement stage of the project, after discussion of the relevant benefits and risks of various implementation strategies with the prospective delivery partners. However I would have expected the risks associated with this approach to have continued to have been considered as the programme progressed.
92. In my 2022 follow up report I recommended that the Government should ensure that the key strategic risks associated with ITS programmes were identified, recorded and managed. Doing this well would support the ability to plan realistic timescales for delivery.
93. The Electronic Patient Record system was implemented as part of the Digital Care Strategy. The November 2020 Business Case demonstrated a sound understanding of the need for the programme and it set out the high-level risks in terms of their impact and probability. The highest identified risk related to the need to complete the clinical transformation – the sometimes radical re - engineering or redesign of processes - required to deliver the expected benefits. The manifestation of this risk became the key issue to the delivery of benefits and the effective operational use of the EPR system.

Risks associated with the Go Live' decision

94. My audit of the ITS programme implementation identified that a Business Readiness Checklist, used to inform the Go Live' decisions, did not focus sufficiently on specific business areas such as health and police, but instead presented a global view of functionality (for example finance and HR) as a whole. The meant that some of the nuances of differences in how specific business areas operate was not assessed adequately. I recommended that these Checklists for future releases include a service-related dimension which provides evidence that each significant service area is satisfied that the functionality has been tested and signed off as meeting business needs.
95. In addition, a key imperative of the ITS programme was to minimise bespoke configuration of the applications which had critical implications for the scope and complexity of the testing and user acceptance processes. There was though a need to fully understand and take into account the nuances of differences in how specific business areas operate. I found that this was not assessed adequately.
96. My audit of the EPR noted that the Go Live' date of Releases One and Two of the programme was put back on two occasions. The reasons for the second delay included specific programme risks that had not been identified and managed effectively.

97. In particular, the new system worked very differently to the old system with the consequence that the user experience of the two systems is very different. While all of the business (clinical) process mapping was approved by Clinical Directors it is not clear how well the implications of the new processes were understood, particularly given the variable engagement from senior clinicians in the programme.

Learning for future IT implementations

L23  Ensure that key strategic risks are identified, documented and managed effectively

at all stages of a project or programme.

L24  Undertake a formal documented risk assessment before agreeing deferrals or

changes to project deliverables.

L25  Deliver structured training to risk owners to confirm their understanding of and

confidence in their role.

L26  Ensure that Go Live' decisions include specific documentation and consideration

of strategic and operational programme and project risks.

L27  Ensure the scope and complexity of the testing and user acceptance processes are

sufficient to understand and take into account the nuances of differences in how specific business areas operate.

Learning on data issues from previous work

eGovernment

98. The C&AG review reported that in January 2016 the eGovernment Programme Board identified a risk that project delivery was compromised due to poor quality departmental data. However, the specific actions to address poor quality departmental data and ensure compliance with a data quality policy were not yet clearly set out. The report recommended that the Government expedite the development and implementation of policies and procedures for data quality and data sharing alongside arrangements to secure compliance.

Integrated Technology Solution

99. My 2022 review identified issues with data migration costs and timeframe which pointed to a lack of routine data maintenance activity at department level.

Electronic Patient Record

100.  During data migration to the new EPR system, it was found there were over

400,000 open patient consultations in the system (for a population of c. 105,000 islanders.) The reasons for delays in the Go Live' dates for the EPR programme included delays in obtaining access to TrakCare (the old' patient information system) data for data migration.

101.  Three months after the EPR Go Live' date, key risks on the programme risk register

that required urgent attention included two relating to the success of data transfer during implementation of new IT:

• the risk that patient records become fragmented as clinicians are updating records on paper and digitally which results in inconsistency; and

• the risk that legacy records, which need to have the planned date on waiting list, had not been corrected in the system (with approximately 1,000 records to be corrected).

Learning for future IT implementations

L28  Document a realistic strategy for data quality and data transfer at the outset of a

project and monitor throughout the project.

Cost and financial controls

Learning from previous work

102.  Capital spending is money that is spent on investment and things that will create

growth in the future. Resource (sometimes revenue) funding includes day-to-day operations of public services, including administrative costs for departments and programme spending (but not on tangible things').

103.  Audits of Government IT implementations have consistently identified learning in

respect of cost and financial controls.

eGovernment

104.  The budget for eGovernment was included in the capital programme. However

the C&AG's 2016 review reported there was no rationale for the budget in the context of the (then) current reformulation of the programme. The picture of cost and spend was further confused as some projects were part or wholly funded by departmental budgets. There were no clear criteria for the funding decisions the Senior Responsible Officer made and reasons were not documented.

105.  The review also found that future revenue costs had not been agreed or set

against planned financial benefits to be realised. The budget for cyber security had not been determined. Recommendations from the 2016 report included validating capital and future revenue budget for eGovernment, to include cyber security and training and to enhance arrangements for making rigorous and transparent decisions on project and workstream funding.

Integrated Technology Solution

106.  In my 2022 review, I reported that the Government Plan 2020-23 included a

dedicated allocation of £28 million for the capital costs of the ITS programme but that the ongoing revenue costs of the programme were not budgeted for separately. Instead they formed part of an allocation for the revenue impact of IT investment.

107.  I also found that the OBC did not include contingency or risks provisions – this was

only added as part of the FBC. I recommended that the Government provide clearer guidance and templates to capture all expected costs of major projects at the OBC stage.

108.  The 2023 follow up found that, although the OBC template asks for revenue and

capital costs to be identified separately, it did not provide a pro-forma for these two categories of cost to be separated.

109.  More generally, whereas funding of £28 million for the ITS programme was

provided in the Government Plan 2020-23 (in line with the OBC) the assumptions underpinning the OBC estimate of expenditure were overly optimistic. Consequently, the OBC excluded significant costs of the programme that should have been better understood and quantified at the time that the OBC was approved. The fact that these costs were not identified and quantified until the FBC meant that they were not included in the Government Plan 2021-24.

110.  The final approved expenditure for ITS is £63.6 million. This is £35.6 million more

that the OBC. In addition, a number of modules originally included in the OBC and FBC are not now being implemented as part of the programme.

Electronic Patient Record

111.  The EPR programme had a long implementation period from October 2021 to the

third quarter of 2024 (three years). During that time the EPR capital funding had not been visible within the EPR programme budget because it is agreed annually and drawn from a wider (then) Health and Community Services Digital Transformation Programme. This made it hard to track whether the EPR programme was overspent. The EPR programme revenue funding was even less visible as this was embedded within large departmental budgets. In addition, the EPR programme budget did not include expenditure on both the existing TrakCare system and the new EPR, as had been set out in the original Business Case.

Risks associated with the learning identified

112.  There is a risk that projects and programmes will face cost pressures if the full costs

of implementation are not identified at the outset. The subsequent impact of cost pressures and overspends may result in projects failing to implement all phases or modules and/or failing to deliver identified benefits.

Learning for future IT implementations

L29  Ensure all expected costs of major projects are captured at the OBC stage and at

the FBC stage. Consider using a challenge process to provide assurance as to the completeness and accuracy of the costs captured.

L30  Align cost and delivery reporting to programme and project boards as indicators

to assess overall programme performance and to highlight risks.

L31  Ensure change request reporting includes clear cost implications associated with

each individual change request.

L32  Produce an ongoing full cost summary for all long running Major and Strategic

programmes, particularly those funded through wider Government or Departmental programmes or where funding is allocated to multiple Government Departments. This summary should be reconciled annually, to ensure whole life programme cost control is visible.

Capacity and management of contractors

113.  For any particular IT implementation and for overall IT implementation ambitions,

the Government should be clear about what capacity and skills it needs, has in house', wants to develop or will contract out.

114.  The capacity and capability needed for IT implementation are not just about

technical skills. There is a need to assess the talent and skills management profile in order to help evaluate and potentially mitigate capacity risks.

115.  In addition, it is important to keep in mind that for projects and programmes of IT

implementation:

• organisations need to plan to manage business as usual' at the same time as managing change; and

• capability and skill requirements change over time and need to be forecast.

Learning on capacity from previous work

eGovernment

116.  The 2016 report noted that although the eGovernment programme had identified

a need to use contractors to augment internal capacity to upskill and develop internal resources, it did not have a plan in place to address the identified need. The report recommended that the Government develop and implement a people and skills plan for eGovernment, including flexible mechanisms for securing skills both within and outside the States and covering the transfer of skills from contractors.

Cyber Security Arrangements

117.  The need to ensure that adequate internal resources are available for IT

implementation was highlighted in my May 2022 Cyber Security Arrangements review.

Integrated Technology Solution

118.  Throughout the ITS programme there have been challenges to successful delivery.

These challenges have been in part related to the capacity of the Government to engage fully with the programme and to make key decisions on a timely basis. There has been a significant reliance on third party contractors to perform key programme roles.

Electronic Patient Record

119.  My review found that pressure on the (then) Modernisation and Digital (now DS)

team meant that additional, unplanned external resources were needed to support the ongoing implementations.

120.  Whereas the FBC identified the need for significant internal resources, for EPR a

decision was taken during the procurement phase that, due to cost, the implementation work would not be outsourced but would instead be undertaken using internal resources.  This placed an increased burden on the internal IT team and significant pressure on Health and Community Services staff.

121.  A specialist external company was to be asked, using an existing draw-down

arrangement with that supplier, to undertake an EPR Readiness Assessment' to determine how the implementation and transformation package could be delivered internally. As the programme progressed, an external supplier was appointed to support the internal team in respect of testing. This procurement was later subject to the declaration of a retrospective procurement breach with a stated value of £180,000.

122.  I also reported that the development of the November 2020 Business Case, which

placed significant reliance on external expertise, increased the risks associated with lack of continuity of staffing and lack of knowledge transfer.

123.  In putting the programme into action, my review found that, late in the process, a

significant number of clinical leads (mainly nursing staff) were seconded to the EPR programme for a period of 18 months to assist in the mapping of business processes. While these staff understood clinical processes they did not have previous specific skills and experience in business process mapping. Additionally, the internal team did not have sufficient experience of testing and ultimately an external supplier needed to be appointed.

Risks associated with learning identified

124.  There remain risks that future IT implementations will fail to deliver on time and to

budget. There also remain risks that Government will not benefit from effective skills transfers from external contractors.

Learning for future IT implementations

L33  At the outset and throughout the programme continually assess the level of

internal resources required to be dedicated to the programme and ensure that the programme timescales set are realistic in light of the resources actually available.

L34  Make best use of scarce internal staff resources in future technology programmes

through confirming availability during the planning phase of a programme and ensuring there is engagement across programme leads to identify activities in common.

L35  Designate internal owners for each workstream in major programmes.

Learning from previous work on engaging and working with commercial partners, including due diligence

125.  The States have a finite amount of specialist commercial services resource

available and so not all projects have a Commercial Manager in place to provide expert advice and to act as Contract Manager. My November 2023 review Major and Strategic Projects, including Capital Projects found there was more to do to ensure that these specialist resources are effective in adding value to key projects.

126.  The National Audit Office's 2023 lessons learnt' report The challenges in

implementing digital change notes the importance of:

• spending enough time and money exploring requirements with commercial partners at an early stage

• adopting a more flexible contracting process that recognises scope and requirements may change

• costing out an exit strategy to change providers if necessary; and

• working towards a partnership model based on collaboration with commercial suppliers

127.  Digital transformation in government: A guide for senior leaders and audit and risk

committees (NAO 2024) notes the things to get right include:

having exploratory discussions with commercial partners before settling on a solution, in order to help the organisation to de-risk the programme and determine what is possible.'

128.  There is though currently no mention of this in the Government's Procurement Best

Practice & Procedures: User Guide & Toolkit.

Integrated Technology Solution

129.  My 2022 review of ICT Cloud Implementation – Integrated Technology Solution

reported that during the procurement phase for the delivery partner and software provider, there was extensive interaction with bidders. Procurement dialogue was held as a series of sessions with the bidder consortium members to increase understanding of bidders' detailed proposals, to mature the scope and to provide constructive feedback to the bidders.

130.  I recommended however that the Government should report formally the

performance of third-party partners against agreed Key Performance Indicators (KPIs) to the ITS Partnership Board. In following up in 2023, I found that the Strategic Oversight Board received a monthly review of supplier performance and I concluded that this complies with expected good practice.

Electronic Patient Record

131.  Negotiations on contractual terms with the preferred commercial partner for EPR

were led initially by an external consultancy firm. A newly appointed Chief Clinical Information Officer however identified significant risks to the Government with the contracts that had been proposed. The decision was made to seek to reduce the contractual risks to Government through further contractual negotiations that took place between July and October 2021.

132.  The final contract that was signed did not reflect the terms and conditions notified

to potential bidders at the procurement stage. While the final terms and conditions were more favourable for the Government, it is not best practice to allow post award contract negotiations to take place.

Learning from non IT C&AG reviews

133.  In January 2021, my Anti-Corruption Arrangements Report included a focus on due

diligence arrangements as a critical part of an organisation's response to the risk of contracting with corrupt suppliers.  I concluded that although the due diligence arrangements are set out in the Government's Procurement Best Practice & Procedures: User Guide & Toolkit, there remained a risk of inconsistency in the procedures used in practice

134.  While work is now underway to provide updated policies and procedures in this

area, consistent, high quality due diligence practice is yet to be established. Delays in implementing actions to improve processes and tighten controls mean that the States continue to be exposed to risk.

135.  In March 2024 I published Use of Consultants – follow up, which reported on the

use of external resources, including in IT implementation. I found there were framework agreements in place with external providers of consultancy services, including in respect digital services, but that more could be done to reduce reliance on these by accelerating work to compile departmental workforce and training plans. In particular, I noted that training plans should take account of identified skills shortages, particularly in connection with project management and digital.

136.  This report also highlighted the risk that the value from some pieces of work

delivered through commercial partnerships might be lost without a more rigorous approach post commission' review, for example where a skills transfer has been a feature of the contract.

Risks associated with learning identified

137.  There are risks that contracts and relationships with commercial partners do not

secure value for money for the Government.

Learning for future IT implementations

L36  Report formally the performance of third party partners against agreed Key

Performance Indicators to the Programme Board.

L37  Document lessons learned from the operation of commercial partnerships for IT

projects and use these to update Government guidance and relevant frameworks.

Tracking and realising benefits

138.  Ensuring that the benefits of an IT implementation project are delivered requires

that:  

• benefits identification and monitoring are built in from the start of a project

• barriers to benefits realisation are understood and addressed; and

• benefits realisation tracking and evaluation continue over an appropriate timeframe to capture evidence and learning.

Learning from previous work

eGovernment

139.  A key weakness of the eGovernment programme was that the wider organisational

and cultural change objectives and benefits were not always identified or identifiable in individual projects. This increased the risk that projects were delivered successfully' but failed to secure – or it could not be demonstrated that they had secured – strategic benefits.

140.  The eGovernment team identified, in a survey issued as part of the C&AG review,

that failure to clearly set out attributable benefits created a perception that the programme was all costs'.

Cyber Security Arrangements

141.  In 2022 I reported that tracking of benefits of the Cyber Security Programme at the

corporate level was limited, making it difficult to assess the impact of the programme on the Government's overall cyber security. I recommended that major programmes routinely evaluate benefits realised and delivery of OBC tasks at programme level, to make it easier for external and senior stakeholders to understand the improvements that are being delivered.

Integrated Technology Solution

142.  My 2022 review found that the initial approach to benefits management adopted

by the ITS programme meant that there was an insufficient central focus on the quantification and realisation of expected benefits. The approach adopted did not

enable the measurement and quantification of benefits at a whole of Government level.

143.  My follow up reported that there was an ITS benefits register which had identified

92 benefits, along with their categorisation (including financial, non-financial and risk reduction) and a nominated benefits owner. However, the Government had de-coupled longer-term benefits realisation from the ITS programme. The formal arrangement for the oversight and governance of the realisation of programme level benefits beyond the closure of the programme was unclear. In addition, ITS - related benefits were not planned to be tracked beyond the end of 2026. Given the long-term investment in the new applications and the slower realisation of benefits than originally anticipated, I would expect the formal monitoring and reporting of benefits to be extended to 2030 if best practice is to be demonstrated.

Electronic Patient Record

144.  The November 2020 Business Case had a clear and realistic view on what good

looks like' and how benefits would be measured. In my view, the financial benefits outlined are not overly ambitious. A benefits tracker was derived from the November 2020 Business Case but this was not being used at the time of my audit. This benefits tracker was not however being actively used at the time of my audit. I identified a risk that the programme could become stuck in short term operational delivery mode and that benefits realisation may be pushed back or even forgotten.

145.  I recommended that programme benefits are identified and tracked as an integral

part of programme delivery during the planning and delivery phases and not left until after programme closure and the move to business as usual.

Risks associated with learning identified

146.  The failure of Government to systematically track and monitor benefits realisation

on major projects has been the subject of multiple C&AG recommendations. If benefits are not systematically tracked and monitored there remains a risk that significant investments by Government may not deliver value for money.

Learning for future IT implementations

L38  Ensure that responsibility for benefits realisation is identified and allocated to

specific business units within departments.

L39  Ensure that the monitoring and communication of benefits realisation is a

centralised responsibility that endures beyond the closure of a programme to a 10-year time horizon.

L40  Produce annual reports on benefits realisation from closed programmes to allow

transparent reporting on long-term benefit realisation.

Appendix One

Approach adopted in preparing this Thinkpiece

The review included the following key elements:

• desk based research into good practice and captured learning; and

• document review

Documents reviewed include:

• Comptroller and Auditor General Reports:

• Anti-Corruption Arrangements (January 2021)
• Cyber Security Arrangements (May 2022)
• eGovernment (May 2016)
• Electronic Patient Record (February 2024)
• ICT Cloud Implementation – Integrated Technology Solution (October 2021)
• Integrated Technology Solution – Follow up (April 2023)
• Major and Strategic Projects, including Capital Projects (November 2023)
• Tackling Fraud and Error (November 2024)
• Use of Consultants – follow up (May 2024)

• Challenges in implementing digital change: A Lessons Learnt' report (National Audit Office (NAO) 2023)

• Corporate Services Scrutiny Panel report on the Government Plan 2021-24 (December 2020)

• Digital progress in local government (Audit Scotland 2021)

• Digital transformation in government: A guide for senior leaders and audit and risk committees (NAO 2024)

• Public Accounts Committee Review of eGovernment (2017)

• States of Jersey documents:

• Automatic Electoral Registration: paper for the Public Accounts Committee from the Greffier of the States (January 2022)
• Budget 2025-2028
• Corporate Portfolio Management Office: Frameworks for Major, Strategic and other projects (May 2021)
• Integrated Technology Solution Programme: Full Business Case (v2.3 February 2021)

• Lessons Learned' Survey (ITS) (October 2021)
• Government Plan 2024-2027
• Group Annual Report and Accounts 2020, 2021, 2022 and 2023
• Public Finances Manual

The fieldwork was carried out by affiliates working for the Comptroller and Auditor General, between mid-October and the end of November 2024.

Appendix Two

Framework to review programmes

Delivery Programme  variation and

Purpose Value set up management

Purpose: Is  Value:  Programme  Delivery and there a  Does the  set-up: Is the  variation strategic need  programme  programme  management: for the  provide  set up in  Are mechanisms programme  value for  accordance  in place to

and is this the  money?  with good  deliver the

right  practice  intended programme to  and are risks  outcomes and meet the  being well  respond to business  managed?  change, and is need?  the programme

progressing according to plan?

Source: National Audit Office: Framework to review programmes updated April 2021

Purpose

• need for the programme – is it clear what objective the programme is intended to achieve?  

• portfolio management and dependencies – does the programme make sense in relation to the Government's strategic priorities?

• stakeholder engagement – have the right people bought into the need for the programme?

Value

• option appraisal – does the option chosen meet the programme's objective and provide long-term value?  

• business case – does the business case demonstrate value for money over the lifetime of the programme?

• cost and schedule – has the programme built up robust estimates of cost and schedule, including all programme components?  

• benefits – does the programme: have a baseline; know what measurable change it is going to make; and actually measure it? Are benefits being achieved?

Programme set up

• governance and assurance – are there structures (internal and external) which provide strong and effective oversight, challenge and direction?

• leadership and culture – does the programme have strong leadership with the necessary authority and influence?  

• resources – has the organisation allocated the resources (staffing, skills, equipment and so on) required to deliver the programme?

• putting the programme into practice – are scope and business requirements realistic, understood, clearly articulated and capable of being put into practice?

• risk management – are key risks identified, understood and addressed?

Delivery variation and management

• delivery strategy – are there appropriate incentives for all parties to deliver (contractual, performance management or other)?

• change control – is there an effective mechanism to control programme alterations?

• responding to external change – is the programme sufficiently flexible to deal with setbacks and changes in the operating context?

• performance management – is progress being measured and assessed, including consideration that the programme is still the right thing to do?

• lessons learned – is the programme learning from experience on the current programme and previous relevant programmes?  

• transition to business as usual – does the programme have a clear plan for transfer to operations/business as usual?

Appendix Three

Summary of Learning for Future IT implementations

Setting a strategic ambition

L1  Finalise the development, approval and adoption of a Digital Strategy for services

to citizens supported by an overarching technology strategy for the States of Jersey.

Articulating the business case

L2  Ensure that pan Government interdependencies are documented fully at the OBC

and FBC stages.

Effective procurement

L3  Ensure that procurement strategies and decisions are revisited where appropriate

during implementation if doing so could result in better value for money being achieved in practice.

L4  Ensure procurement strategies document the options for packaging the

procurement in ways that lower the level of risk to the Government and detail the likely costs of each option under consideration.

L5  Review procurement processes to ensure that all potential suppliers have the

opportunity to submit bids which can be evaluated equally, that terms and conditions of the contract are defined at the appropriate stage and that post award negotiations are avoided.

L6  Ensure that sufficient challenge mechanisms are in place when decisions are

proposed that change the initial procurement approach and result in more work to be undertaken using internal resources.

L7  Ensure that procurement evaluation processes are applied equally and

consistently to all bidders.

L8  Ensure that terms and conditions of the contract are defined at the appropriate

stage and that post award negotiations are avoided.

Stakeholder engagement

L9  Ensure advice or input is sought from Digital Services when considering IT

implementation or when considering the IT aspects of other major projects such as building projects.

L10  Ensure that sufficient time is spent to enhance the skills of key stakeholders

through activities such as:

• workshops with programme stakeholders to identify and prioritise requirements for major programmes; and

• structured briefings at the commencement of stakeholders' involvement in a programme so that they have a clear understanding of their role.

L11  Ensure that communication plans include the communication needs of the States

Assembly and Scrutiny Panels and how these needs will be met.

L12  Consider the use of a user experience venue in a dedicated facility to allow end -

users to be walked through new systems and business processes.

L13  Ensure the full implications of releasing major functionality, in particular in relation

to the impact on the end user community, are fully assessed prior to Go Live'.

L14  Ensure the project and programme have sufficient visibility at an operational level

as well as a senior level during all phases.

L15  Ensure business and user needs are listened to at an operational level as well as a

senior level and, in doing so, ensure communications are focussed on engagement' rather than tell'.

L16  Ensure that there is sufficient and effective proactive engagement with senior end

users. Where senior end users are not sufficiently engaged, consider proactively the implications for the programme and project delivery decisions.

L17  Build in regular sense checks' across a project regarding the understanding of all

key stakeholders of the project's status.

L18  Ensure that informal project meetings are followed up with formal documentation to ensure consistency of understanding of decisions made and actions to be taken.

Project and programme management

Setting realistic ambitions

L19  For major programmes, adopt a set of success measures that can be used to

evaluate the impact of a programme in a clear and straightforward way.

L20  For major programmes, set overall milestones for delivery at programme level and

monitor against those milestones.

L21  Ensure that effective project management arrangements are established at the

start of each project.

L22  Before the start of each project ensure that there is sufficient capacity and

capability within all relevant departments to support the changes being proposed.

Understanding and managing programme uncertainty and risk

L23  Ensure that key strategic risks are identified, documented and managed effectively

at all stages of a project or programme.

L24  Undertake a formal documented risk assessment before agreeing deferrals or

changes to project deliverables.

L25  Deliver structured training to risk owners to confirm their understanding of and

confidence in their role.

L26  Ensure that Go Live' decisions include specific documentation and consideration

of strategic and operational programme and project risks.

L27  Ensure the scope and complexity of the testing and user acceptance processes are

sufficient to understand and take into account the nuances of differences in how specific business areas operate.

Addressing data issues in a planned and managed way

L28  Document a realistic strategy for data quality and data transfer at the outset of a

project and monitor throughout the project.

Cost and financial control

L29  Ensure all expected costs of major projects are captured at the OBC stage and at

the FBC stage. Consider using a challenge process to provide assurance as to the completeness and accuracy of the costs captured.

L30  Align cost and delivery reporting to programme and project boards as indicators

to assess overall programme performance and to highlight risks.

L31  Ensure change request reporting includes clear cost implications associated with

each individual change request.

L32  Produce an ongoing full cost summary for all long running Major and Strategic

programmes, particularly those funded through wider Government or Departmental programmes or where funding is allocated to multiple Government Departments. This summary should be reconciled annually, to ensure whole life programme cost control is visible.

Capacity and management of contractors

L33  At the outset and throughout the programme continually assess the level of

internal resources required to be dedicated to the programme and ensure that the programme timescales set are realistic in light of the resources actually available.

L34  Make best use of scarce internal staff resources in future technology programmes

through confirming availability during the planning phase of a programme and ensuring there is engagement across programme leads to identify activities in common.

L35  Designate internal owners for each workstream in major programmes.

L36  Report formally the performance of third party partners against agreed Key

Performance Indicators to the Programme Board.

L37  Document lessons learned from the operation of commercial partnerships for IT

projects and use these to update Government guidance and relevant frameworks.

Tracking and realising benefits

L38  Ensure that responsibility for benefits realisation is identified and allocated to

specific business units within departments.

L39  Ensure that the monitoring and communication of benefits realisation is a

centralised responsibility that endures beyond the closure of a programme to a 10-year time horizon.

L40  Produce annual reports on benefits realisation from closed programmes to allow

transparent reporting on long-term benefit realisation.

LYNN PAMMENT CBE Comptroller and Auditor General

Jersey Audit Office, De Carteret House, 7 Castle Street, St Helier, Jersey JE2 3BT T: +44 1534 716800 E: enquiries@jerseyauditoffice.je W: www.jerseyauditoffice.je