This content has been automatically generated from the original PDF and some formatting may have been lost. Let us know if you find any major problems.
Text in this format is not official and should not be relied upon to extract citations or propose amendments. Please see the PDF for the official version of the document.
jerseyauditoffice.je
Contents
Introduction........................................................................................................................ 3 Key findings........................................................................................................................ 4 Conclusion ......................................................................................................................... 5
Monitoring and reporting...............................................................................................10 Appendix One ..............................................................................................................................22
- The States of Jersey, independently audited States bodies and States aided independent bodies are responsible for:
• considering recommendations made by the Comptroller and Auditor General (C&AG) and auditors appointed by the C&AG
• determining whether to accept recommendations and, if so, what action to take
• monitoring implementation of agreed audit recommendations and the impact secured; and
• taking corrective action where appropriate.
- During 2019, the States of Jersey introduced a tracker system to monitor the progress being made in implementing recommendations made by the C&AG and the Public Accounts Committee (PAC). The States of Jersey tracker indicates that there is considerable work still required to implement C&AG recommendations made. Exhibit 1 shows data from the States of Jersey tracker.
Exhibit 1: C&AG Recommendations Tracker number of open' recommendations
700 600 500 400 300 200 100 0
End of 2017 End of 2019 End of 2020 End of Q3 2021 Source: States of Jersey Tracker
- This report considers the arrangements made by the States of Jersey to monitor and implement agreed recommendations.
- The key findings from my review were as follows:
• the States of Jersey are ahead of many other jurisdictions in implementing arrangements to track C&AG and PAC recommendations. Implementation of the Tracker by the Government is good practice
• the enhanced focus and challenge of the Tracker by the PAC in 2021 has improved scrutiny of recommendation status and challenged the assurance being provided over the closure process
• the number of open' (yet to be implemented) recommendations has remained roughly the same during 2021 at around 250. This is due to new recommendations being made at the same time as older recommendations are being implemented
• although senior management and the Financial Governance team in the Treasury and Exchequer Department have made good progress in promoting active ownership and update of the Tracker by departments, there remains a significant number of open' recommendations as well as a significant number of overdue recommendations (not implemented within target dates for implementation). There is a need for some focussed work to be undertaken to reduce the number of longstanding and overdue recommendations
• departments and Accountable Officers are not always able to demonstrate effective, diligent and timely implementation of recommendations. Departmental teams also need to improve the explanations provided within the Tracker when target dates for implementation are revised and where actions are delayed
• recommendation owners should be held to account where reporting is not being updated and where timely progress is not being made in implementation of agreed recommendations; and
• there is scope to introduce further analysis and enhanced reporting to support challenge as to the timely and effective implementation of recommendations.
Conclusion
- The States of Jersey are ahead of many other jurisdictions in implementing arrangements to track C&AG and PAC recommendations. Good progress has been made in identifying the recommendations that have not yet been implemented and the reasons why.
- There remains however a large number of agreed recommendations that have not been implemented, many of which are significantly overdue. There needs to be a continued focus on timely implementation with the aim of reducing the number of open' and overdue recommendations.
- Ultimately, the focus should not just be on the Tracker and associated quantitative reporting, but rather about ensuring an appropriate organisational culture that encourages, and ultimately ensures, more timely, effective and appropriate implementation of recommendations. This will help ensure improved governance, value for money and service delivery.
Objectives and scope of the review
- This review has evaluated:
• the arrangements established to manage, monitor and report on implementation of agreed C&AG and PAC recommendations through the recommendation Tracker
• the key reasons for delays in implementing agreed recommendations
• the extent to which the changes made have been evaluated to ensure they address the improvement areas identified before recommendations are shown as complete and are closed on the Tracker; and
• the adequacy of plans to monitor the implementation of open' recommendations.
- The review has considered the arrangements made by the States to monitor and implement agreed recommendations made by the C&AG. It has not considered:
• the implementation of recommendations by States controlled and States established entities; or
• the implementation by the States of Jersey of recommendations made by other auditors such as external auditors and the internal audit function.
- The review provides a commentary as to the assurance that can be taken from the arrangements for tracking recommendations. It does not however focus on or provide assurance on whether individual recommendations have been implemented.
Overall governance of the Tracker
- The States of Jersey approach to tracking recommendations began at the end of 2017 with work undertaken to better understand the range and nature of outstanding recommendations made by the C&AG and the PAC.
- During 2019, an Excel Tracker spreadsheet was populated to record recommendations and monitor implementation progress. A decision was made to include C&AG recommendations from around 2016 onwards and PAC recommendations from 2014 onwards. Earlier recommendations were not included due to their age and the resources required to review and include them. A considerable amount of resource was provided by Internal Audit to populate the Excel Tracker and to check its completeness with individual departments.
- In Quarter Two of 2020, responsibility for the administration of the Excel Tracker was handed over to the Financial Governance team in the Treasury and Exchequer Department. The Financial Governance team has subsequently developed the Tracker working with other Treasury and Exchequer and Chief Operating Office colleagues. The Tracker is now operated through SharePoint rather than Excel. The SharePoint Tracker is much easier to use and administer than its Excel predecessor.
- Implementation of the Tracker by the Government is good practice. The States of Jersey are ahead of many other jurisdictions in implementing formal arrangements to track C&AG and PAC recommendations.
- Using SharePoint has improved the functionality of the Tracker. The Tracker is now more user-friendly, less manual and can be used more effectively in conjunction with other insightful reporting tools. The SharePoint functionality also allows for ongoing, real-time access, update and reporting.
- The Financial Governance team reports to the Chief of Staff in respect of the Tracker. The Chief of Staff is responsible for the Tracker at a senior level. Departmental leads are responsible for ensuring recommendation owners update the Tracker on a quarterly basis. Accountable Officers are ultimately responsible and accountable for the implementation of recommendations in their department. Each quarter, the Tracker reports are considered by senior leadership (at the Operating Committee) and are provided to the PAC.
The control framework
- Members of the Treasury and Exchequer team have administered the Tracker process since mid-2020. Updating did not occur in Q2 2020 (only) due to the COVID-19 pandemic. The COVID-19 pandemic has impacted on the implementation of some recommendations.
- Data integrity has been improved with the introduction of the SharePoint version of the Tracker. However the quality and timeliness of departmental updates and entries could be improved. There is a new actions' field in which recommendation owners record descriptions of actions taken and planned to close recommendations. Often this field is populated with general descriptions of actions rather than Specific, Measurable, Attainable, Relevant and Timely (SMART) descriptions.
- The importance of populating the Tracker fields appropriately has been emphasised by the Financial Governance team and the Chief of Staff. It is important however to ensure this is reinforced by an appropriate tone from the top' for all departments with an emphasis on effective update, implementation of and accountability for recommendations by recommendation owners.
- The Financial Governance team has briefly documented and communicated aspects of the roles and responsibilities for departments and each departmental lead. However, the documented Tracker procedures do not fully cover the associated operation, update, review, assurance and reporting activity. The Financial Governance team intends to develop these procedures into a formal Tracker Manual.
- At the time of my review, there were no clear definitions in place covering all key terms such as for rating progress, reasons for non-implementation and reasons for closure. There is also no clear consensus on what represents an appropriate definition of recommendation closure. Without a consensus on what constitutes closure of a recommendation there is a risk that the reports produced from the Tracker are internally inconsistent.
- New quarterly meetings have been introduced during 2021 between the Financial Governance team and departmental lead officers to review the current status of recommendations. Use of SharePoint has allowed instant update in these meetings.
- Users of the SharePoint system are able to set reminders within the system for when recommendations are due. Administrators of the system also receive an automated notification when a recommendation has been updated. The SharePoint system however is not configured to issue automatic reminders to all
recommendation owners when due dates for recommendations are approaching and subsequently when due dates are reached. The automatic reminder functionality could be used to require recommendation owners to update the Tracker with reduced Financial Governance team intervention.
- A revised recommendations closure process was put in place for Quarter Two of 2021. The Head of Financial Governance reviewed the 32 proposed C&AG and PAC recommendations to be closed for Quarter Two 2021 and rejected closure of 22 recommendations for a variety of reasons. The challenge from the Head of Financial Governance appears robust.
- Internal Audit also carry out periodic checks on the validity of departmental updates and challenge where appropriate. However, Internal Audit does not look at every closed recommendation due to resourcing issues and competing priorities. Internal Audit also aims to pick up on the implementation of previous C&AG recommendations in its ongoing assurance work programme.
- Departments take different approaches to monitoring their own recommendations. For example, the departmental lead in Health and Community Services (HCS) produces a formal report for the senior leadership team of HCS accompanied by a presentation. The report is well produced, references the C&AG reports themselves, the status of the recommendations, those proposed for closure and the recommendations being taken forward into the next quarter. The departmental lead in Children, Young People, Education and Skills (CYPES) prepares the quarterly update and delivers a verbal briefing on recommendations to the monthly performance board.
Recommendations
R1 Review the resources allocated to updating the Tracker and whether the Tracker
management and administration resides in the best place to ensure the highest profile is given to timely, accurate and complete implementation of agreed recommendations to ensure organisational improvement. If not, initiate an improved solution.
R2 Reinforce senior leadership messages about the importance and necessity of:
• progressing implementation of agreed recommendations; and
• completing the new actions' field with SMART descriptions of previous and planned activity to close agreed recommendations.
R3 Review how to include the Tracker in the performance management framework for
senior officers. Consideration should be given to including an objective within relevant officers' annual performance assessments in respect of the implementation of agreed recommendations, together with accurate and timely status reporting of progress being made.
R4 Ensure the planned Tracker Manual includes coverage of roles, responsibilities,
accountabilities and Tracker operation as well as appropriate closure protocols. This should include an agreed and a communicated consensus on what constitutes valid closure of a recommendation.
R5 Improve the narrative explanations on the Tracker where recommendations are
not being implemented to planned timescales. The supporting narrative should enable a quick, clear and consistent understanding of the cause and impact of any delays.
R6 Consider how best to further enhance the Tracker updating processes, exploiting
SharePoint functionality to automate update reminders to recommendation owners.
- The Financial Governance team administers the formal Tracker update and reporting process on a quarterly basis. During 2021 the process has become increasingly embedded and has proved quicker and more efficient with the introduction of the SharePoint Tracker.
- The Tracker Report to the PAC and C&AG in January 2021 showed that there were 250 open' live recommendations, as at the end of 2020. By the end of Quarter Three of 2021 there were 249 open' live recommendations following the closure of 98 recommendations and the addition of 97 new recommendations, as shown in Exhibit 2.
Exhibit 2: Number of open' recommendations each quarter of 2021
250 200 150 100 50 0
Q1 Closed New Q2 Closed New Q3 Closed New Q3 opening opening opening closing
Source: States of Jersey Tracker
- The PAC has continued to be more active in its review of the Tracker in 2021. The PAC has questioned Directors General at public hearings on the status of open recommendations.
Status of recommendations
- A large volume of recommendations is shown as not fully implemented on the Tracker, as set out above in Exhibit 2. During 2021, for most quarters, the number of new recommendations made roughly balanced the number of recommendations marked as closed.
- Officers interviewed as part of my review provided various views on why the number of open' recommendations remains so high, including:
• officers have perhaps been too quick to accept recommendations, without fully thinking through what would be needed for implementation
• officers sometimes set too ambitious implementation dates
• many of the recommendations made are cross-cutting and require considerable action by more than one department to implement
• some of the recommendations on the Tracker are deemed old' and there is reduced buy in' to implementing these recommendations at the current time; and
• competing priorities and a lack of resources, coupled with further pressure due to the COVID-19 pandemic, have caused delays in implementation.
- The colour coding system used on the Tracker is one of percentage completion, with various degrees of amber forming the bulk of the reporting. The measure of percentage completion is a measure of progress made in completion of the agreed action rather than a risk related measure as to the degree to which the outcome intended by a recommendation has been achieved. The latest report for Quarter Three of 2021 shows 72% of recommendations as amber 1' or amber 2' as shown in Exhibit 3.
Exhibit 3: Status of open' recommendations at the end of Q3 2021
Red (no progress) Amber 1 (1%-33% complete) Amber 2 (34%-66% complete) Amber 3 (67%-99% complete)
Source: States of Jersey Tracker
- Open' recommendations are categorised into one of several themes. The Quarter Three update for PAC highlighted themes associated with the (then) 249 recommendations as shown in Exhibit 4.
Exhibit 4: Themes of open' recommendations
70
61
60
50
39
40 32
30
18
20 16 16 15 16
11 9 8 8 10
0
Source: States of Jersey Tracker
- There are more open' recommendations associated with finance, property and health than any other theme.
- The Quarter Three update for PAC also reported the main themes associated with the reason for non-completion of recommendations. Most of the 249 recommendations stated Dependencies' as the reason for non-completion, however only 26 of these have an explanation of the dependency. Many of these explanations are superficial (for example, no change since last update'), while a small number are very comprehensive. Within the 249 open' recommendations, only 79 were assessed as on track for implementation by the target date. Exhibit 5 contains more details.
Exhibit 5: Reasons for non-completion of recommendations
100 90 80 70 60 50
40 30 20 10 0
Dependencies On Track for Lack of Requires a Requires an IT Needs target date Resources Political solution discussion at
decision Executive Leadership
Team
Source: States of Jersey Tracker
Potential areas for enhancement of reporting
- The current reporting could be enhanced to provide further useful analysis to challenge recommendation owners on timely implementation of agreed recommendations. The areas in which reporting could be improved include:
• reporting on what is overdue
• reporting on revisions to target dates
• reporting on expected benefits/outcomes associated with recommendations; and
• reporting on quick wins, short, medium term and strategic actions
Reporting on what is overdue
- There were 259 C&AG recommendations included on the Tracker my review (27 October 2021). Of these recommendations:
.
at the time of
• 164 recommendations were not yet due for implementation
• 21 are currently marked as being implemented
• 46 overdue recommendations (19.3%) currently have no revised target date for implementation
• 12 recommendations (5%) currently have a revised target date that is in the past and are therefore currently overdue; and
• a further 16 recommendations (6.7%) had target dates or revised target dates for implementation at the end of October 2021 with a high risk of becoming overdue.
- There are therefore 74 recommendations (31%) that were overdue for implementation, or very near to being overdue when compared to their latest revised target dates for implementation. This does not include any implemented' recommendations that might be re-categorised on review. Exhibit 6 provides more details.
Exhibit 6: Open' recommendations analysed by due date
21 46
12
16
164
Overdue recommendations with no revised target date
Overdue recommendations with a revised target date in the past
Recommendations that have target dates or revised target dates at the end of October 2021 Recommendations currently within target dates or revised target dates
Recommendations currently marked as implemented
Source: Jersey Audit Office analysis of States of Jersey Tracker
- The C&AG and PAC reports from which recommendations remain open' date back as far as 2014. Exhibit 7 shows the open recommendations by report and by year (up to and including 2020 reports).
Exhibit 7: Open' recommendations by report and year
States Employment Board follow up - Phase 1 COVID-19 Procurement
COVID-19 Public Finances COVID-19 Healthcare Response
CRESS Anti-Corruption arrangements
Annual Reporting States of Jersey Development Company
Handling and Learning from Complaints Insurance
Use of Enforcement Powers Management Information in Education (Follow-Up) Remuneration of Board Members
Financial Management and Internal Control Freedom of Information follow up
Non-Ministerial Departments Housing Repairs and Maintenance: Follow-up Community and Social Services for Adults and Older Role and Operation of the States Employment Board States as a shareholder - follow up
Governance of Health and Social Care Operational Land and Buildings
Arm's Length Organisations - Visit Jersey Governance of the States of Jersey Police
Decision Making - Selecting a Site for the Future Hospital
Private patient income follow up Grants and subsidies
Arms-length organisations Use of consultants
eGovernment Follow up on the utilisation of compromise agreements Freedom of Information
Review of Community and Social Services Private patient income
Use of Management Information in the Health and States as a shareholder - Jersey Telecom
Governance of the States of Jersey Pension Schemes Procurement
0 2 4 6 8 10 12 14 16 18
70 60 50 40 30 20 10 0
2014 2015 2016 2017 2018 2019 2020 Source: Jersey Audit Office analysis of States of Jersey Tracker
- The majority of the 259 PAC and C&AG open' recommendations at the time of my review had original implementation dates in 2020. Exhibit 8 contains more details.
Exhibit 8: Open' recommendations analysed by original target implementation dates
160 140 120 100 80 60 40
20 0
2017 2018 2019 2020 2021 2022 Source: Jersey Audit Office analysis of States of Jersey Tracker
- The Health and Community Services and Treasury and the Exchequer departments have the largest number of longstanding, unimplemented actions. Exhibit 9 contains more details.
Exhibit 9: Recommendations by department with original target implementation dates between 2017 and 2019
5%
10%
33%
3% 41%
8%
- Chief Operating Office Economy
- Health and Community Services Office of the Chief Executive
- Strategic Policy, Planning and Performance Treasury and Exchequer
- Health and Community Services Office of the Chief Executive
Source: Jersey Audit Office analysis of States of Jersey Tracker
- The main areas in which recommendations have been made but have not yet been implemented are spread across the C&AG areas of responsibility as shown in Exhibit 10.
Exhibit 10: Open' recommendations by category
Governance Internal Control Value for Money Source: Jersey Audit Office analysis of States of Jersey Tracker
- There are clearly challenges for departments in implementing agreed recommendations to agreed timetables. There remain weaknesses in governance, internal control and in the delivery of value for money until such time as the recommendations are implemented. The current reporting does not highlight sufficiently the challenges in implementation and the nature of the weaknesses that remain.
Reporting on revised target dates
- As noted above, a significant number of the 259 PAC and C&AG recommendations on the Tracker at the time of my review recorded action required beyond the original target date for implementation. Exhibit 11 contains more detail.
Exhibit 11: Open' recommendations with actions outside target date
1.50%
26.60%
71.80%
Original implementation date not entered Actions within original implementation date Actions outside original implementation date
Source: Jersey Audit Office analysis of States of Jersey Tracker
- Just over a quarter of the recommendations are currently within their original target date, with just under 72% outside the original target date.
- The analysis reinforces my finding that appropriate action to implement recommendations is not generally taking place on a timely basis. For most recommendations, implementation of action lags behind agreed target dates. Implementation dates need to be realistic and firmly agreed, with owners accountable, and held to account, for delivery.
- The current reporting does not highlight sufficiently the recommendations that have had more than one revised target date for implementation.
Reporting on expected benefits/outcomes associated with recommendations
- Reporting on the expected benefits and outcomes associated with the recommendations is not currently undertaken. For this to be implemented, some additional analysis would need to be undertaken to quantify and qualify benefits at the time the recommendation was made. If such an analysis was undertaken if would help to prioritise implementation target dates based on a better understanding of risks and benefits and would highlight the continued risks associated with open' and overdue recommendations.
Reporting on quick wins, short, medium term and strategic actions
- Reporting on the speed required to implement actions is not currently undertaken. For this to be implemented, some additional analysis would be required to
categorise recommendations when they are accepted for implementation. In accepting the recommendation, the owner could state what action will be undertaken to implement the recommendations, and the target date for this. For example:
• quick wins' could be categorised as those which will be implemented within six months
• short-term actions' could be categorised as those which will be implemented within 6-12 months
• medium-term actions' could be categorised as those which will be implemented within 12-24 months; and
• strategic' or 'long-term actions' could be categorised as those which will take longer than 24 months to implement.
Recommendations
R7 Review all open' recommendations on the Tracker and ensure that they have a
valid, future target date or revised target date for implementation. In undertaking the review:
• ensure all implementation dates are realistic and are agreed with recommendation owners; and
• improve the quality of entries on the Tracker to be clear as to who is doing what by when to progress the recommendation.
R8 Enhance the reporting of open' recommendations on the Tracker to include:
• reporting on what is overdue
• reporting on revisions to target dates
• reporting on expected benefits/outcomes associated with recommendations; and
• reporting on quick wins, short, medium term and strategic actions.
The review included the following key elements:
• review of relevant documentation provided by the States of Jersey; and
• interviews with key officers within the Government of Jersey and the Chair of the PAC.
The documentation review included:
• Quarter Two and Quarter Three 2021 Tracker reports to PAC
• Tracker update reports for the Operating Committee Q3 2020, Q4 2020 and Q1 2021
• HCS C&AG and Scrutiny Report Q3 2021
• Notes on Tracker process for Directors General April 2021
• Recommendation closing process
• Simple C&AG, PAC & Scrutiny Recommendation Tracker manual
The officers interviewed included:
• Advisor – Analytics & Management Information, Treasury and Exchequer
• Chief Internal Auditor
• Chief of Staff
• Chief Operating Officer
• Head of Financial Governance, Treasury and Exchequer
• Head of Office, Children, Young People, Education and Skills
• Head of Strategic Planning and Reporting, HCS
• Interim Director of Statistics and Analytics
• Internal Audit Manager
• Public Accounts Committee Officer
I would like to thank all officers and individuals who have contributed to this report.
The fieldwork was carried out by an affiliate working for the Comptroller and Auditor General.
R1 Review the resources allocated to updating the Tracker and whether the Tracker
management and administration resides in the best place to ensure the highest profile is given to timely, accurate and complete implementation of agreed recommendations to ensure organisational improvement. If not, initiate an improved solution.
R2 Reinforce senior leadership messages about the importance and necessity of:
• progressing implementation of agreed recommendations; and
• completing the new actions' field with SMART descriptions of previous and planned activity to close agreed recommendations.
R3 Review how to include the Tracker in the performance management framework for
senior officers. Consideration should be given to including an objective within relevant officers' annual performance assessments in respect of the implementation of agreed recommendations, together with accurate and timely status reporting of progress being made.
R4 Ensure the planned Tracker Manual includes coverage of roles, responsibilities,
accountabilities and Tracker operation as well as appropriate closure protocols. This should include an agreed and a communicated consensus on what constitutes valid closure of a recommendation.
R5 Improve the narrative explanations on the Tracker where recommendations are
not being implemented to planned timescales. The supporting narrative should enable a quick, clear and consistent understanding of the cause and impact of any delays.
R6 Consider how best to further enhance the Tracker updating processes, exploiting
SharePoint functionality to automate update reminders to recommendation owners.
R7 Review all open' recommendations on the Tracker and ensure that they have a
valid, future target date or revised target date for implementation. In undertaking the review:
• ensure all implementation dates are realistic and are agreed with recommendation owners; and
• improve the quality of entries on the Tracker to be clear as to who is doing what by when to progress the recommendation.
R8 Enhance the reporting of open' recommendations on the Tracker to include:
• reporting on what is overdue
• reporting on revisions to target dates
• reporting on expected benefits/outcomes associated with recommendations; and
• reporting on quick wins, short, medium term and strategic actions.
26 | Tracking Comptroller and Auditor General Recommendations