Handling and Learning from Complaints

OFFICE OF THE COMPTROLLER AND AUDITOR GENERAL

HANDLING AND LEARNING FROM COMPLAINTS

08 JULY 2020

R.67/2020

Contents

Summary .................................................................................................................... 2 Introduction ............................................................................................................. 2 Key findings ............................................................................................................ 6 Conclusions ............................................................................................................ 8 Objectives and scope of the review ............................................................................ 9 Detailed findings ....................................................................................................... 11 Principles .............................................................................................................. 11 People .................................................................................................................. 16 Process ................................................................................................................. 19 Analysis ................................................................................................................ 22 Culture .................................................................................................................. 26 Appendix One .......................................................................................................... 28 Audit Approach.................................................................................................. 28

Appendix Two .......................................................................................................... 30 Summary of recommendations ......................................................................... 30

Summary

Introduction

1.  Effective management of complaints from members of the public about public bodies can:

1. provide valuable information about weaknesses in programmes of work, policies and service delivery;  
2. stimulate improvement;
3. reassure the public that an organisation is committed to resolving problems; and
4. enhance accountability and transparency.

3. Poor handling of complaints can:
   1. fail to produce learning and improvement;
   2. lead to repetition and in some cases exacerbation of the underlying issue;
   3. damage an organisation's reputation; and
   4. impact on an organisation's relationship with the public and other stakeholders.

3. In Jersey, the Administrative Decisions (Review) (Jersey) Law 1982 established a Complaints Panel from which a Board can be drawn to consider complaints against Ministers, Departments and those acting for them. Boards consider complaints in public, report their findings publicly and may recommend reconsideration of a matter. Following a report by the Jersey Law Commission in 2017, Improving Administrative Redress in Jersey, the States Assembly voted in favour of a Proposition (P.32/2018), agreeing in principle that in place of the Complaints Panel there should be an Ombudsman and called for further research to be carried out. In November 2018 the Jersey Law Commission published another report, Designing a Public Services Ombudsman for Jersey, identifying and examining options for an Ombudsman and making recommendations. In July 2019 the States launched a 12-week island-wide consultation so that islanders could give their views on the proposed Ombudsman.

4. My predecessor, in her December 2019 report Governance – a Thinkpiece, supported the establishment of a Public Services Ombudsman for Jersey. The establishment of a Public Services Ombudsman, with wide powers reflecting modern legislation in other jurisdictions, sends an important message about transparent, accountable services that embrace feedback and a commitment to improve.
5. The vast majority of complaints are dealt with within Government and do not reach a Complaints Panel and so are unlikely to reach an Ombudsman. Historically, there has been evidence of poor management of such complaints:

• in her September 2018 report, my predecessor highlighted the time taken to respond to and the lack of evidence of learning and sharing from complaints; and  
• Government departments have been the subject of some highly critical findings from the Complaints Boards that appear to reflect issues in engaging with the public. For example, its August 2018 report a Board upheld a complaint about processing a planning application.  

6. At its June 2018 meeting the then Executive Management Board discussed the issues as it saw them with the arrangements in place for managing feedback from the public. These included that:

• each department had adopted a different process with varying levels of success – there was not a single, shared policy;
• there was a fragmented and inconsistent response to customer feedback; and
• for customer complaints in particular, there were no agreed standards for:

• quality and speed of response;
• identifying, recording and analysing complaints; and
• co-ordinating service improvement.

7. In the context of a wider focus on customers and the development of a Customer Strategy, the Executive Management Board decided to develop a Customer Feedback Policy, with a key focus on improving the approach to managing complaints from the public. This was also, in part, a response to the Independent Jersey Care Inquiry report published in July 2017 that recommended a new system that could be easily accessed and in which children and young people could have confidence. The Director General of Customer and Local Services was charged with leading a project to establish a single, standardised approach to managing customer feedback.
8. The new Customer Feedback Policy recognises three distinct stages in handling complaints (see Exhibit 1).

Exhibit 1: New Customer Feedback Policy: stages in complaints handling

9. The Government of Jersey has collated statistics on complaints since October 2019 after the launch of the Customer Feedback Policy. In the five months from October 2019 to February 2020 it recorded a total of nearly 450 complaints (see Exhibit 2).

Exhibit 2: Recorded Complaints October 2019 to February 2020

Chief Operating Office Children, Young People, Education & Skills Customer and Local Services

Growth, Housing & Environment Health & Community Services

Justice & Home Affairs Non-Ministerial Departments

Office of the Chief Executive Strategic Policy, Planning & Performance Treasury & Exchequer

0 20 40 60 80 100 120 140 160

Oct - Dec 2019 Jan - Feb 2020

Source:  Government of Jersey monthly reports

Note 1:  Figures for Health and Community Services exclude February 2020

Note 2:  Figures for Non-Ministerial Departments include only complaints raised on the Government

of Jersey system

Note 3:  Figures for Children, Young People, Education & Skills do not include complaints relating to

schools

Key findings

10. The key findings from my review are as follows:

• the Government of Jersey has prepared and adopted a Customer Feedback Policy that, in most respects, meets the requirements of best practice;

• responsibility for the management of complaints rests mostly with designated officers within departments, with a small corporate team providing an element of co-ordination but without a defined supervisory role;
• a Customer Feedback Management System has been introduced that captures information on complaints across Government;
• a single online channel is now established so that customers can submit feedback, including complaints, about any Government service. Currently more than half of customer feedback is received via that channel;
• other than guidance on the use the Customer Feedback Management System, there are no standardised corporate procedures to support the implementation of the Customer Feedback Policy. Departments rely on their own procedures that have not been updated consistently to reflect the Customer Feedback Policy;
• as noted by my predecessor in her 2019 report Non-Ministerial Departments, there has not always been adequate consideration of the adaptations to corporate activities, including modernisation initiatives, necessary to secure their relevance to Non-Ministerial Departments. While some Non-Ministerial Departments contributed to the development of the Customer Feedback Policy by commenting on draft versions and through participation in workshops, many report a lack of clarity on the terms of their involvement;
• some corporate information on complaints is being generated by the Customer Feedback Management System and shared with the Executive Leadership Team and departmental management. However, the information is limited, in part because of systems limitations, and no information is captured and reported systematically to measure the efficiency and effectiveness of the complaints handling process; and

• the tone from the top' has emphasised the importance of complaints as an integral part of public service provision. However, a consistent shared culture of valuing and learning from complaints is not yet embedded.

Conclusions

11. The Government has taken important steps to improve complaints handling. The adoption of a Customer Feedback Policy, investment in a Customer Feedback Management System, recruitment of a corporate team and designation of departmental staff have shown a commitment to improving complaints handling.
12. More work is required to secure consistent handling of and learning from complaints. In particular, there is a need for a focus on:

• ensuring that the staff handling complaints are people with the right skills, experience, training and supervision;
• ensuring that there are appropriate processes, consistently applied, to facilitate the delivery of the Customer Feedback Policy;
• ensuring that the Customer Feedback Management System is developed where necessary and its capacity fully used; and
• maximising the value that can be secured from the analysis of complaints and their handling.

Objectives and scope of the review

13. The review has evaluated:

• the design and operation of the Government's current arrangements for handling and learning from complaints; and
• the design of proposed changes in the arrangements for handling and learning from complaints.

14. In undertaking the review I focussed on key areas that drive effective complaints management (see Exhibit 3). In high performing organisations the principles, people and process for complaints handling and the analysis of complaints are underpinned by a culture that values complaints.

Exhibit 3: Handling and learning from complaints: areas of focus

Principles  People

Culture

Process  Analysis

Source: Adapted from: Australian Commonwealth Ombudsman Better Practice Guide to Complaint Handling: five elements of effective complaint handling

15. The review has not extended to:

• a detailed review of the work of the Complaints Panel or Ministerial responses to Complaints Board findings;
• the work of the Police Complaints Authority; or
• Arm's Length Organisations.  

Detailed findings

Principles

16. Clear, appropriate principles for complaints handling are a pre-requisite for effective complaints management.
17. The new Customer Feedback Policy reflects many elements of best practice (see Exhibit 4). There are, however, some areas where there is scope for development, especially in relation to:

• responding to particular needs and vulnerable complainants. I recognise that a post-implementation review is planned in this area; and
• complaints relating to more than one service provider, whether in the context of a contracted-out service or a complaint that relates to the Government and another public sector body.

Exhibit 4: New Customer Feedback Policy

Source: Best practice adapted from: Australian Commonwealth Ombudsman Better Practice Guide to Complaint Handling: five elements of effective complaint handling

Recommendations

R1  Include in the planned post-implementation review consideration of solutions

to the current barriers to access, including for:

• children and young people;
• people with sensory disabilities;
• those whose first language is not English; and
• those with learning and reading difficulties.

R2  Explicitly state in the Customer Feedback Policy and in relevant literature and

communications that there is no charge for complaints handling.

R3  Develop and agree a corporate process so that departments can obtain

assurance that:

• contracted out service providers have an accessible and comprehensive complaints management process;
• contracted out service providers and departments have a shared understanding of the scope and responsibilities of each other's complaints processes; and

• there are mutually agreed approaches to handling complaints relating to more than one public body, including coverage of information exchanges, joint working and monitoring the effectiveness of handling such complaints.

People

18. Good complaints handling relies on the engagement and use of staff with the right skills, experience, training and supervision to deal directly with complaints as well as a wider awareness of complaints management amongst all staff in an organisation.
19. The model adopted by the Government of Jersey involves:

• a small corporate team in Customer and Local Services comprising a Customer Feedback Manager and Head of Customer Experience specifically recruited to relevant posts. Officers have advised that this was enabled by restructuring the department; and
• designation of Departmental Feedback Managers and Section Feedback Managers who have a key role to perform in handling complaints. The process of identification of staff to perform these roles on an ongoing basis has in some cases been slower than was planned.

20. The effectiveness of these arrangements has been hindered by:

• the absence of clear role descriptors for departmental staff performing a key role in complaints handling. Such role descriptors aid identification of the right staff to assume these roles. They also provide a basis for identifying the training and other support needed by departmental complaints staff and aid in evaluation of performance in complaints handling. Work is ongoing to develop such descriptors but there is no clear plan for completion of the work;

• the absence of a clear, documented structure for corporate training and support for departmental complaints handlers. A programme of support for Departmental Feedback Managers and training on handling difficult complaints is planned. A series of meetings where Departmental Feedback Managers can propose the support they require was scheduled but has been postponed due to the Covid-19 pandemic; and
• the absence of clear, documented arrangements for oversight of departmental complaints handlers. There is no explicit role for the corporate team in Customer and Local Services and no use of data from the Customer Feedback Management System to monitor the performance of departmental complaints handlers. This increases the risk of complaint handling that is not in accordance with the Customer Feedback Policy.

21. The Government has recognised that, although some staff will focus on complaints, all staff have a role to play in the management of complaints and that wide understanding of corporate policies and processes is vital. The Government has therefore developed mandatory on-line training on the Customer Feedback Policy for all staff, including agency staff. However:

• there is no corporate policy on whether volunteers should undertake the training; and
• take-up of training has been variable across departments (see Exhibit 5). Differences may be attributable to access to computers to undertake the training.

Exhibit 5: Take-up of mandatory corporate complaints training by department as at January 2020

Customer and Local Services Strategic Policy, Planning and Performance Office of the Chief Executive

Chief Operating Office Justice and Home Affairs

Growth, Housing and Environment Treasury and Exchequer

Non-Ministerial Departments and States Assembly CYPES - Non-Schools

Health and Community Services CYPES - Schools

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Source: OneGov Customer Feedback Q4 2019 presentation to ELT (February 2020) Note 1: NMDs means Non Ministerial Departments'

Note 2: CYPES means Children, Young People, Education and Skills'

Recommendations

R4  Develop and adopt a common role description for key departmental staff

involved in handling complaints.

R5  Develop, agree and implement corporate arrangements for support, including

training for and oversight of departmental complaints handlers.

R6  Develop, adopt, roll out and monitor compliance with corporate standards for

promoting awareness of complaints handling processes for volunteers.

R7  Identify barriers to take-up of mandatory on-line training and take appropriate

corrective action.

Process

22. In high performing organisations consistent implementation of corporate principles is facilitated by clear, accessible processes covering all stages of complaints handling, namely:

• initial communication;
• assignment and setting priority;
• planning how to investigate a complaint;
• investigating the complaint;
• providing a response to the complainant;
• providing the opportunity for a complainant to obtain more information about the handling of a complaint or to challenge a decision; and
• considering whether a complaint identifies systemic issues.

23. The Government has developed and rolled out a corporate Customer Feedback Management System to log and capture information about the handling of complaints. However, it has not developed comprehensive corporate procedures to support the handling of complaints. Those dealing with complaints are therefore dependent on departmental procedures. Where these exist they have not, in all cases, been updated to reflect the Customer Feedback Policy.
24. There is therefore an increased risk of both inconsistency and inefficiency in complaints handling. In particular, there is an increased risk that:

• complaints are not appropriately assessed at the outset and the right resources assigned to the investigation;

• short, written plans for handling complaints are not prepared, increasing the risk that they are not handled promptly;
• fair, evidence-based procedures are not consistently adopted in investigating complaints;
• responses are not fit for purpose. For example, they may not systematically give clear reasons for decisions, cover all elements of a complaint or routinely offer complainants the opportunity to seek a review of a complaint;
• complaints are not escalated when it is appropriate to do so;
• complaints that raise serious concerns' about the Government and therefore fall within the scope of its Whistleblowing Policy rather than the Customer Feedback Policy, might incorrectly be considered under the Customer Feedback Policy; and
• interim findings are not routinely communicated in the context of complaints that are taking longer to investigate.

25. The Customer Feedback Management System has for the first time provided a corporate system for capturing data about complaints. However, its value has been limited by:

• the absence of an ability to capture, at the outset, details of the remedy being sought by the complainant; and
• the absence of a structured way of capturing findings from complaints. At the moment, free text is used which inhibits the identification of systemic issues.

26. There is limited corporate oversight of departmental complaints handling processes. In particular:

• there is no corporate review of departmental processes to provide assurance that they are consistent with the Customer Feedback Policy; and
• there is no corporate monitoring of departmental compliance with the Customer Feedback Policy in key areas such as confidentiality.

Recommendations

R8  Develop, adopt and roll out operating procedures and model documentation to

support application of the Customer Feedback Policy covering:

• initial contact;
• complaint assessment and assignment of priority;
• planning of the complaint investigation;
• securing fairness in the investigation;
• complaint response;
• review of complaints; and
• identification of systemic issues.

R9  Ensure that operating procedures adequately address:

• provision of reasons for decisions;
• provision of interim decisions;
• consideration of each element of a complaint;
• criteria for escalation in handling;

• handling of complaints that fall within the scope of the Whistleblowing Policy; and
• offering complainants the opportunity to seek review of a decision.

R10 Prioritise system developments to capture:

• the remedy being sought; and
• systemic issues in a structured way, to facilitate learning and improvement.

R11  Establish and implement procedures for quality assurance of all departmental

procedures for handling complaints.

R12  Establish and implement procedures for monitoring departmental compliance

with corporate policy.

Analysis

27. To secure best value from complaints, handling organisations should look beyond individual complaints to learn in a structured way from complaints as a whole by considering:
    1. lessons to improve public services; and
    2. lessons about complaints handling.

29. The Government has:
    1. in the Customer Feedback Policy, recognised the importance and value of analysis of complaints;

• implemented a system to capture and allow reporting of key metrics about complaints; and
• provided reporting on complaints to the Executive Leadership Team and departmental management.

29. The value of analysis is reduced by the absence of mechanisms to confirm the completeness and accuracy of data captured by the Customer Feedback Management System. Anecdotal evidence suggests that in some areas there is significant failure to identify and log complaints received. For example, management has indicated that complaints associated with the new Income Tax system were not recorded in the Customer Feedback Management System during the period of transition from the old system to the new system. This is despite the Customer Feedback Management System being designed to include feedback on Government policy as well as service delivery.
30. Currently, a limited range of Key Performance Indicators (KPIs) is reported to the Executive Leadership Team and senior departmental management

(see Exhibit 6).

Exhibit 6: Key Performance Indicators in relation to complaints

Source: Government of Jersey CFMS monthly reports

31. The value of the current reporting is limited:

• information is not captured about the characteristics or geographical location of complaints. These dimensions are therefore not reflected in the KPIs;
• information is not captured about whether complaints expose weaknesses or raise questions about integrity or reputation. Nor is information about instances where the conduct of the complainant is regarded as unreasonable captured. There are therefore no KPIs in these areas;
• information is not reported on the extent to which complaints are upheld or not;
• the reporting of themes from complaints is judgemental and does not include quantified measures;
• there are no KPIs reported relating to the efficiency and effectiveness of complaints handling. A software problem means that the Customer Management Feedback System is not currently able to report on the timeliness of handling of complaints. There are no mechanisms for securing, capturing and reporting on feedback from complainants about their satisfaction with the complaints handling process; and
• not all KPIs have targets against which performance in handling and learning from complaints can be measured.

32. At its inception the Customer Feedback Policy set out a number of objectives for complaints handling, such as accessibility, responsiveness, flexibility, objectivity, fairness and confidentiality. Reporting and KPIs do not explicitly reflect measurement of performance against those objectives.
33. The Customer Feedback Management System can be a valuable source of information about the performance of individual members of staff and teams. However, information is not currently routinely used to support performance management.

Recommendations

R13 Develop a clear plan for improvement in data quality regarding complaints and

monitor its implementation.

R14 Expand data capture and KPIs to cover:

• characteristics of complainants;
• geographical spread;
• complaints that expose weaknesses or raise questions about integrity and reputation;
• instances of unreasonable behaviour by complainants;
• the outcome of complaints;
• the themes from complaints on a structured basis; and
• measures of efficiency and effectiveness of complaints handling, including the timeliness of complaints handling and complainant satisfaction.

R15 Set targets for all KPIs.

R16  Ensure that KPIs facilitate measurement of performance against the

objectives of the Customer Feedback Policy.

R17  Develop plans to use information from the Customer Feedback Management

System for performance management purposes.

Culture

34. The effectiveness of arrangements for handling and learning from complaints is underpinned by the culture of an organisation. Complaint handling works best where there is a shared view that complaints are valued as an opportunity to learn and thereby to improve public services.
35. In the context of a wider transformation of public services, the establishment of the Customer Feedback Project, development of a Customer Strategy and development of the Customer Feedback Policy, were important steps in establishing a culture that embraced complaints.
36. However, it will take time and further effort to embed such a culture. Some of the issues identified above, such as low take-up of training and issues with completeness of capture of complaints on the Customer Feedback Management System, may reflect an embedded culture that does not value complaints and sees them as a threat.
37. Some aspects of current arrangements may reinforce elements of the departmental culture that management is seeking to tackle. The relatively limited role of the corporate team discussed above means, for example, that there is no explicit mechanism for considering the adequacy of resourcing of complaints across Government and reallocating resources as appropriate.
38. Tackling those cultural barriers is not easy but may include identifying metrics to evaluate the extent to which the Government's objectives on complaints are understood and owned and using those to drive improvement.

Recommendations

R18  Identify and use relevant metrics to evaluate the extent to which the

Government's objectives in relation to complaints are consistently understood and owned.

R19  Develop plans for a relaunch of the Customer Feedback Policy to staff

following further work to secure management buy-in and implementation of the recommendations about complaints management contained in this report.

Appendix One

Audit Approach

The review included the following key elements:

• review of relevant documentation provided by the States of Jersey;
• review of web pages and online systems; and
• interviews with key officers within the States of Jersey.

The documentation reviewed included:

• drafts of the Customer Feedback Policy;
• the published Customer Feedback Policy;
• presentations to and responses from:

• the Executive Management Board and Executive Leadership Team;
• the Senior Management Group; and
• the Council of Ministers;

• Project Plan and progress reports for the implementation of the Customer Feedback Policy;
• the Customer Feedback Project risk register;
• material from the stakeholder workshops;
• training material;
• Health and Community Services documented complaints procedure;
• early reports from the Customer Feedback Management System;

• reports from the Jersey Complaints Panel; and
• the Customer Feedback project close down review.

The following officers were interviewed:

• Director General, Customer and Local Services;
• Group Director, Customer Services;
• Project Manager, Customer Feedback Project;
• Customer Feedback Manager;
• Head of Customer Experience;
• Departmental leads for customer feedback;
• A complainant;
• Head of Organisational Development; and
• Head of Team Jersey.

I would like to thank all officers who have contributed to this report.

The fieldwork was carried out by an affiliate working for the Comptroller and Auditor General.

Appendix Two

Summary of recommendations

R1  Include in the planned post-implementation review consideration of solutions

to the current barriers to access, including for:

• children and young people;
• people with sensory disabilities;
• those whose first language is not English; and
• those with learning and reading difficulties.

R2 Explicitly state in the Customer Feedback Policy and in relevant literature and

communications that there is no charge for complaints handling.

R3  Develop and agree a corporate process so that departments can obtain

assurance that:

• contracted out service providers have an accessible and comprehensive complaints management process;
• contracted out service providers and departments have a shared understanding of the scope and responsibilities of each other's complaints processes; and
• there are mutually agreed approaches to handling complaints relating to more than one public body, including coverage of information exchanges, joint working and monitoring the effectiveness of handling such complaints.

R4  Develop and adopt a common role description for key departmental staff

involved in handling complaints.

R5  Develop, agree and implement corporate arrangements for support, including

training for and oversight of departmental complaints handlers.

R6  Develop, adopt, roll out and monitor compliance with corporate standards for

promoting awareness of complaints handling processes for volunteers.

R7  Identify barriers to take-up of mandatory on-line training and take appropriate

corrective action.

R8  Develop, adopt and roll out operating procedures and model documentation to

support application of the Customer Feedback Policy covering:

• initial contact;
• complaint assessment and assignment of priority;
• planning of the complaint investigation;
• securing fairness in the investigation;
• complaint response;
• review of complaints; and
• identification of systemic issues.

R9  Ensure that operating procedures adequately address:

• provision of reasons for decisions;
• provision of interim decisions;
• consideration of each element of a complaint;

• criteria for escalation in handling;
• handling of complaints that fall within the scope of the Whistleblowing Policy; and
• offering complainants the opportunity to seek review of a decision.

R10 Prioritise system developments to capture:

• the remedy being sought; and
• systemic issues in a structured way, to facilitate learning and improvement.

R11  Establish and implement procedures for quality assurance of all departmental

procedures for handling complaints.

R12  Establish and implement procedures for monitoring departmental compliance

with corporate policy.

R13 Develop a clear plan for improvement in data quality regarding complaints and

monitor its implementation.

R14 Expand data capture and KPIs to cover:

• characteristics of complainants;
• geographical spread;
• complaints that expose weaknesses or raise questions about integrity and reputation;
• instances of unreasonable behaviour by complainants;
• the outcome of complaints;
• the themes from complaints on a structured basis; and

• measures of efficiency and effectiveness of complaints handling, including the timeliness of complaints handling and complainant satisfaction.

R15 Set targets for all KPIs.

R16  Ensure that KPIs facilitate measurement of performance against the

objectives of the Customer Feedback Policy.

R17  Develop plans to use information from the Customer Feedback Management

System for performance management purposes.

R18  Identify and use relevant metrics to evaluate the extent to which the

Government's objectives in relation to complaints are consistently understood and owned.

R19  Develop plans for a relaunch of the Customer Feedback Policy to staff

following further work to secure management buy-in and implementation of the recommendations about complaints management contained in this report.

LYNN PAMMENT

COMPTROLLER and AUDITOR GENERAL

JERSEY AUDIT OFFICE, DE CARTERET HOUSE, 7 CASTLE STREET, ST HELIER, JERSEY JE2 3BT T: 00 44 1534 716800 E: enquiries@jerseyauditoffice.je W: www.jerseyauditoffice.je