Browse our range of reports and publications including performance and financial statement audit reports, assurance review reports, information reports and annual reports.
Management of Complaints by the National Disability Insurance Agency
Please direct enquiries through our contact page.
Audit snapshot
Why did we do this audit?
- Effective complaints management contributes to transparency, accountability and continuous improvement of service delivery.
- This audit was conducted to provide assurance to the Parliament that the National Disability Insurance Agency (NDIA) has effective processes for managing complaints.
Key facts
- In 2022–23, the NDIA received 28,951 complaints from participants and 983 complaints from providers.
- ‘Dissatisfied with plan’ was the most common complaint theme in 2022–23.
What did we find?
- The NDIA’s management of complaints was partly effective.
- The NDIA’s complaints management framework was largely fit for purpose. Quality assurance results varied significantly during 2022–23.
- The NDIA’s reporting and review of complaints management was partly effective. Reporting of complaints did not include qualitative analysis or sufficient analysis alongside service delivery data to inform continuous improvement.
What did we recommend?
- There were two recommendations to the NDIA, relating to improving its complaints quality assurance framework, and using complaints data to identify service improvements.
- The NDIA agreed to both recommendations.
87%
complaints acknowledged by the NDIA within one day in 2022–23, against a target of 100 per cent.
96.8%
complainants contacted by the NDIA within two days in 2022–23, against a target of 100 per cent.
85.8%
complaints resolved by the NDIA within 21 days in 2022–23, against a target of 90 per cent.
Summary and recommendations
Background
1. The National Disability Insurance Scheme (the NDIS) was established in 2013 under the National Disability Insurance Scheme Act 2013 (NDIS Act) to provide funding for reasonable and necessary supports for people with permanent and significant disability. The NDIS is jointly funded by the Australian, state and territory governments under bilateral arrangements.
2. The National Disability Insurance Agency (NDIA) is the Australian Government entity responsible for delivering the NDIS. It is a corporate Commonwealth entity under the Public Governance Performance and Accountability Act 2013. In 2022–23, the NDIA reported that it employed a total of 5,652 staff with 20.8 per cent of those having a disability.
3. The NDIA receives complaints from a variety of sources on such matters as dissatisfaction with supports in a participant’s plan, payment delays, approvals of plan reviews and engagement with NDIA staff. The NDIA reported in 2022–23 that it received 28,951 complaints from participants (compared to 30,091 in 2021–22) and 983 complaints from NDIS providers (compared to 1,446 in 2021–22). Complaints are also received from various other stakeholders including the NDIS Quality and Safeguards Commission, the Commonwealth Ombudsman, parliamentarians, and state and territory agencies.
Rationale for undertaking the audit
4. In 2021–22 the NDIA reported that it received 30,091 complaints from participants of the NDIS and 1,446 from NDIS providers; and in 2022–23 it reported having received 28,951 complaints from participants and 983 from NDIS providers. Effective complaints management contributes to transparency, accountability and the continuous improvement of administrative decision-making and service delivery. This audit provides assurance to the Parliament that the NDIA has fit-for-purpose complaints management practices.
Audit objective and criteria
5. The objective of the audit was to assess the effectiveness of the NDIA’s management of complaints.
6. To form a conclusion against this objective, the following high-level criteria were applied.
- Does the NDIA have a fit-for-purpose complaints management framework?
- Does the NDIA adequately report on complaints, and outcomes, and review its management of complaints?
Conclusion
7. The National Disability Insurance Agency’s (NDIA’s) management of complaints was partly effective.
8. The NDIA has a largely fit-for-purpose complaints management framework. Supporting internal procedural guidance documents were largely clear, complete, and current. Complaints management is monitored through a monthly quality assurance process assessing four key areas: right person; right process; right referral; and right resolution. The quality assurance results varied significantly from month to month across all four areas in the last financial year, with low results against multiple procedures and no action plan to remedy performance. In 2022–23, the NDIA did not meet its 90 per cent target for resolving complaints within 21 days.
9. The NDIA’s reporting and evaluation of complaints management is partly effective. The NDIA reports internally and publishes regular data on complaints statistics and performance against service standards with limitations on data quality clearly identified. The effectiveness of the NDIA’s suite of internal reporting would be enhanced by more timely reporting on complaints outcomes, detailed qualitative analysis and greater use of complaints data alongside other service delivery data to support continuous improvement activities. The NDIA had not fully implemented monitoring and reporting actions identified by the Commonwealth Ombudsman for lifting its complaints approach to a ‘superior’ maturity level. Recommendations from a 2019 review were not fully implemented. The NDIA’s 2023 review of complaints management lacked baseline evidence and a detailed implementation plan.
Supporting findings
Complaints management policy and practice
10. The NDIA provides a range of methods to make a complaint. Information about making a complaint is on the NDIS website1; it is not prominent in all sections of the website relating to contacting the agency or available in languages other than English. The NDIA’s complaints processes largely align with the key design elements of the Commonwealth Ombudsman’s better practice principles. Supporting internal procedural guidance documents were largely clear, complete, and current. (See paragraphs 2.2 to 2.30)
11. The NDIA undertakes monthly quality assurance checks of its complaints management. There are no internal targets for the results of the quality checks. The significant variability in the results limits the usefulness in assessing compliance with NDIA policy and procedures or identifying priority areas requiring improvement. In a targeted sample of 30 complaints, the recorded actions taken were largely consistent with the requirements of the NDIA’s policies and procedures. (See paragraphs 2.31 to 2.53)
12. The NDIA reports quarterly averages for performance against the 21-day complaints resolution service standard. Based on the quarterly averages for the financial year, the NDIA met its 90 per cent target for this service standard in 2021–22 (achieving 94.5 per cent) and did not meet the standard in 2022–23 (achieving 85.8 per cent). Performance against the 21-day standard was 71 per cent for the December 2023 quarter. In 2022–23, the NDIA’s reported performance against the service standard of one day to acknowledge the complaint was 87 per cent. The reported performance for the service standard of contacting the complainant within two days was 96.8 per cent in 2022–23. Both results were below the NDIA’s 100 per cent target for these service standards. (See paragraphs 2.54 to 2.64)
NDIA reporting and evaluation of complaints
13. The NDIA reports publicly on volumes of complaints received, percentage of complaints closed within the 21-day service standard, sources of complaints, and complaint reason categories. There is regular internal reporting on complaints timeliness, compliance with policies and procedures and work allocation. Outcomes of ministerial complaints and complaints about the NDIA’s handling of a complaint are reported regularly; outcomes of other complaint categories are reported twice per year. The NDIA does not report on implementation of complaint outcomes that require further actions after the complaint was closed. Regular internal reports contain insufficient qualitative analysis of trends or comparison with other service delivery performance data. (See paragraphs 3.3 to 3.25)
14. There is no evidence of the NDIA accepting or rejecting the 14 recommended initiatives from a review of its complaint management process undertaken in 2019. There is evidence of implementation of two and partial implementation of a further three recommendations. There were recurring themes in the recommendations from the 2019 review and a 2023 internal review. The NDIA is yet to fully implement the action plan resulting from its 2023 internal review. The action plan does not specify timeframes for completion of activities. (See paragraphs 3.26 to 3.37)
15. While the NDIA completed a ‘deep dive’ analytical review of complaints data in August 2022 in response to an increased trend in a specific category of complaint, no subsequent ‘deep dives’ have been undertaken to investigate complaints data. There is no monitoring or reporting on the effectiveness of improvements introduced to the NDIA’s complaint management processes. The NDIA’s procedures for identifying continuous improvement opportunities for service delivery require consideration of complaints data. Records of the monthly prioritisation of service delivery issues for further investigation and action do not demonstrate that complaints data form a significant element of this approach. (See paragraphs 3.38 to 3.57)
Recommendations
Recommendation no. 1
Paragraph 2.36
The NDIA implement a fit-for-purpose complaints quality assurance framework to support monitoring of quality and continuous improvement.
National Disability Insurance Agency response: Agreed.
Recommendation no. 2
Paragraph 3.56
The NDIA plan and undertake a program of quarterly reviews of complaints data, matched with other service delivery performance data, including participant satisfaction surveys, to support identification of areas for continuous improvement.
National Disability Insurance Agency response: Agreed.
Summary of entity response
16. The proposed audit report was provided to the NDIA and an extract of the report was provided to the Commonwealth Ombudsman. The NDIA’s summary response is reproduced below, and the full response is included at Appendix 1. The Commonwealth Ombudsman did not make comments on the extract received. Improvements observed by the ANAO during the course of this audit are listed at Appendix 2.
National Disability Insurance Agency response
The National Disability Insurance Agency (NDIA) appreciates the Australian National Audit Office’s (ANAO) conclusion that the NDIA has a largely fit for purpose complaints management framework. Also noted is the ANAO’s observation that the NDIA’s complaints processes largely align with the key design elements of the Commonwealth Ombudsman’s better practice principles, and the ANAO’s acknowledgement of the Commonwealth Ombudsman’s advice that the maturity of the NDIA’s complaints processes is rated four (‘established’) out of a possible five (‘superior’).
Having regard to this, the NDIA has reservations about the overall finding in the report that the NDIA’s management of complaints (including its reporting and evaluation/review) is partly effective. Rather, the NDIA considers that its complaints management system has sound fundamentals and, overall, is effective in the way it responds to, monitors, quality assures and learns from complaints.
The NDIA remains committed to ongoing improvement and strengthening of its complaints system. The NDIA acknowledges and agrees with the identified recommendations and notes the opportunities for improvement identified in the report, and has commenced action to implement those recommendations.
Key messages from this audit for all Australian Government entities
Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.
Performance and impact measurement
1. Background
Introduction
1.1 The National Disability Insurance Scheme (the NDIS) was established in 2013 under the National Disability Insurance Scheme Act 2013 (NDIS Act) to provide funding for reasonable and necessary supports for people with permanent and significant disability. The NDIS is jointly funded by the Australian, state and territory governments under bilateral arrangements.
1.2 The National Disability Insurance Agency (NDIA) is the Australian Government entity responsible for delivering the NDIS. It is a corporate Commonwealth entity under the Public Governance Performance and Accountability Act 2013. In 2022–23, the NDIA reported that it employed a total of 5,652 staff with 20.8 per cent of those having a disability.
1.3 If a person meets the age, residency and disability or early intervention access requirements, they may qualify to enter the NDIS. Once approved to enter the NDIS, these individuals (participants) have their supports costed and funded via a support plan (participant plan). In 2022–23, the NDIA reported participant plan expenses under the NDIS totalled $35.2 billion and covered more than 610,000 participants.2 The NDIS is expected to grow to 1,017,522 participants and $89.4 billion in 2032.
Complaints management
1.4 The NDIA receives complaints from a variety of sources on such matters as dissatisfaction with supports in a participant’s plan, payment delays, approvals of plan reviews and engagement with NDIA staff. The NDIA reported in 2022–23 that it received 28,951 complaints from participants (compared to 30,091 in 2021–22) and 983 complaints from NDIS providers (compared to 1,446 in 2021–22). Complaints are also received from various other stakeholders including the NDIS Quality and Safeguards Commission, the Commonwealth Ombudsman, parliamentarians, and state and territory agencies. Figure 1.1 shows the top five sources of complaints to the NDIA in 2022–23.
Source: NDIA reporting.
1.5 Complaints officers within the NDIA’s Complaints and Participant Incident Team (CPIT) work in different complaints streams to manage complaints.3
- Complaints resolution — manages the largest volume of complaints received by the NDIA, through the primary complaints email and phone contacts.
- Oversight bodies — manages correspondence, including complaints, from external stakeholders about the NDIA, including participant and provider issues.
- Ministerial (formal and informal) — manages complaints received by the Minister’s office.
- Members and Senators complaints — manages complaints received from members of parliament, senators and their electorate office staff.
- Executive complaints — manages complex or sensitive complaints received by the NDIA Board, Senior Executive Service officers, or through media enquiries and social media channels.
1.6 The NDIA categorises complaints into three levels. Until 1 February 2024, these were:
- level 1 complaints — ‘usually a single issue or concern which is straightforward in nature; they do not require detailed investigation’;
- level 2 complaints — ‘usually more complex by nature and may involve more than one issue and may require some investigation’; and
- level 3 complaints — ‘usually sensitive or complex by nature and often involve multiple issues, parties or business areas ….[and] may require more detailed or formal investigation.’
1.7 On 1 February 2024, the NDIA published the following updated descriptions of complaint levels on its website4:
- level 1 complaints — ‘issues that can be resolved by the staff member who first helps you’;
- level 2 complaints — ‘issues or concerns [that] need investigation and/or coordination to resolve’; and
- level 3 complaints — ‘issues that are complex or require formal resolution.’
1.8 Complaints are recorded in the NDIA’s business IT system (Customer Relationship Management, CRM).5
1.9 The Commonwealth Ombudsman’s Better Practice Complaints Handling Guide sets out the best practice approach that entities can apply in developing a complaints management framework or approach.6 The NDIA’s Complaints and Feedback Framework (the Complaints Framework) dated June 2020 sets out the how the NDIA will manage feedback and complaints on its performance, conduct or processes. The NDIA published an updated version of the Complaints Framework on 6 February 2024. Both versions of the Complaints Framework state that ‘This framework also observes the fundamental guiding principles outlined in the Commonwealth Ombudsman: Better practice guide to complaint handling.’ The principles include fairness, accessibility, efficiency, integration and first contact resolution.
1.10 The NDIA has set service standards for the management of complaints, which form part of its Participant Service Charter (see Table 1.1).
Description of service standard in the Participant Service Charter |
Service standard |
Acknowledge a complaint after we receive it |
1 day |
Make contact after we receive a complaint |
2 days |
Resolve 90% of complaints after we receive it. More complex complaints may take longer to address |
21 days |
Source: NDIA Participant Service Charter.
Rationale for undertaking the audit
1.11 In 2021–22 the NDIA reported that it received 30,091 complaints from participants of the NDIS and 1,446 from NDIS providers; and in 2022–23 it reported having received 28,951 complaints from participants and 983 from NDIS providers. Effective complaints management contributes to transparency, accountability and the continuous improvement of administrative decision-making and service delivery. This audit provides assurance to the Parliament that the NDIA has fit-for-purpose complaints management practices.
Audit approach
Audit objective, criteria and scope
1.12 The objective of the audit was to assess the effectiveness of the NDIA’s management of complaints.
1.13 To form a conclusion against this objective, the following high-level criteria were applied.
- Does the NDIA have a fit-for-purpose complaints management framework?
- Does the NDIA adequately report on complaints, and outcomes, and review its management of complaints?
Audit methodology
1.14 The audit methodology involved:
- assessing the NDIA’s complaints management approach against the Commonwealth Ombudsman’s Better Practice Complaint Handling Guide;
- reviewing and analysing policy documents, guidelines, procedures, reports and relevant files;
- examining the management of a sample of complaints;
- site visit with complaints handling teams in Geelong;
- meeting with relevant NDIA staff; and
- reviewing contributions made to the audit contribution facility on the ANAO website.
1.15 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $396,000.
1.16 The team members for this audit were Barbara Das, Renina Boyd, Susan Penn-Turrall, Sonya Carter, Vivian Turner and Alexandra Collins.
2. Complaints management policy and practice
Areas examined
This chapter examines whether the National Disability Insurance Agency (NDIA) has a fit-for-purpose complaints management framework.
Conclusion
The NDIA has a largely fit-for-purpose complaints management framework. Supporting internal procedural guidance documents were largely clear, complete, and current. Complaints management is monitored through a monthly quality assurance process assessing four key areas: right person; right process; right referral; and right resolution. The quality assurance results varied significantly from month to month across all four areas in the last financial year, with low results against multiple procedures and no action plan to remedy performance. In 2022–23, the NDIA did not meet its 90 per cent target for resolving complaints within 21 days.
Areas for improvement
The ANAO made one recommendation aimed at improving the complaints management quality assurance framework (see paragraph 2.36). The ANAO also suggested that the NDIA improve the visibility of its published complaints policy and process information (paragraph 2.9) and consider updating its contact and feedback form (paragraph 2.10).
2.1 The Commonwealth Ombudsman’s Better Practice Complaint Handling Guide (Better Practice Guide)7 states:
Good complaint handling will also help your agency meet general principles of good administration, including fairness, transparency, accountability, accessibility and efficiency. Analysis of complaint data may identify opportunities for the improvement of administrative practices.
Strong quality assurance frameworks will help balance competing imperatives of quality and timeliness.
Does the NDIA have fit-for-purpose complaints management policies and processes?
The NDIA provides a range of methods to make a complaint. Information about making a complaint is on the National Disability Insurance Service (NDIS) website; it is not prominent in all sections of the website relating to contacting the agency or available in languages other than English. The NDIA’s complaints processes largely align with the key design elements of the Commonwealth Ombudsman’s better practice principles. Supporting internal procedural guidance documents were largely clear, complete, and current.
Methods of making complaints
2.2 The NDIA publishes clearly written information about its complaints and feedback process on the NDIS website8, including guidance on how to make a complaint, how the NDIA manages complaints and what to do if dissatisfied with the complaint outcome. A complaint to the NDIA can be made in the following ways:
- visit a local NDIA office and speak in person with staff, such as a planner;
- call the NDIA’s National Contact Centre (NCC);
- contact the NDIA via TTY9 or the National Relay Service;
- live chat online with NDIA staff;
- use the contact and feedback form on the NDIS website;
- post or email to enquiries@ndis.gov.au;
- contact a member of parliament or senator; and
- use the online participant or provider portals.
2.3 In May 2023, a new enquiry phone line was established for the office of the Minister for the NDIS to call a specifically assigned team within the NCC directly. The NDIA advised the ANAO in August 2023 that the new phone line was intended to facilitate a direct connection with the minister’s office. Complaints received via the enquiry line are categorised as complexity level three and referred to a complaints officer (see paragraphs 1.5 to 1.7 for complaints streams and complexity levels).
2.4 The NDIA accepts complaints made anonymously (for example, by phone or not including a last name when using the online form). Guidance directs NCC Customer Service Officers to inform complainants that, depending on the matter, this can limit effective resolution as NDIA staff may be unable to identify all information needed to investigate the complaint.
2.5 Some complaints made to the NDIA are matters for other agencies to resolve. For example, complaints about NDIS service providers should be made to the NDIS Quality and Safeguards Commission (the NDIS Commission), and complainants dissatisfied with the outcome of a complaint managed by the NDIA, or how the NDIA managed it, may ask the Commonwealth Ombudsman to investigate. The NDIA has a ‘no wrong door’ policy for complaints made to the agency and accepts transfer of complaints initially made to other agencies that relate to the NDIA. The NDIA has protocols for inter-agency complaints transfers (see paragraph 2.23).
2.6 The information about making a complaint on the NDIS website10 is not featured on all NDIS website pages relating to contacting the agency. During audit fieldwork, the ‘Feedback and complaints’ webpage did not refer to the option of making a complaint via online portals or by post. On 1 February 2024, the NDIA added a link on the webpage to a new ‘NDIS Enquiries, Feedback and Complaints Policy’ document which refers to these channels. The NDIS website includes key participant information in a variety of languages, including Auslan videos, but this does not include complaints and feedback information.
2.7 The NDIA’s online contact and feedback form allows complainants to identify that their message is about ‘feedback or complaint’, without needing to specify one or the other, and enter free text details. The form also allows participants to select one of the six options given (four relate to seeking support, one to feedback and complaints and one to other general enquiries). During audit fieldwork, the NDIA’s definitions of ‘complaint’, ‘feedback’ and ‘enquiry’ from its internal policy document, as set out below, were not published:
Complaint: An expression of dissatisfaction indicating an experience with the NDIA or a related entity is displeasing or unacceptable and requires a resolution.
Feedback: Feedback is the articulation of an observation made by a participant or their representative on their experience which may take the form of a suggestion, opinion or compliment.
Enquiry: The act of requesting information, knowledge or action.
2.8 The NDIA’s new NDIS Enquiries, Feedback and Complaints Policy includes the following definitions:
- Enquiry — requesting information, knowledge, or action.
- Feedback — sharing your experience, including observations, suggestions, opinions, or compliments.
- Complaint — expressing dissatisfaction about an experience with the NDIA that was displeasing or unacceptable and requires resolution or response.
Opportunity for improvement |
2.9 The NDIA improve linkages on the NDIS website to provide greater visibility to the published complaints policy and process information. 2.10 The NDIA consider updating its contact and feedback form to allow complainants to self-identify that they have a complaint (rather than feedback), and to more easily identify where their matter is both a complaint and a request for access or support. |
2.11 The NDIA identified that some participant groups make disproportionately low numbers of complaints. For example, internal reporting in December 2022 stated, ‘Participants who have identified as First Nations Peoples or culturally or linguistically diverse are less likely to submit a complaint than other participants.’ The NDIA has not identified the causal drivers for such results, including whether these cohorts may experience barriers to submitting complaints.11
2.12 As part of its Complaints Enhancement Project (CEP) (see paragraph 3.30), the NDIA sought feedback on complaints management from its Disability Representative and Carer Organisations (DRCO) Forum in June 2023 and the Participant Reference Group (PRG) in August 2023.12 The feedback included that complaints can be difficult to make, and that the NDIA could provide more easy read documents, clarify there is no negative impact for participants who complain, and consider what channels and support for making a complaint are available for people who are not currently making complaints.
2.13 As part of the CEP, the NDIA drafted a new complaints policy document, including an easy-read version that was published on the NDIA’s website on 1 February 2024 and included advice that ‘no detrimental actions will be taken against people, or their representatives, who raise issues or concerns with us.’ The CEP’s Action Plan included the following activities relevant to the accessibility of the complaints management practices — no timeframes were specified for this work:
Developing and publish translations of the public policy in recommended culturally and linguistically diverse (CALD) languages
Investigating translations in First Nations languages (in line with new NDIS First Nations Strategy13)
Investigating Auslan translation
Investigating an explanatory video for the NDIS website.
Complaints management process
2.14 Complaints made via email or the NDIA’s online form go to the NDIA’s feedback inbox. The NCC monitors this inbox and assesses all content to determine if it is a complaint, the complexity level and any risks involved. The NCC also monitors complaints made via the participant or provider portals.
2.15 NCC Customer Service Officers receive complaints made to the NDIA’s phone line or via the online chat functionality. Other NDIA frontline staff such as planners or NDIA Partners (also known as Local Area Coordinators) may receive complaints directly through the course of their interactions with participants, for example, during a planning meeting or follow-up telephone conversation with a participant.
2.16 The key initial steps for frontline staff receiving a complaint are set out in Table 2.1.
Step |
Description |
Identify |
Identify the matter as a complaint and the key issues. |
Assess |
Assess the complaint’s complexity (level 1, 2 or 3) (see paragraph 1.6 for description of complaint complexity levels). Assess any risks involved in the complaint (e.g., participant will soon run out of support funds) and whether a plan review or reassessment is needed. Record details in the NDIA’s business IT system (CRM or PACE).a |
Acknowledge |
Acknowledge receipt of complaint within one day (complaints received via email are automatically acknowledged).b |
Resolve |
Where the complaint is complexity level 1, resolve complaint where possible and record outcome and closure of complaint within CRM.c |
Refer |
Refer complaints of complexity 2 or 3 to the NDIA’s Complaints and Participant Incident Team and alert them to identified risks.d |
Note a: See footnote 5.
Note b: Where a complaint is received via the Minister, Member of parliament or a Senator, the NDIA acknowledges receipt to the Minister, Member or Senator within one day and to the complainant within two days.
Note c: Some complaints categorised as level 1 complexity may still be referred to a business area for quick resolution and in such cases frontline staff (including NCC Customer Service Officers) may still need to make initial contact within the two-day service standard timeframe.
Note d: Some complaints are out of scope for the complaints teams and the NCC refers directly to the relevant business area, for example, complaints about NDIA compliance with its Model Litigant Obligations are referred to the NDIA’s Legal Services area for assessment and management.
Source: ANAO analysis, based on NDIA documentation.
2.17 The Complaints and Participant Incident Team (CPIT) manages complexity level two and three complaints and is comprised of complaints officers in the following streams: Complaints Resolution; Executive Complaints; Ministerial Formal14; Ministerial Informal15; Members and Senators Complaints Officers (MaSCO); and Oversight Bodies.16 Complaints managed across all streams are subject to the same 21-day resolution service standard (see paragraph 1.10).
2.18 Assigned officers within CPIT monitor the team’s complaints inbox and:
- carry out a further risk assessment to prioritise complaints for CPIT action;
- contact complainants (where not already done by NCC17) — to meet the NDIA’s two-day contact service standard, provide a reference number and outline the complaint process; and
- allocate complaints to CPIT officers.
2.19 NDIA internal procedures describe the actions complaints officers undertake once they are allocated a complaint. A summary of the key steps undertaken is set out in Table 2.2.
Key steps |
Detail |
Preliminary checks |
|
Investigation |
|
Contact complainant |
|
Issue requests for action (RFA) |
|
Review RFA responses and determine outcome |
|
Prepare to close complaint |
|
Contact complainant and close complaint |
|
Note a: Where a complaint involving a participant is made by a third party, NDIA procedures direct staff to check the participant’s record for consent to share information with the complainant, or to directly seek the participant’s consent to do so.
Note b: The NDIA communicates outcomes directly with complainants, including where the complaint has been referred informally from the Minister’s office or is received from an MP or senator. Where the complaint is received via a formal letter to the Minister for the NDIS, the outcome of the complaint is either communicated to the complainant in a formal letter from the Minister, drafted by the NDIA, or in a formal letter from the NDIA where the Minister directs the NDIA to respond on their behalf.
Source: ANAO analysis of NDIA documentation.
Policies and procedures
2.20 The NDIA’s June 2020 NDIS Complaints and Feedback Framework (the Complaints Framework) sets out its policy for complaints management, including definitions of complaints, feedback and enquiries, complexity levels, and complaints roles and responsibilities across the NDIA. The Complaints Framework states that it observes the principles in the Better Practice Guide and refers to the NDIA’s Participant Service Charter principles on how it engages with participants and the public. The Participant Service Charter principles are: transparent; responsive; respectful; empowering; and connected. The Complaints Framework includes the NDIA’s Participant Service Charter commitments to the following complaints management steps:
- acknowledge a complaint within one day of receipt;
- make contact within two days of receipt;
- resolve 90 per cent of complaints within 21 days of receipt (more complex complaints may take longer to address);
- keep the complainant informed about the progress of their complaint at every stage; and
- contact the complainant or their representative to talk about their complaint to better understand it.
2.21 In February 2024, the NDIA updated its Complaints Framework. The new framework includes the same key elements outlined in paragraph 2.20, except it included alternative guidance on making contact with complainants (see paragraph 2.23).
2.22 As at June 2023, the NDIA had 46 internal guidance documents and supporting templates relating to complaints management, located on the intranet in a clearly defined and accessible section.
2.23 The NDIA’s key policy and procedure documents for complaints management were largely complete, current, consistent and clearly written.
- At the time of audit fieldwork, the Complaints Framework had not been updated since 2020, and did not accurately reflect the NDIA’s timeliness service standards, which now refer to calendar days rather than ‘business days.’ The NDIA removed the reference to ‘business days’ in the February 2024 version of the Complaints Framework. The new version states that after receiving contact, the NDIA aims to ‘contact you within 2 days of acknowledgement’ — this is inconsistent with the NDIA’s commitment under its Participant Service Charter to making contact within two days of receipt of the complaint.
- There were inconsistent approaches to responding to complaints about NDIS service providers. NCC guidance directs Customer Service Officers to record complaints, advise callers to complain to the NDIS Commission and offer to lodge complaints with the NDIS Commission on the caller’s behalf. The Complaints Framework directs staff to record any complaints about service providers, advise the complainant to follow the relevant provider’s own complaint processes and then, if necessary, make a complaint to the NDIS Commission.
- The Participant Service Charter states participants will be kept informed about the progress of their complaints at every stage. The February 2024 version of the Complaints Framework reflects this commitment; other internal procedural guidance does not include explicit steps or guidance to support this beyond the initial contact service standards (one and two days).
- While key procedures include steps for preliminary investigations that are likely to be sufficient for most complaints, they did not include guidance to support the more complex investigations. Further guidance, including a broader range of examples for reference, could be added to existing procedures to better support consistent identification of complaint issues, potential remedies and effective planning of more complex investigations.
Alignment with the Commonwealth Ombudsman Better Practice Guide
2.24 The Better Practice Guide sets out eight design principles18 for a better practice complaints management framework. Table 2.3 sets out the extent to which the NDIA’s complaints management framework reflects each of these principles.19
Design principle |
Alignment |
ANAO comment |
The framework should be user-centred, simple to access and easy to use. |
◑ |
Complaints can be made through multiple channels. Information about complaints and feedback is not available on the NDIA’s website in languages other than English (see paragraph 2.13 as to the NDIA’s plan for translations). The NDIA has not assessed if specific cohorts of participants experience barriers to making complaints (paragraph 2.11). |
The framework should support early resolution of complaints. |
● |
The NDIA has set complexity levels to support triaging of complaints for early resolution and service standards to promote timely complaints resolution. |
The framework should be integrated within the overall corporate structure. |
● |
NDIA policy identifies roles and responsibilities for staff across the NDIA relating to complaints management. Central complaints teams have established liaison and escalation links across the NDIA and externally with stakeholders to address complaints. |
Complaints should be recorded in an electronic system capable of producing complaint insights. |
◕ |
Complaints are recorded in CRM and categories of information listed in the Better Practice Guide are captured. CRM has some limitations over data controls and reporting (see paragraph 3.17 to 3.25). |
Complaint handling should be supported by clear process guidance. |
◕ |
Largely complete and clear complaints policy and procedures are accessible on the staff intranet (see paragraph 2.23). |
All staff should have the skills and support they need to deliver better practice. |
● |
The NDIA monitors completion of specific training for complaints officers and complaints management related training provided to other frontline staff (see paragraph 2.27). |
The system should have robust quality assurance and review processes.a |
◑ |
The NDIA does not have a mature complaints quality assurance process (see paragraphs 2.34 and 2.35). NDIA review of the framework is discussed in Chapter 3 (from paragraph 3.26). |
Key: ○ Not aligned ◔ Partly aligned ◑ Half aligned ◕ Largely aligned ● Fully aligned
Note a: The NDIA’s NCC has a separate Quality Assurance Framework and assesses a random sample of calls monthly. The framework applies to calls generally, not just complaints. Call quality is assessed against a range of customer-focused attributes (for example, communicating in a professional and polite manner); business-focused attributes (for example, using the correct procedure); and regulatory attributes (for example, correctly following privacy protocols). Consideration of the NCC framework was not undertaken in assessing alignment against the quality assurance process design principle set out above.
Source: ANAO analysis.
2.25 In addition to key design elements, the Better Practice Guide sets out eight steps that high-quality complaints processes should include. The Complaints Framework includes these steps:
- identify and record a complaint;
- acknowledge the complaint;
- assess and triage;
- resolve early or investigate;
- consider potential remedies;
- communicate the outcome;
- finalise; and
- feedback systemic issues.
2.26 In April 2021, the NDIA responded to a voluntary survey issued by the Commonwealth Ombudsman. The survey responses were used by the Commonwealth Ombudsman to benchmark the maturity levels of participating agencies’ complaints handling processes against its Better Practice Guide. In February 2022 the Commonwealth Ombudsman advised the NDIA that its processes rated as four, ‘established’ (against a maximum of five, ‘superior’). The Commonwealth Ombudsman identified that the NDIA could enhance its continuous improvement processes to raise its maturity level (see paragraph 3.27).
Complaints training
2.27 New complaints officers are required to complete mandatory corporate NDIA training modules, such as privacy and information handling, and a separate complaints induction course, completed over the first two weeks on the job. Training completion rates reported by the NDIA are set out in Table 2.4 below.
NDIA roles |
Training required |
Completed as at 5 January 2024 |
NCC customer service officers
|
NDIA mandatory training, for example privacy and information handling and contemporary disability rights. |
|
NCC induction training modules, including a complaints module. |
|
|
Continuous improvement training (known as CI Connect) — complaints handling |
|
|
Complaints officers
|
NDIA mandatory traininga |
|
CPIT induction training |
|
|
Continuous improvement training (known as CI Connect) relating to complaints handling |
|
|
Other NDIA staff, such as planners, local area coordinators
|
NDIA mandatory traininga |
|
Planning essentials programb |
|
|
CI Connect — complaints handling |
|
|
Note a: NDIA mandatory training applicable to all APS staff, labour hire, contractors and partners comprised of 14 modules, categorised into two groups: ‘Mandatory — it’s the law’; and ‘Mandatory — our ways of working’. NDIA completion rates are recorded as average rates of completion across the modules in both groups.
Note b: Planning Essentials Program includes brief references to staff roles in complaints, such as the need to check for any complaints ahead of participant reassessments. The NDIA refreshed this suite of training in October 2023 — while complaints was still not a focus, the refreshed program includes additional reminders to staff in frontline roles, for example to support the ‘no wrong door’ principle by providing a warm transfer of complaints to other NDIA business areas.
Source: ANAO, based on NDIA documentation.
2.28 New starters in CPIT are assigned another officer as a ‘buddy’ for on-the-job guidance and are required to manage five complaints (two with assistance and three independently) before being rated as competent. The CPIT Continuous Improvement Framework (the Quality Framework) requires further checks of complaints officers’ first 10 independently completed complaints to assess proficiency. These checks occur before the complaints are finalised. Once new complaints officers reach 80 per cent proficiency their complaints are no longer checked before closure. A sample of all complaints officers’ completed complaints are checked monthly as part of the NDIA’s quality assurance approach (see paragraph 2.31).
2.29 For new Customer Service Officers, the NCC induction training includes a complaints module. The NDIA advised the ANAO in September 2023 that Customer Service Officers must complete the NCC complaints induction module before they are assigned to taking calls. Complaints officers can undertake additional periodic training as available, for example in June 2023, 16 complaints officers participated in the Commonwealth Ombudsman’s annual Complaint Handling Forum.
2.30 In September 2023, as part of its work on the CEP, the NDIA’s contracted provider, Nous Group, proposed additional training be provided, such as in communication and writing skills and managing difficult conversations. Nous Group prepared a separate training package on feedback and complaints. As at January 2024, the training had not yet been delivered.
Are complaints managed in accordance with NDIA complaints management policies and procedures?
The NDIA undertakes monthly quality assurance checks of its complaints management. There are no internal targets for the results of the quality checks. The significant variability in the results limits the usefulness in assessing compliance with NDIA policy and procedures or identifying priority areas requiring improvement. In a targeted sample of 30 complaints, the recorded actions taken were largely consistent with the requirements of the NDIA’s policies and procedures.
NDIA monitoring of complaints management quality
2.31 The NDIA’s Quality Framework includes a supporting Quality Framework and Checklist document (the Quality Checklist). The Quality Checklist states that the ‘quality measures to managing complaints within the NDIA are based on four key focus areas of compliance’:
- right person — consent and proof of identity confirmed;
- right process — complaint issue identified and recorded, all actions recorded with a clear resolution, processes followed, correct templates applied;
- right referral — complaint issue clearly summarised, actions recorded, appropriate risk rating applied, correct process and template applied; and
- right resolution — resolution achieved and recorded, complaint outcome or closure communicated, followed up where required.
2.32 The Quality Framework describes the NDIA’s assurance approach for complaints management: monthly quality audits; a quality self-assessment by complaints officers; pre-completion quality checks for new officers; post-completion quality checks; a program of ‘deep dive’ audits; and use of a Continuous Improvement Register for monitoring implementation progress (see paragraph 3.38). The NDIA has not fully implemented the Quality Framework; monthly quality audits and deep dives are not undertaken.20
2.33 The NDIA undertakes post-completion quality checks using a random sample to inform coaching and continuous improvement. The Quality Framework states that an average of at least one check per week is undertaken by senior complaints officers, team leaders, and assistant directors for each complaints officer. The Quality Framework requires results be reported monthly to the executive and managers of complaints teams for discussion in team meetings. Appendix 3 sets out the mandatory complaints management steps assessed within the monthly post-completion quality checks.
2.34 The NDIA has not documented the process to collate the random sample for the post-completion quality checks each month. The NDIA also had not documented its approach to ensuring consistent application of the Quality Checklist, including guidance on examples of qualitative judgement calls required to be made during the quality assurance process.
2.35 The NDIA reports on its post-completion quality checks through monthly quality reports. The NDIA has not set targets for quality check results and does not use data from its monthly reports to analyse trends. The aggregated results of the NDIA’s monthly post-completion quality checks for 2022–23 against the four compliance areas are set out from Figure 2.1 to Figure 2.4 below. Five of the 12 reports in 2022–23 were marked draft, and there was no evidence of NDIA publishing a final version of these reports. The results vary significantly across the financial year for all four key compliance areas. The NDIA has not analysed the drivers of such variability in results.
Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).
Source: ANAO analysis of NDIA monthly quality reports.
Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).
Source: ANAO analysis of NDIA monthly quality reports.
Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).
Source: ANAO analysis of NDIA monthly quality reports.
Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).
Source: ANAO analysis of NDIA monthly quality reports.
Recommendation no.1
2.36 The NDIA implement a fit-for-purpose complaints quality assurance framework to support monitoring of quality and continuous improvement.
National Disability Insurance Agency response: Agreed.
2.37 The NDIA is committed to delivering a quality complaint service. This means genuinely listening to participants, providers and everyone who makes a complaint, and harnessing complaint feedback to drive a culture of continuous improvement.
2.38 A program of work is underway to uplift the quality of complaint services. This program has been underpinned by co-design with the disability community and informed by a review of hundreds of past complaints. The resulting improvement plan and strategic planning activities builds on the sound fundamentals of the agency’s complaints system.
2.39 In this context, the NDIA has recently revised its Complaints Management Framework and, in February 2024, introduced a new NDIS Enquiries, Feedback and Complaints Policy, which is now published on the Agency’s website. Looking forward, as part of its commitment to quality and continuous improvement, the NDIA will be reviewing its complaints quality assurance framework to ensure it is fit for purpose, and uplifting its data analysis, monitoring and reporting capabilities so that complaint insights are more effectively embedded into stronger complaint practices and service delivery. The NDIA will continue to engage with participants and their representatives to improve its complaints practice into the future.
ANAO review of a sample of complaints
2.40 The ANAO reviewed a targeted sample of 30 complaints received by the NDIA in 2022–23 and assessed whether they complied with key steps within NDIA’s policy and procedures that align with the Better Practice Guide process steps (see paragraph 2.25). The complaints selected by the ANAO were those managed by complaints officers in the Complaints Resolution team (see paragraph 2.17) and included complaints received by email (15), phone (11), online form (one), portal (two) and in-person (one). Five of the sampled complaints were categorised as level one (simple), 20 as level two (complex), and five as level three (more complex or sensitive) complaints. The ANAO’s observations are set out in Table 2.5.
Key process step |
ANAO observations |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note a: The NDIA’s acknowledgment emails included most of the content suggested by the Better Practice Guide.
Note b: While the NDIA’s process includes assessing the complexity and risk of complaints, this step is not visible in the NDIA’s CRM system and was not reviewed by the audit team beyond consideration of the complexity level.
Note c: The Better Practice Guide recommends complaints processes include a final step, ‘feed back systemic issues’ raised by the complaint. Sampled complaints were not assessed against this step, as the audit separately assessed the NDIA’s approach to identifying and sharing systemic issues for continuous improvement purposes, discussed in Chapter 3 (discussed from paragraph 3.38).
Source: ANAO analysis.
2.41 The Complaints Framework requires staff to categorise complaint outcomes in accordance with the NDIA’s data dictionary (a glossary of specific terms to support consistent data entry in the NDIA’s business IT system) (see paragraph 3.20). The most common outcome category recorded was ‘desired action completed’ (26 of 30 complaints). Outcomes recorded across the remaining four sampled complaints were: formal reply endorsed and issued; unable to contact – record closed; referred to NDIS Quality and Safeguards Commission; and withdrawn by participant or representative.
2.42 The CRM record also contains free text descriptions of the outcomes of complaints. Recorded outcomes were supported by the outcome descriptions entered by NDIA staff for 25 of the 30 sampled complaints (for four complaints, the outcomes were not recorded according to NDIA guidance and for one complaint the outcome recorded did not accurately reflect the actual outcome). For all 30 complaints, the recorded description of the outcome provided to the complainant indicated the complaints officer addressed the key issues raised in the complaint. In 28 of the 30 sample items reviewed, the outcome description reported that one or more remedies were provided to the complainant.21 The ANAO categorised the descriptions of the remedies provided as follows:
- better explanation or provision of additional information — 16 of 28 complaints;
- corrective action, such as assigning a new Local Area Coordinator — nine of 28 complaints; and
- an apology — seven of 28 complaints.
2.43 The 30 sampled complaints were resolved within a median of seven days and an average of 10 days, with:
- 28 resolved within 21 days, including;
- two resolved within one day;
- 16 resolved within seven days; and
- two taking more than 21 days to resolve (27 days and 33 days). For one of these, the outcome was to inform the complainant they could make a time-sensitive application for an independent review of the decision they were dissatisfied with.22
2.44 The eight complaints recorded as not having included investigation were resolved within a median and average of five days. Timeliness of complaint resolution is discussed from paragraph 2.54.
2.45 The ANAO observed:
- two complaints were resolved within one day and did not require any update to the complainant;
- for eight of the 28 complaints that took more than one day to resolve, complainants were contacted at least once between making the complaint and when it was resolved (see paragraph 2.23); and
- for the two complaints that took longer than 21 days to resolve (out of the 28 complaints that required an update), no updates were provided to complainants.
Complaints about complaints handling
2.46 The NDIA produces a monthly report on complaints received in the prior month about those staff who managed the initial complaint, known by the NDIA as ‘complaints about complaints handling’. In its biannual internal NDIS Complaints and Participant Critical Incidents Analysis (Complaints Analysis) report, the NDIA identified complaints about complaint handling as one of the top 15 complaint reasons for 2022–23, reporting that it had received:
- 384 complaints in the six months to 31 December 2022 (three per cent of all complaints in the period); and
- 329 complaints in the six months to 30 June 2023 (two per cent of all complaints in the period).23
2.47 Reasons for complaints about complaints handling varied and were not consistently described or categorised in the NDIA’s monthly complaint handling reports.
Commonwealth Ombudsman’s reporting on NDIA complaints
2.48 The Commonwealth Ombudsman independently investigates complaints by members of the public about Australian Government agencies, including the NDIA. Complaints are to first be made to the relevant agency, and the Commonwealth Ombudsman then investigates where the complainant is dissatisfied with the way the agency managed the complaint or its outcome.
2.49 In its December 2022 submission to the Joint Standing Committee on the NDIS inquiry into the Capability and Culture of the NDIA, the Commonwealth Ombudsman stated:
In 2021-22, the OCO received 858 complaints about the NDIA. This is a 16 per cent increase in complaints received compared to 2020-21 (741). During the same period the number of NDIS participants increased by 15 per cent. The most common issues raised in complaints were plans (30 per cent), reviews (15 per cent), and service delivery (15 per cent). Within those issues, complaints concerned:
- Plans—the final plan outcome, and dissatisfaction with the plan amendment and implementation process.
- Reviews—dissatisfaction with the process and decision made with the review of a decision, and the process associated with a participant-requested plan review.
- Service delivery—dissatisfaction with the process of NDIA’s handling of complaints made to its complaints service.
In 2021-22, the NDIA was the fourth largest (6 per cent) source of complaints to the OCO in its general Commonwealth jurisdiction. The largest was Services Australia–Centrelink (52 per cent).
2.50 In its 2022–23 Annual Report, the Commonwealth Ombudsman reported the NDIA remained one of its top five Australian Government agencies by number of complaints received, with six per cent of complaints received relating to the NDIA.
2.51 In May 2023 the NDIA received a report from the Commonwealth Ombudsman summarising complaints it had received about the NDIA for the period July 2022 to March 2023, noting that complaints did not always require an investigation. For example, complaints could be finalised without investigation where the Commonwealth Ombudsman was able to assure complainants that the NDIA had acted in a reasonable manner, where complainants had not previously complained to the NDIA, or where complainants had an appropriate review pathway.
2.52 Between July 2022 and March 2023, the Commonwealth Ombudsman reported in its NDIA liaison meeting report that it had finalised 565 complaints relating to the NDIA, of which 44 were investigated, 23 were transferred to the NDIA and 498 were finalised without investigation. Figure 2.5 represents the outcomes of the complaints investigated by the Commonwealth Ombudsman.
Source: Commonwealth Ombudsman report to the NDIA.
Citizen contributions
2.53 As noted at paragraph 1.14, the ANAO received eight citizen contributions to the audit. Six contributions, including one from a peak body representing disability service providers, included comments on NDIA complaint management practices across the following themes:
- the NDIA focusing on timeframes more than effective resolution;
- lack of connection from the NCC back to relevant business areas able to resolve issues;
- inadequate investigation processes;
- poor communications, including a disrespectful approach and errors;
- lack of skills and expertise in complaints handling;
- use of private numbers to contact complainants — meaning complainants might not take the call24;
- delays in complaint resolution causing disruptions in service delivery; and
- poor links (not a warm transfer) of complaints between agencies.
Are complaints resolved within required timeframes?
The NDIA reports quarterly averages for performance against the 21-day complaints resolution service standard. Based on the quarterly averages for the financial year, the NDIA met its 90 per cent target for this service standard in 2021–22 (achieving 94.5 per cent) and did not meet the standard in 2022–23 (achieving 85.8 per cent). Performance against the 21-day standard was 71 per cent for the December 2023 quarter. In 2022–23, the NDIA’s reported performance against the service standard of one day to acknowledge the complaint was 87 per cent. The reported performance for the service standard of contacting the complainant within two days was 96.8 per cent in 2022–23. Both results were below the NDIA’s 100 per cent target for these service standards.
2.54 As discussed at paragraph 1.10, the Participant Service Charter includes timeliness service standards for the management of complaints, which are set out in Table 2.6.
Service standard |
Timeframea |
Acknowledge a complaint after we receive it. |
1 day |
Make contact after we receive a complaint. |
2 days |
Resolve 90% of complaints after we receive it. More complex complaints may take longer to address. |
21 days |
Note a: Timeframes are in calendar days.
Source: NDIA Participant Service Charter 2023.
External reporting
2.55 The NDIA reports on its performance against the 21-day timeframe in the NDIS Quarterly report to disability ministers25, which are published on the NDIS website. These reports provide a view of the NDIA’s overall performance and operations over a three-month period, including complaints. The reported results of the NDIA’s performance for the 21-day timeframe for each quarter from 1 July 2021 to 30 September 2023 is set out in Figure 2.6.
Source: ANAO representation of data in the NDIS Quarterly report to disability ministers, September 2023.
2.56 While the NDIA reports quarterly results, it does not report the result for a financial year. Based on the quarterly averages, the NDIA met its target for the service standard in 2021–22 resolving 94.5 per cent of complaints within 21 days. In 2022–23 it did not meet the target, resolving 85.8 per cent of complaints within 21 days. For the December 2023 quarter, the NDIA reported that it met the timeframe in 71 per cent of cases.
2.57 The NDIA does not report in the NDIS Quarterly report to disability ministers or elsewhere on its website the results of its performance against the timeliness standard of acknowledging receipt of a complaint within one day or contacting the complainant within two days. Timeliness results by complaint stream is also not published.
2.58 The December 2023 NDIS Quarterly report to disability ministers states ‘Work is underway to enhance complaint handling practices and systems which will contribute to more timely complaint resolutions in line with the [Participant Service Guarantee] PSG in coming quarters’.26
Internal reporting
2.59 There are a number of internal reports in place across NDIA teams relating to complaints; the two key reports are the monthly and weekly pulse reports.
NDIS monthly pulse reports
2.60 The NDIA Office of the Scheme Actuary prepares monthly pulse reports, which are provided to the NDIA’s Executive and its Board. The reports include the number of new complaints and contain metrics for the NDIA Corporate Plan measures27 as well as performance against the timeframes in the PSG and the Participant Service Charter 21-day timeframe for resolving complaints. Performance against the other two complaints timeliness service standards in the Participant Service Charter are not included in the monthly pulse reports.
Weekly pulse reports
2.61 In addition to reporting on the 21-day timeframe for complaints resolution, the weekly pulse reports28 report on the one-day and two-day timeframes, for complaint acknowledgement and initial contact with the complainant respectively. The NDIA has a target of 100 per cent for both measures. The results for 2022–23 are set out in Figure 2.7. Based on the average results reported weekly, 87 per cent of complaints were acknowledged within one day and in 96.8 per cent of complaints the first contact was made within two days.
Note: The weekly results have been rolled into monthly results.
Source: ANAO analysis of NDIS weekly pulse reports.
Biannual complaints analysis report
2.62 Timeliness results for each of the complaints streams (see paragraph 1.5) against the 21-day complaints resolution service standard for 2022–23 were included in the NDIA’s June 2023 biannual Complaints Analysis report.29 Results are shown in Figure 2.8.
Source: ANAO analysis of the NDIA’s June 2023 biannual Complaints Analysis report.
Remedial action when timeframes are not met
2.63 There are no documented procedures for when timeframes are not met, however the NDIA advised the ANAO in August 2023 that there are activities in place, such as:
- focus days — days when attention to managing complaints takes priority over, for example administrative tasks30;
- moving workforce — moving staff across the different complaints streams to provide additional support, as needed; and
- operational reporting — to monitor the age of complaints.
2.64 The start date for the 21-day service standard is the date the complaint was entered into CRM. In one of the complaints sampled by ANAO, an existing complaint record was closed, and a new complaint record created to reflect a change in the team managing the complaint (an intention for it to be subject to joint NDIA–NDIS Quality and Safeguards Commission investigation from that point). This meant that while the original complaint had not changed or been resolved, the 21-day service standard did not take effect until the date the second complaint was created, five days later. Complaints data integrity is discussed further from paragraph 3.17.
3. NDIA reporting and evaluation of complaints
Areas examined
This chapter examines whether the National Disability Insurance Agency (NDIA) adequately reports on complaints, outcomes, and reviews its management of complaints.
Conclusion
The NDIA’s reporting and evaluation of complaints management is partly effective. The NDIA reports internally and publishes regular data on complaints statistics and performance against service standards with limitations on data quality clearly identified. The effectiveness of the NDIA’s suite of internal reporting would be enhanced by more timely reporting on complaints outcomes, detailed qualitative analysis and greater use of complaints data alongside other service delivery data to support continuous improvement activities. The NDIA had not fully implemented monitoring and reporting actions identified by the Commonwealth Ombudsman for lifting its complaints approach to a ‘superior’ maturity level. Recommendations from a 2019 review were not fully implemented. The NDIA’s 2023 review of complaints management lacked baseline evidence and a detailed implementation plan.
Areas for improvement
The ANAO made one recommendation aimed at ensuring complaints and service data are regularly analysed to support identification of areas for continuous improvement (see paragraph 3.56). The ANAO also suggested that the NDIA monitors its improvement activities for effectiveness of implementation and evaluation (see paragraph 3.46) and document its methodology and analysis in future case studies or reviews of complaints management (see paragraph 3.54).
3.1 The Commonwealth Ombudsman’s Better Practice Complaint Handling Guide (Better Practice Guide)31 recommends Australian Government agencies have regular internal and public reporting on complaints management, use meaningful complaints data to improve continuously, and review complaint quality assurance and review frameworks every two years. The Better Practice Guide states:
Reflection is the necessary precursor to improvement. Reflective systems are self-critical and ensure complaint handling processes and complaint data are regularly reviewed and analysed. Reflective systems ensure effective actioning of individual complaints, and can identify trends, systemic issues and opportunities for improvement.
3.2 In relation to the types of complaints data agencies should report to management to support oversight of complaints and give insights into broader operations, the Better Practice Guide states:
At a minimum the executive should receive reports about complaint volumes and trends including data about complaint issues, causes and outcomes, systemic issues and relevant complainant data (for example, geographic, demographic, cohort information).
Does the NDIA adequately report on complaints management including implementation of outcomes?
The NDIA reports publicly on volumes of complaints received, percentage of complaints closed within the 21-day service standard, sources of complaints, and complaint reason categories. There is regular internal reporting on complaints timeliness, compliance with policies and procedures and work allocation. Outcomes of ministerial complaints and complaints about the NDIA’s handling of a complaint are reported regularly, outcomes of other complaint categories are reported twice per year. The NDIA does not report on implementation of complaint outcomes that require further actions after the complaint was closed. Regular internal reports contain insufficient qualitative analysis of trends or comparison with other service delivery performance data.
Internal reporting of complaints management
3.3 The NDIA produces 30 ongoing reports that include complaints data. These vary by audience in the frequency and level of detail of complaints data included. The NDIA’s Board and Senior Leadership Team (SLT) receive entity-level performance reports (pulse reports) with summary level complaints data for their monthly meetings. The SLT also receives shorter, weekly versions of the Pulse reports. These reports do not include data on outcomes of complaints or complaints management quality. The managers and executive of complaints teams receive reports with operational level information and data on a daily, weekly, monthly, and quarterly basis. Table 3.1 shows the content of the reports received by the NDIA’s management and the Board. Further information about report frequency is in Appendix 5.
Reports to: |
NDIA’s board and executivea |
Complaints teams’ executive and managers |
ANAO comment |
Volume of complaints |
✔ |
✔ |
For example, volumes of open and new complaints and type of complainant. |
Reasons for or nature of complaint |
✔ |
✔ |
Reports include the most common categories for complaint reasons, discussed at paragraph 3.4. |
Causes of complaints |
✔ |
✔ |
The reasons categories are included; the systemic issues are not explicitly identified. The reason categories in reports lack sufficient detail to understand what complainants are dissatisfied with.
|
Systemic issues arising out of complaints |
✘ |
✘ |
|
Complaint outcomes delivered by NDIA |
✘ |
✔ |
Regular reports for ministerial complaints and complaints about complaints handling. Twice yearly reports on outcomes for all complaint categories. |
Demographics of complainants |
✘ |
✔ |
For example, age group, disability or state or territory. |
Complaint management quality |
✘ |
✔ |
Data on quality of complaints management is not reported to the Board or Executive, although data on complaints about complaints management are included if they are a top complaint reason. Complaints teams receive reports on quality assurance results and complaints about complaints handling (see paragraph 2.46). |
Complaint management timeliness |
✔ |
✔ |
Reports include proportions of complaints completed within set timeliness standards. |
Key: ✘ Not included ✔ Included
Note a: The SLT receives copies of all reports to the Board with complaints data.
Source: ANAO analysis of NDIA documentation.
Reasons for complaints
3.4 Internal management reporting of complaints reasons includes the most common results in the reporting period. Figure 3.1 below shows the top 15 complaint categories across all complaints for each of 2021–22 and 2022–23.
Note a: ‘s48 decisions’ refers to decisions made under section 48 of the National Disability Insurance Scheme Act 2013 (the NDIS Act) to reassess a participant’s plan, either at the participant’s request or the Chief Executive Officer’s initiative.
Source: ANAO analysis based on NDIA documentation.
3.5 While reason categories are aligned with NDIA functions, they generally did not give insight into causes of the complaint that readily would support continuous improvement opportunities. For example, complaints within the category of Supported Independent Living would require further analysis to understand the nature of or driver for the complaint. Where there are high numbers of certain reason categories, particularly over a period of time, the results may indicate systemic issues (see paragraph 3.48).
Complaints outcomes
3.6 There are 10 categories for complaints officers to choose from when recording a complainant’s desired outcome in CRM.32 For example, ‘action expedited’, ‘agency apology or acknowledgement’, and ‘better explanation.’ There are also ten categories for recording the actual outcomes delivered by the NDIA, for example, ‘desired action completed’, ‘review requested’, ‘unable to contact – record closed’ and ‘formal reply endorsed and issued.’ Analysis of complaint outcome can only be conducted when both the desired and actual outcome are matched together. The NDIA advised the ANAO in September 2023 that where existing categories do not apply, it uses free text within CRM to record the actions taken.
3.7 Complaint outcomes are not reported to the NDIA’s Board or SLT. The executive and managers of complaints teams receive regular reports with complaints outcome data for only two complaint categories: ministerial complaints and complaints about complaint handling. These reports record the outcome categories for listed complaints but do not include accompanying analysis of outcomes.
3.8 The executive and managers of complaints teams also receive the biannual NDIS Complaints and Participant Critical Incidents Analysis (Complaints Analysis) report. This report contains the numbers and percentages for the categories that complaints officers used to record the ‘desired complaint outcomes’ for all complaints in the six-month reporting period, for example, ‘action taken/expedited’, ‘explanation’, ‘apology/acknowledgement’ and ‘other’. The report also contains the numbers and percentages for the categories of ‘actual complaint outcomes’ delivered by the NDIA in the reporting period, for example, ‘desired action completed’, ‘formal reply endorsed and issued’, ‘general’, ‘withdrawn by participant/representative’, ‘internal review’ and ‘other’. In 2021–22, ‘desired action completed’ was the largest category of actual outcomes recorded (69.2 per cent), followed by ‘other’ (16.4 per cent).33 Since the December 2022 edition of the biannual Complaints Analysis report, an appendix is included with mapping of actual outcomes against desired outcomes.34 The only analysis of outcomes contained within the December 2022 and June 2023 biannual reports was the identification of the category of the most frequently desired outcome and the actual outcome category with the highest results in the reporting period.
3.9 When a closed complaint still has actions to be completed by another NDIA team, internal policy requires complaints officers to follow up and check the actions occurred. Complaints team leaders can access daily reports to check details of complaints marked for follow up and teams also receive monthly reports to support monitoring of any implementation actions post-closure. The status or timeliness of implementation of agreed outcomes or actions is not reported to responsible business areas or to the NDIA’s Executive.
Quality of complaints management
3.10 The NDIA’s Executive receives reports with complaints timeliness data but does not receive complaint management quality information. The Executive and managers of complaints teams receive the following reports each month with quality information:
- results of monthly quality assurance checks of random samples of complaints closed within the last month (see paragraph 2.35); and
- monthly reports on complaints of complaint handling (see paragraph 2.46).
3.11 The quality assurance reports also include observations as to what was done well that month and suggestions for improvements to be made based on the findings. Reports on complaints of complaints handling do not include analysis of the results. The details of complaints reviewed within the random sample are provided within the reports to support follow up action by team leaders.
Report consistency
3.12 Reports generally use a consistent structure and metrics for reporting on complaints from one reporting period to the next. Reports typically include trend detail, which is most clearly presented for timeliness data. For some metrics, particularly complaint reasons, reports focus on the most common (top) categories in the reporting period, which limits transparency, reliability and usefulness as results may change from one reporting period to the next.
3.13 Reports include additional correlation of data, such as complainant characteristics against complaint volumes or frequency. For example, the Internal Reviews and Complaints Quarterly Analytics and Insights reports include presentation of the top five complaint themes by top five disability types. With the exception of monthly quality assurance reports, regular reports include minimal accompanying qualitative analysis of results presented to give insights into potential scope for actions to be taken to improve results or outcomes.
3.14 Volume, population and complaint management quality data is regularly reported, and workflow and workloads are regularly monitored within relevant teams. The absence of more timely reporting on outcomes of complaints, implementation of outcomes, qualitative analysis of trends or correlation of complaints data with other service delivery performance data limits the effectiveness of the NDIA’s oversight of the complaints management process.
External reporting on complaints management
3.15 The NDIA publishes regular reports with details of its management of complaints on the NDIS website.35
- NDIS Quarterly report to disability ministers — reports include data on complaint volumes, complainant types, who complaints are about, causes and timeliness of resolution.
- NDIA annual reports (up to 2021–22) — include data on the numbers and percentages of complaints open and closed within the NDIA’s 21-day service standard and the number of complaints transferred to or from other relevant agencies.36
- Quarterly Participant dashboards — a series of 15 reports on cohorts of participants with selected disability types (for example psychosocial, autism and spinal cord injury), including data on the rate at which participants with the disability type make a complaint to the agency.37
3.16 Accessible, including ‘easy read’, versions of the Annual Report and NDIS Quarterly report to disability ministers are available on the NDIS website. Participant Dashboards are published alongside transcripts to support accessibility. Between 2019 and 2021, the NDIA also published several one-off reports about certain participant groups38, for example ‘Young adults in the NDIS’ that included data on complaint volumes and data on complaint resolution timeliness.
Integrity of complaints data
Data sources
3.17 Complaints data is entered into CRM manually by staff, except when complaints are made through the NDIA’s online participant and provider portals, where initial data is automatically recorded in CRM.
3.18 In October 2019, a new module for CRM ‘My Customer Requests’ was implemented with fields to support greater data recording, for example, the ability to record multiple related parties as the source of a complaint. The NDIA reported in the quarter three, 2019–20 NDIS Quarterly report to disability ministers that the previous module, ‘My Feedback’, was discontinued following the introduction of the new functionality. The ‘My Feedback’ module is still in use; the management response to a 2022–23 internal audit stated the planned enhancements to CRM did not occur to remove this module and that staff are instructed to use the newer module. The December 2022 biannual Complaints Analysis report states data was reported using both modules.
3.19 The Internal Reviews and Complaints Quarterly Analytics and Insights reports, Pulse reports and the biannual Complaints Analysis reports included caveats that results did not include data on complaints recorded in the new CRM (PACE). The caveats did not include any detail as to the estimated volume of complaints excluded from the reports. The NDIA’s 30 June 2023 and 30 September 2023 NDIS quarterly report to disability ministers included the same caveats and identified the numbers of PACE complaints excluded from the reported results. In November 2023, the NDIA advised the ANAO that PACE complaints data would be incorporated into reports from the December 2023 quarter onwards.
Data integrity
3.20 The NDIA applied the following mechanisms to support data integrity in relation to reporting on complaints:
- internal guidance to support accurate data entry, including a Standard Operating Procedure, Record and update a complaint, and a data dictionary reference guide to assist staff in accurately categorising complaints data;
- internal reports on CRM compliance39; and
- manual reconciliation of data by the NDIA to support reporting.
3.21 The NDIA identified the following limitations to data accuracy in its biannual Complaints Analysis reports:
Some “complaints” are general enquiries rather than complaints so the total count of complaints may be overstated.
There are known data issues with linking participants to a LAC and/or ECEI partner. A complaint linked to a partner is not necessarily a complaint about that partner.
There are fields in the complaints data that are entered as free text and therefore require text mining and manual verification to categorise. As a result, there is a risk of miscategorising complaints.
Complaints with erroneous or missing dates have been excluded from the timeframes analysis.
Timeframes have been calculated using the recorded date of resolution. A complaint will be recorded as closed once the complaints officer has done everything in their power to resolve the issue although outstanding follow up with other business areas may still need to occur.
Not all complaints made about a provider will be lodged with the Agency. Some complaints about a provider will be lodged with the [NDIS Quality and Safeguards Commission (NSQC)] as participants are encouraged to approach the NQSC first.
3.22 While the NDIA acknowledges that the inclusion of some general enquiries it had received in its internal and external reports may overstate the total complaint numbers, the inclusion in reporting of other types of contact is consistent with the Better Practice Guide recommendation that feedback also be analysed for trends alongside complaints. For the June 2023 biannual Complaints Analysis report, the NDIA included the numbers of complaints with erroneous dates, reporting that this represented 0.03 per cent of total complaints received in 2022–23.
3.23 In the June 2023 NDIS Quarterly report to disability ministers, the NDIA stated that Participant Critical Incidents (PCIs)40 ‘were captured in complaints data in previous reports, this is the first report to separate out PCIs.‘ In the 30 June 2022 NDIS Quarterly report to disability ministers, the NDIA reported having received 36,067 participant complaints in 2021–22. This figure was adjusted in the June 2023 version of the report, to 30,089 with 5,723 PCIs for 2021–22.41 While the change in methodology supports greater transparency of both complaints and PCIs, reported participant complaint numbers prior to June 2023 are not directly comparable with current or future reported results.
3.24 The August 2022 deep dive on complaints categorised as ‘dissatisfied with plan’ for the period January to June 2022 found of 500 records reviewed, 315 (63 per cent) per cent were incorrectly categorised and made recommendations to improve categorisation guidance (see paragraph 3.51).
3.25 In May 2023, the NDIA reported findings from a 2022–23 internal audit of complaints management to its Audit Committee, which included agreed management actions to improve data integrity.42 Appendix 4 sets out the audit findings relating to data integrity and agreed resolutions to be implemented. The internal audit also observed that some complaints were received directly in separate email inboxes managed by state and territory-based teams and ‘these complaints might be recorded and managed in off system databases maintained in Excel Spreadsheets or as interactions created in CRM.’ This observation does not appear in Appendix 4 as it was not the subject of a finding that required a resolution. In July 2023, the NDIA advised the ANAO that planned work to address the inconsistent processes was no longer being progressed.
Does the NDIA effectively review its complaints management framework?
There is no evidence of the NDIA accepting or rejecting the 14 recommended initiatives from a review of its complaint management process undertaken in 2019. However, there is evidence of implementation of two and partial implementation of a further three recommendations. There were recurring themes in the recommendations from the 2019 review and a 2023 internal review. The NDIA is yet to fully implement the action plan resulting from its 2023 internal review. The action plan does not specify timeframes for completion of activities.
Reviews of complaints management approach
3.26 In April 2019, NDIA contracted PwC to review its complaints management framework and processes to support an internal Complaints Improvement Program. In June 2019 PwC completed its program of work and provided the NDIA with a draft report. The draft PWC report proposed 14 initiatives to improve complaints management (see Table 3.2 and paragraph 3.33). There was no evidence of the NDIA receiving a final version of the report. PwC received full payment from the NDIA as specified in the contract.
3.27 As discussed in paragraph 2.26, the Commonwealth Ombudsman advised the NDIA in February 2022 that it was rated as level four maturity, based on its survey response. The Commonwealth Ombudsman identified actions the NDIA could take to raise its maturity level:
- undertake customer satisfaction surveys;
- monitor, evaluate and report on changes implemented because of complaints; and
- share insights internally and externally with other complaint handling agencies.
3.28 The findings of a 2022–23 internal audit of the NDIA’s complaints management processes were reported to the Audit Committee in May 2023. The audit was part of the NDIA’s internal audit assurance program for 2022–23 and was conducted between July and October 2022. The internal audit found ‘the end-to-end complaints management process has adequate and effective controls’ and two management actions were agreed to address findings relating to integrity of complaints data (see Appendix 4) and enhancing the quality assurance of complaints management.
3.29 In March 2023, the Chief Executive Officer (CEO) reported to the Board that they had initiated improvements to complaints management processes in January 2023. The CEO’s report set out the main observations of a high-level review and advised the Board the improvement work would be undertaken in three phases; no timeframes were specified. The CEO’s report did not provide a rationale for the review.
3.30 The high-level review of complaints was completed in March 2023 by a Senior Executive Service Band 2. The review led to the creation of a Complaints Enhancement Project (CEP).43 The CEP planned to incorporate three phases of work to improve culture, capability, processes and the operating model and workflow. The CEP was also intended to include a review of the complaints data dictionary. In July 2023, the NDIA engaged Nous Group to deliver ‘short term specialist expertise to lead staff engagement to inform future state design and continuous improvement activities in phase 2 of the project’. The CEP was formally closed in November 2023, following submission of a closure report to the Executive project sponsor.
3.31 Table 3.2 compares key details of the reviews undertaken of NDIA’s complaints management framework since 2019. The table includes discussion of both reports from NDIA’s 2023 review, as one informed the CEP establishment and the other was a key CEP deliverable.
|
PwC review |
NDIA high-level report leading to the CEP |
Nous Group report for the CEP |
Date of report |
June 2019 |
March 2023 |
November 2023 |
Purpose |
Conduct strategic review of the NDIA’s complaints management framework and processes to determine opportunities for improvement against best practice. |
Provide advice to the CEO on how to improve the NDIA’s complaints management arrangements. |
Key elements across two stages include:
|
Methodology |
Review of documents; consultation with staff, observation, participant groups, workshops and advisory groups, data analysis, review of best practice examples, and assessment against maturity model. |
The report did not document a methodology or baseline evidence. The assessment was informed by meetings with complaints staff and the NDIA advised in November 2023 that it reviewed internal documents and some recent complaints. |
Review of documents, discussions with managers and leaders, workshops and review of best practice. The report did not document its methodology or baseline evidence. |
Cost |
$639,108 (GST inclusive) |
Resourced internally |
$203,500 (GST inclusive) |
Reporting |
Draft report provided to project executive sponsor; draft findings presented to NDIA CEO and Executive; summary of draft report provided to Audit Committee. |
Presented to NDIA CEO and Executive in March; and reported in April to SLT. |
Draft report given to project executive sponsor, and CEP working group. NDIA and Nous action plan provided to the NDIA’s Board in November 2023 for noting. The report included identified ‘improvement activities’ rather than recommendations. |
Themes of key recommendations |
|
|
|
Source: ANAO, based on NDIA documentation.
3.32 While there are differences in each of the reviews’ findings and recommendations, the following themes are recurring:
- clarifying standards for complaints-related terminology;
- moving towards a more participant-centric culture that values complaints;
- improving training of staff across the NDIA;
- making improvements to supporting technology; and
- a focus on improved reporting, monitoring and evaluation for continuous improvement.
Implementation of recommendations
3.33 The 2019 draft PwC report set out 14 initiatives and three implementation options to achieve the practice improvements. In August 2019, the Audit Committee was advised by the Acting Deputy Chief Executive Officer, Corporate Services that:
Management needs to consider how it resources and funds the implementation of the staged implementation, plans, and roadmaps resulting from the project and expert consultant review, as well as communicating these initiatives across the Agency and obtaining support from ELT [Executive Leadership Team, since renamed to SLT] and staff.
3.34 There was no evidence of the NDIA agreeing on an implementation roadmap contained in the draft PwC report or having developed an alternative implementation plan to implement the recommended initiatives. Of the 14 initiatives proposed by PwC, NDIA records showed it had fully implemented two — definition of frequently used terminology and tiered management reporting — and partly implemented the following three initiatives or updated equivalents:
- staff training and development;
- performance management and KPIs; and
- technology upgrades.
3.35 As at September 2023, the NDIA had not fully implemented the Commonwealth Ombudsman’s suggestions to track changes implemented because of complaints or to share insights with other complaints handling agencies. While the NDIA undertakes monthly entity-level participant satisfaction surveys and identified the number of participant complaints as one of 11 leading indicators of participant satisfaction, links between complaints data and satisfaction results are not explicitly analysed or reported. The NDIA does not undertake separate complainant satisfaction surveys.
3.36 In response to the 2022–23 internal audit findings, the NDIA updated its Quality Assurance checklist and supporting procedural guidance in June 2023. The NDIA identified that remaining work to address complaints data integrity was reliant on the introduction of the NDIA’s new IT system, PACE, which commenced national implementation from 30 October 2023 (see footnote 5). The NDIA’s records show the action was closed as at 12 December 2023.
3.37 Nous Group completed its draft review report in August 2023 and provided this to the NDIA project owner. The Nous Group report was not included with CEP papers submitted to the SLT’s August 2023 meeting, in which the SLT approved project recommendations to incorporate the CEP activities of triaging and analysing executive complaints into business-as-usual activities. In September 2023, Nous Group submitted a final Action Plan report to the NDIA, setting out opportunities for improvement and recommended actions to achieve these, including updating the data dictionary. The Action Plan was presented to the NDIA Board in November 2023 for its information. The plan suggested action priorities but did not specify implementation timeframes.
Does the NDIA use its complaints management data to improve processes and service delivery?
While the NDIA completed a ‘deep dive’ analytical review of complaints data in August 2022 in response to an increased trend in a specific category of complaint, no subsequent ‘deep dives’ have been undertaken to investigate complaints data. There is no monitoring or reporting on the effectiveness of improvements introduced to the NDIA’s complaint management processes. The NDIA’s procedures for identifying continuous improvement opportunities for service delivery require consideration of complaints data. Records of the monthly prioritisation of service delivery issues for further investigation and action do not demonstrate that complaints data form a significant element of this approach.
Trends in complaints management
3.38 The NDIA’s Complaint and Participant Incidents Team (CPIT) Continuous Improvement Framework (the Quality Framework) identifies the mechanisms to monitor and continuously improve complaints management quality (see paragraph 2.31).
3.39 The NDIA’s monthly series of post-completion quality checks on complaints started in January 2022 for complexity level three complaints and was expanded in July 2022 to include complexity level two complaints (see paragraph 1.6). The questions used for the checks are focused on compliance with NDIA policy and procedure rather than a qualitative assessment of the quality of the overall response to the complaint (see Appendix 3).
3.40 The results of post-completion checks are aggregated into monthly reports (see paragraph 2.35). Pre-completion checks are not reported. As at September 2023, no deep dive audits of complaints management had been ‘undertaken on various programs of work within the CPIT as identified by data and at the request of the Branch Manager’ as specified in the Quality Framework.44
3.41 The NDIA advised the ANAO in July 2023 that complaints teams reviewed the complaints quality data reports in fortnightly team meetings and that team leaders also considered the data to inform one-on-one feedback sessions with staff. The NDIA did not have records of the outcomes of these discussions. The NDIA has not yet consolidated its monthly analyses over 2022–23 to give a longitudinal view of quality.
Complaints management continuous improvement
3.42 Beyond reviewing reported data, the NDIA advised that its complaints teams applied the following business-as-usual approaches to identifying aspects of complaints management that require improvement45:
- informal staff suggestions for potential improvements — these can be made through the Continuous Improvement Form on the complaints teams’ intranet page;
- discussion of complaints case studies in fortnightly complaints team meetings; and
- internal Community of Practice monthly meetings.
3.43 The NDIA has also implemented the following informal improvements to its complaints management processes since July 2019:
- requiring team leaders to check ‘Requests for Action’ before complaints officers send these out to the responsible business areas;
- sending SMS notifications to complainants prior to calling to ensure they are aware of an incoming call from NDIA complaints officers (which appear as ‘private number’); and
- team leaders reviewing written responses to ministerial complaints before sending, to improve their quality.
3.44 Prior to June 2023, the NDIA did not consistently record improvements it had made to its complaints management approach. A centralised complaints continuous improvement register was established in June 2023. As at July 2023 when the register was provided to the ANAO, it contained one action being implemented and one action that was not endorsed by the relevant supervisor but was marked as ‘in progress’. The focus of the register was to track actions taken in response to proposed improvement initiatives; while there was a column for recording feedback, there was no supporting plan or mechanism for reviewing any changes implemented.
3.45 The NDIA does not undertake structured monitoring of complaints improvement initiatives to determine if these were effective. Changes to the NDIA’s complaints practices, made in response to its March 2023 high-level review, were implemented without an identified baseline against which to assess the impact. Baseline data was not evident in further reports generated through the CEP beyond identification of existing complaints processes (see Table 3.2).
Opportunity for improvement |
3.46 The NDIA consider using the central complaints continuous improvement register as a tool for monitoring both implementation and evaluation of the effectiveness of branch-level improvement activities. |
Link between complaints data and service delivery continuous improvement
3.47 At the time of audit fieldwork, the NDIA’s Complaints and Feedback Framework (Complaints Framework) stated that:
Complaints and feedback present an opportunity to support continuous process improvement through enhancing business practices and processes.
…
Managers and leaders at all levels are expected to cascade and discuss complaint reports with staff and teams to support an understanding of themes and systemic issues and how this links in to improving services for participants.46
3.48 As discussed in paragraph 3.5, while the NDIA has standard reports on complaints data that identify the most common complaint reasons or issues, these do not adequately show causes or potential systemic issues to enable identification of areas for continuous improvement.
3.49 The Performance, Workload, Planning and Quality (PWPQ) branch within the NDIA’s Service Delivery group conducts ‘deep dive’ reviews of selected administrative decisions, emerging issues, processes and participant cohorts to inform continuous improvement in service delivery. The PWPQ branch reviews data from a range of sources relevant to the topic of each deep dive review, including:
- results of monthly quality audits undertaken by PWPQ of approved participant plans;
- results of checks on plan quality by other assurance teams;
- complaints data (including monthly complaints data on volumes, timeliness and top complaints reasons; and detailed quarterly and biannual reports on complaints);
- performance data in monthly Pulse reports (discussed at paragraph 3.3); and
- prior deep dive review analysis.
3.50 The PWPQ branch uses a prioritisation matrix template to capture key issues from the data sources above and support its monthly discussions of priority areas for continuous improvement activity. In completed matrices, data on complaints reasons was not explicitly linked to other quality data to inform analysis of areas needing continuous improvement or to inform selection of areas where further deep dive analyses are needed for understanding.
3.51 The PWPQ branch has completed one deep dive related to complaints — an August 2022 review into complaints with the reason category, ‘dissatisfied with plan’. The deep dive report states the complaints teams had identified an increase in complaints in each quarter of 2021–22, with ‘dissatisfied with plan’ complaints consistently in the top complaint categories and a 50 per cent increase in this category of complaint between quarter one of 2020–21 and quarter one of 2021–22. A sample of 500 complaints with this complaint reason category was reviewed; this recorded category was found to be correct for only 185 of the complaints (37 per cent). The deep dive identified improvements the NDIA could make to its data dictionary for complaints and recommended processes support more proactive resolution of some matters by complaints officers. As at October 2023, the NDIA’s review of the data dictionary was still in progress (see paragraphs 3.30 and 3.37)47 and has not yet addressed the other recommendation. The deep dive also made broader service delivery recommendations, namely for teams to improve aspects of planning and communication with participants.
3.52 Regular and comprehensive analysis of complaints data is important as the broad range of potential reasons for complaints can indicate emerging trends or issues not readily apparent from analysis of other NDIA service delivery data alone. The NDIA’s linkage of data on complaints with Administrative Appeal Tribunal (AAT) reviews also requires further analysis as the reports containing these outcomes only identify areas requiring further investigations rather than identifying specific systemic issues.
3.53 The CEP completed three case study reports based on reviews of complaints within the Executive stream on: delays in NDIA decisions on Home and Living supports; multiple attempts by Support Coordinators to resolve plan issues; and resolving participant needs for a top-up of plan funding during an AAT review. Each case study included between four and five relevant complaints. The case study reports identified scope for the NDIA to improve multiple service delivery policies and processes across the organisation, as well as staff training. The case study reports did not include references to structured analyses of supporting evidence, such as that included in the deep dive review undertaken by PWPQ. The links between the observations from the complaints discussed in each case study and the recommended improvement actions were not explicitly documented or clear. The case study reports were submitted to the SLT for noting and shared with relevant NDIA business areas. The NDIA was unable to provide evidence the recommendations had been agreed to or implementation had started.
Opportunity for improvement |
3.54 Future case studies or reviews intended to support process or other operational changes clearly identify the supporting evidence, applied methodology, link between findings and recommendations and reflect coordination with existing or planned work of relevant NDIA business areas. |
3.55 Trends in the management of complaints can be identified from the results of post-completion checks reported to the Executive and managers of complaints teams and shared with other business areas, although thematic results are not included in the reports. As discussed in paragraphs 2.31 to 2.36, the NDIA does not have a mature process for reviewing the effectiveness of its complaints management or considering complaints data alongside other service delivery performance data to identify areas for continuous improvement.
Recommendation no.2
3.56 The NDIA plan and undertake a program of quarterly reviews of complaints data, matched with other service delivery performance data, including participant satisfaction surveys, to support identification of areas for continuous improvement.
National Disability Insurance Agency response: Agreed.
3.57 Please refer to NDIA Response to Recommendation 1.
Appendices
Appendix 1 Entity response
Appendix 2 Improvements observed by the ANAO
1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.
2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.
3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:
- strengthening governance arrangements;
- introducing or revising policies, strategies, guidelines or administrative processes; and
- initiating reviews or investigations.
4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.
- Published a new NDIS Enquiries, Feedback and Complaints Policy document that includes further channels for making a complaint and clearer complaint complexity levels (paragraphs 2.6 and 2.8).
- Updated the Complaints and Feedback Framework in February 2024 (see paragraph 2.21).
- The Planning Essentials Program was updated in October 2023 to include reminders of NDIA’s ‘no wrong door’ policy for receipt of complaints (see note to Table 2.4).
- Re-released the NDIA’s June 2023 biannual Complaints Analysis report to correct an error identified by the ANAO (paragraph 2.46).
- Additional detail included in the biannual Complaints Analysis report to show the proportion of complaints excluded from reporting in the relevant period due to incorrect dates being entered into NDIA’s IT system (see paragraph 3.22).
- Changes to complaints reporting methodology to exclude Participant Critical Incidents from total number of complaints received (see paragraph 3.23).
- A decision to allocate extra resources for complaints data analysis and continuous improvement activities following the Complaints Enhancement Project (see paragraph 3.37).
- Established a central complaints continuous improvement register in June 2023 (paragraph 3.44).
- Analysed a small subset of complaints data through the Complaints Enhancement Project to identify potential areas for continuous improvement (see paragraph 3.53).
Appendix 3 Quality assurance post-completion checks
1. The following table sets out mandatory complaints management steps assessed within the NDIA’s monthly program of post-completion quality checks.
Quality area |
Mandatory steps as described in NDIA documentation |
Right intake process |
|
|
|
|
|
Right person |
|
Right complaints management process |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Right resolution
|
|
|
|
|
|
Note a: ‘POI’ refers to proof of identity.
Source: NDIA documentation.
Appendix 4 Implementation of internal audit recommendations
1. The following table sets out the status of planned resolutions to findings from the NDIA’s 2022–23 internal audit of complaints management, as recorded by the NDIA’s internal audit team.
Finding |
Planned resolution |
Due date |
NDIA reported Status |
1.a) Lack of complaints data integrity due to CRM limitations below in data recording and reporting:
|
|
01/10/23 |
Closed. |
1.b) Lack of complaints data integrity relating to acknowledgements of complaints timeliness, due to errors in manual data entry by NCC staff. |
|
30/01/23 |
Closed. |
|
01/01/23 |
||
|
28/02/23 |
||
|
06/03/23 |
||
|
23/12/22 |
||
Source: NDIA documentation.
Appendix 5 Internal reporting on complaints by audience
1. The following table outlines the content of reports with complaints data that are regularly received by the NDIA Board, Senior Leadership Team (SLT) and managers and executives of complaints management teams.
Audience |
Complaints data included in reports |
Frequency |
Board |
Entity-level performance reports (Pulse reports) with high-level complaints data and trends, largely relating to meeting service standards for complaints, volume, source and subject of complaints. |
Monthly and quarterlya |
Senior Leadership Team (SLT) |
The SLT receives reports on complaints that are provided to the Board as well as shorter, weekly versions of the pulse reports. |
Weekly, monthly and quarterly |
Managers and Executives of complaints management teams |
Analysis of complaints attributes and trends |
Monthly, quarterly and biannual |
Workflow and workload management of complaints teams |
Daily, weekly and monthly |
|
Results of quality assurance reviews for complaints management |
Monthly |
|
Exceptions reports for complaints teams to monitor and take follow up or corrective actions, showing actions not in accordance with policy and procedural requirements |
Fortnightly |
|
Reports capturing complaints referred to or from other relevant agencies |
Monthly and as required |
|
Reports on particular categories of complaints, for example, complaints from ministers |
Daily and weekly |
|
Note a: The board did not hold meetings in December 2022 or June 2023.
Source: ANAO analysis of NDIA documentation.
Footnotes
1 National Disability Insurance Agency, Feedback and Complaints [Internet], NDIA, available from https://www.ndis.gov.au/contact/feedback-and-complaints [accessed July 2023].
2 National Disability Insurance Agency, Annual Report 2022–23, October 2023, available from https://www.ndis.gov.au/about-us/publications/annual-report [accessed 5 February 2024].
3 Complaints officers are allocated to a complaints stream and report to the manager of that stream, with the managers of each stream reporting to the CPIT executive. The different streams reflect the sources of complaints they manage.
4 National Disability Insurance Agency, Enquiries, Feedback and Complaints policy [Internet], NDIA, 2024, available from https://www.ndis.gov.au/contact/feedback-and-complaints [accessed 25 March 2024].
5 The NDIA has built a new IT business system, called PACE, to replace CRM. In November 2022, the NDIA started using PACE for Tasmanian participants and providers for key processes, including complaints. At the time of audit fieldwork, CRM remained the NDIA’s primary business system for managing complaints. On 30 October 2023, the NDIA started implementing PACE nationally. Full implementation of PACE is expected to take up to 18 months with participants transitioning to PACE as they either join the NDIS (new participants) or have a new participant plan or plan reassessment approved. CRM remains in use until all participants have transitioned to PACE. For consistency in describing complaints functions observed prior to the start of PACE national implementation, the audit refers to CRM.
6 The Office of the Commonwealth Ombudsman, Better Practice Complaints Handling Guide [Internet], the Office, 2023, available from https://www.ombudsman.gov.au/__data/assets/pdf_file/0025/290365/Better-Practice-Complaint-Handling-Guide-February-2023.pdf [accessed April 2024].
7 Commonwealth Ombudsman, Better Practice Complaint Handling Guide, 2023, available from https://www.ombudsman.gov.au/__data/assets/pdf_file/0025/290365/Better-Practice-Complaint-Handling-Guide-February-2023.pdf [accessed January 2024].
8 National Disability Insurance Agency, Feedback and Complaints [Internet], NDIA, available from https://www.ndis.gov.au/contact/feedback-and-complaints [accessed July 2023].
9 Teletypewriter — a device that helps people with a speech or hearing disability use a phone to communicate.
10 https://www.ndis.gov.au/contact/feedback-and-complaints [accessed July 2023].
11 The ANAO received eight citizen contributions to the audit. Three contributions, including one from a peak body representing disability service providers, suggested improvements be made to the ease of making a complaint. The peak body’s contribution included suggestions that NDIA provide complaints resources in multiple accessible formats, including plain language and video content with subtitles and sign language interpretation, and optimise its website for mobile use.
12 The NDIA’s DRCO Forum includes CEOs and senior representatives from 27 disability sector organisations. The NDIA’s PRG includes 23 participant and carer representatives from across Australia.
13 The NDIA’s webpage, titled ‘First Nations Strategy’, available at https://www.ndis.gov.au/about-us/strategies/first-nations-strategy [accessed April 2024], was last updated in July 2023 and outlines NDIA’s arrangements for co-designing a new First Nations Strategy; it links to a progress update dated July 2021 that states the refreshed strategy will be completed by 2022.
14 Participant-related written correspondence from the Minister for the NDIS.
15 Participant-related email or phone call from the office of the Minister for the NDIS.
16 Agency communications to the NDIA are through agreed protocols. The CPIT also includes a Participant Critical Incidents (PCI) stream. PCIs are a separate category of matters not included in the audit scope.
17 For some CPIT streams, the initial contact is not with the complainant directly — for example, in the MaSCO stream, it is with the electorate office that made the representation to the NDIA.
18 The eight design principles are included in both the February 2023 Better Practice Guide version referenced in this report and the prior 2021 version which was used to assess the NDIA’s framework dated June 2020 and various other policies and procedures assessed during audit fieldwork.
19 The Better Practice Guide includes an eighth principle that complaints systems are adequately resourced. This was not assessed by the audit.
20 As quality self-assessments by complaints officers were not mandated and pre-completion quality checks were only required for new complaints officers, there was no regular reporting on these results to provide NDIA with further assurance of complaints management quality.
21 For the remaining two sampled items, there was no remedy provided by the NDIA as it referred one complainant to the NDIS Commission, and closed the other complaint following unsuccessful attempts to contact the complainant for more information.
22 Time limits apply to applications for reviews of NDIA decisions — applications must be made within three months of the date of the decision to be reviewed.
23 The NDIA finalised the June 2023 version of its biannual Complaints Analysis report and provided a copy to the ANAO in December 2023. The NDIA issued a corrected version of the report in January 2024 following the ANAO alerting it to a potential error, relating to the reported number of complaints about complaints handling.
24 As noted in paragraph 3.43, in February 2023 the NDIA introduced a practice of sending SMS messages to complainants ahead of calling them to let them know that a complaints officer would be calling them from a private number. Copies of SMS alerts sent were observed in complaint records reviewed by the ANAO as part of the sample discussed at paragraph 2.40 onwards.
25 The reports are provided to all Commonwealth and state and territory disability ministers.
26 The Participant Service Guarantee (PSG) forms part of the Participant Service Charter and provides timeframes for key NDIS processes, such as explaining a decision, approving a plan and making changes to plans. The service standards for complaints are separate to the Participant Service Guarantee timeframes.
27 The metrics in the 2022–23 Corporate Plan were: participant and scheme outcomes; participant and stakeholder sentiment; scheme financial sustainability and integrity; scheme operating performance; market performance; and agency operating performance.
28 The weekly pulse reports include performance information on: participants’ access to the NDIS; eligibility reassessments; plans and plan reviews; decisions on supports; reviewable decisions; complaints, payment matters; and actuarial information.
29 The NDIA’s June 2023 biannual NDIS Complaints and Participant Critical Incidents Analysis report was issued in December 2023.
30 The audit team observed correspondence from July 2023 and December 2023 advising teams of how these activities would occur. The NDIA advised the ANAO in January 2024 that no such activities took place during 2022–23.
31 Commonwealth Ombudsman, Better Practice Complaint Handling Guide, 2023, available from https://www.ombudsman.gov.au/__data/assets/pdf_file/0025/290365/Better-Practice-Complaint-Handling-Guide-February-2023.pdf [accessed March 2024].
32 The NDIA has built a new IT business system, called PACE, to replace CRM. In November 2022, the NDIA started using PACE for Tasmanian participants and providers for key processes, including complaints. At the time of audit fieldwork, CRM remained the NDIA’s primary business system for managing complaints. On 30 October 2023, NDIA started implementing PACE nationally. Full implementation of PACE is expected to take up to 18 months with participants transitioning to PACE as they either join the NDIS (new participants) or have a new participant plan or plan reassessment approved. CRM remains in use until all participants have transitioned to PACE. For consistency in describing complaints functions observed prior to the start of PACE national implementation, the audit refers to CRM.
33 In 2022–23, ‘desired action completed’ remained the largest outcome category (37.1 per cent), followed by ‘formal reply’ and ‘s48/s100 review completed’ (2.5 per cent each).
34 The December 2022 biannual Complaints Analysis Report was issued in July 2023 and the June 2023 biannual Complaints Analysis Report was issued in January 2024.
35 The NDIS website is available at: https://www.ndis.gov.au/about-us/publications.
36 Complaints data is not listed in Schedule 2B of the Public Governance, Performance and Accountability Rule 2014 as a mandatory requirement for inclusion in the annual report of a corporate Commonwealth entity, such as the NDIA, although it is a recommended inclusion in the Better Practice Guide.
37 The NDIA defines the complaint rate as the annualised rate, calculated as the number of complaints in the quarter divided by the active participant exposure in the quarter. On 28 September 2023, the NDIA added two more Participant Dashboard reports to its website, for ‘Other Neurological’ and ‘Other Physical’ disability groups.
38 For example, National Disability Insurance Agency, Young Adults in the NDIS [internet], NDIA, available from https://data.ndis.gov.au/reports-and-analyses/participant-dashboards/previous-participant-group-reports [accessed September 2023].
39 The NDIA’s Executive and managers of complaints teams and National Contact Centre receive fortnightly reports of feedback with incomplete or incorrect aspects of CRM complaint data identified.
40 In its NDIS Quarterly report to disability ministers, the NDIA defines a PCI as ‘circumstances or information about allegations of serious harm occurring to a participant.’
41 The number of participant complaints received in 2021–22 was further updated in the September 2023 NDIS Quarterly report to disability ministers to 30,091.
42 The fieldwork for the internal audit was conducted between July and October 2022.
43 The NDIA noted the purpose of the CEP in its November 2023 report to the Board, following completion of the CEP, stating it had been initiated to address observed ‘increasing levels of escalated issues by participants to the Minister, the NDIA Chair and Senior Executives, where resolution to issues had not been successful.’
44 While no deep dive audits of complaints management were undertaken as part of the Quality Framework, a separate NDIA business area completed one deep dive review into a planning-related category of complaints in August 2022 (see paragraphs 3.24 and 3.49).
45 While not specific to complaints, the NDIA’s National Contact Centre undertakes analysis of caller satisfaction and the drivers behind it (see Table 2.3) and has mechanisms to share its results and learnings with other parts of the agency.
46 In February 2024 the NDIA updated its Complaints and Feedback Framework. The updated framework states, ‘Enquiries, complaints, and feedback provide an opportunity to learn from consumers and enhance business practices and processes. Continuous improvement is an ongoing cycle of identifying and acting on opportunities to improve.’ The updated framework also identifies managers and leaders as responsible for ‘sharing and discussing feedback and complaint data with staff and teams to support a better understanding about: the themes and systemic issues; how this links to improving participant’s services.’
47 The NDIA advised the ANAO in March 2024 that to align the data dictionary with its new business IT system, PACE, it planned to build data dictionary updates into PACE itself through future system enhancements.