Introduction

1.1 The National Disability Insurance Scheme (the NDIS) was established in 2013 under the National Disability Insurance Scheme Act 2013 (NDIS Act) to provide funding for reasonable and necessary supports for people with permanent and significant disability. The NDIS is jointly funded by the Australian, state and territory governments under bilateral arrangements.

1.2 The National Disability Insurance Agency (NDIA) is the Australian Government entity responsible for delivering the NDIS. It is a corporate Commonwealth entity under the Public Governance Performance and Accountability Act 2013. In 2022–23, the NDIA reported that it employed a total of 5,652 staff with 20.8 per cent of those having a disability.

1.3 If a person meets the age, residency and disability or early intervention access requirements, they may qualify to enter the NDIS. Once approved to enter the NDIS, these individuals (participants) have their supports costed and funded via a support plan (participant plan). In 2022–23, the NDIA reported participant plan expenses under the NDIS totalled $35.2 billion and covered more than 610,000 participants.² The NDIS is expected to grow to 1,017,522 participants and $89.4 billion in 2032.

Complaints management

1.4 The NDIA receives complaints from a variety of sources on such matters as dissatisfaction with supports in a participant’s plan, payment delays, approvals of plan reviews and engagement with NDIA staff. The NDIA reported in 2022–23 that it received 28,951 complaints from participants (compared to 30,091 in 2021–22) and 983 complaints from NDIS providers (compared to 1,446 in 2021–22). Complaints are also received from various other stakeholders including the NDIS Quality and Safeguards Commission, the Commonwealth Ombudsman, parliamentarians, and state and territory agencies. Figure 1.1 shows the top five sources of complaints to the NDIA in 2022–23.

Figure 1.1: Sources of complaints to the NDIA 2022–23

Source: NDIA reporting.

1.5 Complaints officers within the NDIA’s Complaints and Participant Incident Team (CPIT) work in different complaints streams to manage complaints.³

• Complaints resolution — manages the largest volume of complaints received by the NDIA, through the primary complaints email and phone contacts.
• Oversight bodies — manages correspondence, including complaints, from external stakeholders about the NDIA, including participant and provider issues.
• Ministerial (formal and informal) — manages complaints received by the Minister’s office.
• Members and Senators complaints — manages complaints received from members of parliament, senators and their electorate office staff.
• Executive complaints — manages complex or sensitive complaints received by the NDIA Board, Senior Executive Service officers, or through media enquiries and social media channels.

1.6 The NDIA categorises complaints into three levels. Until 1 February 2024, these were:

• level 1 complaints — ‘usually a single issue or concern which is straightforward in nature; they do not require detailed investigation’;
• level 2 complaints — ‘usually more complex by nature and may involve more than one issue and may require some investigation’; and
• level 3 complaints — ‘usually sensitive or complex by nature and often involve multiple issues, parties or business areas ….[and] may require more detailed or formal investigation.’

1.7 On 1 February 2024, the NDIA published the following updated descriptions of complaint levels on its website⁴:

• level 1 complaints — ‘issues that can be resolved by the staff member who first helps you’;
• level 2 complaints — ‘issues or concerns [that] need investigation and/or coordination to resolve’; and
• level 3 complaints — ‘issues that are complex or require formal resolution.’

1.8 Complaints are recorded in the NDIA’s business IT system (Customer Relationship Management, CRM).⁵

1.9 The Commonwealth Ombudsman’s Better Practice Complaints Handling Guide sets out the best practice approach that entities can apply in developing a complaints management framework or approach.⁶ The NDIA’s Complaints and Feedback Framework (the Complaints Framework) dated June 2020 sets out the how the NDIA will manage feedback and complaints on its performance, conduct or processes. The NDIA published an updated version of the Complaints Framework on 6 February 2024. Both versions of the Complaints Framework state that ‘This framework also observes the fundamental guiding principles outlined in the Commonwealth Ombudsman: Better practice guide to complaint handling.’ The principles include fairness, accessibility, efficiency, integration and first contact resolution.

1.10 The NDIA has set service standards for the management of complaints, which form part of its Participant Service Charter (see Table 1.1).

Table 1.1: Service standards for complaints

Source: NDIA Participant Service Charter.

Rationale for undertaking the audit

1.11 In 2021–22 the NDIA reported that it received 30,091 complaints from participants of the NDIS and 1,446 from NDIS providers; and in 2022–23 it reported having received 28,951 complaints from participants and 983 from NDIS providers. Effective complaints management contributes to transparency, accountability and the continuous improvement of administrative decision-making and service delivery. This audit provides assurance to the Parliament that the NDIA has fit-for-purpose complaints management practices.

Audit approach

Audit objective, criteria and scope

1.12 The objective of the audit was to assess the effectiveness of the NDIA’s management of complaints.

1.13 To form a conclusion against this objective, the following high-level criteria were applied.

• Does the NDIA have a fit-for-purpose complaints management framework?
• Does the NDIA adequately report on complaints, and outcomes, and review its management of complaints?

Audit methodology

1.14 The audit methodology involved:

• assessing the NDIA’s complaints management approach against the Commonwealth Ombudsman’s Better Practice Complaint Handling Guide;
• reviewing and analysing policy documents, guidelines, procedures, reports and relevant files;
• examining the management of a sample of complaints;
• site visit with complaints handling teams in Geelong;
• meeting with relevant NDIA staff; and
• reviewing contributions made to the audit contribution facility on the ANAO website.

1.15 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $396,000.

1.16 The team members for this audit were Barbara Das, Renina Boyd, Susan Penn-Turrall, Sonya Carter, Vivian Turner and Alexandra Collins.

2.1 The Commonwealth Ombudsman’s Better Practice Complaint Handling Guide (Better Practice Guide)⁷ states:

Good complaint handling will also help your agency meet general principles of good administration, including fairness, transparency, accountability, accessibility and efficiency. Analysis of complaint data may identify opportunities for the improvement of administrative practices.

Strong quality assurance frameworks will help balance competing imperatives of quality and timeliness.

Does the NDIA have fit-for-purpose complaints management policies and processes?

Methods of making complaints

2.2 The NDIA publishes clearly written information about its complaints and feedback process on the NDIS website⁸, including guidance on how to make a complaint, how the NDIA manages complaints and what to do if dissatisfied with the complaint outcome. A complaint to the NDIA can be made in the following ways:

• visit a local NDIA office and speak in person with staff, such as a planner;
• call the NDIA’s National Contact Centre (NCC);
• contact the NDIA via TTY⁹ or the National Relay Service;
• live chat online with NDIA staff;
• use the contact and feedback form on the NDIS website;
• post or email to enquiries@ndis.gov.au;
• contact a member of parliament or senator; and
• use the online participant or provider portals.

2.3 In May 2023, a new enquiry phone line was established for the office of the Minister for the NDIS to call a specifically assigned team within the NCC directly. The NDIA advised the ANAO in August 2023 that the new phone line was intended to facilitate a direct connection with the minister’s office. Complaints received via the enquiry line are categorised as complexity level three and referred to a complaints officer (see paragraphs 1.5 to 1.7 for complaints streams and complexity levels).

2.4 The NDIA accepts complaints made anonymously (for example, by phone or not including a last name when using the online form). Guidance directs NCC Customer Service Officers to inform complainants that, depending on the matter, this can limit effective resolution as NDIA staff may be unable to identify all information needed to investigate the complaint.

2.5 Some complaints made to the NDIA are matters for other agencies to resolve. For example, complaints about NDIS service providers should be made to the NDIS Quality and Safeguards Commission (the NDIS Commission), and complainants dissatisfied with the outcome of a complaint managed by the NDIA, or how the NDIA managed it, may ask the Commonwealth Ombudsman to investigate. The NDIA has a ‘no wrong door’ policy for complaints made to the agency and accepts transfer of complaints initially made to other agencies that relate to the NDIA. The NDIA has protocols for inter-agency complaints transfers (see paragraph 2.23).

2.6 The information about making a complaint on the NDIS website¹⁰ is not featured on all NDIS website pages relating to contacting the agency. During audit fieldwork, the ‘Feedback and complaints’ webpage did not refer to the option of making a complaint via online portals or by post. On 1 February 2024, the NDIA added a link on the webpage to a new ‘NDIS Enquiries, Feedback and Complaints Policy’ document which refers to these channels. The NDIS website includes key participant information in a variety of languages, including Auslan videos, but this does not include complaints and feedback information.

2.7 The NDIA’s online contact and feedback form allows complainants to identify that their message is about ‘feedback or complaint’, without needing to specify one or the other, and enter free text details. The form also allows participants to select one of the six options given (four relate to seeking support, one to feedback and complaints and one to other general enquiries). During audit fieldwork, the NDIA’s definitions of ‘complaint’, ‘feedback’ and ‘enquiry’ from its internal policy document, as set out below, were not published:

Complaint: An expression of dissatisfaction indicating an experience with the NDIA or a related entity is displeasing or unacceptable and requires a resolution.

Feedback: Feedback is the articulation of an observation made by a participant or their representative on their experience which may take the form of a suggestion, opinion or compliment.

Enquiry: The act of requesting information, knowledge or action.

2.8 The NDIA’s new NDIS Enquiries, Feedback and Complaints Policy includes the following definitions:

• Enquiry — requesting information, knowledge, or action.
• Feedback — sharing your experience, including observations, suggestions, opinions, or compliments.
• Complaint — expressing dissatisfaction about an experience with the NDIA that was displeasing or unacceptable and requires resolution or response.

2.11 The NDIA identified that some participant groups make disproportionately low numbers of complaints. For example, internal reporting in December 2022 stated, ‘Participants who have identified as First Nations Peoples or culturally or linguistically diverse are less likely to submit a complaint than other participants.’ The NDIA has not identified the causal drivers for such results, including whether these cohorts may experience barriers to submitting complaints.¹¹

2.12 As part of its Complaints Enhancement Project (CEP) (see paragraph 3.30), the NDIA sought feedback on complaints management from its Disability Representative and Carer Organisations (DRCO) Forum in June 2023 and the Participant Reference Group (PRG) in August 2023.¹² The feedback included that complaints can be difficult to make, and that the NDIA could provide more easy read documents, clarify there is no negative impact for participants who complain, and consider what channels and support for making a complaint are available for people who are not currently making complaints.

2.13 As part of the CEP, the NDIA drafted a new complaints policy document, including an easy-read version that was published on the NDIA’s website on 1 February 2024 and included advice that ‘no detrimental actions will be taken against people, or their representatives, who raise issues or concerns with us.’ The CEP’s Action Plan included the following activities relevant to the accessibility of the complaints management practices — no timeframes were specified for this work:

Developing and publish translations of the public policy in recommended culturally and linguistically diverse (CALD) languages

Investigating translations in First Nations languages (in line with new NDIS First Nations Strategy¹³)

Investigating Auslan translation

Investigating an explanatory video for the NDIS website.

Complaints management process

2.14 Complaints made via email or the NDIA’s online form go to the NDIA’s feedback inbox. The NCC monitors this inbox and assesses all content to determine if it is a complaint, the complexity level and any risks involved. The NCC also monitors complaints made via the participant or provider portals.

2.15 NCC Customer Service Officers receive complaints made to the NDIA’s phone line or via the online chat functionality. Other NDIA frontline staff such as planners or NDIA Partners (also known as Local Area Coordinators) may receive complaints directly through the course of their interactions with participants, for example, during a planning meeting or follow-up telephone conversation with a participant.

2.16 The key initial steps for frontline staff receiving a complaint are set out in Table 2.1.

Table 2.1: Key initial steps in complaint lifecycle

Note a: See footnote 5.

Note b: Where a complaint is received via the Minister, Member of parliament or a Senator, the NDIA acknowledges receipt to the Minister, Member or Senator within one day and to the complainant within two days.

Note c: Some complaints categorised as level 1 complexity may still be referred to a business area for quick resolution and in such cases frontline staff (including NCC Customer Service Officers) may still need to make initial contact within the two-day service standard timeframe.

Note d: Some complaints are out of scope for the complaints teams and the NCC refers directly to the relevant business area, for example, complaints about NDIA compliance with its Model Litigant Obligations are referred to the NDIA’s Legal Services area for assessment and management.

Source: ANAO analysis, based on NDIA documentation.

2.17 The Complaints and Participant Incident Team (CPIT) manages complexity level two and three complaints and is comprised of complaints officers in the following streams: Complaints Resolution; Executive Complaints; Ministerial Formal¹⁴; Ministerial Informal¹⁵; Members and Senators Complaints Officers (MaSCO); and Oversight Bodies.¹⁶ Complaints managed across all streams are subject to the same 21-day resolution service standard (see paragraph 1.10).

2.18 Assigned officers within CPIT monitor the team’s complaints inbox and:

• carry out a further risk assessment to prioritise complaints for CPIT action;
• contact complainants (where not already done by NCC¹⁷) — to meet the NDIA’s two-day contact service standard, provide a reference number and outline the complaint process; and
• allocate complaints to CPIT officers.

2.19 NDIA internal procedures describe the actions complaints officers undertake once they are allocated a complaint. A summary of the key steps undertaken is set out in Table 2.2.

Table 2.2: Complaints officers’ process for managing complaints

Note a: Where a complaint involving a participant is made by a third party, NDIA procedures direct staff to check the participant’s record for consent to share information with the complainant, or to directly seek the participant’s consent to do so.

Note b: The NDIA communicates outcomes directly with complainants, including where the complaint has been referred informally from the Minister’s office or is received from an MP or senator. Where the complaint is received via a formal letter to the Minister for the NDIS, the outcome of the complaint is either communicated to the complainant in a formal letter from the Minister, drafted by the NDIA, or in a formal letter from the NDIA where the Minister directs the NDIA to respond on their behalf.

Source: ANAO analysis of NDIA documentation.

Policies and procedures

2.20 The NDIA’s June 2020 NDIS Complaints and Feedback Framework (the Complaints Framework) sets out its policy for complaints management, including definitions of complaints, feedback and enquiries, complexity levels, and complaints roles and responsibilities across the NDIA. The Complaints Framework states that it observes the principles in the Better Practice Guide and refers to the NDIA’s Participant Service Charter principles on how it engages with participants and the public. The Participant Service Charter principles are: transparent; responsive; respectful; empowering; and connected. The Complaints Framework includes the NDIA’s Participant Service Charter commitments to the following complaints management steps:

• acknowledge a complaint within one day of receipt;
• make contact within two days of receipt;
• resolve 90 per cent of complaints within 21 days of receipt (more complex complaints may take longer to address);
• keep the complainant informed about the progress of their complaint at every stage; and
• contact the complainant or their representative to talk about their complaint to better understand it.

2.21 In February 2024, the NDIA updated its Complaints Framework. The new framework includes the same key elements outlined in paragraph 2.20, except it included alternative guidance on making contact with complainants (see paragraph 2.23).

2.22 As at June 2023, the NDIA had 46 internal guidance documents and supporting templates relating to complaints management, located on the intranet in a clearly defined and accessible section.

2.23 The NDIA’s key policy and procedure documents for complaints management were largely complete, current, consistent and clearly written.

• At the time of audit fieldwork, the Complaints Framework had not been updated since 2020, and did not accurately reflect the NDIA’s timeliness service standards, which now refer to calendar days rather than ‘business days.’ The NDIA removed the reference to ‘business days’ in the February 2024 version of the Complaints Framework. The new version states that after receiving contact, the NDIA aims to ‘contact you within 2 days of acknowledgement’ — this is inconsistent with the NDIA’s commitment under its Participant Service Charter to making contact within two days of receipt of the complaint.
• There were inconsistent approaches to responding to complaints about NDIS service providers. NCC guidance directs Customer Service Officers to record complaints, advise callers to complain to the NDIS Commission and offer to lodge complaints with the NDIS Commission on the caller’s behalf. The Complaints Framework directs staff to record any complaints about service providers, advise the complainant to follow the relevant provider’s own complaint processes and then, if necessary, make a complaint to the NDIS Commission.
• The Participant Service Charter states participants will be kept informed about the progress of their complaints at every stage. The February 2024 version of the Complaints Framework reflects this commitment; other internal procedural guidance does not include explicit steps or guidance to support this beyond the initial contact service standards (one and two days).
• While key procedures include steps for preliminary investigations that are likely to be sufficient for most complaints, they did not include guidance to support the more complex investigations. Further guidance, including a broader range of examples for reference, could be added to existing procedures to better support consistent identification of complaint issues, potential remedies and effective planning of more complex investigations.

Alignment with the Commonwealth Ombudsman Better Practice Guide

2.24 The Better Practice Guide sets out eight design principles¹⁸ for a better practice complaints management framework. Table 2.3 sets out the extent to which the NDIA’s complaints management framework reflects each of these principles.¹⁹

Table 2.3: Alignment of NDIA complaints management framework to better practice design principles

Key: ○ Not aligned ◔ Partly aligned ◑ Half aligned ◕ Largely aligned ● Fully aligned

Note a: The NDIA’s NCC has a separate Quality Assurance Framework and assesses a random sample of calls monthly. The framework applies to calls generally, not just complaints. Call quality is assessed against a range of customer-focused attributes (for example, communicating in a professional and polite manner); business-focused attributes (for example, using the correct procedure); and regulatory attributes (for example, correctly following privacy protocols). Consideration of the NCC framework was not undertaken in assessing alignment against the quality assurance process design principle set out above.

Source: ANAO analysis.

2.25 In addition to key design elements, the Better Practice Guide sets out eight steps that high-quality complaints processes should include. The Complaints Framework includes these steps:

• identify and record a complaint;
• acknowledge the complaint;
• assess and triage;
• resolve early or investigate;
• consider potential remedies;
• communicate the outcome;
• finalise; and
• feedback systemic issues.

2.26 In April 2021, the NDIA responded to a voluntary survey issued by the Commonwealth Ombudsman. The survey responses were used by the Commonwealth Ombudsman to benchmark the maturity levels of participating agencies’ complaints handling processes against its Better Practice Guide. In February 2022 the Commonwealth Ombudsman advised the NDIA that its processes rated as four, ‘established’ (against a maximum of five, ‘superior’). The Commonwealth Ombudsman identified that the NDIA could enhance its continuous improvement processes to raise its maturity level (see paragraph 3.27).

Complaints training

2.27 New complaints officers are required to complete mandatory corporate NDIA training modules, such as privacy and information handling, and a separate complaints induction course, completed over the first two weeks on the job. Training completion rates reported by the NDIA are set out in Table 2.4 below.

Table 2.4: Training completion by NDIA staff involved in complaints, reported by the NDIA as at 5 January 2024

Note a: NDIA mandatory training applicable to all APS staff, labour hire, contractors and partners comprised of 14 modules, categorised into two groups: ‘Mandatory — it’s the law’; and ‘Mandatory — our ways of working’. NDIA completion rates are recorded as average rates of completion across the modules in both groups.

Note b: Planning Essentials Program includes brief references to staff roles in complaints, such as the need to check for any complaints ahead of participant reassessments. The NDIA refreshed this suite of training in October 2023 — while complaints was still not a focus, the refreshed program includes additional reminders to staff in frontline roles, for example to support the ‘no wrong door’ principle by providing a warm transfer of complaints to other NDIA business areas.

Source: ANAO, based on NDIA documentation.

2.28 New starters in CPIT are assigned another officer as a ‘buddy’ for on-the-job guidance and are required to manage five complaints (two with assistance and three independently) before being rated as competent. The CPIT Continuous Improvement Framework (the Quality Framework) requires further checks of complaints officers’ first 10 independently completed complaints to assess proficiency. These checks occur before the complaints are finalised. Once new complaints officers reach 80 per cent proficiency their complaints are no longer checked before closure. A sample of all complaints officers’ completed complaints are checked monthly as part of the NDIA’s quality assurance approach (see paragraph 2.31).

2.29 For new Customer Service Officers, the NCC induction training includes a complaints module. The NDIA advised the ANAO in September 2023 that Customer Service Officers must complete the NCC complaints induction module before they are assigned to taking calls. Complaints officers can undertake additional periodic training as available, for example in June 2023, 16 complaints officers participated in the Commonwealth Ombudsman’s annual Complaint Handling Forum.

2.30 In September 2023, as part of its work on the CEP, the NDIA’s contracted provider, Nous Group, proposed additional training be provided, such as in communication and writing skills and managing difficult conversations. Nous Group prepared a separate training package on feedback and complaints. As at January 2024, the training had not yet been delivered.

Are complaints managed in accordance with NDIA complaints management policies and procedures?

NDIA monitoring of complaints management quality

2.31 The NDIA’s Quality Framework includes a supporting Quality Framework and Checklist document (the Quality Checklist). The Quality Checklist states that the ‘quality measures to managing complaints within the NDIA are based on four key focus areas of compliance’:

• right person — consent and proof of identity confirmed;
• right process — complaint issue identified and recorded, all actions recorded with a clear resolution, processes followed, correct templates applied;
• right referral — complaint issue clearly summarised, actions recorded, appropriate risk rating applied, correct process and template applied; and
• right resolution — resolution achieved and recorded, complaint outcome or closure communicated, followed up where required.

2.32 The Quality Framework describes the NDIA’s assurance approach for complaints management: monthly quality audits; a quality self-assessment by complaints officers; pre-completion quality checks for new officers; post-completion quality checks; a program of ‘deep dive’ audits; and use of a Continuous Improvement Register for monitoring implementation progress (see paragraph 3.38). The NDIA has not fully implemented the Quality Framework; monthly quality audits and deep dives are not undertaken.²⁰

2.33 The NDIA undertakes post-completion quality checks using a random sample to inform coaching and continuous improvement. The Quality Framework states that an average of at least one check per week is undertaken by senior complaints officers, team leaders, and assistant directors for each complaints officer. The Quality Framework requires results be reported monthly to the executive and managers of complaints teams for discussion in team meetings. Appendix 3 sets out the mandatory complaints management steps assessed within the monthly post-completion quality checks.

2.34 The NDIA has not documented the process to collate the random sample for the post-completion quality checks each month. The NDIA also had not documented its approach to ensuring consistent application of the Quality Checklist, including guidance on examples of qualitative judgement calls required to be made during the quality assurance process.

2.35 The NDIA reports on its post-completion quality checks through monthly quality reports. The NDIA has not set targets for quality check results and does not use data from its monthly reports to analyse trends. The aggregated results of the NDIA’s monthly post-completion quality checks for 2022–23 against the four compliance areas are set out from Figure 2.1 to Figure 2.4 below. Five of the 12 reports in 2022–23 were marked draft, and there was no evidence of NDIA publishing a final version of these reports. The results vary significantly across the financial year for all four key compliance areas. The NDIA has not analysed the drivers of such variability in results.

Figure 2.1: NDIA results of post-completion quality checks for ‘right person’ category, Complaints Resolution stream, 2022–23

Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).

Source: ANAO analysis of NDIA monthly quality reports.

Figure 2.2: NDIA results of post-completion quality checks for ‘right process’ category, Complaints Resolution stream, 2022–23

Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).

Source: ANAO analysis of NDIA monthly quality reports.

Figure 2.3: NDIA results of post-completion quality checks for ‘right referral’ category, Complaints Resolution stream, 2022–23

Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).

Source: ANAO analysis of NDIA monthly quality reports.

Figure 2.4: NDIA results of post-completion quality checks for ‘right resolution’ category, Complaints Resolution stream, 2022–23

Note: Complaints Resolution receives the largest volume of complaints (see paragraph 1.5).

Source: ANAO analysis of NDIA monthly quality reports.

ANAO review of a sample of complaints

2.40 The ANAO reviewed a targeted sample of 30 complaints received by the NDIA in 2022–23 and assessed whether they complied with key steps within NDIA’s policy and procedures that align with the Better Practice Guide process steps (see paragraph 2.25). The complaints selected by the ANAO were those managed by complaints officers in the Complaints Resolution team (see paragraph 2.17) and included complaints received by email (15), phone (11), online form (one), portal (two) and in-person (one). Five of the sampled complaints were categorised as level one (simple), 20 as level two (complex), and five as level three (more complex or sensitive) complaints. The ANAO’s observations are set out in Table 2.5.

Table 2.5: Compliance with NDIA policy and procedure for ANAO targeted sample of complaints

Note a: The NDIA’s acknowledgment emails included most of the content suggested by the Better Practice Guide.

Note b: While the NDIA’s process includes assessing the complexity and risk of complaints, this step is not visible in the NDIA’s CRM system and was not reviewed by the audit team beyond consideration of the complexity level.

Note c: The Better Practice Guide recommends complaints processes include a final step, ‘feed back systemic issues’ raised by the complaint. Sampled complaints were not assessed against this step, as the audit separately assessed the NDIA’s approach to identifying and sharing systemic issues for continuous improvement purposes, discussed in Chapter 3 (discussed from paragraph 3.38).

Source: ANAO analysis.

2.41 The Complaints Framework requires staff to categorise complaint outcomes in accordance with the NDIA’s data dictionary (a glossary of specific terms to support consistent data entry in the NDIA’s business IT system) (see paragraph 3.20). The most common outcome category recorded was ‘desired action completed’ (26 of 30 complaints). Outcomes recorded across the remaining four sampled complaints were: formal reply endorsed and issued; unable to contact – record closed; referred to NDIS Quality and Safeguards Commission; and withdrawn by participant or representative.

2.42 The CRM record also contains free text descriptions of the outcomes of complaints. Recorded outcomes were supported by the outcome descriptions entered by NDIA staff for 25 of the 30 sampled complaints (for four complaints, the outcomes were not recorded according to NDIA guidance and for one complaint the outcome recorded did not accurately reflect the actual outcome). For all 30 complaints, the recorded description of the outcome provided to the complainant indicated the complaints officer addressed the key issues raised in the complaint. In 28 of the 30 sample items reviewed, the outcome description reported that one or more remedies were provided to the complainant.²¹ The ANAO categorised the descriptions of the remedies provided as follows:

• better explanation or provision of additional information — 16 of 28 complaints;
• corrective action, such as assigning a new Local Area Coordinator — nine of 28 complaints; and
• an apology — seven of 28 complaints.

2.43 The 30 sampled complaints were resolved within a median of seven days and an average of 10 days, with:

• 28 resolved within 21 days, including;
  • two resolved within one day;
  • 16 resolved within seven days; and
• two taking more than 21 days to resolve (27 days and 33 days). For one of these, the outcome was to inform the complainant they could make a time-sensitive application for an independent review of the decision they were dissatisfied with.²²

2.44 The eight complaints recorded as not having included investigation were resolved within a median and average of five days. Timeliness of complaint resolution is discussed from paragraph 2.54.

2.45 The ANAO observed:

• two complaints were resolved within one day and did not require any update to the complainant;
• for eight of the 28 complaints that took more than one day to resolve, complainants were contacted at least once between making the complaint and when it was resolved (see paragraph 2.23); and
• for the two complaints that took longer than 21 days to resolve (out of the 28 complaints that required an update), no updates were provided to complainants.

Complaints about complaints handling

2.46 The NDIA produces a monthly report on complaints received in the prior month about those staff who managed the initial complaint, known by the NDIA as ‘complaints about complaints handling’. In its biannual internal NDIS Complaints and Participant Critical Incidents Analysis (Complaints Analysis) report, the NDIA identified complaints about complaint handling as one of the top 15 complaint reasons for 2022–23, reporting that it had received:

• 384 complaints in the six months to 31 December 2022 (three per cent of all complaints in the period); and
• 329 complaints in the six months to 30 June 2023 (two per cent of all complaints in the period).²³

2.47 Reasons for complaints about complaints handling varied and were not consistently described or categorised in the NDIA’s monthly complaint handling reports.

Commonwealth Ombudsman’s reporting on NDIA complaints

2.48 The Commonwealth Ombudsman independently investigates complaints by members of the public about Australian Government agencies, including the NDIA. Complaints are to first be made to the relevant agency, and the Commonwealth Ombudsman then investigates where the complainant is dissatisfied with the way the agency managed the complaint or its outcome.

2.49 In its December 2022 submission to the Joint Standing Committee on the NDIS inquiry into the Capability and Culture of the NDIA, the Commonwealth Ombudsman stated:

In 2021-22, the OCO received 858 complaints about the NDIA. This is a 16 per cent increase in complaints received compared to 2020-21 (741). During the same period the number of NDIS participants increased by 15 per cent. The most common issues raised in complaints were plans (30 per cent), reviews (15 per cent), and service delivery (15 per cent). Within those issues, complaints concerned:

• Plans—the final plan outcome, and dissatisfaction with the plan amendment and implementation process.
• Reviews—dissatisfaction with the process and decision made with the review of a decision, and the process associated with a participant-requested plan review.
• Service delivery—dissatisfaction with the process of NDIA’s handling of complaints made to its complaints service.

In 2021-22, the NDIA was the fourth largest (6 per cent) source of complaints to the OCO in its general Commonwealth jurisdiction. The largest was Services Australia–Centrelink (52 per cent).

2.50 In its 2022–23 Annual Report, the Commonwealth Ombudsman reported the NDIA remained one of its top five Australian Government agencies by number of complaints received, with six per cent of complaints received relating to the NDIA.

2.51 In May 2023 the NDIA received a report from the Commonwealth Ombudsman summarising complaints it had received about the NDIA for the period July 2022 to March 2023, noting that complaints did not always require an investigation. For example, complaints could be finalised without investigation where the Commonwealth Ombudsman was able to assure complainants that the NDIA had acted in a reasonable manner, where complainants had not previously complained to the NDIA, or where complainants had an appropriate review pathway.

2.52 Between July 2022 and March 2023, the Commonwealth Ombudsman reported in its NDIA liaison meeting report that it had finalised 565 complaints relating to the NDIA, of which 44 were investigated, 23 were transferred to the NDIA and 498 were finalised without investigation. Figure 2.5 represents the outcomes of the complaints investigated by the Commonwealth Ombudsman.

Figure 2.5: NDIA complaints — outcomes following investigation by Commonwealth Ombudsman, July 2022 to March 2023

Source: Commonwealth Ombudsman report to the NDIA.

Citizen contributions

2.53 As noted at paragraph 1.14, the ANAO received eight citizen contributions to the audit. Six contributions, including one from a peak body representing disability service providers, included comments on NDIA complaint management practices across the following themes:

• the NDIA focusing on timeframes more than effective resolution;
• lack of connection from the NCC back to relevant business areas able to resolve issues;
• inadequate investigation processes;
• poor communications, including a disrespectful approach and errors;
• lack of skills and expertise in complaints handling;
• use of private numbers to contact complainants — meaning complainants might not take the call²⁴;
• delays in complaint resolution causing disruptions in service delivery; and
• poor links (not a warm transfer) of complaints between agencies.

Are complaints resolved within required timeframes?

2.54 As discussed at paragraph 1.10, the Participant Service Charter includes timeliness service standards for the management of complaints, which are set out in Table 2.6.

Table 2.6: NDIA service standards for complaints

Note a: Timeframes are in calendar days.

Source: NDIA Participant Service Charter 2023.

External reporting

2.55 The NDIA reports on its performance against the 21-day timeframe in the NDIS Quarterly report to disability ministers²⁵, which are published on the NDIS website. These reports provide a view of the NDIA’s overall performance and operations over a three-month period, including complaints. The reported results of the NDIA’s performance for the 21-day timeframe for each quarter from 1 July 2021 to 30 September 2023 is set out in Figure 2.6.

Figure 2.6: Closed complaints and percentage completed within the 21-day timeframe per quarter from 1 July 2021 to 30 September 2023

Source: ANAO representation of data in the NDIS Quarterly report to disability ministers, September 2023.

2.56 While the NDIA reports quarterly results, it does not report the result for a financial year. Based on the quarterly averages, the NDIA met its target for the service standard in 2021–22 resolving 94.5 per cent of complaints within 21 days. In 2022–23 it did not meet the target, resolving 85.8 per cent of complaints within 21 days. For the December 2023 quarter, the NDIA reported that it met the timeframe in 71 per cent of cases.

2.57 The NDIA does not report in the NDIS Quarterly report to disability ministers or elsewhere on its website the results of its performance against the timeliness standard of acknowledging receipt of a complaint within one day or contacting the complainant within two days. Timeliness results by complaint stream is also not published.

2.58 The December 2023 NDIS Quarterly report to disability ministers states ‘Work is underway to enhance complaint handling practices and systems which will contribute to more timely complaint resolutions in line with the [Participant Service Guarantee] PSG in coming quarters’.²⁶

Internal reporting

2.59 There are a number of internal reports in place across NDIA teams relating to complaints; the two key reports are the monthly and weekly pulse reports.

NDIS monthly pulse reports

2.60 The NDIA Office of the Scheme Actuary prepares monthly pulse reports, which are provided to the NDIA’s Executive and its Board. The reports include the number of new complaints and contain metrics for the NDIA Corporate Plan measures²⁷ as well as performance against the timeframes in the PSG and the Participant Service Charter 21-day timeframe for resolving complaints. Performance against the other two complaints timeliness service standards in the Participant Service Charter are not included in the monthly pulse reports.

Weekly pulse reports

2.61 In addition to reporting on the 21-day timeframe for complaints resolution, the weekly pulse reports²⁸ report on the one-day and two-day timeframes, for complaint acknowledgement and initial contact with the complainant respectively. The NDIA has a target of 100 per cent for both measures. The results for 2022–23 are set out in Figure 2.7. Based on the average results reported weekly, 87 per cent of complaints were acknowledged within one day and in 96.8 per cent of complaints the first contact was made within two days.

Figure 2.7: NDIA internal reporting of one and two-day timeliness targets

Note: The weekly results have been rolled into monthly results.

Source: ANAO analysis of NDIS weekly pulse reports.

Biannual complaints analysis report

2.62 Timeliness results for each of the complaints streams (see paragraph 1.5) against the 21-day complaints resolution service standard for 2022–23 were included in the NDIA’s June 2023 biannual Complaints Analysis report.²⁹ Results are shown in Figure 2.8.

Figure 2.8: Complaints closed within 21-day timeframe in 2022–23, by complaints stream

Source: ANAO analysis of the NDIA’s June 2023 biannual Complaints Analysis report.

Remedial action when timeframes are not met

2.63 There are no documented procedures for when timeframes are not met, however the NDIA advised the ANAO in August 2023 that there are activities in place, such as:

• focus days — days when attention to managing complaints takes priority over, for example administrative tasks³⁰;
• moving workforce — moving staff across the different complaints streams to provide additional support, as needed; and
• operational reporting — to monitor the age of complaints.

2.64 The start date for the 21-day service standard is the date the complaint was entered into CRM. In one of the complaints sampled by ANAO, an existing complaint record was closed, and a new complaint record created to reflect a change in the team managing the complaint (an intention for it to be subject to joint NDIA–NDIS Quality and Safeguards Commission investigation from that point). This meant that while the original complaint had not changed or been resolved, the 21-day service standard did not take effect until the date the second complaint was created, five days later. Complaints data integrity is discussed further from paragraph 3.17.

3.1 The Commonwealth Ombudsman’s Better Practice Complaint Handling Guide (Better Practice Guide)³¹ recommends Australian Government agencies have regular internal and public reporting on complaints management, use meaningful complaints data to improve continuously, and review complaint quality assurance and review frameworks every two years. The Better Practice Guide states:

Reflection is the necessary precursor to improvement. Reflective systems are self-critical and ensure complaint handling processes and complaint data are regularly reviewed and analysed. Reflective systems ensure effective actioning of individual complaints, and can identify trends, systemic issues and opportunities for improvement.

3.2 In relation to the types of complaints data agencies should report to management to support oversight of complaints and give insights into broader operations, the Better Practice Guide states:

At a minimum the executive should receive reports about complaint volumes and trends including data about complaint issues, causes and outcomes, systemic issues and relevant complainant data (for example, geographic, demographic, cohort information).

Does the NDIA adequately report on complaints management including implementation of outcomes?

Internal reporting of complaints management

3.3 The NDIA produces 30 ongoing reports that include complaints data. These vary by audience in the frequency and level of detail of complaints data included. The NDIA’s Board and Senior Leadership Team (SLT) receive entity-level performance reports (pulse reports) with summary level complaints data for their monthly meetings. The SLT also receives shorter, weekly versions of the Pulse reports. These reports do not include data on outcomes of complaints or complaints management quality. The managers and executive of complaints teams receive reports with operational level information and data on a daily, weekly, monthly, and quarterly basis. Table 3.1 shows the content of the reports received by the NDIA’s management and the Board. Further information about report frequency is in Appendix 5.

Table 3.1: Types of complaints data included in the NDIA’s internal management reports

Key: ✘ Not included ✔ Included

Note a: The SLT receives copies of all reports to the Board with complaints data.

Source: ANAO analysis of NDIA documentation.

Reasons for complaints

3.4 Internal management reporting of complaints reasons includes the most common results in the reporting period. Figure 3.1 below shows the top 15 complaint categories across all complaints for each of 2021–22 and 2022–23.

Figure 3.1: Top 15 participant complaint reason categories, 2021–22 and 2022–23

Note a: ‘s48 decisions’ refers to decisions made under section 48 of the National Disability Insurance Scheme Act 2013 (the NDIS Act) to reassess a participant’s plan, either at the participant’s request or the Chief Executive Officer’s initiative.

Source: ANAO analysis based on NDIA documentation.

3.5 While reason categories are aligned with NDIA functions, they generally did not give insight into causes of the complaint that readily would support continuous improvement opportunities. For example, complaints within the category of Supported Independent Living would require further analysis to understand the nature of or driver for the complaint. Where there are high numbers of certain reason categories, particularly over a period of time, the results may indicate systemic issues (see paragraph 3.48).

Complaints outcomes

3.6 There are 10 categories for complaints officers to choose from when recording a complainant’s desired outcome in CRM.³² For example, ‘action expedited’, ‘agency apology or acknowledgement’, and ‘better explanation.’ There are also ten categories for recording the actual outcomes delivered by the NDIA, for example, ‘desired action completed’, ‘review requested’, ‘unable to contact – record closed’ and ‘formal reply endorsed and issued.’ Analysis of complaint outcome can only be conducted when both the desired and actual outcome are matched together. The NDIA advised the ANAO in September 2023 that where existing categories do not apply, it uses free text within CRM to record the actions taken.

3.7 Complaint outcomes are not reported to the NDIA’s Board or SLT. The executive and managers of complaints teams receive regular reports with complaints outcome data for only two complaint categories: ministerial complaints and complaints about complaint handling. These reports record the outcome categories for listed complaints but do not include accompanying analysis of outcomes.

3.8 The executive and managers of complaints teams also receive the biannual NDIS Complaints and Participant Critical Incidents Analysis (Complaints Analysis) report. This report contains the numbers and percentages for the categories that complaints officers used to record the ‘desired complaint outcomes’ for all complaints in the six-month reporting period, for example, ‘action taken/expedited’, ‘explanation’, ‘apology/acknowledgement’ and ‘other’. The report also contains the numbers and percentages for the categories of ‘actual complaint outcomes’ delivered by the NDIA in the reporting period, for example, ‘desired action completed’, ‘formal reply endorsed and issued’, ‘general’, ‘withdrawn by participant/representative’, ‘internal review’ and ‘other’. In 2021–22, ‘desired action completed’ was the largest category of actual outcomes recorded (69.2 per cent), followed by ‘other’ (16.4 per cent).³³ Since the December 2022 edition of the biannual Complaints Analysis report, an appendix is included with mapping of actual outcomes against desired outcomes.³⁴ The only analysis of outcomes contained within the December 2022 and June 2023 biannual reports was the identification of the category of the most frequently desired outcome and the actual outcome category with the highest results in the reporting period.

3.9 When a closed complaint still has actions to be completed by another NDIA team, internal policy requires complaints officers to follow up and check the actions occurred. Complaints team leaders can access daily reports to check details of complaints marked for follow up and teams also receive monthly reports to support monitoring of any implementation actions post-closure. The status or timeliness of implementation of agreed outcomes or actions is not reported to responsible business areas or to the NDIA’s Executive.

Quality of complaints management

3.10 The NDIA’s Executive receives reports with complaints timeliness data but does not receive complaint management quality information. The Executive and managers of complaints teams receive the following reports each month with quality information:

• results of monthly quality assurance checks of random samples of complaints closed within the last month (see paragraph 2.35); and
• monthly reports on complaints of complaint handling (see paragraph 2.46).

3.11 The quality assurance reports also include observations as to what was done well that month and suggestions for improvements to be made based on the findings. Reports on complaints of complaints handling do not include analysis of the results. The details of complaints reviewed within the random sample are provided within the reports to support follow up action by team leaders.

Report consistency

3.12 Reports generally use a consistent structure and metrics for reporting on complaints from one reporting period to the next. Reports typically include trend detail, which is most clearly presented for timeliness data. For some metrics, particularly complaint reasons, reports focus on the most common (top) categories in the reporting period, which limits transparency, reliability and usefulness as results may change from one reporting period to the next.

3.13 Reports include additional correlation of data, such as complainant characteristics against complaint volumes or frequency. For example, the Internal Reviews and Complaints Quarterly Analytics and Insights reports include presentation of the top five complaint themes by top five disability types. With the exception of monthly quality assurance reports, regular reports include minimal accompanying qualitative analysis of results presented to give insights into potential scope for actions to be taken to improve results or outcomes.

3.14 Volume, population and complaint management quality data is regularly reported, and workflow and workloads are regularly monitored within relevant teams. The absence of more timely reporting on outcomes of complaints, implementation of outcomes, qualitative analysis of trends or correlation of complaints data with other service delivery performance data limits the effectiveness of the NDIA’s oversight of the complaints management process.

External reporting on complaints management

3.15 The NDIA publishes regular reports with details of its management of complaints on the NDIS website.³⁵

• NDIS Quarterly report to disability ministers — reports include data on complaint volumes, complainant types, who complaints are about, causes and timeliness of resolution.
• NDIA annual reports (up to 2021–22) — include data on the numbers and percentages of complaints open and closed within the NDIA’s 21-day service standard and the number of complaints transferred to or from other relevant agencies.³⁶
• Quarterly Participant dashboards — a series of 15 reports on cohorts of participants with selected disability types (for example psychosocial, autism and spinal cord injury), including data on the rate at which participants with the disability type make a complaint to the agency.³⁷

3.16 Accessible, including ‘easy read’, versions of the Annual Report and NDIS Quarterly report to disability ministers are available on the NDIS website. Participant Dashboards are published alongside transcripts to support accessibility. Between 2019 and 2021, the NDIA also published several one-off reports about certain participant groups³⁸, for example ‘Young adults in the NDIS’ that included data on complaint volumes and data on complaint resolution timeliness.

Integrity of complaints data

Data sources

3.17 Complaints data is entered into CRM manually by staff, except when complaints are made through the NDIA’s online participant and provider portals, where initial data is automatically recorded in CRM.

3.18 In October 2019, a new module for CRM ‘My Customer Requests’ was implemented with fields to support greater data recording, for example, the ability to record multiple related parties as the source of a complaint. The NDIA reported in the quarter three, 2019–20 NDIS Quarterly report to disability ministers that the previous module, ‘My Feedback’, was discontinued following the introduction of the new functionality. The ‘My Feedback’ module is still in use; the management response to a 2022–23 internal audit stated the planned enhancements to CRM did not occur to remove this module and that staff are instructed to use the newer module. The December 2022 biannual Complaints Analysis report states data was reported using both modules.

3.19 The Internal Reviews and Complaints Quarterly Analytics and Insights reports, Pulse reports and the biannual Complaints Analysis reports included caveats that results did not include data on complaints recorded in the new CRM (PACE). The caveats did not include any detail as to the estimated volume of complaints excluded from the reports. The NDIA’s 30 June 2023 and 30 September 2023 NDIS quarterly report to disability ministers included the same caveats and identified the numbers of PACE complaints excluded from the reported results. In November 2023, the NDIA advised the ANAO that PACE complaints data would be incorporated into reports from the December 2023 quarter onwards.

Data integrity

3.20 The NDIA applied the following mechanisms to support data integrity in relation to reporting on complaints:

• internal guidance to support accurate data entry, including a Standard Operating Procedure, Record and update a complaint, and a data dictionary reference guide to assist staff in accurately categorising complaints data;
• internal reports on CRM compliance³⁹; and
• manual reconciliation of data by the NDIA to support reporting.

3.21 The NDIA identified the following limitations to data accuracy in its biannual Complaints Analysis reports:

Some “complaints” are general enquiries rather than complaints so the total count of complaints may be overstated.

There are known data issues with linking participants to a LAC and/or ECEI partner. A complaint linked to a partner is not necessarily a complaint about that partner.

There are fields in the complaints data that are entered as free text and therefore require text mining and manual verification to categorise. As a result, there is a risk of miscategorising complaints.

Complaints with erroneous or missing dates have been excluded from the timeframes analysis.

Timeframes have been calculated using the recorded date of resolution. A complaint will be recorded as closed once the complaints officer has done everything in their power to resolve the issue although outstanding follow up with other business areas may still need to occur.

Not all complaints made about a provider will be lodged with the Agency. Some complaints about a provider will be lodged with the [NDIS Quality and Safeguards Commission (NSQC)] as participants are encouraged to approach the NQSC first.

3.22 While the NDIA acknowledges that the inclusion of some general enquiries it had received in its internal and external reports may overstate the total complaint numbers, the inclusion in reporting of other types of contact is consistent with the Better Practice Guide recommendation that feedback also be analysed for trends alongside complaints. For the June 2023 biannual Complaints Analysis report, the NDIA included the numbers of complaints with erroneous dates, reporting that this represented 0.03 per cent of total complaints received in 2022–23.

3.23 In the June 2023 NDIS Quarterly report to disability ministers, the NDIA stated that Participant Critical Incidents (PCIs)⁴⁰ ‘were captured in complaints data in previous reports, this is the first report to separate out PCIs.‘ In the 30 June 2022 NDIS Quarterly report to disability ministers, the NDIA reported having received 36,067 participant complaints in 2021–22. This figure was adjusted in the June 2023 version of the report, to 30,089 with 5,723 PCIs for 2021–22.⁴¹ While the change in methodology supports greater transparency of both complaints and PCIs, reported participant complaint numbers prior to June 2023 are not directly comparable with current or future reported results.

3.24 The August 2022 deep dive on complaints categorised as ‘dissatisfied with plan’ for the period January to June 2022 found of 500 records reviewed, 315 (63 per cent) per cent were incorrectly categorised and made recommendations to improve categorisation guidance (see paragraph 3.51).

3.25 In May 2023, the NDIA reported findings from a 2022–23 internal audit of complaints management to its Audit Committee, which included agreed management actions to improve data integrity.⁴² Appendix 4 sets out the audit findings relating to data integrity and agreed resolutions to be implemented. The internal audit also observed that some complaints were received directly in separate email inboxes managed by state and territory-based teams and ‘these complaints might be recorded and managed in off system databases maintained in Excel Spreadsheets or as interactions created in CRM.’ This observation does not appear in Appendix 4 as it was not the subject of a finding that required a resolution. In July 2023, the NDIA advised the ANAO that planned work to address the inconsistent processes was no longer being progressed.

Does the NDIA effectively review its complaints management framework?

Reviews of complaints management approach

3.26 In April 2019, NDIA contracted PwC to review its complaints management framework and processes to support an internal Complaints Improvement Program. In June 2019 PwC completed its program of work and provided the NDIA with a draft report. The draft PWC report proposed 14 initiatives to improve complaints management (see Table 3.2 and paragraph 3.33). There was no evidence of the NDIA receiving a final version of the report. PwC received full payment from the NDIA as specified in the contract.

3.27 As discussed in paragraph 2.26, the Commonwealth Ombudsman advised the NDIA in February 2022 that it was rated as level four maturity, based on its survey response. The Commonwealth Ombudsman identified actions the NDIA could take to raise its maturity level:

• undertake customer satisfaction surveys;
• monitor, evaluate and report on changes implemented because of complaints; and
• share insights internally and externally with other complaint handling agencies.

3.28 The findings of a 2022–23 internal audit of the NDIA’s complaints management processes were reported to the Audit Committee in May 2023. The audit was part of the NDIA’s internal audit assurance program for 2022–23 and was conducted between July and October 2022. The internal audit found ‘the end-to-end complaints management process has adequate and effective controls’ and two management actions were agreed to address findings relating to integrity of complaints data (see Appendix 4) and enhancing the quality assurance of complaints management.

3.29 In March 2023, the Chief Executive Officer (CEO) reported to the Board that they had initiated improvements to complaints management processes in January 2023. The CEO’s report set out the main observations of a high-level review and advised the Board the improvement work would be undertaken in three phases; no timeframes were specified. The CEO’s report did not provide a rationale for the review.

3.30 The high-level review of complaints was completed in March 2023 by a Senior Executive Service Band 2. The review led to the creation of a Complaints Enhancement Project (CEP).⁴³ The CEP planned to incorporate three phases of work to improve culture, capability, processes and the operating model and workflow. The CEP was also intended to include a review of the complaints data dictionary. In July 2023, the NDIA engaged Nous Group to deliver ‘short term specialist expertise to lead staff engagement to inform future state design and continuous improvement activities in phase 2 of the project’. The CEP was formally closed in November 2023, following submission of a closure report to the Executive project sponsor.

3.31 Table 3.2 compares key details of the reviews undertaken of NDIA’s complaints management framework since 2019. The table includes discussion of both reports from NDIA’s 2023 review, as one informed the CEP establishment and the other was a key CEP deliverable.

Table 3.2: Reviews of the NDIA’s future approach to complaints management

Source: ANAO, based on NDIA documentation.

3.32 While there are differences in each of the reviews’ findings and recommendations, the following themes are recurring:

• clarifying standards for complaints-related terminology;
• moving towards a more participant-centric culture that values complaints;
• improving training of staff across the NDIA;
• making improvements to supporting technology; and
• a focus on improved reporting, monitoring and evaluation for continuous improvement.

Implementation of recommendations

3.33 The 2019 draft PwC report set out 14 initiatives and three implementation options to achieve the practice improvements. In August 2019, the Audit Committee was advised by the Acting Deputy Chief Executive Officer, Corporate Services that:

Management needs to consider how it resources and funds the implementation of the staged implementation, plans, and roadmaps resulting from the project and expert consultant review, as well as communicating these initiatives across the Agency and obtaining support from ELT [Executive Leadership Team, since renamed to SLT] and staff.

3.34 There was no evidence of the NDIA agreeing on an implementation roadmap contained in the draft PwC report or having developed an alternative implementation plan to implement the recommended initiatives. Of the 14 initiatives proposed by PwC, NDIA records showed it had fully implemented two — definition of frequently used terminology and tiered management reporting — and partly implemented the following three initiatives or updated equivalents:

• staff training and development;
• performance management and KPIs; and
• technology upgrades.

3.35 As at September 2023, the NDIA had not fully implemented the Commonwealth Ombudsman’s suggestions to track changes implemented because of complaints or to share insights with other complaints handling agencies. While the NDIA undertakes monthly entity-level participant satisfaction surveys and identified the number of participant complaints as one of 11 leading indicators of participant satisfaction, links between complaints data and satisfaction results are not explicitly analysed or reported. The NDIA does not undertake separate complainant satisfaction surveys.

3.36 In response to the 2022–23 internal audit findings, the NDIA updated its Quality Assurance checklist and supporting procedural guidance in June 2023. The NDIA identified that remaining work to address complaints data integrity was reliant on the introduction of the NDIA’s new IT system, PACE, which commenced national implementation from 30 October 2023 (see footnote 5). The NDIA’s records show the action was closed as at 12 December 2023.

3.37 Nous Group completed its draft review report in August 2023 and provided this to the NDIA project owner. The Nous Group report was not included with CEP papers submitted to the SLT’s August 2023 meeting, in which the SLT approved project recommendations to incorporate the CEP activities of triaging and analysing executive complaints into business-as-usual activities. In September 2023, Nous Group submitted a final Action Plan report to the NDIA, setting out opportunities for improvement and recommended actions to achieve these, including updating the data dictionary. The Action Plan was presented to the NDIA Board in November 2023 for its information. The plan suggested action priorities but did not specify implementation timeframes.

Does the NDIA use its complaints management data to improve processes and service delivery?

Trends in complaints management

3.38 The NDIA’s Complaint and Participant Incidents Team (CPIT) Continuous Improvement Framework (the Quality Framework) identifies the mechanisms to monitor and continuously improve complaints management quality (see paragraph 2.31).

3.39 The NDIA’s monthly series of post-completion quality checks on complaints started in January 2022 for complexity level three complaints and was expanded in July 2022 to include complexity level two complaints (see paragraph 1.6). The questions used for the checks are focused on compliance with NDIA policy and procedure rather than a qualitative assessment of the quality of the overall response to the complaint (see Appendix 3).

3.40 The results of post-completion checks are aggregated into monthly reports (see paragraph 2.35). Pre-completion checks are not reported. As at September 2023, no deep dive audits of complaints management had been ‘undertaken on various programs of work within the CPIT as identified by data and at the request of the Branch Manager’ as specified in the Quality Framework.⁴⁴

3.41 The NDIA advised the ANAO in July 2023 that complaints teams reviewed the complaints quality data reports in fortnightly team meetings and that team leaders also considered the data to inform one-on-one feedback sessions with staff. The NDIA did not have records of the outcomes of these discussions. The NDIA has not yet consolidated its monthly analyses over 2022–23 to give a longitudinal view of quality.

Complaints management continuous improvement

3.42 Beyond reviewing reported data, the NDIA advised that its complaints teams applied the following business-as-usual approaches to identifying aspects of complaints management that require improvement⁴⁵:

• informal staff suggestions for potential improvements — these can be made through the Continuous Improvement Form on the complaints teams’ intranet page;
• discussion of complaints case studies in fortnightly complaints team meetings; and
• internal Community of Practice monthly meetings.

3.43 The NDIA has also implemented the following informal improvements to its complaints management processes since July 2019:

• requiring team leaders to check ‘Requests for Action’ before complaints officers send these out to the responsible business areas;
• sending SMS notifications to complainants prior to calling to ensure they are aware of an incoming call from NDIA complaints officers (which appear as ‘private number’); and
• team leaders reviewing written responses to ministerial complaints before sending, to improve their quality.

3.44 Prior to June 2023, the NDIA did not consistently record improvements it had made to its complaints management approach. A centralised complaints continuous improvement register was established in June 2023. As at July 2023 when the register was provided to the ANAO, it contained one action being implemented and one action that was not endorsed by the relevant supervisor but was marked as ‘in progress’. The focus of the register was to track actions taken in response to proposed improvement initiatives; while there was a column for recording feedback, there was no supporting plan or mechanism for reviewing any changes implemented.

3.45 The NDIA does not undertake structured monitoring of complaints improvement initiatives to determine if these were effective. Changes to the NDIA’s complaints practices, made in response to its March 2023 high-level review, were implemented without an identified baseline against which to assess the impact. Baseline data was not evident in further reports generated through the CEP beyond identification of existing complaints processes (see Table 3.2).

Link between complaints data and service delivery continuous improvement

3.47 At the time of audit fieldwork, the NDIA’s Complaints and Feedback Framework (Complaints Framework) stated that:

Complaints and feedback present an opportunity to support continuous process improvement through enhancing business practices and processes.

…

Managers and leaders at all levels are expected to cascade and discuss complaint reports with staff and teams to support an understanding of themes and systemic issues and how this links in to improving services for participants.⁴⁶

3.48 As discussed in paragraph 3.5, while the NDIA has standard reports on complaints data that identify the most common complaint reasons or issues, these do not adequately show causes or potential systemic issues to enable identification of areas for continuous improvement.

3.49 The Performance, Workload, Planning and Quality (PWPQ) branch within the NDIA’s Service Delivery group conducts ‘deep dive’ reviews of selected administrative decisions, emerging issues, processes and participant cohorts to inform continuous improvement in service delivery. The PWPQ branch reviews data from a range of sources relevant to the topic of each deep dive review, including:

• results of monthly quality audits undertaken by PWPQ of approved participant plans;
• results of checks on plan quality by other assurance teams;
• complaints data (including monthly complaints data on volumes, timeliness and top complaints reasons; and detailed quarterly and biannual reports on complaints);
• performance data in monthly Pulse reports (discussed at paragraph 3.3); and
• prior deep dive review analysis.

3.50 The PWPQ branch uses a prioritisation matrix template to capture key issues from the data sources above and support its monthly discussions of priority areas for continuous improvement activity. In completed matrices, data on complaints reasons was not explicitly linked to other quality data to inform analysis of areas needing continuous improvement or to inform selection of areas where further deep dive analyses are needed for understanding.

3.51 The PWPQ branch has completed one deep dive related to complaints — an August 2022 review into complaints with the reason category, ‘dissatisfied with plan’. The deep dive report states the complaints teams had identified an increase in complaints in each quarter of 2021–22, with ‘dissatisfied with plan’ complaints consistently in the top complaint categories and a 50 per cent increase in this category of complaint between quarter one of 2020–21 and quarter one of 2021–22. A sample of 500 complaints with this complaint reason category was reviewed; this recorded category was found to be correct for only 185 of the complaints (37 per cent). The deep dive identified improvements the NDIA could make to its data dictionary for complaints and recommended processes support more proactive resolution of some matters by complaints officers. As at October 2023, the NDIA’s review of the data dictionary was still in progress (see paragraphs 3.30 and 3.37)⁴⁷ and has not yet addressed the other recommendation. The deep dive also made broader service delivery recommendations, namely for teams to improve aspects of planning and communication with participants.

3.52 Regular and comprehensive analysis of complaints data is important as the broad range of potential reasons for complaints can indicate emerging trends or issues not readily apparent from analysis of other NDIA service delivery data alone. The NDIA’s linkage of data on complaints with Administrative Appeal Tribunal (AAT) reviews also requires further analysis as the reports containing these outcomes only identify areas requiring further investigations rather than identifying specific systemic issues.

3.53 The CEP completed three case study reports based on reviews of complaints within the Executive stream on: delays in NDIA decisions on Home and Living supports; multiple attempts by Support Coordinators to resolve plan issues; and resolving participant needs for a top-up of plan funding during an AAT review. Each case study included between four and five relevant complaints. The case study reports identified scope for the NDIA to improve multiple service delivery policies and processes across the organisation, as well as staff training. The case study reports did not include references to structured analyses of supporting evidence, such as that included in the deep dive review undertaken by PWPQ. The links between the observations from the complaints discussed in each case study and the recommended improvement actions were not explicitly documented or clear. The case study reports were submitted to the SLT for noting and shared with relevant NDIA business areas. The NDIA was unable to provide evidence the recommendations had been agreed to or implementation had started.

3.55 Trends in the management of complaints can be identified from the results of post-completion checks reported to the Executive and managers of complaints teams and shared with other business areas, although thematic results are not included in the reports. As discussed in paragraphs 2.31 to 2.36, the NDIA does not have a mature process for reviewing the effectiveness of its complaints management or considering complaints data alongside other service delivery performance data to identify areas for continuous improvement.

Appendix 1 Entity response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• Published a new NDIS Enquiries, Feedback and Complaints Policy document that includes further channels for making a complaint and clearer complaint complexity levels (paragraphs 2.6 and 2.8).
• Updated the Complaints and Feedback Framework in February 2024 (see paragraph 2.21).
• The Planning Essentials Program was updated in October 2023 to include reminders of NDIA’s ‘no wrong door’ policy for receipt of complaints (see note to Table 2.4).
• Re-released the NDIA’s June 2023 biannual Complaints Analysis report to correct an error identified by the ANAO (paragraph 2.46).
• Additional detail included in the biannual Complaints Analysis report to show the proportion of complaints excluded from reporting in the relevant period due to incorrect dates being entered into NDIA’s IT system (see paragraph 3.22).
• Changes to complaints reporting methodology to exclude Participant Critical Incidents from total number of complaints received (see paragraph 3.23).
• A decision to allocate extra resources for complaints data analysis and continuous improvement activities following the Complaints Enhancement Project (see paragraph 3.37).
• Established a central complaints continuous improvement register in June 2023 (paragraph 3.44).
• Analysed a small subset of complaints data through the Complaints Enhancement Project to identify potential areas for continuous improvement (see paragraph 3.53).

Appendix 3 Quality assurance post-completion checks

1. The following table sets out mandatory complaints management steps assessed within the NDIA’s monthly program of post-completion quality checks.

Table A.1: Mandatory complaints management steps checked in post-completion quality checks

Note a: ‘POI’ refers to proof of identity.

Source: NDIA documentation.

Appendix 4 Implementation of internal audit recommendations

1. The following table sets out the status of planned resolutions to findings from the NDIA’s 2022–23 internal audit of complaints management, as recorded by the NDIA’s internal audit team.

Table A.2: Reported status of implementation as at 21 March 2024 of data integrity findings from a 2022–23 internal audit of complaints management

Source: NDIA documentation.

Appendix 5 Internal reporting on complaints by audience

1. The following table outlines the content of reports with complaints data that are regularly received by the NDIA Board, Senior Leadership Team (SLT) and managers and executives of complaints management teams.

Table A.3: NDIA’s internal reports on complaints for 2022–23

Note a: The board did not hold meetings in December 2022 or June 2023.

Source: ANAO analysis of NDIA documentation.