Introduction

1.1 The Department of Social Services (the department or DSS) has responsibility for delivering:

A sustainable social security system that incentivises self-reliance and supports people who cannot fully support themselves by providing targeted payments and assistance.⁶

1.2 In 2021–22 the department was responsible for welfare payments totalling $124.7 billion, representing around 20 per cent of all Australian Government spending. These payments are made under unlimited special appropriations established by legislation (see Table 1.1).⁷

Table 1.1: Special appropriations for welfare payments, 2021–22

Note a: Payments do not add to total due to rounding.

Source: DSS, Annual Report 2021–22, Commonwealth of Australia, 2022, pp. 214–215.

1.3 Services Australia was established as an executive agency within the social services portfolio in February 2020. Its purpose is to ‘support Australians by efficiently delivering high-quality, accessible services and payments on behalf of the Government’.⁸ Through a longstanding bilateral arrangement, Services Australia is responsible for delivering welfare payments on the department’s behalf through its Social Security and Welfare program.⁹ To facilitate this, the Secretary of the department has delegated powers to assess, determine and make welfare payments to the Chief Executive Officer of Services Australia.¹⁰

1.4 Table 1.2 shows 2021–22 payment expenditure and recipient numbers for the department’s top ten welfare payments. Collectively these ten payments represented around 95 per cent of the department’s total welfare payments in 2021–22. A full list of the department’s welfare payments delivered by Services Australia is at Appendix 3, including coverage of these payments by bilateral key performance measures (KPMs) relating to payment accuracy and timeliness.¹¹

Table 1.2: Top ten welfare payments by expenditure, 2021–22

Note a: Family Tax Benefit recipient information is reported with a two-year lag, so 2021–22 figures were not available.

Note b: Number of individuals and families that started receiving Parental Leave Pay in the financial year.

Source: DSS, Annual Report 2021–22, Commonwealth of Australia, 2022, pp. 46–47 and 243–244.

Bilateral arrangement between the department and Services Australia

1.5 The bilateral arrangement between the department and Services Australia began as a ‘purchaser-provider’ arrangement established in 1997 between the newly created Centrelink and the Department of Social Security. Under a ‘purchaser-provider’ arrangement, one entity purchases services from another entity, with agreed expectations and service levels set out in bilateral agreements. From 1997–98 Centrelink began reporting to the department against bilateral KPMs for the accuracy and timeliness of welfare payments it made on the department’s behalf.

1.6 In 2011 the bilateral relationship changed from a ‘purchaser-provider’ arrangement to an ‘appropriated partnership’ arrangement (when Centrelink became part of the Department of Human Services). Under an ‘appropriated partnership’ arrangement, one entity is accountable for delivering services and prioritising service delivery within its funding budget (appropriation); and the other entity retains policy responsibility for those services.

1.7 For welfare payments, the Secretary of the department is accountable for delivery of welfare payment outcomes and all payment-related functions that have been delegated to Services Australia, including payment accuracy and timeliness.¹² The Chief Executive Officer (CEO) of Services Australia is accountable for whole-of-government service and payment delivery and Services Australia receives direct funding to manage these functions through a departmental appropriation ($4.6 billion in 2022–23).¹³ Services Australia has bilateral arrangements with the department and other entities (such as with the Department of Health and Aged Care for delivery of Medicare services) and needs to prioritise service and payment delivery across these arrangements within its allocated funding. Accordingly, to ensure that welfare payment delivery achieves its policy objectives, the department must establish a robust bilateral arrangement with Services Australia that enables it to effectively oversee the delivery of its delegated functions and resolve competing risk priorities.

1.8 The current Bilateral Management Arrangement (BMA) between the department and Services Australia comprises: a high-level Head Agreement; and a series of subordinate bilateral agreements (called ‘protocols’ and ‘service schedules/arrangements’) outlining how the entities will work together to achieve desired outcomes.¹⁴ Reporting against KPMs for the accuracy and timeliness of welfare payments still forms a central component of the BMA. The current Payment Assurance Service Arrangement (dated April 2016) sets out that: Services Australia is responsible for implementing payment assurance activities, monitoring their effectiveness and providing feedback to the department; and the department is responsible for reporting to government on outcomes and issues.

1.9 Under the BMA, the department reports on welfare payment accuracy in its annual report, Services Australia reports on payment accuracy and timeliness to the Secretary of the department through an annual assurance statement, and Services Australia reports on payment correctness and timeliness in its annual report.¹⁵ In addition, the department and Services Australia provide quarterly performance reporting to bilateral governance bodies comprised of senior executives from both entities.

Payment accuracy

1.10 Welfare payments are considered ‘accurate’ if Services Australia pays ‘the right person the right amount through the right program at the right time’.¹⁶ Inaccurate payments can be either overpayments or underpayments, and can result from fraud, corruption, recipient error (such as inadvertent non-compliance with requirements to report changes in circumstances¹⁷) and/or administrative error (such as errors in Services Australia’s payment processing).

1.11 From July 2002 Centrelink, on behalf of the department, began using the results of a random sample survey of welfare recipients to assess the accuracy of welfare payments. As of June 2023, this survey-based approach, now called the Payment Accuracy Review Program (PARP), was still the primary mechanism for assessing the accuracy of the department’s payments. Broadly, the PARP involves:

• identifying a random sample of welfare payment recipients;
• undertaking reviews of their eligibility and circumstances (PARP reviews) to determine if they are receiving the right payment for the right amount on a particular fortnight; and
• using PARP review results to estimate payment accuracy (also referred to as ‘payment integrity’ or ‘integrity of payment outlays’) for the population of welfare recipients.

1.12 The department and Services Australia use data generated through the PARP to report to bilateral governance bodies on a trimester basis on performance against KPMs for the accuracy of individual welfare payments. Services Australia uses the same data to report on accuracy KPM performance to the Secretary of the department annually through its annual assurance statement. The department also reports on welfare payment accuracy in its annual report using the results of the PARP.

1.13 As shown in Figure 1.1, the overall payment inaccuracy rate for the department’s welfare payments, as measured through the PARP, has increased since 2013–14. The inaccuracy rate went above five per cent (representing the department’s payment accuracy target of 95 per cent) for the first time in 2018–19 and was above the target in 2020–21 and 2021–22. In 2021–22 the department estimated that the total value of overpayments was $7.174 billion (6.0 per cent of total payments), and the total value of underpayments was $514 million (0.4 per cent of total payments). Individual payment accuracy KPMs and reported results from 2018–19 to 2021–22 are reproduced at Appendix 4.

Figure 1.1: Payment inaccuracy rate, 2013–14 to 2021–22

Source: ANAO analysis of DSS annual reports from 2013–14 to 2021–22. DSS did not report an overall accuracy rate in its 2020–21 annual report, so the result for that year is sourced from internal reporting.

1.14 In 2022–23 Services Australia had 150 operational staff working on the PARP (excluding support staff and managers).¹⁸ Figure 1.2 shows Services Australia has conducted an average of around 21,000 PARP reviews of individual welfare recipients per year since 2013–14. In 2019–20 PARP reviews were suspended for trimester three (1 March to 30 June 2020) due to Services Australia’s COVID-19 pandemic response.¹⁹ The department and Services Australia further agreed that three payments (Disability Support Payment, Parenting Payment Partnered and Parenting Payment Single) would not be reviewed in trimester three 2021–22 (1 March to 30 June 2022) due to the diversion of PARP staff to priority activities in response to the continuing COVID-19 pandemic and natural disaster events (particularly, New South Wales and Queensland floods in early 2022).

Figure 1.2: Number of PARP reviews conducted, 2013–14 to 2021–22

Source: DSS annual reports from 2013–14 to 2021–22.

Payment correctness

1.15 Services Australia has also used PARP data to report on welfare payment correctness in its annual report. Centrelink first publicly reported a correctness result based on PARP data in 2004–05, and the calculation method has not changed since then. Services Australia has defined payment correctness as ‘the percentage of [PARP] reviews without a dollar impact error based on information provided by the customer’²⁰, and it has used this performance measure to assess the correctness of its administrative decision making and processing.²¹ As shown in Figure 1.3, the percentage of PARP reviews identifying dollar impact administrative errors has steadily decreased since 2009–10, and has been consistently below five per cent (representing Services Australia’s payment correctness target of 95 per cent).

Figure 1.3: Percentage of PARP reviews with dollar impact administrative errors, 2004–05 to 2021–22

Source: ANAO analysis of Centrelink annual reports from 2004–05 to 2010–11, DHS annual reports from 2011–12 to 2017–18, and Services Australia annual reports from 2018–19 to 2021–22.

Payment timeliness

1.16 Individuals can claim most welfare payments online using a ‘Centrelink online account’ through the myGov platform. In 2021–22 Services Australia reported that it processed more than 4.5 million welfare claims for the department, of which around 3 million were granted and 1.5 million rejected.²²

1.17 ‘Payment timeliness’ is a measure of the time it takes Services Australia to process claims for welfare payments. The department and Services Australia have agreed KPM benchmarks for the timeliness of specific payments — for example, the benchmark for the Crisis Payment is the completion of 90 per cent of claims within two calendar days, whereas for the Disability Support Pension it is completion of 80 per cent of claims within 84 calendar days.

1.18 Services Australia has reported against an aggregate welfare payment timeliness performance measure in its annual report since 2015–16 with a performance target of 82 per cent of claims processed within agreed KPM benchmark standards.²³ Figure 1.4 shows Services Australia’s performance has been variable, achieving the target in four of the past seven years.

Figure 1.4: Welfare claim processing timeliness, 2015–16 to 2021–22

Source: ANAO analysis of DHS annual reports from 2015–16 to 2017–18 and Services Australia annual reports from 2018–19 to 2021–22.

1.19 The department and Services Australia report on timeliness KPMs for individual welfare payments to bilateral governance bodies on a quarterly basis, and Services Australia reports on timeliness KPM performance to the Secretary of the department annually through its annual assurance statement. Payment timeliness KPMs and results from 2018–19 to 2021–22 are reproduced at Appendix 5.

Rationale for undertaking the audit

1.20 Effective management of welfare payment processing to ensure recipients receive ‘the right payment at the right time’ is central to the bilateral arrangement between the department and Services Australia. Accuracy and occurrence of personal benefits is a key audit matter for the ANAO’s financial statements audit program and a key area of financial statements risk for the department due to: the high volume and varying complexity of payments processed by Services Australia on complex IT systems; and the reliance on correct disclosure of personal circumstance information by a large number of recipients across diverse socioeconomic groups.²⁴ This audit was conducted to provide assurance to Parliament on the department’s and Services Australia’s processes for monitoring, reporting and continuously improving welfare payment accuracy and timeliness.

Audit approach

Audit objective, criteria and scope

1.21 The objective of the audit was to assess the effectiveness of Department of Social Services’ and Services Australia’s management of the accuracy and timeliness of welfare payments.

1.22 To form a conclusion against the objective, the following high-level criteria were adopted:

• Has the department established effective oversight arrangements for the accuracy and timeliness of Services Australia’s payment delivery?
• Have the department and Services Australia established effective processes for monitoring, reporting and continuously improving payment accuracy?
• Have the department and Services Australia established effective processes for monitoring, reporting and continuously improving payment timeliness?

1.23 The ANAO’s analysis of payment accuracy and timeliness reporting examined the period of July 2018 to June 2022 and, where relevant, the ANAO examined key events up to and including June 2023.²⁵ The ANAO did not directly examine Services Australia’s fraud and corruption control or debt management activities (including the Online Compliance Intervention System for Debt Raising and Recovery, colloquially known as Robodebt).²⁶ Debt management was the subject of a separate performance audit, which was presented to Parliament in May 2023.²⁷

Audit methodology

1.24 To address the audit objective and criteria, the ANAO:

• examined documentation held by the department and Services Australia, including governance body papers and minutes, internal and external reporting, and email records;
• conducted process reviews for payment accuracy and timeliness monitoring activities, including reviewing procedural documentation and quality assurance processes;
• extracted and analysed monitoring data from 2018–19 to 2021–22 to verify reporting on payment accuracy and timeliness;
• held meetings with staff of the department and Services Australia; and
• invited input from external stakeholders (three contributions were received from two individuals).

1.25 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $614,000.

1.26 The team members for this audit were Daniel Whyte, Anne Kent, Ewan McPherson, James Carrington, Magdalena Carrasco, Nathan Daley, Qing Xue, Zhuo Li, Xiaoyan Lu and Alexandra Collins.

1.27 The ANAO has co-operative evidence gathering arrangements in operation with entities. On 3 February 2023 Services Australia advised the ANAO that it was unable to voluntarily provide certain information requested by the ANAO due to legislative restrictions on the disclosure of requested information. On 9 February 2023 the Acting Auditor-General issued Services Australia with a notice to provide information and produce documents pursuant to section 32 of the Auditor-General Act 1997 to enable it to provide the requested information taking account of legislative requirements. Services Australia provided the information requested within the specified time, following receipt of the notice.

2.1 As noted at paragraphs 1.6 and 1.7, the department and Services Australia have an ‘appropriated partnership’ bilateral arrangement.

• The department is responsible for welfare payment policy and for administration of special appropriations established for welfare payments (see paragraph 1.1). The Secretary has delegated authority for delivering and managing welfare payments to the Services Australia Chief Executive Officer (CEO). The Secretary of the department is accountable for delivery of welfare payment outcomes and all functions delegated to Services Australia, including payment accuracy and timeliness.
• Services Australia receives a direct appropriation for delivering government services and payments. The CEO of Services Australia is accountable for prioritising service and payment delivery within this allocation.

2.2 To support effective oversight of an appropriated partnership, the entity accountable for the services needs to establish robust bilateral agreements with the delivery entity, which should include clear objectives, defined roles and responsibilities, and effective processes for issue identification and dispute resolution. Bilateral governance bodies should be established at appropriate levels to ensure proactive and strategic management and oversight of the services being delivered. Bilateral arrangements should include appropriate assurance processes that provide the accountable entity with an objective and independent assessment of delivery mechanisms.²⁸ Robust processes for identifying, communicating and managing shared risks are also important, particularly when the accountable entity’s and delivery entity’s risk tolerances differ.²⁹ This chapter examines whether the department has established an effective bilateral arrangement with Services Australia that supports its oversight of welfare payment accuracy and timeliness and includes appropriate processes for assurance and shared risk management.

Do bilateral agreements adequately support the department’s oversight of payment accuracy and timeliness?

Structure of the Bilateral Management Arrangement

2.3 A machinery-of-government change in September 2013 brought responsibility for welfare payments together within the department.³⁰ In October 2014 the department established a Bilateral Management Arrangement (BMA) with Services Australia³¹, comprising:

• a head agreement — which outlined the purpose, outcomes and principles of the BMA, defined each entity’s roles and responsibilities, established governance and operational arrangements for managing the BMA, and included a bilateral assurance framework;
• protocols — subordinate agreements outlining processes, frameworks and guidelines to support operational arrangements; and
• service schedules/arrangements — subordinate agreements relating to the payments, programs and services that Services Australia delivered on the department’s behalf.

2.4 In April 2018 the accountable authorities of the department and Services Australia signed a Statement of Intent, which replaced the 2014 BMA Head Agreement. In April 2023 the accountable authorities signed a new BMA head agreement, which superseded the two previous agreements. The 2023 Head Agreement stated that subordinate agreements entered into under the previous agreements remained in effect and bilateral protocols would be revised over the first three to six months of the new agreement (by July to October 2023). BMA agreements in effect as of June 2023 are outlined in Appendix 6.

Bilateral agreement coverage of payment accuracy and timeliness

2.5 As of June 2023 four service arrangements (pre-dating the 2023 Head Agreement) included content related to the accuracy and timeliness of welfare payments. In addition, the 2016 Payment Assurance Service Arrangement includes reference to an additional subordinate agreement with related content — the Random Sample Surveys Standing Operational Statement, which was last updated in May 2019. Table 2.1 outlines the ANAO’s assessment of the completeness and currency of this content. In summary:

• all relevant subordinate bilateral agreements were out of date, with commitments to annual reviews included within the agreements not having been met; and
• content related to payment accuracy and timeliness was incomplete and out of date (all agreements included references to ceased payments and/or did not include all current payments and revised key performance measures, and some agreements did not include entity roles and responsibilities or had outdated references to issues resolution processes).

Table 2.1: ANAO assessment of completeness and currency of BMA subordinate agreement content related to welfare payment accuracy and timeliness

Key: ✔ yes ✘ no

Note a: Agreements were assessed to be incomplete if there was relevant content that was not included (such as new or updated KPMs for welfare payments, entity roles and responsibilities, or issues resolution processes).

Note b: Agreements were assessed to be not current if there was relevant content that was out of date (such as payments or processes that had ceased) or they had not been updated in accordance with the review period defined within the agreement.

Source: ANAO analysis of DSS and Services Australia documentation.

Attempts to refresh the bilateral arrangement

2.6 In 2016 the department commissioned a review of entity accountabilities and the operation of the BMA (‘Review of Accountabilities — The Bilateral Management Arrangement’), led by Ms Peta Winzar.³² The review was finalised in August 2016 and made 22 recommendations for reforming the BMA, including an overarching recommendation that:

DSS and [Services Australia] develop a new, transformative partnership agreement which:

• explicitly aims to transform policy and delivery by collaborating,
• reflects the separate, shared and reciprocal accountabilities of both departments, and
• which uses a control-assure-improve model of accountability to design a performance framework which aligns with the outcomes set out in the DSS Portfolio Budget Statement, and employs a risk-based and proportionate suite of assurance mechanisms.

2.7 The department and Services Australia did not document their responses to the recommendations. Following the Review of Accountabilities, the department and Services Australia initiated a BMA review project in late 2016 with the aim of having revised arrangements in place for 2017–18. In November 2016 the department told the Joint Committee of Public Accounts and Audit that the BMA review would be completed in early 2017.³³ The project was subsequently delayed due to the entities prioritising other activities (such as developing Budget proposals and revising the annual assurance statement process), and a revised timeline was agreed in April 2018 that committed to bilateral agreements being finalised between June 2018 and February 2019. The Statement of Intent (see paragraph 2.4) and some bilateral protocols were finalised in 2018 and early 2019. Core elements of the BMA review project relevant to welfare payment timeliness and accuracy (such as an independent review of bilateral KPMs and a revised assurance framework) were not progressed.

2.8 A November 2019 internal audit of welfare payment processing undertaken by the department found there was limited evidence that the department had accepted or rejected the recommendations of the 2016 Review of Accountabilities or put in place management actions to address them. The internal audit included a recommendation, agreed by management, that the department redesign the existing BMA in consultation with Services Australia, including updating the Statement of Intent and reshaping arrangements and protocols. Implementation of the recommendation has been delayed, with reporting to the department’s audit committee from 2020 to 2022 citing reasons such as: the establishment of Services Australia; the COVID-19 response; prioritising work on addressing payment inaccuracy; and implementing recommendations from an Australian National University review of the methodology for measuring payment accuracy (discussed in paragraphs 3.5 to 3.10).

2.9 A renewed BMA project was initiated in 2022 with the aim of developing a revised suite of bilateral agreements by the end of 2022. The renewed project has also been delayed. As of June 2023, the only output of the project was the April 2023 Head Agreement, which includes a revised governance structure for the refreshed BMA (discussed at paragraph 2.16). None of the proposed subordinate agreements relating to payment accuracy and timeliness had been finalised.

Has the department had adequate oversight of payment accuracy and timeliness?

Bilateral governance structure

Bilateral governance bodies

2.14 As described at paragraph 2.1, the Secretary of the department is accountable for welfare payment outcomes and the CEO of Services Australia is accountable for whole-of-government service and payment delivery. Beneath the accountable authorities, the 2014 Head Agreement established a governance structure centred around a Senior Executive Service (SES) Band 3 level Bilateral Management Committee (BMC). Reporting to BMC were six SES Band 2 level Strategic Business Discussion (SBD) committees³⁴ and a Systems Access and Management Information (SAMI) Committee. In December 2016 the department and Services Australia reduced the number of BMA governance bodies to three, retaining BMC, a single SBD and SAMI. This was originally intended as an interim measure, pending the outcomes of a review of BMA governance arrangements (which was not completed).

2.15 The interim structure was subsequently incorporated into the 2018 Statement of Intent. Under the Statement of Intent, BMC and SBD were responsible for oversight of payment accuracy and timeliness, with the agreement stating that BMC ‘monitors program and payment performance’ and SBD ‘provides oversight of program performance and payment outcomes’. This governance structure, which remained in place until March 2023, is depicted in Figure 2.1.

Figure 2.1: BMA governance structure, as of March 2023

Note a: While not a requirement of the Bilateral Management Arrangement (BMA) until it was included in the 2023 BMA Head Agreement, by convention the Services Australia CEO has provided an annual assurance statement to the Secretary reporting against ‘bilateral assurance areas’ outlined in the 2014 BMA Head Agreement. The five bilateral assurance areas were: 1. Integration of policy and service delivery; 2. Shared understanding of policy and delivery outcomes sought; 3. Systems, data and management information continuity and interoperability; 4. Effective relationship management; and 5. Payment assurance.

Source: ANAO analysis of DSS and Services Australia documentation.

2.16 The April 2023 BMA Head Agreement outlines a new governance structure, which as of June 2023 was in the process of being established, that involves:

• meetings between the Secretary of the department and Services Australia CEO at least twice a year;
• three SES Band 3 level meetings occurring twice yearly covering social security and family payment integrity, community and disability policy and program delivery, and whole of portfolio matters;
• three SES Band 2 level SBD meetings and one SES Band 2 level data forum occurring quarterly³⁵; and
• a series of other ad hoc and scheduled working group and committee meetings at an SES Band 1 level.

Bilateral governance body oversight

2.17 The ANAO examined BMC and SBD meeting papers and minutes from 2018–19 to 2021–22 to assess the adequacy of these bodies’ oversight of payment accuracy and timeliness. BMC and SBD generally met quarterly over this period, except for a ‘pause’ from late 2019 to late 2020 attributed to the establishment of Services Australia and the COVID-19 response.

2.18 As shown in Figure 2.2, all BMC and SBD meetings held over the four years included discussion of payment accuracy and/or timeliness matters. Discussion generally focussed on changes in the quarterly KPM results. Limited progress was made over this period on strategic projects, such as the BMA refresh and development of a revised assurance framework.

• During 2018–19 BMC and SBD discussed both payment accuracy and timeliness issues. In that year most payments fell short of timeliness KPM targets³⁶, and aggregate payment accuracy results failed to meet the department’s 95 per cent target for the first time.
• After bilateral meetings resumed in 2020–21 discussion focused on topics relating to the measurement methodologies (such as reviewing the payment accuracy methodology and timeliness KPMs), particularly at SBD meetings. In addition, there was increased focus on payment accuracy during 2020–21 and 2021–22 as KPM results deteriorated.

Figure 2.2: BMC and SBD discussion topics^a, quarter one 2018–19 to quarter four 2021–22^b

Note a: BMC and SBD discussion topics were determined through analysis of meeting minutes. Agenda items were assessed as having multiple discussion topics when separate topics were discussed under a single item. Topics were classified based on whether they related to: payment timeliness performance; payment accuracy performance; methodologies for measuring payment timeliness or accuracy; BMA matters (for example, the BMA review project); and other matters (for example, policy matters, upgrades to payment processing systems, etc.).

Note b: Quarter in which meeting was held, rather than for which performance reporting was presented. BMC meetings were paused from Q2 to Q4 2019–20 and no BMC meeting was held in Q3 2021–22. No SBD meeting was held in Q1 2018–19 and SBD meetings were paused from Q3 2019–20 to Q1 2020–21.

Source: ANAO analysis of DSS and Services Australia documentation.

Reporting to SBD and BMC

2.19 The department’s 2019 internal audit of welfare payment processing (discussed at paragraph 2.8) found that:

• BMC and SBD had not clearly defined their expectations for reporting; and
• information being provided was not transparent and was insufficient to inform strategic decision-making and demonstrate their responsibilities had been acquitted.

2.20 The audit made a recommendation to clearly outline the information and reporting expectations for each committee. While the terms of reference for BMC and SBD were updated in December 2021 in response to this recommendation, the revisions were superficial and did not address the substance of the recommendation relating to information and reporting expectations.

2.21 The ANAO’s analysis of papers presented to BMC and SBD from 2018–19 to 2021–22 found that the committees received regular reporting on KPM performance. Less frequently, papers were presented that provided detailed analysis of the drivers of accuracy and timeliness performance.

• The department and Services Australia prepare a quarterly performance assurance report, which includes reporting against accuracy and timeliness KPMs and commentary from Services Australia on KPMs not achieving benchmark targets. The report was provided to most SBD meetings and two thirds of BMC meetings held from 2018–19 to 2021–22 (although BMC only received an executive summary in 2018–19).
• One paper was presented over the period on payment timeliness performance. In April and May 2019 the department and Services Australia presented a paper to SBD and BMC that provided quantitative analysis of Newstart Allowance claim processing timeliness, following a rise in the number of unfinalised claims.
• Papers on payment accuracy performance were more frequent, starting from the beginning of 2021 as KPM performance declined. These papers noted a high level of payment inaccuracy risk associated with JobSeeker Payment recipients incorrectly reporting their earned income, and discussed remediation actions being taken by Services Australia.

Accountable authority oversight

2.22 In addition to the regular oversight of payment accuracy and timeliness through bilateral governance bodies, there has been engagement at the accountable authority level. The Secretary of the department has written to the Services Australia CEO on three occasions since 2019 raising concerns about payment accuracy and timeliness results.

• In October 2019 the Secretary expressed concern that Services Australia’s 2018–19 Annual Assurance Statement did not include detail on claim processing timeliness and requested such information be included in future. Services Australia’s subsequent annual assurance statements in 2019–20, 2020–21 and 2021–22 included reporting on payment timeliness KPM performance.
• In April 2021 the Secretary expressed concern about payment accuracy results and requested detailed analysis of factors driving inaccuracy and a remediation plan. The Services Australia CEO responded in May 2021 identifying the key drivers of inaccuracy as: recipients failing to report changes in their circumstances; and a large cohort of new welfare recipients during COVID-19 who were unfamiliar with the welfare system. The CEO’s letter also noted actions being taken to address the deficiencies, including obtaining assurance and advice from KPMG on COVID-19 activities and factors contributing to inaccuracy and using Single Touch Payroll (STP) data to pre-fill recipients’ employment reporting forms.³⁷
• The Secretary wrote to the CEO again in November 2022, in response to commentary on STP in Services Australia’s 2021–22 Annual Assurance Statement, requesting ‘an assurance around the role of STP in improving payment accuracy’. The Services Australia CEO responded in December 2022 providing a progress update on payment accuracy initiatives, including STP. After this correspondence, the department and Services Australia began working together at the Senior Executive Service officer level to analyse the impact of STP on payment accuracy through the Payment Accuracy Review Program (PARP) (this work is discussed further in paragraphs 3.115 and 3.116).

2.23 The department informed the ANAO that the Secretary and Services Australia CEO have had more engagement on payment accuracy since mid-2021 but meetings were not well documented. Talking points prepared by the department for the meetings covered topics such as: actions to address payment inaccuracy; analysis of STP impacts on payment accuracy; and progressing the BMA refresh.

2.24 Raising bilateral governance to the accountable authority level and formalising the engagement through maintaining appropriate records of meetings has the potential to improve the department’s oversight of payment accuracy and timeliness. As noted at paragraph 2.16, the governance arrangements outlined in the 2023 BMA Head Agreement involve a biannual meeting between the accountable authorities, as well as establishing more SES Band 3 and Band 2-level bodies.

Have appropriate assurance arrangements been established to support the department’s oversight?

Bilateral payment assurance arrangements

2.25 As the department’s welfare payments are delivered by Services Australia, the Secretary of the department needs to obtain sufficient assurance that Services Australia’s control framework for welfare payment processing is operating effectively.³⁸

2.26 The 2018 Statement of Intent included a section on ‘Assurance’ that stated:

DSS and [Services Australia] work cooperatively to provide assurance to each Secretary about the efficiency and effectiveness of social welfare policies and administration.

The assurance framework sets out the specific assurance measures used to demonstrate effective collaboration on payment accuracy and risk identification and management across the bilateral relationship. The assurance process is designed to comply with commonwealth risk management policies and supports corporate requirements and responds to contemporary advice from the ANAO.

The assurance framework is outlined in detail in the assurance addendum to the Special Appropriations Service Arrangement.³⁹

2.27 The Special Appropriations Service Arrangement and ‘assurance addendum’ referenced in the Statement of Intent were not developed, meaning there has been no bilateral assurance framework in place since 2018.

2.28 The 2014 BMA Head Agreement had included a bilateral assurance framework that identified two assurance reporting mechanisms:

• the performance assurance report — which includes reporting on performance against bilateral KPMs (such as the payment accuracy and timeliness KPMs) and commentary on program achievements and issues; and
• Services Australia’s annual assurance statement — which is provided by the Services Australia CEO to the Secretary of the department after the end of each financial year.

2.29 From 2018, in the absence of a current bilateral assurance framework, the assurance reporting mechanisms referenced in the 2014 BMA Head Agreement have continued to operate. As noted in Figure 2.1, Services Australia’s annual assurance statement has continued to include reporting against the ‘bilateral assurance areas’ outlined in the 2014 BMA Head Agreement. Nevertheless, neither assurance mechanism provides sufficient assurance to the Secretary over the effectiveness of Services Australia’s welfare payment controls since neither provides an objective and independent assessment of governance, risk management and control processes.

• Reporting against accuracy and timeliness KPMs informs both the performance assurance report and annual assurance statement. While these KPMs provide useful high-level indicators of program performance, they do not provide a direct assessment of the effectiveness of Services Australia’s control framework. When KPM targets are met, it is possible to infer that controls are operating effectively; but when targets are not met, KPM results alone provide limited information about control weaknesses.⁴⁰
• Additional information provided in the performance assurance report and annual assurance statement largely consists of Services Australia’s commentary about likely causes of KPM targets not being met and high-level descriptions of actions Services Australia is taking to respond. As this content is prepared by Services Australia in response to identified performance issues, it cannot be considered objective or independent, and does not serve its purpose of providing assurance to the department.

Entity reviews and audits relating to assurance

2.30 Both the department and Services Australia have commissioned internal reviews and audits in recent years that have identified deficiencies in assurance arrangements relating to activities governed by the BMA (see Box 1). Common themes emerging from these reviews and audits and the relevant entity’s response include that:

• the department’ reliance on high-level KPM reporting and assurances from Services Australia does not provide sufficient assurance over welfare payment control effectiveness;
• the department must work collaboratively with Services Australia to gain a deeper understanding of welfare payment controls (such as by conducting assurance mapping) and determine a more robust bilateral assurance framework; and
• findings of reviews and audits have not been adequately considered at a bilateral level, and recommendations have largely not been implemented.

Note a: Assurance mapping is ‘a mechanism for linking assurances from various sources to the risks that threaten the achievement of an organisation’s outcomes and objectives’. HM Treasury, Assurance Frameworks, December 2012, p. 6.

Exchanging information on welfare payment internal audits

2.31 The 2016 Review of Accountabilities noted that, while the department did not undertake a significant level of internal audits into welfare programs, Services Australia had an extensive internal audit program. It recommended the two entities exchange information on relevant components of their internal audit programs. Following the review, the department and Services Australia began sharing information on their internal audit programs through the BMC in December 2016. However, after June 2018 no evidence was found of internal audits being discussed at BMC meetings.

2.32 In August 2019, after considering Services Australia’s 2018–19 Annual Assurance Statement, the department’s audit committee initiated two action items, requesting that:

• Services Australia acknowledge internal audits performed in future statements; and
• the department’s Head of Internal Audit provide an update to the committee’s next meeting on internal audits agreed between Services Australia and the department under the BMA.

2.33 The first item was closed at the next audit committee meeting in September 2019, with the committee noting that Services Australia’s Branch Manager of Financial Accounting (who had attended the meeting) would ‘take the action on notice’. Subsequently, there was one reference in Services Australia’s 2020–21 Annual Assurance Statement to an internal audit of Paid Parental Leave, and no references to internal audits in either the 2019–20 or 2021–22 statements.

2.34 The ANAO examined internal audits presented to Services Australia’s audit committees between 1 July 2018 and 31 December 2022 to assess their relevance to welfare payment accuracy and timeliness. As shown in Table 2.2, Services Australia undertook 123 internal audits over the period, including 23 that were relevant to payment accuracy and/or timeliness.⁴¹

Table 2.2: Services Australia’s internal audits relevant to payment accuracy and/or timeliness, July 2018–December 2022

Note a: Until 31 December 2022.

Source: ANAO analysis of Services Australia information.

2.35 In November 2019 the department’s audit committee agreed to close the second item and move assurance collaboration to be a regular part of the Head of Internal Audit report. No evidence was found of information on Services Australia internal audits or assurance collaboration being presented to the audit committee after that time.⁴²

BMA refresh

2.36 The 2023 BMA Head Agreement has an attachment setting out high-level requirements for Services Australia’s annual assurance statement⁴³, and includes a ‘strategic principle’ on ‘assurance of program performance’, which states:

Both entities acknowledge the importance of the Australian Government’s and customers’ trust in the successful delivery of social services programs and support. This remains a key element of the unique bilateral relationship between the entities. Performance measurement and reporting is a key component of this agreement, and the entities will have a strong focus on ensuring payment correctness, payment accuracy, payment timeliness and the successful administration of social security programs. The entities commit to the establishment and adherence to the Assurance, Delegations and Performance Management Protocol to ensure that the right programs and services are delivered to the right person at the right time.

The entities acknowledge and recognise the importance of regularly monitoring the reporting obligations, including as part of the governance discussions, to ensure assurance activities remain appropriate and fit-for-purpose.⁴⁴

2.37 The development of the Assurance, Delegations and Performance Management Protocol presents an opportunity for the department to address identified deficiencies in the level of assurance it gains under the bilateral assurance arrangements.

Has the department established effective processes with Services Australia to manage shared risks?

Bilateral management of shared risks

2.42 Of the five bilateral service arrangements related to payment accuracy and timeliness (outlined in Table 2.1), only the 2016 Payment Assurance Services Arrangement includes a reference to risk management. It outlines the following responsibilities for managing shared risks relating to payment accuracy:

• the department is responsible for minimising payment accuracy risks associated with policy issues;
• Services Australia is responsible for minimising accuracy risks associated with payment processes, and developing payment accuracy risk management (PARM) plans; and
• the department and Services Australia are jointly responsible for ongoing review of PARM plans, escalating high- and very high-level risks, and identifying current and emerging risks to inform development of joint compliance strategies.

2.43 The ANAO found no references in these subordinate bilateral agreements to responsibilities for managing shared risks relating to payment timeliness.

Payment accuracy risk management plans

2.44 Services Australia produces annual PARM plans to ‘identify payment accuracy risks’ for specific payments or groups of payments covered by the PARP, ‘detail the current controls and mitigation strategies’, and ‘recommend activities and treatments that will reduce risks’. PARM plans follow a standard format that includes:

• background information on welfare payments, the purpose of PARM plans, the PARP methodology and ‘generic failure’ risk⁴⁵;
• summary analysis of significant inaccuracy risks identified through PARP reviews (largely reproducing information the department provides Services Australia); and
• for each significant inaccuracy risk, a risk assessment section that covers how the risk could happen, consequences, current controls and proposed treatments.

2.45 Table 2.3 outlines the key risks identified in Services Australia’s 2022–23 PARM plans and any proposed treatments noted in the risk assessments. All plans rated the identified ‘significant risks’ as ‘low’ or ‘medium’ after treatment. Where additional treatments were identified, they were often general descriptions of current controls (such as ‘continue monitoring income through current monitoring processes’).

2.46 While PARM plans are prepared for the current financial year, analysis is based on PARP data from the previous year, and most plans are not finalised until halfway through the financial year for which they apply. PARM plans are provided to the department when finalised and circulated to relevant policy teams, but they are not provided to bilateral governance bodies. Risk assessment content within the plans is high-level and lacks detailed consideration of how risks have changed over time or responded to identified treatments. Analysis of PARP data on inaccuracy risks replicates the department’s internal reporting. Consequently, PARM plans do not provide useful information to support effective bilateral management of shared risks.

Bilateral management of strategic risks

2.47 The 2016 Review of Accountabilities (discussed at paragraph 2.6 and in Box 1) made recommendations about improving bilateral risk management that were neither accepted nor rejected and not implemented, including that:

• the BMA risk framework identify shared risks and how the department and Services Australia will deal with them; and
• BMA service arrangements require the department and Services Australia to jointly develop risk management plans for individual payments.

2.48 The department’s 2019 internal audit of welfare payment processing (discussed at paragraphs 2.8 and 2.19 and in Box 1) also found BMA risk management practices could be strengthened. Weaknesses were identified with consideration of shared risks and escalation and resolution of issues at bilateral governance bodies. The audit included a recommendation to conduct a risk assessment to support BMA implementation, which was accepted and remained open as of May 2023.

2.49 The department and Services Australia both have internal risk policies that outline processes for managing shared risks. These policies have largely not been followed in relation to managing welfare payment accuracy and timeliness risks. The department and Services Australia have not developed joint risk management plans or shared risk registers. Engagement at bilateral governance bodies has focussed on managing emerging issues, rather than proactive and strategic discussion of shared risks.

2.50 The April 2023 BMA Head Agreement includes a section on ‘risk management’ that states: ‘Specific risk management arrangements, including the requirement for any risk registers, are to be set out in the individual bilateral arrangements’.⁴⁶ In addition, the Head Agreement states that a ‘Joint Risk Management Protocol’ will be established and reviewed within the first six months of the Head Agreement’s operation. As of June 2023, no additional risk management documents had been developed. The absence of robust bilateral risk management processes limits the department’s capacity to oversee the delivery of its delegated welfare payment functions and to manage associated risks.

3.1 Commonwealth entities must develop performance measures for their corporate plans that use reliable and verifiable sources of information and methodologies, and provide an unbiased basis for the measurement and assessment of their performance.⁴⁷ This chapter examines whether the department and Services Australia⁴⁸ have established a robust methodology and effective operational processes for monitoring payment accuracy through the Payment Accuracy Review Program (PARP), and whether reporting on payment accuracy and correctness has been reliable, verifiable, and free from bias. Since effective analysis of performance information can help to identify deficiencies in current processes and drive a culture of continuous improvement, this chapter also examines whether the department and Services Australia are effectively using performance information to continuously improve payment accuracy.

Has a robust methodology been established for monitoring payment accuracy?

Estimating the accuracy of welfare payments

3.2 The principal objective of the PARP is ‘to provide an estimate of the accuracy of outlays for surveyed payments, and to estimate the impact of administrative errors’.⁴⁹ The department has documented the methodology for estimating the accuracy of welfare payments through the PARP in various internal guides, protocols, process diagrams and other documents. These documents provide limited coverage of the methodology for estimating the impact of administrative errors identified during a review.⁵⁰ Key stages of the methodology are outlined in Figure 3.1.

Figure 3.1: Payment accuracy methodology stages

Note a: DSS stratifies the population of current welfare recipients (that is, divides it into subgroups called ‘strata’) based on payment type, receipt of Rent Assistance, and distance from a Services Australia service centre, and then selects random samples of recipients from these strata based on pre-determined selection quotas.

Note b: Currently surveyed payments are: ABSTUDY, Age Pension, Austudy, Carer Allowance, Carer Payment, Disability Support Pension (DSP), Family Tax Benefit, JobSeeker Payment, Parenting Payment Partnered, Parenting Payment Single, Special Benefit, Youth Allowance (Other), Youth Allowance (Student). Descriptions of these payments can be found at Appendix 3.

Note c: Representativeness and consistency checks include checking to ensure: the sample is of the agreed size; there are no duplicate selections; the sample is broadly representative of the population; and certain categories of recipients have been excluded (for example, deceased, recently bereaved, recently reviewed, subject to current identity fraud investigation, etc.).

Note d: The standard is 98% of all reviews completed and 95% of DSP reviews completed, with no data integrity issues older than 14 days identified.

Note e: When using a stratified random sample to generate a statistical estimate for a population, strata results should be weighted based on the relative sizes of the strata and the samples selected from them.

Source: ANAO analysis of DSS documentation.

3.3 As outlined in Figure 3.1, Services Australia is responsible for conducting PARP reviews and entering results into its electronic databases (the IRS and RRRS). Figure 3.2 provides a summary of the process for PARP reviews and outcomes for welfare recipients who undergo PARP reviews.

Figure 3.2: PARP review process

Note a: If the recipient complies within 14 days, the payment is restored and the PARP review process continues.

Source: DSS and Services Australia documentation.

3.4 The PARP methodology includes a set of 80 questions that Services Australia PARP reviewers answer based on their completed reviews. The PARP question set gathers detailed information on the reasons for payment inaccuracies and any recipient and/or administrative errors that may have contributed to them (see Box 2).

Australian National University methodology review (2021)

3.5 In response to a request from its audit and risk committee, the department engaged the Australian National University (ANU) in June 2021 to conduct a review of the PARP methodology. The scope of the review was to assess the sample selection methodology, data collection methodology (including mode, timing and frequency of collection and data validation and quality assurance processes), and methodology for the statistical estimation and calculation of payment accuracy. The scope did not include systematic testing of the extent to which Services Australia implemented the methodology.

3.6 ANU provided its final report on the PARP methodology review in August 2021. The overarching finding of the review was:

the current [PARP] methodology is generally sound and produces reliable estimates of payment accuracy. There are, however, several aspects of the methodology that can be improved to increase the robustness of the payment accuracy and other related measures, or which would allow for the estimate of these measures at a lower cost.⁵¹

3.7 The review made fifteen recommendations (see: Appendix 7).

• three recommendations related to selecting recipients for review, exclusions for major disasters, and associated weighting methods (ANU recommendations 1, 2 and 6);
• four related to PARP review caseload management, mode of review (face-to-face, video-chat or telephone), and quality control (ANU recommendations 3, 4, 5 and 15);
• two related to extending program coverage to include supplementary payments and reviewing all payments received by recipients (ANU recommendations 7 and 8); and
• six related to the treatment of payment inaccuracy causes in calculations and reporting of results (ANU recommendations 9, 10, 11, 12, 13 and 14).

3.8 The department and Services Australia initially agreed to implement all fifteen recommendations in two phases, with full implementation from July 2023. In March 2023 the Bilateral Management Committee (BMC) agreed not to proceed with implementing ANU recommendation 13, relating to the treatment of DSP inaccuracies. Case study 1 outlines the background to the ANU recommendation and BMC decision. The changes in the treatment of DSP inaccuracies had implications for assessment of control effectiveness (discussed at paragraph 3.32) and payment accuracy reporting (discussed at paragraphs 3.71 to 3.74).

Note a: When assessing eligibility for the DSP, ‘permanent’ is defined as a medical condition that has been ‘fully diagnosed, treated and stabilised, and in light of available evidence, it is unlikely that there will be any significant functional improvement within the next two years’.

Note b: As of March 2023, maximum pension rates were the same for DSP and Age Pension.

Note c: Department of Social Services, Annual Report 2015–16, Commonwealth of Australia, 2016, p. 48.

Note d: M. Gray, K. Reddy, N. Biddle & D. Stanton, Review of Methodology for the Random Sample Survey (RSS) Program, ANU Centre for Social Research and Methods, 24 August 2021, p. 40.

3.9 After the 2021 PARP methodology review was completed, the department commissioned ANU and the Social Research Centre (a subsidiary of ANU) to develop a plan to assist the department and Services Australia with staging and implementing the recommendations. The department has continued to engage ANU and the Social Research Centre on technical aspects of implementation, including:

• developing an ‘optimal allocation’ sampling methodology for the 2022–23 review program (to support ANU recommendation 1)⁵²; and
• conducting a statistical analysis of mode of review, which found there were no significant differences in levels of payment inaccuracy detected by face-to-face reviews and telephone reviews (to support ANU recommendation 4).

3.10 As of March 2023, based on the department’s reporting to BMC, progress in implementing the fourteen agreed recommendations was as follows:

• ANU recommendations 1, 2, 4, 6, 9 and 14 had been ‘implemented’ (although most had tasks still to be completed before they would be closed);
• ANU recommendations 5, 12 and 15 were ‘close to completion’; and
• the other five ANU recommendations were marked as ‘in progress’, although status updates noted that ICT changes were required that may delay implementation and no timeframes had been identified for the relevant ICT changes to have occurred.

3.11 In February 2023 Services Australia reported, as part of its enterprise risk watch list, that funding had not been approved to implement the required ICT changes and that it was investigating an external costing request with the department. In July 2023 Services Australia reported to the Payment Integrity, Payment Accuracy, Debt & Compliance and Automation Strategic Business Discussion that it would assess the operational impact of ANU recommendations that were dependent on ICT changes and report back to the department by 31 October 2023.

Formula for calculating payment accuracy

3.12 The 2021 ANU PARP review made two recommendations relating to the department’s formula for calculating payment accuracy, recommending that supplementary payments (such as Rent Assistance) and/or additional welfare payments received by sampled recipients (where they receive more than one type of payment) be included in PARP reviews and accuracy calculations (ANU recommendations 7 and 8). While the department and Services Australia accepted these recommendations, implementation has been delayed due to the need for ICT changes (discussed at paragraph 3.10 and 3.11).

3.13 The ANAO identified an additional issue with the department’s payment accuracy formula relating to the way it treats underpayments (described in detail in Appendix 8), which means the formula slightly underestimates the accuracy of payments.⁵³ No issues were found with the department’s formulas for calculating underpayments and overpayments in their own right. The issue only arises when payment accuracy is reported as a percentage of total payments.

Benchmark targets for payment accuracy

3.15 The department and Services Australia have an overall KPM benchmark target for welfare payment accuracy of 95 per cent.⁵⁴ Targets for individual payments are also set at 95 per cent, except for Age Pension which is set at 97 per cent. The appropriateness of KPM targets was not within scope of the ANU PARP review.

History of payment accuracy KPM targets

3.16 Services Australia began reporting against KPMs for accuracy of payments it made on behalf of relevant departments in 1998–99. This was against a target of 95 per cent for all payment types (no evidence was found as to why 95 per cent was chosen as the target). With the establishment of the Bilateral Management Arrangement in 2014, the department agreed revised bilateral KPMs with Services Australia. KPM targets for payment accuracy (to take effect from trimester one 2014–15) were set at 95 per cent for most payments. The targets for Age Pension and DSP were set at 97 per cent. In the trimesters prior to the revised targets commencing, Age Pension and DSP had consistently recorded payment accuracy results higher than 97 per cent and had been the highest performing of assessed payments.

3.17 The rationale for increasing the DSP target related to the exclusion of inaccuracies for pre-2012 recipients who were found by the PARP to not qualify against the revised 2012 impairment tables. The DSP target was revised down to 95 per cent from trimester three 2014–15, in conjunction with the PARP methodology change that reincluded inaccuracies related to the pre-2012 recipients (discussed in Case study 1).

3.18 In early 2021 the department and Services Australia considered reviewing payment accuracy KPMs as part of a broader bilateral KPM review project. Services Australia proposed including a separate KPM covering payment correctness, based on the method it uses for its corporate plan performance measure. The department disagreed with this proposal on the grounds that it considered the payment correctness KPM to be ‘biased and incomplete’ (as it did not include all sources of payment inaccuracy), and the planned review of payment accuracy KPMs did not progress further.

Tolerance for payment inaccuracy risk

3.19 Table 3.1 shows the ANAO’s recalculations of welfare overpayments and underpayments using PARP data from 2018–19 to 2021–22. Over these four years, overpayments identified through the PARP represented 5.45 per cent of total welfare payments, which amounts to an estimated $26.7 billion of leakage from Australia’s welfare system (averaging $6.7 billion a year).⁵⁵

Table 3.1: Estimated overpayments and underpayments^a, 2018–19 to 2021–22

Note a: The department reported overpayment and underpayment estimates for the first time in its 2021–22 annual report. The ANAO’s recalculations differ from the department’s estimates for reasons outlined in paragraphs 3.77 to 3.81. Due to rounding, annual figures in this table do not sum to totals.

Source: ANAO analysis of DSS and Services Australia data using business rules provided.

3.20 The ANAO found limited evidence that the department had considered revising payment accuracy KPM targets based on the risk that payment inaccuracy poses to Australian Government welfare payments. When the ANAO raised this matter with the department in February 2023, it agreed that 95 per cent represented a high tolerance for inaccuracy and that there was an argument for adopting different targets for different payments.

Broader objectives of the Payment Accuracy Review Program

3.23 In addition to the principal objective of the PARP noted at paragraph 3.2, other objectives outlined in the 2019 Random Sample Surveys Standing Operational Statement include providing:

• ‘information on causes and sources of errors leading to inaccurate payments’; and
• ‘data for assessment of the effectiveness of payment accuracy controls’.⁵⁶

Information on causes and sources of inaccuracies

3.24 Services Australia’s current compliance model identifies four ‘attitudes to compliance’ that welfare recipients may have, and associated strategies and measures (see Table 3.2).⁵⁷

Table 3.2: Services Australia’s compliance model

Source: Services Australia internal documentation.

3.25 While the PARP question set (see Box 2 above) gathers detailed information on the types of errors leading to inaccurate payments (see Appendix 9), it does not provide information on the underlying causes of errors. For example, there are no questions that relate to whether customer errors represent fraud, opportunistic non-compliance, or inadvertent non-compliance (that is, an error or mistake in the common sense of the word).⁵⁸

3.26 The ANAO’s December 2001 performance audit on Services Australia’s management of fraud and incorrect payments found:

deriving an estimate on the level of fraud and error by income support payment type, could assist both [the department] and [Services Australia] to develop more meaningful indicators to demonstrate that they have been jointly successful in reducing consequent losses. Changes over time in this estimate may be useful in showing that [Services Australia] compliance efforts (encompassing debt prevention, customer education and review activity) are influencing customer behaviour.⁵⁹

3.27 The report noted that the United Kingdom government entity responsible for welfare payments, the Department of Work and Pensions, used random sampling to determine the incidence and magnitude of fraud and error in its welfare programs. The ANAO recommended implementing a mechanism to estimate losses by payment type that distinguished ‘between losses from [Services Australia] error and those resulting from customer error and fraud’.⁶⁰ While both entities agreed to the recommendation, reporting of PARP results by the department and Services Australia has never differentiated between the dollar impact of administrative errors, recipient errors and fraud.

3.28 The ANAO’s 2006 performance audit of the PARP found it provided limited insights on the underlying causes of recipient errors, and PARP reviewers did not seek to uncover why recipients failed to comply with reporting requirements. The report noted the department had obtained legal advice in 2005 (in response to the audit) that suggested PARP reviewers may not be able to inquire into reasons for non-compliance due to the possibility of self-incrimination.⁶¹

3.29 While there may be barriers to inquiring into recipients’ reasons for non-compliance, PARP reviewers could make qualitative judgements about likely causes of non-compliance based on clear and repeatable assessment criteria.

3.30 Gathering additional qualitative information through the PARP on the underlying causes of recipient errors would enable the department and Services Australia to develop better targeted measures to improve payment accuracy.

Data on control effectiveness

3.31 The PARP does not provide a direct assessment of the effectiveness of Services Australia’s controls for welfare payments. The ANAO’s 2006 performance audit of the PARP found:

The [PARP] does not provide a direct measure of the effectiveness of the compliance/control framework because it does not directly test this. Rather, there is an inference that positive results in the [PARP] equate with the compliance/control framework being effective.

The [PARP] questionnaire does not include any questions regarding the outcomes of any compliance/control activity a customer may have been subjected to prior to selection in the [PARP]. There is also no follow-up during the [PARP] process to match back to any compliance activity the customer may have been involved in to see if it was effective.⁶²

These issues remain in 2023. There are no questions in the PARP question set that directly relate to assessing the effectiveness Services Australia’s compliance program or other controls. For example, Services Australia does not record whether recipients have been presented with pre-filled Single Touch Payroll (STP) employment income for confirmation as part of their reporting activities, or whether they have been subject to other data matching compliance activities.⁶³

3.32 The decisions to exclude inaccuracies relating to DSP medical eligibility from PARP reporting in 2019 and from PARP reviews altogether in 2022 (discussed in Case study 1 above) illustrate how ‘providing data for the assessment of payment accuracy control effectiveness’ (see paragraph 3.23) has not been a focus for the PARP. The absence of data on DSP recipients who are no longer eligible for the payment, by virtue of the 2012 changes to the impairment tables or otherwise, limits Services Australia’s ability to implement or assess the effectiveness of other risk-based compliance activities designed to mitigate payment accuracy risk. The papers recommending these decisions correctly argued that finding a DSP recipient is no longer medically eligible through a PARP review does not necessarily mean a recipient error or administrative error has occurred. However, the papers did not acknowledge that it may mean there are weaknesses in Services Australia’s compliance activities for DSP, as the ineligibility was not otherwise identified.

Have effective operational processes been established for payment accuracy reviews?

Operational guidance

3.40 Services Australia has an ‘Operational Blueprint’ intranet site that is accessible to staff and contains operational guidance to support delivery of services. Guidance for PARP reviews is contained within the ‘Payment Accuracy Review’ section of Operational Blueprint, broken down into five sub-topics:

• systems access — arranging access to the IT systems for PARP reviews;
• pre-interview tasks — tasks undertaken prior to commencement of a new PARP trimester;
• PARP interviews — conducting PARP interviews and intensive desktop reviews⁶⁴;
• finalising reviews — post-interview verification and review finalisation tasks; and
• quality processes — quality assurance processes for PARP reviews.

3.41 The ANAO examined PARP Operational Blueprint procedures to assess whether they provided effective operational guidance to support Services Australia staff in conducting PARP reviews. The results of this assessment are in Table 3.3.

Table 3.3: ANAO assessment of PARP Operational Blueprint procedures, as of June 2023

Note a: Procedures were assessed as complete if they covered relevant Bilateral Management Arrangement requirements and had no obvious gaps or weaknesses.

Source: ANAO analysis of Services Australia documentation.

3.42 Services Australia’s Operational Blueprint content for PARP reviews provided clear, logically structured, and largely comprehensive guidance for PARP reviewers. Procedures were underpinned by references to legislative requirements, and content appropriately reflected Services Australia’s PARP responsibilities under bilateral agreements. Services Australia updated its Operational Blueprint procedures on verifying income in November 2022 to include links to guidance on reporting suspected fraud.

3.43 The ANAO tested key PARP controls for its audit of Services Australia’s 2021–22 financial statements and found controls were designed and implemented effectively. This testing found no errors in payments due to legislative errors and total errors were low, indicating that the primary administrative risks to payment correctness related to human errors from staff processing rather than misrepresentation of legislative or procedural requirements.

3.44 The ANAO also conducted targeted testing on 81 randomly selected PARP reviews from trimesters one and two of 2022–23: 27 reviews that resulted in recipients’ payments being cancelled (cancellations); 27 that resulted in downward variations (overpayments); and 27 that resulted in upwards variations (underpayments). This testing found PARP reviewers largely adhered to the requirements in the Operational Blueprint procedures (excluding requirements relating to quality assurance, which are discussed below) with no indications of systemic errors or issues. Of the 81 cases tested, 75 met all tested requirements, with exceptions being the failure to properly record errors or upload review documentation.

Quality assurance processes

3.45 A range of quality assurance processes are in place for PARP reviews, which aim to ensure that information gathered through PARP reviews has been properly assessed and decisions on review outcomes are made in accordance with procedural requirements. Quality assurance processes for the PARP are outlined in Table 3.4.

Table 3.4: PARP quality assurance processes

Source: ANAO analysis of DSS and Services Australia documentation.

Internal quality checking processes

3.46 In July 2022 Services Australia completed a ‘quality stocktake’ based on results from its ‘full quality check’ and ‘quality assurance check’ processes (described in Table 3.4 above) from May to July 2022 and interviews with relevant operational staff. Key findings of the stocktake included that:

• there was a ‘high and static’ rate of discrepancies identified in PARP reviews subject to random checking processes⁶⁵; and
• formal coaching by team leaders was largely not occurring, informal coaching was not being documented, and there was scope for more consistent and structured training.

3.47 The quality stocktake report included 19 recommendations to address identified issues. Services Australia has tracked the implementation of these recommendations through a dashboard. As of May 2023 seven recommendations were recorded in the dashboard as being ‘completed’, one was ‘on hold’, and the remaining 11 were ‘in progress’ (see Appendix 10).

3.48 The ANAO’s targeted testing of 81 PARP reviews resulting in cancellations and variations (discussed in paragraph 3.44) found that Services Australia’s documentation of required ‘pre quality checks’ and ‘partial quality checks’ was inconsistent and incomplete.

• Of 39 PARP reviews that required a pre quality check because a debt was raised, checks were documented for 11 reviews.
• Of 54 PARP reviews that required a partial quality check due to an upward or downward variation, partial quality checks were documented for 28 reviews, and for an additional 10 reviews some form of check was recorded (with insufficient detail to determine if it was a partial quality check).

External validation

3.49 As outlined in Table 3.4, the department conducts external validation of a randomly selected sample of 2.5 per cent of completed PARP reviews (around 500 reviews per year). The department’s external validation officers check that PARP documentation is complete and consistent, payment eligibility criteria have been appropriately assessed, and any changes in circumstances have been correctly recorded in Services Australia’s welfare payment processing systems. After completing its external validation checks for a trimester, the department provides feedback to Services Australia outlining positive observations and details of any inconsistences or issues found.

3.50 The 2021 ANU review of the PARP methodology noted that the external validation checks currently undertaken by the department:

[…] are all reasonable checks, but because they are entirely desk based they are limited in their ability to identify problems with how the RSS review interviews are undertaken. This is of some concern because the quality of the RSS interviews is the single most important determinant of the quality of the data that results from the RSS review and the accuracy of the estimated payment accuracy.⁶⁶

3.51 ANU recommended that the department begin directly monitoring a proportion of PARP interviews for independent quality control purposes, noting it was standard practice in market research to monitor at least five per cent of interviews (ANU recommendation 15). The department agreed to this recommendation and reporting to BMC in March 2023 noted (in different places) that it was ‘close to completion’, ‘completed with continuing refinements’ and ‘ongoing’. In May 2023 the department developed a draft plan to implement more comprehensive external quality control monitoring for the PARP with a target implementation date of 1 July 2023.⁶⁷

Data validation

3.52 One month after the start of a trimester, the department begins undertaking weekly automated data integrity checks to ensure data gathered through the PARP question set is: in the correct format; consistent with the question logic; and consistent with other administrative data. The department provides the results of its data validation testing to Services Australia, and identified data integrity issues are resolved until the PARP review completion standard is met.⁶⁸

3.53 As discussed in Box 2, the PARP question set gathers detailed information on the reasons for payment inaccuracies and any recipient and/or administrative errors that may have contributed to them. The data validation processes conducted by the department and Services Australia focus on data integrity issues with questions one to four of the question set, which relate to the primary reasons for recipients’ payment being varied or cancelled. This data is used for payment accuracy calculations and for reporting on payment inaccuracy risks at a general level (using broad categories such as ‘assets’, ‘change in relationship status’, ‘earned income’, etc.).

3.54 More granular data on recipient and administrative errors is gathered through questions 14 to 80 of the PARP question set. The ANAO’s testing found issues with the completeness and accuracy of this data. The primary issue was that data on the dollar impacts of recipient and administrative errors (recorded in questions 14 to 80) did not reconcile with finalised PARP review data, even after adjusting for known reasons for discrepancies.⁶⁹ These data quality issues mean it is currently not possible to robustly estimate the relative impact of administrative and recipient errors on payment accuracy, which limits the capacity of the department and Services Australia to undertake robust analysis of the underlying causes of payment inaccuracy.

Has reporting of payment accuracy results been reliable, free from bias, and verifiable?

3.61 The department and Services Australia use PARP data to report on payment accuracy KPMs to bilateral governance bodies through the quarterly performance assurance report.⁷⁰ Services Australia includes annual KPM results in its annual assurance statement, which is provided to the Secretary of the department after the end of the financial year. The department also reports the annual KPM results in its annual performance statements. Reported payment accuracy results for 2018–19 to 2021–22 are reproduced at Appendix 4.

3.62 The ANAO assessed whether reporting of payment accuracy results was reliable, free from bias, and verifiable over the period of 2018–19 to 2021–22. Table 3.5 outlines the assessment criteria applied and the ANAO’s overall assessments against these criteria.

Table 3.5: ANAO assessment of payment accuracy reporting

Reliability

Coverage of welfare payments

3.63 Reporting of payment accuracy results from 2018–19 to 2021–22 used a largely robust methodology and appropriate data sources. However, an issue affecting the reliability of accuracy reporting is that the PARP does not cover all of the department’s welfare payments. The PARP covers 15 of the 32 welfare payments listed in Appendix 3.⁷¹ While payments included in the PARP represented 96.6 per cent of the department’s welfare payments in 2021–22, some included payments (such as Special Benefit and ABSTUDY – Tertiary) represent smaller proportions of total payment expenditure than several payments that are excluded.

3.64 The department’s reporting on payment accuracy in its 2021–22 Annual Performance Statements stated:

The [PARP] provides assurance with respect to the accuracy of social security payments. Reviews are conducted by Services Australia using a random sample of the population for the major payment types provided by the department.⁷²

3.65 The department did not include a note identifying which payments were not covered by the PARP or acknowledging that Parental Leave Pay (a payment with expenditure of $2.5 billion in 2021–22, which is means tested and would thus have potential for payment inaccuracy) was not included.

3.66 While some supplementary payments will be picked up through implementing recommendation 7 of the ANU PARP review, there is no documented rationale for why a major payment such as Parental Leave Pay should not be included in the PARP.

Data limitations affecting payment accuracy reporting

3.69 In response to ANU recommendation 9, the department included an addendum to its 2021–22 Annual Performance Statement (APS) which contained the number of PARP reviews conducted in 2021–22 and the financial estimates of overpayment and underpayment risks for each payment type within the PARP. The additional information published by the department improved the reliability of payment accuracy reporting. However, the reporting does not appropriately acknowledge data limitations.⁷³ In addition to not covering all welfare payments, there are various other PARP data limitations with implications for reported results that have not been acknowledged by the department or Services Australia in payment accuracy reporting.⁷⁴

• The ANAO’s 2006 audit of the PARP noted the issue of ‘non-sampling error associated with customer non-disclosure’⁷⁵ (that is, PARP reviews cannot identify all non-compliance or fraud because recipients may conceal factors such cash income or relationship status, which are difficult to detect through other means). ANU noted in its 2021 PARP review that this is a common theme in academic literature and remains an issue for the PARP.
• As discussed in Case study 1, methodology changes implemented in 2019, which excluded certain DSP inaccuracies from calculations, mean DSP results reported since 2019–20 have underestimated overpayments and not been comparable to previous years.
• Family Tax Benefit (FTB) payments are determined based on recipients’ annual income rather than fortnightly income and are subject to a reconciliation process using tax returns lodged with the Australian Taxation Office. PARP reviews for FTB assess eligibility on other grounds but do not assess whether recipients are receiving overpayments or underpayments due to employment income, as it is not possible at the time of review to make this assessment. Reported FTB payment accuracy results thus underestimate both overpayments and underpayments.⁷⁶
• ANU’s 2021 PARP review found accuracy calculations only cover the basic payment for which recipients were selected for PARP reviews and exclude other payments they may be receiving (such as Rent Assistance or other payments in addition to the payment they were selected for). PARP reviews also do not assess whether recipients are eligible for other payments. The implications of these issues are that reported payment accuracy results do not account for the accuracy of all payments received by recipients.

Freedom from bias

Bias in reporting DSP accuracy results

3.71 While reporting for payments other than DSP was unbiased, there was bias in the reporting of DSP results which also impacted on the overall payment accuracy result. As discussed in Case study 1 above, PARP methodology changes were implemented in 2019 that excluded two categories of DSP inaccuracies from calculations (‘change in medical eligibility’ and ‘qualified for another payment’). In 2022 the medical review requirement was removed from the PARP altogether, which has meant that DSP recipients have not been assessed for medical eligibility through the PARP since trimester two 2022–23. These changes have introduced bias to the department’s and Services Australia’s payment accuracy reporting.

3.72 Table 3.6 shows that payment accuracy results for DSP gained a substantial performance increase from the 2019 exclusions. DSP results were between 3.32 and 4.85 percentage points higher from 2019–20 to 2021–22 than they would have otherwise been if no exclusions had been in place. The impact of the DSP result on overall payment accuracy results was an improvement of between 0.47 and 0.65 percentage points over the same period. The 2022 decision to remove the medical review requirement from the PARP altogether means the impact of this bias on the department’s and Services Australia’s payment accuracy reporting will no longer be quantifiable.

Table 3.6: Impact of 2019 DSP inaccuracy exclusions, 2018–19 to 2021–22

Source: ANAO analysis of DSS and Services Australia data using business rules provided.

3.73 The 2019 decision to exclude certain DSP inaccuracies from calculations was taken at a time when overall payment accuracy results fell below the 95 per cent benchmark target. At a BMC meeting in August 2019, when asked about strategies to improve payment accuracy, a Services Australia member noted ‘the methodology change for DSP is expected to see the payment accuracy result return to above the benchmark’. Services Australia’s 2018–19 Annual Assurance Statement, provided to the Secretary of the department in August 2019, cited working with the department to review the DSP methodology as an example of ‘collaborative work to improve payment accuracy’, noting:

If these arrangements had been in place during 2018–19, it is estimated the overall payment accuracy result would have been 95.53 per cent [rather than the reported 94.69 per cent].⁷⁷

3.74 In a July 2019 brief to the Minister for Government Services, Services Australia described the inclusion of ‘change in medical eligibility’ and ‘qualified for another payment’ inaccuracies for DSP as ‘two anomalies in the methodology used by the department to calculate payment accuracy’. However, it is the exclusion of these two inaccuracy reasons for DSP that has created an anomaly, as it has meant the treatment of DSP inaccuracies has been inconsistent with other payments. The two inaccuracy reasons are still included in the PARP question set and have been used for other payments.

Verifiability

3.77 To test the verifiability of reported payment accuracy results, the ANAO:

• obtained the data that the department used to calculate payment accuracy KPM results for 2018–19 to 2021–22; and
• reperformed calculations using documented methodologies, including applying the DSP exclusions discussed in Case study 1.

3.78 The ANAO’s initial verification testing found 50.8 per cent of reported payment accuracy results were verifiable over the assessed period (for detailed verification results see Table A.7 in Appendix 11). After the ANAO presented these results to the department, the department acknowledged that it had made an error in applying weightings when calculating 2019–20 and 2020–21 results, and applied an ad hoc methodological variation to weighting calculations for trimesters two and three of 2021–22. The department had not maintained a register documenting these deviations from its methodology.

3.79 In line with the department’s response, the ANAO conducted the verification testing a second time, recalculating weightings for all four years to address the identified issues. Table 3.7 shows the ANAO could verify 33.5 per cent of reported payment accuracy results over the assessed period using corrected weightings (for detailed verification results see Table A.8 in Appendix 11).

Table 3.7: ANAO revised verification of reported accuracy results, 2018–19 to 2021–22, updated to address weighting issues

Source: ANAO analysis of DSS and Services Australia data using business rules provided.

3.80 Through this second round of analysis, the ANAO identified that the department had made additional calculation errors over the period, including:

• miscalculating the number of reviews occurring within clusters resulting in incorrect weightings for 2018–19;
• incorrectly converting trimester weightings to annual weightings for all payments in 2019–20 and four payments in 2020–21;
• incorrectly dividing the derived strata population size by the sample size when calculating weightings for trimester three 2020–21;
• included partners within weighting calculations for trimester two 2021–22;
• not assigning weights of zero to reviews with active data integrity issues in 2018–19⁷⁸;
• assigning weights of zero to reviews in trimester three 2020–21 that did not have data integrity issues identified; and
• using multiple differing population totals to determine weightings for reviews within the same strata in trimester three 2021–22, where only one should be used, resulting in differing weightings being applied within strata.

3.81 The ANAO’s verification testing highlighted that the department did not use a consistent method for calculating payment accuracy results from 2018–19 to 2021–22 and made frequent calculation errors with strata weightings.

Has reporting of payment correctness results been reliable, free from bias, and verifiable?

3.84 Services Australia defines welfare payment correctness as the percentage of PARP reviews that do not identify any administrative errors with a dollar impact. Such errors are identified through questions 14 to 80 of the PARP question set (see Box 2 above). Until 2020–21 Services Australia used data on administrative errors identified through the PARP to report against a payment correctness performance measure in its APS — ‘Achievement of payment quality standards: Centrelink: delivery of correct customer payments’. In 2021–22 Services Australia reported this result as part of a weighted composite ‘payment quality’ strategic performance measure, which combined the welfare payment measure with a Medicare payment correctness measure into a single entity-wide measure.

3.85 The ANAO assessed whether Services Australia’s reporting of payment correctness results was reliable, free from bias, and verifiable over the period of 2018–19 to 2021–22. Table 3.8 outlines the assessment criteria applied and the ANAO’s overall assessments against these criteria.

Table 3.8: ANAO assessment of payment correctness reporting

Reliability

Coverage of welfare payments

3.86 As noted at paragraph 3.63, the PARP covers 15 of the 32 welfare payments listed in Appendix 3. This also affects the reliability of Services Australia’s payment correctness performance measure as it uses the PARP as its data source. In August 2022 Services Australia informed its Executive Committee that it was testing an alternative methodology for the payment quality strategic performance measure. Rather than using the PARP as the data source for correctness of welfare payments, quality assurance data from its Quality On-Line and Quality Management Application systems would be used as these processes cover a broader set of welfare payments. As of June 2023 no internal reporting had occurred using the alternative methodology.

Adjusting for stratification of the PARP sample

3.87 As explained in Figure 3.1 above, the PARP methodology involves reviewing a stratified random sample of welfare recipients. When using a stratified random sample to generate a statistical estimate for a population, strata results should be adjusted based on the relative sizes of the strata and the samples selected from them.

3.88 Services Australia does not apply any adjustment factors in calculating its payment correctness result. The Australian Bureau of Statistics (ABS) noted this in a 2014 review of the PARP methodology, stating:

• Selection weights are not used — The current method does not take into account the stratification and clustering of the sample design and assumes everyone contributes equally.
• Unduly influenced by sample design — The estimation method is influenced by the proportion of reviews for each payment type. This makes it susceptible to any future changes in sample size or selection methodology. It is preferable to instead take into account the proportion of each payment type present in the population.⁷⁹

3.89 Services Australia did not address these issues and has continued to calculate payment correctness results without adjusting for stratification of the PARP sample.⁸⁰

Measurement of ‘payment correctness’

3.90 The ANAO’s May 2006 performance audit of the PARP noted that Services Australia’s ‘definition of payment correctness is different from the ordinary meaning of the words’.⁸¹ The ordinary meaning of payment correctness would be the percentage of recipients who receive a correct payment. The 2021 ANU review of the PARP methodology also noted this issue and recommended that: ‘The term “payment correctness” be replaced with the term “administrative correctness”’ (ANU recommendation 14).⁸² Services Australia agreed to this recommendation, and reporting to BMC in March 2023 noted it had been ‘implemented’ and ‘corporate documents (Portfolio Budget Statement, Corporate Plan and Annual Performance Statements) were aligned’ in November 2022. None of these documents had been published with the revised terminology as of June 2023, but Services Australia updated internal guidance on its Operational Blueprint intranet site with the revised terminology in April 2023.

Freedom from bias

Exclusion of recipient errors

3.91 Services Australia stated in its 2021–22 Annual Performance Statements that the payment quality performance result (98.9 per cent for welfare payment correctness and 98.5 per cent for its composite entity-wide measure) ‘demonstrates the agency’s ability to pay the right person the right payment at the right time’.⁸³ However, ANAO analysis of PARP results shows 81.4 per cent of welfare recipients received correct payments in 2021–22 (see Table 3.9), with most incorrect payments resulting from recipients not declaring or incorrectly declaring changes in their circumstances. This represents a more meaningful and readily understood way of reporting payment correctness. The difference between the ANAO’s correctness estimate of 81.4 per cent and Services Australia’s reported correctness result of 98.9 per cent is that the ANAO’s estimate accounts for both administrative and recipient errors (which includes opportunistic and inadvertent non-compliance).

Table 3.9: Proportion of welfare recipients receiving correct and incorrect payments^a, 2018–19 to 2021–22

Note a: Results in this table are weighted to account for stratification of the PARP sample.

Note: Rounding means totals may not add to 100%.

Source: ANAO analysis of DSS and Services Australia data.

3.92 As noted at paragraph 3.18, in early 2021 the department disagreed with Services Australia’s proposal to create a bilateral KPM for payment correctness on the grounds it was ‘biased and incomplete’. The department’s assessment of the measure was that it drew focus away from the majority of payment accuracy risks (that is, recipient non-compliance) and did not recognise Services Australia’s role in encouraging compliance and identifying and resolving non-compliance. The ANAO agrees with this assessment. Including recipient errors would improve the completeness of Services Australia’s performance measure. It would also provide a more meaningful assessment of Services Australia’s performance, as it would include outcomes that are within its control (administrative correctness) and within its influence (recipient fraud, non-compliance and error).

Interaction between payment accuracy and correctness

3.97 The ABS’s 2014 PARP methodology review (discussed at paragraph 3.88) found increasing use of online forms to manage welfare payments would lead to automatic improvements in payment correctness, as the number of recipients interacting with Services Australia staff would decrease. It recommended that Services Australia reassess the scope of its payment correctness measure to either include recipient errors in the numerator or exclude self-managed cases from the denominator. Services Australia made no changes to the performance measure in response.

3.98 Services Australia identified in its 2020–21 Annual Performance Statements that payment correctness was expected to improve as it ‘continues to expand on self-service options and automated actions, which reduce the opportunity for staff error’.⁸⁴ In 2021–22 Services Australia reported that 69.7 per cent of welfare payment management tasks were self-managed digitally by recipients.⁸⁵ Services Australia has not considered that its focus on improving administrative correctness may be having a negative impact on payment accuracy. Figure 3.3 shows there has been a strong negative correlation between Services Australia’s payment correctness and accuracy results since 2013–14.⁸⁶

Figure 3.3: Relationship between overall payment inaccuracy and dollar value administrative error rates, 2013–14 to 2021–22

Source: ANAO analysis of DSS, DHS and Services Australia annual reports from 2013–14 to 2021–22.

Exclusion of administrative errors with no dollar impact

3.99 Services Australia’s payment correctness measure also downplays the significance of errors that do not have dollar impacts, as they are excluded from the methodology. The ANAO’s 2006 audit of the PARP noted that a 2004 report by Allen Consulting Group found errors with no dollar impact may contribute to future payment accuracy by reducing the effectiveness of controls. Table 3.10 shows a comparison between 2004–05 PARP error rates (presented in the ANAO’s 2006 audit report) and 2021–22 rates. While the PARP identifies a small proportion of administrative errors with a dollar impact, it has been identifying a substantial and increasing proportion of errors with no dollar impact. In addition, the ANAO found the methodology for Services Australia’s payment correctness measure does not include administrative errors with dollar impacts that occurred outside of the trimester in which the PARP review took place unless a debt was raised.

Table 3.10: Comparison of PARP error rates, 2004–05 and 2021–22

Note a: Errors with no dollar impact include recipients not reporting a new bank account or change in relationship status, or Services Australia incorrectly processing reported changes. These errors may affect future payment accuracy — for example, incorrect bank account details may limit the effectiveness of data matching activities.

Source: Auditor-General Report No. 43 of 2005–06 Assuring Centrelink Payments – The Role of the Random Sample Survey Programme, Table 3.1, p. 55, and ANAO analysis of DSS and Services Australia data.

3.100 If Services Australia continues to use PARP as the data source for an internal administrative correctness performance measure (to supplement a broader external performance measure that includes recipient errors), it should include all administrative errors in its calculations, including errors with a dollar impact in an earlier trimester and errors with no dollar impact. In 2021–22 using this methodology would have meant Services Australia had an administrative correctness result of 79.5 per cent (the percentage of reviews with Services Australia errors was 20.5 per cent in 2021–22, which is calculated by adding the ‘no dollar impact’ and ‘dollar impact’ percentages from Table 3.10).

Verifiability

3.101 To test the verifiability of reported payment correctness results, the ANAO:

• obtained the data that Services Australia used to calculate payment correctness results for 2018–19 to 2021–22; and
• reperformed calculations using the documented methodology.

3.102 As shown in Table 3.11, Services Australia’s payment correctness reporting was largely verifiable, with reported results only deviating from recalculated results in one year by 0.1 percentage points. The reason for this deviation was a minor error in Services Australia’s calculations, which meant a small number of dollar impact administrative errors were inadvertently excluded.⁸⁷

Table 3.11: ANAO verification of payment correctness results^a, 2018–19 to 2021–22

Key: ◆ ANAO recalculation matched reported result ■ ANAO recalculation did not match reported result

Note a: Results in this table are unweighted.

Source: ANAO analysis of DSS and Services Australia data using business rules provided.

3.103 A July 2019 internal audit (conducted by EY Australia) identified that administrative errors falling outside the top four errors (recorded in response to PARP questions 14 to 78) were not being captured and included in payment correctness calculations. The audit included a recommendation that Services Australia develop, document and implement a process to address this. Services Australia closure documentation for this recommendation indicates the PARP question set was updated from trimester two 2019–20 to capture this data. The ANAO’s verification testing shows Services Australia failed to subsequently include the data in its payment correctness calculations up to and including 2021–22. Due to the small number of errors involved, as shown in Table 3.11, this omission had a minimal impact on reported payment correctness results.

Is performance information used to drive continuous improvement in payment accuracy?

Services Australia’s payment accuracy journey

3.104 The payment accuracy result fell below the bilateral KPM target of 95 per cent for the first time in 2018–19. In its 2018–19 Annual Assurance Statement, Services Australia noted that it had developed an Integrity Strategy to ‘improve payment accuracy through evidence-based approaches to policy’.⁸⁸

3.105 The Integrity Strategy, which was endorsed by Services Australia’s Executive Committee in September 2019, identified that the entity had: a high reliance on detective measures, limited information flows, an absence of shared ownership of payment integrity, misaligned resources, obscure compliance requirements, and inconsistent communication. To address these issues, it outlined a work program covering:

• a series of ‘deep dives’ into the Age Pension, DSP, Parenting Payment Single, Carer Payment and Newstart Allowance⁸⁹;
• addressing deficiencies in information systems to support better data use; and
• identifying and disseminating better practice approaches.

3.106 Services Australia completed a deep dive into the Age Pension in November 2019, producing a detailed report identifying 35 opportunities for enhancing payment accuracy, including 11 ‘high yield opportunities for priority implementation’.⁹⁰ It commenced deep dives into the DSP and Parenting Payment Single, but this work was ‘indefinitely paused’ in early April 2020 due to the COVID-19 pandemic response.⁹¹

3.107 In 2020–21 the overall payment accuracy result again fell below the 95 per cent target, prompting the Secretary of the department to write to the Services Australia Chief Executive Officer (CEO) in April 2021 requesting detailed analysis of factors driving inaccuracy and a remediation plan (see paragraph 2.22). In April 2021 Services Australia’s Executive Committee approved an Integrity Strategy Refresh, outlining activities for 2021 and 2022 to:

improve payment accuracy by embedding integrity into upfront design [with a] focus on smart systems, enhanced data-sharing, and modernised procedures and communications embedded at key points throughout the customer journey.

3.108 Also in April 2021, Services Australia undertook a deep dive on payment accuracy, generating a ‘high level roadmap’ of proposed activities between April 2021 and December 2022 to improve payment accuracy. Based on the deep dive findings, Services Australia developed a Payment Accuracy Action Plan, which was endorsed by its Enterprise Business and Risk Committee in July 2021. No evidence was found that Services Australia provided the Payment Accuracy Action Plan to the department at this time. In September 2021 the Services Australia CEO established a new Payment Accuracy Division to ‘drive improvements to payment accuracy in response to 2020–21 [PARP] results’.⁹²

3.109 In October 2021 an Integrity Transformation Roadmap was presented to Services Australia’s Executive Committee, outlining proposed actions to achieve the goals of the April 2021 Integrity Strategy Refresh. The Executive Committee agreed to the actions in the roadmap, but requested that it be combined with the Payment Accuracy Action Plan to form a single Payment Accuracy Roadmap. The combined roadmap was finalised in November 2021 and endorsed by the Customer Sub-Group (a sub-committee of the Enterprise Business and Risk Committee). From late 2021 the Payment Accuracy Roadmap functioned as an organising concept for a range of short-term (up to two-year) initiatives Services Australia was implementing to address payment accuracy, with the specific initiatives that fell under the roadmap shifting over time. Activities falling under the roadmap were communicated to BMC in December 2021 and a presentation on the roadmap was provided to the department in May 2022.

3.110 In June 2022 Services Australia developed an Improving Payment Accuracy Charter, subsuming the earlier Payment Accuracy Roadmap, which outlined a longer-term vision for payment accuracy improvements (see Table 3.12). Reporting on the charter to the Services Australia CEO in October 2022 noted that the entity’s current approach lacked fairness and relied on recipient self-disclosure, and described a ‘future state’ vision involving large-scale use of third-party data, prefilling of recipient forms, near real-time adjustment of payments, and minimal ‘touch points’ for recipients. Reporting to Services Australia’s audit committee in December 2022 noted that the charter would be registered as a program under the entity’s Portfolio Programme and Project Framework, with key activities under the charter required to develop a range of plans covering risks, benefits, data management, stakeholder engagement and communications. As of June 2023 Services Australia had not completed any of the required program planning documentation for the charter.

Table 3.12: Improving Payment Accuracy Charter key activities

Source: ANAO analysis of Services Australia documentation.

Use of performance information to drive continuous improvement

Services Australia’s use of data analysis

3.111 Services Australia engaged KPMG Australia in 2021 and 2022 to analyse PARP data and provide insights on improving payment accuracy. KPMG undertook statistical modelling, combining PARP with other administrative data, to identify inaccuracy risk factors for various welfare payments (Age Pension, DSP, Austudy, Youth Allowance, Carer Payment, Parenting Payment and JobSeeker Payment).⁹³ The key insight identified through this analysis was that recipients receiving partial rates of the relevant payment (rather than the maximum amount) were more likely to receive inaccurate payments (which was found to be a statistically significant factor for most payments). Payment-specific insights included identifying that the following cohorts were more likely to have inaccurate payments:

• Age Pension recipients who were single;
• DSP recipients living in the Australian Capital Territory;
• Austudy recipients who were non-fulltime students;
• Youth Allowance recipients who were non-fulltime students and/or dependents;
• Carer Payment recipients who were caring for only one person; and
• Parenting Payment Partnered recipients who were male or whose partner was not receiving income support.

3.112 Based on its findings, KPMG recommended that Services Australia:

• prioritise its payment accuracy remediation activities towards statistically significant factors with greater potential for payment inaccuracy reduction;
• monitor and measure remediation effectiveness by applying benchmarks and capturing data to measure the impact of interventions; and
• invest in building enduring in-house analytical capability.

3.113 Services Australia has not explicitly responded to these recommendations. Further, Services Australia’s 2022–23 payment accuracy risk management plans (discussed at paragraphs 2.44 to 2.46), which were developed after the KPMG analysis was completed, did not include reference to any of the statistically significant risk factors identified by KPMG.

3.114 For initiatives being implemented through the Payment Accuracy Roadmap and Improving Payment Accuracy Charter, Services Australia has been undertaking ‘discovery’ analysis and research and running ‘proof-of-concept’ trials. This has included analysing PARP data, sometimes supplemented with other administrative data, to identify potential improvement actions. Services Australia has noted in internal reporting that availability of reliable data to baseline and measure effectiveness of its proof-of-concept trials has been a challenge, as:

Significant fluctuations in customer populations over the last two years, and the turning off of several processes in response to natural disasters and the pandemic has meant that comparisons with previous years may not tell an accurate picture.

Assessment of Single Touch Payroll impacts

3.115 As discussed in paragraph 2.22, in November 2022 the Secretary of the department wrote to the Services Australia CEO requesting greater assurance on the role of STP in improving payment accuracy. Subsequently, Services Australia has provided data to enable the department to assess the impact of STP on payment accuracy outcomes, including:

• using administrative data to assess the impact on recipient reporting behaviour of STP automatically prefilling online forms with reported earned income; and
• increasing the number of recipients with STP prefilling in the 2022–23 PARP sample to test the efficacy of STP in reducing payment inaccuracy.

3.116 As of June 2023 due to the time lag in potential benefits from STP being realised, the department’s analysis of available data had not produced clear evidence that STP was driving improvements in payment accuracy results.⁹⁴ Separate Services Australia analysis noted that, after the introduction of STP prefilled data, there was a 64 per cent increase in monthly average cancellations rates due to recipients failing to report fortnightly earnings (from 0.28 per cent over the period of July 2019 to February 2020 to 0.46 per cent in 2021–22).

Scope for greater use of data analysis

3.117 While the department and Services Australia have been using data to design and evaluate payment accuracy initiatives, limitations of the PARP methodology (discussed in paragraphs 3.23 to 3.32) and PARP data quality issues (discussed in paragraphs 3.52 to 3.54) constrain the entities’ ability to drill into deeper payment inaccuracy causes. Many of the initiatives being implemented by Services Australia appear to be based on the assumption that helping welfare recipients to ‘get it right’ by correctly reporting changes in their circumstances will drive improvements in payment accuracy. Services Australia stated in its 2022 incoming government brief to the Minister for Government Services that ‘most non-compliance is inadvertent’. However, as the PARP does not differentiate between recipient fraud, opportunistic non-compliance and inadvertent non-compliance, the evidence base for this statement is limited. Consequently, it is possible that Services Australia may not be effectively targeting the root causes of payment inaccuracy through its Improving Payment Accuracy Charter activities.

4.1 Commonwealth entities must develop performance measures for their corporate plans that use reliable and verifiable sources of information and methodologies and provide an unbiased basis for the measurement and assessment of their performance.⁹⁵ This chapter examines whether the department and Services Australia⁹⁶ have a robust methodology for monitoring payment timeliness, and whether reporting on payment timeliness has been reliable, verifiable, and free from bias. Since effective analysis of performance information can help to identify deficiencies in current processes and drive a culture of continuous improvement, this chapter also examines whether the department and Services Australia are using performance information effectively to continuously improve payment timeliness.

Has a robust methodology been established for monitoring payment timeliness?

Claims processing timeliness methodology

4.2 The department and Services Australia revised their existing bilateral key performance measures (KPMs) in 2014 when the Bilateral Management Arrangement (BMA) was established, which included documenting assessment rules. For welfare payment timeliness the assessment rules were:

All customers are connected to an income support payment in accordance with policy requirements and standards. […]

Customer is any individual enquiring and claiming a payment under the Social Security Act 1991, ABSTUDY Policy Manual or Family Assistance Act 1991.

Connection is the time period from when a customer enquires about entitlement to registration of their income support payment.

4.3 The relevant ‘standards’ referenced in the assessment rules are KPM benchmark targets and timeframes agreed between the department and Services Australia (for example, for Age Pension the KPM benchmark target is 80 per cent of claims processed within a timeframe of 49 days). Current payment timeliness KPMs are outlined in Table A.3 at Appendix 5.

4.4 The methodology for calculating claim processing timeliness is outlined in Box 3.

Note a: Most welfare payments have abridged reclaim procedures if a person loses qualification and reapplies for a payment within 13 weeks of cancellation. Job seekers returning to JobSeeker Payment and Youth Allowance have abridged reclaim procedures extended to 52 weeks.

Claims reopened on review or appeal

4.5 A July 2019 internal audit by Protiviti of Services Australia’s aggregate payment timeliness performance measure identified that claims reopened due to reviews or appeals were being backdated to the original claim receipt date, meaning they generally exceeded KPM timeframes upon reopening. The audit included a recommendation to address the issue, which was agreed by management for implementation by 31 October 2019.

4.6 A brief to the responsible Services Australia Deputy Chief Executive Officer (CEO) in July 2020 noted:

During 2018–19, approximately 24% of claims were rejected, and almost half of these were then reopened either on appeal, or through the provision of additional evidence in support of the claim within 13 weeks.

4.7 The brief proposed a methodology change to implement a new ‘KPI start date’ field (discussed in Box 3 above), claiming it: ‘better represents when we can first start processing the claim and more accurately measure[s] timeliness’. In relation to reopened claims, the relevant date is the date the claim is reopened not the original claim receipt date.

4.8 Other points raised in the brief included that:

• some reopened claims had previously been classified as ‘abridged claims’, which Services Australia excluded from its performance measure calculations, but the department included in bilateral KPM calculations, resulting in ‘confusion and inconsistency’; and
• introducing the new method would allow Services Australia to include ‘abridged claims’ in its timeliness calculations and was likely to provide a performance increase of around six per cent against the corporate plan performance measure.

4.9 The brief noted that it would be important to ‘manage any perceptions this is designed to ensure a better claims timeliness result’. To this end, Services Australia provided a copy of the brief to the responsible Deputy Secretary within the department in July 2020, seeking advice on whether the department had any concerns with the proposed change. The department discussed the proposed change internally, with feedback from areas responsible for welfare payment policy noting:

it may be appropriate to consider the timeliness benchmarks and whether the number days should be reduced (depends on whether the current benchmarks have been set to accommodate the additional time for backdating reopened claims).

…

we are concerned that the proposed change would also exclude periods of delay which are within the control of Services Australia. For example, the period from the original claim date to the date the person is notified of the claim’s initial rejection, and the period of time taken […] to process an internal review. We suggest that there is more information needed and proper process to follow before agreeing on a specific change. We consider it would be appropriate for [Services Australia] to bring forward a proposal to the BMA Governance meetings for agreement.

4.10 No evidence was found that the department raised these or any other concerns about the proposal with Services Australia or responded to the request for advice. The methodology change was agreed by the Services Australia Deputy CEO in July 2020 and implemented for the 2020–21 financial year. The change was not directly considered and agreed by bilateral governance bodies. The implications of this decision for the reliability of reporting on timeliness are discussed at paragraphs 4.35 to 4.40.

Bilateral reviews of payment timeliness KPMs

4.11 Discussion at bilateral governance body meetings of the need to review bilateral KPMs, particularly for claim processing timeliness, began in late 2017. In April 2018 the Bilateral Management Committee (BMC) agreed to commence an independent review of bilateral KPMs after welfare payment infrastructure transformation upgrades had been rolled out or in 12 to 18 months, ‘whichever is earlier’. The BMC chair (a Deputy Secretary within the department) clarified in July 2018:

this does not mean no work on the KPM review is required before the independent review … especially in the context of the ANAO follow-up audit on [Disability Support Pension (DSP)], which is likely to make recommendations relevant to the KPM review.

4.12 In November 2018 Auditor-General Report No. 13 of 2018–19 Disability Support Pension — Follow-on Audit found the timeliness KPM for the DSP was biased as it excluded claims with processing times over 84 days. The audit included a recommendation, agreed by the department and agreed with qualifications by Services Australia, that the entities review and revise external and bilateral KPMs for the DSP.⁹⁷

4.13 The department and Services Australia commenced discussions on KPM review principles and terms of reference for the proposed independent KPM review in late 2018 and early 2019. Comments on iterations of these documents indicate there was disagreement between the department and Services Australia senior executives about the importance of claim processing timeliness KPMs for the bilateral arrangement. Minutes of an October 2019 Strategic Business Discussion (SBD) meeting, where the KPM review process was discussed, noted:

With regard to [Services Australia’s] proposal of moving KPMs to focus on outcomes and removing claims processing timeliness as a KPM in the Principles paper, DSS indicated that both timeliness and accuracy are fundamental for achieving policy outcomes.

4.14 Discussion between the department and Services Australia regarding bilateral KPM review principles recommenced in June 2020. On 3 November 2020 BMC agreed in principle to draft KPM review principles and to progress KPM reviews for three priority programs (DSP, Carer Payment/ Allowance and Debt Management). Over the next year (from November 2020 to December 2021), BMC and/or SBD considered timeliness KPMs for nine welfare payments, agreeing changes for seven payments (see Table 4.1). These bilateral KPM decisions:

• standardised KPM benchmarks at 80 per cent (where changes were agreed);
• addressed bias for three timeliness KPMs that had excluded claims over 84 days;
• made KPMs timeframes longer for four payments;
• made KPMs timeframes shorter for two payments; and
• left KPM timeframes the same for three payments.

4.15 The independent review of KPMs, which BMC agreed to in April 2018, was not progressed.

Robustness of rationales for KPM changes

4.16 Table 4.1 includes the ANAO’s assessment of the robustness of the rationales for changes to bilateral timeliness KPMs presented to the decision-making bodies. Robustness was assessed based on whether the rationale included consideration of:

• claim processing stages and reasonable timeframes for each stage;
• claim pathways (such as automated or manual claims, or simple or complex claims);
• claimants’ needs for timely support;
• Services Australia’s past performance and claim processing capacity; and
• business rules for calculating KPM results.

4.17 A largely robust rationale was provided to BMC to support the decision to change the Carer Payment and Carer Allowance timeliness KPMs in June 2021. The business case outlined claim processing stages for each payment and indicative timeframes for each stage.⁹⁸ While the sum of timeframes for processing stages was 59 days for Carer Payment and 52 days for Carer Allowance, 49 days was assessed to be an achievable KPM timeframe based on previous KPM results and because stages were ‘not strictly sequential’. The paper provided to BMC noted Services Australia’s change to its business rules for calculating claim timeliness, but did not explicitly consider the rules that should apply in calculating KPM results. There was also no explicit consideration of claimants’ needs for timely receipt of payments or notification of the rejection of claims so they may consider alternative financial supports.

4.18 A partly robust rationale was provided for the DSP timeliness KPM change. The business case provided to BMC included consideration of claimants’ needs, noting that DSP claimants have the ‘option to claim an interim income support payment while their claim is assessed’. The business case also included a breakdown of five stages involved in processing DSP claims, with indicative timeframes for each stage.⁹⁹ The rationale for the 84-day timeframe noted that:

• 54 per cent of claims are referred for a comprehensive job capacity assessment by a Services Australia assessor, estimated to take 21 days; and
• 29 per cent of claims are referred for a disability medical assessment by a government-contracted doctor, estimated to take 30 days.

4.19 The business case did not include consideration of creating separate KPMs for routine and complex DSP claims. Instead, the timeframe of 84 days was determined based on completing all five stages, despite only 29 per cent of claims following that pathway.

4.20 For other payments, recommendations were largely based on analysis of the number of days it took Services Australia to process 80 per cent of claims over recent years. KPM timeframes were agreed at levels that were generally achievable, with the analysis showing Services Australia usually processed 80 per cent of claims in fewer days than the KPM. For example, analysis for the Age Pension timeliness KPM showed Services Australia had processed 80 per cent of claims in 49 days or less for 17 out of the last 19 quarters (see Figure 4.1). The rationale for the 49-day timeframe noted:

while ongoing service delivery reform has resulted in improvements to timeliness over recent years (with the temporary workforce surge for claims processing due to COVID-19 further improving timeliness), this will be offset by demographic change to the Age Pension claim population who increasingly have more complex financial circumstances (income and assets) to report and assess as part of their claim.

Figure 4.1: Number of days to process 80 per cent of Age Pension claims, by quarter, 2016–17 to 2020–21

Source: DSS and Services Australia analysis presented to SBD on 16 August 2021.

4.21 The department advised the ANAO that:

• operational planning was a key factor in Services Australia’s claim processing timeliness, rather than business processes; and
• the department was frustrated with its level of visibility of Services Australia’s resource allocation decisions, particularly as they relate to welfare payments.

4.22 The department’s observation about operational planning being a key factor in payment timeliness is supported by internal analysis undertaken by Services Australia. A brief to the Services Australia accountable authority in December 2018 noted that online claiming ‘ensures collection of all necessary information upon claim submission’, which means the need to seek additional information is greatly reduced for payments that do not require additional assessment stages. Consequently, claim processing timeframes for payments such as Family Tax Benefit, JobSeeker Payment and Parenting Payment are largely a function of how long it takes Services Australia to allocate the claim to an officer who has capacity to process it.¹⁰⁰

4.23 As shown in Table 4.1, timeliness KPMs for nine welfare payments were considered by SBD or BMC in 2020 and 2021. As of June 2023, timeliness KPMs for 12 additional working age and student payments had not been considered by BMC or SBD. While initial planning work on revising these KPMs began in late 2021, the department requested that negotiations be put on hold in May 2022 due to competing priorities. Services Australia continued working on the KPM review project internally and provided a proposal to the department for revising 11 of the 12 remaining KPMs in March 2023. In April 2023 the department responded requesting additional analysis to support the proposed changes and noting 8 of the 11 proposed KPMs involved extending existing timeframes. The department also requested information on potential levers available to Services Australia to improve timeliness performance (such as shifting resources or cross-training staff).

Has reporting of payment timeliness results been reliable, free from bias, and verifiable?

4.29 Prior to 2020–21 the department used administrative data from Services Australia’s welfare payment processing system to report against welfare payment timeliness KPMs to bilateral governance bodies through the quarterly performance assurance report. Since 2020–21 Services Australia has used the same data to prepare quarterly bilateral timeliness KPM reporting. Reported quarterly payment timeliness results for 2018–19 to 2021–22 are reproduced at Appendix 5.

4.30 In addition, Services Australia has reported against an aggregate payment timeliness performance measure in its annual performance statements — ‘Achievement of processing service level standards: claims processed within standard’.¹⁰¹ In 2021–22 Services Australia introduced a weighted composite ‘work processed within timeliness standards’ strategic performance measure, combining welfare payment timeliness with timeliness measures for Medicare, child support and emergency payments to produce a single entity-wide result. Services Australia’s reported welfare payment timeliness results since 2015–16 are reproduced in Figure 1.4 above.

4.31 The ANAO assessed whether the department’s and Services Australia’s reporting of payment timeliness results was reliable, free from bias, verifiable over the period of 2018–19 to 2021–22. Table 4.2 outlines the assessment criteria applied and the ANAO’s overall assessments against these criteria.

Table 4.2: ANAO assessment of payment timeliness reporting

Reliability

Coverage of welfare payments

4.32 While the department and Services Australia have used appropriate data sources to calculate payment timeliness reporting, reporting does not cover all of the department’s welfare payments. The department and Services Australia have bilateral timeliness KPMs for 22 of the 32 welfare payments listed in Appendix 3.¹⁰²

4.33 Services Australia included a note in its 2022–23 annual performance statements in relation to its ‘work processed within timeliness standards’ strategic performance measure:

The scope of this performance measure is limited to work types with an agreed timeliness standard. Any work types without an agreed timeliness standard are excluded from the scope of this performance measure. As new timeliness standards are agreed with partner agencies, the measure will be updated to incorporate these new work types.¹⁰³

4.34 There is no clear rationale as to why some welfare payments do not have a timeliness standard. Accordingly, the department and Services Australia should develop additional timeliness KPMs for any welfare payments that are currently excluded.

Distinguishing between categories of claim processing actions

4.35 The original intent of bilateral payment timeliness KPMs was to measure the time that Services Australia takes to process welfare payment claims from the date of a claimant enquiring about a payment (later changed to date of receipt of a completed claim) to the date of a decision to grant or reject a claim. Services Australia’s introduction of the ‘KPI start date’ field from 2020 (discussed in Box 3 and at paragraphs 4.6 to 4.10 above) has meant timeliness KPMs are now measuring three different categories of claim processing actions:

• processing of new claims from claim receipt to initial grant or rejection decision;
• processing of claims reopened following a review or appeal process; and
• processing of abridged claims (where recipients reapply for an income support payment within 13 weeks of losing qualification or within 52 weeks for job seekers returning to JobSeeker Payment or Youth Allowance).

4.36 Inclusion of these claim actions in KPM calculations means timeliness KPMs are not measuring comparable processes. Different KPMs could be developed for different categories of processing actions based on the business processes involved. Analysis of the distribution of claim processing timeframes in 2020–21 and 2021–22 (see Figure 4.2) shows most claims with KPI start dates that differed from their claim receipt dates (suggesting they were reopened for processing) were processed in zero days (that is, on the same day they were reopened).¹⁰⁴ However, KPM timeframes for most welfare payments range from 14 days to 84 days (with the exception of Crisis Payment, which has a KPM timeframe of 2 days). This demonstrates that processing of reopened claims could have a shorter KPM timeframe.

Figure 4.2: Distribution of claim processing timeframes by days taken to process, 2020–21 and 2021–22 (excluding Crisis Payment)

Source: ANAO analysis of Services Australia data.

Comparability of results across periods

4.37 When entities make changes to their performance measurement methodologies, it is good practice to include a note that details the start date for the revised methodology, the changes that occurred and the rationale.¹⁰⁵ Services Australia included such a note in its 2015–16 Annual Report for its welfare payment timeliness performance measure, noting a ‘revised calculation method was adopted for 2015–16, therefore the result is not comparable to earlier years’.¹⁰⁶

4.38 No note was included in Services Australia’s 2020–21 Annual Performance Statements in relation to the introduction of the KPI start date methodology change, despite the increase in its reported result that was due to this methodology change and thus not reflective of any improved performance (the implications of this for reporting bias are discussed at paragraph 4.39 and 4.40). Instead, Services Australia stated that its 2020–21 result of 91.4 per cent was ‘the agency’s best result since reporting on this measure commenced in 2012–13’ (making an explicit comparison between results that are not comparable).¹⁰⁷ Similarly, reporting on timeliness KPM results to bilateral governance bodies and the Secretary of the department did not acknowledge the methodology change and its implications for comparability of results.

Freedom from bias

Bias in reporting timeliness results

4.39 As noted in paragraph 4.38, the introduction of the KPI start date methodology change in 2020 provided an increase to Services Australia’s aggregate welfare payment timeliness results. The methodology change was first introduced for the JobSeeker Payment in late 2019–20, then applied to all payments from 2020–21. As shown in Table 4.3, the impact of this change increased from a 1.1 percentage point increase in 2019–20 to a 5.1 percentage point increase in 2021–22, compared with Services Australia’s previous business rules (which had excluded abridged claims).

Table 4.3: Impact of ‘KPI start date’ methodology change on Services Australia’s reported results for ‘Achievement of processing service level standards: claims processed within standard’, 2019–20 to 2021–22

Key: ◆ ANAO recalculation matched reported result ■ ANAO recalculation did not match reported result

Source: ANAO analysis of Services Australia data using amended Services Australia business rules (adjusting for errors identified by ANAO, discussed in paragraph 4.51).

4.40 Implementing this change with no explanatory notes advising users of the performance information that results were not comparable with previous years introduced bias to Services Australia’s reporting on payment timeliness.

Incentives to prioritise processing of certain claims

4.43 Services Australia has identified that current payment timeliness KPMs create incentives to prioritise certain welfare payment claims over other claims and non-claim work items in three distinct ways.

4.44 Firstly, because Services Australia’s aggregate payment timeliness performance measure is based on the total number of claims processed, it creates an incentive to prioritise meeting timeliness KPMs for payments with higher claim volumes (such as Crisis Payment, Jobseeker Payment and Family Tax Benefit). This was noted in the December 2018 brief to the accountable authority discussed at paragraph 4.21.

4.45 Secondly, once the ‘age’ of a claim has exceeded its KPM timeframe, there is an incentive for Services Australia to prioritise processing of newer claims that are still within their KPM timeframes. This was noted in a February 2019 brief to the accountable authority, which stated: ‘The timeliness measure also creates a tension between the prioritisation of new and older claim processing’. Recognising this tension, in 2018–19 Services Australia decided to shift focus to clearing claim arrears. Services Australia had reported identical results of 82.1 per cent for its aggregate performance measure in 2016–17 and 2017–18 against its target of 82.0 per cent, but in 2018–19 its reported result was 68.0 per cent (a 14.1 percentage point decline). Services Australia noted in its 2018–19 Annual Report that the reduction in performance reflected ‘the priority given to reducing the number of claims on hand, in particular older claims’.¹⁰⁸ An October 2022 internal audit by Protiviti found prioritising performance measure results was still an issue for Services Australia, with the report noting:

When a [claim] is not processed by the agency within the agreed [KPM], the [claim] is not explicitly and effectively prioritised, and remains as an unprocessed work item until agency resources for processing are available.

There is no defined and repeatable process to periodically monitor, review and report on the status and closure of unprocessed [claims]. The review of unprocessed [claims] is ad-hoc.

4.46 Thirdly, the existence of an entity-wide performance measure for the timeliness of work processed within service standards creates an incentive to prioritise processing of work with KPMs over other work items that do not have KPMs (such as processing recipient change in circumstances declarations). The October 2022 Protiviti internal audit also noted:

there are no established [KPMs] for the non-claims processed. If non claim work items remain un-processed for an extended period of time, there is a small risk of customer hardship.

The internal audit recommended that Services Australia develop KPMs for processing non-claim work items and establish processes for effectively monitoring unprocessed claims that have not met their KPM timeframes. The management response noted that Services Australia was reviewing its prioritisation methodology for non-claim work items and an internal KPM would be developed as part of that process, with an implementation date of 30 June 2023.

Verifiability

4.49 To test the verifiability of reported payment timeliness results, the ANAO:

• obtained the data sources that the department and Services Australia used to calculate quarterly payment timeliness KPM results for 2018–19 to 2021–22; and
• reperformed calculations using documented methodologies to verify reported results.

Quarterly KPM results

4.50 The ANAO’s initial verification testing for reported quarterly timeliness KPM results for individual payments found 72.6 per cent of reported results were verifiable over the assessed period (for detailed verification results see Table A.9 in Appendix 12).

4.51 The ANAO identified several errors in the business rules, including that:

• the department did not document the exclusion of claims over 84 days in its business rules for DSP, Carer Payment and Carer Allowance in 2018–19 and 2019–20;
• the department included ‘combined claims’ for Carer Payment and Carer Allowance in 2018–19 and 2019–20¹⁰⁹, and abridged claims for Austudy, Newstart Allowance, Sickness Allowance and Special Benefit in 2019–20, but this was not noted in its business rules;
• the department and Services Australia included ‘non-effective’ claims for Family Tax Benefit and Dad and Partner Pay from 2018–19 to 2021–22, but this was not noted in their business rules¹¹⁰; and
• Services Australia’s business rules noted the Family Tax Benefit KPM timeframe to be 31 days in 2021–22, when it was 33 days.

4.52 Table 4.4 shows, after correcting for the identified errors, the ANAO was able to verify 90.6 per cent of reported timeliness KPM results (for detailed verification results see Table A.10 in Appendix 12).

Table 4.4: ANAO revised verification of reported quarterly timeliness KPM results, 2018–19 to 2021–22, updated to address errors in business rules identified by ANAO

Source: ANAO analysis of DSS and Services Australia data using amended business rules (adjusting for errors identified by ANAO, discussed in paragraph 4.51).

Annual performance measure results

4.53 The results of the ANAO’s verification testing for Services Australia’s annual reporting against its aggregate payment timeliness performance measure are shown in Table 4.5. After adjusting for the errors in business rules identified in paragraph 4.51, ANAO’s recalculated results were close but not identical to Services Australia’s reported results in all years.

Table 4.5: ANAO verification of Services Australia’s reported results for ‘Achievement of processing service level standards: claims processed within standard’, 2018–19 to 2021–22

Key: ◆ ANAO recalculation matched reported result ■ ANAO recalculation did not match reported result

Source: ANAO analysis of Services Australia data using amended business rules (adjusting for errors identified by ANAO, discussed in paragraph 4.51).

Is performance information used to drive continuous improvement in payment timeliness?

Changes in payment timeliness KPMs since the 1990s

4.57 KPMs for the timeliness of welfare claim processing formed part of Centrelink’s first bilateral partnership agreements developed with purchaser entities in 1997. Table 4.6 shows a comparison of bilateral timeliness KPMs in 1999–2000, 2014–15 and 2022–23 for fifteen welfare payments that have continued since 1999.

• Eight of the fifteen KPMs were the same in 2021–22 as they were in 1999–2000.
• For the six KPMs that were different, the agreed changes provided longer timeframes for Services Australia to process these welfare payment claims.

Table 4.6: Timeliness KPMs for selected welfare payments, 1999–2000, 2014–15 and 2022–23

Key: ↑ longer → no change ↓ shorter

Note a: From 2014–15 to 2020–21 the timeliness KPMs for Carer Payment, Carer Allowance and DSP excluded claims over 84 days. These KPMs were changed from 1 July 2021.

Note b: JobSeeker Payment replaced Newstart Allowance from 20 March 2020.

Source: ANAO analysis of Centrelink, DHS, DSS and Services Australia reporting.

4.58 Since 1999, changes to welfare claim processing have made bilateral timeliness KPMs easier to achieve, but KPM timeframes have generally not decreased over the same period.

• As noted in Box 3, the start date for timeliness KPMs was previously the date a claimant first enquired about a welfare payment. The start date is now the date that a claim is available for processing, either when Services Australia receives a fully completed claim form or when a claim is reopened following an appeal or review.
• Upgrades to the Centrelink online account claim form mean claimants need to upload files when prompted for required documentation, otherwise they cannot lodge the claim. This means most claims lodged online can be processed by Services Australia staff on a single day (usually in less than an hour), without the need to request additional information.
• In addition, automated processes such as ‘straight through processing’ enable claims that meet certain eligibility criteria to be automatically processed on the date of lodgement, allowing Services Australia staff to focus on more complex claims. Services Australia internal reporting indicates an increasing proportion of welfare claims are being automatically processed, with close to 100 per cent accuracy rates for decisions.¹¹¹

4.59 As shown in Figure 4.3, a high proportion of claims are processed in zero days, and the next most common timeframe of processing is the last day of the relevant KPM timeframe.

Figure 4.3: Distribution of claim processing timeframes as a proportion of KPM timeframes, 2018–19 to 2021–22 (excluding Crisis Payment)^a

Note a: Timeframes for claim processing records have been converted into proportions of their relevant KPM timeframes and then grouped into ‘bins’ representing 5 percentage point increments. The bars in the figure represent the count of records in each bin.

Source: ANAO analysis of Services Australia data using business rules provided.

4.60 In May 2023 the department advised the ANAO that it seeks to influence Services Australia’s decisions about resource prioritisation, but there are limited levers available to the department to drive improvements in payment timeliness.

Services Australia’s prioritisation of claims

4.61 In November 2016 Services Australia introduced a new workload management system with the aim of effectively managing the allocation and prioritisation of its welfare payment processing workload. Work items covered by the system include new welfare claims and other non-claim work items (such as processing recipients’ reported changes of circumstances, medical certificates or rent certificates). Items are allocated and prioritised as follows:

• each work item is assigned a ‘priority’ (a numerical value indicating when it should be allocated) and ‘capability’ (representing the skills, proficiency or delegation required to complete the task)¹¹²;
• Services Australia processing officers have assigned ‘skill tags’ identifying their capability to process certain tasks; and
• work items are allocated to staff based on a processing officer’s availability and skill tags and the work item’s priority.

4.62 Services Australia workload managers can make adjustments to priorities and allocations using the workload management system, informed by a ‘strategic priorities model’ that outlines a hierarchy of entity priorities (see Figure 4.4), and tactical considerations that are communicated by entity senior executives through weekly operational plans. An example of prioritising ‘ministerial and government outcomes’ occurred in 2018–19. An April 2019 paper to SBD noted:

The Minister for Human Services directed [Services Australia] at the start of the current financial year to prioritise processing of claims for Students and Age Pension requiring a shift in resources from Newstart Allowance claim processing.

Figure 4.4: ‘Strategic priorities model’ from workload management system operational guide

Source: Services Australia documentation.

4.63 Services Australia’s operational guide to the workload management system outlines three approaches to prioritising and allocating work items:

• KPI focus — prioritises claims that are at or approaching their KPM timeframes, then newer claims to ensure further information can be requested if needed, with claims that have passed their KPM timeframes and other work items not prioritised;
• Arrears reducing — prioritises claims that are at or approaching their KPM timeframes, then claims that have passed their KPM timeframes working backwards from the oldest claims; and
• Optimal — prioritises claims that are at or approaching their KPM timeframes, then claims that have passed their KPM timeframes, then newer claims.

4.64 Services Australia explicitly adopted a ‘KPI focus’ approach in 2016–17 and 2017–18, then shifted to an ‘arrears reducing’ approach in 2018–19 due to a large backlog of claims that had passed their KPM timeframes. In May 2023 Services Australia advised the ANAO that it has aimed to adopt an ‘optimal’ approach since 2019–20, while balancing workforce across processing and telephony and responding to additional demand for services stemming from the COVID-19 pandemic and emergency events. All approaches involve prioritising claims that are at or approaching their KPM timeframes, which is consistent with the trend identified in Figure 4.3 above.

Appendix 1 Entity responses

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2022–23 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• The Department of Social Services (the department) and Services Australia signed a revised Bilateral Management Arrangement Head Agreement in April 2023, which raises bilateral governance body oversight of payment accuracy and timeliness to the accountable authority level (see paragraphs 2.4, 2.16 and 2.24).
• In November 2022 Services Australia updated its Operational Blueprint procedures on undertaking Payment Accuracy Review Program (PARP) reviews to include links to guidance on reporting suspected fraud (see paragraph 3.42).
• The department included an addendum to its 2021–22 Annual Performance Statements that included financial estimates of overpayments and underpayments for each payment type within the, which increased the reliability of its reporting (see paragraph 3.69).
• From November 2022 the department and Services Australia started using the PARP and analysis of other administrative data to assess the impact of using Single Touch Payroll (STP) data to pre-fill recipients’ employment reporting forms¹¹³ on payment accuracy outcomes (see paragraphs 3.115 and 3.116).

Appendix 3 DSS welfare payments delivered by Services Australia

Table A.1: DSS welfare payments delivered under Outcome 1 in 2021–22 and key performance measure (KPM) coverage

Key: ✔ yes ✘ no

Source: DSS, Annual Report 2021–22, Commonwealth of Australia, 2022 and payment descriptions published on DSS’s and Services Australia’s websites.

Appendix 4 Payment accuracy results

Appendix 5 Payment timeliness results

Appendix 6 Bilateral Management Arrangement structure

Figure A.1: BMA agreement structure, as of June 2023

Source: ANAO analysis of DSS and Services Australia documentation.

Appendix 7 ANU review recommendations

Table A.4: Implementation status of recommendations from the 2021 ANU review of the Payment Accuracy Review Program (previously known as the Random Sample Survey or RSS) methodology

Note a: Implementation status is sourced from reporting to the Bilateral Management Committee (BMC) on 23 March 2023 and a DSS ‘ANU Recommendations Dashboard’ (Dashboard) updated on 15 March 2023. These sources used different criteria for assessing implementation status, which are noted in the table. The ANAO has not assessed the implementation status of each recommendation.

Source: DSS and Services Australia documentation.

Appendix 8 Treatment of underpayments in payment accuracy formula

1. The formula DSS uses to calculate payment accuracy is outlined in Figure A.2.

Figure A.2: Current formula for payment accuracy

Source: M. Gray, K. Reddy, N. Biddle & D. Stanton, Review of Methodology for the Random Sample Survey (RSS) Program, ANU Centre for Social Research and Methods, 24 August 2021, p. 36.

2. The 2021 ANU PARP review made two recommendations relating to the formula:

• to include supplementary payments (for example, Rent Assistance and Mobility Allowance) received by sampled recipients in payment accuracy reviews (ANU recommendation 7); and
• to include additional welfare payments received by sampled recipients in payment accuracy reviews, rather than just the sampled payment (ANU recommendation 8).

3. Once implemented, the revised formula to calculate payment accuracy will be as outlined in Figure A.3.

Figure A.3: Proposed formula for payment accuracy

Source: M. Gray, K. Reddy, N. Biddle & D. Stanton, Review of Methodology for the Random Sample Survey (RSS) Program, ANU Centre for Social Research and Methods, 24 August 2021, p. 37.

4. The current and proposed formulas are correct in their treatment of overpayments. However, the ANAO identified an issue with the way the formulas treat underpayments. If an individual receives an underpayment, dividing the variation amount by the basic or total amount received at the start of the review period overstates the level of inaccuracy. The variation amount should be divided by the amount received after variations identified by payment accuracy reviews (that is, the amount the individual should have received rather than what they actually received). This is illustrated by the overpayment and underpayment examples below.

• Overpayment — If an individual received $1000 at the start of the review period when they should have received $500 (a $500 overpayment), their payment was 50 per cent accurate with a 50 per cent overpayment.
• Underpayment — If an individual received $500 at the start of the review period when they should have received $1000 (a $500 underpayment), their payment was 50 per cent accurate (variation of $500 divided by $1000 they should have received) with a 50 per cent underpayment. Using the current and proposed formulas leads to the incorrect results that the payment was 0 per cent accurate (variation of $500 divided by $500 they received at the start of the review period) and was a 100 per cent underpayment.

5. To address this issue, the formula should be amended to add the value of underpayments to the denominator, as outlined in Figure A.4.

Figure A.4: Revised formula for payment accuracy

Source: ANAO analysis.

Appendix 9 Analysis of payment inaccuracy reasons

1. Table A.5 shows estimated levels of overpayment and underpayment risk by payment time for 2021–22. Payment types with the highest cancellation risks are working age and student income support payments such as ABSTUDY, Austudy, Parenting Payment Partnered, Youth Allowance and Jobseeker. Jobseeker had the highest estimated level of overpayments involving cancellations at $2.9 billion, representing 44.9 per cent of the total overpayment risk for welfare payments assessed through the PARP.

Table A.5: Overpayments and underpayments by payment type, 2021–22

Note a: Risk is defined as the percentage of expenditure for the payment.

Source: ANAO analysis of DSS and Services Australia data using business rules provided.

2. Table A.6 shows estimated levels of overpayment and underpayment for 2021–22 by primary reason for cancellation or variation. The top five reasons for cancellation accounted for $5.544 billion in estimated overpayments in 2021–22 (85.5 per cent of total overpayments involving cancellations). Four of the top five primary reasons for cancellation related to recipients’ failure to report, or incorrect reporting of, changes in their or their partner’s earned income, or their relationship or study status. The other significant reason for cancellation was ‘generic failure’, where participants fail to engage in PARP reviews.

Table A.6: Overpayments and underpayments by primary reason for cancellation of variation, 2021–22

Note a: Risk is defined as the percentage of 2021–22 total welfare payment expenditure.

Note b: ‘Generic failure’ risk represents cases where participants fail to engage in PARP reviews, such as by failing to attend an interview or provide requested documentation, leading to their payments being cancelled.

Note c: Unearned income refers to income from sources other than salaries or wages (for example, income from property or investments, other government income, gifts, etc.).

Source: ANAO analysis of DSS and Services Australia data using business rules provided.

Appendix 10 Implementation of Payment Accuracy Review Quality Stocktake recommendations

Source: Services Australia PAR Quality Stocktake Dashboard.

Appendix 11 ANAO verification of reported accuracy results

Appendix 12 ANAO verification of reported timeliness results