Introduction

1.1 Launched in July 2015, the jobactive program aims to assist job seekers to obtain sustainable employment, particularly those in receipt of government benefits. The jobactive program is the Australian Government’s primary employment services program.

1.2 The jobactive program objectives are:

• helping job seekers find and keep a job;
• helping job seekers move from welfare to work;
• helping job seekers meet their mutual obligations³; and
• supporting jobactive employment service providers to deliver quality services.

1.3 The jobactive program is administered by the Department of Education, Skills and Employment (DESE). As of 31 December 2021, there were 706,380 job seekers participating in jobactive across 52 employment regions⁴, with an additional 188,121 participating through digital services.⁵

1.4 DESE has contracted around 39 employment services providers (providers) to deliver the jobactive program.⁶ DESE oversees the delivery of the contracted providers. Payments under the jobactive program are made to the providers when the jobactive Deed 2015–2022 (the Deed) and guideline criteria are met. Providers are required to meet compliance requirements set out in the Deed — this is supported by service guarantees and service delivery plans. The payments are split between administration payments and employment outcome payments that are made when jobseeker employment outcomes are met at four, 12 and 26 weeks of employment.⁷ Providers are also reimbursed for services purchased through the Employment Fund General Account (Employment Fund)⁸, relocation assistance provided through the Relocation Assistance to Take Up a Job⁹, and Wage Subsidies¹⁰ as well as for sourcing and managing Work for the Dole¹¹ placements. The DESE Employment Programs and Activation Division administers various employment programs including those listed above that relate to jobactive.

1.5 Figure 1.1 below outlines the spending on jobactive program payments annually since 2015–16.¹²

Figure 1.1: Spending on jobactive

Source: ANAO based on DESE Portfolio Budget Statements and Annual Reports.

Previous reviews

1.6 The ANAO undertook a performance audit of DESE’s design of the jobactive program in Auditor-General Report No.4 2017–18 Jobactive: Design and Monitoring. The audit made two recommendations to DESE to:

• implement a risk-based approach to prioritising the activities required to effectively manage and monitor the delivery of the jobactive program; and
• assess whether the current compliance regime is structured to effectively and efficiently detect and manage non-compliance and adjust as appropriate.

1.7 The Senate Standing Committee on Education and Employment tabled a report in February 2019 titled Jobactive: failing those it is intended to serve.¹³ The report made 41 recommendations with two recommendations relevant to payment integrity. The Australian Government agreed to all recommendations, with the two relevant to payment integrity being:

• Recommendation 36: The committee recommends that the government examine the funding model to ensure that the funding model does not inappropriately incentivise the attainment of short-term or insecure employment outcomes at the expense of more sustainable medium and long-term outcomes. This should include consideration of whether outcome payments are desirable, whether payment timeframes are too short and whether a portion of payments should be clawed back if a participant re-enters the system, and whether outcome payments should be less for insecure jobs; and
• Recommendation 38: The committee recommends that the government review funding arrangements for non-work activities to ensure integrity in the system.

1.8 In 2019, DESE conducted an evaluation of the jobactive program which assessed the effectiveness of the program in achieving job seeker engagement; labour market outcomes for job seekers; and a reduction in regulatory and administrative burdens on providers. The evaluation results have informed guideline and procedures updates relating to the compliance framework. Updates included removing provider requirements for face-to-face training for claiming education outcomes based on participation.

1.9 As a result of economic conditions during the COVID-19 pandemic a higher number of new participants entered the jobactive program. The jobactive caseload increased from 757,316 on 31 March 2020 to 1,432,399 on 30 June 2020.¹⁴ In response to this demand, providers were able to opt in to be paid six weeks of administration fees up-front to help manage the increased caseload.

1.10 Some assurance activities were also deferred by DESE to reduce workloads to assist the providers in meeting their additional caseloads.

Rationale for undertaking the audit

1.11 In 2021–22, the Australian Government budgeted to spend $2,003,915,000 on the jobactive program to help jobseekers move from welfare to work and meet their mutual obligations.¹⁵

1.12 Auditor-General Report No.4 2017–18 Jobactive: Design and Monitoring examined the design of the jobactive program including establishment of monitoring arrangements. There were two recommendations aimed at improving monitoring of the jobactive program and detecting non-compliance. This audit provides assurance to the Parliament on the effectiveness of DESE’s governance and processes in providing assurance on the integrity of payments to providers.

1.13 This audit also has the potential to inform any changes to the jobactive program that could be considered as part of the DESE’s reforms to employment programs that are currently in the development and piloting phase and are due for implementation in July 2022. The implementation of the reform will coincide with the procurement and engagement of new providers and the delivery of a digital platform to support the reforms.

Audit approach

Audit objective, criteria and scope

1.14 The objective of the audit was to assess the effectiveness of the Department of Education, Skills and Employment in managing the integrity of payments to employment service providers.

1.15 To form a conclusion against this objective, the following high-level criteria were considered.

• Has an appropriate framework been established to ensure payment integrity?
• Has DESE effectively implemented the framework to manage and monitor payments to employment service providers?

1.16 The audit examined DESE’s management of the jobactive program, specifically the assurance over the integrity of payments to providers. The audit scope included:

• governance and assurance processes that support payment integrity for providers;
• the IT systems that support payments to providers; and
• monitoring and compliance processes to support payments to providers.

1.17 The audit scope did not include an assessment of the:

• overall effectiveness of the jobactive program;
• integrity of payments to jobactive participants;
• validity or accuracy of the source data from Services Australia systems used in the calculation of payments to providers;
• selection and overall performance of individual providers; and
• payments made in relation to wage subsidies.

1.18 The period in review was from July 2019 to December 2021.¹⁶

Audit methodology

1.19 The audit methodology included:

• examination of procedures and guidelines used to administer jobactive payments including the payment assurance framework;
• undertaking a walkthrough of key systems and processes to review and document the arrangements;
• examination of DESE’s performance monitoring and assurance arrangements for providers;
• undertaking sample testing of payments made to providers to ensure payments are made in compliance with the assurance framework as identified; and
• meetings with relevant entity staff.

1.20 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $249,728.

1.21 The team members for this audit were Synergy Group and Corinne Horton.

2.1 The establishment of an assurance framework to ensure payment integrity assists DESE to effectively manage the risks that arise when processing the large volume of transactions made to providers and helps to facilitate DESE’s achievement of jobactive program outcomes.

2.2 To assess whether DESE has established an appropriate framework to ensure payment integrity, the ANAO examined whether the framework:

• has been appropriately informed by a consideration of risks;
• includes appropriate mechanisms to detect incorrect payments;
• establishes appropriate quality assurance processes; and
• establishes an appropriate monitoring framework.

Has the framework been appropriately informed by a consideration of risks?

2.3 Section 16 of the Public Governance, Performance and Accountability Act 2013 (PGPA Act) requires Commonwealth entities to establish and maintain an appropriate system of risk oversight supported by appropriate internal controls.

2.4 Establishing an effective process to identify risks relating to payment integrity, and aligning the assurance approach to those risks, supports jobactive assurance activities to be directed at the areas where they will be most effective.

Risk management

2.5 DESE’s Enterprise Risk Management Policy and Framework was updated in May 2020 and August 2021 and consists of three components: the Accountable Authority Instructions; the Risk Management Policy; and the Risk Management Framework.

2.6 The Enterprise Risk Management Policy and Framework outlines DESE’s approach to risk management. It specifies DESE’s risk matrix which provides guidance on how to identify, manage and monitor risks of the organisation with consideration to DESE’s risk appetite and tolerance limits.

Payment integrity risks

2.7 Jobactive payment integrity related risks are identified through risk management practices at multiple levels, as required by the Enterprise Risk Management Policy and Framework.

2.8 At the enterprise level, one risk has been identified relating to payment integrity, which is that ‘Payments are not managed effectively, leading to ineligible or incorrect payments, non-compliance and potential fraud.’¹⁷ Specific risks relating to payment integrity are identified in the Payment Integrity Risk Plan, which identifies and rates the effectiveness of risk mitigations measures to form the overall payment integrity assurance approach.

2.9 The Payment Integrity Risk Plan is required to be updated annually and presented to the PISCES. The PISCES is discussed further at paragraphs 2.71 to 2.74. The ANAO reviewed the plans developed since 2018 and found that all were developed and endorsed annually. The Payment Integrity Risk Plan has not changed substantially since 2019 as the program has not changed significantly over this time.

2.10 The Payment Integrity Risk Plan identifies 23 risks, nine of which are relevant to jobactive payment integrity. Table 2.1 below lists the nine jobactive program related payment integrity risks.

Table 2.1: Jobactive related risks

Source: DESE documentation.

2.11 For each identified risk, the Payment Integrity Risk Plan identified causes and existing mitigations.¹⁸ DESE also identify additional treatments to mitigate the risks. The treatments include assurance activities such as RRS review, use of data analytics, targeted assurance and administrative reviews (discussed in paragraphs 2.26 to 2.27). Collectively, DESE applies these treatments to reduce the residual risks related to payment integrity. The results of the implementation of assurance activities are examined in Chapter 3.

Assurance approach

2.12 DESE has developed an Assurance Strategy and Assurance Framework which establishes how DESE’s assurance activities should be developed and targeted at areas of risk.¹⁹

2.13 Assurance functions across DESE operationalise the Assurance Strategy and Assurance Framework by prescribing and undertaking a range of assurance activities. Assurance activities are planned and undertaken by program areas²⁰ as part of managing their programs. The Assurance Strategy and Assurance Framework was established to allow program areas to plan and undertake additional assurance activities across employment programs, including jobactive.

Jobactive Assurance Planning

2.14 Jobactive assurance planning is undertaken by developing an annual assurance planning ‘workbook’ for different activities as part of jobactive. These workbooks are developed in accordance with the Assurance Strategy and Assurance Framework. This involves the program area working with the Assurance Coordination Branch to:

• gather information about the current jobactive assurance environment, to identify assurance objectives;
• undertake a control and risk analysis based on the risks identified in the Payment Integrity Risk Plan to identify which controls (for example, IT controls, contract management process and data monitoring) should be examined through assurance activities;
• examine the types of assurance available to examine the control and meet the desired assurance objectives; and
• based on the above information, plan the assurance activities to be undertaken in the next year.

2.15 Different activities make up the collective jobactive program as a participant moves through the jobactive lifecycle including initial assessment, servicing and employment. The outcomes of the assurance planning workbooks are used to develop annual program assurance plans. Assurance plans are developed for each of the activities of the jobactive program. A total of 10 assurance plans are developed that relate to jobactive. Eight of these are specific to the jobactive program and a further two that relate more broadly across the entire employment program. The jobactive program activities that have specific assurance plans are detailed in Figure 2.1 below.

Figure 2.1: 2021–22 Assurance Plan overview — jobactive participant lifecycle

Source: ANAO analysis based on DESE documentation.

2.16 Assurance plans were endorsed by the relevant program area Assistant Secretary, the Assistant Secretary of the Assurance Coordination Branch and noted by the PISCES in 2021.

2.17 The jobactive Outcomes and administration fees and the Employment Fund make up the most significant value of payments in relation to the overall jobactive program and as such are the most important to payment integrity. The assurance plans summarise the results of the planning workbooks and provide an overview of the annual assurance activities for the jobactive program. Each of the 2021–22 assurance plans have a similar structure and include the following:

• program area overview including program changes since the last review point, annual budget, historical transaction volumes, documentation and evidence requirements, program operations, IT systems and related controls;
• contingency arrangements implemented in response to the COVID-19 pandemic²¹;
• program area risk profile as captured in the Employment Services program level Payment Integrity Risk Plan;
• ongoing monitoring activities for 2021–22; and
• the assurance approach for 2021–22, including details of assurance activities (Table 2.2), links to risk events²², and designated resources and timing.

2.18 Progress against the assurance plans is reported every six months to the PISCES, to review the results of assurance activities and program risks and validate the assurance activities that were planned for the second half of the year. In addition, a review of progress against the assurance plans is undertaken annually.

2.19 A six-month review of all assurance plans commenced in December 2021 concurrently with assurance mapping. Input has been received from the respective program areas on progress with the recommendations in the assurance plans. These are being collated with key messages intended to be distilled for presentation at the April 2022 PISCES meeting.

Provider Assurance Profile

2.20 In addition to the program-level assurance planning process, each provider has an individual Provider Assurance Profile. This profile includes a risk rating for each provider based on several risk indicators including size of provider caseload; range of programs delivered; previous performance; financial viability; previous assurance results; tip-offs and breach actions; quality assurance framework outcomes; and complaints.

2.21 The risk rating for each provider is not directly linked to the assurance workbooks and subsequent plans; however, provider risks are considered as part of targeted assurance activities.

Implementation of recommendations

2.22 Auditor-General Report No. 4 2017–18 Jobactive: Design and Monitoring recommended that DESE:

• implement a risk-based approach to prioritising the activities required to effectively manage and monitor the delivery of the jobactive program.

2.23 In response to the recommendation, DESE reviewed and updated the approach to considering risk as part of their process to develop assurance activities for the jobactive program. Processes have been implemented and are consistently being undertaken, including:

• identifying the risks as they relate to the jobactive program and other programs through consultation and workshops and documenting these risks in the Employment Services Payment Integrity Risk Plan (paragraphs 2.7 to 2.11);
• documenting the effectiveness of risk treatments (paragraph 2.11);
• documenting specific activities to mitigate the risks identified and developing the annual jobactive assurance planning workbook based on these activities (paragraphs 2.14 to 2.15);
• documenting the effectiveness of various treatments to enable prioritisation of assurance activities (paragraph 2.11); and
• undertaking the assurance activities as outlined in this workbook and reporting on these activities (paragraph 3.8).

2.24 The updates made to the approach and the assurance framework were presented to the Employment Services Committee (ESC)²³ on 8 February 2018. These updates were in line with the processes listed at paragraph 2.23 above. Following these updates, the ESC was satisfied the recommendation was addressed.

2.25 To undertake the assurance activities, resources are allocated within the Assurance Coordination Branch to meet prioritised work with additional resources available as needed. This includes where specific activities such as large-scale transactional testing are planned or other sources of resourcing such as contractors or secondments from program areas are used.

Does the framework include appropriate mechanisms to detect incorrect payments?

2.26 DESE conducts a range of management and monitoring activities to ensure that the administration of the jobactive program by providers is in accordance with program requirements. The assurance planning workbooks have been developed each year since 2019–20 and outline assurance activities for the jobactive program which are discussed in Chapter 3.

2.27 Collectively the assurance activities are intended to mitigate the risks associated with payment integrity and other identified risks such as risks associated with privacy breaches. The assurance activities for 2021–22 are described in Table 2.2 below.

Table 2.2: Jobactive assurance activities 2021–22

Source: DESE documentation.

2.28 There is no mechanism to consider the cause of repeated non-compliance in selecting assurance activities or to consider improvements that can be made such as system controls across RRS cycle periods. DESE advised that further analysis of RRS results was undertaken post the completion of this audit. The assurance framework has no mechanisms to consider similar errors in future assurance activities or to review what other corrective action could be undertaken such as further guidance, system fixes or enhancements.

Rolling Random Sample Cycle

2.29 The RRS review is the regular assurance activity applied to each employment program. It is also the activity most closely aligned to payment integrity and is the most complex assurance activity undertaken.

2.30 The RRS review is a planned assurance activity for payments made as part of DESE’s Employment Services programs²⁴, including jobactive payments. The RRS review is a post-payment check where the accuracy of payment processing and the adequacy of documentary evidence supplied by providers are reviewed and assessed for compliance with the jobactive Deed 2015–2022 (the Deed) and guidelines.

2.31 The RRS review is conducted three times (or ‘cycles’) each year. The timing of the cycles has varied as they have moved from a March-to-March period to align to the financial year. Cycles are conducted over two weeks by assessors in DESE’s national office and state and territory offices.

2.32 Assessors examine jobactive payments selected for the RRS review by assessing provider claims for payments for outcomes and the reimbursement of payments and examining documentary evidence for each sampled claim to assess whether it meets program requirements. A sub-sample of assessments is quality assured by separate quality assurance assessors as discussed in paragraph 2.56.

2.33 The processes associated with the RRS review are documented in the RRS Policies and Procedures Manual (the RRS manual). The RRS manual was developed in 2021 and has not been formally endorsed. The RRS manual provides guidance on the administrative steps on how to undertake an RRS review cycle. The RRS manual does not provide guidance on the sampling methodology used to determine the number of payments to be selected or the basis for selection of payments to review.

2.34 A detailed sampling methodology document was tabled at PISCES meetings when the jobactive program commenced in 2015 and again when it was reviewed in 2017. The sampling methodology has not changed since this time. The reporting of the results of each RRS cycle that are provided to the PISCES contains a detailed description of the sampling methodology used.

2.35 The sampling methodology is repeated for each cycle of provider payments and is designed to detect incorrect payments with a 95 per cent confidence level, a 50 per cent expected compliance rate, and a 10 per cent margin for error.

2.36 The Assurance Coordination Branch determines the sample size using the sampling methodology and selects a sample of provider payments for testing each cycle. The sample selection approach is that an initial statistical sample size is calculated and distributed across providers based on an analysis of transaction volumes (payment amounts). Each provider has payments selected and the number of payments selected from each provider is based on the volume of transactions for that provider.

2.37 There are 14 different transaction types included in the RRS review. Not all of these transactions are related to the jobactive program as other employment programs are also covered in the RRS process. The RRS samples are selected from a population of payments that meet the requirements for providers to upload or retain documentary evidence to support the claim for payments.²⁵ The initial sample is distributed across the 14 transaction types in proportion to the number of transactions per provider in each transaction type and the compliance rates for each transaction category. The total sample size is increased where necessary to ensure that a minimum of five transactions are selected for testing for each transaction type for each provider.

2.38 A lack of consistency of application of the sampling methodology was identified. In cycle 16, the sample size for four providers was increased in a manner not consistent with the increase in transaction volumes. There was a general increase in providers sample sizes of 23 per cent in this cycle from cycle 15 because it covered a larger period. The increase in sample sizes for these four providers ranged from 35 per cent to 105 per cent and was not consistent with the increase in transaction volumes. In cycle 17, the sample sizes for these same four providers reduced and was consistent with transaction sizes.

2.39 The RRS review process is described in Figure 2.2 below.

Figure 2.2: Analysis of Rolling Random Sample Review Process

Source: ANAO analysis of DESE documentation.

2.40 The RRS review sampling methodology is independent of other assurance activities. It is a purely random sample based on volume of transactions and the sample sizes are not based on risk. The reasoning for this is to give an unbiased input into the Compliance Indicator. The Compliance Indicator is discussed further at paragraph 2.62.

2.41 Risks for individual providers are considered as part of their assurance profiles (discussed at paragraph 2.20) and factored into the assurance planning workbooks. Additional targeted assurance activities are included to address specific provider risks.

Quality Assurance Framework

2.42 DESE has determined that providers must be certified under ISO 9001:2015 Quality Management Systems²⁶ or the National Standard for Disability Services²⁷ (collectively referred to as the quality standards) under DESE’s Quality Assurance Framework (QAF). Although not directly linked to payments, this helps to ensure providers are working towards quality outcomes which assists with the integrity of their processes and subsequent claims for payments.

2.43 Providers are required to be independently audited by DESE-approved certification auditors against one of these quality standards and the QAF under surveillance and recertification audits as determined by the requirements of the relevant standard. Providers who also provide disability services commit to the National Standard for Disability Services and other providers commit to ISO 9001:2015 Quality Management Systems. The QAF also sets out additional quality principles for the provider which cover the minimum requirements for delivery of Employment Services under the Deed and promotes a strong focus on continuous organisational improvement. The QAF sets the foundation of quality management and supports providers to achieve consistent business processes and drive measurable performance improvements.

2.44 Providers are required to maintain their certifications which includes arranging certification audits. DESE monitors providers to ensure certification is maintained. To maintain certification, providers are required to be subject to certification specific audits, with the audit findings to be reviewed through normal DESE contract management processes to address service issues. The findings are also used to update jobactive Provider Assurance Profiles. To maintain certification, providers must remediate any non-conformance findings identified. A summary of the report findings is provided to PISCES for noting.

2.45 From 2019 to 2021, certification and surveillance audits were completed to maintain certifications. Findings from these audits resulted in two changes to jobactive Provider Assurance Profiles risk ratings with ratings going down. The outcomes of QAF activities are outlined in Table 2.3 below.

Table 2.3: Quality Assurance Framework audit activity

Note a: Certification audits are undertaken by a panel of DESE-approved certification authorities. The 2021 audits had not been finalised and provided to DESE during fieldwork.

Source: ANAO analysis of DESE documentation.

2.46 While outcomes are used from the QAF to consider the risk ratings of individual providers, an overall assessment of the outcomes of QAF audit activities across all providers is not undertaken to identify themes across the jobactive program.²⁸

Administrative arrangements for jobactive payments

2.47 Under the current Deed arrangements, each payment to providers is made on receipt of a claim from the provider. Payment processes are automated through the ESSWeb system.²⁹ The system conducts checks against the Employment Outcome to assess validity, for example of the fortnightly hours worked or income received, to ensure that the Employment Outcome lodged meets the program requirements. DESE has documented ESSWeb process maps to define how payments are made and what triggers exist to initiate a payment.

2.48 Reconciliations of payments are required to be performed monthly between ESSWeb, the DESE general ledger and the DESE departmental bank account. Any discrepancies are investigated and actioned by the ESSWeb team and the Chief Financial Officer (CFO) team. Reconciliation reports are provided to the Chief Information Officer, the CFO and the CFO team. The purpose of this process is to ensure that inconsistencies that could be caused by interface or upload errors are identified early and can be rectified by DESE.

2.49 All providers must keep documentation relating to claims they have lodged where specified in the respective program guideline. Providers are required to upload their supporting evidence for a claim when DESE requests documentation.

Managing non-compliance

2.50 Assurance activities, including the RRS review process, review payments at the transaction level to detect non-compliance. Where potential non-compliant payments are identified, further action is required to be taken.

2.51 Instances of non-compliance are managed through normal contract management by the DESE officer working with the provider to ensure recoveries. For more systemic non-compliance or suspected fraud, DESE has developed a Breach Management Framework. The Breach Management Framework covers any potential breaches of the Deed, not just those related to non-compliance payments. The Breach Management Framework outlines how to identify and record a breach, assess risk, determine remedial actions, and monitor compliance.

2.52 Breach notices issued under the Breach Management Framework require the provider to address the identified non-compliance. Results of breach management activities are discussed further in Chapter 3.

Have appropriate quality assurance processes been established?

2.53 The assurance teams responsible for conducting the RRS review of provider claims for payment are provided with training and guidance material to conduct the payment reviews.

2.54 If any non-compliant payments are detected during the RRS review cycle or through other assurance activities are disputed, a working group (consisting of staff from the relevant areas in DESE) will review the disputes, come to a decision, and issue a final result to the provider. The reviewers are also provided with feedback as to how well they performed the task of reviewing to ensure review standards are maintained in accordance with the assessment handbook and the RRS manual. All of the disputed payments identified as non-compliant go through this additional review process.

2.55 Reports are prepared for all assurance activity results and tabled at PISCES meetings. The reports are signed off by the relevant Assistant Secretary in the program area and the Assistant Secretary of the Assurance Coordination Branch. This sign off process is a key quality assurance process.

2.56 Where claims for payments are found to be compliant, DESE has committed to selecting a sample of five per cent of these claims for review by peer reviewers, however in each of the analysed cycles, over 30 per cent of claims were reviewed. Table 2.4 below shows the percentage of claims initially assessed as compliant which were subsequently assessed through the Quality Assurance process.

Table 2.4: Compliant payments subject to Quality Assurance reviews

Source: ANAO analysis of DESE documentation.

2.57 Each of the assessors is required to complete a conflict of interest declaration as part of onboarding in line with the broader DESE Conflict of Interest Policy. These are centrally tracked through an Excel spreadsheet.

Has an appropriate monitoring framework been established?

2.58 DESE has established mechanisms to monitor provider compliance including the Quality Assurance Framework certification; RRS review; targeted program assurance projects of employment services; and data analytics to identify emerging risks.

2.59 The Assurance Coordination Branch has an annual business plan and reports achievements against the plan.

Jobactive Performance Framework

2.60 The jobactive Performance Framework is designed to monitor provider compliance with contractual obligations as well as their overall performance considering efficiency, effectiveness and quality. The quality is managed through the development and monitoring of Service Delivery Plans. Service Delivery Plans are developed by the provider and articulate how they will deliver the required services to DESE and will be monitored through the jobactive Performance Framework.

2.61 Undertaking the assurance activities detailed in paragraph 3.8 provides DESE with mechanisms to assess whether providers are complying with requirements set out in the agreements between DESE and the providers.

2.62 Since the commencement of the jobactive program in 2015, DESE has used a compliance indicator to measure the compliance of individual providers. The compliance indicator is calculated from the combined results of the previous 18 months of the RRS review, targeted activities, and contract management review outcomes.

2.63 A compliance indicator score of 95 or above out of 100 is considered to meet DESE’s target level of compliance. The average provider compliance indicator score in June 2021 was 95.3, with scores for individual providers ranging from 80.6 to 99.7. Recent compliance indicator results are in Table 2.5 below.

Table 2.5: Compliance indicator results

Source: ANAO analysis of DESE documentation.

2.64 Compliance indicators feed into an analysis of individual provider risk ratings. A detailed analysis is not undertaken for common themes that may lead to changes to provider risk ratings across a range of providers. DESE advised the June 2021 compliance indicator was the last result produced and stated:

During the pandemic, and the various enforced lockdowns across the country, it wasn’t possible to conduct the planned number of assurance activities required, and to have a sufficient number of transactions reviewed, to produce statistically robust Compliance Indicator results; (a certain number of reviews are required for each provider for each type of activity).

2.65 Across the Employment Services Assurance Framework, detailed analysis of trends across providers and identification of causes of issues identified is not undertaken across RRS cycle periods. This includes the RRS review, the quality assurance framework and the compliance indicator outcomes.

Employment Steering Committee

2.68 The ESC³⁰ is the leadership forum for DESE’s employment related programs and activities. ESC has been established to promote the principles of good governance, collaboration, accountability, integrity, efficiency and effectiveness, transparency and openness.

2.69 The ESC has oversight of all employment programs including the jobactive program. The ESC meets weekly. Membership for the ESC includes:

• Deputy Secretary, Employment and Workforce Group;
• Senior Responsible Officer, New Employment Services Model;
• First Assistant Secretaries, Employment and Workforce Group;
• First Assistant Secretary, Digital Services Division; and
• Chief Internal Auditor as an observer.

2.70 The ESC does not monitor the assurance activities undertaken in the assurance framework. Monitoring of assurance activities is undertaken through the PISCES which serves as a sub-committee to the ESC. Minutes of the PISCES are tabled at the ESC.

Program Integrity Subcommittee for Employment Services (PISCES)

2.71 The PISCES provides a high-level forum to support and advise ESC on matters relating to the risk profile and program integrity of all contracted employment services. The PISCES also oversees the implementation of the Assurance Strategy. The summary reporting of assurance outcomes to the PISCES is the key mechanism to monitor that the assurance framework is being adhered to.

2.72 The PISCES meets monthly. Membership includes:

• First Assistant Secretaries and Assistant Secretaries within the Employment and Workforce Group;
• State Managers;
• Chief Internal Auditor;
• Director of Enterprise Risk; and
• representatives from Department of Social Services, National Indigenous Australians Agency and other DESE representatives as observers.

2.73 The PISCES met seven times during the October 2020 to December 2021 period, and members considered 75 papers. The papers include summary reports of the various assurance activities and provide feedback on the assurance activities. Through review of subcommittee minutes, the ANAO identified that the PISCES monitored the progress and outcomes of assurance activities. These are noted as discussion in the minutes and specific action items have not been assigned. As such, the PISCES is not following up and monitoring whether the suggestions it has made to improve assurance activities have resulted in the intended outcomes. Some examples of the discussions in these meetings have been noted in Table 2.6 below.

Table 2.6: PISCES assurance activity discussion

Source: ANAO analysis of DESE documentation.

2.74 The assurance activity information provided to the PISCES was a summary. The ANAO suggests that DESE consider providing detailed analysis of the cause of compliance issues. The PISCES should also consider assigning action items to enable monitoring of all its suggestions around assurance activities and to support better governance over program integrity activities.

3.1 The effective implementation of an assurance framework enables DESE to manage and monitor its payments to contracted employment service providers (providers) appropriately.

3.2 To determine whether jobactive payments are assessed, processed and paid in accordance with the framework, the ANAO examined whether:

• assurance activities are conducted in accordance with the framework;
• IT systems are designed and implemented to ensure automated processes in the framework are effective;
• monitoring activities are effective at managing the assurance framework; and
• non-compliance is managed effectively.

Are assurance activities conducted in accordance with the framework?

3.3 The Assurance Strategy and Assurance Framework (see paragraphs 2.12 to 2.13) details the Assurance Planning Process and the development of the assurance plans for each cycle. The key components of DESE’s assurance framework as summarised in Figure 3.1 below.

Figure 3.1: Assurance activities of the Assurance Strategy and Assurance Framework

Source: ANAO analysis of DESE documentation.

3.4 Under the Assurance Strategy and Assurance Framework, assurance plans are developed every year to outline the assurance activities that will be undertaken. The dates for the planning process have differed slightly over time. The 2019–20 plan was from March 2019 to March 2020, and the 2020–21 version covered the period from March 2020 to June 2021 to align planning to financial years in the future.

3.5 The planning workbooks provide a detailed analysis of the jobactive program, program risks and controls, existing management oversight and assurance activities resulting in recommendations for the assurance approach. The workbooks include:

• program overview including program annual budget, historical transaction volumes, program changes since the last review point;
• the impact of the COVID-19 pandemic on the program;
• documentation and evidence requirements for claim for payments that providers must have when submitting claims for payment;
• program operations, IT systems and related controls;
• ongoing and ad-hoc assurance activities;
• an update of the assurance activities recommended in the 2019–20 plan;
• program risk profile; and
• analysis and recommended assurance activities including rationale for recommendation, alignment to risk events, timing and resources.

3.6 The assurance plans summarise the information developed in the workbooks. This provides an overview of 2021–22 program assurance activities for the jobactive program as discussed in paragraphs 2.14 to 2.19.

3.7 The 2021–22 assurance plans differ from the 2020–21 plans as they cover specific assurance activities to mitigate risks associated with the COVID-19 pandemic.

Implementation of Assurance activities

3.8 The assurance activities that were undertaken in 2019–20 and 2020–21 as they relate to jobactive payments are set out in Table 3.1 below. The ANAO assessed the assurance framework for the jobactive program and determined that activities included preventive, corrective and detective activities.³¹ As discussed in Chapter 2, a range of assurance activities are undertaken to assess payment integrity. The assurance activities related to jobactive payments in 2019–2021 are detailed in Table 3.1 below.

Table 3.1: Jobactive 2019–2021 payment related assurance activities

Source: ANAO analysis of DESE documentation.

3.9 The dates covered for each RRS cycle are outlined in Table 3.2 below.

Table 3.2: Period for RRS Cycle

Note: Some transactions in cycle 16 were completed at a later date. These were reported as one cycle when completed.

Source: ANAO analysis of DESE documentation.

3.10 Interim results were completed in December 2021 for cycle 18.

3.11 Figure 2.2 in Chapter 2 above outlines the steps involved in an RRS review cycle. Reviews can take several months to complete. The samples are large and DESE advised the ANAO that reviewing transactions is time consuming. Once the transaction review is complete there are additional steps that add to the time taken, including:

• providers are sent their claims initially assessed as non-compliant to obtain additional information and are given ten days to respond; and
• disputes are assessed and final results letters are sent to the providers that are assessed as non-compliant, which can take a further two weeks.

3.12 DESE also advised that the length of time has been compounded since the start of the COVID-19 pandemic with provider staff working from home. The longer time to receive responses has been acknowledged by the PISCES.

3.13 The results of all the assurance activities are reported to the PISCES.

Rolling Random Sample Cycles

3.14 As discussed in paragraphs 2.29 to 2.41, the RRS review is the main assurance activity that relates to payment integrity. The sampling methodology used selects payments from all providers except two.³²

3.15 The RRS review cycles commenced in 2015 at the beginning of the jobactive program. Except for RRS cycle 16 that was delayed due to the impacts of COVID-19, RRS cycle reviews have been consistently performed three times each financial year.

3.16 RRS review assessors use a workbook in Microsoft Excel to assess provider claims. The workbook is populated with the details of jobactive payments and checked against information and evidence submitted by providers.

3.17 If an assessor requires additional evidence to complete the assessment, DESE will email the provider to ask that the additional evidence be uploaded within one day. Assessment outcomes are also recorded in the workbook.

3.18 The outcome of an assessment is based on the answers to the questions that the assessor completes in the assessment tool. The assessor classifies the transaction as satisfies requirements; requirements mostly satisfied; requirement partially met (demerit); or requirements not met (recovery). Assessment officers can overwrite the assessment determined by the Assessment Tool if appropriate to the circumstance of that transaction. In some instances when there is systemic non-compliance or suspected fraud, breach actions are referred for further investigation by the assessor.

3.19 Table 3.3 below sets out the assessment results for jobactive claims sampled by the RRS cycles 9 to 17.³³ A compliance indicator score of 95 and above is considered to meet DESE’s target level of compliance with the jobactive Deed 2015–2022 (the Deed) and guidelines. As discussed in paragraph 2.62, the RRS cycle results are an input to the compliance indicator. Evidentiary requirements were assessed by DESE as compliant for more than 95 per cent of claims tested for RRS cycles 9 to 17 with an average compliance rate of 96.5 per cent, with results ranging from 95.6 per cent to 97.8 per cent per RRS cycle.

Table 3.3: Assessment outcomes for jobactive claims 2017–2021

Note: The period from March to July 2020, which was impacted by the COVID-19 pandemic, was included in cycle 17.

Note: Recoveries are sought when transactions are classified as ‘not met’.

Source: DESE documentation.

3.20 A ‘Final Results Minute’ is prepared to outline outcomes of each cycle and noted by the PISCES. The results are discussed in the PISCES meetings as described in paragraphs 2.71 to 2.74. DESE has undertaken analysis on each cycle’s results that is also included in the Final Results Minute. This included identifying providers whose compliance indicator has dropped or identifying trends across providers within that cycle. This analysis does not include detailed analysis of the cause of non-compliance and is not used for continuous improvement or future assurance activities.

3.21 The RRS review cycles have been undertaken as stipulated in the assurance framework other than a period when they were suspended due to the pandemic. As a result of economic conditions during the COVID-19 pandemic a higher number of new participants entered the program. DESE advised that this put pressure on providers from a capacity perspective of having to service more participants. A decision was made to exclude a specific period in RRS cycle 16 to take pressure off providers as they managed their increased workloads. This was endorsed by the PISCES. These transactions were tested later and completed on 29 January 2021.

Are IT systems designed and implemented to ensure automated processes in the framework are effective?

3.22 Services Australia assesses the circumstances and eligibility for jobactive and refers participants to a provider. Services Australia provides data to DESE on participants who are eligible for jobactive.

3.23 Under participant reporting requirements, Services Australia collects payroll data that can also be used to determine employment outcomes under jobactive. This data is transferred from Services Australia to DESE and uploaded into the Employment Services System (ESSWeb).

3.24 The following partnership arrangements have been established which outline responsibilities in relation to the availability and reliability of this source data.

• Letter of Agreement between DESE and the Department of Social Services for the period 1 July 2021 to 30 June 2024.
• Statement of Intent between DESE and Services Australia dated 23 July 2021.
• Letter of Agreement between DESE and the National Indigenous Australians Agency for the period 1 July 2021 to 30 June 2024.

3.25 A Bilateral Management Committee (BMC) peak governance body exists to govern the arrangements between Services Australia and DESE. Membership is at Deputy Secretary/Deputy CEO (SES Band 3) level. The BMC meets quarterly.³⁴

3.26 As part of the year end processes, Services Australia provides DESE with a financial statement assurance certification attesting to the financial assurance processes undertaken relating to the financial information provided by Services Australia to DESE.

3.27 There is also an Employment Services Governance Committee that meets quarterly that provides strategic oversight of the bilateral agenda related to the employment program and assists both parties to monitor their performance and meet their obligations under the bilateral agreement. Membership is at SES Band 2 and Band 1 levels from both Services Australia and DESE. Reporting and the escalation of any unresolved issues or disputes goes to the BMC for consideration and resolution, as required.

3.28 The IT system that supports the administration of the jobactive program is ESSWeb. ESSWeb is a workflow system that is used for several employment programs to administer the programs and process the related payments.

3.29 ESSWeb interfaces with the Department of Finance system called HUB. HUB is a SAP platform administered by the Service Delivery Office within the Department of Finance. Through an automated interface, ESS financial transactions are uploaded into HUB and payments are processed within HUB to deposit payments into provider bank accounts.

3.30 ESSWeb also has an internet portal which allows providers to upload and access information including submitting claims for payments.

3.31 All jobactive transactions are stored in an SQL database³⁵ which acts as the database to support ESSWeb. Transactions within the database are used for various reporting and analysis purposes and are also used as the population source for the selection of the samples for the RRS reviews.

3.32 A partitioned section of this database is used to store the data related to the RRS reviews. The sampled transactions are stored separate from the broader population of data so they can be accessed. Procedural documents or standard operating procedures have not been developed for the extraction of the sample data from the population data and moving this into the partitioned area. The ANAO suggests DESE consider developing procedural documentation to support this process, so the process is repeatable in the event of sudden staff absence or departure.

3.33 ESSWeb and the SQL database have sufficient IT general controls related to access and security, IT change and release management, and backup and recovery. This includes restricting access to transactions and information to the appropriate users both within DESE and at providers. Controls ensure that when changes are made to the systems they are adequately tested before release to confirm the changes work as required and do not adversely impact the system and the information related to the change including approvals and testing are documented. Arrangements to ensure the system can be recovered and operating as required in the event of a disruption event have been regularly tested.

3.34 Auditor-General Report No.8 2021–22 Use and Administration of Wage Subsidies reported that the IT systems and controls considered during the audit were operating effectively. The systems, processes, and controls relevant to the processing of wage subsidy payments are consistent with those applicable to the processing of provider payments.

Are monitoring activities effective at managing the assurance framework?

3.35 A risk-based approach to compliance monitoring activities, including random sampling of payments, supports DESE to identify non-compliance.

3.36 Payment management and monitoring activities were reviewed by the ANAO for the period July 2019 to March 2021. Instances of non-compliance were identified throughout this period by DESE. The ANAO also reviewed the reporting of assurance activity results dating back to July 2017. Since July 2017, DESE reported that the RRS review process has identified 198 outcome payments where outcomes were not met and recovered $367,903. Non-compliance was also identified by other assurance mechanisms as discussed in paragraph 3.44.

3.37 Non-compliance for a small number of providers has been recurring, with providers found to be non-compliant across multiple cycles. DESE provides communication to all providers after each RRS cycle on where deficiencies were identified. For example, in cycle 15 the most common deficiencies were with the documentary evidence not matching the reported earnings; permissible breaks³⁶ not meeting requirements; evidence being uploaded after the claim was lodged; and the evidence uploaded containing payslip or payroll deficiencies. Similar issues did not occur in cycle 16.

3.38 The mechanisms for identifying instances of non-compliance are used to update risk ratings for providers. As identified in Table 3.3 above, non-compliance has been less than 3.5 per cent on average with an average compliance rate of 96.5 per cent.

Assessment of the Assurance Strategy and Framework

3.39 In Auditor-General Report No.4 2017–18 Jobactive: Design and Monitoring, the ANAO identified that an external consultant was engaged to undertake a review of DESE’s Assurance Strategy to align DESE’s current approach with better practice, with the aim of focusing resources on areas of highest priority. The ANAO considered that DESE should:

• assess if the delivery of the Assurance Strategy reflects DESE’s preferred level of compliance;
• effectively guide the allocation of departmental resources towards areas of the highest risk; and
• consider the level of compliance burden on providers.

3.40 The report also included a recommendation that DESE assess whether the current compliance regime was structured to, effectively and efficiently, detect and manage non-compliance, and adjust as appropriate.

3.41 Changes to the assurance framework were finalised in early 2018 in response to the previous ANAO recommendations and the above recommendation was closed by the DESE Audit and Assurance Committee.

3.42 DESE has not undertaken analysis to assess the effectiveness of assurance processes and to determine the efficiency of processes. In addition, an assessment of aligning resources to resource activities to highest risk mitigations has not been completed.

3.43 In late 2021, DESE began an approach to market to engage an actuary to undertake detailed analysis to review the assurance activities. The ANAO was advised this engagement would begin in December 2021. DESE subsequently advised the ANAO that this process commenced in March 2022. DESE advised Taylor Fry was engaged in March 2022 to provide advice on the Rolling Random Sample and was expected to return their advice in late April 2022.

Is non-compliance managed effectively?

3.44 Non-compliance is identified through a range of processes including the RRS review, desktop monitoring, contract management, tip-offs and other processes. Non-compliance is not limited to claims for payment and can include non-compliance with other parts of the Deed including privacy, conflict of interest, Occupational Health and Safety, and IT security. Non-compliance in the first instance is managed through contract management processes. In some instances, non-compliance can escalate to a breach of the Deed.

3.45 DESE has established a Breach Management Framework (BMF) to provide guidance on investigating non-compliance where initial action to manage the non-compliance has been ineffective or the non-compliance is systemic or suspected fraud and determining if it is a breach.

3.46 The breach management process incorporates the following six steps.

• Step 1 — Potential Breach Identified.
• Step 2 — Assess Seriousness of Breach.
• Step 3 — Determine Initial Action.
• Step 4 — Confirm and Rectify.
• Step 5 — Determine Final Actions.
• Step 6 — Ongoing Contract Management.

3.47 Through these steps, the BMF outlines how to identify and record a breach, assess risk, determine remedial actions, and monitor compliance in relation to claim for payments. The BMF is also used for other breaches such as privacy breaches. The Breach Management Team will create a record of the potential breach by creating an entry within the system with links to relevant information. In most instances funds are recovered through offsetting against future payments. In some circumstances recoveries cannot be offset against future payments as the provider will not be receiving future payments. In these circumstances, recovery actions are undertaken to seek repayment.

3.48 Instances of non-compliant claims are mostly identified through the RRS process and other assurance activities as described in Table 3.1. The majority of recoveries reported to PISCES are through the RRS. Table 3.4 below sets out the total recoveries made as a result of the RRS as advised by DESE.

Table 3.4: Jobactive employment outcomes financial recoveries from the RRS

Note: The period from March to July 2020, which was impacted by the COVID-19 pandemic, was included in cycle 17.

Note: Most recoveries are recovered by offsetting future payments.

Source: DESE documentation.

3.49 On 29 April 2022, DESE advised that in the period 1 January 2017 to 30 June 2020, $2,503,160.04 had been recovered from a combination of program assurance activities and actions by the contract manager. From cycle 9 in 2017 until cycle 17 in July 2020, non-compliance issues from RRS identified non-compliance have resulted in recoveries of $367,903.

3.50 In addition to financial recoveries, DESE provides written feedback to providers in respect of claims that were assessed as part of the RRS. The feedback sets out the overall results for jobactive claims assessed in the relevant RRS cycle, a list of the most common deficiencies, and learnings for providers including outcomes of managed non-compliance.

3.51 The focus of reporting is on the percentage of non-compliant transactions and the percentage value of non-compliant transactions is not reported. As shown in Table 3.4, the percentage of value of non-compliance is consistently higher than the percentage of transactions. The RRS value of non-compliance transactions equates to 3.8 per cent of the total value of the transactions sampled through all cycles reviewed.³⁷ The ANAO suggests that DESE evaluate the value of non-compliance when considering assurance activity results.

3.52 The outcomes of the recoveries are reported in the Employment Services and Risk Report which is reported every six months. The outcomes of the assurance activities listed in Table 3.1 above are also presented in this report as this forms the output for assurance activities as detailed in Table 2.2 in Chapter 2 above. The outcomes of assurance activities are outlined in Table 3.5 below. These cover all employment programs and not just jobactive. Across all employment programs privacy breaches represent the majority.

Table 3.5: Breach Management Framework cases across all employment programs in the examined period and since 1 July 2015

Source: DESE documentation.

3.53 As discussed in paragraph 3.45, non-compliance can be raised as a potential breach through the BMF. Payment non-compliance however is rarely escalated, and recovery is generally managed through DESE contract management processes. The BMF is mostly used for non-claim breaches such as privacy.

3.54 Table 3.6 below sets out the jobactive breach cases initiated each year since the inception of the program.

Table 3.6: Jobactive Breach Management Framework cases

Source: DESE documentation.

3.55 Of the three claim breaches, two were reported by job seekers and the other was identified through targeted assurance following a tip-off. No claim breaches were identified through the RRS process. A total of 11 breaches were referred as fraud investigation, four of which were referred directly by the provider, six by job seekers and the remaining one was identified through targeted assurance following a tip-off.

Appendix 1 Entity response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny can promote improved performance. Improvements in administrative and management practices can occur in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021–22 Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions, and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.