Introduction

1.1 Established in 1948, the Adult Migrant English Program (AMEP) provides free English tuition to eligible migrants and humanitarian entrants with no or low English levels.⁹ The program supports an average of 53,000 participants annually (see Appendix 3). The design of the program recognises that learning English can help migrant settlement.

1.2 Contracted delivery of the Adult Migrant English Program (AMEP) has been in place for 25 years.¹⁰ The current contracts commenced in 2017 with AMEP providers contracted in each state and territory, delivering English language lessons in over 300 locations and online, to clients in metropolitan, regional and remote locations in Australia.¹¹ Data published on AusTender initially valued the 15 contracts¹² with 14 providers¹³ at $1.22 billion which had increased by 75 per cent to $2.15 billion as at February 2024, representing an average annual contract value of $287 million. There is also a contract for quality assurance, for which the reported value increased from $6.15 million to $22.52 million.

1.3 A 2019 Investing in Refugees Investing in Australia report¹⁴ found that the program had unacceptably poor results due to too few participants (seven per cent) achieving functional levels of English at the conclusion of their course. The report also found a significant underutilisation of the program, with many participants not completing the hours they had available.

1.4 The COVID-19 pandemic impacted on the delivery of the AMEP by the general service providers, given that the contracts involved face-to-face tuition. The COVID-19 pandemic also impacted on the delivery of onsite assessments by the quality assurance provider.

1.5 In 2021, changes were made to the program. They allowed eligible migrants access to unlimited hours of lessons by removing the previous 510 hour limit; and increased alignment with the AMEP’s partner program, Skills for Education and Employment (SEE) by increasing the standard of eligibility and attainment from ‘functional’ to ‘vocational’ English.¹⁵ The October 2022 Budget included $20 million to provide more flexible delivery options for the program and to increase case management support for students (to deliver on an election commitment).

Administrative responsibility for the program

1.6 Administration of the AMEP contracts transferred into the department from the Department of Education and Training through Machinery of Government changes in July 2019. The procurement process that resulted in the current AMEP contracts was conducted when the program was the responsibility of the then Department of Education and Training (the SEE program is now administered by the Department of Employment and Workplace Relations).

1.7 Changes made since 2019 by the Department of Employment and Workplace Relations to the head agreements that apply to both AMEP and SEE have occurred through engagement with, and input from, the Department of Home Affairs. This is consistent with a ‘Collaborative Agreement’ signed by the two departments in October 2019 under which Home Affairs agreed that the department responsible for the SEE program would be the lead agency. The Collaborative Agreement requires that any variations to the general clauses in the contracts must be agreed by both departments. The two departments have the ability to make variations to their respective schedules within the contracts, and are to inform each other as a matter of courtesy of any variations to their respective parts of the contracts.

Procurement process to replace the contracts

1.8 All contracts were due to end on 30 June 2023, with the request for tender for new contracts to be released on 30 April 2022. The contracts were extended in December 2022 to 30 June 2024 (with work orders under the contracts not due to expire until 31 December 2024), after a government decision to delay implementation of a further new AMEP business model.¹⁶ The request for tender for new AMEP contracts was to be issued by September 2023, and new arrangements to commence 1 January 2025. On 30 November 2023, the Minister for Immigration, Citizenship and Multicultural Affairs approved a further delay¹⁷, with no new release date advised. In February 2024, (22 months after the request for tender was to have been released) the department advised the ANAO that advice to the minister and a new policy proposal were in development and the release date of the request for tender is dependent on government agreement to a new model and implementation date.

Rationale for undertaking the audit

1.9 Contracts for the delivery and quality assurance of delivery of AMEP are valued at over $2.15 billion and will have been in place for at least seven and a half years by the time they are replaced. The ANAO has audited the management of AMEP contracts once previously (in 2001¹⁸), with the audit report including six recommendations, all of which were agreed to. Those recommendations related to improving program performance management and reporting; strategic management and coordination; management of financial risk; and monitoring of contractor performance.

1.10 The audit provides assurance to the Parliament that the department is appropriately administering the Adult Migrant English Program contracts.

Audit approach

Audit objective, criteria and scope

1.11 The objective of the audit was to assess whether the design and administration of AMEP is effective.

1.12 To form a conclusion against the objective, the following high-level criteria were applied:

• Are appropriate contractual arrangements in place?
• Are the service provider contracts appropriately managed?
• Are contracted quality assurance services being delivered to an appropriate standard?

1.13 This audit examined the design and administration of the AMEP contracts entered into in 2017, including associated variations and work orders and preparation and transition planning for the program post-30 June 2024.

1.14 The ANAO did not audit the procurement processes for establishing the AMEP contracts.

1.15 In addition to auditing the management of contracts by the Department of Home Affairs, the audit used the follow the money power provided under paragraph 18B(1)(b) of the Auditor-General Act 1997 (the Act) to examine the performance of Linda Wyse and Associates (LWA), the quality assurance provider for the AMEP.

Audit methodology

1.16 The audit method included:

• examination of records;
• meetings with staff; and
• analysis of reports from information systems.

1.17 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $583,000.

1.18 The team members for this audit were Hannah Conway, Tamara Duncan, Amita Robinson, Calli Stewart and Brian Boyd.

2.1 The adoption of appropriate contractual arrangements is an essential underpinning to achieving the outcomes envisaged through a procurement process. The Department of Finance’s Contract Management Guide states that ‘the aim of contract management is to ensure that all parties meet their obligations to deliver the objectives of the contract’ such as ‘ensuring the goods or services purchased are provided on time, to the agreed standard, at the agreed location and for the agreed price’.

Do service provider contracts clearly outline contracted deliverables?

2.2 The deeds of standings offers executed with each of the 13 general service providers includes a chapter outlining the specifics of the services; a chapter outlining the performance requirements and expectations; another chapter outlining the fees to be charged for each of the AMEP services; and a chapter specifying the contract regions (see footnote 11 and paragraph 3.13) the provider is to deliver the AMEP services.

2.3 The deeds of standing offers with each of the general service providers do not require the department to seek quotes from the providers before entering into a work order under the deed of standing offer. Instead, the standing offers allows the Commonwealth to issue work orders on demand.¹⁹ The deeds with each provider state that ‘The Contractor … irrevocably offers to provide the Commonwealth with the Services as and when the Commonwealth requires those Services’ and that the Contractor agrees to provide the Services ‘for the Fees; within the Contract Regions; from the Sites; and on the conditions’ as set out in the deed.

2.4 The AMEP General Service Provider Instructions (SPIs) are intended ‘to support and clarify the delivery arrangements for the Adult Migrant English Program (AMEP) service providers’. The information in the SPIs usually go to the detail of operationalising the contract, such as providing details on how to enrol, how to use the IT system, what is to be included in reports, timeframes for enrolment and reports. The SPIs were in place from the start of contracts (1 July 2017) to 30 June 2022. Version 11 of the SPIs was released on 5 May 2023 (with an end date of 31 December 2024). As such no SPI was in place for 10 months.

2.5 No one document contains all the AMEP general service providers obligations. The SPIs note that they should be read in conjunction with ‘supplementary tier documents’, ‘your contract and any other relevant reference material issued by the department’. This lack of clarity in where the provider obligations may be recorded was partially addressed in the March 2021 update to the SPIs where the department specified that the SPIs provide ‘supplementary administrative advice and should be read in conjunction with your Deed of Agreement, Work Order, Administrative Advice and all other relevant advice and/or reference material issued by the Department from time to time.’ It is standard practice in the administration of the AMEP for the department to issue correspondence, Communiques and Administrative Advices via email to all providers, or upload to a shared secure site for providers to locate.²⁰

2.6 The version control sections across the 11 versions of the SPIs (most recent update was made in May 2023) note that updates often incorporate changes made to the administration of the program through communication with providers (through Administrative Advices, Communiques and correspondence). Similarly, some variations executed to the deeds of standing offer are made to incorporate changes made to the General Service Provider Instructions.

2.7 The Department of Finance’s Contract Management Guide, notes in section 2.3, that some communications with suppliers ‘can result in a variation of the contract or a waiving of your entity’s rights under the contract’ and recommends that the entity keep ‘note of discussions or agreements and email these to the supplier to avoid uncertainty regarding the discussion, minimise the risk of dispute and ensure you both have a record’. The department only commenced maintaining a register²¹ of such communications in 2020.²²

Were variations to the contracts appropriate, including adequately demonstrating value for money?

2.8 The contracts resulted from a combined procurement process for the AMEP and Skills for Education and Employment Program (SEE) programs.²³ The contracts with the general service providers, as well as the contract with the quality assurance provider, include separate parts specific to each program, as well as parts that apply to both programs.

Extent of variations

2.9 The Department of Finance’s Contract Management Guide advises that:

• ‘The variation process you should follow is usually in the contract’ and that the administering department should ‘document the variation in writing and ensure both parties sign as evidence of agreement to the change (usually through a Deed of Variation)’; and
• the administering department should have, for longer term or more complex contracts, a ‘master version’ of the contract that includes all variations in track changes.

2.10 The extent to which the contracts are varied can indicate that a contract does not provide an appropriate basis for the management and delivery of a program.

2.11 The first AMEP variation was approved in December 2017, less than six months into the contract term, with a total of seven variations (usually referred to as variation numbers two, three, five, nine, 10 and 11) made to clauses that impact on delivery of AMEP. In proceeding with the variation approved in December 2017, the department recorded in April 2018 that:

Since the commencement of the new AMEP contracts on 1 July 2017, the department and the service providers identified contractual inconsistencies, practical problems implementing the new business model and difficulties managing program with the administratively burdensome contingency IT system.

2.12 In addition to the variations specific to AMEP for all service providers, four service providers have had specific variations made to their contracts, as follows:

• in December 2018, a provider’s contract was varied²⁴to incorporate an additional contract region²⁵;
• in June 2022, the contract for one provider (and three previous deeds of variation) was varied²⁶ to change the Australian Business Number to the correct number;
• in July 2023, the contract for one provider was varied to change the fees payable; and
• in December 2023, the contract for another provider was varied²⁷ to change the fees payable.

2.13 Each of the service provider contracts has also been affected by a variation to the general contract terms (that apply to both AMEP and SEE) and a variation to extend the duration of each contract. These variations were undertaken by the department that administers the SEE program.

2.14 The contract with the AMEP quality assurance provider has been similarly subject to numerous variations, with ten deeds of variations executed. Of note:

• Deed of Variation 2 from April 2018 could not be located in departmental records²⁸;
• an ‘in principle agreement’ to vary fees was effected²⁹ via a letter of offer and acceptance, not through a deed of variation;
• two variations³⁰ are titled ‘Deed of Variation 6’, one being executed in February 2021 and another in December 2021 (Deed of Variation 7 was executed earlier in December 2021); and
• Deed of Variation 9 (which was actually the tenth variation) from July 2023³¹ states it is only amending the SEE Part yet varies Part 1 of the whole agreement which impacts both programs.

2.15 Home Affairs has not maintained adequate records of variations to the contracts. For example, in December 2020 (Deed of Variation 4) all contracts for the general service providers were varied to adjust the fees for the providers. Prior to this variation, fees had been varied for four service providers through correspondence and so not reflected as variations in each contract’s variation schedule. These variations through correspondence are not recorded in a contract register, and there is no central repository of them. Some of the fee increases were agreed to through correspondence by the department because of the financial situation of the provider, without the department examining the implications in terms of the competitive procurement process that had concluded that the contractor offered the best value for money for particular contract regions. For example:

• a general service provider was party to two variations specific to its contract:
  • On 28 June 2019, the department agreed through correspondence to increase the fees for the provider effective from 1 January 2019, including backdating fees for a six-month period.³² The reason the provider sought an increase was due to the implementation of a new enterprise agreement for its staff as of 1 May 2018. The department decided to increase the fee as the ‘change to the [enterprise agreement] was unforeseen, outside [the provider’s] contract and is real in terms of the impact on their organisation and their sub-contractors. The [enterprise agreement] was not in place at the time [the provider] tendered to deliver the AMEP and the SEE program.’
  • In December 2019, the same provider again requested a fee increase for financial years 2020–21 and 2021–22. The request was reiterated on 24 June 2020. The justification for the fee increase was the same as previously, ‘unanticipated costs’ related to staffing.³³
• in December 2018 the department agreed through correspondence to increase the child care fee paid³⁴ to a different general service provider, and backdate payment of the child care administrative fee to 1 July 2018.³⁵ The provider had advised the department that its ‘new child care model, which commenced on 1 July 2017, was not working well and [the provider] could no longer support the losses associated with the program.’ The agreement through correspondence was to cease on 30 June 2020.³⁶ The Quarter 1 Invoice for the 2020–21 financial year for this provider continued the varied child care administration fee, despite the agreement ceasing, and applied the Australian Bureau of Statistics (ABS) Wage Price Index to the unit fee, increasing the fee for the 2020–21 financial year.³⁷ In December 2020, a deed of variation was executed³⁸ reestablishing the fee schedules for all providers.
• in July 2018, a different general service provider sought an increase to its fees for the 2018–19 financial year to ‘cover ongoing administration costs associated with the increase in AMEP expenses’. Some seven months later, in February 2019, the department agreed through correspondence to increase fees and for this to be backdated to 1 July 2018. This higher rate was to expire on 30 June 2020.³⁹ In 2019 the department recorded that this fee increase stemmed from ‘concerns about the financial viability of the small service provider delivering in regional and remote contract regions and the administrative burden of delivering the AMEP, which was not well quantified at the time of the tender’⁴⁰ For the 2019–20 financial year, the fees agreed through correspondence were indexed, and increased by the ABS Wage Price Index of 2.3 per cent. Despite the agreed fees expiring on 30 June 2020, for the Quarter 1 invoices for the 2020–21 financial year, the department⁴¹ continued to apply the fees agreed through correspondence with a further index applied. In December 2020, a deed of variation was executed⁴² reestablishing the fee schedules for all providers, and incorporated this expired fee (index annually) in the varied fee schedule.
• in April 2018 the department agreed through correspondence to increase all fees⁴³ for another provider in one of its allocated contract regions, including backdating payments for the revised fees to July 2017, as student participation numbers were lower than the provider anticipated, threatening its ability to continue to deliver services. A further agreement was made for the 2018–19 financial year⁴⁴, and again in June 2019 for the 2019–20 financial year.⁴⁵ There is no evidence of agreement for the 2020–21 financial year.⁴⁶ For the Quarter 1 invoice for the 2020–21 financial year, the department had continued to apply the child care administration fee and supplementary program administration fee as agreed in the correspondence (and indexed each year) despite the agreement for these fee increases expiring. In December 2020, a deed of variation was executed⁴⁷ reestablishing the fee schedules for all providers.

2.16 In February 2024 the department advised ANAO that it ‘does not agree’ that Home Affairs has not maintained adequate records, rather that ‘it maintains a record of the variation that is complete to the best of its knowledge’⁴⁸, and that ‘the record may not encompass all variations, particularly those undertaken via correspondence, prior to Machinery of Government changes.’ ANAO notes that such limitations to the records means that they are not adequate.

2.17 The extent to which the contracts with the general service providers have been varied indicates that those contracts did not provide an appropriate basis for the management and delivery of the program.

Considering value for money

2.21 The Department of Finance’s Contract Management Guide advises that ‘an entity should not seek or allow a contract variation where it would amount to a significant change to the contract or significantly vary the scope of the contract’ if ‘other potential suppliers may have responded differently to an amended contract scope … which may have produced a different value for money outcome’ or if it would ‘compromise the original procurement’s value for money assessment.’

2.22 Home Affairs’ records of each of the decisions to vary the contracts do not clearly record that value for money was considered and therefore do not demonstrate that each of the variations has been appropriate.

2.23 For example, the first variation to the general service provider contracts (see paragraph 2.10) added services additional to those included in the approach to the market that resulted in the contracts being signed. Specifically, as recorded by the department, the ‘original contract did not make provision for Innovative Projects’ with Home Affairs advising the ANAO in April 2024 that this was a ‘drafting oversight’ at the time the tender was conducted. The contracts were varied to include a provision allowing for the conduct of select tender procurement processes for the delivery of additional services (‘innovative projects’). As of February 2024, ANAO has identified 53 innovative projects in departmental records (see paragraphs 2.35 to 2.42, Figure 2.1 and paragraphs 4.65 to 4.67) for a total budgeted cost of $5.94 million (excluding GST).

2.24 Other significant changes made in that variation related to:

• changes to the contracted key performance indicators (see paragraphs 3.11 to 3.19);
• changing the frequency of reporting from quarterly to six-monthly;
• allowing providers to seek exemptions from the department to engage teachers who do not meet the qualifications specified in the contract;
• changing the way child care travel payments were to be calculated and paid (to be formula based rather than based on records submitted by providers); and
• elevating from the General Service Provider Instructions to the contract provision for ‘blended’ classes (where providers with low student numbers may have students from both the pre-employment and social AMEP streams in the same class). This change was made on the advice of the department’s legal team which noted that ‘it is more appropriate to have the provider’s obligations set out on the face of the Contract rather than in the SPIs [Service Provider Instructions]’.

2.25 The brief to the approval delegate within the department did not discuss value for money implications of each of these changes either individually or in totality.

2.26 Another example related to changes made in July 2020, to come into effect in April 2020 (three months before the approval) in response to changes to support virtual tuition modes where face-to-face tuition was not possible due to the COVID-19 pandemic and social distancing measures. The changes included:

• detailing new services (being two new modes of tuition);
• agreement to a ‘One Off Transitional Cost’;
• development and implementation of ‘strategies to encourage adoption of Virtual Participation’;
• development and implementation of ‘a strategy that allows for AMEP clients to engage in a variety of ways most suitable to their circumstances’; and
• ‘Surveying other means of participation with AMEP Clients to determine their existing IT capability’.

2.27 While the department’s approval briefing for the changes discussed various ‘issues’, it did not explicitly address whether the changes (already in place for three months, see paragraph 1.4) represented value for money. The brief sought approval ‘to minimise the risk of 2019–20 program expenditure’ (see Appendix 5 paragraphs 10–12) being ‘recorded against the 2020–21 budget’ and:

A key consideration for desired outcomes is the Departmental budget for the AMEP. The budget model for the program is demand driven and normally increases and decreases according to the number of eligible visas granted. Under normal circumstances the reduction in arrivals from the closure of the Australian border on 15 March 2020 would have resulted in a proportional reduction in overall expenditure. However, the increased cost of delivery means that the reduced client inflow has been partially offset by the increased cost of delivery in the COVID-19 context.

2.28 The changes and associated fees were not incorporated into a deed of variation. Instead, the contract was varied to add a clause stating that Virtual Participation Fees ‘shall be set out in the AMEP SPIs and any applicable Work Order’. A work order was issued, which included a fee table, with fees specific to alternative tuition modes (see Appendix 5 paragraphs 10–12). The department’s approach to including and varying fees and services was inconsistent with the contract and also meant that, in a further contradiction of the contract terms, subordinate documentation, being the General Service Provider Instructions and work orders, was given priority over the contract deed (see Appendix 5 paragraphs 10–12).

2.29 Value for money was also not adequately addressed by Home Affairs in its decision making for the fourth deed of variation. This variation had two key impacts on fees. Firstly, adjusting the fees of all providers to incorporate variations made through correspondence⁴⁹ (see paragraph 2.15), and secondly, the removal of the ‘supplementary program administration fee’ on 31 December 2020. This fee was agreed to be paid until a new information management system was introduced.⁵⁰ When the administration of the AMEP program was moved to Department of Home Affairs in 2019, no new information management system had been introduced. Between 1 July 2017 and 30 June 2021, providers were paid $10.29 million in supplementary program administration fees. Neither the initial brief advising of the issue, nor the brief seeking execution of the deed of variation noted that all fees for all providers were also being varied to align with variations made through correspondence, and did not include a value for money assessment of the changes. Similarly, the department’s approval of an April 2021 variation did not adequately address value for money considerations. The variation was necessary as a result of legislative amendments that:

• removed ‘the 510 hour cap that currently limits free English tuition’ for AMEP students;
• raised ‘the AMEP eligibility threshold (and exit point from the program) from functional to vocational English; and
• removed ‘the time limit for registering, commencing and completing AMEP tuition (for those already in Australia as at 1 October 2020).’

2.30 The department’s variation approval briefing described these variations as ‘administrative only with no anticipated financial implications for the current contracts’. Inconsistent with this statement, removing the time limit on participation in the program (when providers are paid on hours of tuition) and extending the eligibility threshold from functional to vocational English can reasonably be expected to have financial implications.

Have contracted advisers on the program been appropriately engaged?

2.33 The Department of Finance’s Contract Management Guide notes that specialist advisers, such as procurement specialists, technical and operational specialists, and legal advisers, may be used to assist with contract management activities. The Department of Home Affairs’ engagement of advisers to assist with AMEP is required to be consistent with the Commonwealth Procurement Rules, including employing procurement processes that encourage competition and demonstrate the achievement of value for money.

2.34 The CPRs also require that officials maintain appropriate records of the conduct of procurement processes. This includes records that outline the requirement for the procurement, the process that was followed, how value for money was considered and achieved and relevant decisions and the basis of those decisions. As outlined in Appendix 4, for the eight procurements examined by the ANAO:

• there was no competition for five procurements⁵¹ (63 per cent) despite the CPRs stating that encouraging competition is a key element of the Australian Government’s procurement framework, requiring non-discrimination and the use of competitive procurement processes; and
• value for money was not addressed in the record of the decision to spend public money for two procurements (25 per cent) despite the CPRs stating that achieving value for money is the ‘core rule’ of the CPRs (as it is critical to ensuring that public resources are used in the most efficient, effective, ethical and economic manner).

Innovative projects

2.35 The AMEP General Service Provider Instructions include ‘innovative projects’, which are projects to encourage AMEP service providers to ‘look beyond how you currently deliver AMEP services and what you do well, and identify areas that could benefit from adaptation of new ideas and innovative service delivery to enhance client outcomes’. As discussed at paragraph 2.23, the resulting contracts required amendment after being executed to provide for them.

2.36 The amendment to the contracts to permit the contracting of innovative projects set out various matters the department would specify when inviting a service provider to submit a proposal for an innovative project. Those matters included the timeframe for submitting a proposal, the nature of the services required for the innovative project and the delivery timeframe. The contract does not require the department to inform the service provider of the criteria that will be applied in deciding whether to accept any proposal that is received.

2.37 The AMEP General Service Provider Instructions advised providers that the department ‘may conduct funding rounds at least once per financial year’ and projects may be initiated in two ways: open submission from the panel of AMEP providers; and requests for quotation on specific projects identified by the department. Work orders were issued to 10 of the 14 providers to undertake the 56 innovative projects identified by the ANAO from departmental records (see Figure 2.1), for a total budgeted cost of $9.52 million (excluding GST). Innovative project rounds were open to all providers. Not all providers submitted proposals in all years.

Figure 2.1: Distribution and value of innovative projects as at April 2024

Note: Three projects from 2019–20 were varied to extend the life of the project into 2020–21. One further project is yet to be approved for the 2023–24 year, with costs not included in this figure.

Source: ANAO analysis of Home Affairs records.

2.38 Initially, a competitive approach was taken to awarding innovative projects funding, with all providers invited in April 2017 to submit proposals. Seven providers submitted ten proposals in the funding round, of which eight proposals (from seven providers) were successful.

2.39 The innovative projects guidelines released to the AMEP providers included a statement of requirements for the projects and ‘assessment criteria’ that amounted to minimum requirements for projects identified through competitive rounds. The guidelines did not include evaluation criteria for projects where the department asked one or more providers to quote. While initially intended that work orders would be executed with successful providers, once it became apparent to the department⁵² that the contracts did not provide for this, the 2017–18 innovative projects were implemented through separate contracts with each of the successful AMEP providers.

2.40 Subsequently, between May and June 2018, the main contracts with the AMEP providers were varied to include innovative projects. All subsequent innovative projects have been effected through work orders under those contracts.

2.41 After the variations to contracts innovative projects were selected through the ‘request for quote’ process, with the department providing ‘themes’ and all AMEP providers given the opportunity to submit quotes for proposed projects. Not all providers responded to all requests, with an average of seven providers responding with an average of 12 proposals each year.⁵³

2.42 The requests for quotes did not include evaluation criteria, or weightings against the statement of requirements. The requests for quotes did not included information on how the department would evaluate quotes against each other and assess value for money. An evaluation plan was established for the 2023–24 innovative project round, and records the assessment scores awarded to each proposal against five unweighted criteria and determined value for money rankings for projects based firstly on assessed score (highest to lowest total value), followed by assessed risk (lowest risk preferred) followed by value of project. This was also the first time conflict of interest declarations were made by the assessment team.

Have appropriate probity arrangements been effectively implemented?

2.47 The Department of Finance’s Contract Management Guide identifies that, in addition to an overarching contract management plan, entities should consider developing a probity plan that ‘details the mechanisms for assuring probity within the management of the contract’. This guidance recognises that it is important that entity’s address probity risks during the management of contracts, and not only during the procurement process that results in the contract. The Guide identifies ethical considerations that need to be kept in mind during the management of contracts, including in relation to conflicts of interest:

As a contract manager, you should ensure all individuals materially involved with the management of a contract make a conflict of interest declaration and update it on a regular basis, particularly for longer term contracts.

2.48 Probity considerations are not addressed in the department’s contract management plan for AMEP, apart from identifying that advice on probity (and other) matters can be obtained from the department’s Civil, Commercial and Employment Law Branch and that probity risks will need to be managed as part of the transition to replacement contracts. For example, there is no requirement for those individuals materially involved with the management of the contract to make, and keep updated, conflict of interest declarations. The department does not maintain a register of any conflicts of interests that have been identified, or how they have been managed. The absence of a probity plan also means that, for example, there are not arrangements in place to address probity risks relating to:

• variations to the contracts, including where the variations mean the contract terms and conditions differ from the procurement opportunity that was presented to the market (see paragraphs 2.10, 2.24, 2.26, 2.29 to 2.30);
• ‘innovative projects’ (see paragraph 2.23);
• changes being made to the key performance indicators for the general service providers (see paragraphs 3.11 to 3.19); or
• the change in the nature of the services being requested from the contracted quality assurance provider⁵⁴ such that the focus of the work is no longer consistent with the approach to the market (see paragraph 2.14).

2.49 In May 2022, the department adopted a probity plan that applies to consultations with stakeholders for planned program reforms, and the procurement process for replacement AMEP contracts. This probity plan does not apply to the management of the AMEP contracts under which services have been provided since 1 July 2017.

Is an appropriate transition management plan in place?

2.53 The transition from the prior contracts to the current AMEP general service provider contracts that commenced on 1 July 2017 was not well planned and executed. This was reflected in complaints from some general service providers and stakeholders. One provider made a request for compensation, and later sought (and was awarded) increases to its contracted fee, with the lack of student numbers cited as a key reason.

2.54 The contracts include a ‘completion’ date which according to the terms of the agreement is the date the agreement ends (unless terminated earlier). At the time the contracts were signed, this date was 30 June 2023.

2.55 Section 2.13 of the Department of Finance’s Contract Management Guide state that departments can only extend a contract if three conditions are met: the contract has an unused option to extend the contract; it is value for money to extend the contract; and the contract has not yet expired. The AMEP contract does not have a clause providing an option to extend the contract for any addition or set period of time.

2.56 Notwithstanding this situation, in December 2022, all contracts were varied to adjust the ‘completion’ date from 30 June 2023 to 30 June 2024. Essentially, this extended the contract period for a year despite the contract not having an option to extend. In December 2022, the planned request for tender for new contracts was under design and development, and had not yet been opened to the market.

2.57 As part of the December 2022 variation, the clause specifying when a work order made under the contract would cease was also changed. Originally the clause stated that each work order would end on a date specified in the work order or on the contract’s completion date. This was varied to state that the work order would ‘end on the end date specified in the Work Order, unless terminated earlier’ in accordance with the contract. This variation has the effect of allowing the work orders to continue on past the completion of the overarching contract.

2.58 On 30 November 2023, the Minister for Immigration, Citizenship and Multicultural Affairs approved a further delay⁵⁵ to the planned approach to market for new contract arrangements for the AMEP program. The department advised the minister that:

New AMEP Work Orders would need to be put in place to extend arrangements with the current providers a further nine to twelve months. Current AMEP Work Orders are in place to 31 December 2024. These Work Orders sit under an Agreement that is owned by the Department of Workplace Relations (DEWR) which expires 30 June 2024. While new Work Orders can be entered into beyond the Agreement expiry date, any Work Orders must be in place before the Agreement expires and no further variations can be made after that date.

2.59 The department further advised that minister that:

The Agreement does not have any extension options. Extension may be possible through a Deed of Variation (DoV), but this is complex as responsibility for the Agreement sits with DEWR who conducted the approach to market in 2016 and manage the panel, which includes both AMEP and Skills for Education and Employment providers. Any DoV to extend the Agreement date would require consultation with DEWR and it is not clear if DEWR would support this process as they will have no ongoing need for the Agreement beyond 30 June 2024. It may be possible to seek transfer of the Agreement from DEWR to Home Affairs, but this would result in a large body of work for both departments and is therefore considered unlikely.

If the Agreement is not extended, it is possible to put Work Orders in place before the Agreement expires on 30 June 2024 which go beyond that date, as long as the need for services is justified. However, once the Agreement ceases, it is not possible to execute any new Work Orders. This will result in a loss of flexibility in managing the AMEP contracts, especially in undertaking any further Innovative Projects in relation to the Governments $20 million commitment to improving flexible delivery options and increasing case management support to clients in the AMEP. The Department will be able to incorporate Innovative Projects into ms before the expiry of the Agreement, but would need to seek further advice on the extent to which this is possible after the Agreement ceases.

With any DoV or Work Order process, consultation and negotiation would be required with each service provider.

2.60 The minister noted the risks associated with his decision.

Transition plan for end of AMEP contract

2.61 The contracts include ‘Transition-Out’ clauses, which outline the requirements of providers to support transition to new providers and new contract prior to the expiry of the ‘Agreement Period’. The contract defines ‘Agreement Period’ as ‘the period between the Commencement Date and the Completion Date’ (which is now 30 June 2024, although, as set out in paragraphs 2.57–2.60, work orders under the contracts are due to expire on 31 December 2024).

2.62 In terms of the department’s preparation for transitioning to new contracts, as at December 2023, Home Affairs:

• did not have a transition out plan prepared and approved. Drafting of a transition out plan commenced in April 2022, and the department decided in June 2022 to contract Callida at a cost of $250,000 to ‘create and implement a transition-out plan which will detail the processes and activities required to be carried out to ensure a seamless transition of supplier services prior to the expiry of existing AMEP agreements’.⁵⁶ The most recent version of this draft dated November 2023. The document is not complete, has not been considered by the department’s Transition Working Group and has not been approved; and
• preparation of a transition in plan commenced in April 2023, and there have been no further draft versions prepared. Reflecting the significant uncertainty around the delayed procurement process to replace the AMEP contracts, the transition in plan is substantively incomplete. In February 2024, Home Affairs advised the ANAO that:

The department will continue to progress the development and approval of the Transition In Project Plan following finalisation of the new AMEP business model settings and contractual arrangements. As the Transition In Project Plan is a live document, it will continue to be regularly reviewed and updated over the life of the project, including following approval of the detailed transition in plans submitted by the preferred tenderer/s.

2.63 In February 2024 the department advised the ANAO that it ‘is considering options’ to ‘rectify any timeframe issues in relation to transition out clauses, to mitigate against any potential future risk where Work Orders may extend beyond the Completion Date.’ Further extending the completion date of the contract past 30 June 2024 would be inconsistent with Section 2.13 of the Department of Finance’s Contract Management Guide (see paragraph 2.55) which states that entities can only extend a contract if three conditions are met: the contract has an unused option to extend the contract; its value for money to extend the contract; and the contract has not yet expired. The AMEP contract does not have a clause providing an option to extend the contract for any additional or set period of time.

3.1 The aim of contract management is to ensure that parties meet their obligations such that deliverables are provided to the required quantity and standard and within the specified timeframes such that value for money is achieved. The Department of Finance’s Contract Management Guide states that it ‘is important that contracts are managed consistently and actively throughout their life in accordance with their terms’ as this ‘will ensure that supplier performance is satisfactory, stakeholders are well informed, and all contract requirements are met thereby ensuring that the contract delivers the anticipated value for money.’

Is contract information accurately recorded and stored by the department?

3.2 The Department of Finance’s Contract Management Guide advises entities that keeping good records should be seen as an integral part of, rather than incidental to, contract management activities. Good recordkeeping also assists entities manage contracts to achieve their objectives by providing a sound basis to inform contractor performance monitoring and assessment, including decisions about whether it is appropriate to make payments and, where performance is not to the required standard, the exercise of contractual remedies.

3.3 One of the four key performance indicators (KPIs) for the general services providers (see further at paragraphs 3.14 to 3.15) relates to whether data is recorded and reported in a form required by the Commonwealth within required timeframes. Data recording by service providers and reporting to the department was intended to take place through a new Information Technology system with this system to be used to determine client eligibility and entitlements, record participation and provide the basis for the making of contract payments.

3.4 An internal audit in 2018⁵⁷ identified that ‘Implementation of the replacement IT system was delayed from the original release date of 1 July 2017 which resulted in the business area using contingency spreadsheets to supplement the legacy system [referred to as the] AMEP Reporting Management System (ARMS)’. The internal audit noted that the ‘contingency system … relies on multiple spreadsheets, increased manual data entry and back-end reporting.’

3.5 Delays with implementation of the new system resulted in increased program costs. For 2017–18, a ‘supplementary program administration Fee’ was paid to service providers to ‘compensate for the additional administrative work due to the lack of new management information system’, totalling $4.7 million. One provider requested that the department⁵⁸ to continue the payment into 2018–19, and this was agreed, and paid to all providers. As at November 2020, the department⁵⁹ reported a total cost for ‘supplementary program administration fee’ to be $10.29 million over four financial years. In addition to the payments to service providers, the department estimated that its cost to support the current system until a new system is implemented was $1.1 million per annum.

3.6 Work on the development of a new system ceased in July 2019 when responsibility for the program moved to the Department of Home Affairs.

3.7 Non-delivery of the new system has impacted on the measurement of service provider performance. Specifically, the third (of four) key performance indicators (relating to data timeliness) under the contracts has not been reported against by service providers for any year (see Table 3.1 and paragraphs 3.18 to 3.19).

3.8 Non-delivery of the new system as planned has had other adverse effects on contract management:

• The 2018 internal audit noted that ‘provider error rates on upload have been very high, initially 49% in July 2017, and 30% in January 2018’ (see footnote 50). The internal audit report noted that errors dropped to ‘10% in February 2018 following targeted Provider communication.’ Data entered into the system by the AMEP providers was to be quality assured (on a sample basis) by the quality assurance provider. The required quality assurance activities over data entered into the system have declined over time, such that it is not possible to confirm that the data in the system is accurately recorded (see Table 3.1, paragraph 3.19, and paragraphs 4.31 to 4.64).
• The 2018 internal audit noted that ‘payments to providers are made based on information contained in both the spreadsheets and data entered’ into the information management system. As discussed in paragraph 3.28 and Appendix 5 paragraph 6–7, the department identified in December 2023 errors and concerns with regard to relying on the data for invoicing purposes.

3.9 As part of the Department of Home Affairs’ ongoing request for tender preparations a new information management system is, again, to be developed. This system was to align with the design of the new AMEP model to be implemented with the outcome of the new tender.

Are service delivery standards (or KPIs) for the service providers clearly stipulated, reported, and used to inform contract management?

3.10 Guidance to entities from the Department of Finance is that: good performance management is key to delivering value for money; performance standards for contractors should be specified in the contract; and performance management should take place throughout the life of the contract and be based on the performance framework included in the contract.⁶⁰

The contracted performance management framework

3.11 The request for tender conducted in 2016 set out the performance management framework for the general service providers, including the four KPIs that would be applied. The request for tender package stated that the four KPIs represented ‘a minimum performance standard that service providers will be expected to meet and it is an expectation of the department that service providers will strive to deliver above these standards’.

3.12 The contracts signed with the successful tenderers for the general service provider roles to commence on 1 July 2017 included the same KPIs set out in the request for tender. The contract terms were also consistent with the draft contract included in the request for tender, setting out that non-performance against those KPIs provided the basis for the department to take management action including, where a KPI had not been met on two or more occasions, immediately terminating the agreement.

3.13 The design of the program, and the conduct of the procurement process, related to the delivery of services in 58 contract regions which make up the whole of Australia.⁶¹ The request for tender stated that ‘the need for geographical borders such as contract regions is to ensure service delivery is spread equitably and that services are not clustered in highly populated areas.’ Notwithstanding this design feature of the program, the contracts did not require that the KPIs be met in each contract region (contracts with general service providers covered more than one contract region). For some years, the department obtained reporting of performance against the KPIs that identified performance in each contract region (without specifying how performance in each region was to be converted into overall contract performance), in others the service providers were only required to report their performance in aggregate across each contract region. This reduced the visibility of provider performance within the contract regions.

3.14 As originally signed, the contracts required the KPIs be met from the contract commencement date of 1 July 2017. In December 2017, it was decided to seek agreement from the general service providers to delay the commencement of the KPIs to 1 July 2018. The departmental approval of this proposed variation (which was effected through the second deed of variation), which the general service providers agreed to, did not document the merits of making this change. Over time the KPI framework, established in the contracts following the 2017 procurement process, have been varied, as set out in Table 3.1.

Table 3.1: General service provider contracted key performance indicators (KPIs)

Source: ANAO analysis of departmental records.

3.15 While the service provider contracts have been subject to significant variations (see paragraphs 2.22 to 2.30), the contracts have not been amended to replace the KPIs for attainment⁶², data timeliness or accurate assessment. As a result the performance measurement and management provisions of the general service provider contracts have been given relatively little importance in the department’s administration of the contracts.

3.16 An evaluation of the AMEP business model⁶³, commissioned in 2019, notes that ‘the substantial public investment in the AMEP justifies a progress KPI that can be used to report program outcomes to the government and the public.’ As outlined in Table 3.1, the attainment KPI was reduced and ultimately removed, with no contractual replacement made. Departmental data records 1900 students exiting the program since 1 July 2017 based on attaining sufficient English competency such that they have now become ineligible for the program. In April 2024, Home Affairs advised the ANAO that:

Prior to the implementation of the legislative reforms in April 2021, AMEP participants were entitled to up to 510 hours of tuition. AMEP participants were exited from the program when they reached a 510 hour limit or Functional English, and this limitation and context have been omitted. The legislative reforms removed the 510 hour cap on English language tuition; raised the AMEP eligibility threshold and exit point for the program from functional to vocational English; and removed time limits on enrolling, commencing and completing AMEP tuition, for those already in Australia on or before 1 October 2020. These changes have allowed more migrants to access English language tuition through the AMEP for longer and until they reach a higher level of proficiency. Learning a new language is complex and takes time, and the number of hours of tuition required by each individual varies based on many factors, including age, prior education and the linguistic distance of their first language from English. Research shows that 510 hours is insufficient to reach functional English for the vast majority of AMEP clients who enter the program with very low levels of English. For these people, around 2,000 hours may be required.

Quality requirements

3.17 The contracts set out that meeting the KPIs is not the only performance requirement. Specifically, the contracts state that quality of services provided may also be considered in relation to: entry interviews conducted by the service provider; client individual pathway guides; tuition provided to clients; and progress of clients through the AMEP Learning. The provision of quality assurance provider services, which is relevant to this aspect of the department’s management of the general service provider contracts, is examined in Chapter 4 of this audit report.

Service provider reporting against KPIs

3.18 Reporting by service providers of their performance against the KPIs is required to be included in both a mid-year report (required by 31 March each year covering the six months to 31 December) and an annual report (required by 30 September each year covering the year to 30 June).

3.19 As set out in Table 3.1, the first KPI (relating to participants commencing in the program) is only one where there has been reporting to the department throughout the contract period by general service providers of their performance. For the three most recently completed years (2020–21, 2021–22 and 2022–23) service providers have only had one KPI to meet, and report against.

3.20 On those occasions between 2017–18 and 2022–23 where reporting by a general service provider was that they had not met a KPI, departmental records did not outline that department employed the contractual provisions enabling it to issue a notice requiring service improvement. In February 2024 the department advised that ANAO that ‘no notices requiring service improvement have been used under the contract.’

Are invoices verified and payments made in accordance with AMEP contracts?

3.24 Verification of invoices is important to ensure the proper use of commonwealth resources. The Department of Finance’s Contract Management Guide states that departments should only make a payment when the service has been received and meets the required standards; the supplier is compliant with the contract; the invoice is accurate and correct according to the contract; and when all necessary authorisations and approvals have been obtained.

3.25 Although the contracts are constructed similarly, the department has implemented different invoicing processes for the AMEP general service providers compared to the AMEP quality assurance provider.

• Consistent with the contracts, the invoices for the AMEP general service providers are determined by the department. The department determines an estimate that it instructs providers to invoice for, and then makes a quarterly adjustment to the invoices based on data uploaded to Home Affairs systems by each AMEP general service provider.
• Rather than employing the approach set out in the contract, the invoices for the AMEP quality assurance provider are based on work completed in the prior period (usually a month) with no quarterly adjustments process (see paragraphs 4.78 to 4.79 and Appendix 6).

3.26 The approach to the market outlined the services the AMEP general service providers were to provide English language tuition and assessment (across two streams and sub-programs), child care and the Home Tutor Scheme (later renamed the Volunteer Tutor Scheme). In May 2018, variations were made to the AMEP general service provider contracts to allow the department to invite providers to submit proposals for ‘Innovative Projects’ to develop, test, evaluation and report on different approaches and techniques (see paragraphs 2.35 to 2.41).

3.27 Departmental records state that $1.29 billion has been paid to AMEP general service providers since 1 July 2019 (see Figure 2.1). The distribution of this amount has been extensively driven by providers’ invoiced provision of AMEP general services (namely, numbers of tuition hours and associated child card fees). Since 1 July 2019, $8.09 million has been spent on ‘innovative projects’ and additional projects, of which $5.87 million⁶⁴ was paid to the AMEP quality assurance provider. The remaining nearly $2.22 million was shared between 9 of the 13 AMEP general service providers, with payments for projects ranging between $23,255 to $524,469 (see Figure 2.1).

Figure 3.1: Payments made to AMEP general service providers since 1 July 2019

Note: Data limitations as a result of machinery of government changes have prevented analysis from contract commencement (1 July 2017).

Source: ANAO analysis of Home Affairs records.

3.28 Invoicing and payments to the 13 AMEP general service providers has not consistently adhered to the contracts with issues identified by the ANAO in a number of areas, (see Appendix 5 for details) including:

• monthly instalments and quarterly adjustments;
• reliance on unverified data;
• payment for services outside of the contract;
• fee indexation;
• application of GST;
• backdating invoices to accommodate variations to fee; and
• use of fees outside of written agreement.

Does the department have appropriate complaint resolution processes?

3.29 The Commonwealth Ombudsman’s Better Practice Complaint Handling Guide states that good complaints handling will help agencies ‘meet general principles of good administration, including fairness, transparency, accountability, accessibility and efficiency’ and that the ‘analysis of complaint data may identify opportunities for the improvement of administrative practices’.⁶⁵

3.30 Auditor-General Report No. 40 2000–01 Management of the Adult Migrant English Program Contracts, which examined the management of earlier AMEP contracts:

• identified that the service provider contracts required providers to have a complaint handling procedure and to resolve client complaints;
• concluded that the department was not monitoring complaints to service providers including by not obtaining complaint data from the service providers which, in turn, was limiting assurance that issues impacting on contractors’ performance are brought to the department’s attention and that systemic issues are identified and taken into account in broader contract management;
• observed that the results from the department’s Client Satisfaction Survey emphasised that the department needed to better manage the complaints handling mechanisms of providers; and
• recommended that the department collect, monitor and review provider complaints data to inform and improve service delivery. The department agreed to this recommendation.

3.31 The current AMEP service provider contracts include obligations on service providers to have in place mechanisms/processes for the timely and effective handling of complaints, and to make sure clients are aware of the existence of the complaints handling process. In addition, the contracts require that each provider maintain a register of complaints and provide this to the department upon request.

3.32 The Service Provider Instructions issued by the department under the contracts require the 13 general service providers to provide an annual report for each financial year that includes data on number and nature of client complaints received. In comparison to the number of clients, the number of complaints as reported by service providers was as follows in 2021–22:

• a total of 37 complaints were reported to Home Affairs by the general service providers in their AMEP annual reports, representing 1 complaint for every 1,514 enrolments in 2021–22 (or 0.07 per cent)⁶⁶;
• 89 per cent of the complaints were reported by two of the providers with a further four providers reporting that they received a single complaint; and
• seven providers did not report having received any complaints in 2021–22. Of those seven providers, four have not reported receiving a single complaint in any of their annual reports.

3.33 Home Affairs has not:

• analysed the complaints data included in the annual reports from service providers, for example to compare complaints data between service providers;
• sought to satisfy itself:
  • about the completeness of the data being reported to it, such that in some years, providers did not complete the complaints section in the annual report template, with the department not taking steps to have this corrected; or
  • that the general service providers have appropriate complaints handling processes and systems in place; and
• exercised its contractual right to obtain complaints information from providers and so has not analysed client complaints in the manner it had agreed to in response to the earlier ANAO performance audit.

3.34 In April 2024, Home Affairs advised the ANAO that ‘Through the onsite QA process, the QA provider checks service providers’ complaints processes’. As discussed at paragraphs 4.31 to 4.68, the budgeting for, and tasking of, the quality assurance provider by Home Affairs has significantly changed the nature of services provided. In particular, as a result of the department refocusing the work of the contracted quality assurance provider to the delivery of professional development and development of ‘program delivery documents’, the quality assurance activities planned and delivered have not appropriately monitored the performance of the contracted general service providers.⁶⁷ The ANAO has recommended that Home Affairs give greater emphasis to monitoring the quality of services being delivered to students by the contracted general service providers (see paragraph 4.69).

4.1 The 2016 approach to the market outlined the key role the quality assurance provider was to play in ensuring clients receive high quality English language training and assessment services under AMEP. The Department of Home Affairs’ tasks Linda Wyse and Associates (LWA) for the provision of quality assurance services for AMEP.

Has the department appropriately exercised its contractual powers to manage the contracted quality assurance provider’s performance?

4.2 Guidance to entities from the Department of Finance is that: good performance management is key to delivering value for money; performance standards for contractors should be specified in the contract; and performance management should take place throughout the life of the contract and be based on the performance framework included in the contract.⁶⁹

Contracted performance management framework

4.3 The performance framework for the AMEP quality assurance provider was set out in the request for tender for the procurement conducted via an approach to the market in 2017. The request for tender stated that the seven KPIs ‘represent the performance standard that the quality assurance providers will be expected to meet’ and that it ‘is an expectation of the department that the quality assurance providers will strive to deliver above these standards’. The request for tender further stated that ‘failure to meet KPIs may result in the Department applying sanctions as outlined in the [Quality Assurance Provider Instructions]’.

4.4 Consistent with the request for tender, the contract signed with the successful tenderer to commence on 1 July 2017 set out that non-performance against the specified KPIs provided the basis for the department to take management action including, where a KPI had not been met on two or more occasions, immediately terminating the agreement. The KPIs included in the contract are inconsistent from those included in the request for tender and presented to the market (see Table 4.1). Some changes were made in the contract when first signed, with the KPIs varied a further two times after contract execution.

Table 4.1: Key performance indicators — quality assurance provider

Note: Bold text highlights changes made to the key performance indicators.

Source: ANAO analysis of departmental records.

4.5 The first change to the KPIs made after contract signature occurred in February 2018 to include a KPI relevant to the new service requirement to deliver professional development workshops. No further variations have been made the key performance indicators despite the range of other variations made to the contract, including changes in the services to be provided. As a result, the performance measurement and management provisions of the quality assurance provider contract have been given little importance in the department’s administration of the contract (as was also the case for the general service provider contracts, see paragraph 3.15). Changes that have happened and have not been reflected in the contracted key performance measures include:

• descoping of significant parts of the onsite quality assurance assessments in 2018–19 which were only placed back into the contract in 2021 (see paragraph 4.36);
• significantly decreasing the scale of client file verifications from over 3000 to a few hundred (see paragraphs 4.31 to 4.50);
• increasing focus and expenditure on ‘program delivery documents’ (see paragraphs 4.55 to 4.59);
• pausing of quality assurance services during the COVID-19 pandemic (see paragraphs 4.60 to 4.64); and
• inclusion of ‘innovative projects’ to the contract and increased focus and expenditure of these projects (see paragraphs 4.65 to 4.67).

Quality assurance provider reporting against KPIs

4.6 The quality assurance provider is required to report its performance against the KPIs in a ‘quarterly report’ due within 15 business days of the end of each quarter. The quality assurance provider is also required to deliver an annual report (required by 30 September each year covering the year to 30 June) that includes information about the: ‘number of services delivered’; ‘effectiveness’ of the quality assurance activities delivered; the ‘appeals register’; and areas of ‘improvement’ to be made to the delivery of AMEP services.

4.7 The reporting by the quality assurance provider in its annual reports (see Table 4.2) identified that it has frequently not met the KPI relating to delivery of quality assurance activities. Performance has not met the KPI notwithstanding that the annual plans have consistently reduced the number of activities to be delivered. The provider reported that it has delivered the planned number of onsite quality assurance activities in only one year, 2022–23, and has only delivered all the planned file verifications in two years.⁷⁰ Overall, since 2017–18:

• file verifications have fallen short by 2,362 files (27 per cent) across the first six years; and
• onsite quality assurance reviews have fallen short by 30 (20 per cent).

4.8 On the occasions where reporting from the provider was that it had not met a KPI, departmental records did not record whether Home Affairs employed, or considered employing, the contractual provisions which would have started with issuing a notice requiring service improvement.⁷¹

4.9 In February 2024, the quality assurance provider advised ANAO that:

Decisions and actions, external to LWA, such as management of contract transition [see paragraph 2.53, paragraph 4.5, paragraphs 4.31 to 4.50], service provider site establishment/business levels, suspension of QA verification services and KPIs [see paragraph 4.5 and paragraphs 4.60 to 4.64] during COVID-19 and extension of exceptional circumstances [see paragraphs 4.60 to 4.64 and Appendix 5’s paragraph 6], obstructed LWA from delivering files numbers according to the annual plan. It would not have been appropriate for DHA to issue notices requiring service improvement given the contexts cited above.

Departmental administrative arrangements

4.10 The department reduced its ability to appropriately manage the contract with the quality assurance provider by:

• frequent changes in the contract manager, with extensive reliance on hand over notes rather than contract records;
• inconsistency in reporting lines for the provider, depending on the work being delivered. Innovative projects and some program delivery documents have been delivered to the AMEP policy team, some to the AMEP operations team, and others to the AMEP governance and assurance services team; and
• deviating its focus on the work produced by the provider from quality assurance activities to program delivery (consultancy) work, which is also at odds with the procurement opportunity that was presented to the market and the assessment of competing proposals for the quality assurance provider role. Performance measures have not adapted in line with this change in focus, undermining the value of the performance framework in the contract.

Has service provider performance informed planned quality assurance work?

4.15 The Department of Finance’s Contract Management Guide advises entities to ensure that contractual obligations are carried out, including meeting performance measures, as ‘not addressing supplier underperformance when reported may establish new reduced performance levels within the contract’. The intent of quality assurance activity is to ensure services are being delivered to a standard that supports achievement of program outcomes.

4.16 The contract with the quality assurance provider determines what information and inputs the provider may use to develop its annual plan of quality assurance activities. The contract places and obligation on the department to ‘notify the Contractor, by 1 June each year, the: (a) risk profiles of each AMEP Service Provider’⁷², the budget within which the quality assurance work is to be delivered; and the number of client file verifications (for each contract region) and onsite quality assurance activities to be delivered that year.

4.17 The contract requires the quality assurance provider to prepare and submit, by 15 June each year, the annual plan which sets out the proposed quality assurance activity taking into account the information provided by the department. Provider performance against key performance indicators, provider’s annual reports, or briefs from the AMEP general service provider contract managers are not included in this range of information to be drawn on when developing the annual plan. Instead, the department is to include this information to inform the department’s determination of the general service provider risk profiles.

Risk rating

4.18 All three versions of the Quality Assurance Provider Instructions, which form part of the contracting suite⁷³ with the quality assurance provider and the department, stated that:

The risk rating will be dependent on:

• length of time contracted in the program
• Service Provider performance against Key Performance Indicators (KPIs)⁷⁴ and other contractual obligations⁷⁵
• governance arrangements within organisation
• strength of stakeholders relationships.

4.19 The contract and Quality Assurance Provider Instructions specify requirements for the quality assurance provider on how activities with providers must be conducted based on the department’s risk profile for the provider. Provider’s deemed ‘high or very high risk’ are to have quality assurance activities and client file verifications conducted ‘onsite’ using ‘original documentation’, while client file verifications may be conducted offsite for moderate or low risk providers. The contract defines ‘high risk provider’ as a provider that will ‘have a higher percentage of AMEP Client files reviewed every year’ and report to the department how the provider ‘will improve its performance against the KPIs’. These elements of the contract were not varied by any of the nine deeds of variation executed during the life of the contract.

4.20 The department has not met its obligation to produce and provide risk ratings to the quality assurance provider each year. Where ratings have been provided, there are no records to support the allocation of the ratings, and no evidence that the department has considered the inputs as described in the Quality Assurance Provider Instructions (see paragraph 4.18). It is not evident that provider performance informed the allocation of risk ratings.

Table 4.3: Risk category allocated to AMEP general service providers by the department

Note a: The risk ratings provided in 2017–18 were allocated against a mix of providers and contract regions. While this approach is consistent with the design of the program, where services are provided by a single provider in more than one contract region (each with their own inherent regional risks), and may be delivered by sub-contractors, such an approach is inconsistent with the contract, that states that risk profiles will be provided for each service provider (see paragraph 4.16).

Note b: In 2018–19 risk ratings were allocated to contract regions, 25 regions were assigned ‘high’, 6 were assigned ‘medium’ and 22 ‘low’ risk ratings. ANAO has aggregated these allocations to a provider level.

Note c: In 2019–20 risk ratings were allocated to contract regions, three were assigned ‘high’, 19 were assigned ‘medium’ and 30 were assigned ‘low’ risk ratings. A consultant was contracted to review the allocation of quality assurance activity, and developed a ‘proportional’ approach. This had the effect of applying a ‘low’ risk rating to providers where their KPI4 score was 75 per cent or greater; and ‘medium’ where their KPI4 score was 75 per cent or less. The results of this approach have been reflected in the table.

Note d: The risk assessment of ‘low’ for all providers is at odds with the status of the program in June 2021 when risk profiles were to be finalised. At the time general service providers were at varying stages of reopening to onsite teaching after the pandemic lockdowns. Onsite quality assurance assessments (where possible) would have been of increased value at this time.

Note e: This provider was the main provider of services in New South Wales and the largest provider of services for AMEP.

Source: ANAO analysis of departmental records.

4.21 The approach to determining the ‘risk rating’ for providers has varied across the life of the program. In the first two years, risk ratings were allocated at a contract region level, without evidence to support the ratings. By May 2019, the approach developed for the department by ORIMA Research under a contract was adjusted by the department to determine risk based on general service provider results for their KPI4 (see Table 3.1 and paragraph 3.19). Only two risk ratings were applied: ‘low’ a KPI result of 75 per cent or greater; or ‘medium’ a KPI score of 75 per cent or less. It is not clear how it was decided that only two categories (low and medium) of risk would be used⁷⁶; or why 75 per cent was selected as the threshold, particularly when the KPI requirement was that 80 per cent of assessments be accurate.⁷⁷

4.22 The department does not have records explaining or approving:

• the changed approach;
• the rationale to reduce the number of risk categories from an original four down to two; or
• the application of 75 per cent threshold to the KPI assessment rather than the contracted 80 per cent.

4.23 The reduction of available risk ratings from four categories to two (‘medium’ and ‘low’) has an impact for the application of the contract requirements on how ‘high or very high’ rated providers are to be treated (see paragraph 4.19). The approach employed by the department did not provide an option for a provider to be assessed as ‘high’ or ‘very high’ risk. This truncation had the effect of reducing the work required of the quality assurance provider and the department in managing ‘high or very high’ risk providers.

4.24 Risk profiles were not provided or considered in quality assurance planning for the 2020–21 financial year. The department’s records indicated that the team undertaking the planning for quality assurance were concerned that there was not a clear process that was ‘documented, transparent and easily understood’.

4.25 The department’s failure to have a clear, evidenced-based approach to considering and evaluating AMEP general service provider risks, to inform quality assurance activities, resulted in the development of a quality assurance plan for the 2020–21 year that was mostly focused on ‘support’ activities and less on ensuring quality was being delivered. Notwithstanding this was when it would reasonably be expected that risks were high for all service providers (having transitioned to virtual and mixed mode tuition), and addressing COVID-19 impacts on schedules. The department recorded that:

• ‘under the Contract the Department is required to provide the [quality assurance provider] a budget to allow [the quality assurance provider] to plan the next financial year’s delivery of Quality Assurance Support Services. Under normal circumstances [the quality assurance provider] provides an annual plan and costings to the Department for consideration and agreement and that annual plan then forms part of the agreement’. The department made no reference to its obligation to provide the quality assurance provider with risk profiles on which to target the quality assurance plan.
• ‘due to the impacts of COVID 19 and the uncertainty surrounding the extent of that impact into the FY 2020/21’ only a quarterly plan was initially agreed, reflecting ‘what is possible to deliver in a COVID 19 environment’. Despite ‘the impact COVID 19 has had on the ability to undertake’ quality assurance activities the department committed ‘20.6% of the expected $1.2 million’ budget for the first quarter activities. Quality assurance activities included in the first quarter plan were:
  • zero client file verifications;
  • zero client initial assessment file verifications;
  • zero onsite quality assurance activities;
  • three online professional development workshops; and
  • ongoing costs of maintaining the Assessment Task Bank
• The rest of the activities were ‘program delivery’ activities.
• By only approving a quarterly plan the department was provided with ‘time to review the quality assurance needs of the AMEP and determine the direction for Quality Assurance Support Services for the remainder of FY2020/21’.
• ‘the Department can engage [the quality assurance provider] under the Agreement to deliver other related Quality Assurance Support Services. These items fall under … the Program Delivery Documents Clause. AMEP Policy Section, together with AMEP Operations Section, will work with [the quality assurance provider] to define these services.’

4.26 The quarterly plan was approved on 3 July 2020, with the record of approval noting:

The above activities should be considered indicative and are subject to agreement between LWA and the Department about in [sic] Quarter 1:

• Supporting the design, implementation and quality assurance of an interim performance measurement system⁷⁸;
• Consideration of the next stage of the AMEP English Language Proficiency Framework with Melbourne University and key collaborators [see Appendix 4];
• Moving QA away from assessment verification to a more holistic QA that covers: teaching programs, targeting for cohorts, flexibility in delivery, promotional materials, referral to [Distance Learning];
• Providing for the contracting by LWA of experts to assist with AMEP policy design and program operations.

4.27 Records from April 2021 demonstrate that the department was considering how to approach planning for quality assurance for the 2021–22 financial year, and returning some of the original quality assurance activities to the plan. Records outline that the department’s methodology for sampling was to determine risk ratings at a provider level ‘based on KPIs and performance outlined in the annual report’. Departmental records did not evidence that this work was completed, nor that the proposed approach considered that the four KPIs originally planned to be used had reduced to one measured KPI by this time, and that the one KPI remaining related to ‘commencement’ of students, not ‘quality’ of tuition or achievement of student progress (see paragraphs 3.14 to 3.19). The continued determination that all 13 providers were ‘low’ risk, evidenced that no revised approach was applied.

Has the quality assurance provider appropriately monitored service provider performance?

4.31 The request for tender described the AMEP quality assurance services sought as: ‘verification services including client files, work experience placements and onsite QA assessments that include teacher qualifications and curriculum verifications and additional services the following QA services’. The request for tender noted that the focus of the planned quality assurance activities would be ‘directed at establishing the accuracy of assessments used to claim’ achievement against the curriculum and ‘consistent application’ of the curriculum by AMEP providers. The request for tender, and the Quality Assurance Provider Instructions included, provided details on the quality assurance required. As outlined in Table 4.4, the Quality Assurance Provider Instructions and contract executed on 10 April 2017 aligned with the request for tender. The Quality Assurance Provider Instructions were updated twice (these adjustments are highlighted in bold text in Table 4.4) during the course of the contract, with no adjustments made to accommodate changes in quality assurance services during and after the COVID-19 pandemic.

Table 4.4: Quality assurance service scope as outlined in Provider Instructions

Note: Bold text highlights the changes made to the scope of the service.

This is not a complete list of contracted services, it is only services specific to the quality assurance function. Services not included are set out in paragraph 4.32.

Source: ANAO analysis of departmental records.

4.32 Other services included in the contract with the quality assurance provider that did not relate to quality assurance were:

• management and maintenance of the AMEP Assessment Task Bank (later AMEP Hub), including assessment task validation, hosting and promotion of a secure website and an additional website, for publicly available learning resources;
• preparing program delivery documents (described as ‘materials to support Service Providers in the delivery and administration of the program’ and that may include the development of ‘templates; instructions; coversheets; and guidance material’);
• delivery of professional development workshops (added in the January 2018 variation to the contract); and
• innovative projects (added in the December 2021 variation to the contract).

Allocation of funding budgeted for quality assurance work

4.33 The focus of the request for tender for the quality assurance provider role, and the resulting contract, was the delivery of quality assurance services (verification services including client files, work experience placements and onsite quality assurance assessments that include teacher qualifications and curriculum verifications and additional services) and Assessment Task Bank (ATB) management to support the delivery of AMEP.

4.34 In 2017–18, the first year of the contract, funding for the quality assurance provider was consistent with this emphasis as 78 per cent of the budget was approved by the department for quality assurance services and 19 per cent for ATB management. Over time, the allocation of the budget for the quality assurance provider services has changed considerably such that it is no longer consistent with the approach to the market and the resulting contract. While ATB management has been approved for between 17 per cent and 29 per cent of the budget in each year, there has been a marked decline in the budget allocation for quality assurance services. This decline began in the second year (2018–19) of the contract, with the largest reduction occurring when the department approved the plan for the third year (2019–20) of the contract, and has continued such that, for 2023–24, only 15 per cent of the budget was approved for quality assurance services (see Figure 4.1). As a result, the emphasis in service delivery has changed from quality assurance work to other activities. Reflecting this situation, in February 2024, LWA advised the ANAO that ‘LWA’s quality assurance role has been hollowed out over the course of this contract and now LWA’s services do not verify program educational content’.

Figure 4.1: Distribution of funding to quality assurance activities as set out in approved quality assurance annual plans

Note: Innovative projects were only separately reported, and clearly traceable in the annual plans from 2022–23. Prior to that they were incorporated into program delivery document activities (see paragraph 4.58).

Source: ANAO analysis of departmental records.

Quality assurance activities

4.35 The contract with the quality assurance provider requires the department to issue work orders outlining which contracted services are to be delivered. A total of six work orders have been issued.

4.36 The first work order commenced on 1 July 2017 for a period of three financial years, ceasing 30 June 2020. Separate work orders were issued for 2020–21, 2021–22, 2022–23 and 1 July 2023–31 December 2024. No variations to any work orders could be located to account for:

• the five variations during the same period;
• the addition of professional development workshops (see paragraphs 4.51 to 4.53);
• the changes made in September 2018 to the administration of the quality assurance function with administering department’s state networks assuming elements of the quality assurance scope of services⁷⁹ (and the subsequent reinstatement of the work in February 2021⁸⁰);
• changes to program delivery document acquisition process (see paragraph 4.56);
• changes to requirements for the 2020–21 quality assurance planned activities⁸¹ as a result of the COVID-19 pandemic (see paragraphs 4.60 to 4.64); or
• changes in April 2021 to the legislation that underpin AMEP service and service requirements; or
• inclusion of innovative projects into the contract (see paragraphs 4.58 to 4.61 and paragraphs 4.65 to 4.67).

4.37 Instead, changes to service requirements for each year were incorporated into the Annual Plans drafted between the department and the service provider.

4.38 Not all elements of the provider instructions regarding quality assurance services were met. The quality assurance provider reported against service delivery for quality assurance activities are part of its annual KPI reporting (see paragraphs 4.6 to 4.9). As outlined in paragraph 4.34 and illustrated in Figure 4.1, services delivered by the quality assurance provider have deviated from quality assurance activities, with the focus increasingly on program delivery documents and innovative projects.

Volume of quality assurance activities

4.39 The number of client file verification activities (per contract region) and onsite quality assurance activities are to be advised by the department to the quality assurance provider by 1 June each year (see paragraph 4.16). Initially the number of each of these activities was incorporated into the Quality Assurance Provider Instructions (see Table 4.5).

4.40 The quality assurance provider is to allocate the number of client file verifications and onsite quality assurance activities based on provider risk (see paragraph 4.17) across an annualised plan, based on requirements in the contract and the Quality Assurance Provider Instructions. One key change was made to the requirements over time (see Table 4.5), specifically to remove the stated number of client file verifications to be completed.

Table 4.5: Advised distribution of assurance activities

Note a: In response to queries from ANAO, LWA advised on 20 November 2023 that it was not aware of the January 2018 version of the Quality Assurance Provider Instructions. LWA advised that ‘changes in the scope of the quality assurance services resulting from variations to contracts were incorporated into LWA’s work via changes in project work, templates and processes and the issuance of annual plans.’ ANAO notes that this approach by the department to administer the contract is inconsistent with the contract. Rather than a single source of truth, or a clear line of requirement, three versions of requirements exist, the contract requirements, the outdated Quality Assurance Provider Instructions, and the less than detailed work orders which refer back to the contract.

Note b: This version of the Quality Assurance Provider Instructions was redated in March 2020 following machinery of government changes which moved the AMEP to Department of Home Affairs. The only change made to the instructions was the rebranding of the document to carry the Home Affairs document header. No content was changed. The department confirmed to ANAO on 21 November 2023 that the quality assurance provider was not provided with a copy of the rebranded Instructions.

Source: Departmental records.

4.41 Consistent with the August 2018 change in Quality Assurance Provider Instructions, the number of client verifications to be completed was reduced from 2018–19 (see Figure 4.2). It was not evident from department records how the planned figures were arrived at for each year. Despite no changes being made in the contract or the Quality Assurance Provider Instructions on the number of onsite quality assurance activities (52 contract regions are serviced by general service providers, and 7 are serviced by the Distance Learning provider), the number of planned activities declined from 2018–19.

Figure 4.2: Changes in planned and actual quality assurance activities

Note: Data is not yet available for activities completed in 2023–24 as annual reporting from the quality assurance provider is due to be submitted on 30 September 2024.

Source: ANAO analysis of departmental records.

Allocation of quality assurance assessments

4.42 Quality assurance assessment are activities undertaken to:

• assess the suitability of the learning environment (including that: sites are compliant with relevant state and territory safety standards; the facilities are safe, clean and well maintained; equipment, furniture and fittings are sufficient and fit for purpose; and the size and layout meet the learning needs of clients);
• verify teacher qualifications (including: that minimum Registered Training Organisation requirements are met, that AMEP additional requirements are met; and that teachers with appropriate qualifications are teaching appropriate AMEP streams);
• verify the curriculum and resources used for teaching (including: that the curriculum used aligns with the curriculum registered with the department; that the curriculum used is suitable to achieve client outcomes; that the educational resources are maintained and relevant to the curriculum and are in a good condition with sufficient supply and variety to suit the needs of client); and
• undertake client file verifications (for high-risk service providers).

4.43 The KPI in the contract stated that the provider ‘must complete one onsite assessment per service provider, per contract region per financial year, or as specified in the annual plan’ (see Table 4.1). As outlined in Figure 4.2, quality assurance assessments have not been planned or completed against all 53⁸² services contract regions each year.

4.44 In response to ANAO questions about the inconsistency between the Quality Assurance Provider Instruction requirements and delivered activities, the department advised, in October 2023, that ‘the number is determined by the department’ and is based on one per ‘each AMEP provider’, noting that ‘where there are multiple sites in a contract region, [the quality assurance provider] ensures that different sites are visited each year’ and that ‘there is discussion with the provider to organise the site that is visited’. The ANAO further queried whether the approach to selecting sites for onsite assessment was ‘risk based’ or potentially ‘event based’ with the department advising that the quality assurance provider ‘is not informed by service providers of most of the factors you’ve noted (i.e. changes in [service provider] staff, changes in class room venues, complaints, ASQA concerns’ and that ‘selection of sites occurs following discussion with service providers, with sites selected on a rotating schedule’. This is not a risk based approach, nor does the coverage align with the Quality Assurance Provider Instructions approach of each contract region.

4.45 Annual plans have varied the figures over time to only complete one onsite quality assurance assessment per provider per financial year. This means that not all contract regions are planned to be examined. Such an approach relies on strong governance within the service providers to ensure that their internal processes enforce compliance with the AMEP contract requirements and does not consider regional impacts or subcontracting arrangements, without those governance arrangements being assessed as to whether they can be relied upon. This is particularly impactful where:

• the sole provider in the Australian Capital Territory, also provides services in Tasmania and New South Wales, so for the 2019–20, 2020–21 and 2023–24 years no onsite quality assurance activity was planned for the Australian Capital Territory, and no work was completed in 2019–20 and 2020–21;
• no onsite quality assurance activity was planned for the sole provider in the Northern Territory in either 2019–20 or 2020–21, and no work was completed in 2019–20, 2020–21 or 2021–22; and
• where one provider is contracted to deliver services for an entire state, such as in Queensland and South Australia, only one onsite quality assurance activity was planned for 2019–20, 2021–22, 2022–23 and 2023–24.

Allocation of client file verification activities

4.46 While the Quality Assurance Provider Instructions required activities to be ‘distributed nationally based on risk’, the quality assurance plans do not evidence that this occurred.

4.47 In 2017–18, for example, the one provider with no risk rating was allocated 450 client files for verification⁸³ whereas:

• the two providers with a ‘very high’ rating were allocated between 30 and 150 file verifications per contract region, the same number of files also allocated to the three providers with ‘high’ risk ratings; and
• one ‘low’ rated provider was allocated 17 files for verification whereas the other six ‘low’ risk providers were allocated between 125 and 150 client file verifications per contract region and the ‘low’ rated Distance Learning provider was allocated 325 file verifications.

4.48 Changes were made to the approach in each of the next three years. For example, in 2021–22, all providers were allocated low risk, with the only factor actually determining number of client file verifications was the student population size of each provider.

4.49 In 2022–23 and 2023–24, the allocation method from 2021–22 was reused.

4.50 The changes in approach to allocating client file verifications by the department has had little impact on the proportion of client files each provider was required to have verified by the quality assurance provider (see Figure 4.3). Other than for two providers, the allocations are largely consistent (as a portion of total file verifications) across the annual plans and, where there are changes, these are not informed by changes in assessed risk. Instead, the approach simply reduced the number of files verified from an original sample of 3249 files to 1412 files.

Figure 4.3: Changes to proportion of client file verification allocated to provider as a percentage of total client file verification sample size

Note: Client file verification allocated to Tafe NSW have been split, to distinguish between those allocated to the provider in its general service provider capacity, and as the sole Distance Learning provider.

Source: ANAO analysis of departmental records.

Volume of professional development workshops

4.51 Less than seven months after the quality assurance contract was executed it was varied to include a new service, the provision of ‘professional development workshops’ that had not been included in the request for tender. The topics were to be based on the quality assurance provider’s ‘experience of Quality Assurance Assessments and Client File Verifications’ taking into account the ‘developmental needs of the Personnel of the AMEP Providers delivering AMEP Services’.

4.52 The variation stated that the provider was to deliver these workshops ‘at its own cost’. Despite this, the same deed of variation also amended the fee schedule, and included a fee for holding profession development workshops in particular regions.

4.53 Eight months later (September 2018) a variation was made to the contract to include an additional location for professional development workshops to be held, for a fee, increasing the average cost of professional development workshops. None of the subsequent variations removed the clause that specified Professional Development Workshops were to be conducted at the quality assurance providers ‘own cost’. The Department of Home Affairs has paid the quality assurance provider for the Professional Development Workshops held in accordance with quality assurance Annual Plans provided by the quality assurance provider each year.

4.54 In 2017–18, six professional development workshops were held. This increased to 24 in 2018–19 and 2019–20 (representing 21 per cent and 35 per cent of the total budgets for those years). In 2020–21 annual plans of the quality assurance provider included 35 workshops (representing 15 per cent of the total quality assurance budget for the year) to be held online, despite having no online fee agreed within the contract, and no variation made to date to include a fee. Ultimately, 18 workshops were delivered. In 2021–22, 24 online workshops (representing 21 per cent of the total quality assurance budget for the year) were delivered online. The quality assurance provider’s annual plan for 2022–23 included holding 25 online workshops, representing 21 per cent of the total quality assurance budget for the year.

Volume of program delivery document activities

4.55 In addition to the main service sought in the request for tender from a quality assurance provider, the request for tender and initial contract included the provision of ‘additional services’, including development and provision of ‘program delivery documents for use by AMEP service providers’ such as ‘templates, instructions, coversheets or guidance material’.

4.56 Initially the department was to determine ‘a need’ and provide a full scope and details of the required work to the provider. On 24 September 2020 the contract was varied so that the department would ‘consult with the Contractor to determine the full nature and specifics of the documents to be developed’ and if required three quotes for any specialist expert services would be sought.

4.57 A work order would be issued, under the contract, by the department with respect to any program delivery document projects agreed. The ANAO identified five such work orders⁸⁴, with a total budgeted cost of $332,192 (excluding GST):

• Work Order 2, commenced April 2018, to deliver an AMEP Curriculum and Teacher Assessor Qualification Guide;
• Work Order 3, commenced November 2018, to deliver a ‘Streamlined CALD IA/PTA Kit’ including trial and revision of the kit, and publication of report on kit use;
• Work Order 2, commenced December 2019 to map the ‘AMEP Curricula to the Outcomes Framework – Stage 1’ (see Appendix 4);
• Work Order 3, commenced February 2020, to continue the December 2019 work a deliver ‘Outcomes Framework – Stage 2’ (see Appendix 4); and
• Work Order 4, commenced April 2020, to ‘support teachers in their transition to online/remote delivery’ as a result of the COVID-19 pandemic. The quality assurance provider’s existing quality assurance work was ceased, and work was redirected to:
  • revise the Assessor Guide previously created by the contractor for initial assessments and update to facilitate virtual/mixed mode delivery of the initial assessment by AMEP Providers;
  • deliver four training sessions to AMEP Providers about ‘online/remote (booklet) delivery’ and the delivery of the virtual/mixed mode initial assessment;
  • manage a ‘Q&A chat forum … to answer teacher queries and concerns’;
  • introduce a ‘suite of resources’ to the AMEP website to support virtual participate/mix mode delivery of AMEP;
  • deliver a suite of professional development training to highlight best practise for virtual/mixed mode initial assessments and AMEP delivery;
  • provide a report to the department on the innovation of providers, success of transitioning and delivery of virtual/mixed mode service provision, challenges encountered and recommendations to inform future online delivery of the program.

4.58 The annual plan for quality assurance work included ‘program delivery documents’ and later ‘projects’. As illustrated in Figure 4.1, the budget allocated to ‘additional services’, including program delivery documents, and later innovative projects (see paragraphs 4.65 to 4.67), increased over time.

4.59 The work completed through ‘program delivery documents’ does not support the quality assurance activities that monitor the performance of general service providers.

Impact of the COVID-19 pandemic on assurance activities

4.60 In March 2020, the department advised the minister that ‘non-critical work’ was being ‘put on hold’ in response to the COVID-19 pandemic. The list of paused non-critical work did not include any quality assurance activities.⁸⁵

4.61 The following month the minister was briefed on the department’s decision to ‘relax performance monitoring and compliance activities for the next quarter [… including the] suspension of file verifications (audits will not be scheduled for a later date) and onsite assessments for General and DL providers’. While the attachments to the brief informed the minister that the department had suspended application of general service provider key performance indicators two (attainment) and four (accuracy of assessment), no advice was provided regarding the decision to also suspend the quality assurance provider’s KPIs (see Table 4.1).

4.62 On 18 April 2020, the department decided to redirect planned quality assurance activities to instead support service providers ‘transition to Virtual Participation and Mixed Mode delivery of AMEP service’ for an additional cost of $125,909. The work order issued for that changed work did not include quality assurance activities that monitored the performance of general service providers and was issued under contractual provisions relating to ‘program delivery documents’ (see paragraph 4.57, fifth bullet point).

4.63 Two months later, on 25 June 2020, the department held a workshop (led by the quality assurance provider) with AMEP service providers to ‘explore possible interim methods of measuring language progression in the AMEP’. Six weeks later (12 August 2020), the department decided that the quality assurance provider would:

• ‘manage the implementation of interim Progressive Assessments, including … consultation with providers and the Department, to determine appropriate quality assurance verification activities, to be rolled out in Quarter two’;
• ‘work with providers to determine an appropriate verification process, once the implementation of the interim measures rolls out’; and
• return to the decision-maker in ‘Quarter 2 with a proposal for a more holistic QA that covers- teaching programs, targeting for cohorts, flexibility in delivery, promotional materials and referrals to DL (in consultation with [the quality assurance provider]’.

4.64 No variations were made to the contract with the quality assurance provider to accommodate these changes in service requirements.

Innovative projects

4.65 On 2 December 2021 the quality assurance contract was varied to allow the department to ‘invite’ the quality assurance provider to submit a proposal for Innovative Projects, a term not defined in the contract. Innovative Projects are also not defined or expanded on in the Quality Assurance Provider Instructions which support and clarify the delivery arrangements for the contract. Innovative Projects are defined in the AMEP general service providers contractual arrangements (see paragraphs 2.35 to 2.41).

4.66 After variation, the contract with the quality assurance provider required the department to invite the provider to submit a proposal for an innovative project. If the department agreed to accept the proposal, the department is required to issue a work order. The ANAO identified one such work order, which was varied four times.

4.67 Issued four days after the variation to the quality assurance provider’s contract, the work order was for the delivery of a ‘online platform and e-learning content’. The total budget for the work order was $1,598,763 (excluding GST), nearly four times the cost of all ‘program delivery document’ work planned to be completed that year (see Figure 4.1). The work was to be completed by 30 June 2022. By the end of the fourth variation, the budget for the Innovative Project was $5,454,545 (excluding GST), and delivery of the project was varied by a year, to 30 June 2023. The revised budget was nearly four times greater than the original budgeted cost, and 8 times larger than the cost of all other ‘program delivery document’ work planned for 2021–22 and 2022–23 financial years (see Figure 4.1). The quality assurance provider advised ANAO in February 2024 that the change in cost was ‘due to changes in project scope by the department.’

4.68 Further advice to the ANAO from the quality assurance provider in February 2024 was that:

LWA was hired for their expertise in the field and ability to verify educational outcomes of clients in the AMEP. We verified:

• Validity, appropriateness, and reliability of educational outcomes for the learner against their individual learning plan (accurate assessment KPI)
• Timeliness of educational outcome meeting program objectives (attainment KPI)

LWA would like to reapply our expertise to these objectives to ensure program tuition remains valid and appropriate for the AMEP cohort and timely and reliable within and across all providers.

Our current QA activities do identify issues for continuous improvement and reoccurring issues for timed rectification. However, we acknowledge that the current processes do not monitor the quality of services being delivered. Also, the percentage of files we currently verify does not reflect accurate provider risk ratings.

LWA would like to ensure the department retains Quality Assurance services which monitor the quality of services being delivered to students (qualitative KPI), and does not merely monitor quantitative KPIs, such a minimum of 90% of AMEP client exited by the service provider who have not completed a CSWE Certificate of the Pre-CSWE Course have achieved a Statement of Attainment (Service Provider KPI from 2011 AMEP contract).

Has the AMEP service provider performance been improved through the work of the quality assurance provider?

4.73 The Department of Finance’s Contract Management Guide states that ‘good performance management is key to delivering value for money’, and that performance measurement should be based on the performance framework included in the contract. For the AMEP general service providers, this framework included key performance indicators and quality assurance activities. The Contract Management Guide advises that entities should ensure information used to assessment performance is ‘accurate, fair and verifiable’.

4.74 As outlined in paragraphs 3.11 to 3.19, the performance of the general service providers has not been measured consistent with the contracts, and the four key performance indicators established have not been applied for the life of the contract. Only one measure of provider performance remains, and that is student commencements. This measure is not directly impacted by the quality assurance activities completed by the quality assurance provider. In these circumstances, there is insufficient evidence to analyse the performance of AMEP general service providers over the life of the contract.

4.75 The work of the quality assurance provider, as tasked by the department, has deviated from providing primarily quality assurance activities to the provision of professional development workshops, program delivery documents and an online platform for e-learning (see paragraphs 4.39 to 4.50). The volume and statistical significance of quality assurance activities reduced over time (see paragraphs 4.31 to 4.50) such that it cannot be relied upon to accurately and fairly represent performance. The effects of the ‘additional services’ on provider performance have not been measured by the department.

4.76 There is no performance indicator in the quality assurance provider’s contract that addresses the extent to which the contractor’s work has resulted in improvements to general service provider performance.

4.77 Surveys of AMEP students, seeking views on the performance and value of the program, have been conducted by the quality assurance provider since 2020. The benefit of these surveys is limited as:

• response rates have been between 15 and 20 per cent; and
• except for the survey conducted in May 2023, results were provided at a whole of program level, not provider level.⁸⁶

Have payments to the quality assurance provider aligned with the contract and the services provided?

4.78 Verification of invoices is important to ensure the proper use of Commonwealth resources. The Department of Finance’s Contract Management Guide states that departments should only made a payment when the service has been received and meets the required standards; the supplier is compliant with the contract; the invoice is accurate and correct according to the contract; and when all necessary authorisations and approvals have been obtained.

4.79 Invoicing and payments to the quality assurance provider has not consistently adhered to the contract with issues identified in a number of areas, including: monthly instalments and quarterly adjustments; engagement for services outside of the contract; application of goods and services tax; and invoicing and payment anomalies (see Appendix 6 for details).

Appendix 1 Entity responses

Department of Home Affairs response

ANAO comments on the Department of Home Affairs response

1. An important consideration in the ANAO’s conclusions that the design and administration of the contracts has not been effective relates to the Key Performance Indicators (KPIs) for service providers. Of the four KPIs identified included in the contracts that commenced in July 2017 to establish ‘a minimum performance standard’ that service providers were expected to meet, three are no longer in place, including the KPI relating to the desired program outcome of progressive English attainment by students (see paragraphs 3.11 to 3.20).
2. Four KPIs were identified in the request for tender conducted in 2016, and subsequently included in the contracts that commenced in July 2017 (see Table 3.1). As discussed at paragraph 3.11, the KPIs established ‘a minimum performance standard’ that service providers were expected to meet. The KPIs were also to form the basis for assessing provider performance individually and relative to other service providers, and form part of any consideration of contract extensions. Of the four planned and contracted KPIs, only one related to program outcomes, being progressive English attainment by students. As set out at paragraphs 3.11 to 3.20, there remains only one contractual KPI, further: none of the four KPIs were applied for the first 12 months of the contract term; one KPI (relating to data timeliness) has not been applied at all; the target addressing the KPI relating to the accuracy of service provider assessments of client learning outcomes was first reduced and was later paused in November 2021 for the remaining term of the contracts; and the English attainment progress KPI has been removed. The KPIs that have been removed include the one relating to progressive English attainment such that there is no longer a contractual framework under which the department is able to measure and manage provider performance relating to the extent to which clients benefit from AMEP learning activities. As set out in Appendix 3, Table A3.2, 1900 students were recorded as exited from the program as a result of no longer meeting the legislative eligibility threshold of functional or vocational English.
3. The impact of the COVID-19 pandemic on the program, and/or how the impact was addressed or not addressed by the department, is discussed throughout the audit report (paragraphs 1.4, 2.7, 2.26 and 2.27, 3.15, 4.25, 4.31, 4.36, 4.57, 4.60 to 4.68 and Table 3.1).
4. See report’s discussion on exceptional circumstances work orders at paragraphs 4.60 to 4.64 and Appendix 5, paragraph 10.
5. Administration of the AMEP contracts transferred into the Department of Home Affairs nearly five years ago, in July 2019. Relevant records and some key staff moved with the program to the Department of Home Affairs. It would be expected that, if there were further records not transferred at that time which the Department of Home Affairs required as part of its responsibilities for administering AMEP then the Department of Home Affairs would have sought those records from the Department of Education and/or the Department of Employment and Workplace Relations prior to the conduct of ANAO audit work in the second half of 2023 and into 2024. Home Affairs obtaining any missing relevant records would have been consistent with the ‘records follow function’ requirement of a Machinery of Government change.
   
   Changes made since 2019 by the Department of Employment and Workplace Relations to the parts of the head contracts that apply to both AMEP and SEE have occurred through engagement with, and input from, the Department of Home Affairs. This is consistent with the ‘Collaborative Agreement’ signed by the two departments in October 2019 under which Home Affairs agreed that the department responsible for the SEE program would be the lead agency, and that any variations to the general clauses in the contract must be agreed by both departments.
6. See footnote 28.
7. Section 30 of the Auditor-General Act 1997 provides that the information gathering powers of the Auditor-General are not limited by any rule of law relating to legal professional privilege (amongst other things). In short, this means that auditees cannot claim legal professional privilege to refuse access to information for audit purposes. In addition, subsection 30(2) provides protection to auditees to ensure that disclosure within the audit of otherwise privileged information does not waive that privilege for purposes outside the audit.

Linda Wyse and Associates response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s corporate plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO, or advised to the ANAO by Home Affairs, during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• Review of invoicing arrangements and validity of source data for quarterly adjustments (see Appendix 5, paragraphs 1-9).
• An improved process to monitor the quality assurance provider’s progress against contractual KPI requirements (see paragraph 4.8).
• The department remedied a repeated error made as a result of document conversion issues, that created inconsistencies identified the ANAO between the AMEP legislation, the department’s ‘Eligibility Chart’ as shared with AMEP providers in 2021 and 2023, and the table of visas within the information management system. The Chart was reissued during the audit, with the error remedied.
• Review of departmental communications between the program’s policy team, assurance team and operations team to inform a more holistic view of the program (see paragraph 4.10).

Appendix 3 Key program statistics

Table A.3: Number of students by visa type at date of registration for AMEP

Source: ANAO reproduction of Home Affairs report.

Appendix 4 Advisors contracted on the AMEP program (July 2017 to December 2023)

Note a: In November 2021 the value of this contract was increased by $82,547 to $115,265. The provider was re-engaged in 2022 to continue to host AMEPOnline, for $6,569. Variations to this second engagement increased the value of this second engagement by $13,137 to $19,706.

Note b: While noted as a potential probity risk, the department’s evaluation noted (positively) that one of the staff members at the preferred provider ‘has previous experience as a university teacher and as an AMEP volunteer home tutor’ but that none of the staff had ‘formal education or teaching qualifications’. The evaluation report also noted that the preferred provider has ‘benefitted from its previous work with the department on the multi-provider model — access to core documents and SPIs’ indicating that it had information advantages over other potential providers.

Note c: Life Course Centres delivered three projects between 2019 and 2022. The first of the projects was agreed and entered into prior to Machinery of Government changes and the department does not hold records of the procurement approvals. The agreement for provision of the first project expired ‘without the outcomes of the AMEP project being fully realised’. A second and third project were agreed to by Home Affairs with the aim of ‘realising the original intended outcomes of the AMEP project’ and these two projects have been incorporated into the table.

Note d: In response to questions from ANAO about the selection process for Monash University, the department cited email correspondence outlining an unsolicited proposal from the provider. In February 2024, the department advised the ANAO that ‘of particular note’ and ‘the key reason for the selection of Monash University’ was the experience of the ‘lead researcher’ who ‘had also previously worked with a Victorian AMEP provider delivering AMEP in an online environment’. Alternative providers were not considered by the department, and there was no market scan documented. The approvals to proceed with the project did not: identify that the project stemmed from an unsolicited proposal; address the lack of competition in selecting the provider; identify or address the probity considerations that arose as a result of the provider’s very recent AMEP work experience. As Monash University was engaged through LWA rather than directly by the department, the engagement of Monash University was not required to be reported on AusTender. In March 2024 the department advised the ANAO that not reporting the engagement on AusTender ‘was not the intent’ and the engagement had ‘progressed on the understanding this work was within scope of the Program Delivery Document services under the [Quality Assurance] Agreement’. The department has noted that it ‘will develop a Probity Plan for the management of Agreements to mitigate probity risks, including the potential conflict of interests as identified here’, that ‘there is scope to address VFM [value for money] more clearly’ and the department ‘will introduce stronger governance arrangements over the process by which it engaged service providers under the AMEP, under the existing panel.’

Note e: Scope of work was to ‘create and implement a transition-out plan which will detail the processes and activities’ and to ‘ensure a seamless transition of supplier services prior to the expiry of existing AMEP agreement’. In April 2024, Home Affairs advised the ANAO that: ‘the scope of the department’s contract with Callida is broader than assisting with transition planning, as has been highlighted in the Report. The scope of work also includes ‘to support timely implementation of the…AMEP reform project and support work as required’, which allowed the department to seek support from Callida in costing the new AMEP business model.’

Note f: Variation was made to this contract increasing the value to $300,000, see paragraph 2.62 and footnote 56. In April 2024, Home Affairs advised the ANAO that the variation and increase related to exercising an extension option.

Note g: In October 2022 the value of this contract was increased by $283,620 to $958,454.54. In April 2024, Home Affairs advised the ANAO that the variation and increase related to exercising an extension option.

Source: ANAO analysis of Home Affairs records.

Appendix 5 Invoicing issues identified with general service providers

Monthly instalments and quarterly adjustments

1. The approach to market set out that invoicing and payment arrangements for the AMEP general service providers would require that invoicing and payments be made in monthly instalments, with quarterly adjustments. Consistent with the approach to the market, the contracts signed with the successful tenderers reflected the approach to market in that it provides for monthly instalment payments of fees based on an estimated fixed fee, with quarterly adjustments.

2. The estimated fixed fee used by the department to determine a quarter’s first two monthly invoices is based on ‘the previous year’s expenditure divided by twelve months’. This is recorded in internal departmental guidance, finalised in January 2024. Neither this methodology, nor the resulting amount, are recorded in the AMEP general service provider contracts or AMEP Service Provider Instructions.

3. The third quarterly fee is an ‘adjusting’ invoice, in that the department determines, based on provider reported delivery over the quarter (reported actual volumes and classification of tuition and child care), the total amount that a provider should be paid for the quarter, less that already paid in Month 1 and Month 2 of the quarter. The quarter’s third invoice is for the balance. As such, over payments in Month 1 and 2 of the quarter would be reflected in a smaller Month 3 payment, while under payments in Month 1 and 2 of the quarter would be reflected in a high payment in Month 3.

4. While not covered by the AMEP general service provider contract nor the AMEP Service Provider Instructions, internal departmental guidance allows for the fixed monthly rate to be adjusted ‘where there is significant over or underspend through the quarterly reconciliation process, indicating a change of the demand for the delivery in the regions and adjustments to the fixed payment is imminent.’

5. ANAO notes that the methodology to determine the fix monthly price is flawed as taking ‘the previous year’s expenditure’ and simply divided by 12 does not allow for:

• the Wage Price Index planned for in the contract to be applied to fees on 1 July each year and, as such, the previous year’s expenditure would be best estimated to also increase by the same index;
• demand for the program including impacts of changes to migration rates, rates that the Department of Home Affairs reports to the ABS and historical AMEP enrolment rates; and
• variations made to fees throughout the year, that impact on the total expenditure (for example, in December 2020 agreeing to vary fees for providers would have a six-month impact on the 2020–21 financial year and an annualised impact on the 2021–22 expected total cost).

Reliance on unverified data

6. As detailed in Chapter 4, the quality controls and verification activities required by the department from the quality assurance provider have declined over the life of the AMEP quality assurance Contract. Not only did this work inform the assessment of general service provider performance against the fourth key performance indicator (accuracy of assessment) it also provided a degree of assurance over the accuracy of the data used for invoicing and payments (see paragraphs 3.2 to 3.9).

7. The ANAO raised with the department the lack of control and verification of provider inputted data, and reliance on its accuracy for reporting, measurement and payments. In December 2023, the department advised the ANAO that it had identified a potential issue that it was working to quantify. The department identified that controls and verification of data used for the quarterly reconciliation were insufficiently robust, and lead to a potential risk of duplicated or repeated payments, or alternatively missed payments.

8. In determining the quarterly adjustment invoice, the department relies on the data input by providers. The department requires, through the AMEP General Service Provider Instructions, that providers ‘enter all AMEP activity data for each month into [the information management system] by the 10^th Business Day of the following month.’ While the department may apply contract clauses to enforce compliance it has instead relied on invoice generation to motivate compliance with the timeframe.

9. The department identifies the adjustment invoice amount based on a point in time extract from the department’s reporting system. The department has not reconciled this report to previous reports to ensure that payments are not duplicated. For example, commencement fees are to be paid (in general) once per student. In August a provider may claim to have commenced a student. In September’s adjusting invoice this was included in the report and paid. In October the provider may change the date of the commencement of the student, and so appear to have claimed commencement for the student in October.⁸⁷ In December’s adjustment, this same student would appear in the report generated of activities for payment, and the commencement fee paid again. The department does not have controls between the AMEP management system and its financial management system to prevent duplicated payments. The department has commenced application of an interim manual monitoring process to reconcile reports between quarters. As at December 2023, the department had ‘identified discrepancies in the last financial quarter of 2022–23 was approximately $4000’. The department has not yet completed a full review of all payments in all fee types across the years to quantify the full impact. In February 2024, the department advised the ANAO that it has also commenced:

• running a ‘final reconciliation report for the year’ to identify ‘where duplicate payments (quarterly and monthly) have occurred and payments adjusted’;
• providing ‘client detailed reports’ to providers to enable them to ‘reconcile and check their local records’; and
• use of a system script ‘to rule out the second Initial Assessment data for the same client for expenditure calculation’ as well as running ‘point in time ad hoc reports to examine changes to records (possible duplicates) overtime’.

Services outside of the contract

10. The ‘Exceptional Circumstances’ Work Order was issued between 8–10 July 2020, for COVID-19 adjusted services to be delivered from 1 April 2020 (three months prior to the work order being issued) to 30 September 2020. This work order essentially added additional services and associated fee schedule to the arrangement with all AMEP general service providers for a fixed period, and included four options to extend the work order for a period of three months each. The Exceptional Circumstances Work Orders were all extended on 1 September 2020 to 31 December 2020.

11. It is not clear that there was sufficient and appropriate authority within the contract to develop such a work order, and to vary the fees in such a manner. Internal advice from April and May 2020:

• recommended that the program team ‘conduct a detailed review of the AMEP Agreement at first instance, to determine if the requirements … set out in the Agreement … are suitable, or if a variation to the AMEP Agreement itself (through a Deed of Variation) may also be required’;
• noted that the ‘AMEP Agreement contains no force majeure clause, or options for [service] postponement, which may have otherwise assisted’ to response to the COVID-19 pandemic circumstances; and
• advised that the AMEP Agreements at the time include a ‘narrowly defined’ option for virtual participation, and broadening the definition through a deed of variation could accommodate the department’s needs, noting that the ‘negotiated Fees for each service provider to undertake virtual services should be introduced within these DoVs at Chapter 4 of the AMEP Agreement under a separate subheading along the lines of “Virtual participation Fees”.

12. The department decided to vary the AMEP Agreement to broaden the definition of virtual participation. It did not seek to vary the agreement to included virtual participation fees as ‘the fees [are] not necessarily appropriate beyond this situation’.

Indexation

13. The contract stipulated that the fees in the fee schedule were to be indexed on 1 July each year ‘in accordance with the Wage Price Index (WPI) for all industries and sectors for total hours, but excluding bonuses, using the March-to-March increase published in May of each year’ for services rendered on or after 1 July.

14. The Australian Bureau of Statistics (ABS) provides three version of the WPI: the original; the seasonally adjusted; and the trend. The contract does not specify which version of the WPI the department will use to index the fees.

15. In December 2023 the department advised the ANAO that it uses the seasonally adjusted version of the WPI. The evidence cited by the department was an email from October 2016 (prior to execution of the current contracts, which occurred in April 2017) which stated:

Seasonally adjusted or original series are unlikely to have much if any variance as it is comparing the same quarter, but most people are comfortable with seasonal adjustment so I thought we should go that way … I recall this being agreed at a TWG meeting.

16. The department’s internal AMEP finance manual from October 2023 does not make it clear which of the three ABS WPI indices to use.

17. The ANAO’s analysis is that, given the long term nature of the AMEP contracts, and the design of the term scheduling in AMEP, the program and its fees are not impacted by ‘seasonal influences’ described by the ABS: ‘the effect of the weather, social traditions, or administrative practices’ or ‘other kinds of calendar related variations, such as the number of trading days, Easter or the proximity of significant days in the year (e.g. Christmas)’.

18. The WPI Original Index and the WPI Seasonal Index were different in only two years over the life of the contract, at the end of 2019–20 and at the end of 2022–23 (impacting the expected fees for the 2020–21 and 2023–24 financial years). This has an impact on alignment of expected fees from 1 July 2020. Between 1 July 2020 and 30 June 2023, for the 2020–21 financial year:

• services were varied, through the implementation of the ‘Exceptional Circumstances Work Order’.⁸⁸ This had the effect of implementing a second service and associated fee schedule, the basis of which was used to determine invoice amounts. The ‘Exceptional Circumstances’ Work Order included two service and associated fee tables, one for the period 1 April 2020 to 30 June 2020, and another for the period 1 July 2020 to 30 September 2020.⁸⁹ As such the effect of inconsistent application of the WPI index was removed from invoicing for this period; and
• fees were varied to existing services through a deed of variation on 17 December 2020. This has the effect of removing the inconsistent indexation for those AMEP General Services included in the contract.

19. ANAO also examined the 2023–24 fee schedule generated by the 2022–23 WPI Index. A 0.1 per cent difference between the two ABS indices, resulted in a difference between what the fees were expected to be, and what the department used to determine the fees payable. For the first quarter this resulted in a total difference, across all provider invoices of $83,488.40. Variations to fee schedules were agreed through deeds of variation with two providers, one effective as at 1 August 2023, and the second effective as at 1 January 2024.

20. The application of the seasonal WPI index will create continued divergence between the contract’s expected fees schedule and the fee schedule developed and implemented by the department.

21. The agreements through correspondence (see paragraphs 2.15 and 2.28) with the contractor have been incorporated by the department into the fee schedules applied when generating invoices for quarter one payments, these variations through correspondence are contract variations however are not recorded in the department’s Contract Management Tracking Register.⁹⁰

GST

22. Part 2, paragraph 4.5 of each of the General AMEP service provider contracts set out the fees (GST exclusive) that each provider may claim for the range of AMEP Services. Each provider’s fees are different, and specific to individual contract regions. Each provider’s contract only includes the fees for the contract regions that they have been contracted to service.

23. The AMEP providers are required to all be registered training organisations, and are a mix of state/territory government entities and private sector providers. In February 2017, prior to execution of the AMEP contracts, the Australian Taxation Office issued a private ruling to the department to determine whether GST was payable by AMEP general service providers on any of the AMEP general services. The Australian Taxation Office determined that the supply of AMEP General Services is for a GST-free service, as ‘an education course … under section 38-85 of the GST Act’. The advice noted that a supply of services is ‘GST-fee’ if the services is an ‘education course’ or ‘administrative services directly related to the supply of such a course’. As such the when the department generates the monthly invoices (including adjustment invoices) that it seeks the providers to issue so the department may make payment, all invoices are exclusive of GST.

24. The ruling from the Australian Taxation Office came before innovative projects were added to the AMEP general service provider contracts through the second deed of variation made between 2 May and 4 June 2018. Although innovative projects are not required to be an education course, an updated ruling was not sought to address GST payments on innovative projects.

25. Reports extracted from the department’s finance management system record an inconsistent treatment of innovative projects. Reports provided by the department in March 2024, the system recorded 42 purchase orders, for 43 innovative projects, of which 12 are categorised as to have GST applied⁹¹ and 30 are categorised as not to have GST applied.

26. An additional three work orders for innovative projects to be delivered by three providers between 16 December 2019 and 30 June 2020, were not clearly described in the department’s finance management system as innovative projects. Instead these three purchase order were given, and continue to have, descriptors which implied they were for AMEP general services, all three recorded GST to be applied. The ANAO notes that GST was included in the fee schedule for all three work orders:

• One innovative project was for a provider to develop a ‘realistic workplace introduction Program where students develop their language, numeracy and workplace knowledge and skills, out of the classroom’. This innovative project was expected to cost of $87,968 (excluding GST).
• A work order for another provider was executed to deliver a program that focused on ‘developing English language and numeracy skills for the hospitality industry’ where students would be ‘immersed in a working kitchen, allowing them to understand the expectations and requirements of the Australian workplace.’ This was to cost of $89,550 (excluding GST).
• Another work order for the second mentioned provider was executed for a program focused on transitioning ‘AMEP Mature-Age’ student to work for a cost of $83,776.54 (excluding GST).

Backdating of invoices to accommodate variations and in-principle agreements

27. As early as March 2018, internal advice was being sought on the validity of varying fees and backdating an increase to fees to the beginning of the contract. The advice from the procurement team was that the department’s procurement and contract policy ‘does not stop you from taking action to increase pricing on fees or indeed backdating an increase to fees’ but that ‘ultimately it is for the delegate to agree to any recommendations taking into account legal and policy advice plus your expertise in the management of the programme and desired outcomes.’

28. The ANAO, in locating the agreements through correspondence, identified seven letters with four providers. Of these letters, five (with three providers) specifically state that payments would be backdated to accommodate changes to the fee schedule. The ANAO identified a further two letters (with two providers) where retrospective changes were made to the fee schedules, but has not been clear as to whether backdated payments would be made for services already rendered.

Fees outside of written agreements

29. Over the period 1 July 2019 to 30 June 2023, the fees used to generate quarter one invoices for two providers did not align with either the contract (and expected indexation, including application of seasonal index) or agreements made through correspondence:

• The fees for one contract region were not correctly updated in the invoice to reflect the December 2020 variation to fees. The hourly tuition fees were less than agreed in the contract. For the Quarter 1 invoice this resulted in an underpayment of $5,200. When identified by the provider, the department undertook to rectify the error in the following quarterly payment.
• Child care fees for another contract region did not align with contract or agreements through correspondence for the 2019–20 Quarter 1 invoice, and was overstated in the 2020–21 Quarter 1 invoice as well. The contract region impacted is a very small contract region. The department is responsible for maintaining an accurate fee schedule, and using that fee schedule to determine invoice amounts. If this same overstatement occurred in the provider’s other contract region, or with the providers detailed above and below, the impact would be significant.
• Continuation of fees agreed through correspondence beyond the agreement’s expiry date:
  • The child care fees for another provider were increased for the period 1 July 2018 to 30 June 2020 through correspondence. The department continued to apply the increased fee as the basis of invoices, despite no variation made to the contract, and no additional agreement through letters to extend the term of the agreement. As a result, the invoice generated for Quarter 1 calculated a child care administration fee of $624,045, which is $366,205 greater than fee provided for by the contract.
  • The initial assessment fee for another provider was overstated in its 2020–21 Quarter 1 invoice. The provider was party to an expired agreement through correspondence, with the fee increased from that expired agreement subsequently incorporated into the December 2020 variation. The amount used to generate the invoice for Quarter 1 was either the expired rate from the agreement through correspondence or the December 2020 variation rate, despite not yet being in effect. As a result, the invoice generated was overstated for the commencement of five students that quarter.
  • The child care fee for another provider was overstated for the 2020–21 Quarter 1 Invoice. The fee used reflects an agreement made through correspondence In June 2019, where the higher fees were only to apply until 30 June 2020. As result, the invoice generated for Quarter 1 was overstated by $70,005. No variations were made to the contract, and there were no additional agreements through correspondence to extend the term of the arrangement.

Appendix 6 Invoicing issues identified with quality assurance provider

Monthly instalment payments and quarterly adjustments

1. The approach to market set out that invoicing and payment arrangements for the quality assurance provider would be similar to those for the General Services Providers, and this was the basis on which respondents tendered. Specifically, that invoicing and payments would involve monthly instalments and quarterly adjustments. Consistent with the approach to the market, the contract that was signed with the successful tenderer reflected the approach to market in that it provides for monthly instalment payments of fees based on the annual plan, with quarterly adjustments.

2. The invoicing and payment arrangements do not involve monthly instalments based on the annual plan with a quarterly adjustment. Rather, the quality assurance provider invoices the department for the services provided in the prior period (which is often one month but has been longer).

Engaging for services outside the contract

3. Agreeing services and fees outside of formal contracting arrangements undermines the integrity of a contract.

4. Under the contract, the annual plan is to perform a key role in the invoicing and payment arrangements. The contract states that the quality assurance provider ‘has been engaged to provide Quality Assurance for AMEP Services’ as described in the Part 3 of the contract upon receipt of a work order issued by the Commonwealth. The quality assurance provider is to submit an annual plan to the Commonwealth for approval (clause 2.3), and the Commonwealth may approve that plan or make amendments to that plan. Quality assurance services to be ‘incorporated into any Work Order’ are specified in the contract (clause 2.7). Where services outside of the specified quality assurance services are identified as required, the department is to specify the scope of services, seek a quote, and on acceptance of the quote issue a work order.

5. Notwithstanding the terms of the contract, the annual plan has been used by the department to significantly vary the work to be delivered by the quality assurance provider such that quality assurance comprised only 15 per cent of work in 2023–24, compared to 78 per cent in the first year of the contract (as illustrated above by Figure 4.1). In the Annual Plan approvals, no value for money considerations were documented, and no reference was made to deviations from the original scope of the requested works at the time of tender evaluation. By way of example:

• In 2021–22, five online professional development workshops were included on the invoice, despite the contract’s fee schedule never including a fee for such a service.
  • In December 2023, the department advised ANAO that the ‘2021–22 Annual Plan provided for Online Workshops were approved as part of the Annual Plan, not via a formal variation.’
  • The department advised ANAO that this approach was consistent with clause 2.3 of Part 3 of the Agreement. Clause 2.3 of Part 3 of the Agreement requires the preparation of an Annual Plan and this clause does not allow for variations outside the specified quality assurance services outlined in the contract. Rather, the contract requires that work outside of the specified services (specifically program delivery documents) are to be quoted on, approved, and a work order issued.
• In 2022–23, two Communities of Practice Workshops held online were included on an invoice, despite the contract’s fees schedule not including a fee for such a service. ANAO could not locate a separate work order or quote for such a service. ANAO notes that the fee for these workshops was greater than the fees for the online workshops invoiced for in 2021–22.
  • In December 2023, the department advised ANAO that the ‘22-23 Annual Plan provided for COP Workshops were approved as part of the Annual Plan, not via a formal variation.’
• In 2019–20, invoicing for a Professional Development workshop to be held in Orange NSW, when the contract does not specify a rate for Orange NSW. ANAO could not locate delegate approval for a variation to the contract, or for an in principle (outside of contract) agreement to use another region rate. Previously the department had varied the contract to incorporate Wagga Wagga as a costed region when a workshop was to be held there.
  • In December 2023, the department advised the ANAO that ‘Orange has not been added to the Agreement’ and that instead this was ‘approved via the 2019–2020 Annual Plan process’, where the invoiced amount was included in the approved Annual Plan. The ANAO notes that the Annual Plan processes does not allow for variations outside the specified quality assurance services outlined in the contract, and that work outside of the specified services (specifically program delivery documents) are to be quoted on, approved, and a work order issued. There was no separate work order, or variation to existing work order to incorporate Orange into the contract.

GST

6. The approach to market enabled respondents to tender fees that were either GST inclusive or GST exclusive, or both. The contract provides that the fees are on a GST exclusive basis, as it states that an additional amount must be paid to the supplier for any taxable supplies.⁹² Notwithstanding the clear terms of the contract, invoicing and payments have been administered on the basis that the contracted fees are inclusive of GST.

7. In responding to the ANAO on this issue, the department did not reference the terms of the contract instead citing an email from 2017 where an acting Director of the AMEP’s ‘Assurance Unit’ sought confirmation from a team member that the fee schedule aligned with the contract, and advised the junior team member that ‘the prices in the attachment [the fee schedule] are gst inclusive.’

8. The junior team member identified misalignment between item costs. The acting Director then replied with a ‘final version’ of the prices for contracted quality assurance services. This final version stated that the fees included GST. ANAO notes that neither party to this correspondence had authority to vary the contract.

Invoicing and payment anomalies

9. Errors were identified by ANAO in the sample of invoices examined. These included:

• In 2017–18, incorrectly charging travel to Regional NSW for activity in the ACT, when the fee schedule specified a lower rate for ACT travel. This error was not corrected by the department.
  • In December 2023, the department advised ANAO that this ‘may be an error that was not picked up when the invoice was progressed for payment in 2017, however due to passage of time and MOG changes we are unable to verify this.’
• In 2018–19, incorrectly charging travel to Regional QLD for onsite quality assurance activity of the Distance Learning provider that is based in metropolitan NSW.
  • In December 2023, the department advised ANAO that supporting evidence from the time recorded that the travel taken was only to NSW not Queensland and that the invoicing ‘may be an error that was not picked up through invoice payment process, however due to the passage of time and MOG changes we are unable to verify this.’
• In 2019–20, unexplained and unevidenced increase to the fee for Work Experience File Verifications.
  • In December 2023, the department advised ANAO that ‘this may be an error that that was not picked up when the invoice was progressed for payment.’
• For 2020–21, the delegate approved on 4 September 2020, an annual plan with a fee schedule that did not align with the contract terms (indexation and GST application). The contract manager (who did not have sufficient delegation) subsequently confirmed with the quality assurance provider the fee schedule on 1 October 2020.⁹³ No subsequent deed of variations were entered into to reflect this agreement from the delegate or the contract manager in the contract.
  • In December 2023, the department advised ANAO that ‘Approval of the 2020–21 QA Annual Plan … was completed on 4 September 2020’ and that ‘[the contract manager] did not seek to vary the agreed rates through this exchange’. The department noted that the rates referred to by the contract manager were the same ‘as approved by the delegate on 4 September 2020’.
• In the 2020–21 annual plan the cost of the Assessment Task Bank Management was set to $299,839. This does not align with the expected annual fee of $306,136. The following year, the annual plan reduced the fee below the expected fee to $282,480. No records could be identified to explain these changes, with the department confirming that ‘a data entry error … led to the misalignment between the ex-GST rates as documented in this Plan’ and the fees in the contract.