Introduction

1.1 The Australian Skills Quality Authority (ASQA) was established as the national vocational education and training (VET) regulator under the National Vocational Education and Training Regulator Act 2011 (NVR Act), and forms part of the Education, Skills and Employment portfolio. ASQA is a non-corporate Commonwealth entity under the Public Governance, Performance and Accountability Act 2013.

1.2 ASQA’s purpose is ‘to provide nationally consistent, risk-based regulation of VET that contributes to an informed, quality VET sector that meets Australia’s needs.’² ASQA is an independent statutory authority established to regulate:

• providers that deliver VET qualifications and courses;
• providers that deliver VET courses to overseas students;
• accredited VET courses; and
• certain providers that deliver English Language Intensive Courses to Overseas Students.

1.3 ASQA does not regulate Registered Training Organisations (RTOs) in Victoria or Western Australia that do not offer courses outside of those jurisdictions.³ These organisations are regulated by the Victorian Registration and Qualifications Authority and the Training Accreditation Council respectively.

1.4 ASQA’s regulation of the VET sector is supported by a framework of legislation and standards. The relevant legislation and standards are detailed in Appendix 2. In particular, the purpose of the Standards for VET Regulators 2015 is to ensure ASQA:

• uses a risk-based approach to regulation that is consistent, effective, proportional, responsive and transparent;
• has consistency in implementation and interpretation of the RTO Standards and the Standards for VET Accredited Courses; and
• has accountability and transparency in undertaking its regulatory functions.

1.5 The Standards for VET Regulators 2015 describe a series of outcomes that ASQA must achieve as a regulator, but do not prescribe how to achieve these outcomes. The six outcomes to be achieved under the standards are illustrated at Figure 1.1.

Figure 1.1: Standards for VET Regulators which ASQA must achieve

Source: ANAO analysis of the Standards for VET Regulators 2015.

ASQA clients

1.6 In 2019, an estimated 4.2 million students were enrolled in VET with an Australian training provider. Of those VET students: 52.0 per cent were male; 3.8 per cent identified as Indigenous; 5.0 per cent reported having a disability; and 4.4 per cent were international students.

1.7 In 2019, 3.0 million students (71.4 per cent) were enrolled in nationally recognised training at private training providers, 779,200 (18.6 per cent) at TAFE institutes, 489,100 (11.6 per cent) at community education providers, 124,400 (3.0 per cent) at enterprise providers, 108,000 (2.6 per cent) at schools and 77,600 (1.8 per cent) at universities.⁴

Impetus for reform

Review history

1.8 Reviews of the VET sector and ASQA since 2017 have contributed to the reform objectives of ASQA. In June 2017, the Australian Government commissioned a review of the NVR Act. The review was part of the government’s commitment to ensuring the quality of the national VET sector into the future. The report, All eyes on quality: Review of the National Vocational Education and Training Regulator ACT 2011 (Braithwaite Report) — released in June 2018 — assessed the suitability of the legislative framework and its capacity to support a responsive, effective and efficient approach to regulation. The Braithwaite Report made 23 recommendations intended to drive continuous improvement in the quality of education and training, and the protection of students’ rights, both of which are integral to desirable student outcomes.

1.9 In March 2019, the report Strengthening Skills: Expert Review of Australia’s Vocational Education and Training System (Joyce Review)⁵ was released to the Department of Education, Skills and Employment (the department). The review recommended a range of systemic policy and structural reforms targeted at boosting confidence and strengthening the sector’s ability to deliver skills now and into the future. The Joyce Review identified the need for ASQA to reform elements of its regulatory approach, including its engagement with the sector and its educative approach, as well as consistency of regulation.

1.10 In March 2020, the Rapid Review of the Australian Skills Quality Authority’s Regulatory Practices and Processes (Rapid Review)⁶ was reported to the department.⁷ The purpose of the review was to evaluate the effectiveness of ASQA’s internal regulatory practices and processes. The scope of the review encompassed:

• evaluate the effectiveness of ASQA’s internal regulatory practices and processes (including ASQA’s Regulatory Risk Framework, how it undertakes audits, how it makes and internally reviews regulatory decisions, and processes related to the review of ASQA decisions)
• identify and recommend any changes to ASQA’s processes to support consistent, risk-based and contemporary regulatory decision-making and education
• identify and recommend any changes required to ASQA’s governance arrangements to clarify roles and responsibilities, improve accountability, improve efficiency of resources, and improve focus on strategic direction and performance
• identify areas in which changes could be made to the VET Quality Framework to drive improvements across the sectors.

1.11 The Rapid Review made 24 short, medium and longer-term recommendations to improve ASQA’s regulatory practices and processes.

1.12 A chronology of reviews and key events with relevance for ASQA is illustrated at Figure 1.2.

Figure 1.2: Chronology of reviews and key events

Source: ANAO analysis of publicly available information.

Rapid Review findings and recommendations

1.13 The focus of the Rapid Review was on how ASQA’s regulatory practices and processes might be improved. A number of areas requiring ‘significant adjustments’ were identified, including that ASQA:

• promote and facilitate a culture of effective self-assurance as an integral part of a provider’s routine operations;
• drive cooperative relationships between ASQA, providers and other stakeholders;
• consistently understand provider performance against standards and in the context of their operating environment;
• embed a variable-touch approach to regulation, guided by regulatory necessity, risk and proportionality;
• communicate regulatory decisions and advice in a way that is clear, consistent and timely;
• coordinate and appropriately share information with other regulatory agencies, policy makers and funding bodies; and
• restructure governance.

1.14 The 24 recommendations were grouped using nine theme areas. These theme areas are summarised in Figure 1.3. A complete list of the recommendations can be found at Appendix 3.

Figure 1.3: Rapid Review recommendation theme areas

Source: ANAO analysis of the Rapid Review.

1.15 The Rapid Review recommendations are not solely the responsibility of ASQA. There are a number of recommendations which rely on actions to be led by the department, for example amendments to the NVR Act and supporting legislation; and consultation, review and updates to relevant standards including the Standards for Registered Training Organisations (RTOs) 2015 and the Standards for VET Regulators 2015. The amendments to enabling legislation and standards will impact on how ASQA’s regulatory functions are performed and what outcomes are to be achieved. It will also impact on the timing in which ASQA is able to fully implement the Rapid Review recommendations.

1.16 The Rapid Review identified a number of instances where ASQA’s service standards were not met. For example, ASQA’s service standards in 2018–19 required that providers receive an interim audit report within 30 calendar days of the site visit; but in 2018–19, this timeframe was only met for 16.7 per cent of audits for initial applications, and 36.2 per cent of compliance monitoring audits and post-initial registration audits.

1.17 ASQA must meet its service standards under paragraph 157(1)(o) of the NVR Act. ASQA has continued to monitor the achievement of its service standards as part of its annual performance reporting. ASQA’s 2019–20 Annual Performance Statement identified that whilst its performance had improved in the reporting period, it continued to fail to meet its service standards. For example, for audit reports, ASQA met its 2019–20 service standard for timeliness 59 per cent of the time.

Reform funding

1.18 To support its reform activities, ASQA was provided with $24.1 million over five years (2019–20 to 2023–24). This included $3.7 million in capital funding (see Table 1.1).

Table 1.1: Funding provided to assist ASQA’s reform activities

Source: ANAO analysis of funding proposals.

1.19 The funding was provided as part of two policy proposals:

• Skills Package — Strengthening the national training system ($8 million); and
• Skills Recovery — Getting Australians back into jobs — VET, Skills and Employment Service Options ($16.1 million).

Rationale for undertaking the audit

1.20 The ANAO’s Annual Audit Work Program is designed to anticipate and respond to current and emerging risks and challenges impacting on public administration. In determining the 2020–21 audit work program, ANAO considered prior-year audits and other review findings and what these indicate about portfolio risks and areas for improvement. This included consideration of the planning and implementation of reforms by agencies. Auditor-General Report No. 31 of 2016–17 Administration of the VET FEE-HELP Scheme identified that the VET FEE-HELP scheme was not effectively designed or administered.

1.21 ASQA was established as the national VET regulator under the NVR Act. The Rapid Review identified the need for ‘significant adjustments’ to ASQA’s regulatory approach. ASQA has been undertaking reform activities since April 2020. ASQA is nearing the completion of the first year of the reform program and the audit provides an opportunity to consider its design and early implementation.

Audit approach

Audit objective, criteria and scope

1.22 The audit objective was to examine the effectiveness of ASQA’s planning and implementation of reform to the regulation of the VET sector. To form a conclusion against this objective, the following high-level criteria were applied:

• Does ASQA’s strategic planning effectively address the requirements for reform?
• Is ASQA effectively transitioning to the proposed regulatory approach?
• Has ASQA effectively engaged with the VET sector about its transition to the proposed regulatory approach?

Audit methodology

1.23 The ANAO:

• examined documentation held by ASQA including strategy, policies, procedures, frameworks, guidelines, risk, regulatory frameworks, stakeholder engagement and internal reporting;
• reviewed specific documentation relating to action plans and project management to implement reform initiatives; and
• interviewed relevant entity executive and staff.

1.24 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of $308,000.

1.25 Team members for this audit were Axiom Associates Pty Ltd, Zoe Pilipczyk, Dr Cristiana Linthwaite-Gibbins and Alex Wilkinson.

2.1 The Rapid Review of the Australian Skills Quality Authority’s Regulatory Practices and Processes (Rapid Review) has triggered the Australian Skills Quality Authority (ASQA) to reform its regulatory model for the vocational education and training (VET) sector. ASQA is transitioning from a focus on compliance to a focus on working with the VET sector to support providers in building capability for self-assurance.

Self-assurance is not the same as self-regulation. Self-regulation often involves industry taking responsibility for regulation, with government having a limited role. The concept of self-assurance requires providers manage their business to ensure a focus on quality, continuous improvement and ongoing compliance with standards set by governments. The role of ASQA, as the regulator is to promote this culture, to monitor each provider’s performance against the Standards, to identify any non-compliance and to support providers to return to compliance (or to take necessary sanctions action where this is not possible).⁸

2.2 This chapter examines whether ASQA’s strategic planning and governance effectively addresses the requirements for reform.

Does ASQA’s strategic planning support the implementation, monitoring and reporting of the reform?

2.3 Planning is integral to the establishment of an organisation’s strategic priorities, its management of budgets and its program and service delivery approach. This section reviews whether ASQA’s strategic planning supports the implementation, monitoring and reporting on the reform of VET sector regulation.

Strategic planning

2.4 The corporate plan is the primary planning document published by an entity, setting out the purposes, the operating context in which it will operate, the key activities it intends to pursue, and how performance will be measured and assessed over the reporting periods.⁹

2.5 In addition to ASQA’s purpose outlined in paragraph 1.2, ASQA’s 2020–21 Corporate Plan (dated 30 June 2020) states:

ASQA operates in a complex VET environment that is currently undergoing considerable change, with the redesign of almost every aspect of the VET sector through a comprehensive program of reform being implemented by the Australia Government and various state and territory governments.¹⁰

2.6 The strategic approach outlined in the Corporate Plan includes six elements to address ‘risk-based regulation’ and ‘engagement and support’ (see Figure 2.1).

Figure 2.1: ASQA’s strategic approach for 2020–21

Source: ANAO analysis of 2020–21 ASQA’s 2020–21 Corporate Plan.

2.7 For each of these six elements, ASQA identifies planned activities for 2020–24, reform initiatives for 2020–21¹¹ and 2020–24¹² and key strategies for 2020–21.

2.8 The Department of Finance (Finance) advises that ‘a corporate plan can be varied during a reporting period if an accountable authority considers it appropriate.’¹³ ASQA’s operational environment continued to change to the proposed regulatory model after the 2020–21 Corporate Plan was published. Despite Finance’s advice, the Corporate Plan was not updated and in January 2021, ASQA prepared the ASQA’s ‘Four Year Plan’ (an internal document).

2.9 The Four Year Plan was intended to deliver against the Ministerial Statement of Expectations National Vocational Education and Training Regulator¹⁴, and to address the Rapid Review’s recommendations. The Four Year Plan outlines six strategic deliverables, and performance indicators to evaluate ASQA’s success (see Figure 2.2). The Four Year Plan is neither linked to the 2020–21 Corporate Plan, nor does it outline reform activities beyond 2020–21.¹⁵

Figure 2.2: ASQA’s Four Year Plan strategic deliverables

Source: ANAO analysis of ASQA’s Four Year Plan.

2.10 For this year of transition to the proposed regulatory approach (see paragraph 2.1), ASQA is monitoring the outcomes outlined in both the 2020–21 Corporate and Four Year Plan. During the audit, ASQA provided the ANAO with a document that identified how each of the strategic deliverables were associated with the Rapid Review recommendations. ASQA has not identified how the strategic deliverables would address the Rapid Review’s recommendations.

Reform time-period

2.11 ASQA determined that a four year approach to the reform was required to fully address the recommendations of the Rapid Review. This will allow ASQA to work with other stakeholders, such as the Department of Education, Skills and Employment (the department), to ensure the new approach to regulation is reflected in relevant legislation and standards, including the Standards for Registered Training Organisations (RTOs) 2015, and the Standards for VET Regulators 2015. ASQA has established regular meetings with the department to facilitate this process.

2.12 ASQA prepared an ASQA Rapid Review Recommendation Implementation Tracking (Implementation Tracking) document which identifies when implementation activities to address the Rapid Review recommendations will commence. It identifies whether the implementation will commence in Wave 1¹⁶ or in future waves of change (2021–2025). In March 2021, the Implementation Tracking document identified that the implementation of 11 recommendations would commence in Wave 1; actions to implement a further 12 recommendations would commence in future waves; and one recommendation was not assigned to a wave as it was to be led by the department.

2.13 During the audit, ASQA developed the Rapid Review Recommendations: Overview document. This document included expected timeframes in which the implementation of each Rapid Review recommendation will be completed. However, ASQA has not yet identified the actions, deliverables or timing of activities to deliver each of the accepted recommendations.

Alignment with the Australian Government’s Regulator Performance Framework

2.17 As part of ASQA’s reform activities, ASQA must ensure that its new regulatory approach aligns to the Australian Government’s Regulator Performance Framework.¹⁷ The Regulator Performance Framework was introduced in 2014 as part of the Australian Government’s red tape reduction programme.

The Government has committed to reducing the cost of unnecessary or inefficient regulation imposed on individuals, business and community organisations by at least $1 billion a year. In order to achieve the Government’s goal of reducing the burden of regulation, it will be essential to improve the performance of regulators, including by supporting regulators to adopt consistent, risk-based approaches to administering regulation.¹⁸

2.18 The Regulator Performance Framework provides a structure to allow regulators to report objectively on the outcomes of their efforts to administer regulation fairly, effectively and efficiently. The Regulator Performance Framework requires ASQA to assess its performance through annual externally validated self-assessments. All regulators subject to the Regulator Performance Framework must self-assess their performance once every 12 months.¹⁹

2.19 ASQA’s last completed regulator performance self-assessment was for the period 2017–18. For the periods 2018–19 and 2019–20, ASQA linked the six key performance indicators under the Regulator Performance Framework to the performance measures outlined in its 2018–19 and 2019–20 corporate plans, and annual performance statements. The regulatory performance measures used, and the level of analysis on the achievement of these measures included in these documents is not sufficient to meet the requirements of the Regulator Performance Framework. A description of ASQA’s performance measures included in ASQA’s 2020–21 Corporate Plan and Four Year Plan is outlined above in paragraphs 2.4 to 2.10.

Have appropriate governance arrangements been established to oversee the reform?

2.20 Appropriate governance ensures direction and control are exercised in the achievement of the government’s objectives for VET sector regulation. Governance arrangements should ensure that implementation of the reform to VET sector regulation is effective and efficient, and takes appropriate account of risk. This section reviews the appropriateness of ASQA’s governance arrangements to oversee the reform.

2.21 An illustration of ASQA’s key governance arrangements²⁰ is included in Figure 2.3.

Figure 2.3: ASQA’s key governance arrangements

Source: ANAO analysis of ASQA’s organisational charts.

Accountable authority

2.22 The National Vocational Education and Training Regulator Amendment (Governance and Other Matters) Act 2020²¹ required that a Chief Executive Officer (CEO) replace the existing governance structure of a Chief Commissioner and two Commissioners. The CEO is the accountable authority of the regulator. This change was made to support a role more consistent with that of an agency head, including leading the long term and short term strategy and making top-level managerial decisions that determine the objectives, resources and policies of ASQA.

2.23 The Ministerial Statement of Expectations National Vocational Education and Training Regulator also identifies that the CEO has responsibility for leadership of ASQA, and drives organisational and cultural change when implementing the reform agreed in the Rapid Review.

Strategic Leadership Committee

2.24 Between September 2020 and January 2021, one Senior Executive Service (SES) Band 2 officer (to be the Deputy CEO) and four SES Band 1 officers were appointed. From 1 January 2021 the SES officers joined the Strategic Leadership Committee (SLC) (see Figure 2.4). Each of these SES positions has a documented position description, and are assigned specific responsibility for ASQA’s strategic deliverables. The purpose of the SLC is to:

Oversee, strengthen and monitor the efficiency and effectiveness of ASQA’s system of governance. Ensure ASQA achieves its strategic outcomes and deliverables. Ensure that there is a one-ASQA focus on building capability in the following priority areas: leadership; organisational and cultural change, including effective communication; regulatory best practice; risk management; and continuous improvement.

2.25 The SLC receives a range of information in order to monitor the implementation of the reform including:

• monthly reports on the progress of ASQA’s Four Year Plan;
• weekly status updates from the Reform Program Steering Committee; and
• monthly updates from its management sub-committees, particularly related to areas of interest and/or risk.

2.26 The reporting arrangements are examined further in Chapter 3 (see paragraphs 3.57 to 3.64).

Figure 2.4: Strategic Leadership Committee membership

Source: Strategic Leadership Committee Terms of Reference.

2.27 The CEO endorsed the establishment of a number of sub-committees of the SLC. The SLC committed to developing, for new sub-committees, or revising for its existing sub-committees, terms of reference that reflect reform objectives and activities. The sub-committee structure is illustrated in Figure 2.5.

Figure 2.5: Sub-committee structure

Note a: Darker blue represents new or revised terms of reference for the sub-committee to reflect the reform initiatives were approved before 31 March 2021.

Note b: Lighter blue represents sub-committee meetings to formally approve the new or revised terms of reference for the sub-committee to reflect the reform initiatives to occur on or after 1 April 2021.

Source: ANAO analysis of Strategic Leadership Committee Terms of Reference.

2.28 To support the role of the SLC, in September 2020, the Reform Program Taskforce was established:

To attend to priority workstreams related to reform … (t)he Reform Program Taskforce seeks to integrate current reform activity and manage and control delivery of reform until all 24 recommendations of the Rapid Review have been implemented and adopted by ASQA and promoted to the sector.

2.29 The reform activities are managed by the Director, Reform Taskforce, the Deputy CEO and/or the CEO. ASQA has engaged mpconsulting (the author of the Rapid Review) to provide advice on the design and implementation of the new regulatory approach to ‘maintain fidelity with the Review’s intent.’ For the financial years 2019–20 and 2020–21, 43 per cent ($3.2 million) of the available reform funding is forecasted to be spent on consultants and contractors.²² This level of reliance on the work of consultants and contractors requires ASQA to maintain a fit-for-purpose framework of accountability to ensure that value-for-money is achieved from these contracts and that there is appropriate skills transfer.

Advisory Committees

2.30 ASQA’s advisory committee governance structure has been updated to better reflect the objectives and strategic deliverables of the reform. Its two advisory committees are the Audit Committee and the (yet to be established) National Vocational Education and Training Regulator Advisory Council (Advisory Council).

• The Audit Committee Charter has been refreshed and will be finalised in June 2021. These changes will see the committee comprise three independent persons appointed by the CEO. It will also have a renewed focus on the review of systems of risk oversight and management, and systems of internal control to better align to the reform.
• The National Vocational Education and Training Regulator Amendment (Governance and Other Matters) Act 2020 gave effect to the Advisory Council. The Advisory Council will provide advice to ASQA on the regulator’s functions. The Advisory Council will consist of a chair and no more than nine other members. The department is responsible for the establishment of the Advisory Council and it is expected that the Advisory Council members will be appointed by the Minister before September 2021. The Advisory Council is expected to provide high-level expert advice on ASQA’s strategic objectives and approach to regulation.

3.1 As outlined in paragraph 2.1, the Australian Skills Quality Authority (ASQA) is reforming its regulatory model for the vocational education and training (VET) sector. The successful implementation of the reform requires program management, risk management and performance monitoring and reporting. This chapter examines how ASQA has considered these three elements to effectively transition to the proposed regulatory approach.

Does ASQA have a fit-for-purpose program management framework for reform?

3.2 Reform programs require fit-for-purpose program management frameworks to ensure work is completed to the expected standard, within available resources and to schedule. To meet the requirements for implementing the Rapid Review of the Australian Skills Quality Authority’s Regulatory Practices and Processes (Rapid Review) in an effective and timely manner, ASQA requires a fit-for-purpose program management framework. This section reviews the appropriateness of ASQA’s program management framework, including ASQA’s consideration of risk.

Program management framework

Program Plan

3.3 In September 2020, ASQA established a Reform Program Taskforce to manage ASQA’s reform program of work, including key elements of reform that ASQA has determined require an integrated program management methodology. The aim of this program of work was ‘to integrate … reform activity and manage and control the delivery of reform until all 24 recommendations of the Rapid Review have been implemented and adopted by ASQA.’ The Reform Program Taskforce works within two management frameworks:

• Managing Successful Programmes — for managing change at the programme level; and
• Managing Successful Projects with PRINCE2 — for managing projects in the portfolio.

3.4 In March 2021, ASQA finalised the Program Plan, which outlines which activities the Reform Program Taskforce is responsible for. ASQA determined that some activities, for example, certain governance arrangements and changes to organisational risk management, would not be managed through the Reform Program Taskforce.

3.5 The Program Plan covers the four years of the reform, but only outlines activities related to the first year through to 30 June 2021. The Program Plan was scheduled for review in April 2021. In May 2021, the Strategic Leadership Committee (SLC) (see paragraph 2.24) confirmed its approach to integrating the Rapid Review’s recommendations under ASQA’s Four Year Plan. On 9 June 2021, the Reform Program Steering Committee agreed to the scope of work to be managed under the Reform Taskforce Program Plan: 2021–22, which is intended to replace the Program Plan and support reform activities from 1 July 2021 to 30 June 2022.

3.6 The current Program Plan outlines five workstreams of activities to be undertaken (see Table 3.1).

Table 3.1: Program Plan workstreams

Source: ANAO analysis of ASQA’s Program Plan.

3.7 The Program Plan states that a structured approach to reform planning and reporting will be undertaken. It also identifies a range of program controls to be used to assist in planning and reporting of activities. These program controls are summarised at Figure 3.1.

Figure 3.1: ASQA reform program controls

Source: ANAO analysis of ASQA’s Program Plan.

Program risk

3.8 The Program Plan includes a risk matrix to be used for the reform and was used to prepare the Risk and Issues Register program control documents. The Program Plan’s risk approach is inconsistent with ASQA’s Risk Management Framework because:

• the Program Plan does not provide guidance on how to apply the risk matrix, such as definitions for assessing risk likelihood and consequence; and
• the risk thresholds do not include relevant escalation requirements to align with the ASQA Risk Management Policy Statement²³, which states that all ‘elements of the reform agenda will have a risk appetite of low.’ Alignment with this policy statement would require any risk rated above ‘low’ to be escalated.²⁴

3.9 The February 2021 Risk and Issues Register included target risk ratings (risk ratings post-risk mitigation treatments) above the rating of ‘low’ set in ASQA’s risk appetite for the reform program. The February 2021 Risk and Issues Register identified six risks which related to the reform:

• business operations are not ready to ‘go live;’
• implementation of new regulatory approach is not consistently applied;
• the sector is not properly informed in the transition to the new regulatory approach;
• system change to support key elements of new process cannot be delivered in the timeframe;
• the pace and volume of required changes results in adverse regulatory outcomes; and
• staff wellbeing is compromised.

3.10 The risk treatments in the Risk and Issues Register prioritise time over other project variables such as cost, quality, scope and benefits. As at February 2021, four of these risks had a medium or high risk rating, after identified controls were applied. The mitigation treatments were to ‘re-scope 6^th April Go Live activities, creating a backlog’ or ‘detailed planning to identify required changes.’ Focusing mitigation treatments on achieving short term timeframe objectives may impact on the longer term outputs, outcomes and benefits achieved from the reform.

3.11 ASQA has a number of other risk registers that deal with reform risks, particularly the organisational risk register and the risk assessment included in ASQA’s Four Year Plan. The Program Plan does not outline how its application of risk management will feed into and/or support other ASQA risk assessments which include specific reform risks. The Risk Management Framework (March 2021) states that:

Work will be conducted during 2021 to consolidate and coordinate these specialist risk registers and processes further within the ASQA risk management approach and ASQA Risk Register to ensure efficiency and consistency in managing risks, as well as integrated reporting and monitoring of ASQA’s overall risk profile.

Program quality

3.12 Managing successful projects with PRINCE2 guidance emphasises the quality theme as an important part of a project management framework. It states that ‘the purpose of the quality theme is to define and implement the means by which the project will verify that products are fit-for-purpose.’²⁵ It is particularly important that products are fit-for-purpose before the change is implemented.

3.13 The Program Plan outlines quality controls to ensure that the reform deliverables are completed to the required level of quality. Six quality objectives are outlined in the Program Plan.

3.14 The quality controls outlined in the Program Plan do not consider whether:

• deliverables address scope requirements and/or the relevant recommendations outlined in the Rapid Review;
• benefits identified for the project, process redesign and/or tools have been achieved;
• technical aspects of control redesign have been sufficiently identified, considered and addressed (including compliance with relevant legislation and standards in force at the time of implementation); or
• organisational service standards and cost information have been sufficiently identified, considered and addressed (particularly impacts on efficiency and effectiveness of new processes/templates).

3.15 In March 2021, ASQA published an Assurance Policy Statement. This policy statement states that:

A key part of this transformational journey is the development of mature and coordinated assurance practices across the agency, which provide confidence in the efficacy and quality of what we do and that continuously improve how we regulate and operate to best manage risk and achieve our strategic objective. Effective assurance practices will ensure that we are

• complying with legislative requirements
• producing outcomes to a quality standard
• undertaking work which is consistent with best practice
• focussed on continuous improvement
• have a robust system of governance to monitor our efficiency and effectiveness

3.16 In March 2021, an Assurance Framework was also published. This document outlines ASQA’s approach to assurance which sets out a three lines of assurance model.²⁶ This model provides details of each of the lines of assurance and how they may be used to monitor and assure quality outcomes, risk management practices, compliance, and the effectiveness of controls.

3.17 ASQA has also refreshed its Internal Quality Assurance (IQA) function which reports directly to the Chief Executive Officer (CEO). The IQA undertakes a program of work as one component of ASQA’s Assurance Framework. In addition, ASQA has developed an Assurance Maturity Assessment which includes an implementation plan to improve assurance over the period from March 2021 to December 2021.

3.18 The focus of the Assurance Framework and the IQA work program is on providing assurance over the quality of activities and deliverables during or after implementation. The PRINCE2 methodology highlights an additional need for assurance over the design of new processes and controls.

Meeting expectations

3.21 The Program Plan identifies that when activities are closed, a lessons learned log will be updated to assist with sharing insights into the reform process over the four year time-period.

3.22 To provide additional assurance over the progressive implementation of the reform, ASQA has periodically reviewed key changes implemented to confirm the adequacy of implementation and achievement of desired outcomes. For example, in November 2020, ASQA reviewed the reform achievements in the assessment process — post-auditor training — and changes to the assessment and audit practice implemented from August 2020. The changes primarily focused on improved analysis of evidence and positioning of information and findings in audit notes and audit reports.

3.23 The review identified a high degree of variation in how auditors are undertaking the task of positioning of information and findings in audit notes and audit reports. The review found that despite general improvements in the inclusion of summaries of evidence, a number of reports contained findings without the appropriate rationale. To address this, the review focused on:

• matters identified for the purpose of refresher auditor training
• suggested changes to the audit notes and audit report templates

3.24 The review did not consider the root causes of its findings. For example, the review did not consider the quality of guidance provided (particularly the level of information on how to analyse evidence and express findings), or the appropriateness of supervision arrangements and issue escalation protocols. In April 2021, IQA commenced an assurance review of the assessment methodology. This review is intended to provide insights into the achievement of ASQA’s reform of the assessment methodology.

Managing relationships with partner agencies

3.25 Where there are shared deliverables, entities should be aware of each other’s priorities and risk tolerances, and manage instances where these differ. Developing a shared understanding of acceptable risk can support efficient and effective decision-making through program implementation.

3.26 As previously noted in paragraph 1.15, the Rapid Review recommendations are not solely the responsibility of ASQA. There are a number of recommendations which are dependent on the Department of Education, Skills and Employment (the department) for amendments to the National Vocational Education and Training Regulator Act 2011 (NVR Act) and supporting legislation; and consultation, review and updates to relevant standards including the Standards for Registered Training Organisations (RTOs) 2015 and the Standards for VET Regulators 2015.

3.27 Under the Public Governance, Performance and Accountability Act 2013 (PGPA Act) entities are required to put in place arrangements for understanding and managing shared risks.²⁷ ASQA’s Program Plan does not consider how shared risks and responsibilities related to the reform will be identified and managed. The Program Plan identifies the department as a key Australian Government stakeholder, but does not identify how shared responsibilities for implementing the reform will be managed.

Is ASQA’s approach to transition informed by relevant systemic and VET provider risks?

3.28 As outlined in paragraph 2.1, as a result of the Rapid Review, ASQA is transitioning from a focus on compliance to a focus on working with the VET sector to support providers in building capability for self-assurance. Section 16 of the PGPA Act requires the accountable authority to establish and maintain appropriate systems of risk oversight and management for the entity. This section reviews ASQA’s consideration of risk in transitioning to the new regulatory model.

Organisational risk

3.29 In November 2020, the SLC (see paragraph 2.24) identified that ASQA’s existing Risk Management Framework and Policy and related risk registers required ‘a substantial refresh and an investment to lift the maturity for the overall application of risk management.’

3.30 In January 2021, the CEO released a new Risk Management Policy Statement, which outlined ASQA’s risk appetite during the reform.

At this point in time … ASQA has a low appetite in relation to the following enterprise risks:

• workforce-related issues, including Work Health and Safety and capability
• implementation of the reform agenda
• systems of governance.

This means that we are prepared to invest resources to closely and actively manage these risks. As ASQA’s transformational journey delivers improved systems, processes, governance and outcomes, and our risk management capability matures, I expect that ASQA’s risk appetite will change. I anticipate that, during the current four-year planning cycle, ASQA will move to a point where Work Health and Safety remains the only risk for which ASQA has a low appetite.

3.31 In March 2021, the ASQA Risk Management Maturity Assessment was prepared to guide the update to the Risk Management Framework. ASQA used the Australian Government ComCover Risk Management Capability Maturity Model to conduct a self-assessment of its current risk management practices. The model provides a maturity matrix with five levels of maturity from Level 1 (Simple) to Level 5 (Advanced). ASQA self-assessed its current risk maturity level at Level 1, with a desired maturity state of Level 3 (Defined) to be achieved within 12–15 months. ASQA has prepared an implementation plan and calendar up to June 2022, including key actions to facilitate maturing its risk management across the organisation.

3.32 In March 2021, ASQA developed a new Risk Management Framework and Risk Management Procedure. The Risk Management Procedure was based on the Risk Management Toolkit for NSW Public Sector Agencies, produced by the NSW Department of the Treasury. The Risk Management Framework includes the identification and management of strategic risks, enterprise-wide risks and specialist risks.²⁸

3.33 The Risk Management Procedure outlines how risks will be identified, analysed and treated. The analysis of risks includes the assessment of worst case or inherent risk analysis, a control effectiveness assessment and a current risk analysis. Risk treatments are also to be identified to manage risks within ASQA’s stated risk appetite. A summary of the key documents used to direct ASQA’s organisational risk management is included at Figure 3.2.

Figure 3.2: ASQA’s organisational risk management approach

Source: ANAO analysis of ASQA’s risk management documents.

3.34 ASQA has commenced revision of its strategic and enterprise-wide risk registers, and specialist risk registers, to apply the new Risk Management Procedure. ANAO review of the risks in the risk register indicates that:

• the risk descriptions require development in order to target the risk events and to clearly identify the causes of the risks;
• the risk register requires greater support for the rating of control effectiveness assessments; and
• controls and treatments require clearer articulation so that assessments of the impact on risk likelihood and consequence can be analysed.

Regulatory risk

3.35 ASQA’s approach to regulation is stated in its 2020–22 Regulatory Strategy (Regulatory Strategy).²⁹ The Regulatory Strategy is founded on ASQA’s understanding of provider risk and systemic risk.³⁰

3.36 The Regulatory Strategy identifies, specific focus areas for regulatory activities, strategic initiatives to be implemented and any training products or standards of concern, which will be subject to targeted regulatory scrutiny.

3.37 A number of the Rapid Review recommendations relate to clarifying and modernising ASQA’s regulatory approach and regulatory risk management, including:

• refocusing on using intelligence to effectively monitor risk and provider performance;
• emphasis on supporting self-assurance;
• adopting more appropriate and proportionate regulatory outcomes; and
• providing more meaningful information on provider performance.

Regulatory Operating Model

3.38 In February 2021, ASQA developed a Regulatory Operating Model and Regulatory Risk Framework to set the foundations for undertaking its regulatory functions and delivering on its Regulatory Strategy.

3.39 The Regulatory Operating Model is illustrated in Figure 3.3. The Regulatory Operating Model has a structured approach to:

• defining regulatory outcomes;
• engaging strategically about risk;
• assessing risk and designing regulatory responses;
• using data and intelligence to understand risks and regulated entities;
• understanding regulatory impact and outcomes; and
• interacting with regulated entities.

Figure 3.3: ASQA’s Regulatory Operating Model

Source: ASQA’s Regulatory Risk Framework.

3.40 The Regulatory Operating Model outlines an integrated framework for ASQA’s regulatory functions. It articulates how ASQA will manage risk and continuously improve quality in VET. It supports a ‘one ASQA’ regulatory culture, approaches and practices. The Regulatory Operating Model has been developed to provide:

• renewed and expanded approach to the management and use of data as a key input to regulatory operations seeks to strengthen the use of data and methods of delivery of intelligence to ASQA.
• better use of data insights from performance information and feedback from regulatory operations to understand impact and effectiveness of regulatory activities and continuously improve regulatory design.
• design activities define a response to specific regulatory risk, making greater use of behavioural insights that is co-ordinated and makes the best use of regulatory tools across, engagement, communication and regulatory operational activities.
• engaging strategically about risks. Communication and engagement expands as a key tool in the regulator toolkit. Sharing information with stakeholders to better understand risk, raising awareness and providing clear feedback and support for providers to self-assure and continuously improve.
• through all our foundational risk treatments education, registration and approvals, course accreditation, monitoring and performance assessment, compliance management and internal review of decisions – we seek to apply risk-based principles, proportionate, fit for purpose responses.

3.41 As part of reform activities, ASQA is designing and developing policies, procedures, practices, guidance and tools to support implementation of the new Regulatory Operating Model.

Regulatory Risk Framework

3.42 Section 190 of the NVR Act allows the Minister to make guidelines for ASQA to use when assessing risk. Subsection 157(4) of the NVR Act states that ‘in performing the National VET Regulator’s functions, the Regulator must apply the Risk Assessment Framework.’

3.43 In 2011, the Minister released a Risk Assessment Framework which set out:

… a suggested risk assessment process for ASQA to use in making decisions about:

• assessing and responding to the risk of non-compliance by applicants against the VET Quality Framework;
• assessing and responding to the risk of non-compliance by Registered Training Organisations (RTOs) against the VET Quality Framework; and
• monitoring the compliance of RTOs against the VET Quality Framework.

3.44 In February/March 2021, ASQA redeveloped its Regulatory Risk Framework. The Regulatory Risk Framework was reviewed by the Reform Program Taskforce (see paragraph 2.28) and approved at the Reform Program Steering Committee and the SLC. The Regulatory Risk Framework was implemented from March 2021 and made publicly available on 6 April 2021. ASQA sought advice from the department who advised:

that the documentation that has been utilised by ASQA to date sets out an approach that applies and is consistent with the Risk Assessment Framework … We are yet to review the risk framework as released by ASQA in mid April but will undertake the same analysis and provide advice accordingly.

3.45 The Regulatory Risk Framework outlines the approach to assessing risks and designing regulatory responses. ASQA has not fully defined all components of the specialist regulatory risk matrix in terms of definitions, escalation protocols and how it feeds into specific regulatory functions, such as education, registration and approvals, course accreditation, monitoring and internal review of decisions. ASQA is still to develop procedures to guide the application of the Regulatory Risk Framework across all functions.

Has ASQA developed effective monitoring and reporting on its transition?

3.49 ASQA has accountability to the Parliament and the public for its performance. This includes for the use of additional funding for the reform activities, and for the changes to performance expected and arising from the reform. These accountabilities are primarily discharged through ASQA’s corporate plan and associated annual performance reporting.

Performance measures and targets for reform — strategic planning level

3.50 ASQA’s approach to the reform is to integrate reform activities into its strategic and business planning. Paragraphs 2.4 to 2.10 identified ASQA’s strategic planning was contained in two key documents — the ASQA’s 2020–21 Corporate Plan and Four Year Plan.

3.51 The performance framework outlined in the 2020–21 Corporate Plan includes information on ‘our approach’, ‘performance criteria’ and ‘evidence of performance/measures.’ The 2020–21 Corporate Plan identified ASQA’s expectation that:

… during the coming year of intensive reform, we may be challenged in meeting some of our performance criteria and service standards. During this time, we will also be reviewing the adjustments to our performance framework necessary to take account of our new regulatory approach and activities, in order to ensure that our performance framework remains relevant and effective.³¹

3.52 The performance criteria and measures outlined in the 2020–21 Corporate Plan include both qualitative and quantitative measures of performance. The Corporate Plan outlines a number of reform initiatives to be undertaken in 2020–21 and also across the four year reform time-period (2020–24). However, no performance measures or targets were identified for these strategic reform initiatives. ASQA advised that as part of its 2020–21 annual performance reporting, it intends to include progress reporting on the implementation of the Rapid Review recommendations as supplementary performance information.

Measuring benefits realisation

3.53 In March 2021, ASQA prepared a draft Reform Implementation and Immediate Benefits Realisation document which was designed to articulate the benefits expected from implementing each of the 24 recommendations from the Rapid Review. These benefits are aligned to the new strategic deliverables set out in ASQA’s Four Year Plan (see Figure 2.2). The performance measures identified in the Four Year Plan do not meet the Department of Finance’s requirements for defining performance measures.³² In particular, they identify the source of performance information, but not how performance will be measured.

Monitoring and reporting — senior leadership level

3.57 The SLC receives monthly reports (see paragraph 2.25) on the progress of the strategic deliverable actions identified in ASQA’s Four Year Plan. The progress reports outline:

• the number of actions scheduled for each strategic deliverable;
• whether all actions scheduled to have commenced have commenced, and any reasons for delays;
• if there are any substantial slippages in the performance of actions; and
• the risk rating of work commenced to assess whether risks are being appropriately managed.

3.58 The monthly progress reports also capture whether there are any amendments to the relevant actions, owners or commencement dates. ASQA continues to update and refine action items associated with ASQA’s Four Year Plan, and changes to the actions are considered and agreed during SLC meetings.

Weekly status updates from the Reform Program Steering Committee

3.59 The SLC is provided with weekly updates from the Chair of the Reform Program Steering Committee (Deputy CEO). To support the progress update, copies of the weekly Program Dashboard, and Critical Path — Priority Reforms, prepared by the Director, Reform Taskforce are also provided.

3.60 The Program Dashboard outlines:

• key decisions and considerations made over the week;
• key issues and risks, including agreed mitigations;
• upcoming stakeholder engagement;
• impact indicators (traffic lights) on whether the reform implementation is on schedule, within budget, scope status and resource status; and
• critical activities and milestones for the next period.

3.61 The Critical Path — Priority Reforms document outlines a schedule of activities, and any dependencies between the key activities, over the next six month period.

Monthly updates from the sub-committees

3.62 The SLC intends to revise the terms of reference for its sub-committees by May 2021 to better reflect the reform objectives and activities (see Figure 2.5). Part of the responsibility of the sub-committees will be to monitor reform activities within their respective areas of responsibility. For example the Resources and Finance Committee will ‘oversee resource allocation risks that may impact on achievement of ASQA’s strategic outcomes and deliverables’ and provide ‘advice to the Strategic Leadership Committee in relation to ASQA’s financial performance.’

3.63 Currently, the reporting provided to the SLC to monitor reform program implementation focuses on tasks and/or actions being undertaken, immediate priorities and decision-making over the next six months. It does not provide information over the four year time-period to allow strategic consideration of scope achievement, benefits realised, dependencies, costs and resourcing.

3.64 ASQA established two sub-committees under the SLC to examine ASQA’s regulatory performance and compliance. These sub-committees came into effect in April 2021, and include:

Regulatory Impact Committee (Chair: General Manager, Regulatory Insights and Impact) — Advise and apply an appropriate methodology for evaluation of regulatory impact. Monitor risks to inform the development and monitoring of a regulatory program of work to meet ASQA priorities. Consider the trends and patterns across the regulatory system (including risk (tactical, operational, strategic (including external)) to identify opportunities for regulatory treatments, improvements to regulatory practice and to influence policy reform work. Develop the narrative around the regulatory system, and the story we tell about our impact.

Regulatory Operations Committee (Chair: General Manager, Quality Assessment and Compliance) — Determine effective processes to ensure that ASQA’s capacity is effectively utilised to manage the regulatory risks identified in the VET sector. Establish systems and processes for ensuring consistency in approach for applications, assessment and compliance. Ensuring resources level and allocation processes across the regulatory functions are designed to most effectively utilise the capacity of the group. Oversight of service standards and identifying trends and areas for improvement. Instilling culture across the teams that reflects ASQA’s regulatory posture.

Costs and resources

3.65 ASQA has been provided with funding of $24.1 million over five years (2019–20 to 2023–24) to undertake reform activities (see Table 1.1). This includes $3.7 million in capital funding. ASQA manages and monitors reform spending as part of its general financial reporting. The Resources and Finance Committee monitors ASQA’s financial performance.

3.66 A recommendation of the Rapid Review was to ‘realign ASQA’s cost recovery arrangements to reflect ASQA’s new approach to performance monitoring.’ In addition, ASQA will be moving to full cost recovery arrangements from 1 January 2022. ASQA advised that it is currently reviewing how it will move to these arrangements.

3.67 The reform to the proposed regulatory approach will impact on how ASQA organises and allocates its resources. Although the average staffing level allocated to ASQA has not changed (199), how ASQA deploys these resources will change, for example Regulatory Operations was split into two teams, one for Quality Assessment and one for Compliance from 1 March 2021. This reflects the changed approach to the assessment methodology, to assess provider performance, and a more focused risk-based compliance approach.

3.68 The resources allocated to each of the teams will need to reflect the redesigned processes, new performance expectations and changed supervision hierarchy. ASQA is developing a resource and cost model associated with its reform activities.

3.69 In addition, on 9 February 2021, the SLC agreed that:

• going forward, establishment briefings need to support SLC’s consideration of a whole-of-agency approach to vacant positions (cf. a focus on pool positions and substantively vacant positions) to ensure the agency is agile in allocating resources in priority areas that align with the achievement of strategic deliverables.
• agreed that positions should not be held/recruited to existing levels, roles or teams, but rather consideration of the creating of/recruitment to positions should be based on: (a) the overall budget available; (b) where resources are required in order to align with the four year plan and meet the strategic deliverables; and (c) where resources are required to mitigate identified risks, including in relation to the achievement of external performance measures.

3.70 On 22 February 2021, ASQA updated the Terms of Reference for the Resources and Finance Committee to include the following responsibility:

Oversee a whole-of agency approach to the allocation of staffing to ensure the agency is agile in allocating resources in priority areas that align with the achievement of strategic deliverables.

4.1 As outlined in paragraph 2.1, Australian Skills Quality Authority (ASQA) is currently reforming its regulatory model for the vocational education and training (VET) sector. The Rapid Review of the Australian Skills Quality Authority’s Regulatory Practices and Processes (Rapid Review) highlighted the importance of sector-wide engagement. This chapter examines whether ASQA has effectively engaged with the VET sector about its transition to its proposed regulatory approach.

Has ASQA developed an effective VET sector engagement strategy for the reform?

4.2 The Rapid Review highlighted the importance of cooperative relationships between ASQA and the VET sector.

Currently there is a significant trust deficit between the regulator, peak bodies and some regulated entities. The reasons for this are multifaceted and influenced by recent history, including some significant policy, regulatory and industry failures.

Acknowledging this history and committing to moving forward in a spirit of cooperation and mutual understanding will be critical if reform is to be delivered.

ASQA will play an important role in this by strengthening its focus on engagement and education and working with the sector to co-design new regulatory tools. Providers will also play an important role by proactively engaging with ASQA to understand ASQA’s expectations on an ongoing basis and when non-compliance is identified, and by working with ASQA to address some of the significant systemic issues that impact the sector more broadly.³³

4.3 Effective engagement by ASQA with the VET sector is required to ensure the sector understands the proposed regulatory approach. This section reviews the effectiveness of ASQA’s engagement strategy.

Stakeholder engagement approach

4.4 ASQA’s 2020–21 Corporate Plan identified three aspects to its approach to engagement and support which were:

• we engage with stakeholders to inform our understanding of risks;
• we provide accurate information to support stakeholders’ understanding of the VET sector and relevant obligations; and
• we provide advice on strategies to improve VET regulation and the VET sector.³⁴

4.5 In July 2020, ASQA reorganised its internal structure to establish dedicated stakeholder engagement, and education and guidance teams to ensure full accountability for the delivery of these functions within a single area. In December 2020, ASQA commenced an external capability review of the communications team to assist in identifying areas for improvement related to the structure and capability of the resources involved in external communications and engagement.

4.6 ASQA has also taken measures to improve the effectiveness of, and control over, its engagement with stakeholders. For example ASQA engaged a consultant to document the first stage of the development process for a Stakeholder Engagement Strategy based on a desktop review of key documents. The consultant’s report was delivered in February 2021. In addition, in April 2021, ASQA publicly released a consultation draft of the ASQA Stakeholder Engagement Strategy. This document outlines engagement principles and an engagement model. It also indicates that performance measures will be developed collaboratively with stakeholders to meet ASQA’s external reporting requirements, such as for the Public Governance, Performance and Accountability Act 2013 and the Regulatory Performance Framework.

4.7 Successful implementation of the reform will require changes to ASQA’s engagement approach to overcome the ‘trust deficit’ identified in the Rapid Review. This includes ASQA demonstrating a strategic approach to setting engagement priorities, preferred methods of engagement and planned stakeholder reach. ASQA has not developed stakeholder engagement baseline performance information that would facilitate an objective assessment of the success of stakeholder engagement over the term of the reform.

Stakeholder engagement activities

4.8 Over the last twelve months, ASQA has undertaken a range of activities to support improved stakeholder engagement including the use of a variety of communication techniques such as: digital and online communications; social media; direct mail; ASQA Info Line responses; media; and the production of communications products such as fact sheets, reports and videos.

4.9 ASQA has also developed and delivered webinars³⁵ on topics related to reform activities including:

• working towards effective self-assurance (August 2020);
• changes to audit practice — ASQA’s reform agenda (September 2020);
• ASQA’s approach to compliance — consistency, collaboration and continuous improvement (December 2020);
• advice to the sector on next steps in self-assurance (December 2020);
• ASQA’s strategic view into building online capability in the VET sector (December 2020); and
• understanding assessment (February 2021).

4.10 ASQA has also used its stakeholder engagement forums to discuss the aims, progress and key elements of the reform, including the use of the Provider Roundtable and the Stakeholder Liaison Group (and relevant sub-groups).

4.11 ASQA has used its 2020 Annual Provider and Course Owner Survey to obtain feedback and information relevant to the application and understanding of self-assurance. ASQA has also ensured that relevant information on the reform is included on its external website.

Incorporation of stakeholder feedback into the regulatory approach

4.12 The Rapid Review recommended that the regulatory reform should be based on the adoption of a self-assurance model by the VET sector. Effective engagement by ASQA with the VET sector is required to ensure the sector understands the proposed regulatory approach and its requirements to effectively self-assure and report.

4.13 ASQA sought stakeholder feedback on the development of its new regulatory approach. ASQA has used a range of mechanisms to seek stakeholder feedback (see paragraphs 4.8 to 4.10).

4.14 In addition to the above mechanisms, in August 2020, ASQA conducted its annual survey of all training providers and course owners that it regulates. The survey sought provider views on ASQA’s key processes, communication and engagement and established a baseline for providers’ knowledge and preparedness for self-assurance. The survey included questions to help inform ASQA’s reform activities, particularly the understanding and use of provider self-assurance. Some of the results related to self-assurance are highlighted in Table 4.1.

Table 4.1: Highlights from the Provider and Course Owner Survey 2020

Source: Acil Allen Consulting, Provider and Course Owner Survey 2020 Final Report, September 2020.

4.15 In July 2020, ASQA developed a consultation paper on Working Together Towards Effective Self-Assurance to outline the definitions and movement towards self-assurance, and what regulation of the VET sector may look like over the short, medium and longer-term. In November 2020, a Self-Assurance Consultation Paper Feedback Report³⁶ was published which provided a consolidated collation and analysis of the feedback received. It identified a range of further steps ASQA would take towards monitoring the effectiveness of self-assurance.

4.16 In November 2020, ASQA released a consultation paper on the Draft Compliance Policy and undertakings to remedy non-compliance (draft compliance policy). ASQA also engaged consultants to obtain stakeholder feedback on its draft compliance policy and to determine how this feedback could be used to effectively implement the compliance policy.

4.17 The consultants concluded that ‘Overall, the proposed content of the Policy and the concept of the Undertakings were well received.’ The consultant’s report identified 24 recommendations to improve the draft compliance policy with regard to content, implementation, communication and publication, and education and engagement. ASQA has incorporated this stakeholder feedback into the Regulatory Operating Model and supporting policies, procedures, practices, guidance and tools.

Engagement to facilitate the identification of emerging risks related to COVID-19

4.18 ASQA developed a COVID-19 Risk Monitoring Strategy which was considered by ASQA’s Senior Executive Committee on 27 May 2020. The purpose of the monitoring strategy was to ‘identify key risks arising from the changing provider behaviour in response to the impact of COVID-19 on the broader economy and community by identifying key data and information sources to monitor provider activity.’

4.19 The development of this strategy was designed to address the intent of the recommendations of the Rapid Review. In particular, ASQA designed the monitoring strategy to:

• enable key messages to be given to providers as quickly as possible in light of changing circumstances;
• influence provider behaviour by providing advice to providers seeking to comply with their obligations and alerting the sector, that ASQA is monitoring delivery risk;
• collaborate with peak and funding bodies to ensure that they are aware of the risks and support ASQA’s approach to responding to these risks; and
• communicate with students to help them choose quality VET providers.

4.20 The COVID-19 Risk Monitoring Strategy identified the three most significant risks within ASQA’s jurisdiction arising from COVID-19 to be:

• risk 1 — some providers will add products to their scope of registration in order to meet changes in demand, without the necessary industry engagement and expertise in those areas;
• risk 2 — some providers will transition to online or other distance modes of delivery without adequate resourcing, student support and/or validation of assessment; and
• risk 3 — some providers will certify students as competent who have not completed required work placements and/or been assessed in a workplace in line with the training package or accredited course requirements.

4.21 The COVID-19 Risk Monitoring Strategy identified how these risks would be monitored and examined by ASQA, including education and information material to be provided to the VET sector, internal processes and controls, internal issue escalation protocols and reporting.

4.22 In addition, ASQA developed and delivered a range of webinars specifically related its COVID-19 Monitoring Strategy, including:

• ASQA’s approach to COVID-19 (May 2020);
• distance learning — compliance essentials (May 2020); and
• resumption planning — continuing delivery in a changed world (July 2020).

4.23 ASQA finalised an evaluation of the COVID-19 Risk Monitoring Strategy in January 2021 to assess whether the approach was effective. This evaluation considered the extent to which the actions implemented achieved the desired purpose, effectiveness of those actions and identified areas for improvement to influence similar risk response strategies in the future. The overall evaluation results are summarised in Table 4.2.

Table 4.2: ASQA’s evaluation of its COVID-19 Risk Monitoring Strategy

Source: ASQA’s COVID-19 Risk Monitoring Strategy Evaluation.

4.24 ASQA’s stakeholder engagement has facilitated the identification of emerging provider and VET sector risks for COVID-19. ASQA has developed and evaluated its approach to VET sector engagement related to COVID-19.

Has ASQA effectively planned for and supported VET providers for the implementation of provider self-assurance?

4.25 Education and engagement are essential components of effective regulation. Reflecting this, and in line with the direction of Ministers, ASQA emphasised engagement with, and education of, VET sector stakeholders. ASQA’s shift to a more consultative approach to engagement aims to inform ASQA of the issues facing the VET sector and help to shape the guidance and regulatory tools provided to promote and support continuous improvement.

Responding to VET provider concerns

4.26 Over the last twelve months ASQA has undertaken a range of activities to support how it responds to VET provider concerns including:

• its provision of information and guidance to the sector on ASQA’s regulatory approach and interpretation of relevant standards, training package requirements, and obligations under relevant legislation;
• how it responds to provider and stakeholder queries received through ASQA’s Info Line in a consistent, timely and accurate manner; and
• engagement with stakeholders to gather and share regulatory data and information to identify risks to quality and effective treatment strategies to inform ASQA’s regulatory approach.

4.27 ASQA’s 2020–22 Regulatory Strategy (Regulatory Strategy) is the primary document outlining the focus areas, strategic initiatives, approach to education and guidance and any training products or standards of concern. The Regulatory Strategy highlights the aim to continue to share information and respond to VET provider concerns using:

• regular interactive webinars, video discussions with sector participants and targeted guidance materials;
• an annual series of provider briefings and to continue to use established forums such as the Provider Roundtable and the Stakeholder Liaison Group (and related sub-groups);
• guidance information published on ASQA’s website in response to key risks identified by providers;
• monthly ASQA update newsletters, with summaries of the latest activities impacting on the sector; and
• direct emails to providers, as needed, to circulate important information.

4.28 ASQA also conducts an annual environmental scan of existing and emerging issues in the VET sector to identify risks across the VET sector or in a proportion of providers. This environmental scan informs the development of the Regulatory Strategy and helps to target actions to respond to change and support the VET sector address changes in ASQA’s regulatory approach.

4.29 ASQA has commenced the development of a forward education program in response to stakeholder feedback on key topics of interest, with a focus on explaining ASQA’s reforms and how these changes are impacting Registered Training Organisations (RTOs).

Action to support the VET sector to address changes in its regulatory approach, particularly provider self-assurance

4.30 ASQA has been consulting with the VET sector on provider self-assurance through interactive webinars, formal consultation papers and stakeholder engagement forums (see paragraphs 4.8 to 4.11).

4.31 In addition, a focus of ASQA’s stakeholder engagement to date has been on gathering and sharing intelligence, both data and information, with VET providers to assist with greater emphasis on VET provider self-assurance.

4.32 ASQA has reviewed the 2021 Annual Declaration of Compliance to be prepared by RTOs to ensure functionality issues identified through consultation were rectified for use in 2021. Minor amendments to the 2021 Annual Declaration of Compliance were also aimed at improving the ease of use of the declaration, and its supporting self-assessment tool to highlight the focus on monitoring, reviewing and continuously improving systems.

4.33 ASQA has communicated its intent to co-design the Statement of Compliance to be used for 2021–22 to provide a strong focus and support on provider self-assurance.

4.34 ASQA has commenced consultation on, and co-design of, self-assurance tools and sector guidance materials. VET sector feedback will be used to review and co-design tools to support the Annual Declaration for Compliance once the revised Standards for Registered Training Organisations (RTOs) 2015 is released.

4.35 ASQA has also identified a number of aspects of the Standards for VET Regulators 2015 that are no longer fit-for-purpose or aligned to the focus of ASQA’s reform agenda, including a greater emphasis on provider self-assurance. These have been discussed with the Department of Education, Skills and Employment to be considered for incorporation into the revised standards.

Appendix 1 Australian Skills Quality Authority response

Appendix 2 Legislation and standards

1. The Australian Skills Quality Authority’s (ASQA’s) regulation of Australia’s vocational education and training (VET) sector is supported by a framework of legislation and standards.

2. ASQA was established on 1 July 2011 through the enactment of the:

• National Vocational Education and Training Regulator Act 2011;
• National Vocational Education and Training Regulator (Consequential Amendments) Act 2011; and
• National Vocational Education and Training Regulator (Transitional Provisions) Act 2011.

3. ASQA is subject to other legislation and standards including:

• the VET Quality Framework which comprises the:
  • Standards for Registered Training Organisations (RTOs) 2015;
  • Fit and Proper Person Requirements 2011;
  • Financial Viability Risk Assessment Requirements 2011;
  • Data Provision Requirements 2012; and
  • Australian Qualifications Framework.
• Standards for VET Accredited Courses 2021;
• Standards for VET Regulators 2015; and
• legislation and standards relating to education services for overseas students which comprises the:
  • Education Services for Overseas Students Act 2000;
  • Education Services for Overseas Students Regulations 2019;
  • National Code of Practice for Providers of Education and Training to Overseas Students 2018;
  • ELICOS Standards 2018; and
  • other legislative instruments.

Appendix 3 Rapid Review Recommendations

1. In March 2020, the Rapid Review of the Australian Skills Quality Authority’s Regulatory Practices and Processes (Rapid Review) was reported to the Department of Education, Skills and Employment.³⁷ The purpose of the review was to evaluate the effectiveness of ASQA’s internal regulatory practices and processes. The Rapid Review made 24 short, medium and longer-term recommendations for improvement to ASQA’s regulatory practices and processes. All of the recommendations in this report were accepted by the Australian Government.

2. Summarised below are the recommendations from the Rapid Review, using the nine recommendation theme areas included in the report.

Long term vision for ASQA: Moving from input and compliance controls to a focus on self-assurance and excellence in training outcomes

1. Develop a shared understanding of what ‘quality’ and ‘outcomes’ look like, how these should be reflected in outcomes-focused standards and performance assessed by ASQA.

2. Develop new standards in consultation with the sector, with a view to decreasing prescriptive detail and increasing the focus on quality training delivery and outcomes for students and employers.

3. Develop resources to support providers to meet the standards including self-assessment tools to encourage continuous improvement.

4. Adjust ASQA’s approach to performance monitoring in line with revised standards and a focus on provider self-assurance.

5. Publish more information regarding provider performance to support students and employers to differentiate between providers.

6. Realign ASQA’s cost recovery arrangements to reflect ASQA’s new approach to performance monitoring.

Clarifying and modernising ASQA’s role and regulatory culture

7. Build a common understanding of ASQA’s role and regulatory approach, what stakeholders can and cannot expect of ASQA and areas of shared stakeholder responsibility.

Aligning governance arrangements with ASQA’s renewed focus

8. Change ASQA’s organisational structure and focus to strengthen strategic oversight and managerial capacity to:

• support implementation of critical reforms (including in relation to engagement and education, performance assessment and proportionate response to non-compliance)
• support change management
• strengthen internal systems and quality controls.

9. Establish an advisory council to improve access to high-level ongoing expert advice including in relation to ASQA’s strategic objectives and approach to regulation.

Strengthening engagement and education

10. Strengthen ASQA’s strategic stakeholder engagement and education, including to:

• build provider capacity for self-assurance
• co-design new regulatory tools with the sector (for example, to support self-assessment)
• identify common areas of non-compliance and develop consistent guidance for external stakeholders and for ASQA.

11. Strengthen strategic engagement with SSON, IRCs and SSOs to establish a feedback loop regarding broader VET reform, training packages, their intent and their implementation by providers.

Using intelligence to effectively monitor strategic risk and provider performance

12. Expand ASQA’s post-market monitoring to include risk-based and routine monitoring to support public confidence in the regulation of the sector.

Aligning audit practice to focus on self-assurance

13. Review the provider self-assessment tool and the annual declaration on compliance, such that they better support providers to identify and address non-compliance and drive continuous improvement.

14. Adopt a range of monitoring activities that can be selected based on risk and the purpose for monitoring, to better align regulatory effort to risk.

Appropriate and proportionate regulatory action where non-compliance is identified

15. More clearly distinguish the functions of monitoring provider performance and determining the most appropriate regulatory response where non-compliance is identified to improve consistency of audit outcomes and proportionality of regulatory response.

16. Give providers opportunity to remedy any identified non-compliance within 20 working days, or in accordance with an undertaking to remedy (providing a longer period within which to address the non-compliance on a more systemic and sustained basis).

17. Enhance opportunity to undertake early dispute resolution.

Meaningful reporting on provider performance

18. Revise the report that is given to providers following audit to more comprehensively describe any non-compliance as identified issues against the student-centred journey; reduce the amount of non-essential text; and distinguish between minor non-compliances or areas for improvement and more significant non-compliances.

19. Notify government agencies of provider non-compliance after the provider has had the opportunity to respond to the audit report (except where there is significant and immediate risk to public funding, students or others as a result of the identified non-compliance).

20. In the short term, build on the information ASQA publishes in respect of regulatory decisions to include the main reasons for the decision, so as to improve transparency of decision-making.

21. In the long-term, publish a summary of ASQA’s findings regarding the provider’s performance against the standards.

Supporting ASQA staff to deliver the outcomes

22. Strengthen induction training for ASQA auditors and decision-makers, including to support consistent regulatory practice, decision-making and documentation.

23. Implement a formal program for continuing professional development for ASQA auditors and decision-makers.

24. Implement a program of internal quality assurance, including regular:

• opportunities for moderation of audit outcomes and decision-making
• internal review of audit reports and compliance outcomes.