Background

The Department of Education, Science and Training's (DEST's) International Education Group (IEG), previously known as Australian Education International Group (AEI), and DEST's International Network, have primary responsibility for administering the Australian Government's international education and training policy framework.¹

DEST estimates that the total value to the Australian economy of all international education activity was $5.7 billion in 2003, making education the third largest service export. The industry mainly provides educational services in Australia to students who are not residents of Australia. Total enrolments for 2004 were 322 776, up six per cent on 2003.

IEG has undergone major reform in the past three years, which has considerably increased its role in Australia's international education and training export industry with a concomitant increase in Budget funding for the Group's activities.

IEG, together with the DEST International Network, administers five key roles in support of the Australian international education and training export industry. These roles are: facilitating the export of Australian education and training services; industry regulation, compliance, enforcement and the assessment of overseas qualifications; policy development, coordination and advice; stakeholder engagement; and the administration of grant programs. In 2004-05, the Budget funding provided to undertake these roles is in excess of $60 million.

The objective of the audit was to assess the administrative effectiveness of DEST's support for the Australian education and training export industry, including its regulatory and associated roles, and how it monitors and reports on its performance in undertaking these roles.

Key findings

Effectiveness of IEG's activities relating to Australia's education and training export industry (Chapter 2)

The ANAO identified a number of weaknesses in IEG's administration of the wide range of activities it undertakes in support of the Australian education and training export industry and relevant international relationships.

Planning

While IEG has developed a draft strategic plan, it is yet to finalise this plan. The ANAO considers that IEG, as a priority, needs to finalise its strategic plan to inform business planning in the Group. The IEG Group and Branch business plans need to be supportive of, and consistent with, the Group's strategic plan. To support the successful implementation of these plans, appropriate project plans, including risk management schedules, should routinely be developed. In relation to the export facilitation role undertaken by IEG, it is important that future monitoring and reporting on the implementation of relevant plans include marketing and promotional outcomes.

Overseas skills recognition

The National Office of Overseas Skills Recognition (NOOSR) is incorporated within IEG. In 2004, the median turnaround time for completion of assessments by IEG of overseas qualifications was about nine weeks.²  This is much longer than performance targets IEG has recently introduced, which require assessments to be completed within 30 days, and advice to individuals and other clients on assessments to be provided within 15 days.

IEG's main system for automated processing, and recording of assessment decisions, is the NOOSR Information Intranet System, which was introduced in 2001. IEG advised the ANAO that the NOOSR Information Intranet System could be more effective particularly with respect to speed, storage and retrevial of data. IEG has successfully sought funding through DEST's Business Technology Improvements process to enhance this system, and this work has been underway since 2003.³

Universities, and post secondary non-university institutions, make assessments of overseas students' qualifications for their own purposes using information from IEG's Country Education Profiles. Many of the Country Education Profiles, covering education systems in some 90 countries, have not been updated for over 10 years, and are not available online through the AEI-Online website.

IEG commenced development of a project plan, in July 2004, to have its Country Education Profiles made available through the AEI-Online website. The purpose of this project was to make access easier for institutions and others, and potentially reduce the number of enquiries to the Educational Standards Branch-NOOSR. Although funding for the project commenced in July 2004, IEG advised the ANAO that efforts to date had focussed on modelling IT infrastructure options, analysing client requirements and researching and updating Country Education Profile content. Consequently, only a small number of County Education Profiles will be converted to online format by July 2005.

A large proportion of overseas students coming to Australia reside in a relatively small number of countries (such as Korea, Hong Kong, Indonesia, and India). Given this, and the limited progress achieved thus far with this project, the ANAO suggests that that IEG take into account the relative user demand for individual Country Education Profiles in deciding the priority order for making Country Education Profiles available through the AEI-Online website.

Industry regulation

IEG undertakes a range of compliance and enforcement activities in relation to international education providers. These include: on-site assessments of providers' compliance with the Education Services for Overseas Students Act 2000 (ESOS Act); joint compliance reviews with relevant State and Territory government agencies; provision of industry seminars and workshops; and answering hotline and mailbox queries concerning compliance with ESOS legislation.

To assist in its compliance and enforcement role, IEG has developed a risk matrix model that assesses the risk of providers committing breaches of ESOS legislation and related policies and practices. The model contains risk weightings relating to: offences; facilities and resourcing; complaints; enforcement activity; financial viability; and overall risk rating.

This matrix is useful in directing IEG compliance and enforcement resources towards providers that are relatively likely not to comply with ESOS legislation and related policies and practices. However, the matrix does not contain standard risk management approaches, such as separating likelihood and consequence, nor does it feed into a clearly defined framework for controlling risks, determining residual risks, and monitoring and evaluating risk treatments. Further, the ANAO found that few risk assessments have been undertaken, and IEG Branch plans contain little reference to compliance and enforcement.

IEG advised the ANAO that it is reviewing its compliance activities, and is developing new tools to assist in administering its compliance responsibilities. These include a compliance package and case management system.

Annual Registration Charge

IEG recovers some of the costs of administering the ESOS Act, including its compliance activities, through a compulsory Annual Registration Charge (ARC) levied on providers of international education and training services. The charge comprises a base fee plus a charge per student enrolment. In 2003 04, DEST collected some $6 million from the ARC.

ANAO noted that IEG had detected errors in the annual registration fees remitted by providers in 2003-04, resulting in overpayments of $200 000 and underpayments of $400 000. Providers calculate the amount of the charge they assess they are liable for and remit this to IEG. However, the ESOS Act does not specify that providers must use the enrolment data contained in IEG's Provider Registration and International Students Management System (PRISMS) computer system. Accordingly, currently providers may use their own enrolment data instead, which may be different from the data in the PRISMS system, affecting the calculation of the charge. Additionally, providers may make an error in calculating the fee payable.

To address these issues, the ANAO has recommended that IEG make it clear to providers, that PRISMS data is available to use when calculating the ARC. The ANAO also recommends IEG undertake an annual reconciliation between the payments received from providers and the ARC calculated as payable on the basis of the data in PRISMS, to identify significant variances and, where cost effective, take remedial action (see Recommendation No. 1).

Administration of grant programs

IEG does not have a centralised grant management system. This makes it difficult for IEG to comprehensively determine what activities have been undertaken, and how such activities contribute to meeting IEG's key deliverables and outputs. The ANAO considers that the implementation of such a grant management system has the capacity to:

• accommodate systems-generated monitoring of key milestones; 
•  monitor actual grant payments against budgets; 
•  interface with other organisational systems, for example the Department's Financial Management Information System; and 
•  facilitate exchange of information and or documentation between, and within, IEG and grant recipients.⁴

The ANAO suggests that IEG's administration of grants could be improved, through adopting those better practices outlined in the ANAO's May 2002 Better Practice Guide that are not currently included in its grant management frameworks or practices. In addition to the development of a centralised grants management system, these include:

• an improved reporting framework, setting out the status of each grant program, and the linkage to IEG's outputs and outcome; and
• regular reviews to provide assurance to IEG management that the grant programs are being appropriately administered.

IEG's performance reporting framework (Chapter 3)

IEG is responsible for delivering Output Group 3.3 of DEST's current Outcome/Output Framework namely, ‘Support for the Australian education and training export industry and international relationships'.⁵ While it is likely that IEG's wide range of activities are beneficial in terms of achieving the aims of Output Group 3.3, and so contributing to DEST's Outcome 3⁶ , the ANAO considers that a lack of adequate performance information and indicators is significantly hampering IEG's capacity to both gauge and demonstrate the level of effectiveness of its activities and the extent of its contribution.

This has two effects on DEST's performance measurement and reporting structure:

• the achievement of the IEG functions cannot be adequately measured, and, therefore, the performance of IEG cannot be adequately assessed; and 
•  a flow-on effect, whereby the reliability and quality of the higher-level departmental indicators of performance, which rely on these lower level measurements as their basis, are compromised.

Therefore, in assessing DEST's performance information at an organisational level, the ANAO found it difficult to ascertain how well IEG had achieved its outputs relating to Australia's international education and training industry or the extent of its contribution to the relevant part of DEST's Outcome 3, that is ‘Australia is engaged internationally on science, education and training to advance our social development and economic growth.'

The fieldwork for this audit was primarily undertaken in IEG's National Office in Canberra between May and September 2004. Accordingly, ANAO's analysis was undertaken against DEST's 2004–05 Portfolio Budget Statements (PBS). However, Outcome 3 and Output Group 3.3 remain the same in the department's 2005-06 PBS.

The DEST 2004–05 PBS includes two strategic priorities to explain the Australian Government's medium-term goals relating to IEG's outcome and outputs⁷ . However, on the basis of the performance information currently available, it would be difficult for the Department to know when it had achieved its desired performance relating to these priorities.

ANAO also noted that IEG's performance measurement framework did not incorporate the expectations of key stakeholders concerning IEG, and their perspectives of IEG's performance in satisfying these expectations. The ANAO considers that there would be benefit in IEG including customer satisfaction surveys and/or other feedback tools as key components of its performance monitoring framework.

Overall audit conclusion

As noted above, it is likely that there are benefits to Australia's international education and training industry from IEG's activities. However, the lack of effective performance monitoring and reporting across all of IEG's main functions made it very difficult for the ANAO to identify even the broad magnitude of such benefits.

An effective performance reporting and monitoring system is a key aspect of a well-governed agency. Good governance requires that the agency have a structured and regular system of performance monitoring and review. This system should be aligned with the agency's outcomes and outputs framework and generate information that is appropriate for both internal and external performance management needs and external reporting requirements such as the annual report.⁸

In this context, it is important that IEG and DEST implement a performance framework that effectively monitors and reports on IEG's performance. This framework needs to provide measurable indicators of effectiveness, from the individual branch level through to DEST's reporting to the Parliament. This will allow DEST to better demonstrate how it is achieving its outcome and outputs relating to Australia's international education and training export industry.

The ANAO also identified opportunities for IEG to improve the administration of its various activities, particularly:

• planning, both at the Group level and across all of IEG's branches, with most current IEG plans lacking measurable performance indicators and targets, and some plans being incomplete, or inconsistent in format and content; 
•  the speed of assessments of overseas qualifications by the National Office of Overseas Skills Recognition within IEG; 
•  IEG's risk management approach to compliance/enforcement activities undertaken by the Group as part of its regulation of the Australian education and training export industry; 
•  controls to ensure the correctness of Annual Registration Charges remitted under the ESOS Act by providers of education services to overseas students; and 
•  scope for improvement to IEG's grants administration systems and processes.

Recommendations

The ANAO made four recommendations to improve the administrative effectiveness of DEST's support for the Australian education and training export industry.

DEST response

Appendix 2 of the audit report contains detailed comments provided by DEST on the proposed report. DEST's summary response appears below:

The Department of Education, Science and Training generally concurs with the findings of the ANAO report, and fully agrees with its recommendations. It is noted that there are several areas identified in the Report where International Education Group (IEG) could improve its administrative processes and performance management framework. These areas became evident in the course of the audit and, in the main, immediate action was taken by the Department to address these aspects of administration, management and reporting. IEG has thus already made significant progress in implementing the Report's recommendations and the various suggestions to identify appropriate performance indicators and targets, and to enhance IEG's performance reporting framework.

Footnotes

1 DEST's Australian Education International Group was restructured, and renamed IEG in late 2004. During the course of the fieldwork for this audit, IEG was known as AEI. Also, IEG together with the DEST International Network, is known outside the Department as AEI.

2 IEG advised of this median assessment time, and noted that the long time frame was partly because many assessments require information from overseas, and that some cases are referred to the Education Standards Branch-NOOSR expert advisory panel, which only meets twice a year.

3 IEG has advised the ANAO that in 2005–06 the Group is hoping to enhance the reporting and tracking features of the NOOSR Information Intranet System.

4 ANAO Audit Report No. 16 2002–03, The Administration of Grants (Post–Approval) in Small to Medium Organisations, p. 42.

5 Portfolio Budget Statements 2005-06, Education, Science and Training Portfolio, p. 36.

6 Portfolio Budget Statements 2005-06, Education, Science and Training Portfolio, Outcome 3: ‘Australia has a strong science, research and innovation capacity and is engaged internationally on science, education and training to advance our social development and economic growth'.

7 In the DEST 2005-06 PBS, these are slightly reworded and Strategic Priority 13 renumbered to 14.

8 Better Practice in Annual Performance Reporting, Department of Finance and Administration and Australian National Audit Office, April 2004, p. 1.