Introduction

1. The objective of Australia’s aid program is ‘to assist developing countries to reduce poverty and achieve sustainable development, in line with Australia’s national interest.’ In support of this objective, Australia’s aid investment in education is expected to be around $744 million, or 19 per cent of Australia’s Official Development Assistance (ODA) in 2010–11. Tertiary training is a major focus of Australia’s aid for education spending, receiving around one third of the total allocated to education, and close to seven per cent of total ODA. Other aid for education spending is largely focused on basic (primary and lower secondary) education initiatives, including, for example, the aid program’s large school building program in Indonesia.

2. Tertiary training—which encompasses post-secondary education, including vocational and technical training institutions and universities—is widely recognised as being critical to a country’s development prospects_.^[1] The World Bank has observed that:

Tertiary education can offer better opportunities and life chances for students from low income and other minority groups, thereby increasing their employability, income prospects and social mobility and decreasing income inequality. At the same time, the norms, values, ethics and knowledge that tertiary education can impart to students contribute to the social capital necessary to construct healthy civil societies and socially cohesive cultures, as well as to achieve good governance and democratic political systems. [2]

3. Aid for tertiary training has been an important element of Australia’s aid program since the 1950s, when scholarships to study in Australia were offered under the Colombo Plan. Scholarships to study in Australia continue to be a major component of tertiary training assistance, accounting for close to 80 per cent of tertiary training expenditure. In addition, assistance is also provided in the form of scholarships for study in non-Australian institutions, direct support for overseas tertiary training institutions, and direct provision of tertiary training overseas, notably through the Australia–Pacific Technical College (APTC). The APTC delivers technical and vocational training courses in four campuses across the Pacific.[3]

4. The Australian Agency for International Development (AusAID) is the main Australian Government agency responsible for managing the aid program, and provides the bulk of tertiary training assistance. A small number of in-Australia scholarships are also provided by other Australian Government departments. These are mostly administered by the Department of Education, Employment and Workplace Relations (DEEWR) through its Endeavour scholarships program.

5. Tertiary training assistance forms one component of AusAID’s country programs, which are implemented by a network of overseas country offices. Country offices are guided in their implementation of country programs by country strategies or Pacific Partnerships for Development, which provide a basis for agreement with aid recipients on how Australia will contribute to the country’s development objectives. These are multi-year (usually five-year) plans developed to explain the Australian Government’s position on ODA engagement in a particular country.

6. From 2004–05 onwards, AusAID has managed a significant expansion of the aid program, from $2.7 billion (in real prices) to an estimated $4.2 billion in 2010–11.[4] Aid flows are expected to increase to over $8 billion annually in the next five years, to meet the Government’s objective of increasing the proportion of ODA to gross national income to 0.50 per cent by 2015–16.[5] In the area of education and training, aid spending could amount to as much $1.6 billion by 2015–16, a tripling of current allocations.

Audit objective and scope

7. The objective of the audit was to assess the effectiveness of AusAID’s management of tertiary training assistance. Particular emphasis was given to:

• planning of tertiary training assistance;
• management of in-Australia scholarship programs;
• management of other forms of tertiary training assistance, including support for overseas tertiary training institutions, and direct provision of tertiary training overseas;
• devolved management of tertiary training assistance; and
• monitoring, evaluation, and reporting.

8. For the purposes of the audit, tertiary training assistance is defined as the support provided by the aid program to enable individuals to obtain formal qualifications at an advanced (that is post-secondary school) level, including through universities, and technical and vocational training institutions.

9. As previously noted, tertiary training is recognised as being critical in providing the human capital required to support sustainable development. However, it is but one component of Australia’s aid program, and of education assistance in particular. Funding for tertiary training depends, in part, on judgements by the Australian Government, and AusAID, in consultation with aid recipients, about the importance of tertiary training relative to other funding priorities for individual countries. As such, consideration was also given to how activities outside the tertiary training sector may affect the tertiary training assistance offered by Australia, and the effectiveness of this assistance.

Overall conclusion

10. Australia provides aid to over 80 countries—small and large, fast-growing and slow, peaceful and unstable—each of which has its own needs. In some countries or regions, Australian aid plays only a niche role. In others, particularly in the Pacific, Australia is the dominant donor. The amount and types of tertiary training assistance offered by Australia depend on judgements by the Government (informed by advice from AusAID), and AusAID itself, about the relative needs of, and funding priorities for, individual countries. Importantly, it also depends on the role governments consider the aid program should play in supporting national interest or foreign policy objectives.

11. As every country’s circumstances are different, it is difficult to generalise about the outcomes of Australian aid or tertiary training assistance in particular. The outcomes of aid, including any unintended consequences, can usually only be assessed in the specific contexts of the countries that are supported. Even at the country level, it is important to recognise that the outcomes of aid may be constrained by factors that are outside the direct control of aid agencies. This means that, even if interventions are well-conceived and effectively implemented, their effectiveness can be undermined by corruption, conflict, weak institutions, or chronic funding shortages within countries receiving aid.

12. In spite of these constraints, many of Australia’s tertiary training interventions are having a positive impact. Evaluations of Australia’s large in-Australia scholarship programs have consistently established that individuals benefit from the program. Often they are promoted to more senior positions, and are able to have a greater influence in their professional roles. Beyond the benefits to individuals, there is also evidence to suggest the training provided has made a contribution to reform efforts in certain countries. In addition, direct support for tertiary training institutions (including scholarships to study at them) has made a tangible contribution to improving tertiary training outcomes, particularly in the Pacific region, where the bulk of Australia’s support has been focused.

13. Overall, AusAID’s management of tertiary training assistance has been broadly effective. In designing tertiary training initiatives, AusAID targets assistance to the needs of aid recipients, and implements tertiary training initiatives in a way that is appropriate to local contexts. AusAID’s regular monitoring and review of the performance of initiatives is robust, and the outcomes of these reviews are a major influence on the design of future initiatives. AusAID has also made good progress in coordinating the delivery of tertiary training programs in the Pacific by establishing joint management arrangements with the New Zealand Government’s aid program. The assistance provided by AusAID is generally well regarded by aid recipients.

14. Consistent with international agreements on reforming how aid is delivered, the Australian Government has committed to more closely aligning assistance with the financing needs of aid recipients, and increasing the predictability of aid. Continued progress in this area will be particularly important in countries where Australian aid accounts for a large proportion both of total aid flows, and of recipient government resources. In pursuing these objectives, a notable gap in AusAID’s management of tertiary training assistance has been a failure to build greater certainty into the implementation of its country programs with longer-term financing commitments to priority sectors that are closely aligned to the needs and policies of aid recipients.

15. This gap has increased the potential for misalignment between aid allocations and needs on the ground. The impact of this gap has been particularly evident in the Pacific, where AusAID has struggled to develop a well-balanced and predictable program of tertiary training assistance, both as a mechanism for addressing the higher order capacity needs of recipient countries, and as an effective complement to other education and aid assistance. With the aid program undergoing a rapid expansion, the effectiveness of AusAID’s country offices in providing more stable, balanced and predictable assistance will depend on AusAID corporately achieving greater strategic clarity about where, and to what sectors additional resources will be allocated.

16. The ANAO has identified a number of other areas where AusAID’s management of tertiary training assistance could be improved. In particular, proliferation of scholarships schemes has created unnecessary administrative complexity and could confuse potential candidates. Streamlining of scholarship programs would help to improve the targeting of tertiary training assistance and the quality of information collected about the focus and scale of assistance. This information will help AusAID to better explain the rationale for its tertiary training assistance program, and to better understand its contribution to development efforts. A stronger focus on monitoring scholarships alumni after they complete their studies, and the establishment of a consistent methodology for evaluating the impacts of scholarship programs, as well as better geographical coverage of evaluations, would provide stronger evidence about the impact of scholarship programs.

17. The ANAO has made three recommendations aimed at improving AusAID’s management of the aid for tertiary training.

Key findings

Planning of tertiary training assistance

18. Aid policies under successive governments have generally not stated the proportion of aid that will be devoted to education; where education resources will be allocated; or how resources will be divided between the different education sub-sectors. Planning of education assistance has instead relied on AusAID’s country offices to identify investment priorities that are appropriate to the specific needs and circumstances of partner countries, and that reflect aid policy objectives and Australia’s national interests. The focus of the assistance to be provided, in terms of priority sectors and initiatives, should then be formalised in country strategies, which are multi-year (usually five-year) plans developed by AusAID. Country strategies are expected to define the Australian Government’s position on what priority sectors Australia's aid will target, why those priority sectors have been chosen, and how aid objectives will be achieved.

19. Historically, AusAID has found it difficult to maintain up-to-date country strategies. Until recently, nine of the top 20 recipients of country program aid did not have approved country strategies. In addition, completed strategies have generally not contained a clear delineation of aid priorities, or been supported by long-term spending commitments that will address those priorities. These shortcomings have reduced the effectiveness of country strategies in supporting coherent and disciplined resource allocation decisions by AusAID, including those affecting tertiary training assistance. They have also made it difficult for AusAID to integrate its support with the budgets and policies of recipient country governments.

20. Against this background, development of a well-balanced and sustained approach to providing education and tertiary training assistance across the aid program has presented challenges for AusAID. In PNG, the Pacific and Timor-Leste, the region with the highest financing needs and poorest performing education systems, education assistance reduced from over $150 million in the early 2000s to less than $100 million by 2006–07. While subsequent increases in support for education returned education spending to historical levels, around 60 per cent of the new education spending in the region has come from initiatives that are not integrated with the budgets and policies of recipient country governments, such as the APTC, and scholarships to study in Australia.

21. The release in early 2011 of 12 country strategies means they now cover most of AusAID’s major country programs. AusAID has also revised its approach to country strategy development, which has provided a basis for more substantive consideration, in a small number of cases, of partner country needs in specific sectors and multi-year resource commitments to them. Continued progress in this area will be important in supporting a sharper delineation of aid priorities, and a more consolidated focus on addressing these priorities.

22. With the aid program growing rapidly, and education assistance projected to almost triple current spending levels to $1.6 billion by 2015–16, the capacity of AusAID’s country offices to develop more focused, predictable education and tertiary training assistance programs will depend on achieving greater certainty about where, and to what sectors, it will be provided.

23. To provide this certainty, AusAID has developed a strategic framework to guide the expansion of education assistance. The influence of the strategic framework would be increased by translating its intent into long-term, indicative budget allocations that address aid recipients’ education sector needs in a balanced manner, and appropriately reflect Australia’s national interests. Within its overall budget for education, AusAID will need to consider the appropriate level of investment in: post-secondary education relative to other education sub-sectors; direct support for aid recipients’ tertiary training systems and institutions; and indirect support provided through avenues such as the APTC and scholarships to study in Australia.

24. Building on its strategic framework for education assistance, AusAID is currently developing an education strategy, which will provide a vehicle for these considerations. This could also provide an avenue for publicly explaining the rationale for choices that will be made about the focus of increased education and tertiary training assistance.

Scholarships to study in Australia

25. In 2009–10, the aid program provided over 2000 scholarships to study in Australian tertiary institutions, to recipients from over 70 countries. They represent around five per cent of Australian aid expenditure globally, and a significant share of aid expenditure in all of the regions that are the focus of the aid program, including in many countries that receive no other assistance.^[6] By 2015–16, the number of scholarships offered is expected to double, and the pool of recipient countries will expand considerably.

26. The global focus of scholarships to study in Australia reflects, in part, the role they play in Australia’s foreign policy agenda, including their role in improving people-to-people links between Australia and its partners. The presence of Australian educated alumni within partner institutions facilitates and supports Australia’s diplomatic relationships with partner countries. Australian foreign affairs officials consulted during the audit emphasised the contribution scholarships to study in Australia made in improving access to senior government officials, and communication with partner government officials across cultural boundaries. The ANAO observed during fieldwork to Indonesia, PNG, and Fiji that Australian scholarships alumni were prominent in senior positions in partner government agencies that are important to AusAID and other Australian Government agencies.

27. Evaluations of AusAID’s in-Australia scholarships programs have consistently highlighted the benefits they provide to recipients, including promotion to more senior positions, and increased capacity to exert influence in their professional roles. Evaluations of many of AusAID’s largest scholarships programs have found that most alumni return home and derive benefit from their newly found skills. This was particularly evident for programs in Indonesia, the Philippines, Vietnam and Laos, which together account for over 40 per cent of annual scholarships expenditure. There is also evidence to suggest that the training provided has had an impact beyond benefits to individuals in certain countries. In Laos and Samoa, reviews undertaken by AusAID in 2003 concluded that scholarships to study in Australia had made a measurable contribution to reform efforts by recipient governments.

28. The ANAO identified a number of countries where scholarship allocations has increased substantially at a time when country offices have been unable to attract sufficient numbers of suitable candidates. In the case of AusAID’s $17 million annual investment in scholarships to PNG, high course extension and failure rates have incurred costs amounting to about 17 per cent of scholarship program expenditure over the past five years. In some smaller programs, such as the Solomon Islands and Kiribati, AusAID evaluations and performance reports have observed that increased scholarships assistance has placed heavy demands on the limited human resource pools in those countries, and put AusAID’s broader capacity development efforts at risk.

29. Identifying the appropriate number of scholarships to offer to aid recipients depends, in part, on effective coordination between AusAID and other donors. AusAID has made substantial progress in coordinating the delivery of scholarships programs with other donors and, in particular, through the joint management of programs in the Pacific with the New Zealand Government. In addition, AusAID has also played a lead role in efforts to improve donor coordination in Indonesia.

30. Improved coordination between AusAID and DEEWR scholarship programs, which forms part of the current Government’s forward agenda, has been more challenging. The Government’s Australia Awards initiative marks a second attempt in recent times to better integrate the scholarship programs managed by the different departments. The first attempt under the previous Government, the Australian Scholarships initiative, presented a number of challenges from its inception. While progress was made in some areas, most of the joint activities envisaged by the initiative were not fully implemented. In countries with small human resource pools, such as most of the countries in the Pacific, it is important that DEEWR and AusAID work together to mitigate the risk that the number of scholarships offered will undermine the effectiveness of other capacity development efforts.

Other tertiary training assistance

31. In addition to scholarships to study in Australia, AusAID provides a number of other types of tertiary training assistance. This includes direct support for aid recipients’ tertiary training systems and institutions, and scholarships to study at developing country institutions. It also includes the provision of tertiary training overseas, through initiatives such as the APTC, which provides technical and vocational training in four countries using Australian registered training providers.

32. Over the past decade, the major focus of AusAID’s direct support for tertiary training systems and institutions has been on the Pacific and Timor-Leste, where tertiary training outcomes have generally either stagnated, or declined. The cornerstone of AusAID’s tertiary training assistance in the Pacific has been the long-standing support it has provided to the University of the South Pacific (USP) in Fiji, which has amounted to over $76 million in real terms since 1995–96. The impacts of this support are visible and tangible, including in the form of improved information systems, libraries, classrooms, laboratories and regional campuses. Australia’s regular funding contributions have helped the university deal with some significant financial challenges, and to maintain its position as the premier university in the Pacific region (excluding Australia and New Zealand). In addition, AusAID’s program of scholarships to study at USP and other regional institutions (almost $150 million over the last 15 years) has been a cost-effective and valuable way of supporting tertiary training institutions and outcomes in the region.

33. AusAID’s direct support for other tertiary training systems and institutions in the Pacific and Timor-Leste has been smaller in scale, and less sustained. While beneficiaries have generally valued the assistance provided, its impacts are difficult to discern at a sectoral level. Two factors have contributed to this situation. Firstly, the broad focus of most country programs has made it difficult to provide sustained support on a sufficient scale to have a discernable impact on tertiary training outcomes. Secondly, the availability of resources to support education and tertiary training efforts has been affected by decisions taken over the last decade to allocate aid to other priority areas. This includes the deployment of expert consultants, to provide (where it does not exist) or to build the capacity of government institutions in these countries. In combination, these factors have contributed to a decline in the resources AusAID provides to directly support tertiary training systems and institutions, from over $34 million annually at the beginning of the decade to around $18 million in more recent times.

34. A recent addition to AusAID’s tertiary training assistance in the region has been the establishment of a regional technical and vocational training institution, the Australia–Pacific Technical College (APTC). The APTC accounts for over 20 per cent of AusAID’s education spending in the Pacific. AusAID’s establishment of the college following its announcement in 2005 and its learning outcomes have been very successful. The APTC has trained some 2424 graduates who have received Australian standard qualifications in a range of disciplines. Observations during field visits to two of the college’s campuses in Fiji and PNG and discussions with employer groups confirmed that, in a relatively short space of time, the APTC has established a good reputation among students and employers.

35. While the APTC represents a valued addition to the tertiary training provision in the region, there is scope to improve its sustainability and effectiveness. The APTC is Australian owned and operated, which means it does not directly support aid recipients’ technical and vocational training systems. The APTC endeavours to avoid competing with local providers by assuming a niche role at the upper end of the training market not covered by local suppliers. However, partner government stakeholders consulted during the audit raised the potential for the APTC to compete with and undermine the capacity of local providers, which has affected their support for the initiative. In addition, delivering Australian standard technical training in developing countries is expensive. A submission to the recently established independent review of the aid program has suggested that the cost of a qualification from the Fiji National University is about one quarter that of an equivalent qualification obtained through the APTC. Given the relatively high cost of the initiative (over $30 million annually), there will be an ongoing need to monitor and improve the value for money of courses delivered by the APTC.

Management of tertiary training assistance

36. To cater for the unique characteristics of each of the countries they support, AusAID has established a devolved management model, which delegates responsibility and accountability for the implementation of its development strategy in a country to its country program offices. These offices have responsibility for program design, implementation and management, in-country policy dialogue, managing local stakeholder relationships, monitoring and performance assessment.

37. The devolved management approach provides country offices with the flexibility to tailor the design of aid interventions to country contexts. In administering tertiary training programs, this includes consideration of: the merits of coordination with other donors; the relative merits and costs of in-country or regional versus in-Australia scholarships; the way that training is targeted to the needs of countries and to aid objectives; and the amount of pre-departure training required to support scholars to perform well academically. The approach has also supported the development of greater country knowledge within AusAID, and stronger relationships with partner government personnel. It has also supported improved coordination with other donors.

38. There are a number of areas where AusAID could preserve the strengths of its devolved management model, while at the same time address some of the unintended outcomes that have resulted from the way it has been implemented. These include:

• proliferation of different scholarship programs, which has created a high level of administrative complexity and duplication of effort, and made it difficult for country programs to coherently target tertiary training assistance;
• inconsistent approaches to monitoring and evaluation, which have reduced the ability to compare the performance of different programs; and
• deficiencies in information management, which have contributed to a loss of central oversight over the scope and scale of tertiary training initiatives globally.

Monitoring, evaluation and reporting

39. Evaluating outcomes from development investments, which may only be realised over a long time period, is very difficult. In the case of overseas scholarships programs, for example, some years will elapse before a returning student might contribute to their country’s development, and even then, it would be very difficult to establish a causal connection between that individual’s efforts, and broader economic or developmental outcomes.

40. Recognising this, monitoring and evaluation is taken very seriously by AusAID. AusAID closely monitors the performance of tertiary training initiatives and their contribution to country program objectives. Regular independent reviews and evaluations have an important influence over the design of tertiary training initiatives and, taken together, provide valuable insights into their performance, and the benefits and pitfalls of education aid in the various contexts in which AusAID operates. In addition, the commencement of joint reviews of Pacific scholarship programs with the New Zealand Government is providing a stronger basis for considering the overall effectiveness of donor scholarships and, by implication, Australia’s contributions.

41. This body of research provides AusAID with a strong foundation to draw from in addressing weaknesses in the quality and coverage of its evaluative work. While coverage by evaluations of programs in East Asia has been strong, AusAID has not completed any impact evaluations in South Asia and Africa. The results of the few evaluations AusAID has completed in the Pacific have been inconclusive because of methodological flaws. This means there is limited empirical evidence about the performance of scholarships in many countries or regions where substantial investments are being made. At a time when in-Australia scholarships assistance is being expanded, including to a larger number of countries, it will be important that AusAID strengthens the evaluation of the long-term impacts of scholarship investments, and advises the Government accordingly.

42. Until recently, there has been limited external reporting on tertiary training support. Since 2005, AusAID has completed over 25 significant reviews and evaluations of its support for tertiary training, which provide insights into what is working, what is not, and the reasons why. However, none of this work has been made available to external stakeholders, contrary to AusAID’s internal policy position that internal evaluation reports should be released publicly. Similarly, until recent times, reporting on the scope, scale and performance of tertiary training initiatives has been minimal, and inconsistencies in the information that has been provided has made it very difficult to get an accurate picture of AusAID’s allocation of resources between different education sectors and programs.

43. AusAID has made some progress in improving the quality and quantity of information that is available about the aid program, including information about tertiary training expenditure. This includes the reintroduction, after a three-year hiatus, of a detailed statistical summary of aid expenditure, and increased public release of reviews and evaluation reports. To build on this progress, greater attention could be given to the clarity and consistency of external reporting, and to disseminating the results of reviews and evaluations of tertiary training initiatives, most of which remain unpublished.

Summary of agency response

AusAID

44. AusAID welcomes the ANAO’s report AusAID’s Management of Tertiary Training Assistance, in particular, its recognition that AusAID’s management of tertiary training assistance has been broadly effective, targeting the needs of recipients and implementing activities in ways that are appropriate to local contexts.

45. The report highlights a number of areas in the education sector, and in particular scholarships, where AusAID is already implementing reforms. These include developing a new education strategy to guide investments in education over the next five years, increasing the flexibility and efficiency of our tertiary assistance, improving the management of scholarships, and implementing a strengthened approach to monitoring and evaluation of scholarships.

46. The recommendations of the report focus on areas where work is already well progressed. The Government’s response to the Independent Review of Aid Effectiveness, commissioned in November 2010 and submitted to the Government in April 2011, will also help determine how AusAID implements the recommendations of this present audit.

Footnotes

[1] According to the World Bank, tertiary education broadly refers to all post-secondary education. This includes not only universities but also a range of public and private tertiary institutions such as advanced technical training institutes, community colleges, nursing schools, and teacher training institutions. Together, these form a network of institutions that support the production of higher-order human capital required for development. See: <http://web.worldbank.org/WBSITE/ EXTERNAL/TOPICS/EXTEDUCATION/
0,,contentMDK:20298183~menuPK:617592~pagePK:148956~piPK:216618~theSitePK:282386,00.html> [Accessed 28 February 2011].

[2] World Bank, Constructing knowledge societies: New challenges for tertiary education, 2002, p. 5. See:<http://siteresources.worldbank.org/TERTIARYEDUCATION/Resources/Documents/Constructing-Knowledge-Societies/ConstructingKnowledgeSocieties.pdf> [Accessed 28 February 2011].

[3] In Fiji, Samoa, Papua New Guinea, and Vanuatu.

[4] Commonwealth of Australia, Budget 2010–11, Australia’s International Development Assistance: A Good International Citizen, Statement by the Honourable Stephen Smith, MP, Minister for Foreign Affairs, and the Honourable Bob McMullan, MP, Parliamentary Secretary for International Development Assistance, 11 May 2010, Table 17.

[5] ANAO Audit Report No.15 2009–10, AusAID’s Management of the Expanding Australian Aid Program, p. 30.

[6] Scholarships to study in Australia accounts for over 10 per cent of aid expenditure in at least 12 countries. In countries such as Mongolia and Bhutan they account for almost all of the assistance provided.