The Indigenous Advancement Strategy

1.1 Through the Indigenous Advancement Strategy (IAS) the Australian Government funds and delivers a range of programs specifically for Indigenous Australians. Initially, $4.8 billion was committed to the IAS over four years from 2014–15. In the 2019–20 Budget, the Australian Government allocated an additional $5.2 billion over four years from 2019–20 to 2022–23 to the IAS, for grant funding processes and administered procurement activities.

1.2 The National Indigenous Australians Agency (NIAA) administers the IAS. NIAA was established as an executive agency within the Prime Minister and Cabinet portfolio on 1 July 2019 and is the lead entity for Commonwealth policy development, program design, implementation and service delivery for Aboriginal and Torres Strait Islander peoples. Between 2013 and 2019 the Department of the Prime Minister and Cabinet (PM&C) was the lead agency for Aboriginal and Torres Strait Islander affairs within the Australian Government and administered the IAS.⁶

1.3 There are six IAS programs⁷:

• Program 1.1 — Jobs, Land and Economy;
• Program 1.2 — Children and Schooling;
• Program 1.3 — Safety and Wellbeing;
• Program 1.4 — Culture and Capability;
• Program 1.5 — Remote Australia Strategies; and
• Program 1.6 — Research and Evaluation.

1.4 The Children and Schooling program and the Safety and Wellbeing program are examined in this report. The objectives of the Children and Schooling program are to:

Get children to school, particularly in remote Indigenous communities, improving education outcomes and supporting families to give children a good start in life. This program includes measures to improve access to further education.⁸

1.5 The program’s focus is on increased school attendance and improved educational outcomes that lead to employment. A broad range of activities are funded under the program, including activities that aim to improve family and parenting support; early childhood development, care and education; school education; youth engagement and transition; and higher education.⁹ In 2019–20, the Children and Schooling program’s administered budget was $305.04 million.¹⁰ As at 30 June 2020, 709 grant activities were funded through the Children and Schooling program.

1.6 The objectives of the Safety and Wellbeing program are to:

Ensure that the ordinary law of the land applies in Indigenous communities, and ensure that Indigenous people enjoy similar levels of physical, emotional and social wellbeing enjoyed by other Australians.¹¹

1.7 The program seeks to increase levels of community safety and individual wellbeing by funding initiatives that aim to: reduce all kinds of violence and provide support for victims; reduce alcohol and substance misuse; reduce the rates of offending or recidivism; and enhance connection to family and community and build the capacity of individuals to respond to life stressors.¹² In 2019–20 the Safety and Wellbeing program’s administered budget was $279.88 million.¹³ As at 30 June 2020, 535 grant activities were funded through the Safety and Wellbeing program.

The National Indigenous Reform Agreement

1.8 The IAS Grant Guidelines note that Australia will only achieve its true potential when all Australians, including Indigenous Australians, have equal opportunity to participate in all aspects of society.¹⁴ Research has shown that Indigenous people have worse health, higher mortality, lower literacy and numeracy, and higher rates of child abuse and adult imprisonment than non-Indigenous Australians.¹⁵

1.9 In 2006 the Council of Australian Governments (COAG) agreed to an intergovernmental approach to ‘closing the gap in outcomes between Indigenous Australians and other Australians’¹⁶, which led to the establishment of the National Indigenous Reform Agreement (Closing the Gap) in 2008. Through the agreement, the parties committed to working together with Indigenous Australians to close the gap in Indigenous disadvantage. It committed the Australian, state and territory governments to a detailed framework of Closing the Gap objectives, outcomes, outputs, performance measures and targets. The National Indigenous Reform Agreement contained six targets, one of which one of which was reframed in 2015 after expiring in 2013, and COAG committed to another target in 2014. The seven Closing the Gap targets are outlined in Table 1.1.

Table 1.1: Closing the Gap targets

Note a: The target was reframed in 2015 following the expiry of the previous remote early childhood education target.

Source: COAG Communiques and Closing the Gap Prime Minister’s Report 2020.

1.10 The gap in outcomes between Indigenous and non-Indigenous people in Australia has been reported in twelve successive Closing the Gap Prime Minister’s reports. The Closing the Gap Prime Minister’s Report 2020 reported that, while there has been progress, only two of the seven Closing the Gap targets are on track to be met — early childhood education and Year 12 attainment.¹⁷ The other five targets are not on track.

1.11 In 2019 the ANAO examined the Closing the Gap framework. The audit concluded that: arrangements for monitoring, evaluating and reporting progress towards Closing the Gap had been partially effective; and reporting on the high-level Closing the Gap targets had been maintained, but little work had been undertaken to monitor and evaluate the contribution of Australian Government programs towards achieving these targets.¹⁸

1.12 In late 2016 COAG announced a refresh of the Closing the Gap framework. A partnership agreement between COAG and the National Coalition of Aboriginal and Torres Strait Islander Peak Organisations came into effect in March 2019, creating a Joint Council on Closing the Gap. In July 2020 the Joint Council met to discuss the details of the draft National Agreement on Closing the Gap, which identifies priority reforms and includes new accountability measures. The National Agreement on Closing the Gap was released on 30 July 2020.

COVID-19 pandemic measures for Indigenous Advancement Strategy funded organisations

1.13 On 2 April 2020 the Minister for Indigenous Australians announced a number of targeted COVID-19 measures to support Aboriginal and Torres Strait Islander people, including $38 million to enhance the delivery of critical social support programs delivered under the IAS (including alcohol and other drug services, social and emotional wellbeing projects, family support and youth engagement, community safety and school nutrition).

1.14 Modified arrangements for grants administered under the IAS were also established temporarily. These included adopting a flexible approach to ensure that funded organisations are supported and remain viable during the COVID-19 pandemic. To this effect, on 30 March 2020 NIAA wrote to IAS funded organisations offering the opportunity to discuss available options. For organisations concerned about their ability to deliver services consistent with their grant agreements, the options included a review of their funding agreement.

Rationale for undertaking the audits

1.15 The IAS is one of the means through which the Australian Government has been trying to improve the lives of Indigenous Australians. Among the six IAS programs, the Children and Schooling and Safety and Wellbeing programs have the second and third largest administered budgets respectively. ANAO performance audits¹⁹, as well as Parliamentary inquiries²⁰ and departmental reviews, have shown that there have been shortcomings in the administration of the IAS. Auditor-General Report No.35 2016–17 Indigenous Advancement Strategy concluded that the ‘department’s grants administration processes fell short of the standard required to effectively manage a billion dollars of Commonwealth resources’. This report discusses NIAA’s progress in implementing relevant recommendations from Auditor-General Report No.35 2016–17 and provides assurance to Parliament and the public about the effectiveness of the administration of the IAS, focusing on the Children and Schooling and the Safety and Wellbeing programs.

Audit approach

Audit objective, criteria and scope

1.16 The ANAO conducted separate audits of the IAS Children and Schooling program and the IAS Safety and Wellbeing program, the findings and conclusions of which are presented in this report. The objective of the audits was to assess the effectiveness of PM&C’s and NIAA’s administration of the IAS Children and Schooling and the Safety and Wellbeing programs.

1.17 To form a conclusion against the objective of the audits, the ANAO adopted the following high-level criteria:

• Have the programs been designed and implemented to support the Government’s objectives?
• Are grant assessments consistent with the Commonwealth Grants Rules and Guidelines (CGRGs) and program guidelines?
• Is the management of grants consistent with the CGRGs and program guidelines?
• Does the performance framework support the effective administration of grants and enable ongoing assessment of progress towards outcomes?

Audit methodology

1.18 The audit methodology included:

• examination of PM&C and NIAA documentation about the design and implementation of the Children and Schooling and the Safety and Wellbeing programs;
• analysis of system data for 257 Children and Schooling and 237 Safety and Wellbeing grant applications submitted between 1 July 2016 to 30 June 2019, excluding applications that were withdrawn or under assessment as at 16 September 2019;
• analysis of documentation relating to a sample of 67 Children and Schooling and 65 Safety and Wellbeing grant applications submitted and assessed between 1 July 2016 to 30 June 2019;
• interviews with:
  • PM&C and NIAA personnel in National Office and in the Regional Network in Eastern New South Wales, Victoria/Tasmania, South Australia, Greater Western Australia, Kimberley, Central Australia, Top End and Tiwi Islands and Far North Queensland; and
  • 38 grant recipients in all states and territories around Australia;
• analysis of feedback from 132 recipients of grants under the Children and Schooling program, the Safety and Wellbeing program, or both programs.

1.19 The audit of the IAS Children and Schooling was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $495,000. The audit of the IAS Safety and Wellbeing was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $656,000.

1.20 The team members for these audits were Dr Isabelle Favre, Natalie Maras, Hugh Balgarnie, Clarina Harding, Elizabeth Robinson and Deborah Jackson.

2.1 The CGRGs establish the overarching Commonwealth grants policy framework and articulate expectations for all non-corporate Commonwealth entities in relation to grants administration. ‘Robust planning and design’ is one of the CGRG’s seven key principles for better practice grants administration that apply to all grant opportunities.²¹

2.2 To assess whether the Children and Schooling and the Safety and Wellbeing programs were designed and implemented to support the Australian Government’s objectives, the ANAO examined whether:

• the objectives of the programs align with government objectives;
• effective governance arrangements have been established;
• fit-for-purpose systems support staff to assess and manage grants; and
• the IAS Grant Guidelines are consistent with the CGRGs.

Are the objectives of the Children and Schooling and the Safety and Wellbeing programs aligned with government objectives?

2.3 The CGRGs state that:

The objective of grants administration is to promote proper use and management of public resources through collaboration with government and non-government stakeholders to achieve government policy outcomes.²²

2.4 The IAS is a key program for achieving the government’s three priorities to improve outcomes for Indigenous Australians:

• education;
• employment, economic participation and social participation; and
• a healthy and safe home and community.²³

2.5 Auditor-General Report No.35 2016–17 Indigenous Advancement Strategy noted that the Australian Government’s areas for priority investment in Indigenous affairs broadly aligned with the IAS programs.²⁴

Children and Schooling program

2.6 When established, the objectives of the Children and Schooling program, as outlined in the 2014 IAS Grant Guidelines, were:

To support families to give children a good start in life through improved early childhood development, care, education and school readiness; get children to school, improve literacy and numeracy, and support successful youth transition to further education and work.²⁵

2.7 These objectives are consistent with the government’s priorities and the objectives of the program outlined in the 2014–15 Department of the Prime Minister and Cabinet’s (PM&C’s) Portfolio Budget Statements (PBS). The 2014–15 PBS states that the objectives of the program are to: get children to school, improve education outcomes and support families to give children a good start in life. The objectives are described in the same way in PM&C’s 2015–19 corporate plan and have remained consistent in subsequent IAS guidelines and budget statements.

2.8 As discussed in paragraph 1.9, in 2006 the Australian Government agreed to an intergovernmental approach to closing the gap in Indigenous disadvantage. The agreement outlines seven building blocks to support closing the gap as measured by six (then seven) targets (see Table 1.1). The building blocks are: early childhood; schooling; health; economic participation; healthy homes; safe communities; and governance and leadership.

2.9 The objectives of the Children and Schooling program reflect the broader objective of the Closing the Gap framework and are aligned to two of the building blocks (early childhood and schooling). Five of the Closing the Gap targets are closely related to the Children and Schooling program outcomes and objectives. Figure 2.1 demonstrates this alignment, as articulated in the current guidelines and PBS.

Figure 2.1: Alignment of Children and Schooling program with government objectives

Note: When NIAA was established as an executive agency, Outcome 2 and its related programs transferred from PM&C to NIAA and are now presented as Outcome 1. There have been no other changes to the outcome or program structure since the publication of the 2019–20 PBS.

Source: ANAO analysis based on Closing the Gap targets, PM&C Portfolio Budget Statements 2019–20 and the IAS Grant Guidelines 2019.

2.10 The 2014 IAS Grant Guidelines stated that the strategy was designed to be broad in scope and flexible enough to support a wide range of activities. This has made it possible for NIAA to fund a broad range of activities under the Children and Schooling program, which align to the program objectives and the Closing the Gap targets.

Safety and Wellbeing program

2.11 When established, the objectives of the Safety and Wellbeing program, as outlined in the 2014 IAS Grant Guidelines, were:

To ensure the ordinary rule of law applies in Indigenous communities, and to ensure Indigenous people enjoy similar levels of physical, emotional and social wellbeing enjoyed by other Australians by fostering the ability of Indigenous Australians to engage in education, employment and other opportunities.²⁶

2.12 These objectives are consistent with the objectives of the program outlined in the 2014–15 PM&C’s PBS. The 2014–15 PBS states that the objectives of the program are to: ensure that the ordinary law of the land applies in Indigenous communities; and ensure Indigenous people enjoy similar levels of physical, emotional and social wellbeing enjoyed by other Australians. The objectives are described in the same way in PM&C’s 2015–19 corporate plan, and have remained consistent in subsequent IAS Guidelines and budget statements.

2.13 The objectives of the Safety and Wellbeing program reflect the broader objective of the Closing the Gap framework and are aligned to two of the building blocks referred to in paragraph 2.8 (healthy homes and safe communities). Two of the Closing the Gap targets are related to the program outcomes and objectives. Figure 2.2 demonstrates this alignment, as articulated in the current guidelines and PBS.

2.14 The Safety and Wellbeing program objectives also include elements that are broader than the Closing the Gap building blocks. For example, physical, emotional and social wellbeing (Safety and Wellbeing program) encompasses a broader range of issues than healthy homes and safe communities (Closing the Gap). This has resulted in the funding of a large range of activities.

Figure 2.2: Alignment of Safety and Wellbeing program with government objectives

Note: When NIAA was established, Outcome 2 and its related programs transferred from PM&C to NIAA and are now presented as Outcome 1. There have been no other changes to the outcome or program structure since the publication of the 2019–20 PBS.

Source: ANAO analysis based on Closing the Gap targets, PM&C PBS 2019–20 and the IAS Grant Guidelines 2019.

Recent changes to the Indigenous Advancement Strategy strategic direction: the Indigenous Policy and Investment Framework

2.15 In December 2018 PM&C commissioned a strategic review of the Children and Schooling and Safety and Wellbeing programs. The purpose of the review was to develop ‘policy architecture’ for the two programs that could guide future evidence-based program design and strategic decision-making. The review was completed in June 2019. It presents three theories of change for each program.²⁷ The theories of change provide a map of where investment decisions, including grant allocation, should be made to increase assurance that policy objectives of a program are met and expected results achieved.

2.16 The ANAO found that there is alignment between the proposed theories of change for each program and the 2019–20 PM&C PBS objectives and activity outcomes, and to the relevant Closing the Gap targets.

2.17 The theories of change were tested in a subset of the Children and Schooling program in July–August 2019. In November 2019, NIAA advised the ANAO that the review had been expanded to encompass all IAS programs, including the Children and Schooling in its entirety and the Safety and Wellbeing program, with the aim of developing an entity-wide organising framework: the Indigenous Policy and Investment Framework. This framework comprises 12 evidence-based theories of change aligned to the Closing the Gap draft outcome areas. In February 2020, NIAA executive endorsed the following key implementation steps:

• gaining support across Commonwealth agencies to use the framework to coordinate and guide policy and investment priorities across government;
• implementing the framework at the regional level, with an initial focus on IAS investment, with a view to develop a regional approach to investment that is guided by national priorities and shaped by the local context, community-identified needs, and local solutions; and
• developing a performance, monitoring and evaluation framework to enable NIAA and Commonwealth agencies to assess the impact of investments and inform decision-making around investment.

2.18 The Indigenous Policy and Investment Framework aims to realign policies, investments and evaluations to better deliver on the Government’s Closing the Gap goals across NIAA and across government. It is presented by NIAA as ‘a critical mechanism for NIAA to deliver on its mandated role to lead, coordinate and advance a whole-of-government approach for Indigenous Australians and deliver on the Commonwealth’s commitments under Closing the Gap.’²⁸ NIAA informed the ANAO that it expects to finalise the Indigenous Policy and Investment Framework in tandem with the refreshed Closing the Gap National Partnership Agreement and its implementation plans.

2.19 Five years after the start of the IAS, this foundational work has the potential to provide a stronger strategic direction to the programs’ investments. Using theories of change more clearly links activities to intended outcomes, which in turn better informs where investment should be directed.

Have appropriate governance arrangements been established?

2.20 Under the Public Governance, Performance and Accountability Act 2013 (PGPA Act) an accountable authority of a Commonwealth entity must govern the entity in a way that promotes the: proper use and management of public resources for which the authority is responsible; achievement of the purposes of the entity; and financial sustainability of the entity.²⁹ To ensure compliance with the PGPA Act, it is important for entities to establish effective governance arrangements. The Commonwealth Governance Structures Policy supports creating and maintaining fit-for-purpose governance structures for government activities.³⁰

2.21 The current IAS governance structure is illustrated in Figure 2.3. The structure, and the roles and responsibilities of NIAA’s bodies, are largely similar to PM&C’s governance arrangements when PM&C was responsible for the IAS.

Figure 2.3: NIAA internal governance structure

Note: Yellow shading indicates governance bodies with responsibility for IAS program performance.

Source: ANAO based on NIAA documents.

2.22 The Executive Board, which meets monthly, was ‘the primary executive decision-making body with a focus on strategic planning and policy, culture, organisational performance, resource allocation, risk management, staff management and workplace safety’.³¹

2.23 The Policy and Delivery Committee was created and met for the first time on 28 November 2019. Its role is helping to ‘drive and operationalise the strategic agenda of the NIAA through improved oversight of our policy, and implementation and delivery activities, ensuring they are aligned with government priorities.’³²

2.24 The Program Performance Committee’s purpose is to enable senior management oversight of the IAS. Until March 2020 the Program Performance Committee was referred to as the Program Management Board. Prior to the establishment of the Policy and Delivery Committee, the Program Management Board was a sub-committee of the Executive Board.

2.25 The Program Management Board’s role included ‘making decisions and providing advice to programme owners and the Executive on the strategic directions and implementation of Indigenous-specific programmes, in particular the Indigenous Advancement Strategy (IAS)’.³³ The Program Management Board’s responsibilities also included providing oversight regarding funding decisions; ensuring that adequate planning and resources are in place to manage the IAS program workload; driving continuous improvement and innovation; ensuring that program design, implementation and delivery is consistent with NIAA policies; and providing updates to the Executive Board. Program Management Board meetings were intended to be conducted every three weeks since September 2017 (every four weeks before this date).

2.26 To assess the effectiveness of the Executive Board’s and the Program Management Board’s oversight of the Children and Schooling and Safety and Wellbeing programs, the ANAO examined the meeting minutes for the two boards between January 2017 and December 2019.

2.27 The Executive Board met quorum and meeting frequencies for the majority of 2017 (with one exception in December). The Board did not meet in February, March, May, June and December 2018; or in January or April 2019. When it did meet, the Executive Board discussed funding, reform, risk and policy priorities, consistent with its terms of reference.

2.28 The Program Management Board also did not meet as regularly as scheduled. The Board discussed the Children and Schooling and Safety and Wellbeing programs at almost all meetings until August 2018. Between September 2018 and December 2019 the programs were mentioned in only three meeting minutes and were not specifically mentioned between April to December 2019. ANAO analysis indicates that while the programs were not specifically named in all minutes, the PMB did discuss strategic and operational IAS issues that could directly affect programs, including the development of the Policy and Investment Framework, the new IAS Grant Guidelines and the IAS Evaluation and Performance Framework.

Are systems to support staff to assess and manage grants fit-for-purpose?

2.29 Technology, processes and people are key components underpinning successful program administration. The ANAO examined the grant assessment and management systems used to manage the grant programs, as well as guidance and training available to staff.

Grant assessment and management systems

2.30 The two main information technology (IT) systems that NIAA uses to assess and manage IAS grants are the Grants Processing System (GPS)³⁴ and FUSION. GPS, the primary system used, is an online application that is administered by the Community Grants Hub.³⁵ GPS functionality includes:

• standard grants functionality to support block funding and some individualised funding;
• grants rounds and application assessment;
• review functionality for reporting; and
• grant agreement creation and management including financial commitment recording and payment processing.³⁶

2.31 There are three Community Grants Hub packages under which the Hub provides services to manage parts of the grant lifecycle. NIAA’s current service agreement is for a Basic Package, which gives NIAA access to the GPS Digital Platform, helpdesk support and limited reporting.³⁷

2.32 To complement GPS, NIAA uses FUSION to record Indigeneity, record activity coding, create and assess ceasing activities³⁸ and to generate reports. FUSION (previously known as the Indigenous Reporting Tool) was initially established to unify grant information from a number of grant management systems. One of the primary functions of FUSION is to provide end-to-end reporting of a single grant from initial proposal, application assessment, activity management, performance reporting and Ceasing Activity Assessments.

2.33 Figure 2.4 shows the systems used at different stages of the grant assessment and management process.

2.34 During the audit the ANAO identified a number of issues that resulted from NIAA’s use of two systems to assess and manage grants. While NIAA has recently worked to address these issues, they impacted on the administration of the Children and Schooling and the Safety and Wellbeing programs between 2016 and 2019. The issues identified included interface issues between GPS and FUSION that required manual intervention to resolve.

2.35 Under the Partnership Agreement between NIAA and the Department of Social Services, NIAA has access to a standard suite of reports generated on data held in GPS. In addition, since January 2019 NIAA has started to implement reporting solutions specific to its grant administration business requirements, accessing data stored in FUSION (including GPS data uploaded regularly into FUSION) and additional datasets as required. NIAA’s reporting solutions have allowed the development of automated weekly grant administration reports that are delivered weekly to NIAA executives since December 2019 and regional summary reports delivered to regional mailboxes since March 2020. Reporting to NIAA’s executive is further considered in paragraphs 5.29 to 5.31.

Guidance

2.36 NIAA has developed policies, procedures and resources for the IAS based on the PM&C grants management framework and the Commonwealth Grants Policy Framework. Where there is not a specific NIAA policy, the relevant PM&C policies apply until they are replaced by an agreed NIAA policy. The main source of guidance provided to staff to assist them to assess and manage grants is the online Grants Administration Manual. Other sources of guidance include: GPS systems guidance and intranet resources available through a SharePoint. The IAS Grant Guidelines are also a key policy document and are examined in paragraphs 2.45 to 2.51.

Grants Administration Manual

2.37 NIAA maintains a Grants Administration Manual to guide staff in the administration of grants. Available on NIAA’s intranet, it steps staff through the five stages of the grant management cycle (design, select, establish, manage, and evaluate and improve). Links are provided to relevant guidance and templates supporting each stage. Specific sections address the key areas of risk outlined in the CGRGs. The resources available from the Grants Administration Manual are updated periodically. Overall, the manual is clear, detailed and comprehensive.

2.38 In five of the eight regional network offices visited by the ANAO, NIAA staff indicated that the manual is useful but that it can be difficult to navigate the abundance of material it contains and identify whether the material available is current. In November 2019 NIAA commenced a project to redesign the Grants Administration Manual to improve its useability, content and maintenance. The project is scheduled to be completed in the second half of 2020.

Grants Processing System guidance and the Grants Systems Support and Reporting SharePoint

2.39 GPS task cards help users understand and perform different functions and include business rules and data conventions. The 120 task cards, managed and maintained by the Community Grants Hub under the standard service offer, are in various stages of revision, with some (for example, task cards for the ‘Select’ stage) yet to be updated to reflect current processes in GPS.

2.40 NIAA has also implemented a Grants Systems Support and Reporting web-based collaborative platform (SharePoint) that is designed to achieve consistency in assessment and management and help new users navigate GPS and perform different functions. The Grants Systems Support and Reporting SharePoint has three main parts:

• Grants Systems Support — helps set up and change user profiles and escalate systems issues to the Community Grants Hub;
• Grants System Training — contains task cards, links to e-learning modules and access to instructor-led training (such as ‘bite-sized’ training and webinars); and
• Grants Reporting — provides access to a team assisting with development of custom data reports.

2.41 The platform is an effective way of providing users with technical support contacts and links to detailed operational guidance. As the same information is available in the Grants Administration Manual and the Sharepoint, there may be merit in linking the systems (so that each system refers to the other where relevant).

Training

2.42 Staff training is fundamental to an entity-wide culture of quality and consistency in the assessment and management of grants. Auditor-General Report No.35 2016–17 Indigenous Advancement Strategy reported that PM&C was unable to provide a training register or other documentation recording which staff had attended training. In 2017, following the Auditor-General’s report, Joint Committee of Public Accounts and Audit Report 464: Commonwealth Grants Administration recommended that PM&C measures the number of staff formally trained with regard to the CGRGs and quickly rectify skills gaps so as to ensure compliance with CGRGs.³⁹ In response to the Joint Committee of Public Accounts and Audit, PM&C advised that it had developed a ‘comprehensive staff training package aligned with the Grant Administration Manual’, including elements of core training mandatory for staff working in grants administration, and that improved records were being maintained of staff attendance at training sessions.⁴⁰

2.43 Between August and September 2019 NIAA introduced a mandatory cloud-based Learning Management System, which can measure the number of staff completing training. Through the Learning Management System, NIAA piloted grants administration training consisting of five interactive modules explaining obligations under the PGPA Act and the CGRGs. The training also outlined the key steps of the grant lifecycle, including key grants administration processes and responsibilities. In April 2020 NIAA advised that of the 963 staff included in the initial pilot of the training, 86 per cent completed the training.⁴¹

2.44 NIAA also offers face-to-face and video conference training, provided annually or as needed, for example where staff in a region routinely omit information from one of the key fields in GPS. Training includes acquittals, organisation and activity risk management, grants advice, Working with Vulnerable People guidance, performance reporting, fraud awareness and the centralised briefing process. A helpdesk providing support with grant systems is also available to NIAA staff.

Are the IAS Grant Guidelines consistent with the Commonwealth Grants Rules and Guidelines?

2.45 The CGRGs emphasise the importance of promoting open, transparent and equitable access to grants, including through ensuring that the rules of grant opportunity guidelines are clear in their intent.⁴² The CGRGs note that the rules of grant opportunities should be simply expressed, clear in their intent and effectively communicated to stakeholders.⁴³

2.46 The IAS Grant Guidelines were published in 2014, revised in 2016 and updated in August 2019 to reflect the transfer from PM&C to NIAA. They include a description of program objectives, eligibility and assessment criteria, assessment process, grant agreement requirements and monitoring and evaluation approaches. The guidelines contain a comprehensive description of the Children and Schooling and Safety and Wellbeing programs, including activities that are in-scope and out-of-scope for funding. They also present the three approaches that NIAA uses to allocate funding under the IAS (summarised in Table 2.1).

Table 2.1: Funding approaches in the IAS Grant Guidelines

Source: ANAO analysis, based on the IAS Grant Guidelines 2019.

2.47 As required by the CGRGs, the guidelines are available on NIAA’s website (previously on PM&C website) and are supplemented, when relevant, by an application kit. In accordance with the CGRGs, grant opportunities have been listed on GrantConnect since 2017, with links to the IAS Grant Guidelines and the responsible entity’s website.⁴⁴ The information provided on GrantConnect is consistent with the information provided on the website(s).

2.48 The ANAO’s analysis found that the IAS Grant Guidelines, in conjunction with the application kits, include most of the information that the Department of Finance recommends should be present in grant guidelines⁴⁵ and, to that extent, are compliant with the CGRGs. The CGRGs do not require entities to publish the amount of funding available. However guidance from the Department of Finance advises that grant guidelines should outline the total funding available over a period of time, how much funding is available for each grant and whether there are limitations on the amount that can be applied for.⁴⁶

2.49 The IAS Grant Guidelines only indicate the overall IAS budget ($5.2 billion over four years to 2022–23), which does not provide sufficient information on the amount of funding available for a specific program, or whether there are limitations on the amount for which applicants can apply.⁴⁷ As shown in Table 2.2, there is a material difference between the budget allocated to the IAS; the budget allocated to the Children and Schooling and the Safety and Wellbeing programs; and the uncommitted budget available for each program at any point in time.

Table 2.2: Children and Schooling and Safety and Wellbeing programs’ budgets

Note a: IAS Grant Guidelines 2019, p. 6.

Note b: PM&C 2019–20 Portfolio Budget Statements. The budgets for the Children and Schooling and Safety and Wellbeing programs are for administered expenses.

Note c: The uncommitted budget is the amount of funding available once legal commitments and planned allocations are subtracted.

Source: ANAO, based on NIAA documents.

2.50 Auditor-General Report No.35 2016–17 Indigenous Advancement Strategy also found that the IAS Grant Guidelines and the grant application process could have been improved by identifying the amount of funding available under the grant funding round.⁴⁸

2.51 The ANAO has made a recommendation that, to assist applicants to appropriately scope activities relative to funding available and to increase transparency in relation to the limitations on the amounts that can be applied for, NIAA publicly communicate up-to-date information about funding available under the Children and Schooling and Safety and Wellbeing programs (Recommendation no. 1).

New IAS guidelines

2.52 NIAA advised that new guidelines were being developed, with an expected publication date of December 2020. Instead of one overarching set of IAS guidelines, two sets of ‘agency collaborates’ guidelines would be developed:

• ‘agency collaborates’ guidelines — non-competitive — to fund a proposal that has been developed with an eligible applicant (current community led approach) and to directly approach an organisation, where an approach to market cannot be undertaken;
• ‘agency collaborates’ guidelines — competitive — to approach a group of potential grantees to select the best provider, for example where a critical service gap exists but the most suitable provider needs to be identified.⁴⁹

2.53 In addition, dedicated program-specific guidelines for each IAS grant opportunity would be issued.

2.54 Drafts reviewed by the ANAO indicate that the two sets of ‘agency collaborates’ guidelines include only the total funding envelope ($5.2 billion over four years to 2022–23). NIAA advised that guidelines developed for dedicated grant opportunities would provide increased transparency regarding funding available, by publishing the amount of uncommitted (available) funds for each opportunity.⁵⁰

3.1 The concept of transparency in grants administration underpins the CGRGs. Transparency: provides assurance that grants administration is appropriate and that legislative obligations and policy commitments are met; minimises concerns about equitable treatment; and provides assurance that relevant money has been spent for the approved purposes and is achieving the best possible outcomes.⁵¹

3.2 The ANAO analysed whether:

• selection processes are applied in a manner that demonstrates the achievement of value for money outcomes;
• grant assessments are compliant with the CGRGs and the IAS Grant Guidelines;
• grant assessments consider regional strategies, and potential gaps and duplications; and
• advice provided to the minister complied with the requirements of the CGRGs.

Are selection processes applied in a manner that demonstrates the achievement of value for money outcomes?

3.3 The CGRGs allow for a range of approaches for grant selection, including competitive and non-competitive rounds, demand-driven processes and one-off grants to be determined on an ad hoc basis. The CGRGs specify that using a non-competitive approach to allocate grant funding may be appropriate, in particular when the number of service providers is very limited and these providers have a well-established record of delivering the grant activities. However, the CGRGs also state that competitive, merit-based processes can achieve better outcomes and value with relevant money and should be used to allocate grants (unless otherwise agreed).⁵²

3.4 As summarised in Table 2.1, NIAA uses different approaches to allocate funding under the IAS:

• agency invites applications;
• agency responds to community led proposals; and
• agency approaches organisation (also called direct approach); under the direct approach, the IAS Grant Guidelines state that NIAA may also use a simplified assessment process.

3.5 Figure 3.1 presents NIAA’s grant assessement processes under the three approaches.

Figure 3.1: Grant assessment processes, July 2016 to June 2019

Note: Before June 2019 the minister was the only delegate except for direct approaches under the Regional Managers Discretionary Fund, which allows Regional Managers to fund small scale, short-term, one-off projects.

Source: Based on NIAA documents.

3.6 As illustrated in Figure 3.1, assessments to inform a funding decision follow a similar process under the ‘agency invites applications’ and ‘agency responds to community led proposals’ approaches. Assessment processes differ when the direct approach is used.

3.7 As the direct approach is not competitive and does not assess the relative merits of the applicant, NIAA internal guidance requires that approval to use the approach is obtained from the delegate (the minister) before approaching the organisation. This is in accordance with the CGRGs, which state that where a method, other than a competitive merit-based selection process is to be used, this should be specifically agreed to by a minister or delegate.⁵³ NIAA policy and internal guidance specify that the simplified process is to be used sparingly and as an exception:

[A simplified direct approach] should be used as an exception, as the standard approach in accordance with the CGRGs is to undertake a competitive approach to market…. You must provide a strong business justification to explain why you are not using a competitive approach.⁵⁴

3.8 In the ministerial briefs seeking approval for the funding approach, the standard justifications PM&C used to support its recommendation for a non-competitive direct approach included that the provider: has extensive experience in effectively delivering activities; has established relationships with key stakeholders; has expressed interest in continuing service provision; does not have a high risk rating; and is the only known provider able to deliver the activities at the time.

3.9 Table 3.1 demonstrates that, between July 2016 and June 2019, PM&C used the non-competitive ‘agency approaches organisation’ direct approach to allocate 90 per cent of the Children and Schooling program funding and 95 per cent of the Safety and Wellbeing program funding.

Table 3.1: Funding distribution by funding approach, July 2016 to June 2019

Source: ANAO analysis of NIAA data. Some totals do not add up due to rounding.

3.10 As illustrated in Figure 3.1, NIAA also uses a simplified direct approach under the direct approach. NIAA guidance indicates that the simplified direct approach can be used in two circumstances:

• to address urgent needs, for instance to prevent an unexpected service delivery gap; and
• to assess whether to continue or cease activities already funded. These activities are described by NIAA as ‘ceasing activities’.

3.11 Ceasing activities are activities that are due to cease within the next six months and that NIAA assesses in order to decide whether to continue funding the activity beyond the current agreement’s expiry date. In most cases NIAA does not require an application as the organisations that are approached are receiving or have previously received IAS funding to deliver the same or similar activities. NIAA uses the information and knowledge it holds about the organisations’ experience and performance to conduct the assessment. The IAS Grant Guidelines also state that using the simplified assessment process minimises administrative workload on providers. This approach is consistent with the CGRGs.

3.12 Table 3.2 shows that, between 1 July 2016 and 30 June 2019, ceasing activities accounted for 84 per cent of all funding allocated under the Children and Schooling program, and 90 per cent of all funding allocated under the Safety and Wellbeing program.

Table 3.2: Funding allocated through the ‘agency approaches organisation’ approach, July 2016 to June 2019

Note a: The data for funding allocated through the direct approach includes a number of grants allocated using a simplified approach.

Source: ANAO analysis of NIAA data. Some totals do not add up due to rounding.

3.13 The ceasing activities assessment process can result in a range of outcomes:

• activity ceases: due to the activity being completed without expectation of further funding or capacity to continue (for instance, construction of a building or attendance to a conference); or due to changing community needs or poor performance, governance issues or other risk factors related to the provider;
• activity continues with the same provider: delivering the same activity without substantial change (under the same agreement and schedule); or with some changes to the nature or scope of the activity or key performance indicators (under the same agreement with a new schedule); and
• activities continue with a new provider: the same activity is transitioned to a new provider either as soon as the existing provider’s agreement is completed or over a specific period.

3.14 Between 1 July 2016 and 30 June 2019, the ceasing activities assessment process resulted in the activities being continued for 80 per cent of Children and Schooling and 87 per cent of Safety and Wellbeing funding (see Table 3.3). This means that the majority of program funding was reallocated to the same providers delivering mostly the same activities.

Table 3.3: Ceasing activities funding outcomes

Source: ANAO analysis, based on NIAA data. Some totals do not add up due to rounding.

3.15 As mentioned at paragraph 3.3, competitive, merit-based selection processes can achieve better outcomes and value with relevant money.⁵⁵ NIAA allocates the majority of the funding for the Children and Schooling and Safety and Wellbeing programs using a non-competitive approach. This is not consistent with the principles underlying the CGRGS and with NIAA’s policy and guidance (outlined at paragraph 3.7). The extensive use of a non-competitive approach, combined with the allocation of the majority of the programs’ funding to the same providers, restricts the opportunity for new providers to compete for funding and limits NIAA’s ability to demonstrate that value with relevant money is achieved.

Are assessments compliant with the Commonwealth Grants Rules and Guidelines and the IAS Grant Guidelines?

3.21 The CGRGs state that officials should assess grants in a manner that is consistent, transparent and accountable and minimises concerns about equitable treatment.⁵⁶ The CGRGs state that processes applied to grant assessments should follow the rules presented in the guidelines. The CGRGs also define eligibility criteria as the mandatory criteria that must be met to qualify for a grant; and assessment criteria as the specified principles or standards against which applications are judged.⁵⁷

Assessment requirements

3.22 Under the IAS Grant Guidelines, grant applications must meet certain eligibility criteria. Applicants must : be incorporated; not be bankrupt or subject to insolvency proceedings; have an Australian Business Number and be registered for GST purposes; be financially viable for the purpose of the grant, as assessed by NIAA; and not have been named as non-compliant under the Workplace Gender Equality Act 2012. Applicants are required to provide evidence to support their eligibility when applying for a grant.

3.23 The ANAO found that, in a sample of 63 Children and Schooling and 46 Safety and Wellbeing grant applications submitted between 1 July 2016 and 30 June 2019, NIAA assessed eligibility in all Children and Schooling applications and 98 per cent of Safety and Wellbeing applications.

3.24 Once an application has been assessed as eligible, it should be assessed against the assessment criteria. The application should then be endorsed (or otherwise) by the regional manager and recommended (or otherwise) by the program owner. An organisational risk assessment should be conducted concurrently with the assessment process. The ANAO analysed a sample of 63 Children and Schooling and 46 Safety and Wellbeing grant applications submitted between 1 July 2016 and 30 June 2019, and determined that the assessment process as described in the IAS Grant Guidelines and in NIAA documentation was largely followed (Table 3.4).⁵⁸

Table 3.4: Compliance with requirements when undertaking assessments

Note a: This test does not apply to the simplified direct approach (see Figure 3.1). As a result, n = 35 Children and Schooling applications and n = 34 Safety and Wellbeing applications.

Source: ANAO analysis of NIAA data.

Assessment criteria

3.25 The CGRGs state that assessment criteria are ‘the specified principles or standards against which applications will be judged. These criteria are also used to assess the merits of proposals and, in the case of a competitive grant opportunity, to determine application rankings’. The CGRGs further state that assessment criteria should be clear to enable to selection of applications in a consistent, transparent and accountable manner.⁵⁹

3.26 The IAS Grant Guidelines specify that assessment criteria are used to answer two questions:

• Will a proposed activity lead to improved outcomes within the target community or group that would not occur without the grant?
• Do the intended outcomes represent value for money, that is do the intended outcomes justify the Government providing the requested amount of grant funding?⁶⁰

3.27 The guidelines then state that applications for grant funding may be assessed against the following four assessment criteria⁶¹:

• Need — the activity is needed to provide improved outcomes and there is a demand for the activity from the target Indigenous community or group;
• Quality — the organisation that will deliver the proposed activity is committed to and capable of working with the target Indigenous community or group;
• Efficiency — the proposed activity will support the intended outcomes in a way that appropriately manages risk, is cost effective and is coordinated with relevant stakeholders in the target community; and
• Effectiveness — the proposed activity can deliver the intended outcomes and sustain the outcomes into the future.⁶²

3.28 NIAA rates applications against the four assessment criteria. Each criterion is scored on a scale of one to seven (maximum score of 28). As shown in paragraph 3.9, the majority of funding is allocated using non-competitive approaches. This means that the merits of applications are not rated against each other, but against the assessment criteria.

3.29 The ANAO examined the correlation between assessment scores and funding outcomes for the 257 Children and Schooling and 237 Safety and Wellbeing applications received between 1 July 2016 and 30 June 2019 (Figure 3.2 and Figure 3.3).⁶³

Figure 3.2: Children and Schooling application outcomes relative to assessment score, 1 July 2016 to 30 June 2019

Source: ANAO analysis of NIAA data.

Figure 3.3: Safety and Wellbeing application outcomes relative to assessment score, 1 July 2016 to 30 June 2019

Source: ANAO analysis of NIAA data.

3.30 Figure 3.2 and Figure 3.3 show that in a number of instances high-scoring applications are not funded and low-scoring applications are funded:

• a score of 16 or below is rated by NIAA as ‘poor’:
  • of the 151 Children and Schooling funded applications, 48 (32 per cent) received a score of 16 or less and received funding of $16.6 million in total;
  • of the 164 Safety and Wellbeing funded applications, 20 (12 per cent) received a score of 16 or less and received funding of $13.8 million in total; and
• a score of 24 or above is rated by NIAA as ‘exceed expectations’ and ‘significantly exceed expectations’:
  • of the 106 Children and Schooling applications not funded, 26 (25 per cent) received a score of 24 or above;
  • of the 73 Safety and Wellbeing applications not funded, seven (10 per cent) received a score of 24 or above.

3.31 A number of factors explain the differences between individual assessment scores and funding outcomes:

• NIAA guidance, which indicates that the scoring is done by assessors only. Regional managers and program owners are expected to provide their recommendations and endorsement, including against the four published criteria, but are not able to change the score determined by the assessors;
• the IAS Grant Guidelines, which note that NIAA may use a number of information sources when assessing and verifying claims in applications, including information that it or other Commonwealth entities hold about the applicant; and that decisions will be subject to funding availability⁶⁴; and
• NIAA’s advice that, while some applications may rate poorly, there are situations where no other provider is available to deliver the same service and not funding that activity would leave a gap in service delivery. Conversely, some highly rated applications may not be funded if the program budget was already allocated for the period.

3.32 When seeking funding approval from the minister, NIAA submits briefs that include the assessment score, details of whether and how the application meets the IAS assessment criteria, strengths and weaknesses of the proposal, value for money considerations and other significant details. However, the factors described in paragraph 3.31 are not reflected in the scores that are included in the brief to the minister. As a result, the link between scores, whether the proposal meets the assessment criteria and whether the proposal should be funded is not always clear (Table 3.5 provides some examples).

Table 3.5: Examples of inconsistency between assessment scores, whether the proposals met the criteria and funding recommendations in ministerial briefings

Source: ANAO analysis, based on ministerial briefings.

3.33 Documenting how additional factors considered during the assessment process impact on the scoring against the four criteria, when applied, would improve NIAA’s ability to demonstrate to the decision-maker that its recommendations are consistent, transparent and demonstrate value for money.

3.34 Auditor-General Report No.35 2016–17 Indigenous Advancement Strategy also found that some grants that were awarded a high score against the selection criteria and need were not recommended for funding and some low-scoring applicants were recommended. The report recommended that PM&C publishes adequate documentation that clearly outlines the assessment process and PM&C’s priorities and decision-making criteria; and consistently implements the process. In its response, PM&C stated that the 2016 revised IAS Grant Guidelines ‘clearly describe[d] the application assessment process and the criteria that an application [is] assessed against’.⁶⁵

Do grant assessments consider regional strategies and potential gaps and duplications?

3.35 To ensure that grant assessments consider regional strategies and potential gaps and duplications, NIAA uses its Regional Network officers, primarily supported by two mechanisms: regional blueprints and the Investment Analysis Tool.

Alignment with Government and regional strategies

3.36 NIAA describes regional blueprints as high-level internal planning documents providing a snapshot of key opportunities in a region to maximise economic and social benefits for Indigenous Australians, the priority areas for action and what will be accomplished.

3.37 A regional blueprint has been developed by each of the 12 Regional Network regions. Of the 12 blueprints the ANAO examined in February 2020, four were in draft form since December 2018 and three had not been updated since January 2018. In July 2020 NIAA advised that updates currently occur as needed. NIAA also advised that it is revising its approach to updating blueprints. In future, blueprints would be reviewed annually and would evolve in line with NIAA’s broader Policy and Investment Framework. Ensuring that the regional blueprints are finalised and remain up-to-date would more effectively support the alignment of policy and activities to regional priorities.

3.38 Since 2017 an Investment Analysis Tool has been available to guide decision-making about funding for ceasing grant assessments. The tool is mandatory and directs Regional Network assessors to analyse funded activities across five domains:

• Strategic alignment: assesses whether the currently funded activity aligns with regional or IAS programme priorities, or whether it should be supported by other Commonwealth or state and territory government agencies.
• Activity design: considers the logic of funded activities, both internally and within context.
• Outcomes focused management: looks at how well the organisation is set up to administer the funds provided to implement the activity effectively, with a view to outcomes and long-term impact.
• Cultural strength: looks at the degree to which funded activities demonstrate a strong capacity to working with local Aboriginal and Torres Strait Islander people to create sustainable change at the grassroots level.
• Evaluation: considers the quality of the evaluation processes the organisation has planned or in train.⁶⁶

3.39 Box 1 describes the domain related to strategic alignment of the activity with regional and IAS program priorities.

3.40 The ANAO reviewed 20 Children and Schooling and 20 Safety and Wellbeing assessments in which the tool was used. In 18 and 19 assessments respectively, NIAA had addressed regional priorities with reference to regional blueprints.

3.41 The ANAO’s review of the tool indicates that it provides a practical framework to analyse, in a consistent and systematic manner, the potential of an activity to achieve the program objectives and gain assurance that funding decisions are aligned to government, regional and program priorities. The tool is used for ceasing grant assessments. There would be benefit in NIAA extending the use of the tool to all grant assessments.

Gaps and duplications

3.42 The CGRGs recognise that grants can be funded by Commonwealth, state or territory, and local government bodies, private trusts and foundations or national or state coordinating organisations. To help reduce fragmentation and unnecessary overlaps, the CGRGs encourage officials to work collaboratively with government and non-government stakeholders as well as consider what impact a particular grant opportunity may have on other government or non-government funded activities.

3.43 NIAA applies a range of approaches aimed at ensuring that gaps and duplications in service delivery are identified. This includes the Investment Analysis Tool discussed above and the requirement for grant applicants to demonstrate that there is a need for the service they propose to deliver. This requirement is one of the assessment criteria in the IAS guidelines, and is separately examined in the assessment process. The assessment criterion is:

The activity is needed to provide improved outcomes and there is a demand for the activity from the target Indigenous community or group.

a) There is evidence the proposed activity is needed and will support improved outcomes in the target Indigenous community or group.

b) The target community or group supports the proposed activity and has been involved in its design.⁶⁷

3.44 At an organisational level, one of the Regional Network’s primary purposes is to support active engagement with communities and for intelligence gathered by the Network to be fed to centralised policy areas. As part of the assessment process, regional managers are expected to endorse proposed grant applications based on a consideration of community and regional needs and priorities, alignment of the applications to the regional strategy and duplication of services. Auditor-General Report No.7 2018–19 Management of the Regional Network identified that each region was engaged with a large range of stakeholders, including Australian and state government departments, land councils, peak bodies, community groups and service providers. In addition the Network has a permanent in-community presence in seven regions. The level of engagement of Regional Network staff with local communities provides some assurance that needs for services are identified, and gaps and duplications are avoided.

Did advice to the minister comply with the requirements of the Commonwealth Grants Rules and Guidelines?

3.45 The CGRGs state that officials must provide written advice on the merits of the proposed grant or group of grants to the minister, where ministers exercise the role of an approver, and that this advice must be received before the minister approves the grant. The CGRGs also outline what must be included in the advice.⁶⁸

3.46 In the sample of 62 Children and Schooling and 42 Safety and Wellbeing applications, the ANAO found that in all cases PM&C had provided written advice to the minister before grants were approved. The ANAO assessed whether the advice covered the terms, value, timeframe and risk of the proposed grant; and whether a rationale was provided in support of all funding recommendations. In all cases the advice provided to the minister appropriately documented the basis for the recommendation and was sufficient to enable the minister to make a decision. However, as noted in paragraph 3.32, there is a lack of consistency between: scores; whether the proposal meets the assessment criteria; and whether the proposal should be funded.

3.47 The ANAO’s analysis, shown in Table 3.6, that there is alignment between the recommendation and the minister’s decision.

Table 3.6: Alignment of recommendation and decision

Note a: n = 62 Children and Schooling applications and n = 42 Safety and Wellbeing applications.

Note b: The relevant applications for this test were funded applications (34 Children and Schooling applications and 22 Safety and Wellbeing applications).

Source: ANAO analysis of NIAA data.

3.48 Under the CGRGs, the minister must report annually to the Minister for Finance instances where they have decided to approve a particular grant that the relevant officials had recommended be rejected.⁶⁹ Where the minister approved funding for an amount that differed to the amount recommended, the minister noted approval for a higher or lower amount on the ministerial brief.⁷⁰ One Children and Schooling grant and one Safety and Wellbeing grant was approved that the department did not recommend for funding. The minister reported these instances to the Minister for Finance as required. Outside the sample, the ANAO identified one grant approved contrary to the department’s recommendation, and that had not been reported to the Minister for Finance. NIAA advised the ANAO that this was due to an administrative error. This error, subject to advice from the Department of Finance, was noted in the 2019 annual report to the Minister for Finance.

Timeliness

3.49 The CGRGs indicate that timely appraisal of grants avoids possible inequities and waste that may arise through unnecessary delays.⁷¹ The ANAO reviewed the timeliness of the grants assessment process, including the provision of advice to the minister.

3.50 Prior to February 2019 NIAA aimed to advise applicants of the outcome of their application within three months of its receipt. In February 2019 NIAA set the following service standards for processing applications:

• 90 per cent of applications assessed and advice provided to minister within 45 days; and
• 100 per cent of applications assessed, advice provided to minister and applicants notified within 90 days.

3.51 Table 3.7 shows that a considerable percentage of applications (37 per cent of Children and Schooling and 25 per cent Safety and Wellbeing applications) took over 180 days from the date they were submitted to the date the applicant was notified of the outcome.

Table 3.7: Application processing times, July 2016 to June 2019

Note a: NIAA was not able to provide data for five Children and Schooling and three Safety and Wellbeing applications, which were removed from the sample. As a result, n = 58 Children and Schooling applications and n = 37 Safety and Wellbeing applications.

Note b: NIAA was not able to provide data for four Children and Schooling and six Safety and Wellbeing applications, which were removed from the sample. As a result, n = 63 Children and Schooling applications and n = 59 Safety and Wellbeing applications.

Source: ANAO analysis of NIAA data.

3.52 NIAA has improved its timeliness since 2016–17 (Table 3.8), with 81 per cent of Children and Schooling and 100 per cent of Safety and Wellbeing applications assessed within 90 days in 2019–20.

Table 3.8: Applications processed within 90 days by financial year, July 2016 to May 2020

Note: Data to 25 May 2020. During the period, 293 applications were submitted under the Children and Schooling program and 267 applications were submitted under the Safety and Wellbeing program. Applications submitted during the caretaker period associated with the Federal Election of 18 May 2019 were excluded from the analysis (9 Children and Schooling and 17 Safety and Wellbeing applications).

Source: ANAO analysis of NIAA data.

4.1 In relation to the management of grant activities, the CGRGs establish the importance of a well-drafted and fit-for-purpose grant agreement. This helps establish a shared understanding of objectives and expectations against which to monitor performance. Once an agreement has been established, ongoing monitoring and management arrangements throughout the grants lifecycle should assure entities that grant opportunities are proceeding as planned and that relevant money is being appropriately managed.⁷² To promote proper use of and management of public resources, entities should also have in place mechanisms to address situations where the purpose of the grant is not being achieved.

4.2 In October 2019 NIAA finalised the IAS Grant Performance Framework.⁷³ The framework is a principles-based document that is not specific to individual programs. The processes to monitor grant performance outlined in the framework include grant agreements; grant deliverables and performance measures; and agreement manager monitoring of grants.

4.3 The ANAO examined whether, for the Children and Schooling and Safety and Wellbeing programs:

• grant agreements are appropriate and risk-based;
• grants are monitored for progress against grant objectives; and
• mechanisms are in place to address situations where the purpose of the grant is not being fulfilled.

Are grant agreements appropriate and risk-based?

4.4 The CGRGs state that grant agreements should be well-drafted and fit-for-purpose, as this will contribute to good governance and accountability and that a well-designed grant agreement will help establish the basis for effective working relationships based on collaboration and respect between the grantee and the entity. Agreements should apply the principle of proportionality, whereby an appropriate balance is struck between complexity, risks, outcomes and transparency. Agreements should also clearly document the expectations of all parties in relation to the grant.

Appropriate

4.5 The CGRGs are supported by a number of initiatives, including whole-of-government tools and templates to standardise grants administration processes.⁷⁴ In December 2018 the Department of Finance released a suite of whole-of-government grant agreement templates for use by entities when they enter into grants. The templates include simple and standard grant agreements, a letter and deed of variation, and a notice of change.

4.6 NIAA’s intranet does not reference the Department of Finance templates and user guides. NIAA’s head agreement (July 2019), which is published on NIAA’s website, is based on a 2015 Commonwealth template, not the current whole-of-government head agreement template.

4.7 The ANAO also notes that the terminology NIAA uses in key IAS documents and information sources is not consistent. For example, a grant agreement is referred to as a ‘project agreement’ in the IAS guidelines and an ‘IAS funding agreement’ and ‘head agreement’ on the website.The CGRGs state that it is important that officials develop clear, consistent and well-documented grant opportunity guidelines and other related documentation. Using inconsistent terminology does not accord with better practice principles outlined in the CGRGs.

4.8 The ANAO assessed a sample of 14 Children and Schooling and 14 Safety and Wellbeing grant agreements executed between July 2016 and June 2019 against key characteristics that demonstrate better practice outlined in the CGRGs and the Grants Administration Manual and found:

• The project description included in 12 Children and Schooling and 11 Safety and Wellbeing agreements was prescriptive, clearly and succinctly detailing what, how, when and where the project should be delivered. These elements allow NIAA to assess grantee performance and progress against agreed objectives.
• In seven Children and Schooling and four Safety and Wellbeing agreements the level of monitoring applied to the projects (including frequency and nature of performance and financial reports) was not commensurate with the risk presented by the organisation and the project.⁷⁵ The reporting requirements included in some agreements indicate that the level of monitoring would be more than necessary given the grantee’s risk rating as assessed by NIAA and the value of the grant. The converse is true for other agreements with higher risk ratings and value. This indicates that not all agreements apply the CGRG principle of proportionality.
• KPIs in 10 Children and Schooling and seven Safety and Wellbeing agreements were not aligned to the project’s objective and purpose; did not always have targets or benchmarks; did not measure outputs and deliverables; or did not measure outcomes. Four out of the ten Children and Schooling agreements and three of the seven Safety and Wellbeing agreements had been executed after the KPIs were redesigned and used the redesigned KPIs (the redesigned KPIs are discussed below).

4.9 Auditor-General Report No.35 2016–17 Indigenous Advancement Strategy examined the performance information included in IAS grant recipients’ funding agreements, and concluded that the performance indicators were not sufficiently outcome-based. As a result, PM&C was not able to link performance reporting at the project level to the achievement of outcomes. PM&C agreed to a recommendation to identify the outcomes and results to be achieved through the IAS and analyse performance information to measure progress against these outcomes.⁷⁶

4.10 In 2017–18 PM&C redesigned the IAS grant performance indicators, aiming to develop a suite of revised KPIs, including measures that would be clearly linked to grant recipients’ activity deliverables, and identify data sources and data collection methodologies.⁷⁷ As a result of this work, from early 2018:

• a revised set of 72 performance measures became available for Children and Schooling activities, which included 40 output quantity measures, 18 output quality measures and 14 outcome measures. These performance measures were grouped into 14 categories and 6 sub-programs.
• a revised set of 151 performance measures became available for Safety and Wellbeing activities, which included 78 output quantity measures, 40 output quality measures and 33 outcome measures. These performance measures were grouped into 24 categories and 8 sub-programs.

4.11 Agreement managers are expected to apply the relevant measures from each category to all new grant agreements. In addition, each activity agreement must include five mandatory performance measures, which relate to core service provision and the employment of Indigenous Australians. NIAA advised that the new KPIs are also to be incorporated whenever a funding agreement is reviewed and renegotiated. A suite of templates and manuals were developed and distributed to staff and providers to guide the selection of performance measures for each activity, collection of data and assessment of performance against the measures. The ANAO analysed the redesign and the revised KPIs and found that:

• PM&C sourced external advice when redesigning the KPIs;
• the new performance measures are defined and measurable, as they identify a method and source of data collection, a benchmark or target (when relevant) and the reporting period; and
• the measures are specific and focused, which means that the results are clearly attributable to the activities.

4.12 The KPIs were redesigned before the Policy and Investment Framework, including theories of change, were developed for the Safety and Wellbeing program (the Policy and Investment framework is discussed at paragraphs 2.15 to 2.18). NIAA has acknowledged that once the Policy and Investment Framework is finalised, the KPIs would likely need to be reviewed.

Risk-based

4.13 Prior to 2019 PM&C’s risk management framework was focused on the development of a provider risk profile that considered the provider’s governance, financial management systems, service delivery and intelligence collected by PM&C.⁷⁸ The risks associated with activities were not assessed. For high and very high risk providers, a risk plan had to be developed explaining the controls to be applied. Potential mitigation strategies included more frequent communication with and reporting by the provider. Risk plans were optional for low/minor and moderate risk providers.

4.14 From January 2019 a new risk management framework — the Grant Risk Management Guidelines — is being implemented as new agreements are executed or existing agreements are varied. Under the framework, an activity risk assessment is conducted that takes into account: the organisation’s risk rating; the annualised value of the grant (grants with an annual value greater than $1 million are defined as presenting a major or severe risk); and the nature of the activity or potential impact on beneficiaries. Based on the assessed activity risk, a set of standard controls are included in the funding agreement for each activity.

4.15 For low risk grants, the agreement is in the form of a letter of offer and requires minimal performance and financial controls (for example, a financial declaration and acquittal). For medium, high and extreme risk levels, a standard grant agreement is established, with graduating types, numbers and levels of controls. The controls include: performance reports and reviews; site visits; financial reports and acquittals; and payments linked to the achievement of milestones.

4.16 The 2019 risk management framework establishes a proportional approach that balances the need for accountability and transparency with the risk profile of the provider and the activity. In addition, including controls in agreements provides a sound basis for staff to monitor and manage grantee performance.

4.17 To assess whether the changes to the risk management framework had resulted in improved agreements, the ANAO examined a sample of 10 Children and Schooling and 10 Safety and Wellbeing agreements executed between January 2019 and February 2020. The ANAO found that the framework had been applied appropriately, resulting in a better balance between risk and monitoring requirements and improved NIAA’s ability to measure the activities’ achievements against the grant objectives.

Are grants monitored for progress against grant objectives?

4.18 To ensure that the objectives of the grant activity are met, grant agreements should also be supported by ongoing communication, active grants management and risk-based performance requirements.⁷⁹ Two of the mechanisms NIAA uses to monitor a grant’s progress against its objectives are: provider performance reports; and site visits.⁸⁰

Provider performance reports

4.19 Providers are required to submit a report, usually every six months, describing their progress against the agreement’s objectives and performance against the KPIs.⁸¹ In the ANAO sample of 67 Children and Schooling and 65 Safety and Wellbeing grants, 30 Children and Schooling and 33 Safety and Wellbeing agreements had reports due between July 2016 and June 2019. The analysis found that for 29 out of 30 Children and Schooling grants, and 30 out of 33 Safety and Wellbeing grants, the required performance reports had been submitted. There was no evidence recorded in the Grant Processing System (GPS) that NIAA had taken action to address the four instances where reports were not submitted. ANAO also found that there was a record of NIAA’s assessment of submitted performance reports for 27 (93 per cent) Children and Schooling agreements and 30 (100 per cent) Safety and Wellbeing agreements.

4.20 A 2018 PM&C report examining how performance reports are assessed by agreement managers identified that over 90 per cent of the provider reports submitted between July and December 2017 were assessed as ‘satisfactory’ or better by agreement managers. The report suggested that ‘a possible cause of this tendency might be a perception [that] the allocation of any rating of ‘satisfactory’ or better is necessary in order to release a payment’. Therefore, a positive rating may not be an accurate reflection of performance.

4.21 In mid-2017 PM&C introduced a new template for assessing performance reports. Agreement managers must record the completed template in GPS. The 2019 version of the template requires that each KPI be assessed and rated separately, and explanatory notes included for any KPIs assessed as not ‘satisfactory’. A rating for the overall performance of the project must also be recorded, supported by explanatory notes. The ANAO examined the performance report assessments for 10 Children and Schooling and 10 Safety and Wellbeing agreements that covered the reporting period between 1 July 2017 and 30 June 2019. Most assessments followed NIAA guidance, with each KPI consistently assessed and rated separately and explanatory notes providing details of the organisation’s achievement of progress against program objectives.⁸²

4.22 The ANAO also noted some weaknesses that limit the effectiveness of the KPIs in measuring grant progress against their objectives:

• Developing appropriate KPIs that measure the impact or effectiveness of a program in achieving outcomes is challenging. PM&C developed a set of identical KPIs across programs and sub-programs, which has enabled the aggregation of KPI data at program level and reporting against PBS program performance measures. However, the KPIs are not always relevant to the specific objectives of individual activities and, as a result, are not always effective in measuring outcomes at activity level. A number of providers consulted by the ANAO indicated that the KPIs do not align with the services delivered or capture the extent of their achievement.⁸³ Providers provided similar feedback to NIAA in Grant Activity Reviews (GARs) (GARs are examined at paragraphs 5.33 to 5.38).⁸⁴
• The data reported in providers’ performance reports is collected, for at least 90 per cent of Children and Schooling and 80 per cent of Safety and Wellbeing performance measures, by the providers (that is, the data is self-reported). The Grants Administration Manual advises agreement managers to apply their knowledge of the organisation and the community in which the activity is based to determine whether they agree with the information provided in the reports. This provides some assurance about the reliability of the data. However, NIAA does not validate the integrity of providers’ records, for example by requiring agreement managers to inspect a sample of provider records during site visits.⁸⁵
• Seven of the 14 Children and Schooling outcome KPIs and 24 of the 33 Safety and Wellbeing outcome KPIs rely on surveys or client feedback forms. Surveys or feedback forms are not always an appropriate data collection tool when administered by the provider and when targeting populations with low levels of literacy or who may be tempted to provide what they perceive to be the response desired by the provider.

Site visits

4.27 The risk management framework (discussed in paragraph 4.14) requires that a schedule of site visits is included in funding agreements. Activities rated as medium risk must receive an annual site visit and activities with a risk rating of high or extreme must receive two annual visits. All activities, regardless of risk level, are also expected to receive site visits in response to issues as they arise, in addition to other forms of engagement by NIAA, for example, by telephone or email.⁸⁶ To support the implementation of mandatory site visits, in September 2019 NIAA released guidance and templates to prepare for and record visits. Prior to September 2019 there was no consistency between the regional approaches used to document site visits.

4.28 NIAA policy requires that site visits be documented in GPS. The ANAO found that site visits, when they occurred, were not always documented in GPS records. Consequently, site visit records were not easily traceable or uniformly retrievable from GPS. The ANAO reviewed a sample of 10 site visit reports for each program recorded in GPS from 2019 to 2020 and found that the reports were completed with a sufficient level of detail to monitor the delivery of the activity and its impact. This indicates that site visit reports, when they are documented in GPS as required by NIAA, have the potential to inform program performance.

4.29 ANAO fieldwork indicates that the quality of engagement between NIAA and grantees is variable across the Regional Network: approximately half the 132 providers consulted reported that Regional Network staff were not sufficiently supportive, engaged or knowledgeable about their contract.

4.30 As part of its December 2019 restructure, NIAA established Regional Grant Units. The units are expected to undertake the administrative elements of grant management, assess and manage risk, and ensure that contractual issues are appropriately addressed. The restructure has the potential to improve the quality of the Regional Network’s engagement with service providers and Indigenous communities.

Quality control

4.31 In early 2019 NIAA established a Grants Assurance Office to implement an internal quality control and assurance framework to ensure grants are developed and managed in accordance with policy and legal requirements. The Grants Assurance Office work programs for 2019 and 2020 each included eight quality assurance reviews (two ‘spot checks’ of a sample of financial acquittals; and six reviews of a sample of grant activities focusing on key steps of the grant assessment and management process).⁸⁷ As at June 2020 six of the eight scheduled quality assurance reviews on the 2019 work program were completed, one was carried over to the 2020 work program and one had not been conducted. One review from the 2020 work program had commenced.

4.32 The completed reports were provided to the Program Management Board (now the Program Performance Committee) for discussion and endorsement. The reports did not include a management response. Following endorsement, completed reviews are shared with relevant work areas and are available on NIAA’s intranet. NIAA advised that future reports would require a management response, which is expected to provide greater visibility of how agreed actions are proposed to be addressed. NIAA also advised that the role of the Grant Assurance Office was being reconsidered, with a view to enhance timely advice on the quality and consistency of key grant processes.

4.33 Implementing an effective quality assurance program is good administrative practice. As part of the reconsideration of the Grant Assurance Office’s role, NIAA should ensure that adequate mechanisms are developed to support the effectiveness of the quality assurance framework, including ensuring that opportunities for improvement are acted upon.

Are there mechanisms in place to address situations where the purpose of the grant is not being fulfilled?

4.34 In accordance with the CGRGs, where the Australian Government’s purpose or the grantee’s objectives described in the grant agreement are not being fulfilled, officials need to address the situation to ensure proper use of public resources. If NIAA identifies, for example via assessment of performance reports or site visits, situations where the purpose of the grant is not being fulfilled, it should have mechanisms in place to address compliance and/or performance matters.

4.35 The IAS Grant Guidelines provide that NIAA will focus on working with organisations to achieve the intended outcomes of the project or activity. The guidelines indicate that if a grant risks not meeting its objectives or outcomes, NIAA may increase engagement with the grant recipient and affected stakeholders to ‘address the factors contributing to this risk and increase the likelihood of the activity delivering outcomes’. In circumstances of non-compliance with the grant agreement, the guidelines state that NIAA will consider an appropriate response, but does not prescribe the form of that response.⁸⁸

4.36 A Provider Compliance Framework was developed in 2015 to facilitate management of grantee compliance with funding agreement requirements and to support transparency and consistency in decision-making. The framework is complemented by a Provider Compliance Toolkit, which aims to provide staff with additional guidance to operationalise the framework. The framework outlines three levels of non-compliance:

• minor: accidental and/or self-reported minor non-compliance;
• significant: opportunistic or deliberate non-compliance; and
• major: serious and systemic non-compliance, with potential criminal intent.

4.37 The framework states that minor and significant non-compliance should be treated by the agreement manager or regional manager. NIAA staff informed the ANAO that in situations where concerns about providers’ compliance with the conditions of the agreements arise, they will consider the provider’s explanation to decide on what action to take. For instance, a compliance management approach may not be appropriate in situations where service delivery is prevented by factors outside the provider’s control, such as the inability to fill, for a time, a position crucial to the delivery of an activity or to deliver a service during the wet season. In other cases, agreement managers may consider that the provider lacks sufficient experience and capability to run fully effective activities, and that an appropriate course of action is the provision of support and time to improve service delivery.

4.38 Major non-compliance should be escalated to the Program Compliance and Fraud branch (formerly referred to as the Compliance Operations Section) in Canberra for an assessment and possible compliance review. The branch also provides compliance-related advice, referred to as ‘intensive support’, to agreement managers.

4.39 Table 4.1 presents the number of active intensive support matters and compliance reviews for the Children and Schooling and the Safety and Wellbeing programs undertaken by the Program Compliance and Fraud branch between 2016–17 and 2019–20. As cases can remain active across financial years, cases can be counted in more than one year.

Table 4.1: Non-compliance intensive support matters and reviews, 2016–17 to 2019–20

Note: Data to 30 March 2020.

Source: ANAO analysis, based on NIAA data.

4.40 Between 1 July 2016 and 30 March 2020 NIAA closed 56 Children and Schooling and 55 Safety and Wellbeing intensive support matters. Nine of the finalised matters resulted in the agreement being terminated (four for the Children and Schooling program, five for the Safety and Wellbeing program). The distribution of finalised matters across financial years, and the time taken to finalise those matters, is presented in Figure 4.1 and Figure 4.2.

Figure 4.1: Number and duration of intensive support matters in the Children and Schooling Program, 2016–17 to 2019–20

Note: Data is to 30 March 2020. An intensive support matter is excluded from the figure to improve presentation — one Children and Schooling matter was active for 1124 days (closed in 2018–19).

17 intensive support matters pertained to a service provider that delivered grant activities from the Children and Schooling and Safety and Wellbeing Programs, and have been represented in this figure and in Figure 4.2.

Source: ANAO analysis based on NIAA data.

Figure 4.2: Number and duration of intensive support matters in the Safety and Wellbeing Program, 2016–17 to 2019–20

Note: Data is to 30 March 2020. An intensive support matter is excluded from the figure to improve presentation — one Safety and Wellbeing matter was active for 1102 days (closed in 2019–20).

17 intensive support matters pertained to a service provider that delivered grant activities from the Children and Schooling and Safety and Wellbeing Programs, and have been represented in this figure and in Figure 4.1.

Source: ANAO analysis based on NIAA data.

4.41 Between 1 July 2016 and 30 March 2020 NIAA closed 20 Children and Schooling and 25 Safety and Wellbeing compliance reviews. One of the finalised reviews resulted in the agreement being terminated (for the Safety and Wellbeing program). The distribution of finalised reviews across financial years, and the time taken to finalise those reviews, is presented in Figure 4.3 and Figure 4.4.

Figure 4.3: Number and duration of compliance reviews in the Children and Schooling Program, 2016–17 to 2019–20

Source: ANAO analysis based on NIAA data.

Figure 4.4: Number and duration of compliance reviews in the Safety and Wellbeing Program, 2016–17 to 2019–20

Source: ANAO analysis based on NIAA data.

4.42 Changes within NIAA have the potential to impact on the detection and management of non-compliance. These include:

• implementation of a new organisational structure from December 2019 — one of the objectives of the restructure is to focus the work of agreement managers on engagement with providers and other stakeholders, which should help identify and manage under- and non-performance (including through increased support to providers);
• roll out of new KPIs — agreements executed since early 2018 include the new KPIs, targets and benchmarks, which should help agreement managers to assess grant performance; and
• progressive implementation of theories of change — should assist agreement managers identify program expectations and grant objectives and purposes by providing a more robust policy framework.

5.1 The Public Governance Performance and Accountability Act 2013 (PGPA) provides the basis for the Commonwealth performance framework, which allows an assessment of entities’ progress against their purpose. It is comprised of three inter-dependent elements: PBS, corporate plans and performance statements. The PGPA Act requires that performance information should demonstrate the extent to which a Commonwealth entity is meeting its purposes through the activities it undertakes.⁸⁹ Alignment across the elements of the Commonwealth performance framework is intended to improve the line of sight between the use of public resources and the results achieved by entities. The Indigenous Advance Strategy (IAS) Grant Performance Framework outlines the relationship between these elements.

5.2 The ANAO examined the performance framework for the Children and Schooling and the Safety and Wellbeing programs, reporting of program performance to the executive and whether lessons learnt inform program administration.

Has an appropriate performance framework for the programs been implemented?

5.3 The Department of Finance’s guidance on corporate plans indicates that ‘it is important that the performance measures in [the PBS and the corporate plan] are consistent and work together to enable a coherent set of performance results to be included in the annual performance statements (that is, enable a ‘clear read’).’⁹⁰

5.4 In February 2020⁹¹ NIAA published its first corporate plan.⁹² The plan outlines nine activities against which it measures its performance. Activity 6 addresses ‘supporting early childhood development and wellbeing, school attendance, attainment and improved post school pathways’. Activity 2 is ‘improving mental health and wellbeing outcomes for young Indigenous people and supporting suicide prevention in remote communities’. In addition, Activity 8 relates to all Indigenous Advancement Strategy programs.

Alignment between portfolio budget statements and corporate plan

Children and Schooling program

5.5 Figure 5.1 demonstrates that there is a close alignment between the Children and Schooling program in the PBS and NIAA’s combined Activities 6 and 8.

Figure 5.1: Alignment between NIAA corporate plan and PBS for the Children and Schooling program

Note: When NIAA was established as an executive agency, Outcome 2 and its related programs transferred from the Department of the Prime Minister and Cabinet (PM&C) to NIAA and are now presented as Outcome 1. There have been no other changes to the outcome or program structure since the publication of the 2019–20 PBS.

Source: 2019–20 PM&C PBS and NIAA Corporate Plan 2019–20.

Safety and Wellbeing program

5.6 Figure 5.2 illustrates the PBS objectives for the Safety and Wellbeing program and NIAA’s relevant activities.

Figure 5.2: Alignment between NIAA corporate plan and PBS for the Safety and Wellbeing program

Source: 2019–20 PM&C PBS and NIAA Corporate Plan 2019–20.

5.7 The scope of Activity 2 is narrower than the objective and performance criteria of the Safety and Wellbeing program as outlined in the PBS. NIAA’s corporate plan focuses on youth mental health and suicide prevention. It does not address key parts of the PBS objective, which includes the physical, emotional and social wellbeing of all Indigenous communities.⁹³

5.8 In developing the corporate plan, NIAA sought the Department of Finance’s advice. The Department of Finance suggested that NIAA’s corporate plan explains the approach to reporting against the performance measures that were included in PM&C’s PBS (or not reporting if NIAA decided to report against different measures) so it is clear what would be included in NIAA’s 2019–20 annual performance statements. NIAA corporate plan states that it will use the performance measures to report on its performance in its 2019–20 Annual Performance Statement. However, the corporate plan does not include an explanation of its approach to reporting against a different objective than was outlined in the PBS, which limits the ability of the reader to have ‘clear read’ between the PBS and the new NIAA corporate plan. NIAA should remedy this in its next corporate plan.

Appropriateness of performance information

5.9 As shown in Figure 5.1, the Children and Schooling program as presented in the PBS has one performance criterion against which progress is assessed: ‘Indigenous Advancement Strategy activities contribute towards increased Indigenous school attendance and improved educational outcomes’. The Safety and Wellbeing program also has one performance criterion: ‘Indigenous Advancement Strategy activities contribute towards improved wellbeing and/or reduced levels of offending, violence and substance abuse’ (Figure 5.2).

5.10 For each program, the criterion is met if ‘At least 70 per cent of funded activities within the program achieved the primary outcome key performance indicator (KPI) specified in funding agreements’. Table 5.1 summarises the ANAO’s assessment of the appropriateness of performance information for the Children and Schooling and Safety and Wellbeing programs as reported in PM&C’s annual performance statement. Appendix 2 provides more detail of the ANAO’s analysis.

Table 5.1: Appropriateness of performance information for the Children and Schooling and the Safety and Wellbeing programs

Note: Auditor-General Report No.17 2018–19 Implementation of the Annual Performance Statements Requirements 2017–18, pp. 68–69.

Source: ANAO analysis.

5.11 The ANAO found that the performance information is not fully appropriate because it is:

• partially reliable because activity KPIs do not always measure outcomes, there are shortcomings in the methodology for calculating performance information, assessment relies on surveys and self-reported data and the quality assurance mechanisms are not fully effective (free from bias); and
• partially adequate because: it does not assess important aspects of program operations (collective); and it focuses on one short term quantitative measurement type and there is no evidence that better practice was followed when developing the case study that is presented in the performance information (balanced).

5.12 The annual performance statement in PM&C’s 2018–19 annual report states that the target of ‘At least 70 per cent of funded activities within the program achieved the primary outcome KPI specified in funding agreement’ was:

• achieved for the Children and Schooling program: 71.83 per cent of funded activities achieved their primary outcome KPI; and
• substantially achieved for the Safety and Wellbeing program: 68.14 per cent of funded activities achieved their primary outcome KPI.

5.13 The ANAO assessed the methodology supporting these results. As at December 2018, 810 activities were funded under the Children and Schooling program and 512 activities under the Safety and Wellbeing program. However, to calculate the proportion of activities that achieved their primary outcome KPI, the responsible entity used only the activities that were reporting against the revised KPIs from July to December 2018 and had valid data: 465 activities for the Children and Schooling program (57 per cent of all funded activities); and 113 activities for the Safety and Wellbeing program (22 per cent of all funded activities).

5.14 As previously discussed (paragraph 4.11), each activity agreement includes at least five KPIs. To identify the primary outcome KPI, the responsible entity selected one of the KPIs belonging to the category addressing the question ‘Is anyone better off?’. When no outcome KPI was available (either because the agreement did not include any outcome KPIs or because the provider had not reported against these KPIs), PM&C selected the mandatory KPI measuring employment of Indigenous Australians.⁹⁴ Table 5.2 presents the detail of PM&C’s calculation supporting the performance results for the Children and Schooling and the Safety and Wellbeing programs published in PM&C’s 2018–19 annual report.

Table 5.2: PM&C’s calculation supporting the performance result for the Children and Schooling and Safety and Wellbeing programs

Source: ANAO analysis.

5.15 PM&C annual report included a footnote that provided some explanation of the basis for the calculation:

Results [are] based on IAS provider reporting for the period July to December 2018, and for completed reviews only (meaning they have been assessed and accepted by the Department). As the PBS criteria relates to the percentage of activities which achieved their ‘primary outcome KPI’, PM&C selected the most ‘robust’ outcome KPI where there were multiple outcome KPIs reported against for an activity.⁹⁵

5.16 However, the annual report does not mention the fact that the result presented is based on only 57 per cent of all Children and Schooling funded activities; and 22 per cent of all Safety and Wellbeing funded activities. Further, while Indigenous employment is an important input in the two programs, it is not a primary outcome for the programs.⁹⁶ Using results reported against the employment KPI is not an appropriate approach to measuring the programs’ performance, as it does not inform on achievement against the Children and Schooling program’s objectives (getting children to school, improving education outcomes, improving access to further education); or the Safety and Wellbeing program’s objectives (ensuring the ordinary law of the land applies to indigenous communities, ensuring indigenous people enjoy similar levels of wellbeing enjoyed by other Australians).

National Indigenous Australians Agency’s corporate plan

5.22 NIAA’s corporate plan includes one activity that relates to the Children and Schooling program (Activity 6); and one activity that relates to the Safety and Wellbeing program (Activity 2). In addition, Activity 8 relates to all IAS programs.

5.23 Table 5.3 presents the ANAO’s assessment of the performance measures for each activity, and shows that some of the measures could be improved by addressing outcomes for both activities. Further, for the Children and Schooling program, the ‘attainment’ element of Activity 6 was not addressed. In August 2020 NIAA advised that the performance measures were designed for the activities of the Corporate Plan 2019–20 to address the new priorities of the Minister for Indigenous Australians and the Government, rather than the PM&C PBS programs. NIAA also advised that it will review the performance measures and their alignment to the PBS as part of the Corporate Plan 2020–21 and budget process.

Table 5.3: Performance measures for Activities 6, 2 and 8 — NIAA 2019–20 corporate plan

Source: National Indigenous Australians Agency, 2019–20 Corporate Plan, pp. 10–12.

Does reporting support executive consideration of the performance of the programs?

5.29 Effective internal reporting processes assist management to oversee grant processes and make informed decisions about the administration of a grants program, including associated risks and the performance of an entity’s operations.

5.30 Auditor-General Report No.35 2016–17 Indigenous Advancement Strategy identified that analysis of IAS performance data was constrained due to the absence of a mechanism for classifying activities by type. Data analysis could only be conducted through resource intensive manual approaches, which limited PM&C’s ability to report against outcomes, monitor compliance and benchmark similar projects. The ANAO made a recommendation to identify the outcomes and results to be achieved through the IAS and analyse performance information to measure progress against these outcomes.⁹⁷ In response to the ANAO’s recommendation, PM&C implemented a classification system whereby each grant activity was coded against three category levels and a reporting solution to support performance analysis and reporting on grants.⁹⁸

5.31 The reporting solution enables NIAA policy and regional staff to generate tailored reports on grant results, including the reporting and comparison of performance results for similar grants across service providers and locations where services are delivered. The combination of the activity coding and the reporting solution has also enabled NIAA to generate a number of reports for its executive. For example, the Program Management Board, now referred to as the Program Performance Committee, has access to monthly dashboard reports on IAS grantee performance and outcomes. The ANAO reviewed a selection of these reports, which could be effective in supporting executive consideration of the performance of the programs as they contain detailed and regionally specific information

Do lessons learnt inform the administration of the programs?

5.32 Lessons from grants administration should be used to inform improvements to programs and processes. The IAS Performance Framework notes that feedback obtained from processes at the activity-level (such as performance reporting and site visits), and as part of Grant Activity Reviews (GARs) and evaluations, should inform ongoing policy, program and grant design. Performance reports are assessed at paragraphs 4.19 to 4.22. Site visits are examined at paragraphs 4.27 to 4.30. GARs and evaluations are discussed below.

Grant Activity Reviews

5.33 PM&C developed GARs in 2017 to ‘address the need for deeper analysis and information on how the grant agreements operate and how services are delivered’⁹⁹, and complement the information drawn from performance measures included in the funding agreements. Their primary purpose is now to assist NIAA improve its grant management practices and better understand place-based outcomes and impacts achieved through the IAS.

5.34 GARs involve a desktop analysis and on-site visit to observe and gather information, and result in a report on each grant activity reviewed. NIAA also aimed to develop Target Area Reports, which analyse GARs from the same geographical area; and Quarterly Summary Reports, which provide a summary of the GARs and Target Area Reports undertaken during the previous quarter.

5.35 As at 31 December 2019, the responsible entity had completed:

• 158 GARs — 60 relating to the Children and Schooling program; and 61 relating to the Safety and Wellbeing program;
• two Target Area Reports — for Western New South Wales and Far North Queensland; and
• no Quarterly Summary Reports.

5.36 The ANAO reviewed a sample of 56 GARs for the Children and Schooling program and 59 GARs for the Safety and Wellbeing program undertaken between March 2018 and December 2019 across five states and territories and found:

• the methodology applied to conduct the GARs followed a consistent structure, which facilitates the collation and analysis of GAR results across regions or types of activity delivered; and
• the reports deliver detailed place-based information in relation to four focus areas (benefits delivered; business drivers; strengths and opportunities; and grant management). They have the potential to provide valuable information to NIAA agreement managers and policy staff, and providers.

5.37 NIAA regional staff and providers stated that they found the examination of the activities’ operation a valuable and instructive process; NIAA policy staff indicated that the detailed information provided valuable material to inform their decisions in relation to the management of specific programs. In August 2019 an internal review found that GARs provided a ‘crucial opportunity to build capacity and identify capability gaps, for grant recipients as well as NIAA staff’, with a significant amount of evidence, insights and observations being collected.¹⁰⁰

5.38 GARs contain comprehensive information, including on issues that impact on the management on grants. NIAA advised in March 2020 that each GAR report is circulated to relevant program areas and that future work is planned to provide targeted communication to relevant work areas and stakeholders highlighting individual and aggregate GAR findings. Since February 2020 the GARs have also been available through FUSION, with more work planned to further expand the accessibility of GAR information through NIAA’s intranet.

Evaluations

5.39 In February 2018 PM&C released the IAS Evaluation Framework to guide the evaluation of programs and activities. Auditor-General Report No.47 2018–19 Evaluating Aboriginal and Torres Strait Islander Programs examined the effectiveness of the design and implementation of the evaluation framework for the IAS and concluded that ‘five years after the introduction of the IAS, the department [was] in the early stages of implementing an evaluation framework that has the potential to establish a sound foundation for ensuring that evaluation is high quality, ethical, inclusive and focused on improving the outcomes for Aboriginal and Torres Strait Islander peoples’.¹⁰¹

5.40 An Annual Evaluation Work Plan, which lists the IAS programs identified as a priority for evaluation, complements the IAS Evaluation Framework. Four evaluations for the Children and Schooling program and eight evaluations for the Safety and Wellbeing program were listed in the 2018–19 and 2019–20 work plans (see Table 5.4).

Table 5.4: Status of Children and Schooling and Safety and Wellbeing evaluations

Note a: Blue shading denotes variance between the 2018–19 and 2019–20 work plans.

Note b: The 2019–20 work plan also refers to a published review of Petrol Sniffing under the Safety and Wellbeing program. Stage One of the Alcohol and Other Drug Treatment Services Evaluation Report, which detailed the processes and outcomes to develop an assessment tool and evaluation framework to evaluate individual treatment components, was published August 2018.

Note c: NIAA advised that Outback Stores Pty Ltd was undertaking an evaluation of sugar reduction in remote communities and would provide findings to NIAA.

Source: ANAO analysis, based on the 2018–19 and 2019–20 Annual Evaluation Work Plans.

5.41 As at June 2020 there was evidence that the three completed Children and Schooling evaluations have informed changes to the program. For example, following the Remote Schools Attendance Strategy (RSAS) evaluation PM&C conducted a survey seeking staff and service provider feedback on the RSAS training strategy. The survey results showed that future RSAS training should be place-based, tailored to address the issues and sensitivities specific to each community and delivered through local and regional workshops. Subsequently, NIAA developed a RSAS workforce development and training strategy. NIAA advised the ANAO that the completed Safety and Wellbeing evaluation had informed changes to the program.

Integration of information gathering processes

5.42 The processes used by NIAA to collect information on the Children and Schooling and Safety and Wellbeing programs and activities generate a large volume of data. NIAA has developed a number of templates that standardise information collection (for instance for provider reports and site visits). Staff also have the ability to use NIAA’s IT systems (in particular, GPS and FUSION) to store, search and retrieve the information pertaining to each process.

5.43 The different information gathering processes are not yet integrated, which limits NIAA’s ability to use the information effectively to inform the administration of the program. This issue has been recognised by NIAA.¹⁰² The Policy and Investment Framework makes provision for the development of a performance monitoring and evaluation framework, which aims to set up a structured approach to program data management and use.

Appendix 1 National Indigenous Australians Agency response

 

 

Appendix 2 Assessing performance information

1. The PGPA Act requires that performance information should demonstrate the extent to which a Commonwealth entity is meeting its purposes through the activities it undertakes.¹⁰³ While criteria that set a minimum standard for the quality of performance information are not defined in the PGPA Act, the Department of Finance has provided guidance to entities on the characteristics of ‘good’ performance information — relevant, reliable and complete.¹⁰⁴ In the absence of formal criteria in the PGPA Act, the ANAO drew on the Department of Finance’s guidance, and other relevant reference points, to develop audit criteria for assessing the appropriateness of performance information. Two of the characteristics of appropriate performance criteria are relevance and reliability. Table A.1 outlines how relevant and reliable are described in Department of Finance guidance and how the ANAO assesses these elements.

Table A.1: Assessing relevance and reliability

Source: Department of Finance, Resource Management Guide No. 131 Developing Good Performance Information and related Quick Reference Guide, May 2020.

2. For the review of activities that do not relate to an entire entity the ANAO examines the adequacy of performance measures within an activity area rather than whether they are complete. Performance indicators are considered to be adequate where they:

• collectively address the purpose of the activities identified in the PBS; and
• provide a balance between: effectiveness and efficiency indicators, quantitative and qualitative data, and short, medium and long-term performance.

3. Table A.2 presents the ANAO’s assessment of the appropriateness of performance information for the Safety and Wellbeing program.

Table A.2: Appropriateness of performance information for the Children and Schooling and Safety and Wellbeing programs

Note a: Auditor-General Report No.17 2018–19 Implementation of the Annual Performance Statements Requirements 2017–18, pp. 68–69.

Source: ANAO analysis.