Introduction

1. Over recent years, successive Australian governments have adopted various policies aimed at reducing carbon emissions. On 10 July 2011, the then Government released its Securing a Clean Energy Future Plan (the Clean Energy Future Plan). The Clean Energy Future Plan included placing a price on carbon emissions¹ and setting a new long-term target to reduce carbon pollution by 80 per cent by 2050 compared with year 2000 levels. This was in addition to an existing commitment to reduce carbon pollution by five per cent from 2000 levels by 2020.

2. As part of the Clean Energy Future Plan, the then Government committed to providing transitional assistance for manufacturing businesses to adjust to the carbon price. This assistance was to be delivered through the $1.2 billion Clean Technology Program, which comprised the:

• Clean Technology Investment Program (CTIP)—which was allocated $800 million between 2011–12 and 2017–18 to assist manufacturers to invest in energy-efficient capital equipment and low-emissions technologies, processes and products;
• Clean Technology Food and Foundries Investment Program (CTFFIP)—which was allocated $200 million between 2011–12 and 2016–17 to help manufacturers in the food and foundries industries to invest in energy-efficient capital equipment and low-emission technologies, processes and products; and
• Clean Technology Innovation Program—which was allocated $200 million between 2012–13 and 2016–17 to support research and development, proof-of-concept and early stage commercialisation activities that lead to the development of new clean technologies and associated services, including low emission and energy efficient solutions that reduce greenhouse gas emissions.

3. The then Department of Industry, Innovation, Science, Research and Tertiary Education (DIISRTE, now the Department of Industry (the department)) was responsible for the design and implementation of the Clean Technology Program. Specifically, the Manufacturing Policy Branch of the department was responsible for program design, while the AusIndustry Division was responsible for program implementation.

4. The CTIP and CTFFIP (the programs), which are the focus of this audit², had similar objectives³ and reflected a policy articulated in the Clean Energy Future Plan that:

for many Australian manufacturers, improvements in energy efficiency will be the most effective way that carbon cost impacts can be managed to ensure long-term competitiveness. While a carbon price will provide incentives for these manufacturers to reduce energy consumption, the Government will also help manufacturing businesses identify and implement technologies that will improve energy efficiency and reduce their exposure to changing electricity prices.⁴

5. The objective of the programs was supported by the four merit criteria listed in the program guidelines, which are set out in Table S.1. As illustrated by this table, the extent of the reduction in carbon emissions intensity⁵ (merit criterion one) had a higher weighting compared to the other criteria. As a result, the criteria weightings reflected that the main objective of the programs was to reduce the carbon emissions intensity of manufacturers. Further, in this respect, and consistent with the Clean Energy Future Plan, the extent to which a project maintained and improved the competitiveness of the applicant’s business was of significantly less importance.

Table S.1: Merit criteria for the programs

Source: CTIP and CTFFIP program guidelines.

6. A feature of the programs, when compared with many grant programs, was that they operated through a continuous application and assessment process, rather than discrete funding rounds. In this respect, the program guidelines set out that a staged assessment and approval process would be employed, including:

• an initial assessment of the eligibility and completeness of applications by the department;
• merit assessment of eligible applications by Innovation Australia (IA)⁶; and
• a funding recommendation from IA to the decision-maker.⁷

7. The programs opened to applications in February 2012 and closed in October 2013.⁸ A total of 1171 applications seeking $773.5 million were received by the department. The amount sought in these applications ranged from $25 000, which was the minimum amount of funding available under the programs, to $20.8 million.

8. Grants were awarded to manufacturers in a range of industry segments and for different types of emissions reduction measures. However, there were instances where the successful applicant chose not to: execute the funding agreement; or proceed with the project despite having an executed funding agreement in place. In this respect, a total of:

• 603 projects were awarded $314.9 million under the programs, with funding distributed equally between the programs; and
• funding agreements were signed in respect to 569 projects for $295.9 million in funding, with funding distributed equally between the programs. Of these agreements, 16 were later terminated. The remaining 553 projects had an executed funding agreement in place as at 2 October 2014 involving $265.2 million in funding. Of this amount, 54 per cent of funds related to the CTIP and 46 per cent of funds related to the CTFFIP.⁹

Audit objective, scope and criteria

9. The audit objective was to assess the effectiveness and equity of the award of funding under the Clean Technology Program in the context of the program objectives and the Commonwealth’s grants administration framework.

10. In June 2013, the Auditor-General received a request from Senator Simon Birmingham, Liberal Senator for South Australia, then Shadow Parliamentary Secretary for the Murray-Darling Basin and then Shadow Parliamentary Secretary for the Environment to undertake an audit of the Clean Technology Investment Program. The Senator raised a number of concerns about the process for awarding grants under the programs and, in relation to the Clean Technology Investment Program, the distribution of funding in electoral terms. A potential audit of the Clean Technology Program was included in ANAO’s Audit Work Plan for 2013–14 as the resources to undertake an audit were not available at the time of the Senator’s request. Audit resources became available later in 2013, with this audit commencing in November 2013. This audit also examined the specific matters raised by the Senator.

11. The scope of the audit included the design of the programs, as well as the assessment and decision-making processes in respect to the 1171 applications that were received. The audit scope was focused on the application and assessment processes up to the point at which funding was awarded and a funding agreement signed, and also included analyses of the distribution of funding (including in electorate terms) and the announcement and reporting of grant funding. The scope did not include the administration of the Clean Technology Innovation Program (see paragraph 2).¹⁰

12. The audit criteria reflected relevant policy and legislative requirements for the expenditure of public money and the grants administration framework, including the Commonwealth Grant Guidelines (which have now been replaced by the Commonwealth Grants Rules and Guidelines). The criteria also drew upon ANAO’s Administration of Grants Better Practice Guide.

Overall conclusion

13. The Clean Technology Investment Program and the Clean Technology Food and Foundries Investment Program were well received by industry, with 1171 applications made seeking over $773 million in Australian Government funding.¹¹ By the time the programs were closed, after some 18 months of operation, 603 projects had been awarded nearly $315 million in grant funding.¹² This result was achieved due to a range of factors, particularly the strong demand for funding and the significant level of assistance provided to applicants by the department’s AusIndustry Division. In this latter respect:

• a high proportion of applications (84 per cent) proceeded through the eligibility checking stage to merit assessment;
• a similarly high proportion of applications (74 per cent) that proceeded to merit assessment were recommended and approved for funding; and
• expenditure proceeded more quickly than originally budgeted, such that $160 million in program funding was brought forward to 2014–15.

14. The significant amount of grant funding provided to industry over the relatively short period of time the programs were in operation was consistent with AusIndustry’s culture of assisting businesses to access programs it administers. However, the approach that was taken to assessing applications was not sufficiently focussed on maximising program objectives and treating applicants equitably. As a result, it was common for funding to be approved for projects that did not have high expectations as to the extent of the reduction in carbon emissions intensity they would deliver.¹³ Further, a number of the approaches employed to maximise the assistance to industry did not sit comfortably with the operation of a competitive grants program under the Australian Government’s grants administration framework. Specifically:

• some incomplete applications were permitted to proceed to the departmental merit assessment;
• some applications were ‘reframed’¹⁴ to improve their assessed merit in terms of published criteria.¹⁵ The published program material, and internal program documentation, did not clearly establish the circumstances in which reframing assistance would be provided to applicants by the department, and the extent of this assistance. Further, applicants were not required to re-submit reframed applications;
• the assessment and selection method employed was inconsistent with the approach approved by the then Government, which had referred to a competitive grants program. In particular, applications were not ranked in terms of the extent to which they had been assessed as meeting the published merit criteria; and
• the program guidelines stated that applications would be assessed by IA, but 41 per cent of applications were assessed by a committee of departmental officers operating under IA delegation. This helped to expedite the processing of grant applications¹⁶, but meant that specialist knowledge was not used to assess a large number of applications.

15. A key message from a number of ANAO audits of grant programs over the years, which has also been included in both the 2010 and 2013 versions of ANAO’s Grants Administration Better Practice Guide, is that selecting the best grant applications promotes optimal outcomes. In this respect, it was originally expected that projects that delivered a significant reduction in carbon emissions intensity would be funded under the programs. However, the assessment of applications against the primary merit criterion was not solely focused on the extent of the reduction in carbon emissions intensity. Instead, so as to ensure that large manufacturers would not be penalised for making small percentage improvements, the department’s scoring against the emissions intensity reduction criterion also incorporated an assessment of the amount of grant funding requested per tonne of carbon abated.¹⁷ As a result, while the design of the programs intended that only those projects which delivered a significant reduction in carbon emissions intensity would be funded, projects offering relatively low emissions intensity reductions were also successful.¹⁸

16. More broadly, the department did not set a performance target for the programs in terms of the amount of carbon savings, with the indicator it established¹⁹ relatively low in comparison to the then Government’s target to reduce carbon emissions by five per cent from year 2000 levels.²⁰ The department has advised ANAO that it has estimated that the nearly 200 projects that have been completed will abate over 1730 kilotonnes of carbon, a figure slightly above what was expected when funding was recommended and approved. However, the use of grants to fund emissions reduction measures by businesses was a relatively expensive means of reducing carbon emissions in comparison to the carbon price, which the department cited as a reference point for assessing value for money.²¹ Departmental figures indicate a cost per tonne of more than $54²² for the savings expected across all contracted projects, of which the Australian Government is contributing $18 per tonne with industry to contribute the remainder. Grant funds per tonne of carbon abated ranged from $1.52 to $140.27 for individual projects.²³

17. Against this background, the ANAO has made four recommendations that are designed to:

• improve aspects of grant program design and governance arrangements;
• promote equitable access to grant funding opportunities that operate through continuous application and assessment processes;
• improve the merit scoring approaches adopted for competitive, merits-based grant programs; and
• adopt a stronger outcomes orientation when advising decision-makers on which grant applications should be approved for funding.

18. An important message from this audit is that there would be benefit in the department reconsidering the balance that it strikes between AusIndustry assisting business to access funding opportunities, and better practice grants administration principles and practices. In particular, there are risks to be understood and managed when those responsible for assessing applications competing for funding are also able to suggest, or make, changes to an application so as to improve its chances of success.

Key findings by chapter

Program Design (Chapter 2)

19. The design of the programs was challenging for the department given the diversity of projects, and the broad range of technologies of varying complexity eligible to be funded. In this context, design of the programs was informed by extensive stakeholder consultation, with the department responsive to the feedback that it received.

20. To provide a sound overall foundation for implementing the programs, a range of program documentation was developed by the department. However:

• a probity plan was not developed; and
• the programs were not designed to guard against the approval of funding for projects that were largely complete at the time of application or had been committed to prior to receiving any Australian Government funding.

21. In addition, the department developed two sets of guidelines, Ministerially approved program guidelines and ‘customer guidelines’ (which were more comprehensive than the program guidelines). The use of separate, but related program guidelines and customer guidelines made it easier for the department to make changes over time to the assessment of applications. However, having a consolidated set of guidelines is generally regarded as better practice because it provides prospective applicants with a single reference point for key information, including the assessment and selection process, and reduces the risk of inconsistencies between documents. In this respect, a key element of the implementation of the programs involved the use of two indicators to assess applications against the first merit criterion. The existence of two indicators was not identified in the program guidelines, and the second indicator that was used in the assessment of applications²⁴ was not identified until the fifth version of the customer guidelines, promulgated in December 2012.²⁵

22. More broadly, the assessment and selection method identified in the program guidelines was inconsistent with the approach approved by the then Government, which referred to a competitive grants program. In this regard, the assessment and selection process that was implemented reflected elements of a merit-based, non-competitive program. In particular, the programs were not implemented in a way that applications competed for the available funding by being ranked in order of merit. Rather, projects were approved for funding so long as:

• the application had been assessed as eligible and as having some merit (rather than being rated highly against each merit criterion as was required by the program guidelines)²⁶; and
• sufficient program funding remained available.

23. In addition, the department did not develop a sufficiently structured process for assessing applications against the merit criteria, with the assessment of applications involving a range of factors that the department and/or IA considered pertinent. In this respect, one IA committee chair advised ANAO that the committee took into account a range of objective and subjective matters that were not reflected in the records of the assessment process. Further, while the department provided committee members with an assessment template to facilitate the assessment of applications by IA, any completed templates were not retained by the department. In these circumstances, the basis for funding recommendations made by IA to the program delegate was not evident.²⁷

24. As an outcomes orientation is one of the key principles for grants administration included in the CGGs, it is important for agencies to develop an evaluation strategy during the design phase of the granting activity and appropriate performance indicators. The department did not, however, develop an evaluation strategy prior to the closure of the programs. In addition, performance reporting has aggregated the number of projects with an expected reduction in carbon emissions intensity of at least five per cent. This approach:

• does not reflect actual outcomes achieved by the funded projects; and
• was not consistent with the desired outcome set out in the Clean Energy Future Plan.

Access to Funding (Chapter 3)

25. The program was well received by the manufacturing industry, with the department receiving 1171 applications for funding. In total, 159 applications were not processed due to the closure of the programs in October 2013. The remaining 1012 applications proceeded to the eligibility checking stage with relatively few applications (61 or six per cent) assessed by the department as not meeting the eligibility requirements. The department retained documentation to support the majority of eligibility assessments, and affected applicants were provided with advice as to the reasons their application had been assessed as ineligible.

26. The process that was established by the department for assessing applications against the eligibility and merit criteria was focussed on helping applicants access funding under the programs. In particular, the department:

• permitted some incomplete applications to proceed to departmental merit assessment; and
• proposed changes to applications that it considered were not likely to receive funding in their original state.

27. In this latter respect, rather than requiring affected applicants to resubmit a reframed application, it was common for departmental assessors to reframe the project activities, expenditure and/or underlying assumptions for some applications to improve the application’s assessed merit in terms of the published criteria. In these cases, the departmental assessors often made changes to the project or the information in the application²⁸ by:

• adjusting the project, or claims made in the application, themselves and seeking agreement from the applicant to the changes; or
• advising the applicant to exclude components of their project that were not considered competitive.

28. The approach adopted for the programs went well beyond clarifying information included in applications and seeking to address any minor information missing from applications. More broadly, combining advisory and assessment roles is an approach not well suited to maintaining an objective assessment of competing applications. In this context, where government decides that an advisory role should be performed in addition to the assessment of applications, it is preferable that a clear separation be maintained between the roles so as to maintain the objectivity of the assessment stage. There are also challenges that arise in treating applicants equitably due to the risk that the level of assistance provided to applicants will vary.²⁹

Reduction in Emissions (Chapter 4)

29. Consistent with the program objective, the extent of the reduction in carbon emissions intensity associated with projects was the most highly weighted of the merit criteria. There were two indicators used to assess eligible applications against this criterion (see Table S.2). A Carbon and Energy Savings Calculator was developed to calculate results for the two indicators for individual projects and a carbon scoring tool was developed to promote consistent scoring against both indicators for this criterion.

Table S.2: Merit criterion one indicators

Source: CTIP and CTFFIP customer guidelines and program procedures manual.

30. The first indicator was consistent with the published criterion. However, the percentage reduction calculated was highly dependent on the project boundary³⁰, with a smaller boundary resulting in a higher percentage reduction being calculated. This led to inconsistencies in the way that the reduction in carbon emissions intensity was measured and meant that applicants with projects that delivered the same amount of carbon savings could be scored differently depending on the project boundary selected.³¹ Further, one of the approaches adopted during the assessment of applications to improve the chances of an application being approved for funding was to adjust the project boundary being used for emissions intensity reduction calculation purposes (but without the project itself actually changing) so as to increase the assessment score to a level at which the project could be recommended for funding.

31. Departmental records outlined that the second indicator for the first merit criterion was intended to ensure that large manufacturers would not be penalised for making small percentage improvements, as these small improvements could still yield large carbon savings. However, the assessment approach for the second indicator involved the department calculating the grant funding requested per tonne of carbon abated whereas the customer guidelines had identified this indicator as relating to total carbon savings over the life of the conservation measure. The department advised ANAO that this was seen as a way to compare and consistently score total carbon savings over the life of the project. However:

• ANAO’s analysis indicates that such an approach was inconsistent with the first merit criterion, which related to the extent of any reduction in carbon emissions intensity with the second indicator used in the assessment, instead, relating to an application’s cost-effectiveness or ‘value for money’³²; and
• by reducing the requested grant amount³³, applicants or assessors could improve the score achieved against the indicator, without any change to the carbon savings expected to be achieved over the life of the conservation measure.³⁴

32. The existence of the value for money indicator was not identified in the program guidelines and was not identified until the fifth version of the customer guidelines, which were promulgated in December 2012. The inclusion of this indicator had a significant effect in that, had the merit criterion score solely related to each application’s assessed performance in terms of the predicted percentage reduction in carbon emissions intensity, 57 successful applications may not have been awarded a total of $30.6 million in grant funding.

33. A more appropriate and transparent approach, consistent with a range of other Australian Government funded grant programs audited by ANAO, would have been for a separate project cost-effectiveness criterion to have been included in the published program guidelines.

Advice to the Program Delegate and Funding Decisions (Chapter 5)

34. The program delegate within the department was responsible for making all funding decisions under the programs. Of the 849 applications that were assessed against the merit criteria³⁵, 35 were not considered by the program delegate due to the closure of the programs.

35. The grants administration framework has a particular focus on the establishment of transparent and accountable grant decision‐making processes. In this context, the program delegate accepted all recommendations that were received in terms of those applications that should be awarded funding, and those that should be rejected. However, there were a number of shortcomings in the advice provided to the delegate to inform funding decisions. Specifically:

• the records that supported the IA assessment did not demonstrate that each application was assessed against each of the merit criteria;
• the advice provided to the delegate did not demonstrate that the recommended applications rated highly against each of the merit criteria; and
• the advice to the delegate did not clearly identify the expected outcomes from funding each recommended project.³⁶

Reporting and Funding Distribution (Chapter 6)

36. The department’s website reporting of grants made under the programs was largely consistent with the reporting requirements of the CGGs and associated guidance. Further, the application, assessment and decision-making processes for the programs guarded against the award of funding being politicised. ANAO analysis of the distribution of funding awarded under the programs did not identify any political bias. Of note in this respect was that, although the total value of grants to electorates held by the then Government was greater, the approval rate for grants to electorates held by then Opposition was slightly higher.

Summary of entity response

37. The proposed audit report was provided to the department and extracts were provided to the former chairs of the Clean Technology Investment Committee and the Clean Technology Food and Foundries Investment Committee. The department provided formal comments on the proposed report and these are summarised below, with the full response included at Appendix 1:

The Clean Technology Investment Programs were delivered as part of the former government’s Clean Energy Future Plan. The Programs were designed to deliver on two goals of carbon abatement and maintaining the competitiveness of the manufacturing sector by improving energy efficiency through replacement of capital stock. This required innovative program design particularly in regard to the technically complex requirements for estimating and assessing carbon abatement. These methodologies were developed in consultation with key stakeholders including the former Department of Climate Change and Energy Efficiency.

Demand for the Programs were high with 1171 applications received and 603 projects awarded grants. The Programs accommodated a wide variety of projects, ranging from a $50,000 solar panel installation in a small winery to a $70 million dual site consolidation for concrete manufacture. Carbon abatement varied significantly depending on the nature of the emissions reduction measures with supported projects estimated to deliver between 0.5 kilotonnes and 1000 kilotonnes of carbon savings.

The department notes the report acknowledges that the application, assessment and decision-making processes for the programs guarded against the award of funding being politicised and that there was no evidence of political bias in the distribution of funding.

The department acknowledges the contribution the Australian National Audit Office (ANAO) makes to enhancing administration around granting programs through the observations in the report, particularly in relation to administrative transparency. This will assist the department to strengthen its existing grants management frameworks.

Recommendations

Background

1.1 Over recent years, successive Australian governments have adopted various policies aimed at reducing carbon emissions. On 10 July 2011, the former government released its Securing a Clean Energy Future Plan (the Clean Energy Future Plan). An element of the Clean Energy Future Plan provided transitional assistance for manufacturing businesses³⁷ to adjust to the carbon price through the $1.2 billion Clean Technology Program, comprising:

• Clean Technology Investment Program (CTIP)—which was allocated $800 million between 2011–12 and 2017–18 to assist manufacturers to invest in energy-efficient capital equipment and low-emissions technologies, processes and products;
• Clean Technology Food and Foundries Investment Program (CTFFIP)—which was allocated $200 million between 2011–12 and 2016–17 to help manufacturers in the food and foundries industries to invest in energy-efficient capital equipment and low-emission technologies, processes and products; and
• Clean Technology Innovation Program—which was allocated $200 million between 2012–13 and 2016–17 to support research and development, proof-of-concept and early stage commercialisation activities that lead to the development of new clean technologies and associated services, including low emission and energy efficient solutions that reduce greenhouse gas emissions.

1.2 The CTIP and the CTFFIP (the programs), which are the focus of this audit³⁸, reflected a policy articulated in the Clean Energy Future Plan that recognised:

for many Australian manufacturers, improvements in energy efficiency will be the most effective way that carbon cost impacts can be managed to ensure long-term competitiveness. While a carbon price will provide incentives for these manufacturers to reduce energy consumption, the Government will also help manufacturing businesses identify and implement technologies that will improve energy efficiency and reduce their exposure to changing electricity prices.³⁹

1.3 The then Department of Industry, Innovation, Science, Research and Tertiary Education (DIISRTE, now the Department of Industry (the department)) was responsible for the design and implementation of the programs. Specifically, the Manufacturing Policy Branch of the department was responsible for program design, while the AusIndustry Division (AusIndustry) was responsible for implementation. Innovation Australia (IA) was also engaged, as a subject specialist, to assess grant applications..

Program funding

1.4 The programs opened to applications in February 2012 and closed in October 2013. A total of 1171 applications were received seeking $773.5 million.⁴⁰

1.5 The maximum amount of funding available for each project depended on total project expenditure, business turnover and the likelihood of the manufacturer being liable to pay the carbon price, as shown in Table 1.1. The minimum amount of funding available was dependant on the type of project. Specifically, a minimum of:

• $25 000 was available for: changes to the energy sources of existing or replacement plant or processes; or replacement of, or modification to, existing plant, equipment and processes; and
• $1.5 million was available for the replacement of, or modification to, established manufacturing production facilities for new products which offered significant energy or carbon savings during their in-service life.

Table 1.1: Assistance available under the programs

Source: CTIP and CTFFIP program guidelines.

Note 1: ‘Covered emissions’ was defined in the Clean Energy Act 2011 (Cth). This Act was repealed in July 2014. The Clean Energy Regulator defines ‘covered emissions’ as scope one emissions, which are the emissions released by the combustion of coal, gasoline or other fuels as part of a facility’s operations. Manufacturers may also produce scope two emissions, which are generated outside of a facility’s operations. For example, when a manufacturer uses electricity that has been purchased from an electricity provider, the emissions generated are scope two emissions.

Overview of the assessment process

1.6 After an application was submitted, the department assessed it against the eligibility criteria, which related to the type of business, activities undertaken, type of project, and project expenditure. If assessed as eligible, the department then assessed the application against the merit criteria and provided that assessment, along with relevant information from the application, to one of three committees that were given delegated authority by IA to assess applications.

1.7 The IA committees undertook the final merit assessment and provided a funding recommendation to the program delegate. Once a recommendation was provided, the delegate, who was a senior employee of the department, confirmed that funding was available before deciding whether to approve, or not approve, an application.⁴¹ The process followed was dependent on the value of the grant sought, as shown in Figure 1.1.

Figure 1.1: Grant assessment process

Source: ANAO analysis.

Projects funded under the programs

1.8 A total of 603 projects were awarded $314.9 million under the programs.⁴² Grants were awarded to manufacturers in a range of industry segments and for different types of emissions reduction measures. As shown in Figure 1.2, the wine industry received the highest number of grants, but the meat processing industry received more than double the amount of funding.

Figure 1.2: Top ten industries funded under the programs

Source: ANAO analysis.

1.9 The emissions reduction measures funded under the programs are shown in Figure 1.3. The most common measures funded included photovoltaic (PV) system installation and equipment replacement. However, funding awarded for PV system installation projects ($20.3 million GST exclusive) was significantly less than that awarded to equipment replacement project ($160.7 million GST exclusive).

Figure 1.3: Types of projects funded under the programs

Source: ANAO analysis.

Closure of the programs

1.10 On 28 August 2013, the then Opposition announced its intention to discontinue funding for the programs as part of its commitment to deliver savings by abolishing the core elements of the Clean Energy Future Plan. The programs were closed to new applications on 22 October 2013. Details of the closure of the programs were published on the AusIndustry website on 11 November 2013 and AusIndustry contacted applicants directly with advice on the status of their application the following day. The advice was tailored to the status of each application, for example, applicants:

• who had received a letter of offer, but did not have an executed funding agreement were advised that program arrangements were still being determined; and
• whose application was not fully considered prior to the date of program closure were advised that their application was unsuccessful.

1.11 The final program arrangements were decided by the incoming Government following a review of all granting activities. As a result of this process, it was decided to progress with the 106 grant offers that had been made before program closure.⁴³

Audit objective, criteria and methodology

1.12 The audit objective was to assess the effectiveness and equity of the award of funding under the Clean Technology Program in the context of the program objectives and the Commonwealth’s grants administration framework.

1.13 In June 2013, the Auditor-General received a request from Senator Simon Birmingham, Liberal Senator for South Australia, then Shadow Parliamentary Secretary for the Murray-Darling Basin and then Shadow Parliamentary Secretary for the Environment to undertake an audit of the Clean Technology Investment Program. The Senator raised a number of concerns about the process for awarding grants under the programs and, in relation to the Clean Technology Investment Program, the distribution of funding in electoral terms. A potential audit of the Clean Technology Program was included in ANAO’s Audit Work Plan for 2013–14 as the resources to undertake an audit were not available at the time of the Senator’s request. Audit resources became available later in 2013, with this audit commencing in November 2013. This audit also examined the specific matters raised by the Senator.

Audit criteria

1.14 To form a conclusion against this audit objective, ANAO adopted the following high-level criteria:

• the assessment process promoted open, transparent and equitable access to funding and led to those projects most likely to contribute to the cost-effective achievement of the program objective being consistently and transparently recommended for funding approval;
• departmental assessments and advice from Innovation Australia informed the funding decisions that were taken;
• the expected outcomes and distribution of funding were consistent with the objective of the programs and the program guidelines; and
• the announcement and reporting of grants awarded under the programs was adequate, accurate and transparent.

Audit scope and methodology

1.15 The scope of the audit included the design of the programs as well as the assessment and decision-making processes in respect to the 1171 applications that were received. The audit scope was focused on the application and assessment processes up to the point at which funding is awarded and a funding agreement signed, and also included analysis of the distribution of funding (including in electorate terms) and the announcement and reporting of grant funding.

1.16 The scope did not include the administration of the Clean Technology Innovation Program (see paragraph 1.2). The audit scope also did not include the management of grant agreements with successful applicants (and, therefore, did not examine the department’s measurement and verification regime for completed projects) or the evaluation of program outcomes (apart from any steps taken by the department to plan and prepare for program evaluation).

1.17 The audit examined departmental records on the design, implementation and administration of the programs, including:

• application and eligibility assessment records for 1012 applications⁴⁴; and
• assessment records for the 849 applications that were assessed by the IA committees against the merit criteria.⁴⁵

1.18 ANAO also interviewed officials from the department and discussed the implementation of the programs with the chairs of the Clean Technology Investment Committee (CTIC) and the Clean Technology Food and Foundries Investment Committee (CTFFIC).

1.19 The audit has referenced the grants framework that was in place at the time that the programs operated (in particular, the Finance Management and Accountability Act 1997 (FMA Act), Financial Management and Accountability Regulations 1997 (FMA Regulations) and the Commonwealth Grant Guidelines (CGGs)). The framework changed after the programs had closed, with implementation of the grants-related elements of the Public Governance, Performance and Accountability Act 2013 (PGPA Act) taking effect from 1 July 2014. In this respect, unless stated otherwise, similar arrangements exist under the current framework (PGPA Act and the Commonwealth Grant Rules and Guidelines (CGRGs)).

1.20 The audit was conducted in accordance with ANAO auditing standards at a cost to ANAO of $754 000.

Report structure

1.21 The structure of the report is outlined in Table 1.2.

Table 1.2: Structure of the report

Introduction

2.1 From the time that the programs were announced in July 2011, the department was responsible for the design and implementation of the programs. Accordingly, ANAO examined the extent to which the department designed an appropriate framework for administering the programs through the:

• program guidelines and other documentation;
• objective of the programs;
• program governance arrangements; and
• key program documentation.

Program guidelines

2.2 A key obligation under the CGGs was for all grant programs to have program guidelines in place. Program guidelines play a central role in the conduct of effective, efficient and accountable grants administration, by articulating the policy intent of a program and the supporting administrative arrangements for making funding decisions.⁴⁶ Reflecting their importance, the program guidelines represent one of the policy requirements that proposed grants must be consistent with in order to be approved for funding.⁴⁷

2.3 The CGGs indicated that, where appropriate, consulting stakeholders on grant arrangements could help achieve more efficient and effective grants administration. In this context, the department facilitated an extensive stakeholder consultation process that was used to inform program design. This process involved:

• consultations with key industry groups between August 2011 and September 2011;
• a discussion paper that was released in September 2011;
• 14 targeted public consultations that were attended by 439 representatives from business, government and industry groups between October 2011 and December 2011; and
• 94 written submissions, received between October 2011 and December 2011, in response to the discussion paper.

2.4 The department was responsive to the feedback provided during the consultation period. For example, the programs were originally designed to provide a grant of up to 25 per cent of eligible costs, but after feedback was received that manufacturers would have difficulty financing investments at a maximum government grant contribution of 25 per cent, the tiered funding ratios presented in Table 1.1 on page 35 were adopted.

2.5 After the public consultation period had concluded, the department finalised the program guidelines. At that time, the requirements of the grant framework relating to the approval of program guidelines were dependent on a risk assessment by the department. Both programs were rated as ‘medium risk’ and the program guidelines and risk assessments were reviewed by the Department of the Prime Minister and Cabinet and the then Department of Finance and Deregulation (Finance). Final approval of the guidelines was given by the then Prime Minister in January 2012.

Customer guidelines

2.6 The CGGs described program guidelines as being a ‘single reference source’ for policy guidance, administrative procedures, appraisal criteria, monitoring requirements, evaluation strategies and standard forms.⁴⁸ In a previous ANAO audit⁴⁹, instances where agencies were producing more than one document that outlined important aspects of the grant selection process were observed. In this regard, ANAO identified that:

• the better approach is for a single program guidelines document to be prepared (and approved) that represents the reference source for guidance on the grant selection process, including the relevant threshold and assessment criteria, and how they will be applied in the selection process; but
• where more than one document is produced and each outlines important aspects of the grant selection process, it is important that agencies recognise that, collectively, all such documents constitute the program guidelines for the purposes of the CGGs. Accordingly, these documents should collectively be subject to the grant program approval requirements and made available to stakeholders.

2.7 The program guidelines provided an overview of the programs, as required by the CGGs. However, a separate document (referred to by the department as the customer guidelines) was also prepared. The department’s intent in providing customer guidelines was to give prospective applicants a more comprehensive reference point for application development and the assessment and selection process. Despite this intention, the customer guidelines did not provide potential applicants with an accurate overview of the assessment process in relation to the:

• opportunity for applicants to provide additional information after application submission;
• department assisting applicants to reframe applications to exclude proposed capital investment activities that were not considered to provide a relative contribution to carbon savings that was commensurate with the share of project costs; or
• approach used to assess applications against the merit criteria, with specific reference to the indicators used to assess merit criterion one and a range of undocumented factors considered important by the IA committees in arriving at funding recommendations.⁵⁰

2.8 The customer guidelines were not subject to the approval process outlined in the CGGs on advice taken from Finance. This situation made it easier for the department to make changes over time, particularly in relation to the assessment approach. However, as previously outlined by ANAO, the approach of having two separate sets of guidelines for a program, with important detail not included in the guidelines subject to the grants administration framework approval processes was not consistent with better practice.

2.9 For example, the program guidelines outlined the merit criteria for the programs, but the customer guidelines provided further details regarding the implementation of those criteria (see Figure 2.1). Of particular significance was that the program guidelines did not identify that two indicators were being used to assess applications against the first (and most heavily weighted) merit criterion, or the relative weighting applied to those two indicators.⁵¹

Other program documentation

2.10 To assist applicants, the department also repeated some of the information in the program and customer guidelines in the electronic application form (a ‘smartform’) and in a series of fact sheets.⁵² In addition to the published program documentation, the department developed a procedures manual that:

• described the roles and responsibilities of departmental officials when processing and assessing applications;
• included a framework for allocating merit scores to applications; and
• provided a template for the departmental assessment.

Figure 2.1: Information on merit assessment published in the customer guidelines

Source: ANAO analysis of customer guidelines.

2.11 The primary purpose of the departmental assessment was to present the relevant IA committee with a report that included:

• an assessment of: the application against each of the merit criteria in the form of a score, with the scores summed to give the overall merit score; and the quality of the evidence supporting the application;
• options to reframe the project to exclude any ineligible activities⁵³ or eligible activities that the department considered did not provide value for money;
• additional commentary to highlight information that had the potential to significantly impact the final decision, but was not easily identifiable in the application; and

• an assessment recommendation.

Probity plan

2.12 The CGGs also outlined that an important requirement in grant administration is ensuring probity and transparency, such that decisions relating to the granting activity are impartial, appropriately documented and publicly defensible. In this respect, the department did not develop a probity plan or engage a probity advisor.⁵⁴

Objective of the programs

2.13 Program objectives that are clearly linked to the outcomes set by government were required by the CGGs. As such, it was important that the objective of the programs was consistent with the outcome set in the Clean Energy Future Plan, which was to reduce carbon pollution and assist manufacturers to reduce their exposure to rising energy costs.⁵⁵

2.14 The objective of the programs, as stated in the program guidelines, was:

To assist Australian [manufacturing] businesses to invest in energy efficient capital equipment and low emissions technologies, processes and products in order to maintain the competitiveness of Australian manufacturing businesses in a carbon constrained economy.

2.15 In practice, the objective of the programs was implemented through the assessment of applications against the merit criteria, using the weightings in Table 2.1.

Table 2.1: Merit criteria and weightings

Source: CTIP guidelines and CTFFIP guidelines.

2.16 As illustrated in Table 2.1, the extent of the reduction in carbon emissions intensity (merit criterion one) had a higher weighting compared to the other criteria. As a result, the criteria weightings reflected that the main objective of the programs was to reduce the carbon emissions intensity of manufacturers and that the competitive position of businesses was of significantly less importance.⁵⁶ Accordingly, the objective that was implied by the weighting of the merit criteria was consistent with the outcomes set out in the Clean Energy Future Plan.

Value with public money and additionality

2.17 As noted in ANAO’s Grants Administration Better Practice Guide, value with public money should be considered at two levels:

• in the context of grant allocation, the extent to which a population of projects maximises the achievement of specified objectives within the available funding; and
• in the context of selecting individual projects for funding, selected applications should be eligible, have met the selection criteria, involve reasonable cost and have a risk profile that is acceptable to the Commonwealth.⁵⁷

2.18 With regard to the value of individual projects, there were four merit criteria for the programs, but none specifically addressed the reasonableness of project costs (as shown in Table 2.1). However, the assessment of applications against merit criterion one included consideration of the grant funds per tonne of carbon abated. This indicator provided a measure of the fiscal cost of abatement⁵⁸ and was used as a basis for identifying and promoting changes to projects that the department considered were ‘unlikely to represent value for money’.⁵⁹ In June 2014, the department advised ANAO that the cost of abatement measure was ‘simply a tool that allowed us to compare and consistently score total carbon savings over the life of the project’. Despite this advice:

• the department’s customer guidelines released in August 2013 (near the time of program closure) stated that ‘total carbon savings in the context of value for money (grant dollars for each tonne of carbon abated)’ was a factor considered in the assessment of merit criterion one; and
• both IA committee chairs advised ANAO that value for money was considered during committee deliberations.

2.19 Value with public money is also promoted by considering the extent to which the funding being sought by an applicant will result in an outcome that is additional to those that are likely to occur regardless of whether the application is successful. This is referred to as ‘additionality’. There was no criterion or provision in the program guidelines that prevented projects that were being implemented without funding assistance from being assessed as eligible or meritorious. This reflected that the programs were administered as ‘transitional adjustment assistance’ for the carbon price. In this regard, the program manager noted in May 2012 that:

Some applicants are likely to choose to order and/or part pay for plant and equipment with significant supply lead times prior to making an application or the project start date. There is no additionality requirement in the Program and it is consistent with the policy intent of the Program to allow applicants to claim remaining costs for these plant items that are paid within the project period.

It is recognised that there may be risks associated with meeting the Australian National Audit Office Granting Guidelines value for money test for funding project activities for an ERM [emissions reduction measure] that is largely complete at [the] time of application. An extreme example would be a project to put a roof on an otherwise completed replacement manufacturing facility.

This risk is mitigated by the requirement for the project to rate highly against all program merit criteria.

2.20 However, ANAO analysis was that:

• the lack of an additionality criterion was inconsistent with the policy because the implementation arrangements in the policy proposal stated that funding would not be provided for projects that were intended to be undertaken privately in the absence of the programs;
• none of the program merit criteria specifically required consideration of additionality, nor was information that would inform the consideration of additionality sought in application forms; and
• it was not until December 2012 that benchmarks were set to reflect the score required for applications to be considered to rate highly against all program merit criteria.⁶⁰

2.21 The risk of funding projects that would have proceeded without government funding was not addressed in practice, as highlighted by the examples in Table 2.2.

Table 2.2: Projects largely complete at the time of application submission⁶¹

Source: ANAO analysis of departmental records.

2.22 As part of an internal review of the programs in November 2013, the department identified that:

A consequence of no need-for-funding requirements is that some approved projects would have already been implemented without grant funding. In fact, some approved projects had already undergone internal final approvals and were already underway at the time of application.

2.23 Although in some cases there were significant periods of time between the date of lodgement of an application, decision and execution of a funding agreement, there were also:

• 25 funding agreements with grants totalling $6.5 million that were executed on or after the project end date listed in the funding agreement; and
• a further 109 funding agreements with grants totalling $38.5 million executed within three months of the project end date listed in the funding agreement.⁶²

Assessment and selection process

2.24 An important consideration in the design of any grants program is the process by which potential grant recipients will be able to access grants. At the time that the assessment and selection processes for the programs were determined, the CGGs outlined that:

Unless specifically agreed otherwise, competitive, merit based selection processes should be used, based upon clearly defined selection criteria.⁶³

2.25 While guidance on a competitive selection process was not provided in the 2009 version of the CGGs, the following guidance was available to the department in the ANAO’s 2010 Grants Administration Better Practice Guide⁶⁴:

An appropriately conducted competitive, merit-based grant selection process involves all eligible, compliant applications being assessed in the same manner against the same criteria, and then being ranked in priority order for receipt of the available funding based upon the outcome of those assessments.⁶⁵

2.26 In December 2011, Finance advised the department to provide a clear statement of the selection process in the proposed program guidelines, advising the department that:

It notes in the guidelines that the program is a competitive merit (page 2) based grants program. However, it also states that applications can be lodged at any time. It needs to be clearer what type of merit selection process will be used. For example, will applications be assessed against the selection criteria, or will applications be ranked against other applicants’ applications. During the seven year period, will there be points in which all applications will be ranked, or will applications be assessed on a case-by-case basis, with recommendations going to the delegate on an ad-hoc basis. It would be useful if more information be [sic] included so potential recipients can see more clearly how the merit selection process will operate.

2.27 The process that the department implemented involved applications being individually assessed against the merit criteria, with funding decisions made on an ongoing basis. In the context of guidance available at that time concerning grant selection processes, this approach was not consistent with the policy proposal and program guidelines, which referred to a ‘competitive grants program’. In this respect, the department advised ANAO in June 2014 that:

[The 2009 version of the CGGs] do not provide any additional guidance in relation to competitive selection processes.

On the basis of those Guidelines the program was correctly described as a competitive granting program in the [policy proposal] and the selection process of individual assessment against the merit criteria is consistent with that description.

2.28 Notwithstanding the above advice, the programs were launched at a time when the department’s Customer Service Charter clearly stated that a competitive grants program was one in which applicants compete for program funding as follows:

AusIndustry offers both entitlement and competitive grant programs.

For grants-based products, customers compete for limited funds, based on the merits of their application.

For entitlement programs, such as the R&D Tax Concession, customers make claims in accordance with pre-determined criteria.⁶⁶ [ANAO emphasis]

2.29 Further, the department advised ANAO in June 2014 that:

Innovation Australia may also choose to rank applications considered at a particular meeting. Ranking would be undertaken where it is necessary to restrict approvals in accordance with available funding. This has not been required to date. [ANAO emphasis]

2.30 This is inconsistent with the advice that the department provided to the then Minister for Industry and Innovation in February 2012 that:

Innovation Australia’s newly formed Clean Technology Investment Committee requires appointment of a Chair and members to assess and merit rank applications submitted under both the Clean Technology Investment Program and the Clean Technology Food and Foundries Investment Program. [ANAO emphasis]

2.31 The reasons for the department’s decision not to rank applications may relate to the difficulty in establishing the relative merits of applications for projects that involve substantively different activities. In this regard, the department separately advised ANAO in June 2014 that:

There is significant variation in project size, complexity, activity and outcomes depending on the nature of the emission reductions measures to be undertaken and industry sector. The value of projects considered to date ranges from $50 000 to $70 million. Projects can encompass activities ranging from replacement of lighting fixtures, modifying an existing manufacturing process, installing a co-generation plant, covering anaerobic lagoons or replacing an entire manufacturing site. Reductions in emissions intensity will vary significantly depending on the nature of the emissions reduction measures included in the project. Given the significant variation between projects, applications are assessed individually against the program merit criteria rather than against other projects within a funding round.

2.32 However, these factors were not documented at the time that the assessment and selection process was decided upon.

2.33 A program in which applications were individually assessed against merit criteria would more accurately be described as a merit-based, non-competitive program (which is the definition adopted by the CGRGs, as well as in ANAO’s Grants Administration Better Practice Guide). However, the process used also exhibited some of the characteristics of a demand-driven program as 74 per cent of the applications that were considered by the program delegate were approved.⁶⁷ Following advice from ANAO in June 2013 (prior to the commencement of the audit), the department acknowledged that the programs were not competitive grants programs, in the context of the 2013 CGGs, by updating the program guidelines to reflect that applications would be individually assessed against the merit criteria. The department advised ANAO in October 2014 that ‘despite this reclassification, there was no change to the assessment process’.

2.34 Given that there was no change to the assessment process or the definition of a competitive grants program that was provided in the ANAO’s 2010 Grants Administration Better Practice Guide, the department did not obtain agreement, from the then Government, to change the assessment and selection process from a competitive grants program (as identified in the policy proposal) to an open, non-competitive or a demand-driven grants program.

Program governance arrangements

2.35 Key roles and responsibilities in the administration of the programs were set out in program governance documentation and the program guidelines. Responsibility for the program’s design and delivery was shared between the manufacturing policy area of the department and AusIndustry. The manufacturing policy area was responsible for designing the programs and providing ongoing policy advice, while the day-to-day management and delivery of the programs was carried out by AusIndustry. Oversight was managed through monthly reports from AusIndustry to the department’s executive group. These reports provided a financial management summary and summarised key issues, risks and opportunities relating to the program.

Role of the program delegate

2.36 On 1 February 2012, the then Minister for Industry and Innovation appointed the departmental official who was acting in the role of General Manager, Clean Technology Investment Branch, AusIndustry as the program delegate to:

• take all necessary decisions and carry out all necessary functions in relation to the administration of the programs;
• authorise employees of the department to take decisions and to carry out functions as are specified in the program guidelines; and
• award funding to applicants under the programs, subsequent to a merit assessment and recommendation from IA.

2.37 The program manager was responsible for the day-to-day operations of the programs including overseeing the delivery network, but the delegate attended the majority of IA committee meetings. In this regard, the department advised ANAO in September 2014 that:

Neither the program delegate or program manager were members of the Clean Technology Investment Committee or Clean Technology Food and Foundries Investment Committee. They could elect to attend meetings, and often did, as observers to provide advice to the committee on the program rules if asked, and to gain insight into the factors considered by the committee(s) in recommending a grant application for approval.

2.38 However, as outlined in ANAO’s Grants Administration Better Practice Guide, irrespective of whether the approver is a Minister or an official, it is prudent for the approver to remain at arm’s length from the assessment process. This separation avoids the potential for perceptions to arise that the approver has influenced the funding recommendations subsequently put forward for the approver’s consideration.

Role of Innovation Australia

2.39 It is relatively common for an advisory committee to be used to provide advice and/or recommendations to grant program decision‐makers. Where a program relates to a specific area, such committees are able to bring relevant knowledge, experience and judgement to bear in formulating, or assisting to formulate, funding recommendations. In this context, it is important that the basis on which an advisory committee is to be involved in an assessment and selection process is clearly defined, and that the committee’s deliberations and recommendations are appropriately documented.

2.40 Against this background, the role of Innovation Australia (IA) was to:

• undertake a merit assessment of each eligible application;
• provide a recommendation to the program delegate (or to the Cabinet of the Australian Government, where relevant); and
• provide more general advice to the program delegate in relation to the programs and the development of the customer guidelines.

2.41 To carry out this role, an IA committee was created and, on 22 March 2012, it received delegated authority from the Board of IA to make funding recommendations to the program delegate up to a maximum of $5 million.⁶⁸ All of the members of the IA committee had skills in manufacturing or engineering and business administration, but only three members had experience in clean technology and two members had experience in energy and carbon efficiency.

2.42 In October 2012, the IA committee was separated into three committees due to the significant workload associated with the programs. The responsibilities of these committees in assessing, and making recommendations on, applications were as follows:

• departmental committee—low-risk grants under $300 000. This committee consisted of members from the program management area and manufacturing policy area of the department and also included managers from AusIndustry’s State Office Network and officers with responsibility for the delivery of the programs;
• CTIC—general investment applications up to a maximum value of $5 million. This committee consisted of six members who had experience in manufacturing or engineering and business administration, two members who had experience in clean technology and one member who had experience in clean energy and carbon efficiency; and
• CTFFIC—food and foundries applications up to a maximum value of $5 million. This committee consisted of five members who had experience in manufacturing or engineering and business administration, two members who had experience in energy and carbon efficiency and one member who had experience in clean technology.

2.43 While the members of the IA committees were not employees of the department, the functions they were responsible for performing carried specific record keeping obligations. In this context and as outlined in the CGGs and ANAO’s Grants Administration Better Practice Guide, where the advice provided by a committee directly informs a decision about expenditure—for example, where the committee assesses applicants against particular criteria, and/or recommends supporting particular projects or distributing funds to particular applicants—committee members are officials of the relevant agency.⁶⁹ Importantly, where advice and recommendations received from a panel are intended to be relied upon by a decision‐maker in forming a view on the merits of providing grant funding, the assessment of applications and funding recommendations of the panel should be formulated in a manner that is consistent with the program guidelines. This provides a sound basis for the departmental decision-maker concluding that a recommendation to approve funding is consistent with the program guidelines as a policy of the Commonwealth (as was required by FMA Regulation 9).⁷⁰

Assessment of applications by the departmental committee

2.44 The creation of the departmental committee assisted with the management of the workloads for the CTIC and the CTFFIC. This arrangement, however, meant that a number of merit assessments were undertaken by the department rather than IA, as described in the program and customer guidelines, such that specialist knowledge was not brought to bear in the assessment of applications.

2.45 As shown in Figure 2.3 on page 63, 345 applications (41 per cent) were assessed by the departmental committee. In 75 per cent of these assessments, the departmental committee agreed with the score provided by the departmental assessor. By way of comparison, the CTIC and the CTFFIC agreed with the score allocated by the departmental assessor in 18 per cent of cases. Given that IA had a role in providing specialist knowledge in the assessment of applications, the use of a different arrangement did not allow for the specialist knowledge that was available to be applied. As a consequence, the transparency of the assessment process was reduced for applicants and the knowledge applied differed across applications.

Documentation retained to support recommendations

2.46 The procedures manual set out the role of departmental officials in assisting the IA committees with the final merit assessment. This included compiling an ‘application deck’, which included the departmental assessment report, the submitted application and other relevant supporting documentation. The application deck was intended to provide committee members with ‘the core documents needed to review and assess the application’ and was only to include information ‘that is necessary and sufficient for the committee to make an informed decision’.

2.47 While the department developed a scoring framework to be used in the departmental assessment that was presented to the IA committees, the evidence provided by the department did not reflect that the IA committees used the same framework in scoring applications against the merit criteria. In this respect, the program management area of the department advised IA committee members, in April 2012, that ‘committee members may wish to score on some other basis’. In October 2014, the department confirmed this in advice to ANAO that:

The purpose of the Carbon Scoring tool is to ensure that AusIndustry Customer Service Managers take a uniform approach to scoring against this criterion. Committee Members may wish to score on some other basis.

2.48 With regard to IA committee deliberations one of the IA committee chairs advised ANAO in September 2014 that:

The assessment was complex because there were many inputs that the committee considered, with the final scores as presented by the CSM [Customer Service Manager] being only some of them. Other critical issues that had different bearings on various applications included:

• the type and age of existing vs new equipment, the suppliers, the amount of data provided on estimated performance of the new equipment; depreciation rates etc
• the assessment of the validity of all the assumptions throughout the application
• the business profile of the applicant, its industry, competitors, the applicant’s financial history and robustness of their future cashflow (in addition to the necessary citing of the Accountants Declaration)
• the production forecasts and how believable they were, particularly how they related to sales forecasts - and how this impacted the merit criteria
• the complexities around suitable boundary definitions for calculations
• the value for money in relative and absolute terms
• the necessity to [sic] being consistent and fair in approach and where appropriate taking into account many evolved learnings gained over time with many many assessments completed
• changing government policy on treatment of carbon emissions.

2.49 The consideration of these particular matters was not well recorded, as discussed in paragraphs 2.50 to 2.54.

2.50 To facilitate the assessment of applications by the IA committees, the department provided committee members with an assessment template. This template did not provide a basis for scoring applications; rather it provided a means of the committee member, who was allocated as a spokesperson for the application, recording recommended scores. Using the template, committee members could:

• record a score for the application against each merit criterion after considering the departmental assessment report, the original application and any relevant attachments; and
• provide their scores against each criterion to the Secretariat in advance of the relevant meeting.⁷¹

2.51 In addition, the template enabled committee members to do more than simply record an assessment score against each criterion and indicate whether the member supported or did not support the award of funding. In particular, the template provided a means for committee members to record the rationale for their scoring, which is important in promoting accountability in assessment processes. However, the department advised ANAO in May 2014 that:

The [templates] were not designed to be provided back to the Secretariat to keep as a permanent record and were not kept by the Secretariat. Members are told that they can leave papers/documents at the end of the meeting to be destroyed should they not wish to keep these themselves.

Scores are placed by the Secretariat into a template that lists all applications for that meeting (these are kept as a record by the Secretariat). At the meeting the scores are then moderated following discussion of each application by the committee (those members with conflicts did not provide scores and were not present during the discussion or the final decision of the committee on an application where they had a conflict). The final decision and recommendation of the committee is put into a decision sheet and recorded in minutes.

2.52 Where templates were provided by committee members to the department, they were not retained in departmental records. In this respect, in October 2014, the department advised ANAO that:

The Committee Member assessment records were not destroyed. The offer to destroy the Committee Member’s [sic] personal notes is a long standing security procedure to support members’ appropriate management of papers in their personal possession.

The members individual merit scores provided to the meeting are kept as noted and the Committee’s recommendation on the merit of an application is set out in the decision sheet and minutes.

2.53 As a consequence of the department’s decision not to retain the templates completed by committee members, the documentation that was provided in support of the committees’ recommendations was limited to:

• a spreadsheet for each meeting that included scores provided by the committee members who were the spokespersons for each application;
• the signed minutes of each meeting;
• a signed recommendation for each application; and
• informal notes that supported the meeting minutes.⁷²

2.54 However, the information that was contained in the meeting minutes and individual recommendation sheets was identical and, as shown in Figure 2.2, did not provide sufficient details regarding the assessment of applications, including the basis for the recommendations made by the IA committees or the range of matters ANAO has been advised were taken into account (see paragraph 2.48).

Figure 2.2: Example of a recommendation sheet from the CTFFIC

Source: ANAO adaptation from IA Committee Recommendations.

2.55 It would be reasonable to expect that the value added by the specialist knowledge of the IA committees with external members would be reflected in there being greater variation between committee assessment scores and departmental assessment reports. The department has advised ANAO that it anticipated that this would be the case, but ANAO’s analysis was that the program was not implemented in a way that ensured the reasons for differences were appropriately recorded for accountability and related purposes.

2.56 In this context, Figure 2.3 demonstrates that there was greater consistency between the department’s records and the records of the departmental committee compared to the other IA committees. Specifically, for 75 per cent of applications considered by the departmental committee the scores were consistent with the departmental assessment; but this fell to 18 per cent of applications for the other IA committees. Accordingly, while the departmental assessment reports provide some basis for explaining three-quarters of the recommendations made by the departmental committee, they not surprisingly provide only limited explanation for the recommendations of the other IA committees.

Figure 2.3: Comparison of merit score in the documentation provided

Source: ANAO analysis of AusIndustry data.

2.57 In September 2014, an IA committee chair advised ANAO that the relevant committee had held ‘detailed, meaningful and complex’ discussions and that:

As you know there were no transcripts of these discussions and hence the final recommendation is merely that, a recommendation based on significant inputs (firstly of course the CSMs recommendation) and the Merit Criteria summaries, and finally in most cases intense deliberation. It is for this reason that the assessment recommendations were not always perfectly correlated with absolute merit criteria scores. The recommendations necessarily and properly in my view, incorporated many more issues, both objective in some cases and subjective in others where the expert committees experience were well utilised.

2.58 The situation in relation to the programs was similar to that identified in an earlier ANAO audit of a program implemented by the department, as follows:

The minutes of Board meetings in the second funding round describe the approach taken by the Board to assess applications and contained details of the Board’s overall merit assessment for each application. Other details relating to the Board’s assessment of each application—such as the Board’s overall assessment comment and details of assessments against the individual evaluation criteria—were not reflected in the final minutes. Rather, the department advised that this information was documented in supporting spreadsheets. While details of the Board’s overall assessment comments were available, details of the Board’s assessments at the individual criteria level were unable to be located by the department during the course of the audit.⁷³

2.59 The department accepted the subsequent recommendation made in the earlier audit for improved documentation of the assessment process.

Disclosures of interest

2.60 The program guidelines and IA’s disclosure of interest guidelines set out how potential conflicts of interest were to be identified and managed for IA committee members. If a conflict was declared prior to a meeting, the committee member did not receive a copy of the relevant application. In addition, committee members were invited to declare any additional disclosures for which they became aware of at the start of each meeting, following receipt of the meeting papers. Committee members then left the meeting room before discussion of the relevant application and this was documented in the minutes. To support committee members in making conflict of interest disclosures, the department developed and circulated ‘disclosure of interest’ statements relating to applicants, which contained information on collaborative partners, directors, board members and primary shareholders.

2.61 Notwithstanding the requirements of IA disclosure of interest guidelines, in 28 of the 92 material disclosures, the signed minutes did not record when committee members left the room as a result of a conflict of interest.⁷⁴ In this respect, the department advised ANAO in May 2014 that:

There was an issue with the accuracy of the documents. Additional supporting documents confirm that the process was undertaken with members departing the room.

Action was undertaken to rectify the records once these issues were identified internally with the relevant Committee Manager putting a file note confirming that conflicted members left the room in accordance with the Disclosure of Interest Table which accompanied the minutes.⁷⁵

Further action has been undertaken to reinforce standard operating procedures for the Secretariat in minute taking as per secretariat procedures manuals for all committees where there is a requirement for the primary minute taker to capture the arrival/departure of meeting attendees and movement of attendees where material disclosures exist.

2.62 There were also nine instances in which the chair had a disclosed material conflict, but the minutes did not note that a replacement chair had been appointed for the period of the chair’s absence. However, in all of these cases the recommendation that was made to the program delegate was signed by a committee member who did not have a disclosed interest.

2.63 Further, while the practices were sound in terms of the disclosure of conflicts of interest for the majority of applications, there were instances in which practices could have been improved. For example, there were two applications for which a disclosed conflict of interest was assessed by the department as being immaterial, an assessment that raises some questions given:

• the assessment of one application was informed by one of the committee member spokespersons having previously visited the site and having a past association with one of the directors of the applicant company; and
• the committee member held shares in the company that was supplying the power infrastructure upgrade for the project.

2.64 In both cases, the committee member that had the conflict was recorded in the minutes as being a spokesperson for the relevant applications.

Program performance measurement

2.65 The CGGs stated that grants administration should focus on the delivery of government outcomes.⁷⁶ ANAO’s Grants Administration Better Practice Guide also highlights the importance of developing a performance/evaluation framework during the design phase of a program. For the programs, the responsibilities of the program delegate and the grant recipient, in terms of monitoring and evaluation, were identified in the program guidelines, but were not supported by an evaluation strategy. The department advised ANAO in July 2014 that:

It was anticipated that the Program would be evaluated in 2015/16. A formal evaluation strategy has not yet been prepared. However, [the department has] given detailed consideration to the project metrics that would be required to support a program evaluation.

2.66 This advice was updated by the department in October 2014 to reflect that the evaluation is planned for 2016–17 rather than 2015–16 and that this timing has been reflected in the department’s forward evaluation plan. The department further advised ANAO that:

A robust measurement and verification regime has been introduced with more than 70 different metrics from completed projects and that due diligence is being undertaken on reported data⁷⁷ with a view to preparing a detailed evidence base for the evaluation.

As at 16 September 2014, 192 projects have been finished (this does not include projects that have been terminated). After measurement and verification it is estimated that these projects will abate over 1,730 kilotonnes of carbon which exceeds and is generally consistent with what was recommended by the Committee and approved by the Program Delegate (1,713 kilotonnes).

Program Management undertakes a thorough due diligence process. It also undertakes measurement and verification training with the State Office network.

Program Management review each finalised project and compares the KPIs at the post-project stage with that approved by the Committee.

2.67 As outlined at paragraph 1.16, the audit scope did not include the department’s measurement and verification regime for completed projects.

Program performance indicator

2.68 The key performance indicator (KPI) reported in the department’s 2012–13 Portfolio Budget Statements was the ‘proportion of companies assisted under the Clean Technology Investment Programs reporting projects with a minimum five per cent reduction in carbon intensity’. In July 2014, the department advised ANAO that this target:

• was developed in consultation with the then Department of Resources, Energy and Tourism;
• was selected because it aligned with the then Government’s plans to reduce carbon pollution by five per cent from 2000 levels by 2020; and
• reflected the outcomes of the Energy Efficiency Opportunities Program.

2.69 This KPI was appropriate in the context of the primary merit criterion, which was ‘the extent of the reduction in carbon emissions intensity, including through improvements in energy efficiency arising, from the project’. However, it was not consistent with the then Government’s broader policy objective to reduce carbon pollution by five per cent from 2000 levels by 2020. Specifically, total carbon emissions had increased by 13.2 per cent from 2000 to 2010, as demonstrated by Figure 2.4, but the starting point for calculating the reduction in carbon emissions under the funding agreements was not based on 2000 levels. As a consequence of that change, a reduction of 16 per cent from 2010 emissions levels would have been required to be consistent with the broader policy target set by the then Government.

Figure 2.4: Total carbon emissions produced in Australia, 2000–2010

Source: The World Bank.

2.70 After ANAO pointed out that the program’s KPI was not consistent with the then Government’s broader policy objective to reduce carbon pollution by five per cent from 2000 levels by 2020, the department advised ANAO in October 2014 that:

It was never the intent to directly link the program to the Government’s target. Rather, the program indirectly supported the achievement of the Government’s policy objectives through helping manufacturers invest in low pollution technologies.

2.71 In November 2014, the department further advised ANAO in relation to the program KPI that:

The Department determined the KPI of 5 per cent based on the available evidence base at the time. Consultations with Department of Climate Change and Energy Efficiency (as policy owner of the overarching initiative) and Department of Resources, Energy and Tourism indicated that the most relevant available data set was provided by the Energy Efficiency Opportunities Program. The Department drew on analysis from the Program Report Continuing Opportunities 2011. [The report] notes ‘Corporations in the EEO program have combined adopted energy savings equivalent to 5.4 per cent of the energy assessed to date, compared to 5.1 per cent in 2010, 4.5 per cent in 2009 and 4.2 per cent in 2008.’ The Department considered this target to be conservative noting that the EEO customer cohort are typically large emissions intensive businesses. The Department had no evidence base to confirm the likely emissions reductions that might be achieved by smaller less emissions intensive businesses.

Reporting on performance

2.72 In its 2012–13 Annual Report, the department stated that 98.7 per cent of companies assisted under the programs reported a minimum five per cent reduction in carbon emissions intensity. In relation to performance reported under the programs, the department advised ANAO in July 2014 that:

This figure [98.7%] was calculated using the number of applications approved in 2012–13 from both the Clean Technology Investment Program and Clean Technology Food and Foundries Investment Program. Out of 476 approved projects that accepted the offer, 6 projects predicted less than 5% reduction in carbon intensity.

2.73 However, the expected outcomes of projects were not: recorded in the IA committee recommendation (as discussed in paragraph 5.14) or the decision made by the delegate; or consistently recorded in funding agreements. In this respect, the department advised ANAO in September 2014 that:

In any instance where the Committee considered a reframed project, as part of recording the Committee’s recommendation AusIndustry also recorded the agreed carbon metrics that formed the basis for that decision.

2.74 ANAO examined the spreadsheet of agreed carbon metrics (referred to in this audit report as the expected outcomes of the projects) and compared this data with outcomes listed in funding agreements. This analysis revealed that:

• the agreed metrics provided by the department did not match the metrics stated in 72 funding agreements (13 per cent of executed funding agreements) with a total value of $62.2 million. The average difference between these metrics was seven per cent;
• expected outcomes, in terms of the reduction in carbon emissions intensity, were not included in 34 funding agreements (six per cent of executed funding agreements) with a total value of $21.6 million;
• for projects that were reframed, the carbon metrics listed in the spreadsheet were not identified as having been revised; and
• carbon metrics were entered without a clear explanation of the source of that metric (the possible sources included the application form, the departmental assessment or the IA committee assessment).

2.75 In addition, the KPI aggregates a number of different emissions intensity reductions that are not measured in the same way. Specifically, as discussed in paragraphs 4.9 to 4.13, the expected reduction in carbon emissions intensity could be calculated using an equipment, process or site-wide boundary, with the selected boundary influencing the expected outcomes of projects. As a result, the department is not well placed to measure the outcomes of the programs using this KPI.

2.76 Similarly, in an earlier audit of another program that was implemented by the department, ANAO found that the indicators did not provide insights into the continuing performance of the program, including measuring the program’s broader impacts and outcomes.⁷⁸ The department accepted the resulting recommendation from that audit that it assess the long-term performance of the program and report against relevant KPIs.

Conclusion

2.77 A range of key program documentation was developed by the department that was informed by extensive stakeholder consultation. While this documentation provided a sound overall foundation for implementation of the programs:

• not all of the program documentation clearly identified that the programs were to be focused on reducing carbon emissions (rather than assisting entities to maintain their competitiveness);
• there were multiple reference points for applicants seeking information about the programs, including separate but related program guidelines and customer guidelines, but only the program guidelines were approved in accordance with the grant program approval requirements; and
• there would have been benefits in a probity plan being developed.

2.78 Further, the assessment and selection method identified in the program guidelines was inconsistent with the approach approved by the then Government, which had referred to a competitive grants program. In this regard, the assessment and selection process that was implemented reflected elements of a merit-based, non-competitive program as well as a demand-driven program. In particular, the programs were not implemented in a way that applications competed for the available funding. Rather, so long as they were assessed as eligible and as having some merit, and sufficient program funding remained available, they were approved for funding.

2.79 The decision to establish the program was based on advice that funding would not be provided for projects that were intended to be undertaken privately in the absence of the programs. However, there were no mechanisms in place to prevent the approval of funding for an application that an applicant had committed to or was largely complete at the time of application. In this respect, there were 134 funding agreements with grants totalling $45 million where the funding agreement was signed either after the reported project end date, or within three months of the reported project end date.

2.80 The department provided committee members with an assessment template to facilitate the assessment of applications, however, the only records retained by the department were the final merit score and recommendation. In this context, an IA committee chair has advised ANAO that, in deciding which applications to recommend, the committee took into account a range of objective and subjective matters which were not reflected in the records of the assessment process. In the absence of records being made and retained that reflected the assessment of applications against the published criteria and the additional matters considered by the IA committees, the basis for the funding recommendations made to the program delegate was not evident. Of particular note was that, for 58 per cent of the applications that were assessed by the IA committees, the documentation available did not provide a clear basis for the recommendations that were made.⁷⁹

2.81 Performance reporting for the programs to date has aggregated the number of projects with an expected reduction in carbon emissions intensity of at least five per cent. This approach however, does not reflect actual outcomes, but expected outcomes (which, in turn, could relate to one of three different approaches to calculating emissions intensity reductions⁸⁰). In addition, the five per cent target set for the reduction in carbon emissions intensity did not reflect that the outcome set in the Clean Energy Future Plan to reduce emissions by five per cent was from year 2000 levels. Specifically, based on 2010 emissions levels, the reduction required was in the order of 16 per cent.

Recommendation No.1

2.82 To improve the design of, and governance arrangements for, future grant programs, ANAO recommends that the Department of Industry:

1. develops a single set of program guidelines that is approved in accordance with the grant program approval requirements;
2. includes, as an eligibility criterion, a requirement that excludes projects that are largely complete, or would otherwise proceed, without Australian Government funding, in circumstances where government intends not to fund such projects;
3. ensures that the basis for recommendations to the program delegate is appropriately documented, with documentation retained by the department; and
4. develops performance indicators that align with broader government policy outcomes.

Department of Industry’s response:

2.83 Part (a): Agreed in-principle. For future grant programmes the Department of Industry will consider this aspect as part of programme design.

2.84 Part (b): Agreed. The department notes in adopting such a criterion that this would not exclude circumstances where value for money can be achieved by bringing forward the timing of the funded activity where this is consistent with the policy intent.

2.85 Part (c): Agreed.

2.86 Part (d): Agreed.

Introduction

3.1 Effective grants administration is supported by agencies adopting application and assessment processes that promote open, transparent and equitable access to grants.⁸¹ It is also recognised as good practice for agencies to design a grant application process that is cost-effective, accessible and likely to maximise the attraction and selection of high quality applications.

3.2 In relation to the ability of applicants to access funding, the programs were characterised by:

• a high proportion of applications (84 per cent) proceeding through the eligibility checking stage to merit assessment;
• a high proportion of applications (74 per cent) that proceeded to merit assessment being recommended and approved for funding; and
• the 2013-14 Budget bringing forward $160 million in program funding from 2015-16 and 2016-17 to 2014-15 so as ‘to allow industry to access support for clean energy investment and research and development projects earlier and to more closely align funding with anticipated demand’.⁸²

3.3 In this context, the ANAO examined the:

• 1012 applications that were assessed against the eligibility criteria; and
• 849 applications that were assessed by the IA committees against the merit criteria.

Eligibility assessment

3.4 The department received 1171 applications for funding, of which 159 were not processed due to the closure of the programs in October 2013.⁸³ The remaining 1012 applications proceeded to the eligibility checking stage.

3.5 As noted in paragraph 2.36, the program delegate was responsible for determining whether an application was eligible for merit assessment. In making this decision, the program guidelines required that the delegate consider whether the:

• applicant was an eligible applicant;
• project to which the application related was an eligible project and involved eligible activities;
• application was complete and contained sufficient information to undertake a merit assessment; and
• application was in the form required.

3.6 The responsibility for eligibility assessment was delegated to departmental officers across the AusIndustry State Office Network. The assessors used an eligibility checklist that reflected the main eligibility requirements that were outlined in the program guidelines.⁸⁴

3.7 Relatively few applications (61, or six per cent) were identified as not meeting these eligibility requirements. However, for two per cent of applications assessed, completed checklists to support the results of the eligibility checking stage were not available.⁸⁵ The checklists for the remaining 938 applications were largely complete, but a relatively small number contained some shortcomings that did not provide an accurate assessment of eligibility as the checklists:

• did not identify whether the application was eligible and complete for 27 applications (13 applications were approved, 12 were not approved and two were withdrawn following eligibility assessment);
• were marked ineligible and/or incomplete for 18 applications that proceeded to merit assessment (10 applications were approved and eight were not approved); and
• were marked eligible and complete for three applications that were assessed as being ineligible.

3.8 These errors occurred despite the department developing a procedure that each eligibility checklist be reviewed by a more senior official from AusIndustry’s State Office Network.

3.9 Following the eligibility assessment process, the department’s procedures required that applicants be provided with written advice that their application was either: ineligible; incomplete; or eligible. Given the potential for applicants to revise and/or resubmit applications at any time, it was important for accurate feedback to be provided in correspondence for ineligible and incomplete applications. In this respect, the applicant for each of the 61 applications assessed as ineligible received a letter advising them of the reasons why their application was ineligible.

Approach taken to incomplete applications

3.10 Determining whether additional information will be sought or accepted from applicants to assist in the assessment process should be based upon appropriate consideration that weighs any risks to maintaining equity and probity against the need to ensure funding decisions are appropriately informed.⁸⁶ While the department adopted an application process that allowed applicants to submit applications at any time, the customer guidelines for the programs identified that incomplete applications would not be assessed, noting:

Your application is the sole source of information available to AusIndustry for the assessment of the eligibility and merit of your application. No further written or oral explanation, or further documentation, should be required. Before you submit an application, check to ensure all information is complete and accurate. An application that is incomplete will not be assessed.

3.11 In practice, if an application was found to be incomplete during the eligibility assessment, the applicant would receive a letter or email detailing the information required to complete the application. In this respect, the procedures manual outlined that:

If there is a minor omission or oversight in an application, a line manager may agree to provide the applicant with a brief opportunity to address this, and subject to this being done, may deem the application complete as at the date of receipt. Alternatively an application may be deemed complete at a later date where it takes more time for an applicant to provide all mandatory data and attachments.

3.12 However, a ‘minor omission’ was not defined or illustrated in the procedures manual. In June 2014, the department advised ANAO that:

AusIndustry CSMs [Customer Service Managers] have a significant level of experience in program delivery. Our culture is to assist businesses to access our programs where possible. Typically where applications are incomplete, we would contact applicants and give them an opportunity to provide the outstanding information. Generally speaking only in those circumstances where customers did not address information gaps as requested, would applications be formally determined to be incomplete and therefore ineligible for merit assessment.

3.13 The department also implemented processes relating to additional information requests that did not promote equitable access to funding. For example, the department advised ANAO in June 2014 that:

During the high application volume period of the program, guidance was provided to the network in which streamlined assessment processes were recommended for small grants [less than $300 000].

3.14 The process, referred to by the department, resulted in applicants who were seeking less than $300 000 in grant funds not being afforded the same opportunity to provide additional information as applicants seeking more than $300 000 in funds. In this respect, the departmental assessors were advised to not engage the applicant in extensive information requests, but rather seek the minimum level of information needed to complete the assessment and provide the applicant with a specified response date after which ’AusIndustry will complete its due diligence with the information available and forward it to [the] Committee for merit assessment and Delegate decision‘. It is not clear from the assessment records how many applications were assessed using this process. However, 245 applications that sought less than $300 000 in grant funding were submitted, merit assessed and considered by the program delegate after this guidance was issued to the department. Of these applications, 69 per cent were approved by the delegate. This is slightly lower than the overall rate of approval of applications, which, as discussed in paragraph 2.33, was 74 per cent.

3.15 In September 2014, the department advised ANAO that its streamlined processes for projects seeking less than $300 000 in grant funds was designed to encourage staff to ‘limit their due diligence in accordance with the complexity and risk of applications’. However, for the wider group of applications, the department’s due diligence went beyond simply clarifying and confirming the information presented in each application and extended to initiating and promoting changes to projects that were unlikely to be funded under the programs in their original form.⁸⁷ In this context, the approach of seeking to limit departmental assistance to some applicants and not others was inequitable, particularly when considered against the advice given in the customer guidelines that incomplete applications would not be considered.

Reframing of applications by the department and Innovation Australia

3.16 In addition to the 61 applications identified as ineligible, the grant amount was changed by the applicant, department or IA committee during the assessment process for a further 215 applications.⁸⁸ Of particular note in this respect was that:

• a number of applications were reframed to exclude ineligible expenditure items⁸⁹ with 83 applications reframed during the departmental assessment and five applications reframed during the IA committee assessment⁹⁰;
• applications were also reframed to exclude eligible expenditure items that did not provide value for money with 56 applications reframed to exclude eligible items during the departmental assessment and 24 reframed during the IA committee assessment.⁹¹ In this regard, the department made changes to application forms in December 2012 to require further information on the individual components of projects with multiple emissions reduction measures (including the costs and predicted electricity, fuel or direct carbon emissions abated by each measure)⁹²; and
• 11 applications were reframed so that the applicant would receive the maximum amount of funding available under the grant ratios (all of these changes were made during the departmental assessment).⁹³

3.17 Other reasons for reframing applications included changes to project costs and to projects in instances where the application did not provide sufficient evidence to support the claims made in relation to a component of the project (where that component was separable). A clear reason for changing the grant amount was not documented in respect to 13 of the reframed applications.

3.18 The department’s perspective on the reframing of applications was outlined to ANAO in October 2014, as follows:

The Department acknowledges that the approach adopted is not suitable for all granting programs, but was considered appropriate for the Clean Technology Programs which involved investing in capital equipment. Applications and assessments contained objective information on the capital equipment, for example manufacturing specifications, energy audits and utility bills. This enabled individual emission reduction measures to be easily dissected for analysis.

A ‘reframed project’ refers to a class of projects where the original scope of the project as submitted in the application was: changed by the Committee during the merit assessment process; or changed by the applicant, or by the Customer Service Manager in consultation and full agreement with the applicant, during the due diligence process before merit assessment by the Committee.

Many of the projects included a number of different capital investment activities, often unrelated. Part of Innovation Australia’s consideration of application competitiveness related to consideration of whether the relative contribution to carbon savings of each of the proposed capital investment activities was commensurate with the share of project costs. In a number of instances, Innovation Australia elected to recommend support of a subset of the project activities. As part of the assessment process, and at the request of Innovation Australia, the Customer Service Manager would identify uncompetitive emissions reduction measures elements of the project and highlight these in the assessment materials. In other instances, following advice from a Customer Service Manager, a customer might elect to remove an emissions reduction measure.

The level of analysis undertaken by the Department was commensurate with the size and complexity and risk profile of the project. Typically projects with multiple emission reduction measures required further analysis. This should not imply the assessor took a more active approach, but rather were undertaking normal due diligence.

Applicants were not required to submit a new application form but may be asked to provide additional supporting information. This is consistent with a business focused approach, reduces the burden upon businesses seeking to apply to the program, and was aimed at getting the best outcomes for the program. Any changes to projects undertaken at the request of the customer were documented in the assessment materials and dealt with in the merit assessment process and recorded in the decision outcome.

3.19 As indicated by the department’s advice, the level of applicant involvement in reframing projects varied, with some projects reframed by, or in consultation with, departmental officials (discussed below in paragraphs 3.21 and 3.22) and others reframed by IA committees without consultation with the applicant. In this context:

• the reframing of applications in the manner outlined by the department was not clearly set out in the program guidelines;
• a clear distinction was not maintained between the department’s and/or committee’s changes and the applicant’s changes, or the initial and final applications;
• assessment records did not clearly outline the reframing advice that was provided, when that advice was provided and when the changes were made; and
• the department’s reframing of applications was not accepted by all applicants, with the substantial reframing of the project given as one of the reasons for rejecting a funding offer in at least five instances. For these applications, the average reduction in the grant amount, from application submission to funding offer, was $220 142 (33 per cent of that sought).

3.20 While departmental advice was provided with the intention of assisting applicants, there are challenges in maintaining the objectivity of the assessment process and with treating applicants equitably when this assistance is provided as part of the assessment of applications. The preferred approach, to preserve the integrity of the assessment process, is to separate the provision of advice from the assessment process, other than clarifying minor inconsistencies or omissions that are not significant (noting it is common for program guidelines to outline that incomplete applications will not be assessed).⁹⁴ The sections following highlight some of the challenges where reframing occurs as part of the assessment process.

Departmental merit assessment

3.21 The aim of the departmental merit assessment was to ‘provide value adding commentary to assist IA and the program delegate to decide which applications should be funded under the program’. However, the assessment process extended beyond simply analysing the application information originally submitted by applicants. For at least 22 per cent of the departmental merit assessments, assessors took a more active approach to reframing applications by presenting scores, to the IA committee, that were based on a reframed project (or multiple possible variations to a reframed project) with modified project activities, expenditure and assumptions.

3.22 For the applications that were reframed, the departmental assessment report included:

• variations to project scope, such as removing elements of a project that were considered to be ineligible expenditure or not considered to provide ‘value for money’ in terms of carbon abatement, as discussed in paragraph 3.16;
• alternative scores calculated using various interpretations of the evidence provided in support of carbon savings, such as a reduction in the expected production levels; and
• a revised grant amount.

Amending the cost of carbon abated

3.23 Consistent with the department’s advice that its culture is to ‘assist businesses to access our programs where possible’, the department worked with the applicant to reframe the scope of projects that were ‘unlikely to be supported’ in their original form.⁹⁵ This approach blurred the line between departmental officials assessing applications for the purpose of informing committee recommendations and officials acting as advocates for projects.⁹⁶

3.24 One of the ways in which the prospect of obtaining funding for an application was improved was by reducing the fiscal cost of abatement, which was referred to, in the assessment of applications, as grant funds per tonne of carbon abated. However, the decision to reframe an application was subjective as the scoring framework did not contain a benchmark or upper limit on grant funds per tonne of carbon abated. In August 2012 (six months into the programs), the department advised its assessors that:

With input from the committee we are starting to gather a pool of knowledge about how this criterion is being addressed by applicants. The current data would suggest that an estimated $ per tonne calculation that exceeds $80 is unlikely to represent value for money as savings of this magnitude are not commensurate with the level of investment.

3.25 In the August 2012 guidance, departmental assessors were also advised to consider how the ‘competitiveness’ of the proposal could be improved through the removal of project costs from the application, with acceptance by the applicant, as follows:

Where the figure is in this order, [Customer Service Managers] CSMs should critically review the project budget to see if the applicant has included costs that are not linked directly to generating carbon or energy emissions savings. The applicant should be advised to remove these costs and submit a revised budget, otherwise the application is unlikely to be competitive. It is important that the assessment includes your analysis of the various components of the application and the interactions the CSM has had with the applicant in these circumstances. Following discussion, if the estimated $ per tonne calculation still exceeds $80 and the applicant wishes to proceed, the CSM should give careful consideration as to whether the application should be recommended for support. The assessment will need to document the rationale behind the decision.

3.26 Further guidance was issued to assessors in December 2012 noting that for ‘uncompetitive applications’ departmental assessors should:

• remove activities that do not directly contribute to energy and carbon savings or do not represent value for money;
• if still uncompetitive, look to see how far it is away from similar projects approved; and
• include in the assessment that the company would be prepared to accept a reduced grant amount.

3.27 Eligible activities could be an essential component of the project, but separable in circumstances where grant funds per tonne of carbon abated was considered, by the department, to be uncompetitive. This approach was reflected in guidance to assessors in March 2013, which noted:

Applicants can improve the competitiveness of their project by removing any eligible activities that do not directly contribute to significant carbon savings. For example site preparation activities which are for the dominant purpose of supporting implementation of the emissions reduction measure can be eligible activities. However, such activities do not directly contribute to carbon and energy savings.

3.28 Specifically, if an application was considered to be ‘uncompetitive’, the program management area of the department advised assessors to remove project costs in the first instance⁹⁷ and to then consider reducing the amount of grant funding sought.⁹⁸ Further guidance on reducing the grant amount was provided to assessors in March 2013 and noted that the IA committees considered that:

the core of any project should have significant carbon and energy savings, and if the project overall is good value for money then a lower grant ratio is acceptable. However, if the project appears to be primarily about productivity gains and energy and carbon savings are only marginal, then a reduced grant ratio to achieve a good Merit Criterion 1 score is not likely to be acceptable.⁹⁹

3.29 In this respect, there were:

• 34 applications that were reframed to improve the competitiveness of the project and were approved by the program delegate; and
• 22 applications that were reframed to improve the competitiveness of the project, but were not approved by the program delegate.

3.30 Of the 34 applications that were approved, there were at least four cases in which integral components, such as pre-project activities, were excluded from the project and two cases in which the grant amount was reduced to an acceptable amount. However, there were also applications for which it was not clear whether the component that was excluded was integral to the project. In these cases, the assessor’s commentary was limited to the fact that the component did not provide a ‘competitive dollars of grant funding per tonne of carbon abated’ or did not contribute substantially to carbon savings.

3.31 As discussed in paragraph 3.24, any judgement about whether a project provided ‘good value for money’ was necessarily subjective in the absence of well-founded benchmarks. As a result, the success of an application was ultimately dependent on the skills and experience of the departmental assessor.

3.32 Considering that some applications were reframed because they were identified by the department as not likely to be ‘competitive’ if the assessment was based on the application that had been submitted, the changes that were made to these applications by departmental officials increased the likelihood that applications were funded. However, the approach adopted by the department was not consistently applied, highlighting the challenges to the department in achieving equitable outcomes when providing advice as part of the assessment of applications.

Innovation Australia Committee merit assessment

3.33 The published program guidelines provided that IA must undertake a formal merit assessment of all eligible applications. As with many committee arrangements, this provided a mechanism to bring specific knowledge to the task of assessing the merits of applications. In fulfilling this role, the Board of IA delegated its authority for conducting merit assessments and preparing recommendations for the decision maker to the IA committees.¹⁰⁰

3.34 The active approach that was adopted by the departmental assessors in reframing applications was also adopted by the IA committees, with a least five per cent of applications considered by the IA committees reframed.¹⁰¹ In this respect, there were:

• 24 applications that were reframed to exclude eligible expenditure items that did not provide value for money;
• seven applications that were reframed, but no clear reason was given; and
• five applications that were reframed to exclude ineligible expenditure.

Probity of requesting additional information and reframing applications

3.35 In previous audits, ANAO has observed that, where a department has sought to amend a selection process, probity advice has generally recommended that assessment and selection documentation be amended to provide for a transparent resubmission process. This includes allowing all applications or a subset of applications that have been assessed and shortlisted as meeting primary program objectives (as reflected by all or the most highly weighted merit criteria) to revise and resubmit information.

3.36 For example, in relation to the Building Better Regional Cities Program (see ANAO Audit Report No. 25 2013–14), an additional step was added to the application and assessment process to allow all applications in a particular cohort to be revised and re-submitted.¹⁰² In that program, the administering department, in consultation with its probity adviser, decided that an addendum should be added to the Assessment and Selection Plan for the program to allow a resubmission process to be employed for all shortlisted applications. Applicants were informed as to those elements of their application that they were being invited to re-submit, with departmental preferences in terms of those responses that would be more favourably considered in the merit assessment process clearly identified to each applicant that was invited to re-submit.

3.37 As noted at paragraph 2.12, the department did not develop a probity plan or engage a probity advisor for the programs. Further, the department did not establish a similar approach to that observed in respect to the Building Better Regional Cities Program, or similar arrangements, for seeking additional information or reframing applications to the programs. In particular, the program guidelines did not identify the circumstances under which the department or the IA committees could seek additional information from applicants¹⁰³, including by soliciting improved proposals with respect to one or more merit criteria. Such an approach was only adopted at the very end of the program in relation to one particularly significant government policy change.¹⁰⁴

3.38 In September 2014, the department advised ANAO that it sees a distinction between the role of the department as a facilitator to program access and IA committees as assessors of merit. Specifically, the department advised ANAO that:

A key part of the AusIndustry customer service function is to assist customers to access government programs. This extends to advising customers on what they can do to make it more likely that their application will be considered to be competitive. It is the role of Innovation Australia to consider applications against the merit criteria and make recommendations to the Program Delegate. This process ensures that only the most meritorious applications are recommended for funding. In the event that the Committee considered that the application was not sufficiently meritorious, it would not be recommended for funding. This is the appropriate process for ensuring that the Program outcomes are maximised.

3.39 Consistent with this advice, the departmental assessment process¹⁰⁵ focussed on increasing the likelihood that applications would be considered, by the IA committee, to be ‘competitive’ against the merit criterion. However, where an application as originally submitted was identified to have scored sufficiently well¹⁰⁶ that it was expected to be recommended for funding, the department did not examine whether reframing could lead to a more meritorious project being undertaken.

3.40 In addition, the distinction between the departmental assessment and the IA committee assessment, referred to in paragraph 3.38, was not maintained in the administrative arrangements for the programs, with the IA committees also reframing applications to improve the merit of projects.

Grants above $10 million

3.41 The program guidelines assigned a role to the Cabinet of the Australian Government in considering applications for grants of $10 million or more that were recommended for funding by the IA committees. Such applications needed to demonstrate:

to the satisfaction of the Cabinet of the Australian Government, that there will be a superior level of benefit to the Australian economy.

3.42 In total three applications were considered by Cabinet, with two applications following the process outlined in the guidelines. A third application seeking funding of $23.0 million was part way through the IA committee assessment process when the decision was taken to fund the project outside of the program using a one-off discretionary grant. Specifically, the application was assessed and recommended by the department’s assessors and was in the process of being considered by an IA committee. The IA committee had deferred further consideration of the application on 19 June 2013, subject to the provision of a substantive volume of additional information from the applicant including:

• financial projections for the project;
• information on company governance changes;
• a detailed project plan for bringing one of the applicant’s products to market;
• a commissioning plan for the applicant’s biodigester plant; and
• information on the results of a pilot project.

3.43 A briefing prepared in November 2013 identified that on the same day as the Committee’s decision to request additional information (19 June 2013), the department advised its Minister’s Office that the IA committee had deferred consideration of the application. On the following day (20 June 2013), the Minister’s Office advised the department that it had decided to present the project for Cabinet consideration.

3.44 Cabinet considered and agreed to fund the project on 24 June 2013 subject to conditions that replicated the information requested by the IA committee. This incorporated a decision to fund the project as a new policy proposal outside of the programs, but with the $23.0 million in funding required to be offset from the programs.

3.45 In November 2014 the department advised ANAO that:

• the funding agreement included a number of conditions the company must satisfy before payments are made and the company has provided information in relation to all the conditions;
• the company has submitted a request to vary the funding agreement proposing a change to the technology;
• the department was undertaking due diligence on the information provided with a view to putting a recommendation to the program delegate shortly; and
• no payments had been made to date.

Conclusion

3.46 The process that was established by the department for assessing the eligibility and merit of applications was focussed on helping businesses access funding under the programs. However, the level of assistance provided does not sit comfortably with the operation of a competitive grants program under the Australian Government’s grants administration framework. Rather than asking applicants to resubmit a reframed application, it was common for those responsible for assessing applications to:

• permit some¹⁰⁷ applications that had been identified as incomplete to proceed to departmental merit assessment¹⁰⁸; and
• reframe the project activities, expenditure and/or underlying assumptions for some applications to improve the application’s assessed merit in terms of the published criteria. The focus was on assisting applicants whose application was otherwise unlikely to score well enough to be recommended for approval, rather than seeking to reframe all applications where the department considered the applicant could have proposed a more meritorious approach (as assessed against the published criteria).

3.47 The approach adopted for the programs went well beyond clarifying information included in applications and seeking to address any minor information missing from the application. More broadly, combining advisory and assessment roles is an approach not well suited to maintaining an objective assessment of competing applications. In this context, where government decides that an advisory role should be performed in addition to the assessment of applications, it is preferable that a clear separation be maintained between the roles so as to maintain the objectivity of the assessment stage. There are also challenges that arise in treating applicants equitably due to the risk that the level of assistance provided to applicants will vary.

Recommendation No.2

3.48 To promote equitable access to grant funding and objective assessment of competing grant applications, ANAO recommends that, where the Government decides that advisory assistance should be provided, the Department of Industry separate the provision of this assistance from the task of assessing applications.

Department of Industry’s response:

3.49 Agreed. For future granting programmes, the roles of officials and other delivery partners are to be clarified and publicly documented in relation to advisory and assessment activities.

Introduction

4.1 As the primary objective of the programs was to reduce the carbon emissions intensity of manufacturers, the assessment framework developed by the department to score applications heavily weighted merit criterion one.¹⁰⁹ In this respect, the department advised ANAO in June 2014 that:

The rationale behind the weightings of the merit criteria was to ensure that only those projects which delivered a significant reduction in carbon emissions intensity as outlined in the Guidelines could be funded. This reflects the Program objective. The weighting for this criterion ensured that applications which scored poorly against this criterion but performed very well against the remaining program criteria could not be funded. This was done at the direction of the Department of Prime Minister and Cabinet.

4.2 Accordingly, ANAO examined the:

• approach taken to assessing the extent of the reduction in carbon emissions intensity claimed in applications; and
• consistency of the assessment approach with the published merit criteria, objective of the programs and the original policy intent.

Scoring methodology developed for merit criterion one

4.3 The program guidelines described merit criterion one as ‘the extent of the reduction in carbon emissions intensity, including through improvements in energy efficiency arising from the proposed project’. Although carbon emissions intensity was not defined in the program guidelines, it was defined in the customer guidelines as the number of tonnes of greenhouse gases that are emitted for each unit of output produced by a manufacturer.¹¹⁰ Further, in the customer guidelines, the department communicated to prospective applicants that the reduction in carbon emissions intensity would be assessed using the:

• predicted percentage reduction in carbon emissions intensity following project implementation (indicator one); and
• total predicted carbon savings over the life of the conservation measure (indicator two).

4.4 The values for these indicators were to be calculated using the Carbon and Energy Savings Calculator (the calculator) provided on AusIndustry’s website. To generate these values, applicants were required to:

• identify the activities on a site (or multiple sites) that related to the manufacturing activities for which funding was sought, as opposed to other activities such as corporate management;
• select an activity boundary from which emissions were estimated—this could have been the whole or part of a manufacturing site;
• identify a period of time over which to measure baseline emissions intensity that was typical of the normal production process and did not include one-off or unusually large periods of activity;
• estimate future production levels (based on realistic sales forecasts) and associated energy use; and
• identify the effective life of the emissions reduction measure(s).

4.5 The scoring methodology developed by the department then allocated a score against merit criterion one using the two indicators. The score for indicator one was based on the percentage reduction in carbon emissions intensity, while the score for indicator two was based on grant funds per tonne of carbon abated (which was the total carbon savings over the life of the conservation measure relative to the amount of grant funds requested).¹¹¹ The contribution of these indicators to the merit criterion one score was dictated by the weightings provided in Table 4.1.

Table 4.1: Weightings for merit criterion one

Source: ANAO analysis of the procedures manual.

4.6 In June 2014, the department advised ANAO that the second indicator listed in the customer guidelines was assessed using grant funds per tonne of carbon abated, rather than total carbon savings over the life of the conservation measure, because:

The dollars per tonne calculation was simply a tool that allowed us to compare and consistently score total carbon savings over the life of the project. In the absence of this tool it would not be possible to objectively measure performance in terms of total carbon savings. Carbon savings varied enormously across applications depending on the size of the project and the nature of the emission reduction measure, ranging from 1064 kilotonnes to 0.35 kilotonnes.

4.7 This methodology was approved by the manufacturing policy area of the department in November 2011 and was based on advice from the Department of Climate Change and Energy Efficiency (DCCEE), the Department of Resources, Energy and Tourism, Low Carbon Australia and IA. The department advised ANAO in November 2014 that the customer guidelines were cleared in February 2012 by the then AusIndustry General Manager responsible for the programs after being reviewed by DCCEE and IA. However, the department was unable to identify to ANAO who made the decision not to include the fiscal cost of abatement in the customer guidelines.

Percentage reduction in carbon emissions intensity

4.8 The reduction in carbon emissions intensity is the predicted change in the tonnes of greenhouse gases emitted for each unit of output produced by a manufacturer once the proposed emissions reduction measure has been implemented. Due to the way that this reduction was calculated, departmental assessors were required to consider the assumptions that were made by applicants when using the calculator to estimate the expected outcomes of a project.

4.9 One of the assumptions made by applicants that had a significant impact on the predicted reduction in carbon emissions intensity was the selection of an activity boundary. The department recommended that applicants choose a site-wide energy baseline to easily align with the information available from utility bills, but also allowed applicants to choose smaller areas in their manufacturing site(s) to align with a process or a piece of equipment. Figure 4.1 shows the three types of boundaries allowed under the programs. For the 841 applications for which the activity boundary was identified in the departmental assessment, the most commonly used was a site-wide boundary.

Figure 4.1: Types of boundaries that could be selected for a project

Source: CTIP and CTFFIP customer guidelines.

4.10 In November 2013, the department noted in a review of the programs that:

The percentage reduction is highly dependent on the project boundary being nominated by the applicant, with a smaller boundary resulting in a higher percentage reduction. This led to inconsistencies and meant that applicants with projects that delivered the same amount of carbon savings could be scored differently depending on the project boundary selected.

4.11 For example, one applicant that received funding under the programs initially used a site-wide boundary to generate a reduction in carbon emissions intensity of 4.1 per cent. This level of reduction would not have been sufficient for the project to be recommended for funding without changes to the application. During the departmental assessment, the application was reframed by changing the boundary to a process boundary, which was calculated to deliver a reduction in carbon emissions intensity of 71 per cent (and therefore increased the assessment score to a level at which the project could be recommended for funding) despite the project not having changed.

4.12 In respect of the 849 merit assessed applications:

• a change in the boundary was recorded in the departmental assessment for eleven applications (five of these applications were funded for a total value of $462 033); and
• an increase in the predicted percentage reduction in carbon emissions intensity was recorded for a further 102 applications (71 of these applications were funded for a total value of $44.3 million). The average increase for these applications was 12 per cent.¹¹²

4.13 The approach of allowing applicants to select more narrowly defined boundaries was adopted not to reflect a change in the scope of the project or to improve the carbon emissions intensity reduction outcomes expected from projects but, rather, with the intention of improving their scoring against the merit criteria so as to maximise the amount of funding being paid to industry. In this context, where the boundary used for assessment purposes does not incorporate all the emissions associated with producing the manufacturer’s output, the extent of the reduction in carbon emissions intensity associated with a project has been overstated so that the project can appear to have greater merit under the programs and, therefore, be awarded grant funding.

4.14 Another factor that influenced the reduction in carbon emissions intensity (indicator one) was the size of existing emissions levels. For example, the department estimated that a 40 kilowatt photovoltaic (PV) system can reduce an applicant’s electricity consumption from the grid by 58 400 kilowatt hours (kWh) per annum. This correlates to a 10 per cent reduction for an applicant that consumes 584 000 kWh of electricity per annum, but only a five per cent reduction for an applicant that consumes 1 168 000 kWh of electricity per annum.

Total carbon savings over the life of the conservation measure

4.15 As previously noted, indicator two (as identified in the customer guidelines) was generated using the calculator. Advice provided to departmental staff by the program management area of the department in February 2012 noted that total carbon savings were included as a second indicator because:

An absolute measure means that large manufacturers won’t be penalised for making small percentage improvements, as these small improvements can still yield large carbon savings.

4.16 In October 2014, the department provided further advice to ANAO that the two indicators measured the short-term and long-term extent of the reduction in carbon emissions intensity. Specifically:

Indicator 1 provided a relative measure that demonstrated the immediate impact of a project compared to the customer’s existing manufacturing processes. This was done by measuring the percentage reduction in emissions intensity in the first year following project completion.

Indicator 2 provided a more long-term representation of the impacts of the reduction in emissions intensity to be delivered by the project. This indicator was based on Indicator 1, but further incorporated the effective life of the emissions reduction measure and the manufacturer’s expected production levels during this period (e.g. the next 10 years). This long-term view was achieved by representing the reduction in emissions intensity in terms of the carbon savings over the life of the conservation measure.

Where a project involved electricity savings, Indicator 2 also accounted for the impact of varying state and territory electricity emissions factors on the emissions intensity reduction to be delivered by the project. This provided a more complete picture of the emissions intensity reductions associated with each project at a national level. Projects that reduced grid electricity consumption from emissions-intensive grids (e.g. Victoria, with electricity generated from brown coal) were recognised as delivering higher reductions in emissions intensity.

4.17 In this respect, the reduction in carbon emissions intensity indicator reflected the short-term impacts of the project (generally 12 months), while the total carbon savings over the life of the conservation measure measured the longer-term impacts (from 10 years to 100 years). By combining the two indicators under the assessment of merit criterion one, the assessment of the primary merit criterion was inconsistent with the policy proposal for the programs. Specifically, the policy proposal identified that the programs were designed to deliver ‘transitional, targeted assistance for manufacturing companies’ that ‘will assist in achieving the objective of the carbon price by helping these businesses to reduce their energy consumption in the short-term while remaining competitive in Australia’. The combination of the indicator scores resulted in a score which reflected a mixture of short- and long-term benefits.

4.18 However, as discussed in paragraph 4.5, indicator two was replaced, in assessing applications against merit criterion one, by grant funds per tonne of carbon abated, which is a relative cost-effectiveness measure. In other grant programs audited by ANAO where the cost per unit is an important consideration in the award of funding, agencies have included a separate value for money/cost effectiveness merit criterion.¹¹³ This approach provides greater clarity to applicants as to how the merit of their application will be assessed, and promotes a clear line of sight between the program objective and the key policy criterion or criteria.

4.19 The department used the cost-effectiveness indicator to assess applications against merit criterion one from the start of the programs, but did not communicate this to applicants until the fifth version of the customer guidelines (December 2012). In this version of the customer guidelines, rather than identifying the cost-effectiveness indicator as the second indicator used in the assessment of applications, the department identified this indicator as a factor that was taken into account. In June 2014, the department advised ANAO that:

We took the view early in the program not to reference dollars per tonne in the customer documentation as we felt that this might drive applicants to artificially adjust their projects to achieve a “competitive” dollar per tonne figure. We were also cautious about managing customer messaging around what a competitive figure looked like. This is because there could be significant variation depending on the nature of the emissions reduction measure. However, due to customer feedback on this issue we included a clear reference to dollars per tonne in the customer guide published on 12 December 2012 and merit criteria fact sheet published on 16 October 2012.

4.20 In light of the approach taken to assess applications using different activity boundaries (as discussed in paragraph 4.10) IA placed more emphasis on grant funds per tonne of carbon abated when assessing applications against merit criterion one. In this context, the decision not to publish the second indicator used in assessing applications at the start of the programs reduced the transparency of the assessment process, particularly for those applicants who submitted an application prior to December 2012.

Implementation of the scoring methodology for merit criterion one

4.21 The department implemented the scoring methodology for merit criterion one using two rating scales. These rating scales, and the assessment procedures used by the department, evolved over the life of the programs. In this context, ANAO examined the implementation of the scoring methodology in respect to the two indicators used in assessing applications against this criterion, including the rating scales and the relative weightings that resulted from changes made to those rating scales during the life of the programs.

Predicted reduction in carbon emissions intensity

4.22 A maximum of 42 points was available for the predicted percentage reduction in carbon emissions intensity and, as shown in Table 4.2, there were two versions of the rating scale that the department applied to allocate a score for this indicator. The rating scale most frequently used by departmental assessors was the scale that was applied from June 2012 to July 2013. This scale was used to score over 80 per cent of applications.

Table 4.2: Indicator one rating scale: predicted reduction in carbon emissions intensity

Source: ANAO analysis of the department’s carbon scoring tool.

4.23 The second version of the rating scale was adopted following a meeting of the IA committee in May 2012 at which the committee members’ discussion was as follows:

[Committee member one] was concerned the current indicator 1 scoring is too “tough”. The applicant would only ever achieve the top score if they achieved 80% reduction which may be unrealistic… However, the 60/40 split between indicator 1 and 2 seemed fine.

[Committee member two] suggested we come up with a new scoring system to reward the really good applicants but not punish the ones that have already done a lot of energy efficiency measures.

[Committee member three] suggested a logarithmic scale for the score, and the committee members agreed.

4.24 As the discussion demonstrates, committee members considered that the reduction in carbon emissions intensity that could be achieved by different manufacturers would be driven by the extent to which those manufacturers had already implemented emissions reductions measures. This approach was reflected in the logarithmic scale developed to score the predicted percentage reduction in carbon emissions intensity (Figure 4.2).

Figure 4.2: The logarithmic scale for indicator one

Source: ANAO analysis of logarithmic scale used for indicator one that was developed by the Clean Technology Investment Committee and the department.

4.25 The use of such a scale, however, did not clearly differentiate between the degree of merit of each application because:

• the size of an emissions intensity reduction proposed by an applicant may have had no relationship with whether the applicant had previously implemented emissions reduction measures¹¹⁴;
• the emissions intensity reduction levels implicit in the scale were not based on evidence of the reduction in emissions that could be achieved by implementing specific emissions reduction measures and instead reflected the IA committee’s preference for rewarding large reductions while not penalising other applicants.¹¹⁵ In this regard, the department advised ANAO in November 2014 that, following consultation with industry, Low Carbon Australia and within government, it determined that there was no existing body of evidence that could provide a foundation for benchmarks or key performance indicators for emissions reduction measures in the manufacturing sector; and
• the scores allocated to each application did not demonstrate how much better the application was when compared to another. For example, the highest rating of 10 equated to an intensity reduction of at least 80 per cent, but an applicant could achieve half that rating (a score of 5) with a reduction of one eighth of the size (that is 10 per cent).¹¹⁶

Grant funds per tonne of carbon abated

4.26 A maximum of 28 points was available for the grant funds requested per tonne of carbon abated and, as shown in Table 4.3, there were three versions of the rating scale that the department applied to allocate a score for this indicator. The rating scale applicable from June 2012 to July 2013 was used for more than 80 per cent of applications.

Table 4.3: Indicator two rating scale: grant funds requested per tonne of carbon abated

Source: ANAO analysis of the department’s carbon scoring tool.

4.27 The application of a single fixed scale to determine the score allocated for grant funds per tonne of carbon abated also presented challenges in differentiating the degree of merit associated with applications. Specifically, the indicator value was primarily influenced by the amount of funding requested by the applicant, which was capped at:

• 50 per cent for applicants that:
  • had covered emissions from facility operations of 25 000 tonnes or more, but less than 100 000; or
  • were seeking less than $500 000 and had a turnover of less than $100 million in the last financial year;;
• 33 per cent for applicants that sought between $500 000 and $10 million or sought less than $500 000 and had a turnover of more than $100 million in the last financial year; or
• 25 per cent for applicants that sought more than $10 million.

4.28 In addition, grant funds per tonne of carbon abated was considered in comparison to similar emissions reduction measures and with reference to the carbon price. Following the then Government’s decision to bring forward the introduction of an emissions trading scheme in July 2013, the department advised the CTIC that the:

• reference point for value for money had changed from $23 to $6 per tonne;
• assessment of value for money needed to be ‘tougher’; and
• approval ratings were expected to go down significantly.

4.29 This advice prompted a committee member to identify that the changes to the assessment, in terms of value for money, would mean that certain types of emissions reduction measures would not be able to achieve a grant dollars per tonne of carbon abated that was acceptable under the programs. This was not, however, communicated to prospective applicants.

Relative weightings of indicators

4.30 As the department did not establish clear benchmarks in regard to the reduction in emissions intensity or the grant funds per tonne of carbon abated, ANAO examined the extent to which the scoring framework reflected:

• the published performance indicator of a reduction of five per cent in carbon emissions intensity; and
• applications seeking grant funds per tonne of carbon abated being more likely to receive funding if closer to the carbon price of $23.¹¹⁷

4.31 Table 4.4 shows that the nominated weightings of 60 per cent on indicator one and 40 per cent on indicator two were not consistently applied through the scoring methodology.

Table 4.4: Calculation of relative weightings in assessing applications against merit criterion one

Source: ANAO analysis of departmental records.

4.32 In particular, all of the 849 applications considered by the IA committees were scored using an approach that placed a higher relative weighting (55 per cent from the start of the programs to June 2012 and 64 per cent from June 2012 onwards) on indicator two.

4.33 In practice, the average scores of applications indicated that applications claimed a reduction in carbon emissions that was greater than five per cent, but grant funds per tonne of carbon abated was higher than the reference point (of $23 per tonne of carbon abated, if the applications were assessed between February 2012 and July 2013, or $6 per tonne if the applications were assessed after July 2013). The distribution of the scores allocated for merit criterion one is provided in Figure 4.3.

Figure 4.3: Scores allocated for the merit criterion one indicators

Source: ANAO analysis.

Merit score as a predictor of funding outcomes

4.34 The scoring methodology did not define a minimum reduction in carbon emissions intensity that was considered acceptable under the programs. For example, between June 2012 and July 2014 an application with a reduction in carbon emissions intensity of between two and four per cent and less than $10 in grant funding per tonne of carbon abated, would score 36.4 out of 70. In the context of merit criterion one only, the department would have considered this score sufficient to be approved for funding.

4.35 In addition, the department did not draw a distinction between whether an investment that generates a reduction in emissions intensity of between two and four per cent, for example, was considered to be a ‘business as usual’ investment or an investment in low emissions technology. In this respect, replacing a piece of equipment that is at the end of its effective life may result in a reduction in emissions intensity, but a greater reduction may be achieved if the fuel used in the manufacturing process is changed or a process is re-engineered so that it is more efficient. The predicted percentage reduction in carbon emissions intensity was:

• less than five per cent for eight successful applications with a total value of $797 872;
• between five and 10 per cent for 49 successful applications with a total value of $14 893 290;
• between 10 and 15 per cent for 70 successful applications with a total value of $17 997 758; and
• between 15 and 20 per cent for 61 successful applications with a total value of $27 690 168.¹¹⁸

4.36 Further, the inclusion of the value for money indicator in merit criterion one impacted on those applicants that were awarded funding, and those that were unsuccessful. Specifically, as shown in Figure 4.4, had the merit criterion one score solely related to each application’s assessed performance in terms of the predicted percentage reduction in carbon emissions intensity:

• 57 successful applications may not have been awarded funding, at a saving of $31 million (or an average of $545 000 for each application); and
• 126 unsuccessful applications may have been awarded funding at a cost of $61 million (or an average of $486 800 for each application).¹¹⁹

4.37 There were also 64 successful applications that received a score of 50 per cent against indicator one and were awarded $17 million in funding (or an average of $262 345 for each application).

4.38 This situation highlights that the implementation of the scoring methodology, using the rating scales for indicators one and two, assisted the department to increase the amount of financial assistance that was provided to manufacturers. However, the assessment scoring methodology did not ensure that only those projects which delivered a significant reduction in carbon emissions intensity could be funded. Such a result was at odds with the rationale behind the weightings of the merit criteria (see paragraph 4.1).

Figure 4.4: Scores allocated to applications for indicator one, merit criterion one and the overall merit score¹²⁰

Source: ANAO analysis.

Conclusion

4.39 It was originally expected that the programs would only fund projects that would deliver a significant reduction in carbon emissions intensity. However, more than 100 projects with low carbon emissions intensity reductions were awarded funding of $47.6 million.¹²¹ This situation was the result of the high weighting adopted for the merit criterion relating to reductions in carbon emissions intensity (merit criterion one) not being supported by other aspects of the design and implementation of the assessment process. Specifically a significant proportion of the score against this criterion did not relate to reductions in carbon emissions intensity, but an assessment of the grant funds per tonne of carbon abated. The inclusion of this indicator had a significant effect in that, had the merit criterion one score solely related to each application’s assessed performance in reducing carbon emissions intensity:

• 57 successful applications may not have been awarded funding, at a saving of $30.6 million (or an average of $536 400 for each application); and
• 126 unsuccessful applications may have been awarded funding at a cost of $61.3 million (or an average of $486 800 for each application).

4.40 A more robust and transparent approach in the context of the grants administration framework would have involved:

• developing a consistent, evidence-based approach to assessing applications and publicising that approach in the program guidelines;
• specifying a minimum score that an application must achieve against the first merit criterion in order to be approved for funding. This approach guards against applications scoring very highly against less important criteria being awarded funding notwithstanding a poor score against the most important criterion; and
• including the value for money/cost-effectiveness of projects as a separate merit criterion.

4.41 A further shortcoming related to the reframing of applications to improve their assessed merit irrespective as to whether there was any change in the underlying project proposed to be funded, in part, by the programs. Specifically, if an application was considered to be ‘uncompetitive’, the program management area of the department advised assessors to remove project costs in the first instance and to then consider reducing the amount of grant funding sought. Another approach was to change the way in which the estimated reduction in emissions intensity was calculated so as to increase the percentage reduction used for assessment purposes. These approaches assisted the department to increase the amount of funding awarded under the programs, but do not sit comfortably in the context of a program that was publicised as awarding funding through a competitive, merits-based process.

Recommendation No.3

4.42 In the administration of competitive, merit-based grants programs, ANAO recommends that the Department of Industry design, publicise and implement merit assessment scoring approaches that promote a clear alignment between the published program objective, the merit criteria, the weighting for those criteria and any scoring indicators.

Department of Industry’s response:

4.43 Agreed.

Introduction

5.1 The program delegate was responsible for making all funding decisions under the programs. Consistent with the delegate’s broader responsibilities under the Australian Government’s financial management legislation, the delegate also needed to satisfy the requirements of an ‘approver’ for the purposes of the FMA Regulations.¹²² In this context, ANAO examined the final merit assessment, recommendation and decision of the 814 applications that were considered by the delegate.¹²³

Innovation Australia committee assessment

5.2 In April 2012, the program management area of the department advised IA committee members that:

The purpose of the carbon scoring tool is to ensure that AusIndustry Customer Service Managers take a uniform approach to scoring against this criterion. Committee members may wish to score on some other basis.

For Merit Criterion 1, Committee Members are asked to give applicants a score out of 70. We are assuming that your score will reflect your level of confidence in the values for the indicators provided in the application and your consideration of the competitiveness of the values of those indicators and their relative importance in overall performance against this criterion.

5.3 In September 2014, the department advised ANAO that the process for the CTIC and CTFFIC’s consideration of applications included:

• committee members being asked to provide their scores against each of the merit criteria prior to meetings (scores were then collated and tabled in meetings using the process discussed in paragraph 5.6) ¹²⁴;
• two or more spokespersons (appointed prior to meetings) leading the discussion of each application by ‘typically’ giving a general overview of a project, followed by their assessment of the application against each of the merit criteria;
• committee members discussing the application, agreeing on whether to recommend the application for funding and providing a final committee score; and
• a discussion of any conditions to be placed on the recommendation and the identification of a reason if the committee was not recommending an application.

5.4 In support of this advice, the department provided agenda documents setting out this process as a proposed approach for conducting meetings, but was otherwise unable to substantiate the content of committee deliberations. As was noted in paragraphs 2.50 to 2.54, the retained records of CTIC and CTFFIC meetings (including meeting minutes and recommendation sheets) did not provide a basis for identifying the matters discussed in meetings.

5.5 Given that a large majority of applications were approved for funding, the absence of recorded reasons for recommending applications has led to a lack of transparency regarding the factors that were considered important in forming individual funding recommendations.¹²⁵ ANAO therefore examined the content of CTIC and CTFFIC meeting notes taken by departmental officers.¹²⁶ During the audit, the department had advised ANAO that these notes ‘were used to prepare the minutes and decisions which were cleared by the committee’, but subsequently advised ANAO that they do not ‘reflect a complete and accurate record of the meetings’.
Notwithstanding any limitations of the notes, they are the only records available to evidence committee discussions.¹²⁷ The notes indicate that the deliberations of the IA committees focussed on particular issues raised by committee members rather than the merits of each application against each criterion with:

• grant dollars per tonne of carbon abated discussed in 31 per cent of applications; and
• the reduction in carbon emissions intensity mentioned in respect to 15 per cent of applications.¹²⁸

Calculation of the total merit score

5.6 The department advised ANAO in October 2014 that the IA committees provided a total average score for their final decision in relation to each application. To generate the average score, two¹²⁹ spokespersons were nominated by the respective committees to score the application, based on the information provided in the departmental assessment report. The average score was then ‘moderated’ during the IA committee meeting, as stated in paragraph 5.3. A simplified version of this scoring process is provided in Table 5.1.

Table 5.1: Calculation of merit score by the CTIC

Source: ANAO analysis of departmental records.

5.7 The IA committee members’ scores in Table 5.1 reflected a consistent score for merit criterion two, but significant differences in scoring for the other criteria. In this context, the calculation of average scores for merit criterion one and three did not demonstrate that the members reached agreement on the reasons for the application demonstrating merit against the criterion.

5.8 In another example, an assessment report presented to the IA committee identified three possible variations of a project, including one scenario in which the grant amount would be reduced. The IA committee members in attendance at the meeting scored the application using at least two of the three possible project variations, with two committee members scoring the original application (using four merit criteria as the original grant amount requested was greater than $1.5 million) and three committee members scoring the reduced funding scenario (using three merit criteria as the reframed grant amount was less than $1.5 million). In this case, all scores, including the scores given against merit criterion four, were averaged to determine the overall merit score even though the final grant amount was less than $1.5 million.

5.9 The appropriateness of the scoring was documented as an ongoing concern of the IA committee members who, in February 2013 (a year into the programs), noted that it was difficult to ‘determine if projects are meritorious in carbon savings or financial savings’. These concerns were reflected in the recommendations made to the program delegate with 53 applications that scored over 60 per cent in aggregate against the merit criteria not recommended by the IA committees. The high number of applications that were not recommended, despite receiving a merit score of more than 60 per cent, was a direct consequence of an informal benchmarking approach that was used to consider the grant funds per tonne of carbon abated indicator jointly developed by the IA committee and the department.¹³⁰ Specifically, as discussed in paragraph 3.24, the department issued guidance to its assessors in August 2012 advising:

It was always envisaged that the extent of the reduction in carbon emissions intensity (merit criterion one) would be the most difficult to assess. However, to date 36 applications have been assessed and 30 applications worth $26.4 million have been approved under both programs. With input from the committee we are starting to gather a pool of knowledge about how this criterion is being addressed by applicants. The current data would suggest that an estimated $ per tonne calculation that exceeds $80 is unlikely to represent value for money as savings of this magnitude are not commensurate with the level of investment.

5.10 Further, following the then Government’s decision to bring forward the introduction of an emissions trading scheme in July 2013, the department advised the IA committees that:

Before up to $100 level was being considered and now there needs to be an adjustment to this number. Around $30 per tonne is now starting to look like it is no longer value for money. We are just changing [the] reference point for value for money. Committee to exercise judgement where the boundaries of these are applied to each project and to determine what looks meritorious. We expect the approval rating to go down significantly. Political factors to contend with now as well as a result of the lower money pot. There is not much money to be allocated. We have to be tough on value for money. They have to look good to be acceptable now.

5.11 From the information provided in paragraphs 5.9 and 5.10, it is clear that the focus of the assessment was on grant funds per tonne of carbon abated. This focus was also documented in an internal email, sent by the program management area of the department in July 2013, that stated:

The Committee have tended to give less weight to the emissions intensity than the $ per tonne.

5.12 However, this indicator was inconsistently applied in assessing applications, as demonstrated by the case studies in Table 5.2.

Table 5.2: Case studies reflecting the assessment of ‘grant funds per tonne of carbon abated’

Source: ANAO analysis of departmental records.

Recommendations to the program delegate

5.13 A clear recommendation that identifies whether funding should be approved promotes the transparency of the decision-making process. In addition, funding recommendations should explicitly address the extent to which the recommended grants are expected to result in the granting activity achieving its objectives.

5.14 In all cases, IA provided a clear funding recommendation to the program delegate. However, the recommendation did not document:

• the process that was used in forming recommendations, including any limitations of the assessment performed;
• the reduction in carbon emissions or total carbon savings that were expected to result from funding the project.¹³¹ As a result, the executed funding agreements for approved grants did not provide a:
  • clear target for the reduction in carbon emissions intensity for 35 approved grants; and
  • target for the reduction in carbon emissions intensity that matched the estimated reduction in the application, or the revised reduction where applicable, for 100 approved grants;
• whether the recommendation was based on the original application or a reframed application (only 57 of the 215 reframed applications that were identified in Chapter 3 and recommended by the IA committees were explicitly identified to the delegate as a reframed project); or
• whether the application scored highly against each merit criterion, as required by the program guidelines.

5.15 In respect to the program guidelines’ requirement that recommended applications ‘score highly against each merit criterion’, there was no minimum benchmark set even though the IA committees’ recommendations were based, in part, on the allocated merit score. Advice was not provided, by the program management area of the department, on how this requirement should be interpreted until late December 2012 (10 months into the programs). Specifically, on 21 December 2012, email advice was provided to departmental staff noting a ‘couple of instances where the score was below 50 per cent (for example, a pass mark) but was supported’; and advising that the overall merit score must be greater than 50 out of 100 (for grants of less than $1.5 million) or 60 out of 120 (for grants of $1.5 million or more) for the project to be approved. This was supplemented with a further email to assessors in May 2013 advising that the department had identified differences in how the scoring approach was being applied and that:

• the score had to be 50 per cent to constitute a pass; and
• the merit criterion one score had to be at least 35 out of 70, merit criteria two and three scores had to be at least eight out of 15. The final score had to be at least 50 out of 100 or 60 out of 120 (as relevant).

5.16 In this respect, there were a number of recommended applications that were allocated a score of less than 50 per cent against at least one of the merit criteria, as shown in Table 5.3.

Table 5.3: Recommended applications that scored less than 50 per cent against the merit criteria¹³²

Source: ANAO analysis of departmental records.

5.17 Overall, there were six¹³³ recommended and approved applications that scored less than 50 per cent of the total available score after the departmental assessment and two applications that scored less than 50 per cent of the total score available following the IA committees’ assessment.¹³⁴

Funding decisions

5.18 The program guidelines stated that the program delegate was responsible for taking decisions on whether to approve funding for each grant application made under the programs subject to:

• accepting that the application was eligible;
• referring the application to IA for merit assessment;
• receiving an IA funding recommendation¹³⁵; and
• confirming that funding was available.

5.19 Following receipt of a recommendation, the program delegate took decisions on applications until the programs were closed by the Government in October 2013. To assist the approver, the procedures manual for the programs allocated responsibility to officers in the department’s committee secretariat and program management areas to prepare decision sheets ‘based on the IA recommendations and the availability of funds’. These sheets were then provided to the delegate for review and, if satisfied, approval for the commitment of program funds was given.

5.20 The delegate accepted all of the funding recommendations made by the IA committees.¹³⁶ As a result, the recommendations made by the IA committees formed the basis for the approver’s reasonable inquiries that each grant proposal recommended was a proper use of resources. The two key factors referred to in the program delegate’s decision as forming the basis for reasonable inquiries were that the:

• merit assessment process had been conducted in accordance with the agreed framework; and
• the applicant was competitive against the relevant merit criteria established for the program.

5.21 However, these statements were not supported by the IA committee assessment process. In particular, the IA committee assessments did not result in an agreed score against each of the merit criteria, as discussed in paragraphs 5.6 to 5.8. Further, as was noted in paragraph 5.14, the recommendations provided to the delegate by IA committees did not document:

• the process that was used in forming recommendations;
• expected project outcomes¹³⁷; or
• a clear statement that the recommendation was based on an original or reframed application.

Feedback to unsuccessful applicants

5.22 The provision of feedback to applicants has been emphasised by the Joint Committee of Public Accounts and Audit as an important element of grant administration practice.¹³⁸ In the context of an open, non-competitive program, the provision of specific feedback was critical in assisting applicants to apply for future grants, under the programs, where applicants choose to submit a revised application.

5.23 The most common reason recorded for not recommending an application was that the IA committees considered that the applicant had not demonstrated that the total carbon savings over the life of the project, and in some cases the reduction in carbon emissions intensity, would be commensurate with the level of investment. Specifically, of the 211 applications that were not recommended by the IA committees, this reason was the only reason cited for an application not being successful in 146 cases (69 per cent of unsuccessful applications) and a contributing reason for another 53 cases (25 per cent of unsuccessful applications). The department advised ANAO that ‘commensurate with investment’ reflects the IA committee’s views in relation to performance against the factors identified in the customer guidelines. These factors include:

• performance against both indicators: reduction in carbon emissions intensity and total carbon savings;
• quality of evidence provided to support estimated carbon and energy savings;
• total carbon savings in the context of value for money (grant dollars per tonne of carbon abated);
• project activities: projects that include activities which account for a significant proportion of the total project cost but do not deliver a proportionate contribution to total carbon savings are unlikely to be competitive; and
• carbon and energy savings outcomes for similar projects in the same industry.

5.24 As the feedback provided to unsuccessful applicants may refer to one or more of five factors listed in the customer guidelines, there was a risk that the advice was too general in nature to be useful to applicants should they have sought to revise applications or prepare new applications for other projects. In the context of feedback being provided to unsuccessful applicants, in October 2014 the department advised ANAO that:

At a minimum the Customer Service Manager and Program Manager attended all Committee meetings. They listened into the discussion and answered any questions posed to them by the Committee. They could ask questions to clarify that they understood the reasons for the decision. The decision reflected the merit criteria(s) where the application was not competitive and this was relayed to the applicant by the Customer Service Manager.

5.25 Against this background, after being advised by the department that their initial application was not successful, or not likely to be successful, there were 47 applicants that submitted a revised application that was based on a similar project. Of these applicants:

• 25 applicants (53 per cent) were recommended and approved for funding;
• 12 applicants (26 per cent) were not recommended for funding and a grant was not approved¹³⁹; and
• 10 applicants (21 per cent) were not successful due to the closure of the programs.¹⁴⁰

Conclusion

5.26 A feature of the programs was the high proportion (74 per cent) of applications that proceeded to the merit assessment stage being recommended and approved for funding. In this context, there were a number of shortcomings in the advice provided to inform funding decisions:

• the records supporting the IA committee assessments did not demonstrate that each application was assessed against each of the merit criteria. ANAO analysis of the CTIC and CTFFIC meeting notes showed that merit criterion one indicators were only explicitly discussed in less than half of the applications considered. Further, for the 60 per cent of recommendations made by these IA committees, the overall merit score was calculated using an average of the committee members’ scores, rather than an agreed score against each criterion;
• the advice provided to the program delegate did not demonstrate that recommended applications rated highly against each of the merit criteria; and
• the advice to the delegate for some applications did not identify the expected outcomes from funding recommended projects.

5.27 Nevertheless, the program delegate accepted all of the recommendations.

Recommendation No.4

5.28 To promote a stronger outcomes orientation in the administration of future grant programs, the ANAO recommends that the Department of Industry:

1. clearly identifies, in advice provided to decision‐makers, the extent to which assessed projects are expected to deliver outcomes that are consistent with the overall program objective and related performance targets; and
2. include, as a requirement in respective funding agreements, the expected outcomes that informed decisions to award funding.

Department of Industry’s response:

5.29 Part (a): Agreed.

5.30 Part (b): Agree in-principle. The Department notes that it would not always be appropriate to include programme level outcomes as a contractual obligation for individual grant recipients.

Introduction

6.1 The Commonwealth’s grants administration framework addresses the distribution of funding awarded under grant programs and includes various reporting obligations in relation to approved grants. In this context, ANAO examined the website reporting of approved grants and the distribution of funding awarded under the programs.

Grant reporting

6.2 Website reporting of individual grants was introduced by the Australian Government to promote transparency and accountability.¹⁴¹ At the time that the program delegate made the majority of funding decisions, the requirement to publish information on individual grants was set out in the 2009 version of the CGGs. The CGGs required that each agency publish information on individual grants on its website no later than seven working days after the funding agreement for the grant took effect. These reporting obligations were updated in the CGGs issued in June 2013, with the main change being an extension of the timeframe for agencies to publish information on their websites from seven to 14 days. The department advised ANAO in October 2014 that:

Over the life of the program there were 25 breaches recorded as part of the Certificate of Compliance process where the reporting was not undertaken within the 7 or 14 day period out of a total of 603 applications.

6.3 In addition to timely reporting, it is also important to publish consistent information on grants. This matter was highlighted in the July 2008 report of the Strategic Review of the Administration of Australian Government Grant Programs. The information to be reported for each grant, and a template to be used for reporting purposes was outlined in Finance Circular 2009/04 Grants—Reporting Requirements released in June 2009 and Finance Circular 2013/02 Australian Government Grants: Briefing and Reporting released in May 2013. The mandatory data fields that must be populated by agencies when publishing grant details include grant recipient, value (inclusive of GST), approval (or commencement) date, grant term, location and postcode.

6.4 ANAO compared the information reported on the department’s website with information in submitted applications and executed funding agreements. This analysis confirmed that the grant information relating to ‘grant funding location’ and ‘postcode’ reported on the department’s website was accurate in most instances, except for:

• five applications for which the head office address was reported; and
• 37 applications which involved projects at multiple locations.

6.5 In this respect, Finance Circulars 2009/04 and 2013/02 both required the ‘grant funding location’ to describe the area where funding is to be used. For grants that are provided to multiple locations, details are required that best describe the area where funding is to be provided, such as Riverina, NSW or national.

6.6 As was required by the CGGs, the department’s website reporting related to the signing of funding agreements rather than the award of individual grants. In this respect, the most common method of reporting used by the department was to report the date that the funding agreement was executed as the commencement date and the project end date in place of the grant term, as shown in Figure 6.1.

Figure 6.1: Grant reporting

Source: ANAO analysis of departmental records.

6.7 In March 2013, the department was advised by Finance that, as long as the total duration (for example, the start and end dates) was provided in the website reporting of grants, the grant end date would suffice. However, in the department’s website reporting:

• a negative grant term was reported for 42 executed grants, with the commencement date reported as being after the end date; and
• a grant term of up to 10 days was reported for another 11 executed grants.

6.8 Reporting of at least seven per cent of grants was, therefore, inconsistent with the CGGs requirement to report the ‘grant term’ in months and, failing that, Finance’s advice that the total duration must be reflected in the website reporting.

AusIndustry reporting

6.9 In addition to publishing information on grants executed under the programs on the department’s website to comply with the CGGs requirements, grants approved under the programs were published on AusIndustry’s website to ‘assist potential customers better understand the type of projects that were supportable’.¹⁴²

6.10 Publication of both the executed and approved amounts on two different websites had the potential to confuse stakeholders, as the reporting used inconsistent grant amounts (reporting the executed amount inclusive of GST on the department’s website, but the approved amount exclusive of GST on the AusIndustry website). In this respect, the department advised ANAO that the grant recipient details for publication were generated using two different processes. The list of grantees published on the department’s website was generated using its finance (payments) system, while the list published on the AusIndustry website was based on information in the department’s grants management system, with the project description confirmed with the grant recipient prior to the information being published.

6.11 The department has committed to streamlining its reporting process. In this context, since the closure of the programs, the AusIndustry website has been merged with similar websites to form a single consolidated website for business.¹⁴³

Distribution of funding

6.12 Since December 2007, agencies have been required to publish on their website details of individual grants. In addition, since January 2009, the grants administration framework has required Ministers who are Members of the House of Representatives to report to the Finance Minister each instance in which they approve a grant in their own electorate.

6.13 As outlined in an ANAO performance audit on the grant reporting obligations¹⁴⁴, where audit reports or public commentary have raised questions about the political distribution of grant funding, the concerns raised have generally related to a wider issue than grants approved by a Minister in his/her own electorate. Specifically, the concern has more often been whether the total distribution of approved grants under a particular program has favoured the party in government, rather than just the electorate of the particular Minister who was making the decisions. This was the case in respect to the programs with concerns being raised with ANAO about the extent to which the funding had been awarded to projects located in electorates held by the Australian Labor Party (ALP), with comparatively fewer grants awarded to projects located in electorates held by the Coalition.

6.14 An indicator of the equity and impartiality of decision-making that is frequently applied is the distribution of approved funding across party electorates. In this context, and as noted at paragraph 6.13, concerns had been raised with ANAO about the extent to which the funding had been awarded to projects located in electorates held by the then Government. Those concerns were based on analysis of data reported on the AusIndustry website for the CTIP. Analysis of that data, as advised to ANAO by the Parliamentarian, had suggested ‘a significant skew, of about two to one, in funding towards electorates with Government representation’. This analysis was based on the 94 grants that had been announced and were reported on the AusIndustry website as at 31 May 2013.

6.15 ANAO conducted a similar analysis as the one provided by the Parliamentarian using the grants that were required to be reported on the department’s website by 31 May 2013. The results of this analysis are presented in Table 6.1. Although there was a skew toward ALP-held electorates at this date under the CTIP, there was a skew in funding of a similar magnitude towards Coalition-held electorates under the CTFFIP. The combined results of these analyses show that ALP-held electorates had received $82.8 million (56.6 per cent) in funding as at 31 May 2013 and Coalition-held electorates had received $63.6 million (43.4 per cent) in funding.

Table 6.1: Funding distribution by major political party as at 31 May 2013

Source: ANAO analysis of departmental records.

Note 1: For CTIP, one grant related to a project located in an electorate held by an independent. The value of this grant was $128 533. For CTFFIP, two grants related to projects located in an electorate held by an independent. The value of these grants was $377 605.

6.16 The award of funding was undertaken through a public, open call for applications and was wholly consistent with the IA committees’ recommendations as to which application should be approved. In the context that 62 per cent of approved funding related to projects located in electorates held by the governing party, ANAO also examined whether there was any evidence of the electorate status influencing the process by which applications were received, assessed as to their eligibility and merit, and recommended to the program delegate. As no funding decisions were made by the delegate after the 2013 Federal election was announced, the distribution of funding, provided in Table 6.2, was based on the seats held after the 2010 Federal election.¹⁴⁵

Table 6.2: Distribution of funding by political party based on seats held after the 2010 Federal election¹⁴⁶

Source: ANAO analysis of departmental records.

6.17 As illustrated by Table 6.2, applications relating to projects in ALP-held electorates sought greater funding than those involving projects in Coaliation-held electorates. This is reflected in the former representing 56 per cent of total funding sought, and the latter representing only 42 per cent of total funding sought. However, the approval rates, in Table 6.2, demonstrate that there was no bias in approved funding across party electorates, with approval rates for each party being between 50 per cent and 60 per cent.

6.18 Further, the objective of the programs did not involve any imperative for funding to be allocated other than to those eligible applications that had demonstrated merit in terms of the published criteria. In this respect, there was no evidence that electorate distribution played any role in the application, assessment or approval of grant funding. Specifically:

• applicants were asked to identify the location of the project, but not in electorate terms;
• recommendations to the program delegate did not identify the electorate in which the projects were located;
• the program delegate, who was an official in the department, accepted all of the funding recommendations made by IA committees; and
• there was no ministerial involvement in decision-making for grants under $10 million.¹⁴⁷

Conclusion

6.19 The department’s website reporting of grants made under the programs was largely consistent with the reporting requirements of the CGGs and associated guidance.

6.20 The application, assessment and decision-making processes for the programs guarded against the award of funding being politicised.¹⁴⁸ ANAO analysis of the distribution of funding awarded under the programs did not identify any political bias. Of note in this respect was that, although the total value of grants to electorates held by the then Government was greater, the approval rate for grants to electorates held by then Opposition was slightly higher.

Appendices

Please refer to the attached PDF for the Appendices:

• Appendix 1: Department of Industry response

 

Abbreviations

Glossary