Background

This Government believes that innovation—developing skills, generating new ideas through research, and turning them into commercial success—is key to Australia's future prosperity. Innovation is not only the province of new or high tech industries, but also essential to the future of many of our traditional sectors such as agriculture, manufacturing and mining.¹

The Australian Government has a range of strategies designed to assist and build Australian innovation.² One of these is the R&D Start program, which provides Australian companies with grants and loans to undertake research and development (R&D) and for commercialisation of technical innovations. Since the program's inception in 1996 some $1.3 billion in grants have been approved.

The program supports projects that aim to develop new or improved products, processes, or services. Grants of up to $15 million are available, though typically they range between $100 000 and $5 million. Loans are also available to meet some project costs.

The objectives of R&D Start are to:

• increase the number of projects involving R&D activities with a high commercial potential that are undertaken by companies;
• foster greater commercialisation of the outcomes of those projects;
• foster collaborative R&D activities in industry and between industry and research institutions;
• encourage successful innovation in small companies by supporting commercialisation of internationally competitive products, processes and services;
• increase the level of R&D activity in Australia that is commercialised in a manner that will benefit the Australian economy; and
• increase the level of R&D activities conducted that provides national benefit. ³

The Industry Research and Development Board (the Board) has administrative responsibility for the program under the Industry Research and Development Act 1986 (the Act).⁴ Most administrative responsibilities have been delegated to sectoral committees⁵  or to AusIndustry, a division of the Department of Industry, Tourism and Resources.

In 2004, the R&D Start program was merged with other innovation programs to form the Commercial Ready program. The objectives and delivery strategies for the Commercial Ready program are similar to those of R&D Start, but no loans are offered under the new program. Applications for grants under the Commercial Ready program commenced on 1 October 2004.

Administration of existing R&D Start grants will continue for up to 10 years.

This audit

The objective of the audit was to assess the Commonwealth's administration of the grants component of the R&D Start program. Lessons for the new Commercial Ready program have been identified in the audit. Accordingly, recommendations arising from this audit are directed, when appropriate, to the Commercial Ready program.

As most financial assistance is in the form of grants, the loans component of the program was excluded from the audit.

Key findings

Aspects of program governance (Chapter 2)

Under the Act and ministerial directions, the Board is the approver of spending proposals. The Secretary of the Department is accountable, under the Financial Management and Accountability Act 1997, for the proper expenditure of public money appropriated to the Department.

The Board has delegated most of the administrative activities for R&D Start to the three sectoral committees and AusIndustry. Documents outlining the delegations and the Board's policies and procedures provide a clear articulation of, and framework for, program administration.

The Department did not have procedures in place to be assured that the Board, by approving spending proposals, had not over committed the program's annual allocation. This weakness resulted in the program over committing its allocation by $60 million in 2001–02. The shortfall was met by a $20 million reallocation of funds from other innovation programs, and an advance of $40 million from future appropriations for R&D Start.

Because of the over commitment of program funds, the Board ceased approving grants from January 2002. The program was formally suspended in May 2002. It was reopened in November 2002. Prior to its reopening, the Board and the Department put in place new financial management arrangements to prevent a reoccurrence of such over commitments. The key corrective action was the delegation of responsibility for the financial control of R&D Start funds to a senior executive officer of AusIndustry. However, the delegation did not resolve the overlap in the legislated financial management accountability for R&D Start between the Board and the Department.

The legislative provisions relating to the Board's financial management responsibilities were removed from the Act and ministerial directions prior to the commencement of the new Commercial Ready program in October 2004. Under the revised legislative arrangements, DITR is responsible for the administration of Commercial Ready.

The program's performance measurement framework has some limitations that affect its usefulness in measuring the achievement of objectives and outcomes. For example, the relevance of some Key Performance Indicators to measure program outcomes is not clear. Also, specific targets have not been defined for Key Performance Indicators, to enable performance to be assessed.

Prior to 2004, two independent evaluations were conducted to assess the program's effectiveness. In 2004, AusIndustry commenced biannual surveys to collect information on a sub-set of the program's outcome measures and Key Performance Indicators. A low response rate to the first two biannual surveys undermined the reliability of the initial results, as the results could not be readily extrapolated to the target grant population.

Notwithstanding the limitations, the survey results suggest that participation in R&D Start has produced positive results for grant recipients and the wider community.

The Board and committee members are drawn largely from the private sector. This provides commercial and technical expertise to inform decisions. Inevitably, this means that many of these members have links to industry, creating a need to manage the risk of perceived or actual conflicts of interest.

For the Board and its committees, the Act requires disclosure of direct or indirect pecuniary interest by a member in a matter being considered, or about to be considered. The ANAO found that the Board's Conflict of Interest and Statement of Private Interests policies are in accordance with recognised better practice principles, and meet the requirements of the Act. The ANAO observed the policies working well in practice.

Access to the program (Chapter 3)

R&D Start, and its successor, Commercial Ready, have been promoted as part of AusIndustry's suite of innovation programs. Market surveys are conducted to assess the effectiveness of the awareness raising activities, but these are not conducted regularly. The most recent survey of the target audience, conducted in 2002, indicated that promotional and marketing activities are effective at raising awareness. Some 71 per cent of businesses and 67 per cent of professional bodies and industry groups were aware of the program.

As professional bodies and industry groups are a cost effective way to increase the awareness of potential applicants, AusIndustry has targeted them in its awareness raising activities since 2002. However, it has not measured the effectiveness of its increased targeting. This is important to evaluate and inform the development of such strategies.

AusIndustry allocates a Customer Service Manager to a potential applicant so as to provide a single point of contact, simplifying access to information and advice. The ANAO found that AusIndustry places a high priority on matching the Customer Service Manager's skills with the potential applicant's project or, if possible, allocating a Customer Service Manager to a potential applicant with whom they have worked previously.

Approximately 80 per cent of respondents to the 2003 and 2004 Customer Satisfaction Surveys⁶ agreed that AusIndustry staff provided advice that was clear, consistent and comprehensive.

Processing of applications (Chapter 4)

The decision to approve or reject an R&D Start application was made in four stages.

Firstly, AusIndustry confirmed that the application met the program's eligibility requirements. Where eligibility could not be established clearly by AusIndustry (for example, the ability of the applicant to meet its share of project costs), the application was sent to the relevant committee to determine eligibility. AusIndustry implemented several initiatives to improve the ability of its staff to determine eligibility but has not measured the effectiveness of these initiatives.

At the second stage, AusIndustry assessed the strengths and weaknesses of each eligible application, and provided the relevant committee with its assessment, as well as a recommendation on whether the application should be supported. A merit rating between 1 (low merit) and 6 (very high merit) was assigned to each criterion, and the individual merit ratings were then summed to give a total rating score that was used to determine competitiveness.

In the third stage, having regard to AusIndustry's ratings, a committee assessed the project, and recommended to the Financial Delegate whether the project was sufficiently competitive to justify financial assistance. In 2003–04, 80 per cent of AusIndustry's assessments of the competitiveness of applications were the same as those of the relevant committee. In 20 per cent of cases, committees recommended the application be rejected, whereas AusIndustry recommended it be supported.

AusIndustry had no structured approach to analysing the reasons for the different recommendations, so as to identify improvements that could be made to the quality of advice to committees. The ANAO found that, in 22 per cent of cases, AusIndustry had recommended supporting the application where the total rating score was high, but it had given one merit criterion a low rating. Committees had not supported any of these applications.

A common process for appraising applications accords with grant administration better practice. Although appraisal procedures had only been documented for one of the four committees, each committee developed substantially the same procedures for assessing an application's level of competitiveness.

However, in 2003–04, committees used different minimum total rating scores to recommend support for applications. The different levels were not based on a risk, merit based, or other rationale, nor was the reason for the different levels documented. Although the observed differences in the ratings were small, the lack of a rationale for the difference undermines transparency and risks claims of bias. Without adequate controls, there is a risk that the difference will increase over time, resulting in applications from different industry sectors being subjected to quite different standards of competitiveness without a clear reason why.

The Board and AusIndustry did not undertake regular comparative analyses of the proportions of applications supported by committees, or the extent to which any differences in these proportions were reflected in the subsequent commercial success of projects.

The ANAO found that the proportion of applications supported by the Information Technology and Telecommunications Committee was lower than other committees. AusIndustry advised that this is likely to be a result of the committee receiving a higher rate of less meritorious applications than other committees.

On the other hand, small grant applications had a relatively high rate of support (by the AusIndustry R&D Start Committee). Available data on project outcomes indicate that these smaller grant projects also have a higher rate of subsequent commercial success.

The final stage in the processing of applications was the approval or rejection of the application by the Financial Delegate. Prior to approval, the Financial Delegate confirmed that the recommendation was consistent with the program's objectives and there were sufficient uncommitted funds to meet the new financial obligations. For the applications examined by the ANAO, the Financial Delegate approved grants in accordance with the financial management delegation.

Contractual arrangements (Chapter 5)

The Board, on behalf of the Commonwealth, established a formal contractual arrangement with the successful applicant(s) before grant payments commenced. This contract is called a Deed of Agreement (the Agreement).

AusIndustry used a template document to produce Agreements. The ANAO found that the Agreement provides sufficient information to allow AusIndustry to determine that the grant is used for the agreed purpose, and that grant payments are made according to progress of the project. The Agreements also clearly articulate the requirements that must be met by grant recipients for payments to be made.

The ANAO found that most terms and conditions, such as project start and finish dates, timings of milestones and annual expenditure profile, were accurately reflected in the Agreements examined. Some due dates for progress reports were inconsistent with the program's reporting guidelines.

These Agreements were also signed in accordance with the Board's requirements.

Compliance with contractual arrangements (Chapter 6)

The grant recipient reports to AusIndustry each quarter on the project's progress. If progress is assessed as satisfactory, AusIndustry makes quarterly payments to the grant recipient.

The proportion of progress reports submitted to AusIndustry by the due date has almost doubled from 31 per cent in 2001–02 to 60 per cent in 2004–05. Overdue reports have little impact on the program in the short term as AusIndustry does not make a grant payment without a report having been received and assessed. No penalties have been imposed on a grant recipient for late reporting.

Assessment of progress reports can be by a desk audit of the claims made in the report or by visiting the grant recipient. The targets for site visits are inconsistent across different components of AusIndustry's risk management strategy.

AusIndustry did not achieve its targets for the number of visits undertaken. In 2004–05, only 25 per cent of high-risk projects were visited, against a target of 100 per cent. Approximately 60 per cent of all projects were visited in 2004–05, also against a target of 100 per cent.

There is discretion to conduct end-of-project visits for low-risk projects. However, AusIndustry does not monitor the use of that discretion, despite this being the last opportunity for AusIndustry to assure itself that projects achieved the outcomes in accordance with Agreements. Over the period 2001–02 to 2004–05, AusIndustry visited approximately half of the projects at their completion.

Once progress has been assessed as satisfactory, AusIndustry calculates the grant payment. The ANAO found that the steps in calculating the level of the payment are adequately defined in the Procedures Manual. Proformas and checklists are used to ensure all steps are undertaken.

The payments examined by the ANAO were made according to the approved amounts set out in Agreements, or had been appropriately varied. Also, payment calculations were generally appropriately checked and authorised, and retained on the project file, providing an audit trail.

AusIndustry does not confirm that project expenditures claimed in quarterly progress reports comply with the program's eligible expenditure guidelines. Instead, it relies on an annual audit certificate issued by an independent auditor appointed by the grant recipient.

As part of its compliance management strategy, AusIndustry is planning to implement an audit program that aims to substantiate the veracity of independent audit certificates submitted by the grant recipients. AusIndustry planned to conduct 20 audits as a pilot in 2004–05, but only conducted nine. AusIndustry advised in response to the draft report that, following an analysis of the results of the nine completed audits, it has set a target of 12 substantiation audits for 2005–06.

Overall audit conclusion

The ANAO concluded that the delivery of the R&D Start program is generally well managed by AusIndustry. Improvements in some areas would further strengthen the framework, improve the efficiency of the delivery of grant financial assistance, and provide greater transparency to stakeholders.

The key governance arrangements are well established and roles and responsibilities of the Industry Research and Development Board, its committees, Innovation Division and AusIndustry are clearly articulated.

However, prior to 2002, arrangements for addressing financial management accountabilities were insufficiently clear, leading to an over commitment of funds and the suspension of the program. New financial management arrangements were introduced prior to the reopening of the program. These arrangements addressed the weakness in the previous system relating to lack of clarity of financial management accountabilities.

Sound procedures are in place to manage the potential for perceived and actual conflict of interest.

The Innovation Division-AusIndustry Business Partnership Agreement defines the program's performance measurement framework. However, there are limitations in the framework that affect its usefulness, including the relevance of some Key Performance Indictors and the lack of specific targets that enable performance to be assessed. In addition, surveys conducted to collect outcome information have low response rates. Addressing these gaps would provide AusIndustry and the Board with better information against which to assess program performance, improve public reporting, and properly evaluate the program.

Based on the available performance information, survey findings suggest that the program has had positive impacts on companies that have received R&D Start grants. Also, an independent evaluation conducted in 2003 estimated that the national economic benefits resulting from the program have been of the order of $4.50 for every R&D Start dollar.

The procedures for appraising applications generally followed better practice. However, committees selecting applicants for financial assistance used different minimum rating scores, creating relative differences in the level of competitiveness required to receive financial assistance.

The contractual agreement between AusIndustry and grant recipients provides a sound accountability framework for the management of grants.

AusIndustry's approach to monitoring compliance against the obligations in the Agreement is supported by detailed standard operating procedures, which are reasonably well documented and implemented. Improvements are still required to ensure grant recipients adhere to the due dates of their progress reports, so as to minimise the risk that the program's budget allocation will be underspent.

Applicants are generally satisfied with the quality and timeliness of the service provided by AusIndustry.

Lessons learned from administering R&D Start have been incorporated in administrative processes, either for R&D Start, or for the new Commercial Ready program.

Recommendations and responses

The ANAO made six recommendations aimed at strengthening the administration of R&D grants under the R&D Start and Commercial Ready programs. All recommendations were agreed.

The Department provided the following summary comments:

DITR is pleased with the ANAO's conclusion “that the delivery of the R&D Start program is generally well managed by AusIndustry”. In addition, the ANAO has found that lessons learned from administering R&D Start have not only been used to improve the administration of R&D Start, but have also been adopted in the new Commercial Ready program.

DITR agrees with the ANAO's recommendations and acknowledge their potential impact in strengthening or further improving the delivery of the program. However, DITR notes that in any assessment process involving a range of views, there will always be degree of disparity in overall scores reflecting individual judgments, and there is a specific role for the committees who have the technical and industry expertise in the assessment process.

The Industry Research and Development Board provided the following summary comments:

The Board is pleased with the ANAO's conclusion that the delivery of the R&D Start program is generally well managed by AusIndustry and the key governance arrangements between the Board, its committees and DITR are well established and clearly articulated. The Board is assured by the ANAO's findings that its conflict of interest policies are “in accordance with recognised better practice principles” and sound procedures were found to be in place to manage conflicts of interest.

Footnotes

1 John Howard The Hon, M.P., Backing Australia's Ability (2000).

2 These strategies include the 1998 Investing for Growth Statement, the 2001 Backing Australia's Ability strategy, and the 2004 Backing Australia's Ability—Building our Future through Science and Innovation strategy.

3 Commonwealth of Australia, R&D Start Program Directions No. 3 of 2002, 28 November 2002, section 5.

4 The Governor-General has appointed members with relevant industry, management and academic backgrounds to the Board.

5 There are three sectoral committees: Biological; Engineering and Manufacturing; and Information Technology and Telecommunications.

6 AusIndustry collects information from successful and unsuccessful applicants in its Customer Satisfaction Surveys.