Background

In June 1999 and May 2001, the Government announced the Telstra Social Bonus 2 (TSB2) and Telecommunications Service Inquiry Response (TSI Response) funding packages, respectively. These packages, totalling over $830 million, are designed to enhance telecommunications infrastructure development and service availability across Australia, and address associated inequities in regional, rural and remote areas compared with capital cities.

All 20 programs under the TSB2 and TSI Response packages are administered by the Department of Communications, Information Technology and the Arts (DCITA). As at March 2006, most of the programs have been completed or are nearing completion.

The audit objectives were to examine the extent to which selected TSB2 and TSI Response programs:

• are achieving or had achieved their objectives; and
• had been administered effectively by DCITA according to better practice principles. To evaluate this aspect, the audit assessed DCITA's compliance with the better practice principles outlined in the Administration of Grants Better Practice Guide (May 2002) produced by the Australian National Audit Office (ANAO). There are 19 separate principles covered under the broad areas of:
  • Planning for effective grant programs;
  • Selecting projects;
  • Managing and monitoring funding deeds; and
  • Evaluating and reporting grant program performance.

The ANAO examined the following programs (the ‘programs subject to audit'):

• Building on Information Technology Strengths (BITS) Incubator Program;
• BITS Advanced Networks Program (ANP);
• BITS Intelligent Island Program;
• National Communications Fund (NCF) Program;
• Towns over 500 Program;
• Information and Communications Technology (ICT) Incubators Program; and
• Advanced Networks Program 2 (ANP2).

Two of the programs subject to audit from the TSB2 package, BITS Incubator and ANP, were, in effect, extended by two new programs, ICT Incubators and ANP2 respectively. These were funded from the Backing Australia's Ability 2 package of programs announced in June 2004.

The delivery mechanisms for the programs subject to audit varied from contracts with telephony providers, to grants through or to consortia of private or (state) public sector organisations.

The better practice principles for grant management serve as a guide to agencies to help them to administer their grant programs efficiently, effectively and equitably. The ANAO considers that compliance with the principles will lead to sound grant management practice. However, there may be occasions where agencies consider that relevant principles should not be applied to their grant programs for various reasons including the existence of adequate compensatory management controls. In these cases, the ANAO considers that the transparency of agencies' grant management would be well served if they formally documented the reasons for partially complying or not complying with the relevant principles of better practice.

Overall audit conclusion

The ANAO considers that all programs subject to audit, except one, are effectively achieving their stated objectives. Significant delays to the Intelligent Island Program have resulted from a number of outstanding issues that will need to be resolved before the program could be considered to be on track to achieve its objectives.

The ANAO considers that the ICT Division of DCITA, where the programs subject to audit have been managed, demonstrated a high degree of understanding of the principles of better grant management practice. The ANAO commends DCITA's attempt to further reinforce sound grant management practice within the department through its recently-produced Practical Guide to Program Management.

The ANAO concluded that DCITA's management of the seven programs subject to audit demonstrated a high level of compliance with the better practice principles of grant management. Overall, DCITA planned the programs subject to audit well, selected the best projects to fund, and managed the projects well given their complexity. The program mid-term reviews and evaluations undertaken were well established and measured the programs' performance against their respective objectives.

Notwithstanding this, the ANAO considers there are certain areas of DCITA's grant management practice, impacting more than one audited program, that only partially complied with relevant better practice principles. In this regard, processes could be improved in areas of:

• applying formal risk management methods to program management (affecting all programs subject to audit, except NCF)
• establishing performance indicators early in the life of some programs that comprehensively measure and link the programs' performance against their respective objectives (BITS Incubator, ANP, ANP2 and Intelligent Island programs);
• reporting program performance against all of the programs' respective objectives or key performance indicators (all programs subject to audit, except Towns over 500); and
• ensuring that independent acquittals of grant funds are always received in a timely manner (Intelligent Island and NCF programs).

The ANAO concluded that processes surrounding the calling for, and assessment of, submissions to extend existing projects under the ICT Incubators and ANP2 programs were not as rigorous as for their preceding programs. This resulted in some partial compliance or non-compliance with better practice principles in respect of:

• guidance provided to applicants (affecting ICT Incubators and ANP2 programs);
• DCITA's appraisal of the submissions (ANP2); and
• maximising the programs' value for money (ICT Incubators and ANP2 programs).

The ANAO concluded that, although the performance monitoring regime for the Intelligent Island Program was well established, DCITA has not monitored the achievement of the program's objectives using the agreed performance measures. This does not represent sound grant management practice. The ANAO also considers that DCITA could have taken earlier action to progress the Intelligent Island Program, which has experienced significant delays throughout its life.

The ANAO's assessment of DCITA's compliance against the 19 grant management principles for each program subject to audit is summarised in Table 1 below.

The ANAO made two recommendations aimed at enhancing DCITA's administration of grant programs.

DCITA Response

The Department welcomes the ANAO findings that its management of all audited programs demonstrated a high level of compliance and understanding of the better practice principles of grants management and that the Department has a disciplined process for budgeting and running costs which allocates resources efficiently and effectively. The Department also welcomes the ANAO's finding that programs subjected to audit are effectively achieving their objectives.

In regard to Recommendation 1, the Department notes that the decision about appropriate processes for extending programs will need to take into account any relevant assessments made in the review process of the original programs. The Department also notes that the issues of submission quality may not always arise because, for example, the program is not submissions-based or because the program has generated submissions of adequate quality in the past. As outlined in paragraphs 2.35 and 2.45 of the Report, the Department considers that, for the ICT Incubators Program and ANP2, the processes for extending were appropriate given the particular circumstances for these programs.

The Department agrees with a proactive approach to the management of risks and has developed a comprehensive and contemporary risk management framework. In regard to Recommendation 2, the Department agrees, while noting that other whole of government considerations may be relevant to the approach to be adopted.

Table 1 - Summary of DCITA's compliance against the better practice principles of grant management

Key findings

Planning for effective grant programs (Chapter 2)

The ANAO found that for all programs subject to audit DCITA had adopted sound practices with respect to:

• Determining the need for the programs or the areas to be targeted by the programs—using appropriate combinations of interdepartmental committees, scoping consultancies and consultations with program stakeholders;
• Defining operational program objectives—in terms of measurable outcomes;
• Designing the programs for accountability—during the selection process, and ongoing management and monitoring of the projects and programs;
• Selecting funding strategies—by adequately considering the risks and benefits of alternative funding strategies and by adequately considering cash management principles when establishing payment profiles for each grant and grant program; and
• Considering taxation issues—by, primarily, informing potential applicants through program guidelines how the goods and services tax applied to the programs and that successful applicants would be responsible to the Australian Taxation Office for any taxation requirements.

For the majority of programs subject to audit, DCITA's program risk management only partially complied with better practice principles. Although DCITA satisfactorily managed most of the risks applicable to the programs subject to audit, improved formal risk management practices would provide greater assurance that all significant risks to programs have been identified and mitigation strategies put in place.

The ANAO considers that the administrative controls for the BITS Incubator, ANP, Intelligent Island, NCF and Towns over 500 programs, as implemented and exercised by DCITA, provide assurance that DCITA complied with the better practice principle of maximising their value for money. However, greater leverage may have been achieved from grant funding in the ICT Incubators and ANP2 programs.

The ANAO considers that the competitiveness of the selection process for the ICT Incubators Program was adversely affected by DCITA notifying BITS incubators in advance of the possible post-selection outcomes in the program guidelines. This led to DCITA's assessment that some incubators did not make an obvious effort to prepare high quality submissions—including incubators selected by DCITA. Increased competition may have led incubator owners to propose greater contributions of cash and in-kind outgoings from themselves for their incubatees. In addition, when establishing funding deeds, DCITA set low standardised performance targets that did not present a challenge to most ICT incubators. The risk with setting performance targets too low is that, once achieved, grantees will not make every effort to maximise their performance.

The ANAO also considers that had ANP grantees provided more structured ANP2 submissions (based on additional guidance from DCITA), DCITA could have assessed more thoroughly the relative merits, and therefore the value for money, of each grantee's submission. In addition, there was no documentation on file evidencing DCITA's assessment of these submissions. As grantees were asked for submissions, it could reasonably be expected that DCITA would assess their relative merits, in conjunction with grantees' performance to date. This assessment would have improved the advice provided to the Minister for Communications, Information Technology and the Arts for funding the grantees.

The performance measurement and monitoring strategies for the ICT Incubators, NCF and Towns over 500 programs complied with better practice principles. The performance indicators of the remaining programs subject to audit only partially complied with better practice principles as they did not comprehensively measure and link the programs' performance against their respective objectives early in the lives of the programs.

At the time of the calls for proposals or tenders, DCITA developed sound program guidelines or equivalents for all programs subject to audit except ICT Incubators and ANP2. The ANAO assessed program guidelines for the ICT Incubators and ANP2 programs as partially compliant and non-compliant with better practice principles, respectively, for the reasons noted above.

Selecting projects (Chapter 3)¹

The ANAO found that for all programs subject to audit, except ANP2, DCITA had complied with better practice principles in respect of:

• handling and appraising project applications and tenders consistently and fairly; and \
• documenting decisions, and reasons for decisions.

As noted above, the ANAO considers that greater guidance to ANP grantees would have improved the quality of their submissions for further funding under ANP2. In addition, there was insufficient documentation available to enable the ANAO to assess the quality of DCITA's appraisal of ANP2 submissions.

For all programs subject to audit, DCITA adopted sound practice by:

• making grant or contract offers to successful applicants and the successful tenderer; and
• advising unsuccessful applicants and the unsuccessful tenderer within reasonable times of selection decisions.

Managing and monitoring funding deeds (Chapter 4)

Given the complexity of the projects funded, DCITA established funding deeds, MOUs or a contract under the BITS Incubator, ANP, Intelligent Island, NCF, and Towns over 500 programs in compliance with better practice principles. Nevertheless, the ANAO considers that the process for establishing funding deeds for complex or high risk operations could be improved if DCITA:

• formally assessed the benefits of obtaining internal or external technical advice during deed negotiations on a case-by-case basis;
• set and formally updated deed negotiation timetables to urge grant recipients to finalise negotiations in a timely manner; and
• required each grantee to develop a risk management plan at the start of their project and to regularly monitor and report on the management of the project's risks.

The sustainability (or ongoing financial viability) of the funded projects after the conclusion of grant programs was a feature of all the programs subject to audit, except Intelligent Island. The ANAO found that project sustainability was handled well in the funding deeds for the ANP2, NCF and Towns over 500 programs, but not as well in the funding deeds for the BITS Incubator, ICT Incubators and ANP programs. Project sustainability under the ICT Incubators Program is particularly important as it formed part of the justification for the program extension and was a program objective. Nevertheless, the ANAO recognises the experimental nature of the BITS Incubator and ICT Incubators programs meant that some of the incubator business models would be successful while others could encounter difficulties in becoming self-sustainable. The ANAO considers that had project sustainability received a higher profile, DCITA could have asked grantees to nominate sustainability measures and targets they considered appropriate. Incorporating these into the deeds does not guarantee their achievement or the longer-term viability of the incubators. However, they do provide benchmarks against which the grantees would formally and regularly report their progress on to DCITA.

The ANAO considers that the quality of funding deeds under the ICT Incubators and ANP2 programs could have been improved had DCITA:

• tailored challenging but achievable performance targets for each ICT incubator rather than using standardised targets;
• better taken into account measures to aid the ICT incubators along the path to longer-term self-sustainability; and
• documented its critical assessment of the ANP grantees' submissions for further funding under ANP2.

DCITA established strategies to monitor the performance of the projects of all programs subject to audit, except Intelligent Island, in compliance with better practice principles. The ANAO considers that the monitoring framework for the Intelligent Island Program will be enhanced once its planned procedures manual is developed.

The ANAO considers that DCITA monitored grantees' performance against the funding deed, MOU or contract requirements according to sound practice for all programs subject to audit, except BITS Incubator and Intelligent Island. The ANAO questions the timeliness of, and justification for, varying one BITS Incubator funding deed that waived the injection of previously agreed payments from its consortium members. Although the performance monitoring regime for the Intelligent Island Program was well established, DCITA has not monitored the achievement of the program's objectives using the agreed performance measures. This does not represent sound grant management practice.

The ANAO considers that all programs subject to audit, except the Intelligent Island Program, are effectively achieving their stated objectives. Significant delays to the Intelligent Island Program have resulted from a number of outstanding issues that will need to be resolved before the program could be considered to be on track to achieve its objectives. DCITA could have also taken earlier action to progress the Intelligent Island Program.

Audited financial reports are a key control mechanism to ensure Australian Government funding is spent on the projects to which it was allocated. The ANAO considers that DCITA acquitted the funding for the BITS Incubator, ICT Incubators, ANP, ANP2 and Towns over 500 programs in compliance with better practice principles. However, DCITA did not receive independent acquittals for Intelligent Island funding and funding for one of the eight NCF projects when they fell due. Where significant amounts of funding have been disbursed, such as the case with the Intelligent Island Program and the NCF project referred to above, DCITA should require the grantees to provide audited financial reports in a timely manner.

Evaluating and reporting grant program performance (Chapter 5)

The ANAO considers that DCITA established the requirements for mid-term reviews and evaluations, and final reports from grantees for all relevant programs subject to audit, in accordance with sound practices.

The mid-term reviews or evaluations complied with better practice principles as they measured the performance of the projects and the programs against the programs' stated objectives. The ANAO also considers that the reviews and evaluations provided a reasonable encapsulation of the performance of the respective programs and funded projects at the time the reviews and evaluations were conducted. Final project reports, when received, have helped measure the programs' performance against their objectives and key performance indicators. However, the ANAO notes there is a risk that DCITA will not receive final project reports from the two BITS incubators that received no further government funding.

The ANAO considers that DCITA's annual reporting for most programs subject to audit would be enhanced if DCITA placed greater emphasis on regularly measuring and reporting publicly the programs' performance against their objectives. Reporting of the performance of the Towns over 500 Program complied with better practice principles.

Footnotes

1 As the scope of the audit did not extend to examining the merits of decisions taken by the Intelligent Island Board or Project Executive to fund particular projects, the better practice criteria for selecting projects are not applicable to the Intelligent Island Program.