Background

Since 1997, the Government has introduced a number of different programs and initiatives in excess of $1 billion that are designed to enhance telecommunications infrastructure and services in regional, rural and remote areas of Australia. This has included $494 million provided through telecommunications grants programs.

Two Commonwealth departments are responsible for the administration of these grant programs. The Department of Communications, Information Technology and the Arts (DCITA) administers six programs under the banner of Networking the Nation (NTN) This includes the General Fund and the Social Bonus 2 programs. The Department of Transport and Regional Services (DOTARS) is responsible for the Rural Transaction Centres (RTC) program.

At 30 June 2003, 797 projects have been approved under the NTN programs at a cost of $351.1 million of which 373 have been completed; 164 Rural Transactions Centres have been approved under the RTC program of which 80 are operational. In addition 119 Electronic Point of Sale (EPOS) centres have been established. Expenditure and commitments on the RTC program totalled $56.5 million at 30 June 2003.

The primary objective of the audit was to assess whether DCITA and DOTARS are administering the grant programs according to better practice. The audit was also aimed at determining whether DCITA had implemented the recommendations of an earlier audit of the NTN General Fund, Audit Report No.43, 1998–99, Networking the Nation—The Regional Telecommunications Infrastructure Fund (the 1999 audit).

In 2002, the Senate Environment, Communications, Information Technology and the Arts Reference Committee requested an audit of the telecommunications grant programs administered by DCITA. This encompassed the extent to which DCITA is able to assess the success or otherwise of NTN.

The audit also covered the administration of the RTC program by DOTARS.

Key audit findings

Planning for Effective and Efficient Grant Programs (Chapter 2)

The ANAO found that neither DCITA nor DOTARS translated the Government's program objectives into operational objectives that would have helped to establish an appropriate performance management framework to monitor the efficiency and effectiveness of program delivery. In the case of the NTN General Fund this was, in part, associated with a shortcoming of the needs assessment, which did not involve a qualitative and quantitative assessment of the nature and extent of the needs of the target client groups. In the case of the Social Bonus initiatives no quantitative needs assessment was undertaken. However, DCITA did implement the recommendation from the 1999 audit that the Department brief the Minister on options for consulting with other Ministers on the changes made to the program objectives.

The RTC program was slow in getting off the ground, in part, because of a lack of a needs assessment during the planning phase to identify and target likely communities. However, some 22 months after the launch of the RTC program, DOTARS took steps to better assess priority communities and generate greater demand for the program.

Neither department conducted a formal risk analysis during the planning phase of their programs although there was evidence that risk management activities had been subsequently conducted in DCITA to identify and manage key risks. DOTARS has also taken steps to identify risks to the RTC program as a whole, but not at the individual project level.

The RTC program had a high level of budgeted administrative costs. As a percentage of total program funds administrative costs were an estimated 26.9 per cent, compared with an estimated 4.2 per cent for the suite of NTN programs. The high administrative costs for the RTC program are mainly attributable to the appointment of a separate field officer network to promote and administer aspects of the program. Although staff numbers have fluctuated over the life of the program, at the time of the audit fieldwork there were some 39 staff administering the RTC program (including field staff) compared to 29 for NTN. DOTARS has advised that staff numbers have subsequently been substantially reduced.

The ANAO found that, as recommended in the 1999 audit, DCITA had reviewed opportunities to streamline the application and approval processes for NTN and restructured the NTN secretariat to reduce internal workload inequities.

Both programs highlight the need to comprehensively plan evaluations early in the process. Although DCITA planned for a major evaluation at the outset the program, its evaluation plan did not establish baseline information needs and measurable program targets as part of a coherent performance information framework for monitoring program outcomes. There was no evidence that DOTARS had developed an evaluation strategy for the RTC program from the start of the program. However, despite a lack of an initial strategy some RTC evaluation activity has taken place and more is planned.

As a result of shortcomings in the planning process, neither program had a mechanism for feeding information gained from the evaluation of individual projects into an evaluation of the efficiency and effectiveness of the programs as a whole.

Program Promotion and Project Selection (Chapter 3)

Both Departments have actively promoted their programs. DOTARS' initial efforts, however, were not successful and some time after the launch of the program less than 50 applications had been received. DOTARS response was to establish a field officer network to promote the program, which resulted in a larger number of applications.

The ANAO did not examine selection processes for the NTN program as they had been examined previously in the 1999 audit and had not changed. The criteria used in the selection process for the RTC program were appropriately designed for the selection task. However, there have been a number of shortcomings in the guidelines and training for staff particularly as far as financial issues are concerned. The guidelines for staff are now satisfactory and remedial measures have been taken to improve financial skills. However, the re-examination and approval of applications previously rejected, raises questions about the way the selection criteria have been applied and the effectiveness of quality assurance systems in place at the time of the original assessment.

Both Departments had transparent decision-making processes. There was clear documentation of reasons for decisions and there was no evidence of inequities in the decision making process. The Departments have also put in place measures to reduce the risk of double dipping by applicants.

The 1999 audit of the NTN General Fund identified cost shifting as an area of high risk for DCITA to manage. This risk has been further exacerbated by the introduction of the Social Bonus 2 initiatives. This audit found that the relevant recommendation from the 1999 audit to examine more stringently the risk of cost shifting has been implemented.

Management of Funding Agreements (Chapter 4)

Both Departments have standard templates to assist staff in preparing funding agreements. The design of DCITA's agreement involved a better structure and a greater level of detail than DOTARS'. It was consistent with better practice to the extent that it required a description of the objectives and outputs expected from a project, linked performance indicators to the measurement of activities required to achieve individual project objectives, linked payments to project performance, and retained funds until satisfactory project completion. The ANAO found that DCITA's template had been consistently applied across NTN, providing the Department with a useful project management tool.

Both Departments provided grant recipients with details of what was expected in their project progress reports. DCITA monitors project progress through NTN Online which, although having some highly desirable features, also has some shortcomings. These shortcomings have resulted in the development of shadow computer systems. The Department advised that these shortcomings were being addressed as part of a proposed move to a department–wide Grant Management System (GMS) based on NTN Online.

There is scope for improvement with respect to DOTARS' monitoring arrangements. The most significant shortcomings were the timing of progress reports and the absence of a link between progress reports and project payments. Progress reports are not required until after an RTC has been opened. This is of little help as a mechanism for oversighting progress and managing projects during their establishment phase. The payment of all grant funding without evidence of the completion of milestones, also provides no incentive for the grant recipient to complete projects on time or to account to the Commonwealth for financial and outcomes performance.

Both Departments had significant numbers of outstanding project reports and acquittals. In response to this situation, DCITA has increased its efforts to reduce the number of outstanding progress reports and acquittals including through the establishment of a dedicated Compliance Team. DOTARS has written to recipients requesting progress reports and acquittals for 2001–02 but has also experienced difficulty in maintaining up-to-date records of outstanding reports and acquittals.

Program Monitoring and Evaluation (Chapter 5)

Both Departments have databases to monitor financial expenditure and collect performance information on projects. NTN Online provides monthly reports on both financial and performance aspects of the NTN programs. Although the financial reports are of a high standard, the performance reports are more limited. While there are no links between DCITA's financial management system and NTN Online, the two systems are reconciled. The development of the GMS, noted earlier, is designed to address some of these issues but it will not remedy the shortage of some basic performance information.

DOTARS uses a number of small, independent databases to monitor the RTC program. However, these have some serious limitations for both financial and performance monitoring. Like NTN Online, they are also not linked to the departmental payments system but, unlike DCITA, DOTARS does not reconcile the two systems. Some improvements have been made since an internal review in August 2001 and more are expected. However, the current limitations restrict the Department's ability to monitor progress against program outcomes as well as program expenditure. A GMS is also being developed by DOTARS that may address some of these issues.

Both DCITA and DOTARS have evaluated some aspects of their respective programs, and have other evaluation work in progress. DCITA abandoned its first attempt to evaluate the NTN General Fund after three years of trying as a result of methodological and data collection problems. A second evaluation exercise, which commenced in 2002, has also experienced difficulties with data collection.

DCITA is aiming to report on the extent of the increase in telecommunications infrastructure development in regional areas. However, as no baseline data was collected at the start of the program and no targets were set, it will be difficult to demonstrate the success, or otherwise, of the program.

DOTARS' review of the RTC program in 2002 found the program appeared to be meeting its objective to improve access for small rural communities to basic government, financial and other services. It also identified the financial viability of some of the Centres as a key constraint that could affect the longer-term success of the program. DOTARS, however, has taken a number of steps to try to overcome this issue.

At present, DOTARS is not in a position to determine the efficiency and effectiveness of the RTC program. However, the Department has advised that it is now undertaking a data collection exercise that is designed to enable further efficiencies in assessment processes to be implemented and to estimate the net present value of costs and benefits of the program.

Some of the shortcomings with both the evaluation of NTN and the RTC program can be traced to insufficient planning at the start of the programs. Neither Department established the range of performance information and targets needed for evaluation purposes, nor the associated data collection processes that needed to be put in place. In future, it is important that both Departments ensure that evaluations are properly planned at the outset of programs to be able to contribute to efficient, effective and accountable outcomes.

Meeting External Accountability Requirements (Chapter 6)

The ANAO found that there were shortcomings in the public reporting of both NTN and the RTC program. Current reporting through the Departments' Annual Reports does not provide sufficient information for stakeholders to make an informed assessment of program performance over time. This is because reporting on performance is primarily focused on Departmental activities, rather than their level of achievement against outcomes sought from the programs.

DCITA has not reported against the performance indicators articulated in its Portfolio Budget Statements. DOTARS does not report on key measures consistently over time. The ANAO considers that more reporting on trends, risks and challenges for the future would also allow stakeholders to make informed judgements about overall program performance.

Audit conclusions

The ANAO found that although DCITA's and DOTARS' administration of the programs demonstrated elements of better practice there was scope to improve the administration of both NTN and, more particularly, the RTC program. DOTARS is aware of many of the shortcomings with the administration of the RTC program and is working towards resolving them.

The ANAO found that although Departmental Annual Reports and the NTN Board Annual Report provide information on levels of program activity, it is difficult for stakeholders to get a sense of what outcomes have been achieved by the various programs and how they have contributed to the achievement of the government's broader policy objectives.

With respect to the implementation of the recommendations from Audit Report No. 43, 1998–99, Networking the Nation—The Regional Telecommunications Infrastructure Fund, the ANAO is satisfied that DCITA has implemented the three recommendations.

Agency responses

DCITA

DCITA welcomes the ANAO findings that the NTN programs exhibited transparent decision making processes with clear documentation of reasons for decisions; that there was no evidence of inequities in the decision making process; and that our funding arrangements were consistent with better practice.

DCITA acknowledges the importance of operational objectives as a central element in an overall performance information framework and is working to improve the performance indicators for all the programs that it administers. It notes that, because a very specific focus was established for each of the Social Bonus initiatives, its ability to report on the efficiency and effectiveness of its administration of these initiatives, and on their achievements and outcomes, is considerably enhanced.

DCITA agrees with the Report's conclusion that it is important that future evaluations are properly planned at the outset to ensure efficient, effective, and accountable outcomes.

In response to Recommendation No 4, DCITA acknowledges that the quality of its reporting could be improved and has addressed this by undertaking a major review of its Outcomes and Outputs structure and related performance indicators. The new structure will form the basis of reporting from the 2003–04 financial year.

DOTARS

The Department notes that the RTC Programme was initiated in an environment of very constrained resources and under significant time pressures. The program is driven by demand from communities. DOTARS welcomes the recognition in the report that the Department has made a range of improvements to the administration of the Programme. The RTC arrangements now form part of Regional Partnerships Programme which is based on the guiding principles set out in the ANAO Better Practice Guide. This includes establishing clear objectives, an accountability framework and performance measures in the design of the programme. Regional Partnerships also uses Standard Funding Agreements and effective monitoring arrangements in managing accountability and risk. DOTARS is also implementing a comprehensive evaluation mechanism for the new programme as well as progressively installing modules of a comprehensive grants management support system.