Introduction

1. Against a background of increasing domestic electricity prices and a greater focus on encouraging Australians to embrace energy efficiency measures, the Smart Grid, Smart City Program was announced by the Australian Government in the 2009–10 Federal Budget. The then Government committed up to $100 million for the program to:

create, in one Australian city, town or region, an energy network that integrates a smart grid with smart meters in homes, thereby enabling greater energy efficiency, reduced emissions and use of alternative energy sources such as solar power.¹

2. A smart grid combines advanced communication, sensing and metering infrastructure (including smart meters) with the existing electricity network. A smart grid can also improve the reliability of electricity services by remotely identifying and resolving faults on the electricity grid, better managing voltage and identifying infrastructure that requires maintenance. In addition, smart grids have the potential to assist consumers in managing their individual electricity consumption by providing real-time information on electricity use, and enabling the use of ‘smart appliances’ that can be programmed to operate on off-peak power.

3. The Smart Grid, Smart City Program had the following four high-level objectives: deploying a demonstration and/or commercial-scale smart grid rollout; building public and corporate awareness of the economic and environmental benefits of smart grids; gathering robust information and data on smart grid applications; and investigating synergies with other infrastructure.

4. The program was established as a competitive, merit-based grant program, with the aim of selecting one consortium led by an Australian electricity distribution network service provider (DNSP) to deliver the program. The grant application period opened in late October 2009, with four consortia submitting applications. The preferred applicant, EnergyAustralia (later renamed Ausgrid), was announced in June 2010. A funding agreement between the Australian Government and Ausgrid, with a total value of $93 million, was signed in October 2010. The program was to be delivered in Newcastle, New South Wales, surrounding districts in the Hunter Valley, and some areas of Sydney.

5. The key deliverables under the funding agreement with Ausgrid included:

• the deployment of smart meters into consumers’ homes;
• the trialling of new electricity tariff regimes and feedback technologies (such as an internet portal showing real-time electricity consumption and costs);
• testing new technologies in the electricity network to enhance reliability and assist in the integration of renewable energy sources (referred to as grid-side applications);
• a trial of 20 electric vehicles (EVs) to gather information about their potential broader rollout in Australia; and
• testing potential smart grid compatibility with the National Broadband Network (NBN), and other technologies, such as ‘smart’ gas and water metering.

6. The rollout of individual projects under the funding agreement commenced in late 2010, with the program originally scheduled to be completed by September 2013. However, in early 2013 the Government announced that one component—trialling new electricity retail tariffs together with smart meters and feedback technologies—would be extended to the end of February 2014.

7. At the conclusion of the program, the funding agreement required Ausgrid to agree with the Australian Government on a decommissioning strategy for each of the projects implemented under the program. Where this strategy included the sale of assets, proceeds of the sale were to be returned to the Australian Government.²

Administrative arrangements

8. The responsibility for the administration of the Smart Grid, Smart City Program has been transferred across four administering departments since it was established in May 2009. The former Department of the Environment, Water, Heritage and the Arts (DEWHA—now the Department of the Environment) administered the program from May 2009 to March 2010, including designing the program and conducting the grant assessment and selection process. The former Department of Climate Change and Energy Efficiency (DCCEE) administered the program from March to September 2010, including finalising the grant assessment and selection process and negotiating the funding agreement.³ The former Department of Resources, Energy and Tourism (RET) was responsible for the majority of the program’s implementation, from September 2010 to September 2013, when the Department of Industry assumed responsibility for the program.⁴

Audit objective, criteria and scope

9. The objective of the audit was to assess the effectiveness of the administration of the Smart Grid, Smart City Program, including the establishment, implementation and ongoing management of the program.

Criteria

10. The criteria used by the ANAO to address the objective examined the:

• program’s design and establishment, including governance and oversight arrangements;
• grant assessment process to select the provider for the program;
• negotiation and management of the funding agreement; and
• monitoring, reporting and evaluation arrangements put in place to determine the extent to which the program has achieved its objectives.

Scope

11. The audit focused on the implementation of the Smart Grid, Smart City Program by the responsible departments. It did not include a technical assessment of the various projects implemented under the funding agreement. Broader issues associated with smart grids and smart meters, such as potential advantages or disadvantages of time-of-use pricing regimes, or any health and safety concerns that may be associated with smart meters were also not examined.

Overall conclusion

12. The $100 million Smart Grid, Smart City demonstration program was established to implement or trial a range of new technologies in a challenging environment. These challenges included technological issues, consumer resistance to smart metering technologies, regulatory reform in the electricity sector, and responsibility for the program being transferred across four departments between 2009 and 2013. While a number of staff transferred with the program, changes in administrative responsibility occurred at key stages of the program’s implementation (such as the approval of the successful grant applicant) and resulted in changed oversight arrangements and administrative policies and procedures. The changes also made it more difficult to manage program knowledge, including the creation and retention of program records.

13. As a demonstration program, a key outcome from the Smart Grid, Smart City Program is data and information that contributes to greater knowledge and understanding regarding the rollout of smart grids. To date, reports from the grant recipient, Ausgrid, and the department indicate that many of the program’s trials have been successfully implemented, with a range of data collected.⁵ Projects that were completed largely in accordance with the funding agreement included:

• grid-side applications that tested new technologies to assist distribution network service providers (DNSPs) to better manage electricity supply;
• energy resource management projects to test the potential impact of wide-scale renewable energy generation (such as rooftop solar panels or wind turbines) on the existing electricity grid, and trial storage batteries and other technologies that can assist to manage peak electricity demand and integrate with energy generated from renewable sources;
• an electric vehicle (EV) trial that involved the operation of 20 vehicles over short and longer-range journeys for an 18-month period; and
• a ‘network’ trial that rolled out smart meters to customers’ homes and tested their interaction with feedback technologies providing information on real-time electricity use.

14. Key components of the program have, however, presented additional challenges, such as the retail trial. This trial, which cost $20 million and represented the largest component of the program, sought to test consumers’ interaction with a range of electricity retail tariffs and feedback technologies.⁶ Technological difficulties, combined with customer resistance and problems in securing an electricity retail provider, contributed to significant delays in rolling out the retail trial and, ultimately, the achievement of lower‑than‑expected numbers of customers participating.⁷

15. Overall, the administering departments established appropriate arrangements to support the implementation of the Smart Grid, Smart City Program. The commissioning and completion of a pre-deployment study, coupled with the engagement of a broad range of stakeholders, informed the design of the program, while oversight arrangements, including an interdepartmental Steering Committee guided the program’s implementation. In addition, the arrangements established to manage the Smart Grid, Smart City funding agreement, including a structured reporting framework to underpin grant payments, enabled the department to monitor whether project milestones were being met and projects were delivered to the required standard.

16. There was, however, scope for improvement in several areas of the departments’ administration of the program, including: aspects of the grant assessment and selection process, including probity arrangements; and the measurement and reporting of program performance.

17. While the grant assessment and selection process involved three levels of review undertaken by the then Department of the Environment, Water, Heritage and the Arts (DEWHA), an Expert Panel and an Independent Assessment Panel, there was scope for aspects of the process to be strengthened to enhance transparency and accountability. In particular, it was not evident from departmental records that the Independent Assessment Panel had assessed each applicant against the five published merit criteria. The Chair of the Independent Assessment Panel and the Department of the Environment informed the ANAO that each panel member did conduct an assessment of each application against the published merit criteria, with the assessments subsequently used to rank applicants. However, these assessments have not been retained⁸ and the assessment report of the panel did not clearly set out this process.

18. Probity processes help to ensure that grant assessment and selection processes are transparent and accountable. While DEWHA sought probity advice on aspects of the grant assessment and selection process, overall, the probity arrangements established for the process were not in keeping with those expected for a grant of this scale and complexity. Given the scale of the program and the involvement of independent assessors, there would have been merit in the department: preparing a probity plan; requiring the probity adviser to attend assessment panel meetings; and reviewing the panel’s selection report to confirm that the assessment process undertaken aligned with the published program guidelines and that any identified conflicts of interest had been appropriately managed.

19. Over the course of the program’s implementation, the departments have provided information on the progress of various program activities through a range of communication channels, such as stakeholder meetings, program seminars, presentations to conferences and the publication of Ausgrid reports. This information has not, however, specifically addressed the extent to which the program’s objectives are being achieved. Developing and reporting against an appropriate set of outcome-focused key performance indicators would have better: informed the department’s senior managers, Parliament and other stakeholders about the progress being made towards achieving the program’s objectives; and supported the planned program evaluation.

20. This audit has highlighted the challenges in maintaining administrative continuity for complex programs that are transferred across agencies. In these circumstances, sound record-keeping and regular program reviews help to ensure the continuing appropriateness of governance and program administration arrangements. It is particularly important for departments assigned responsibility for continuing programs to undertake a ‘health check’ at the point of transfer, to ensure the key governance elements in place are appropriate and operating effectively.

21. The ANAO has made two recommendations to enhance the administration of grant programs in those agencies that have had responsibility for implementing aspects of the Smart Grid, Smart City Program. The first recommendation (directed to the Department of Industry) relates to measuring and reporting program achievement against established objectives. The second recommendation (directed to the Department of the Environment) relates to implementing appropriate probity arrangements, and appropriately documenting grant assessment and selection processes.

Key findings by chapter

Program planning and design (Chapter 2)

22. The Smart Grid, Smart City Program was announced by the then Government in the 2009–10 Federal Budget, with limited input from the agency tasked with its design and implementation (DEWHA). DEWHA’s immediate priority once it was assigned responsibility for the program was to establish administrative arrangements, including the commissioning of a pre‑deployment study. DEWHA managed this process effectively, drawing on relevant expertise and taking into account a range of stakeholder views. The pre-deployment study informed the subsequent detailed design of the Smart Grid, Smart City Program, including the preparation of program guidelines (the grant guidelines).

23. DEWHA generally met the requirements of the Commonwealth Grant Guidelines (CGGs) for approving the program’s guidelines (although the department advised that a signed approval was not retained). Stakeholder feedback was also considered prior to the finalisation of the guidelines.⁹ The guidelines provided relevant information for potential applicants, such as the program’s objectives, weighted selection criteria and an indicative timeline for the assessment and selection process. However, there was scope to further clarify several eligibility criteria (for example, the criteria used to determine a ‘commercial-scale’ project or a ‘credible’ operational plan). It would also have been helpful to potential applicants if more information had been provided about the: planned grant assessment and selection process; decision-making arrangements; and complaint/review mechanisms.

Program governance arrangements (Chapter 3)

24. In general, the established governance mechanisms for the program incorporated appropriate oversight arrangements. The Department of the Prime Minister and Cabinet (PM&C) established a Steering Committee for the program, involving senior officials from a range of relevant departments. Several departmental officials interviewed by the ANAO who participated on the committee considered that it was effective in providing high-level guidance for the development of a new program that involved complex technologies. The absence of records of Steering Committee meetings, however, limited the departments’ ability to demonstrate the extent to which the committee discharged its terms of reference, in particular providing oversight of the design and implementation of the program.

25. Oversight arrangements established by the administering departments, including senior management engagement and approval at key stages of the program, helped the program to progress as intended. Once the funding agreement with the preferred applicant, Ausgrid, was signed and the program had commenced, progress was also subject to review by a senior executive committee in RET with oversight of all departmental projects. Departments also established sound risk plans for the program, with evidence of the risks being reviewed and amended in line with developments throughout the life of the program.

26. There was a range of information provided to stakeholders and the general public about activities conducted under the program, including through the establishment of an industry stakeholder committee, presentations at conferences and seminars, and the regular release of Ausgrid reports. However, the absence of a set of outcome-focused key performance indicators (KPIs) has made it difficult for stakeholders to assess the extent to which the Government’s objectives are being achieved. The reporting against the activity‑level KPIs that were included in the administering departments’ Portfolio Budget Statements (PBS) has provided limited progress information for stakeholders. The adoption of more appropriate outcome-focused KPIs (including, where appropriate, intermediate outcome measures), would have better assisted administering departments to communicate the achievements of the program.

Grant assessment and selection (Chapter 4)

27. DEWHA outlined the proposed timeline and grant assessment and selection process in a range of documents, including the pre-deployment study, detailed submissions to government and briefings to the Steering Committee. However, the establishment of a grant assessment plan, which outlined in a single document the manner in which the assessment and selection process was to be implemented, clarified roles and responsibilities and documented probity arrangements, would have better placed the department to demonstrate that it had undertaken a transparent assessment and selection process.

28. Probity processes help to ensure that grant assessment and selection processes are transparent and accountable. While DEWHA appointed a probity adviser for the grant assessment and selection process, the adviser had limited oversight of the process. The probity adviser did not prepare a probity plan or attend assessment panel meetings. Strengthening arrangements for the management of conflicts of interest, and confirmation from the adviser that the assessment and selection process conformed to the published guidelines would also have provided DEWHA with a greater level of assurance regarding the probity of the process.

29. DEWHA sought to implement an assessment and selection process that drew on a range of expertise, as had been recommended in the pre‑deployment study. This involved: an eligibility assessment of the four grant applications by departmental staff; a concurrent assessment by a 17‑member Expert Panel drawn from across government and five private sector companies; and a separate assessment by an Independent Assessment Panel (IAP) comprising six members with a range of relevant expertise.

30. The Expert Panel review process was complex, involving the assessment of each application against the five high-level selection criteria (including 32 sub-criteria, which were allocated to Expert Panel members based on their relevant expertise). A number of the 17 assessors on the Expert Panel did not, however, provide an assessment against all the criteria assigned to them.¹⁰ Further, the 32 sub-criteria used to assess applications had not been clearly identified in the program guidelines and associated guidance documents.

31. The merit assessment process included each IAP member undertaking an assessment of each grant application, against the five published selection criteria. These individual assessments were not, however, retained by DEHWA. The Chair of the IAP advised the ANAO that the panel subsequently made a joint assessment of each application against the five selection criteria, taking into account the findings of the Expert Panel. However, documentation, including the minutes of the IAP meetings and its final selection report, did not clearly evidence this assessment. The IAP also undertook a secondary assessment of the two highest ranked applicants, intended to provide additional assurance regarding their suitability to deliver against the objectives established for the program. There would have been merit in outlining the potential for an additional assessment in the program guidelines and internal guidance materials.

32. Appropriate advice was provided to the relevant Minister and the Government on the grant assessment process and the recommendation of a preferred applicant. On the basis of this advice, the Minister and the Government supported the panel’s recommendation, and the relevant departmental official subsequently approved the spending proposal of $93 million (as required under Regulation 9 of the Financial Management and Accountability Act 1997). The decision-maker (approver) for the grant was not, however, clearly identified in internal planning documents or published guidance materials. Further, the roles of the Minister and Government in the approval process were not clearly defined, with various briefs and submissions to Ministers and Government seeking ‘approval’ or ‘agreement to’ the selection of the preferred applicant, approval to announce the preferred applicant, and approval of a spending proposal (which would be the form of words generally expected for a Regulation 9 approval). There is merit in clearly identifying decision-making arrangements for grants programs, to avoid misunderstandings and to ensure transparency and accountability in the process.

Managing the funding agreement (Chapter 5)

33. The funding agreement negotiated by the Australian Government and Ausgrid established detailed project deliverables to be achieved by Ausgrid. It also established a clear framework for reporting, releasing payments and managing the relationship between the two parties. The administering department (the then Department of Resources, Energy and Tourism—RET) and Ausgrid developed a productive and professional working relationship, with RET’s detailed review of Ausgrid’s reporting resulting in improved information and data being provided over the life of the funding agreement.

34. In general, the majority of the grant milestone payments were made in accordance with the funding agreement requirements and were appropriately documented. There were, however, some cases where records did not clearly indicate the final acceptance and approval of deliverables, such as reports from Ausgrid.

35. In June 2012, RET made an early payment to Ausgrid for several milestones that were yet to be achieved. The department informed the ANAO that the payment was made with the intention of ensuring that the Australian Government could meet its obligations under the funding agreement.¹¹ However, while carefully considered by RET, this approach was not in keeping with generally accepted principles of sound program management and presented a number of risks to RET’s effective management of the agreement—in particular its ability to manage potential under-performance by Ausgrid, by withholding future grant payments.

36. RET’s monitoring of Ausgrid’s compliance with its obligations under the funding agreement relied primarily on regular reports from Ausgrid (as had been set out in the funding agreement). The department’s review of these reports was informed by technical advisers engaged to provide assurance over a range of activities and projects being reported by Ausgrid. The compliance arrangements established for the program through the funding agreement were not, however, supported by a documented compliance strategy. The early development of a compliance strategy by DEWHA during the design phase, which identified key compliance risks and outlined a framework to address these risks, would have informed potential applicants of their compliance obligations and better placed RET to establish and monitor compliance under the funding agreement.

Program achievements (Chapter 6)

37. The objectives of the Smart Grid, Smart City Program reflected the intention of the Government for the program to provide useful and valid data to inform a potential broader rollout of smart grid technologies in Australia. The program was also intended to identify the issues that would need to be addressed by both government and industry to help achieve this objective.

38. A final report from Ausgrid to the Government outlining the detailed findings of the program is due to be published in early 2014. Internal and external reporting for the Smart Grid, Smart City Program, available as at October 2013, indicated that the majority of projects had been completed in line with the funding agreement requirements. These projects had gathered significant data and information to support further development of smart grids in Australia.

39. Some elements of the program did not, however, progress as originally envisaged. Ausgrid’s grant application and the funding agreement had foreshadowed a retail trial involving ‘up to’ 20 000 participants. A trial design study commissioned early in the program by Ausgrid, and verified by RET consultants, reduced the target to a minimum of 4453 (with a ‘stretch target’ of 8333). However, challenges in the retail trial implementation included customer resistance to the technologies¹², delayed availability of appropriate smart meters, technological issues and difficulties in securing an electricity retail partner for the trial. As a result, the final number of participants in the retail trial fell just short of the minimum target (at 4000 participants), and the trial was not fully in place for the optimal two summers (to gather the most comprehensive range of data).

40. Notwithstanding the reduced number of participants, the planned cost of the trial remained unchanged, primarily due to increased implementation costs. The estimated cost of the retail trial (around $20 million—or $5000 per participating customer) accounted for just over one-fifth of the total expected expenditure for the program. The retail trial did, however, help to identify some of the issues that government and industry will need to consider if implementing a broader rollout of smart meters and their associated technologies.

Summary of agency responses

41. The Department of the Environment’s and the Department of Industry’s summary responses to the proposed report are provided below, with their full responses at Appendix 1:

Department of the Environment

The Department of the Environment notes the ANAO’s findings that overall, the administering departments established appropriate arrangements to support the implementation of the Smart Grid, Smart City program and is grateful for the opportunity to respond to the audit report. The Department also notes that the audit has made recommendations for improvement of some aspects of the grants administration process and these recommendations will be incorporated into current and future grants programs.

Department of Industry

The Department welcomes the conclusion of the Australian National Audit Office that overall the four administering departments of this program established appropriate arrangements to support the implementation of the Smart Grid, Smart City Program.

The Department of Industry agrees with Recommendation 1. As noted by the ANAO in the report, the Department provided extensive information on the progress and outcomes of the program to stakeholders through a range of communication channels. The Department acknowledges that it is best practice to ensure indicators report on the extent to which the programmes objectives and outcomes are being achieved.

Recommendations

Footnotes

[1] Australian Government, Budget Paper No.2: 2009–10, Expenses Measures for the Department of the Environment, Water, Heritage and the Arts, Commonwealth of Australia, Canberra, 2009, p. 199.

[2] In December 2013, the Department of Industry advised that decommissioning costs for each of the projects were likely to exceed any profit made from the sale of Smart Grid, Smart City assets.

[3] As part of revised administrative arrangements, DCCEE was abolished by the previous Government in March 2013.

[4] As part of revised administrative arrangements, RET was abolished by the current Government in September 2013.

[5] In a number of cases, information reported by Ausgrid has been verified by technical advisers engaged by the former Department of Resources, Energy and Tourism.

[6] This trial, which is separate from the network trial outlined earlier, involved new electricity tariffs in addition to feedback technologies facilitated by smart meters. For example, one retail trial tariff involved participants paying for their electricity use on a ‘time of use’ basis, with participants able to monitor their electricity use and costs in real-time over the internet.

[7] The program guidelines and funding agreement had stipulated that an electricity retail provider must be included in the consortium to deliver the program. This was because electricity retailers ‘owned’ the customer relationships, which would be important to the success of the retail trial.

[8] The stated roles and responsibilities of the Independent Assessment Panel (IAP) did not clearly outline whether these records should have been destroyed or returned to DEWHA. The Terms of Reference for the IAP stated ‘all copies and notes prepared in the course of the assessment should be destroyed’. However, the contract for each Panel member stated that IAP members must deliver all Contract Material to the department when the contract ended. Contract material was defined as 'any material created for the purposes of this contract, provided or required to be provided to the Department as part of the Services, or derived at any time from the material'.

[9] The grant application process was accessible to a small pool of eligible applicants (the 16 electricity distribution network service providers in Australia). These providers were aware of the announcement of the program in the 2009–10 Federal Budget and had participated in the stakeholder consultations undertaken to inform the pre-deployment study.

[10] The Department of the Environment advised that it tested the final averaged scores and concluded that it was highly unlikely that the missing assessments would have changed the order in which the applications were ranked by the Expert Panel.

[11] In general, agencies that have not spent program funding in a financial year can seek approval from government to reallocate the funding to the next financial year. However, in 2012–13, the then Government determined that such reallocation would only occur in exceptional circumstances, with all other savings to be returned to government. On this basis, RET considered that there was a risk that funding would not be available in 2013–14 to meet the Government’s commitments under the funding agreement. To address this risk, RET approved the release of the early payment.

[12] Ausgrid experienced a high customer drop-out rate after initial sign-up to the trial (up to 20 per cent). As at November 2013, Ausgrid was yet to conduct detailed surveys to determine causes for the drop‑outs.