Introduction

1. Water resource policies in Australia have historically been focused on promoting economic development and employment. In 1994, the Council of Australian Governments (COAG) established a water reform framework to address the degradation of water resources and improve efficiency and sustainability in the water industry. In June 2004, following continuing drought conditions, COAG endorsed the National Water Initiative (NWI), in which all governments indicated a commitment to a range of measures to increase the efficiency of Australia’s water use.

2. By 2006, concerns about water security were being expressed at the highest levels of government, with Premiers, Chief Ministers and the then Prime Minister meeting to discuss emergency measures for water supply in the Murray–Darling Basin.¹ At the time, however, the development of an appropriate and proportional policy response was constrained by poor quality information on water resources. An alliance, led by the Commonwealth Scientific and Industrial Research Organisation (CSIRO) and involving industry, research organisations and Australian Government agencies, was formed to review the availability of information on water resources. The alliance found significant limitations on the availability, comparability and quality of water information. Many datasets that were expected to be available were not and obtaining data required entering into restrictive licensing arrangements with some jurisdictions.

3. In January 2007, the then Prime Minister announced a $10 billion National Plan for Water Security, including a $480 million investment to improve water information.² Subsequently, the Australian Government announced the $12.9 billion Water for the Future initiative in May 2008, which maintained the investment in improving water information. While the drought conditions eased from the autumn of 2010, Australia’s climatic variability suggests a continuing need for water information to assist governments and communities in managing climate risks.

Improving Water Information Program

4. The Improving Water Information Program was designed to enhance the quality of water information systems in Australia. The Water Act 2007 (the Act), which establishes the legal framework for the program, came into effect in March 2008 and provides the Australian Government with a specific mandate to enable water resources to be managed in the national interest. The objects and provisions of the Act are designed to enable the Government, in conjunction with the Murray–Darling Basin states, to manage Basin water resources in the national interest. It also provides for the collection, collation, analysis and dissemination of information about Australia’s water resources and the use and management of water in Australia.

5. The Water Regulations 2008, which were established under the Act and came into effect on 30 June 2008, define those organisations that must give specified water information to the Bureau, and the timeframe and format in which this information must be given. As at October 2013, 232 water organisations were named under the Regulations. Typically, these organisations are state and territory water agencies, other state or Australian Government agencies, hydroelectricity generators, major water storage owners or operators, rural or urban water utilities, catchment management authorities and local councils.

6. The Bureau of Meteorology (the Bureau) was given responsibility for the administration of the Improving Water Information Program, with funding of $450 million that was originally budgeted over a 10‑year period from 2007.

7. The program funds were primarily to cover Bureau expenses such as staffing, accommodation and information technology systems to: enable the collection and harmonisation of water data; produce new water-related products; provide improved forecasting services; and expand the data available to improve policy and infrastructure decisions and evaluation. There was also an allocation of $80 million in administered funding to assist water data providers to strengthen water monitoring arrangements through the Modernisation and Extension of Hydrologic Monitoring Systems Program (M&E Program). The Bureau provided funding of $78.1 million for 463 projects over five rounds of the M&E Program from 2007–08 to 2011–12. The majority of funding was allocated to projects that focused on modernising and extending monitoring equipment and networks and improving water data management systems and the quality and accuracy of water data.

8. The water information products and services developed under the Improving Water Information Program were intended to improve the understanding of water availability, storage and flows in the landscape and to answer questions such as:

• How much water is available today, and how does that compare with the past?
• How is the quantity and quality of water in our rivers and aquifers changing? and
• What are the hydrologic impacts of land management changes and climate change?

9. Since the commencement of the program, the Bureau has released a range of new water information products and services, including: annual national water accounts; water resources assessments; water storage information; and seasonal streamflow forecasts.

Administrative arrangements

10. In 2007, water information was largely a new function within a new division of the Bureau focusing on the information shortfall that was constraining water management reform in Australia. The Bureau expected to address information shortfalls by:

• using the authority of the Act to obtain water data;
• establishing partnerships with research organisations (such as the CSIRO);
• enhancing water monitoring systems and data quality through financial assistance to water data providers across Australia; and
• developing and embedding national standards for water measurement.

11. A key consideration for the Bureau was that the vast majority of streamflow monitoring gauges for flood and water resources are the responsibility of the states, territories and local government. Consequently, the Bureau was largely dependent upon state and territory agencies, local government and other parties for critical data to meet the requirements of the Act.

Audit objective and criteria

Objective

12. The objective of the audit was to assess the effectiveness of the Bureau of Meteorology’s implementation of the Improving Water Information Program.

Criteria

13. To form a conclusion against this audit objective, the ANAO adopted the following high-level criteria:

• sound planning processes and governance arrangements were established;
• effective arrangements for collaborating with water organisations and providing financial assistance to water data providers were developed; and
• arrangements for collecting and managing water data and for producing high-quality water information products were appropriate.

Overall conclusion

14. The Improving Water Information Program is a relatively new and complex area of activity for the Commonwealth. The program was introduced in 2007 as a key element of the wider national water reforms designed to improve water management in Australia. At the time, governments in Australia were under considerable pressure to address water supply problems that were accentuated by prolonged drought conditions. However, effective policy responses were constrained by poor water information and a lack of nationally consistent data in areas such as water availability, allocations and entitlements.

15. In the six years that the program has been in place, the Bureau has expended $186 million and collected more than 21 million water data files containing more than four billion time‑series observations. This data covers thousands of water monitoring sites. From this data and meteorological information, the Bureau has introduced a range of new products and services that have improved the comparability and quality of available water information. These have included: annual national water accounts; water resources assessments; tracking of water storages; and seasonal streamflow forecasts.

16. The Bureau has developed effective arrangements for collaborating with water data providers that supply much of this data, with these providers generally complying with the legislative provisions. High participation in data licensing arrangements has also helped to maximise the utilisation of the Bureau’s water data by third parties and increased the availability of water data to the Australian community.

17. In addition, the Bureau has improved the collection of water information nationally through its collaboration with, and financial assistance to, water data providers. The Modernisation and Extension of Hydrologic Monitoring Systems Program (M&E Program) delivered financial assistance to eligible data providers to assist them in modernising and extending their hydrologic monitoring networks. The Bureau received a total of 789 applications across the five rounds of the program. Of the eligible applications, 60 per cent were awarded funding that totalled $78.1 million. The majority of funding was allocated to projects that focused on modernising and extending monitoring equipment and networks and improving water data management systems and the quality and accuracy of water data. The Bureau effectively administered the M&E Program, with funded activities collectively contributing to improved accuracy and quality of water data and better equipping policy‑makers to manage Australia’s water resources.

18. While stakeholders generally viewed the program and the effectiveness of the Bureau’s implementation positively, there has been a gap between the expectations of users and the range and completeness of the Bureau’s products and services currently provided. Stakeholders are seeking increased coverage and better quality products and services, including data downloads, so that they can address their own specific product and service needs. Services, such as data downloads, were included as a priority in the Bureau’s 2008 strategic plan for improving water information, but have yet to be fully delivered. While the Bureau’s forward work program is designed to address a number of these concerns, closer consultation with key agencies through established forums (such as the Jurisdictional Reference Group on Water Information) would further assist in managing expectations.

19. A key constraint on the effectiveness of the program’s implementation and the capacity of the Bureau to meet expectations has been the limited functionality available through the system designed to manage national water data—the Australian Water Resources Information System (AWRIS). The development of AWRIS was a key program objective and was fundamental to the Bureau delivering improved national water information. The functionality of AWRIS is severely limited and this has constrained the range and timely development and release of new products and services. Overall, the development of AWRIS was characterised by technical and governance shortcomings, changes in design and approach, unanticipated cost increases and delays. As a consequence, the Bureau has not achieved its vision for AWRIS as a reliable, national repository for water information. Further, the level of expenditure has not been proportional with the level of functionality obtained, with the Bureau expending $38.5 million on AWRIS and associated systems and applications as at 30 June 2013.

20. The issues encountered by the Bureau in this information technology implementation emphasise the importance of agencies understanding their business environment and the likely operational risks and challenges they will be facing when developing new systems. Clarity as to the requirements of users is important, along with the recognition that these may evolve or change over time requiring enhancements to functionality over and above planned business as usual processes. In the case of AWRIS 1, clearly defining business and system requirements and establishing governance arrangements that were commensurate with the risk profile of the project, would have better positioned the Bureau to develop and deliver a system with greater functionality within more reasonable timeframes. The limited functionality that led to the decision to decommission the data warehouse component of AWRIS 1, which is estimated to have cost $12.5 million, raises questions regarding the value for money achieved from the investment in new information technology.

21. Determining the extent to which the Improving Water Information Program is achieving its objectives has been affected by changes in performance measures over the course of program implementation. The program’s early key performance indicators (KPIs) were broad and difficult to measure. While the KPIs have become more measurable over time, the program’s current KPIs do not readily inform an assessment of the extent to which the program is achieving its outcomes. Having a set of specific, measurable and consistent KPIs that provide the basis for reporting on the program would better position the Bureau to inform stakeholders of program progress and the challenges involved in achieving program outcomes.

22. The ANAO has made two recommendations designed to strengthen the development and management of information technology systems and improve the measurement and reporting of program performance.

Key findings by chapter

Planning, oversight and reporting (Chapter 2)

23. Following the introduction of the Improving Water Information Program in 2007–08, the Bureau has developed and implemented planning processes to assist it to meet the requirements of the Water Act 2007 and the policy outcomes expected by government. Management and advisory structures established by the Bureau were appropriate and consistent with a collaborative model of service delivery. The technical and scientific challenges in building capability in data management and producing a suite of water information products and services were substantial, requiring investment in applied research with a range of partner agencies, such as the CSIRO and the National Water Commission.

24. An appropriate approach to stakeholder communication and engagement has been established and will be enhanced through an overarching stakeholder framework planned for the period 2013–17. While the Bureau gave appropriate and early attention to risk management and compliance, some risks, such as a failure to develop effective systems for web‑based delivery and data management, were underestimated. This risk was realised with functionality constraints adversely impacting on the Australian Water Resources Information System (AWRIS).

25. The program’s early KPIs were broad and difficult to measure, but have become more specific and measurable over time. However, the program’s current KPIs do not readily inform an assessment of the extent to which the program is achieving its outcomes. In addition, the performance indicators for the program have changed over time making it difficult to gain an understanding of program performance over the six years of the program. Having a set of specific, measurable and consistent KPIs from year to year would better position the Bureau to inform stakeholders of program progress. A greater coverage of the challenges and constraints in progress reports would also assist stakeholders to better understand the implementation issues facing the Bureau.

Collaboration and compliance management (Chapter 3)

26. The Bureau has provided sufficient guidance to water organisations on their responsibilities under the Water Act 2007 and the Water Regulations 2008, with water organisations generally expressing satisfaction with the level of communication and guidance received from the Bureau.

27. The development of national water information standards is continuing, with the Bureau engaging with stakeholders through the Water Information Standards Business Forum. The Bureau has taken a three‑tiered approach to standards development, with a small number of standards to be mandated and others to be adopted on a voluntary basis. The development of standards and guidelines assists with harmonising water data collection, analysis and reporting across Australia and ultimately improves the quality of the data provided to the Bureau.

28. The licensing of water data under Creative Commons Attribution licences³ has been effective in allowing users to freely copy, distribute, transmit and adapt water data. The Bureau has negotiated with water data providers and, as at October 2013, had achieved a 90 per cent participation rate in the Creative Commons Attribution licensing arrangements. The Bureau’s achievement of high participation in these licensing arrangements has helped maximise the utilisation of the Bureau’s water data by third parties and has increased the availability of water data to the Australian community.

29. The Bureau has developed and implemented a compliance strategy and an escalated response policy to manage identified non‑compliance by water data providers. Initially, the focus of compliance activities was on establishing a data-provision relationship with each named organisation. Since the commencement of the Regulations, there have been only three cases of non‑compliance identified by the Bureau where a formal response has been required. In each case, the compliance issues were resolved after the first formal response level and no further escalation was required. As at October 2013, all of the non‑exempt named organisations had provided data to the Bureau.

Financial assistance to water data providers (Chapter 4)

30. The M&E Program was a key component of the implementation of the Improving Water Information Program as it provided grant funding to enable data providers to modernise and extend their hydrologic monitoring networks and to improve the accuracy and quality of the water data submitted to the Bureau. The Bureau provided appropriate information to potential applicants and published funding guidelines for each round of financial assistance under the M&E Program. These guidelines provided detailed information for stakeholders and generally aligned with the requirements of the Commonwealth Grant Guidelines. The assessment and recommendation of applications was conducted in an open and accountable manner, with assessment documentation generally being retained and the decision‑maker making reasonable inquiries before approving funding.

31. A robust monitoring regime was established to assist the Bureau to determine if grant recipients had met agreed milestones, and the arrangements established by the Bureau to manage grant payments were generally appropriate. In some cases, however, the extent of monitoring was not proportionate to the amount of funding provided. The monitoring arrangements could have been more streamlined within the context of a risk‑based approach.

32. The M&E Program has assisted grant recipients to modernise and extend their water monitoring systems and improve the accuracy, quality and frequency of the data transferred to the Bureau. In particular, the replacement of obsolete mechanical water gauges with electronic systems and the upgrading of water data management and transfer systems have enhanced the Bureau’s access to more timely and accurate water information from data providers.

Data systems, collection and management (Chapter 5)

33. The development of the Australian Water Resources Information System (AWRIS) was a key objective of the Improving Water Information Program. AWRIS was to enable the Bureau to receive, store and manage national water data and to underpin the delivery of high-quality water information. The Bureau is receiving approximately 10 000 new data files containing time-series observations each day. It has been managing more than 21 million water data files containing more than four billion time‑series observations since the Water Regulations came into effect on 30 June 2008. This data relates to 105 parameters across 10 categories and covers thousands of hydrologic monitoring sites.

34. While the Bureau has been receiving and cataloguing this data, there have been major challenges and constraints in using AWRIS to manage the data for the production of new products and services. The development of AWRIS has been problematic with unclear business and system requirements, inadequate technical solutions, shortcomings in governance arrangements, changes in design and approach and unanticipated costs and delays that have limited the functionality of the system.

35. At the outset, project planning, governance and management did not follow established IT system design, development or implementation practices. Decisions were also taken that increased the complexity of the system to such an extent that it is difficult to maintain and is constrained in its functionality. Further, the level of expenditure on AWRIS, particularly when contrasted with planned expenditure and functionality has also raised questions regarding the value for money achieved. As at 30 June 2013, the Bureau had expended $38.5 million on AWRIS and associated IT systems and applications. This included $23.2 million that has been capitalised and $15.3 million in operational expenses. The Bureau has advised that, in the preparation of its financial statements, the accounting treatment for the $12.5 million in expenditure capitalised during the development of the AWRIS 1 data warehouse will be reviewed to take into account its limited functionality and early decommissioning.

36. The achievement of operational requirements for a national information technology system with maximum interoperability and flexibility has not been achieved. There have been significant consequences with delays in the rollout of products and services and ‘work-arounds’ to enable key products and services to be produced. In the case of AWRIS 1, clearly defining business and system requirements and establishing governance arrangements commensurate with the risk profile of the project, would have better positioned the Bureau to develop and deliver a system with greater functionality within more reasonable timeframes.

37. The decision to decommission AWRIS 1 and build a new AWRIS 2 system has enabled the Bureau to review and enhance its approach to IT development, with AWRIS 2 based on accepted industry standards for data warehousing. On the basis of the results of early testing, AWRIS 2 is progressing towards its performance and functionality targets for data ingest and storage. However, the effective implementation of AWRIS 2, planned for December 2014, and the expected enhancement to functionality will depend upon a clear understanding of user requirements, strong strategic IT planning, executive oversight and project management.

38. The comprehensive data quality approach originally envisaged for end‑to-end water data collection, management and analysis, has not yet been realised. The challenge of standardising data from a large number of sources, and with considerable variation, has been a significant task even after six years of program implementation. While the Bureau has been progressively introducing improvements to data quality, extensive and time-consuming product-specific manual checking on the quality of data remains necessary because of the limitations of AWRIS. Extension of the Water Data Transfer Format (a standardised data transfer format developed by the Bureau and the CSIRO for water data providers)⁴ would assist the Bureau to achieve greater consistency. However, an ongoing risk for the Bureau relates to decisions by data providers to reduce the number of monitoring stations that they maintain. Deterioration in the monitoring network has the potential to affect the capacity of the Bureau to maintain or enhance data quality over time. This risk will need to be considered in the context of the future directions of the program.

Water information products and services (Chapter 6)

39. Since the inception of the Improving Water Information Program in July 2007, the Bureau has introduced a suite of new products and services with nationally consistent data, such as annual national water accounts, water resources assessments, water storage information and seasonal streamflow forecasts. As a result, a broader range of better quality water information is available.

40. However, most products have been introduced later than originally planned by the Bureau with varying degrees of coverage and completeness. There have also been considerable challenges in managing data from approximately 200 providers, with the Bureau needing to undertake or commission research to develop its products and services because of the absence of acceptable models that could be readily applied. Further, as outlined earlier, the limitations of the Bureau’s information technology system as a national repository for water information has contributed to delays in the release of new products. Although the Australian Water Resources Assessment has national coverage, the Bureau does not envisage full national coverage for the National Water Account (NWA). The Bureau has advised that the capacity of reporting partners, the availability of data and resources and the priorities of stakeholders have determined the order in which regions have been included in the NWA.

41. The level of collaboration with many different agencies across Australia reflects well on the Bureau’s approach and suggests a wide degree of commitment to the program and its products and services. Although not complete, the Bureau’s current suite of water information products and services provide governments with important data to inform better policy decisions in relation to water services and infrastructure investment.

42. In general, stakeholders have indicated a positive view of the Improving Water Information Program. There is, nevertheless, a gap between the expectations of users and the capacity of the Bureau to deliver. Services such as data downloads, were a priority in the Bureau’s 2008 strategic plan for improving water information, but have yet to be fully delivered. Stakeholders have also suggested a need to increase the coverage and quality of products and services available. While the Bureau’s forward work program has been designed to address a number of these concerns, closer consultation with key agencies through established forums would further assist in managing expectations.

Summary of Bureau response

43. The Bureau’s summary response to the proposed report is provided below, with the full response at Appendix 1:

The Bureau welcomes the Audit Report and is implementing the two recommendations:

• Key performance indicators have been redesigned, at enterprise and program levels; reported to management and in the Annual Report. Major IT projects (including AWRIS) are now identified as major deliverables in the Bureau’s operational plan and progress reported to senior management.
• From July 2013, the Bureau established an Information Systems and Services Division to enhance capability and increase flexibility and responsiveness. An enterprise-level Portfolio Management Board is being implemented to enhance project governance and provide regular reviews of the status, cost and overall progress.

The Bureau was pleased with the ANAO assessment that the Bureau has developed effective arrangements for collaboration with water data suppliers, and will continue engagement with stakeholders to deliver product and services and manage expectations having regard to available resources.

Recommendations

Footnotes

[1] Vertessy, RA, ‘Water information services for Australians’, Australian Journal of Water Resources, 16 (2), 2013.

[2] This was subsequently reduced to $450 million.

[3] Creative Commons Attribution licences allow material to be copied, distributed and adapted as long as the ownership of the data is attributed to the data provider.

[4] The Water Data Transfer Format is currently used by 66 of the 232 named water organisations.