Background

1. As a party to the United Nations Framework Convention on Climate Change (UNFCCC) and subsidiary agreements, Australia has agreed to meet targets to reduce human-induced greenhouse gas (GHG) emissions. Measuring and tracking greenhouse gas emissions and removals and projecting future emission levels assists the Australian Government to:

• meet its international reporting obligations;
• monitor progress towards achieving its emission reduction commitments; and
• develop and implement policies and programs to meet emissions reduction commitments.

2. The Department of the Environment and Energy (the department) is responsible for accounting and reporting Australia’s past, and projecting future, GHG emissions. Emissions estimates¹ and projections are calculated using published and unpublished data sourced from government and non-government entities or from modelling by external providers.

3. Australia is required to submit annual estimates of its past GHG emissions—termed national or GHG inventory estimates—to the UNFCCC across five sectors: Energy; Industrial Processes and Product Use; Agriculture; Land Use, Land Use Change and Forestry (LULUCF); and Waste. The latest National Inventory Report 2014 (revised) was submitted to the UNFCCC in August 2016. Australia also submits projections of future greenhouse gas emissions to the UNFCCC as part of National Communications (every four years) and Biennial Reports (every two years), with the most recent Biennial Report submitted to the UNFCCC in December 2015.² In addition, the department recalculates or updates Australia’s emissions projections annually, with the latest report on GHG emissions projections to 2030 published in December 2016.

4. Each country’s National Inventory Reports (and supporting data), National Communications and Biennial Reports are subject to a UNFCCC technical review. These reports and the UNFCCC technical review results are published on the UNFCCC’s website (unfccc.int).³

Audit approach

5. The objective of the audit was to assess the effectiveness of the Department of the Environment and Energy’s arrangements for the preparation and reporting of Australia’s greenhouse gas emissions estimates and projections. To form a conclusion against this objective, the ANAO adopted the following high-level criteria:

• Were robust processes established to prepare and report emissions estimates and projections?
• Were sound arrangements in place to support the preparation and reporting of emissions estimates and projections?

6. Providing opinions on the accuracy of the reported GHG emissions estimates and projections contained in the National Inventory Report 2014 (revised) and December 2016 projections report, respectively, was not within the scope of the audit.

Conclusion

7. The arrangements established by the department for the preparation and reporting of Australia’s greenhouse gas emissions estimates and projections were largely effective.

8. Appropriate processes have been established to prepare, calculate and publish Australia’s greenhouse gas emissions estimates to June 2014 and emissions projections to 2030. The emissions estimates contained in the National Inventory Report 2014 (revised) and the December 2016 projections report have been calculated using relevant contemporary data. Appropriate quality assurance and control procedures are in place for the preparation of most of the emissions estimates and projections, but could be: better applied in relation to data entry to improve inventory accuracy and completeness; and expanded to better encompass the estimates projections for all sectors and abatement measures. The publication of additional key input data, assumptions, formulas and methods would increase the projections’ transparency and utility to stakeholders and users. On the whole, the quality of Australia’s inventory compares well to the inventories of other Annex I (developed) countries.

9. Governance arrangements for the preparation and reporting of inventory estimates and emissions projections are generally effective, with the exception of risk management which requires strengthening. Monitoring arrangements have facilitated the timely preparation and reporting of inventory estimates and emissions projections that met UNFCCC submission deadlines. The department has engaged stakeholders throughout the preparation and reporting process and significantly improved the efficiency of inventory estimates preparation and reporting over recent years.

Supporting findings

Estimates of Australia’s past greenhouse gas emissions

10. The reporting of inventory estimates to June 2014 by the department was based on relevant data, with no material errors identified in the testing undertaken by the ANAO. The UNFCCC has also assessed Australia’s National Inventory Report 2014 (revised) and supporting information as consistent, timely and mostly complete and not requiring adjustment. While the IT systems that support the preparation of Australia’s annual GHG inventory are generally effective, controls could be improved to reduce risks to data accuracy and security.

11. Over three-quarters of the recommendations from the UNFCCC’s 2015 technical review of Australia’s GHG inventories had been resolved by the time the following year’s technical review was undertaken. All recommendations outstanding from earlier technical reviews had been satisfactorily resolved by the department.

12. The department has developed generally appropriate quality assurance and control procedures over the preparation of inventory estimates. Limited documentation, however, makes it more difficult for the department to demonstrate its implementation of procedures related to data entry into the Australian Greenhouse Emissions Information System (AGEIS). In addition to better documenting its implementation of these quality control procedures, there is scope to enhance inventory transparency by improving referencing to source data.

13. The quality of Australia’s inventory estimates has generally improved over time. Inventory improvement projects, most of which are planned in response to UNFCCC technical reviews of the GHG inventory, are being implemented—albeit in a less timely manner than originally envisaged. The quality of Australia’s inventory compares well to other countries’ inventories on most parameters examined by the ANAO, with improvement also in the quality of Australia’s inventory over time in respect of some of the parameters.

Projections of Australia’s future greenhouse gas emissions

14. The most recent emissions projections (December 2016) are based on relevant data from contemporary sources. Projection calculations contained in spreadsheets examined by the ANAO are operating as intended with the exception of a small number of data and formula errors that had an immaterial impact on emissions projections results.

15. The relevant Biennial Report review recommendation from the UNFCCC technical review related to emissions projections has been addressed by the department in advance of the next Biennial Report, primarily by the inclusion of emissions projections to 2030 in the December 2016 projections report.

16. The department has instituted generally appropriate quality assurance and control procedures over the preparation of emissions projections for all sectors apart from the Land Use, Land Use Change and Forestry (LULUCF) sector where there is scope for improvement. Better documentation of the rationale behind the application of judgment and the quality assurance testing undertaken would enhance the transparency of the LULUCF sector projections. In addition, expanding quality assurance procedures for abatement measure projections, where calculated separately to sector projections, would provide greater assurance regarding their accuracy and robustness.

17. The December 2016 emissions projections have improved in quality on past emissions projections in some areas, but not others. The inclusion of sensitivity analyses has increased the robustness of the projections by forecasting quantitative emission ranges from changes to some key projection assumptions. Nevertheless, the latest projections include less quantitative information than earlier projections about the impact that Australian Government abatement measures are projected to have on future emissions. The utility of the emissions projections would be further improved by the department publicly releasing key data inputs, assumptions, formulas and methods sufficient to allow users to recalculate emissions projections (within a reasonable degree of precision) and adapt them for their own purposes.

Governance of greenhouse gas emissions estimates and projections

18. Appropriate planning documentation has been established by the department to guide aspects of the preparation and reporting of inventory estimates and emissions projections. Nevertheless, refinements to overarching project plans for inventory estimates and emissions projections would strengthen governance arrangements and provide a basis for mitigating the risk to future inventory quality and timeliness from the loss of corporate knowledge due to staff turnover.

19. The department has not retained documentation to demonstrate that risks to preparing and reporting GHG emissions estimates and projections are being actively managed and the implementation of risk treatments monitored. The department’s risk management planning documentation for the preparation and reporting of inventory estimates and emissions projections is insufficient and, in the case of inventory estimates, is incomplete.

20. Effective arrangements are in place to monitor the timeliness of the preparation and reporting of inventory estimates and emissions projections. Inventory estimates and emissions projections have been completed and published within UNFCCC submission deadlines.

21. Data providers and external reviewers of draft inventory estimates and emissions projections have been effectively engaged throughout the data collection, preparation and reporting processes. These stakeholders expressed to the ANAO their general satisfaction with the content and quality of the estimates and projections and made suggestions to enhance the timeliness, consistency and transparency of emissions projections. Obtaining feedback from a broader range of end users would further enhance stakeholder engagement.

22. The overall efficiency of the preparation and reporting of inventory estimates has increased significantly over recent years, while the efficiency of the emissions projections’ preparation and reporting has remained relatively stable.

Recommendations

Summary of entity responses

23. The Department of the Environment and Energy’s summary response to the proposed report is provided below, while its full response is at Appendix 1.

The Department agrees with the recommendations in the report. The Department is grateful for the assistance and cooperation of the Australian National Audit Office in assessing the performance of its greenhouse gas emissions estimation and projections, accounting and reporting systems.

The Department has in place comprehensive and advanced inventory and projections systems that are developed in accordance with Intergovernmental Panel on Climate Change (IPCC) guidelines and rules established under the United Nations Framework Convention on Climate Change (UNFCCC). UNFCCC expert review teams test the inventory and projections, and their related systems, for compliance with these international rules. These expert review team reports are available on the UNFCCC website.

The Department is committed to the continuous improvement of its systems and welcomes the recommendations and suggestions for improvement in this report. The Department takes this opportunity to apologise for its inadvertent breach of the Auditor-General Act 1997 resulting from the inclusion of some findings from the ANAO audit report, prior to its tabling in the Parliament, in the National Inventory Report 2015, State and Territory Greenhouse Gas Inventories 2015, and National Inventory by Economic Sector 2015. Once alerted to this serious breach, the Department undertook the following actions:

• The inventory publications were withdrawn from the Department’s website.
• A revised National Inventory Report was submitted to the UNFCCC and substituted for the original version on the UNFCCC website.
• Revised versions of the inventory publications were posted on the Department’s website and sent to State and Territory Government and key inventory user contacts, who were asked to delete or destroy any copies of the original versions.

The Department is also taking steps to ensure that such a breach is not repeated in the future, including through discussion at departmental SES forums and revisions to standard operating procedures. The Audit Office was kept informed of each of the steps outlined here.

Auditor-General’s comment

24. While the Auditor-General is disappointed that the Department of the Environment and Energy breached its confidentiality obligations under the Auditor-General Act 1997, the Auditor-General is satisfied with the department’s actions outlined above to respond in this instance and to minimise the risk of future similar breaches.

Introduction

1.1 As a party to the United Nations Framework Convention on Climate Change (UNFCCC) and subsidiary agreements, Australia has agreed to meet targets to reduce human-induced greenhouse gas (GHG) emissions. Measuring and tracking greenhouse gas emissions and removals and projecting future emission levels assists the Australian Government to:

• meet its international reporting obligations;
• monitor progress towards achieving its emission reduction commitments; and
• develop and implement policies and programs to meet emissions reduction commitments.

1.2 The Department of the Environment and Energy (the department) is responsible for accounting and reporting Australia’s past, and projecting future, GHG emissions.

United Nations Framework Convention on Climate Change

1.3 The UNFCCC was adopted in 1992 at the United Nations ‘Rio Earth Summit’ and has since been ratified by 197 Parties of the United Nations (including Australia). The ultimate objective of the UNFCCC is to stabilise GHG concentrations:

at a level that would prevent dangerous anthropogenic (human induced) interference with the climate system [and that] such a level should be achieved within a time-frame sufficient to allow ecosystems to adapt naturally to climate change, to ensure that food production is not threatened, and to enable economic development to proceed in a sustainable manner.⁴

1.4 In 1997, the Kyoto Protocol was established under the UNFCCC, with its Annex I Parties (developed countries) committing to internationally binding emissions reduction targets over: an initial period from 2008 to 2012; and a second period from 2013 to 2020 under an amendment to the Protocol in 2012.⁵ Subsequently, in December 2015, UNFCCC Parties reached a further agreement to accelerate and intensify the actions and investments needed for a low carbon future over the period from 2020 onwards (the Paris Agreement).

1.5 The key targets for the Australia’s reduction of GHG emissions over the period from 2008 to 2030 are provided at Figure 1.1.

Figure 1.1: Timeline of Australia’s international commitments to reduce greenhouse gas emissions

Note a: Australia’s emissions reduction target under the second commitment period of the Kyoto Protocol is effectively a re-expression of the emissions reduction target under the 2010 UNFCCC Cancun Agreement of a five per cent reduction on 2000 emissions over the period 2013–2020.

Source: ANAO based on departmental information.

Measuring and projecting greenhouse gas emissions

Greenhouse gas emissions estimates

1.6 Australia is required to submit annual estimates of its past GHG emissions—termed national or GHG inventory estimates—to the UNFCCC Secretariat across five sectors: Energy; Industrial Processes and Product Use; Agriculture; Land Use, Land Use Change and Forestry (LULUCF); and Waste.⁶ GHG emissions are estimated using calculations based on methods and models developed by the department to achieve consistency with emission estimation guidance from the Intergovernmental Panel on Climate Change (IPCC) and adopted by the UNFCCC.⁷

1.7 Each country’s National Inventory Reports (and supporting data) are subject to an annual UNFCCC technical review, with National Inventory Reports and the UNFCCC technical review results published on the UNFCCC’s website (unfccc.int).⁸

1.8 The greenhouse gases reported in Australia’s annual GHG inventory include carbon dioxide (CO2); methane (CH4); nitrous oxide (N2O); hydrofluorocarbons (HFCs); perfluorocarbons (PFCs); and sulphur hexafluoride (SF6). Emissions of these greenhouse gases are aggregated after conversion to a common format—carbon dioxide-equivalent (abbreviated to CO2-e)—a measure based upon the gases’ global warming potential over a 100-year period relative to that of carbon dioxide.⁹ Figure 1.2 illustrates Australia’s GHG emissions by gas type measured in million tonnes of CO2-e (Mt CO2-e) as reported in the most recent National Inventory Report.

Figure 1.2: Australia’s GHG emissions by gas type in 2014 (Mt CO2-e)

Source: Adapted from Common Reporting Format tables supporting the National Inventory Report 2014 (revised).

1.9 The National Inventory Report 2014 (revised) submitted to the UNFCCC in August 2016 indicates that Australia’s GHG emissions in 2014 were 524 Mt CO2-e. Figure 1.3 shows the estimated level of GHG emissions and the contribution by sector since 1990.

1.10 Based on Australia’s National Inventory Reports, the UNFCCC determined that Australia surpassed its Kyoto Protocol first commitment period target by 128 Mt CO2-e.

Figure 1.3: Net CO2-e emissions by sector 1990–2014 (Mt CO2-e)

Source: Department of the Environment and Energy, National Inventory Report 2014 (revised) Volume 1.

Greenhouse gas emissions projections

1.11 Australia is required to submit projections of future greenhouse gas emissions to the UNFCCC for review and publication as part of National Communications (every four years) and Biennial Reports (every two years).¹⁰ Australia submitted its most recent Biennial Report to the UNFCCC in December 2015, with the next National Communications and Biennial Report due for submission by 1 January 2018. In addition, the department has recalculated or updated Australia’s emissions projections annually, with the latest report on greenhouse gas emissions projections to 2030 published in December 2016.

1.12 Projections involve judgment and rely on assumptions about the growth of future global and domestic economies, policies and measures, technical innovation and human behaviour. The uncertainty about the accuracy of projections increases the further into the future emissions are projected. In recognition of this uncertainty, the department has included sensitivity analysis in the December 2016 projections report to accompany the ‘baseline’ projections that provide a range for the projections by varying some key assumptions.

1.13 As shown in Figure 1.4, the department’s December 2016 projections estimate that:

• Australia is expected to surpass its Kyoto Protocol second commitment emissions reduction target by 97 Mt CO2-e (which is the net of the two green-shaded periods of target over and under-achievement illustrated in Figure 1.4)¹¹; and
• to meet its first abatement target under the Paris Agreement, Australia will be required to reduce total projected emissions by at least 842 Mt CO2-e over the period 2021–2030 (the orange-shaded area in Figure 1.4).¹²

Figure 1.4: Australia’s emissions trends, 1990–2030

Source: ANAO adapted from Department of the Environment and Energy, Australia’s emissions projections 2016.

Governance arrangements

1.14 The preparation of GHG emissions estimates and projections is the responsibility of two branches—the National Inventory System and International Reporting Branch, and the Climate Change Policy Branch—within the Department of the Environment and Energy. Emissions estimates and projections are calculated using published and unpublished data sourced from government and non-government entities or from modelling by external providers. The department maintains two purpose-built management information systems (AGEIS¹³ and FullCAM¹⁴) to calculate inventory estimates, while GHG emission projections are calculated using spreadsheets.

Audit approach

1.15 The objective of the audit was to assess the effectiveness of the Department of the Environment and Energy’s arrangements for the preparation and reporting of Australia’s greenhouse gas emissions estimates and projections. To form a conclusion against this objective, the ANAO adopted the following high-level criteria:

• Were robust processes established to prepare and report emissions estimates and projections?
• Were sound arrangements in place to support the preparation and reporting of emissions estimates and projections?

1.16 The audit examined arrangements established by the department to prepare the National Inventory Report 2014 (revised) and December 2016 projections report, including governance processes for the reporting process, and the collation and processing of a sample of sub-sectors’ data that collectively comprised greater than 50 per cent of Australia’s annual GHG emissions. Providing opinions on the accuracy of the reported GHG estimates and projections was not within the scope of the audit.

1.17 In conducting the audit, the ANAO examined departmental records relating to the preparation of the estimates and projection reports including: UNFCCC and departmental guides; implementation plans and quality assurance/quality control documents; general governance documentation; and spreadsheets supporting the entry of data into AGEIS and projections calculations. The ANAO also examined key IT controls supporting the AGEIS and FullCAM systems, interviewed departmental staff and sought input from the public and key stakeholders.

1.18 The audit was conducted in accordance with the ANAO’s Auditing Standards at a cost to the ANAO of approximately $459 000.

1.19 The team members for this audit were Grant Caine, Sonya Carter, Amanda Reynolds, Mark Rodrigues and Mark Simpson.

Are inventory estimates based on relevant, authoritative source data?

Examination of source documentation

2.1 The department’s GHG emissions estimates have been calculated using data from national and international public and private sector entities and research institutions, including:

• published and unpublished data from government entities (including the Clean Energy Regulator, the Australian Bureau of Agricultural and Resource Economics and Sciences (ABARES) and the Australian Bureau of Statistics) and industry;
• commissioned, academic and industry research;
• the department’s geospatial data collection systems that collect and process satellite imagery to determine land use changes in collaboration with Geoscience Australia and Commonwealth Scientific and Industrial Research Organisation (CSIRO);
• project data collected under the Emissions Reduction Fund by the Clean Energy Regulator;
• the National Greenhouse and Energy Reporting (Measurement) Determination 2008 (reviewed, and if necessary amended, annually) that contains country-specific default values for some sectors (including energy content factors and emission factors) used in emissions calculations¹⁵; and
• the 2006 IPCC Guidelines for National Greenhouse Gas Inventories, and other supporting guidelines adopted under the UNFCCC and Kyoto Protocol.

2.2 The source data is entered into the Australian Greenhouse Emissions Information System (AGEIS)¹⁶ by the department via spreadsheet-based data entry templates. While activity data (for example, production statistics and quantity of fuel consumed) changes from year-to-year, other data types (typically related to energy content factors and emission factors) tend to be updated less frequently. In some cases, the raw source data requires pre-processing (for example, data aggregation/disaggregation) before it is entered into the AGEIS. In total, some 25 000 pieces of data are entered into the AGEIS each year.¹⁷

2.3 The ANAO examined sub-sector line-items¹⁸ to verify emissions to its source data and ensure the data is captured by formulas within the AGEIS to calculate GHG emissions. Of the 5250 data points across 158 data types entered into the AGEIS for these sub-sectors, all were satisfactorily verified by the ANAO to pre-processing spreadsheets (where applicable) and source documentation except for:

• 32 data points from one data type for one Agriculture sub-sector examined where the department was unable to provide the source documentation to the ANAO; and
• 80 data points from 27 data types across all sub-sectors examined where data errors had occurred due to human error (involving data transposition/transcription, data not being updated, or incorrect values used). The department informed the ANAO that these data errors resulted in a cumulative error across the 2014 GHG emissions estimates of 0.1 per cent. In the context of over 5000 data points and inventory emissions examined, the errors are not material.

Extent to which Australia’s inventory estimates meet UNFCCC requirements

2.4 In August 2016, the department published and submitted to the UNFCCC the most recent National Inventory Report (for emissions estimates to the year ended June 2014). Australia’s National Inventory Report 2014 (revised)¹⁹ contained information on:

• the inventory’s context and background (including framework, governance arrangements, general quality assurance/control measures, common methodologies and data sources, uncertainty evaluation and a general assessment of completeness);
• emission trends (by sector, per capita and per dollar of gross domestic product) and drivers of the trends;
• each emissions sector, including:
  • total emissions and emissions/trends by sub-sector;
  • methodologies (including formulas)/emission factors for calculating key emissions;
  • sector-specific activity data sources and quality assurance/control procedures;
  • uncertainty and time-series consistency analysis;
  • recalculations of past annual sector/sub-sector emissions and their reasons; and
  • planned improvements to future inventory estimates for the sector/sub-sectors; and
• supplementary Kyoto Protocol reporting requirements.²⁰

2.5 Each Annex I country’s annual GHG inventory is the subject of a technical review by an UNFCCC Expert Review Team undertaken in accordance with the UNFCCC guidelines for review of annual inventories. The technical reviews are designed to, among other things, provide ‘a thorough, objective and comprehensive technical review of all aspects of the implementation’ of the UNFCCC’s annual GHG inventory requirements.²¹ Technical reviews comprise:

• an initial assessment of the annual GHG inventory submission by the UNFCCC Secretariat that considers the consistency, completeness and timeliness of the submission; and
• a review of annual GHG inventory by a UNFCCC Expert Review Team²² that assesses the inventory’s contents for accurate, consistent and relevant information prepared and reported in accordance with UNFCCC requirements.²³

2.6 The UNFCCC Secretariat’s technical review determined that Australia’s National Inventory Report 2014 (revised) and supporting data were consistent, timely and mostly complete (64 of 69 tables of data had been provided, with the remainder relating to some required LULUCF sector data). The Expert Review Team did not raise questions of implementation and did not require any adjustments to the contents of Australia’s 2014 GHG inventory. The report of the technical review of Australia’s 2014 GHG inventory also:

• determined the extent to which the 80 improvement recommendations from technical reviews of Australia’s past GHG inventories had been resolved (discussed later from paragraph 2.10); and
• made 24 additional findings and recommendations to improve Australia’s future GHG inventories (of which 18 related to inventory transparency, four related to inventory comparability and two related to inventory accuracy).²⁴

Australian Greenhouse Emissions Information System (AGEIS) and Full Carbon Accounting Model (FullCAM)

2.7 The department develops the GHG emissions estimates using the AGEIS and, for the LULUCF sector, the Full Carbon Accounting Model (FullCAM). The AGEIS has been designed to meet the UNFCCC’s requirements for national inventory systems and is an essential part of the inventory preparation and publishing processes. In particular, AGEIS has been designed to fully integrate quality control procedures into the compilation process, as well as centralising emissions estimation, inventory compilation and reporting, and data storage activities. FullCAM is a specialised system for the LULUCF sector that models complicated biological processes using multiple data sets, including spatial data.

2.8 An effective control environment within the AGEIS and FullCAM, encompassing user and administrator/IT support access, change management and data security, is important for maintaining data integrity and calculating emissions accurately and reliably. The ANAO’s review of the AGEIS and FullCAM found that the department’s control environment was generally effective. While IT control arrangements to support data integrity had been implemented effectively for the AGEIS, there was scope to improve:

• the oversight of user and administrator/IT support access to the AGEIS and FullCAM systems/database by: regularly reviewing those staff that require system access; establishing a means of logging system access by administrators (with privileged access); and rationalising the number of departmental IT support staff with unlogged access to the unencrypted database²⁵; and
• FullCAM’s change management controls by more effectively documenting: testing undertaken prior to system changes being implemented; and the system changes that have been, or are yet to be, implemented in the annual FullCAM production release.²⁶

2.9 The department informed the ANAO that it would take action to improve the IT control environment of AGEIS and FullCAM in the areas identified during the audit. The department further advised that some of systems/data access issues will require an upgrade to the systems’ IT platform.

Has the department responded to inventory estimates review recommendations from the UNFCCC?

2.10 Australia’s implementation of previous recommendations is assessed as part of the UNFCCC Expert Review Team’s technical review of Australia’s annual GHG inventory. The 2015 technical review made 80 recommendations to improve future GHG inventories. Of the 80 inventory improvement recommendations, the 2016 technical review determined that Australia had resolved 61 (76.3 per cent)²⁷, was addressing 12 (15.0 per cent) and had not resolved seven (8.8 per cent). Table 2.1 on the following page outlines the implementation status of the 2015 technical review recommendations by their impact type on the annual GHG inventory as determined by the 2016 UNFCCC technical review.

2.11 Of the 19 recommendations being addressed or not resolved from the 2015 technical review, 16 (84.2 per cent) related to the Industrial Processes and Product Use (IPPU) and Land Use, Land Use Change and Forestry (LULUCF) inventory sectors. Recommendations classified as being addressed included those where implementation was underway, where the department was waiting for relevant data to become available, or that had only been partly addressed. The Expert Review Team had also documented that the department had planned to implement at least four of the seven ‘not resolved’ recommendations in future GHG inventories.

2.12 Recommendations outstanding from the previous technical review of the 2014 inventory (12) have all been resolved according to the UNFCCC’s most recent technical review, including two LULUCF recommendations that had remained outstanding for two and three consecutive technical reviews, respectively.

Table 2.1: Australia’s implementation of the UNFCCC 2015 technical review recommendations

Source: Report on the individual review of the annual submission of Australia submitted in 2016.

Has the department instituted appropriate quality assurance and control procedures over the preparation of inventory estimates?

2.13 In accordance with UNFCCC requirements, the department develops an inventory Quality Assurance/Quality Control Plan in advance of preparing each year’s annual GHG inventory.²⁸ The IPCC Guidelines describe the requirements or good practices for effective quality assurance and quality control procedures (as well as outlining the minimal general quality control procedures that should be applied).

Source data quality controls

2.14 The quality control framework for the annual GHG inventory (all sectors except for LULUCF) relies on the quality control procedures employed by other government entities and industry over the data sourced from these entities. The department’s inventory Quality Assurance/Control Plan 2015–16 documents the key quality control procedures for some of the principal data sources, including the Clean Energy Regulator (collected under the National Greenhouse and Energy Reporting Scheme), the Australian Bureau of Statistics (agricultural data) and the Department of Industry, Innovation and Science’s Office of the Chief Economist. However, the Plan does not describe the quality control procedures employed by other principal data sources, including ABARES and state/territory waste agencies. Expanding the description of the key quality control procedures employed by all principal data sources would better position the department to demonstrate its understanding of the risks to data quality across all principal source data providers.

2.15 In contrast to other inventory sectors, much of the source data for the LULUCF sector is generated by consultants contracted by the department. Each year, the department oversees the updating or generation of spatial data sets (relating to land clearing/regrowth, and climatic and soil quality variables) covering most of Australia’s land area for entry into FullCAM, which facilitates the calculation of greenhouse gas emissions/sinks from annual land use changes.

2.16 Before the spatial data sets are used to estimate LULUCF emissions, the department monitors the application of quality assurance procedures by its contractors and applies a number of its own quality assurance procedures. In relation to its most significant spatial data set—the land use change spatial data set—the department:

• oversees the quality assurance/control procedures instituted by its lead contractor (CSIRO) over the work undertaken by other private sector entities contracted to the department to acquire, process and classify satellite data using algorithms;
• manually checks the land use change spatial data obtained from the CSIRO to either confirm, expand, reduce or remove the potential land use change areas identified; and
• reviews the manual checks (in total or on a sample basis).

2.17 The results of the latter quality assurance reviews of the manual checks are appropriately documented in the review reports, although:

• documentation of adjustment/errors by type would facilitate subsequent analysis and the refinement of instruction and training for departmental staff undertaking the manual checks; and
• in cases where land use changes are reviewed on a sample basis, additional information regarding the number of data points reviewed would assist the department to identify the adjustment/error rates to give indications of the potential for, and the acceptability of, errors in the population.

Quality assurance/controls over inventory preparation

2.18 Inventory Preparation Manuals established by the department to guide its preparation of each sector’s inventories contain quality control and assurance procedures to verify the accuracy and completeness of the manual entry of source data onto templates for entry into AGEIS. However, the comprehensiveness of the procedures varies between sectors and evidence is not retained to verify that the procedures had been appropriately implemented.

2.19 The effective implementation of inventory preparation quality assurance procedures would have reduced or eliminated the data errors identified by the ANAO during its examination of the source data for a selected sample of inventory sub-sector line-items (see paragraph 2.3). Many of the errors occurred due to human error during the department’s development of the data entry templates or the pre-processing of source data for inclusion in data entry templates. In addition, all metadata references archived in the AGEIS were found to be nonspecific, out of date, inaccurate, or absent. As a result, the ANAO required extensive assistance from the department to trace the data entry template inputs to the source data. The metadata referencing weaknesses inhibit the transparency of the annual GHG inventory and create a risk to the timely and accurate preparation of future inventories through the loss of corporate memory from staff turnover.

2.22 In addition to the AGEIS input quality controls/assurance procedures contained in the Inventory Preparation Manuals, the department prepares an annual Quality Assurance/Quality Control Plan that identifies the risks to inventory quality and the quality control measures to be undertaken by the department during and after inventory preparation. Table 2.2 on the following page outlines the risks to 2014 GHG inventory quality identified in the Quality Assurance/Quality Control Plan 2015–16.

2.23 The Quality Assurance/Quality Control Plan 2015-16 contains 52 quality control measures that are used to evaluate activity data, emissions factors, uncertainty analysis, UNFCCC technical review results and inventory recalculations. Each quality control measure:

• is mapped to the applicable quality objective(s);
• describes the quality control/assurance procedure(s) in the IPCC guidelines that is/are met; and
• has a pre-established quantitative or qualitative target. Quantitative targets are expressed for measures that involve internal/external data reconciliations or comparisons (for example, a target may require a variance of less than 0.1 per cent). Qualitative targets require ‘compliance’ with the quality control measure.

2.24 Quality control measures have been designed to cover each quality objective for all inventory sectors/sub-sectors, with the exception of measures relating to the UNFCCC’s technical reviews of Australia’s annual GHG inventory. The current measures require a reduction in the number of UNFCCC technical review recommendations over time, yet the department determines compliance with these measures by reference to its acceptance of the recommendations. The number of recommendations arising from an inventory technical review is determined by the Expert Review Team, and recommendation acceptance does not necessarily equate to its timely implementation. There is scope to improve the UNFCCC technical review quality measures by redesigning them to relate to the UNFCCC’s acceptance of the annual GHG inventory without adjustment and the department’s implementation of recommendations within specified timeframes.

Table 2.2: Risks to inventory quality

Note a: Tier 2 and 3 models relate to country- and facility-specific methods of estimating emissions from greenhouse gas producing activities. (Tier 1 models are those that are specified by the IPCC).

Source: Quality Assurance / Quality Control Plan 2015–16 prepared by the department.

2.25 The extent to which quality control measures for the annual GHG inventory have been addressed are evaluated by the department and reported in the Evaluation of Outcomes report. The Evaluation of Outcomes 2015–16 report indicated full compliance with 51 quality control measures and partial compliance with the remaining measure.²⁹

External review of draft inventory estimates

2.26 As part of the inventory’s quality assurance framework, the department has established two committees/groups—the National Greenhouse Gas Inventory Committee and National Greenhouse Gas Inventory User Reference Group—to review and provide feedback on the draft national inventory report prior to its finalisation and submission to the UNFCCC. In addition to departmental representatives, the National Greenhouse Gas Inventory Committee includes representatives of each state/territory government, and the National Greenhouse Gas Inventory User Reference Group includes representatives from academia, science, the environment and industry. Both bodies met in March 2016 at which time the department presented the draft 2014 National Inventory Report, highlighting recalculations undertaken and improvements made on the previous year’s report. Both bodies raised no concerns with the contents of the draft 2014 National Inventory Report.

Has the quality of inventory estimates improved over time?

Inventory Improvement Plans

2.27 At the conclusion of each year’s GHG inventory development process, the department updates its Inventory Improvement Plan. The plan outlines the projects and activities that the department has designed to ensure the national greenhouse accounts:

• remain relevant to the GHG emissions policies accepted by the Australian Government and fit-for-purpose (by incorporating: revisions to UNFCCC/Kyoto Protocol reporting requirements; more sophisticated IPCC Tier 3 methods; results from empirical research; and carbon abatement from mitigation measures);
• mitigate risks to Australian Government interests (including potential financial penalties/costs arising from inventory non-compliance and compromised data quality/integrity³⁰); and
• respond to emerging international developments (including: new international guidelines; UNFCCC technical review feedback; and new GHG estimation methods).

2.28 Of the 27 inventory improvement projects listed in the Inventory Improvement Plan 2016–18 for future implementation:

• all require the implementation of system changes to AGEIS and/or FullCAM;
• 21 projects arose from recommendations/suggestions from an UNFCCC technical review of a previous GHG inventory, with the remainder scheduled to implement a UNFCCC decision (three) or self-initiated (three);
• 19 projects relate to the LULUCF sector of the inventory; and
• seven projects will facilitate the integration of enhancements to national data sets into future inventories.

2.29 An examination of the previous year’s Inventory Improvement Plan indicates that full implementation of nearly half of its 27 projects³¹ had been delayed by one or more years. The department indicated that the primary reasons for delays were a lack of resources and changing priorities. The department informed the ANAO that the implementation delays may elevate the risk of future UNFCCC technical reviews commenting adversely on the quality of future GHG inventories.

Quality of Australia’s inventory estimates over time and in comparison to those of other countries

2.30 The consistent application of agreed methodologies and formats for making estimations and reporting GHG inventories makes each country’s annual GHG inventories (particularly the supporting spreadsheet data) comparable among Annex I parties in any given year. In addition, the consistency of inventory reporting structures and formats over time also allows a country’s annual GHG inventory reporting to be compared from one year to the next.

2.31 In this context, the ANAO analysed Australia’s GHG inventory over time to determine the extent to which its quality has changed. The quality of Australia’s GHG inventory over the past 10-12 years, as measured by the parameters outlined in Table 2.3, has improved in some areas (most notably emission calculation methods, UNFCCC technical review assessments and ‘not estimated’ emission categories) while remaining relatively stable in other areas.

Table 2.3: Quality of Australia’s GHG inventory over time

Note a: LULUCF sector emissions were excluded from this analysis due to the large variability in its year-to-year contribution to Australia’s GHG emissions. Nevertheless, the sophistication of LULUCF emission methods increased from 2013 when some methods were upgraded: from Tier 1/Default to Tier 2/Country-specific; and from Tier2/Country-specific to Tier 3/Method.

Note b: Analysis takes into account only those reasons cited that are within the Party’s control (that is, ‘activity data unavailable’, ‘emissions not significant’, and where no substantive reason was provided).

Note c: Reporting formats for UNFCCC technical reviews have changed over time, which has restricted the comparison analysis.

Source: ANAO analysis of UNFCCC information.

2.32 Using the same parameters, the ANAO also analysed the quality of Australia’s GHG inventory relative to the quality of other Annex I parties’ inventories. This analysis indicated that the quality of Australia’s GHG inventory compares well to the inventories of other Annex I parties, with the exception of ‘not estimated’ emission categories, as outlined in Table 2.4.

Table 2.4: Quality of Australia’s GHG inventory relative to other countries

Note a: The number of countries for each analysis type varies according to the extent to which country-specific information is available in English on the UNFCCC’s website related to the 2014 GHG inventory year (unless otherwise specified).

Note b: Analysis takes into account only those reasons cited that are within the Party’s control (that is, ‘activity data unavailable’, ‘emissions not significant’, and where no substantive reason was provided).

Note c: The UNFCCC Expert Review Team technical reviews of the 2012 GHG inventories are the latest published on the UNFCCC website for most countries. As a consequence, meaningful comparisons of Australia’s performance to those of other countries in relation to later inventories (2013 and 2014) could not be undertaken.

Source: ANAO analysis of UNFCCC information.

Are emissions projections based on relevant, authoritative source data?

3.1 In December 2016, the department published the most recent report on Australia’s GHG emissions projections (Australia’s Emissions Projections 2016). The report indicated that as a proportion of total emissions from 2015 to 2030:

• emissions from electricity generation are projected to remain Australia’s greatest emissions contributor, albeit decreasing (by approximately four percentage points);
• emissions from the LULUCF sector are projected to increase by three percentage points (from a very low base); and
• emissions from the remaining sectors are projected to remain a similar proportion of total emissions (within ± 1.2 percentage points).

3.2 Figure 3.1 illustrates emissions estimates and projections from each sector of the Australian economy over the period from 1990 to 2030.

Figure 3.1: Domestic emissions estimates and projections, 1990 to 2030

Source: Department of the Environment and Energy, Australia’s emissions projections 2016, p.6.

3.3 The department’s greenhouse gas emissions projections have been calculated using published and unpublished forecast data from government entities, activity data projections from external providers and public announcements by major carbon-emitting entities. The primary data sources for the December 2016 projections include the government entities listed earlier in paragraph 2.1 and external providers with sector-specific expertise such as ACIL Allen Consulting (Electricity) and ABMARC (Transport). At times, the department also applies its own judgment, usually in relation to reconciling conflicting forecast data from multiple sources or determining activity data from sub-sectors with insignificant emissions.³²

3.4 In most cases, the December 2016 projections were based on the latest available data. There are, however, several exceptions, which included:

• the LULUCF forest conversions projections calculated for the earlier April 2016 projections update being carried over to the December 2016 projections.³³ The use of older data did not significantly alter the emissions projection, which was determined by its comparison to (unpublished) forest conversion projections applying a varied set of accounting rules that used more recent data³⁴; and
• the use of forecast activity data series for aspects of the Direct Combustion and Industrial Processes and Product Use sectors that became superseded when more recent activity data became available. In some cases, the department had documented its reasons for, and the (insignificant) impact from, its decision not to use the more up-to-date data. In other cases, using older data did not significantly alter the sectors’ emissions projections.

3.5 In those cases where the department does not use contemporary data in the future for emissions projections, the department should document its reasons and the likely impact on emissions projections calculations.

3.6 Projection calculations are contained in multiple spreadsheets prepared for each emissions sector and/or sub-sector. The spreadsheets supporting the sampled emissions sub-sectors examined by the ANAO³⁵ contained formulas and calculations that operated as intended with the exception of seven data discrepancies and 48 formula discrepancies.³⁶ These discrepancies collectively resulted in an immaterial understatement of future emissions over the period 2015 to 2030 of 17 Mt CO2-e (0.4 per cent of the cumulative emissions of the sub-sectors examined by the ANAO).

Has the department responded to emissions projection review recommendations from the UNFCCC?

3.7 As outlined earlier, Australia’s greenhouse gas emissions projections are reported to the UNFCCC in National Communications (prepared every four years) and Biennial Reports (prepared every two years). These projections are subject to a technical review by an Expert Review Team. The most recent projections submitted to the UNFCCC were those contained in Australia’s Biennial Report of December 2015.³⁷

3.8 The UNFCCC Annex I reporting guidelines on national communications outline, among other things, the mandatory and discretionary requirements for reporting projections.³⁸ The key mandatory requirements, and the UNFCCC technical review assessment of Australia’s second Biennial Report compliance, are outlined in Table 3.1.

Table 3.1: UNFCCC’s assessment of Australia’s reporting of projections in its second Biennial Report

Note a: Australia was the only UNFCCC Annex I Party that did not include emissions projections to 2030 in its 2015 Biennial Review.

Source: UNFCCC Annex I reporting guidelines on national communications and Report of the technical review of the second biennial report of Australia (July 2016).

3.9 The department had since taken steps to address the UNFCCC’s projection-related recommendation in advance of the next Biennial Review due for submission on 1 January 2018. The December 2016 projections report contained emissions projections to 2030.

Has the department instituted appropriate quality assurance and control procedures over the preparation of emissions projections?

Sectoral emissions projections

3.10 The department has developed a quality assurance and control template that is to be applied to the projections for each emissions sector prior to publication. In respect of the December 2016 projections, the template was completed satisfactorily for each sector examined by the ANAO which:

• listed (and linked to) the ‘master’ spreadsheet and other spreadsheets supporting the sectors’ projection calculations;
• outlined the Australian Government policy considerations included in the calculations (such as the Emissions Reduction Fund);
• outlined how the projections have been scaled to the latest inventory estimates;
• outlined emission projection plots and trends for the sector and sub-sectors (including sensitivity analyses, where applicable), and contained a brief explanation of the trend and variances from past emissions projections for the sector; and
• contained a quality assurance checklist that indicated, among other things: assumptions had been tested with the Technical Working Group (discussed later at paragraph 3.15), and departmental staff with sectoral expertise, as necessary; at least two team members had checked the accuracy of the spreadsheet calculations; and results had been tested with staff with sectoral expertise, as necessary.

3.11 The department has documented brief qualitative explanations of variations from past sector emissions projections. Quantitative reconciliations of the variance between the current projections release and the previous release would provide greater insights into the extent to which changes to data, assumptions and models each contributed to the variance. There is scope for the department to enhance the rigour of future projections with the insights provided through quantitative reconciliations.

3.12 With the exception of the LULUCF sector projections, the department provided additional contemporaneous evidence to support its satisfactory:

• understanding and testing of projection assumptions (including those underpinning data: sourced directly from government and non-government sources; and within projection models prepared by departmental consultants);
• checking of spreadsheet calculations (by two officers); and
• testing of projection results for the sectors examined by the ANAO.

3.13 In contrast to other sectors, the LULUCF sector projections rely heavily on: complex models developed within the department to calculate emissions projections; and the application of expert judgment regarding key variables and manual adjustments to the models.³⁹ Over time, successive adjustments to models have been added to previously adjusted models without clear documentation of the rationale. The ANAO’s examination of the ‘layers’ of departmental adjustment applied to the LULUCF forestry conversion projections identified a 0.5 Mt CO2-e per annum discrepancy in the 2015 to 2030 projections caused by a failure to recalibrate earlier model adjustments (the rationale for which had not been appropriately documented).

3.14 The department informed the ANAO of its intentions to ‘rebase’ its LULUCF projections model rather than continue to overlay additional adjustments on previously adjusted models. The transparency of LULUCF sector projections would be further enhanced were the department to improve the documentation of:

• its application of judgment throughout the calculation of this sector’s projected emissions; and
• the quality assurance testing (see paragraph 3.12) that has been undertaken.

3.15 In addition to the department’s internal quality control and assurance procedures for projections, in 2014 the department established the Emissions Projections Technical Working Group—an inter-departmental group that is responsible for ‘provide[ing] technical, methodological and review assistance to ensure Australia’s emissions projections are of the highest possible quality and consistent with the broader approach to forecasting across the Australian Government’.⁴⁰ The Technical Working Group meets twice a year where the department presents its overall and sectoral methodology, sensitivities and key issues (Meeting 1) and its draft projections results by sector (Meeting 2). Those action items arising from the 2016 Technical Working Group meetings in relation to the areas examined by the ANAO had been satisfactorily recorded and addressed by the department.

Abatement projections

3.16 The December 2016 projections report indicates that its emissions projections take into account the Emissions Reduction Fund⁴¹ and the Renewable Energy Target policies of the Australian Government designed to abate carbon emissions.⁴² The department separately calculated the Emission Reduction Fund’s projected abatement (which has been deducted from ‘gross’ sectoral projections to arrive at the published sectoral projections inclusive of the Emissions Reduction Fund). Sectoral projections have been calculated inclusive of the abatement from the Renewable Energy Target.

3.17 In contrast to the sectoral projections, the department has not prepared a quality assurance and control document for the Emission Reduction Fund abatement projection calculations nor has the Technical Working Group been involved in reviewing the assumptions underpinning, or results of, the Fund’s projected abatement. The department advised that those officers responsible for the policy settings of the Emissions Reduction Fund within the department were consulted regarding its assumptions and results but the feedback obtained has not been documented.⁴³ The department also presented the assumptions, data inputs and calculation methodology underpinning the Fund’s abatement projections to the Emissions Reduction Fund Programme Steering Committee (comprising representatives from the department and the Clean Energy Regulator). This Committee did not raise concerns regarding the department’s approach to calculating Emission Reduction Fund abatement projections.

3.18 The introduction of quality assurance procedures for abatement measure projections calculated separately from sectoral emissions projections would provide the department with greater assurance regarding their accuracy and robustness.

Has the quality of emissions projections improved over time?

Accuracy and longevity of emissions projections

3.19 The length of time that emission projections remain accurate is one measure of their quality. The longevity of accurate emissions projections relies on the longevity of the key assumptions underpinning activity data and emissions factors used in their calculation. Australia’s projected greenhouse gas emissions to 2020 and 2030 have been revised downwards in successive reports published since 2008 (from 849 Mt CO2-e in 2008 to 592 Mt CO2-e in 2016—see Figure 3.2).

Figure 3.2: Projected emissions in 2030, 2008—2016

Source: Australia’s Emissions Projections 2016, p.7.

3.20 While the latest projections provide a more current representation of projected future emissions in 2030 than earlier projections, this does not necessarily indicate that the later projections are better than earlier projections. Each set of projections are ‘point in time’ forecasts of future emissions based on a given set of assumptions related to predicted future activity (and therefore, emissions). New information can significantly change assumptions on which future emissions are based. For example, the closure of the Hazelwood Power Station from April 2017 (announced in November 2016) reduced Australia’s predicted GHG emissions to 2030 by a net 88.3 Mt CO2-e (which is in excess of one per cent of Australia’s emissions over the same period).⁴⁴ The unpredictable nature of assumption changes makes the longevity of accurate emissions projections an unreliable measure of projections quality.

Quality of Australia’s emissions projections in comparison to those of other countries

3.21 The quality of Australia’s GHG emissions projections can be measured in comparison to those of other countries. The UNFCCC technical review of second Biennial Reports rates each Annex I country’s ‘Progress in achievement of targets’—which contain each country’s emissions projections—in terms of the quality measures of completeness and transparency. The UNFCCC Expert Review Team rated Australia’s ‘Progress in achievement of targets’ as mostly complete and mostly transparent, which was in line with the ratings assigned to most countries’ second Biennial Reports (see Table 3.2).

Table 3.2: UNFCCC technical review of Annex I Parties second Biennial Reports

Note: Grey-shading indicates Australia’s rating by the UNFCCC.

Source: ANAO analysis of UNFCCC information.

Usefulness of emissions projections to stakeholders and users

3.22 Stakeholders and users of Australia’s GHG emissions projections are wide and varied, including governments (Australian, state/territory and local), industry, academia and environmental organisations. Maximising GHG emissions disclosures, including the key data and assumptions underpinning the projections, will maximise the utility of the projections for stakeholders and users⁴⁵ and give effect to the Australian Government Public Data Policy Statement that requires the release of non-sensitive data by default.

Disclosures in the December 2016 projections report

3.23 The December 2016 projections report contained information designed to assist stakeholders and users to understand the content of, and calculations underpinning, emissions projections—including:

• qualitative descriptions of the Australian Government policies taken and not taken into account in the projections, the reasons for the trends observed at the overall and sectoral emissions levels, and the core assumptions underpinning the sectoral assumptions; and
• sensitivity analyses in addition to ‘baseline’ projections that calculated alternative projections (at an aggregated sectoral level) based on variations to baseline assumptions. Two alternate projections have been published that resulted in emissions projections greater and less than the baseline (see the hash-shaded area in Figure 1.4).⁴⁶

3.24 While sectoral disclosures and sensitivity analyses have enhanced the transparency of the December 2016 projections, the lack of disclosures regarding projected abatement from Australian Government policy measures detracts from the transparency of the projections. The December 2016 projections report does not quantify projected abatement from the Renewable Energy Target or Emissions Reduction Fund.

Renewable Energy Target

3.25 Sectoral emissions projections have been calculated inclusive of the Renewable Energy Target policy, reflecting its integral nature to the calculations themselves. The department informed the ANAO that it is often difficult to disaggregate the impacts of market-based emissions abatement policies (such as the Renewable Energy Target), once implemented, from a range of other prevailing economic and other factors. Once such policies are embedded as a ‘business as usual’ factor, identifying their contribution to emissions reduction over time requires a range of assumptions (which may not have a high degree of associated confidence) and may necessitate detailed modelling (of questionable cost-benefit value)⁴⁷.

3.26 The ANAO acknowledges that projecting emissions abatement from market-based emissions abatement policies, in particular, can be difficult to determine accurately for the reasons cited by the department above. Nevertheless, increasing quantitative disclosure (including through the use of abatement ranges, and descriptions of key assumptions and the extent to which the department has confidence in the ranges and assumptions) would enhance the transparency of emissions projections.

Emissions Reduction Fund

3.27 Sectoral emissions projections have been calculated exclusive of the Emissions Reduction Fund policy and then reduced by the projected abatement from the Emissions Reduction Fund calculated separately by the department. The department informed the ANAO that it was concerned not to undermine the effectiveness of the Emissions Reduction Fund policy by disclosing information regarding abatement projections from future auctions that may affect future auction bidding behaviour by market participants.⁴⁸ The ANAO considers that where concerns exist that disclosures can adversely influence policy effectiveness, the department should maximise its disclosures (quantitative emissions projections and key assumptions) to the point where policy effectiveness is unlikely to be adversely impacted in a significant manner.

3.28 The decision taken not to disclose projected abatement from the Emissions Reduction Fund in the December 2016 projections report is in contrast with the April 2016 projections update where projected abatement to 2020 (92 Mt CO2-e) was disclosed. The December 2016 projections report implicitly includes projected abatement from the Emissions Reduction Fund of 82.2 Mt CO2-e over the period 2015–2020 (representing emissions reduction of 2.5 per cent), and 242.2 Mt CO2-e over the period 2021–2030 (representing emissions reduction of 4.2 per cent). The key assumptions that underpin the Emission Reduction Fund calculations include:

• abatement projections include contracted and post-contracted abatement⁴⁹ from completed and future auctions⁵⁰, but exclude abatement credited to Fund projects not purchased in auctions;
• contracted projects will not be impacted by: the non- or delayed satisfaction of conditions precedent in Fund contracts; or delays to the delivery of contracted abatement and non-remediable Fund contract defaults;
• all uncommitted funding remaining after the November 2016 auction will be contracted in the April 2017 auction⁵¹; and
• contract abatement delivery schedules for the April 2017 auction will mirror those for the November 2016 auction (without any allowance for time lags).

3.29 As the Emissions Reduction Fund is a primary mechanism for the Government to reduce GHG emissions, the transparency of the assumptions underpinning the Fund’s abatement calculations becomes increasingly important. However, documentation retained by the department supporting the calculations did not sufficiently justify the Emission Reduction Fund’s key assumptions and reflected an optimistic scenario. The department has informed the ANAO that:

• an alternative set of assumptions would not necessarily improve the abatement projections or lead to a material difference; and
• variances would mostly result in the re-profiling of abatement to later years rather than significantly varying the total abatement from the Fund.

Increasing the availability of projections information

3.30 Increasing the release of information underpinning the department’s projections would improve the utility of the emissions projections to stakeholders and users. In this regard, there would be benefit in the department publishing projections key input data (with sensitive data obscured by aggregation, averaging or redaction), assumptions, formulas and methods sufficient to allow users to recalculate emissions projections within a reasonable degree of precision. Such information would:

• provide stakeholders and users an opportunity to better scrutinise the projections and provide feedback to the department that could enhance the projections’ accuracy and robustness; and
• enable stakeholders and users to adapt the department’s projections (by varying the key data inputs or assumptions) for their own purposes.

Have appropriate planning processes been established to guide the preparation and reporting of inventory estimates and emissions projections?

4.1 Effective planning frameworks for the preparation and reporting of emissions estimates and projections, among other things, provides assurance to those responsible for approving inventory estimates and emissions projections that all relevant considerations will be taken into account. As noted earlier, separate branches of the department are responsible for the preparation of emissions estimates and projections.

Inventory estimates

4.2 The National Inventory Systems and International Reporting Branch’s preparation and reporting of the annual GHG inventory is supported by a number of plans covering particular management aspects, including the:

• Branch Business and Risk Management Plan that outlines, among other things, staffing/resources and a risk register (discussed later)⁵²;
• annual National Inventory Report submission preparation timeline and responsibilities document that outlines branch staff responsibilities for preparing aspects of the annual GHG inventory and timelines for the inventory’s preparation;
• Inventory Preparation Manuals for each inventory sector that outlines: data sources and timeframes; instructions for pre-processing source data (where required) and data entry into the AGEIS via spreadsheet; and quality control measures to verify the accuracy and completeness of AGEIS data entry and the calculated emissions;
• Quality Assurance/Quality Control Plan that, among other things, identifies the quality control measures to be undertaken during and after the preparation of the annual inventory to verify it meets the quality objectives;
• Inventory Improvement Plan that contains projects and activities designed to ensure the National Greenhouse Gas Accounts (including the annual GHG inventory) remains fit-for-purpose and reflects the latest information and requirements; and
• AGEIS and FullCAM Strategic Plans that provide the rationale and direction for the planning and management of the AGEIS and FullCAM in support of Australia’s National Inventory System.

4.3 While the department updates these plans annually, it has not finalised an overarching project plan to govern the preparation and reporting of Australia’s annual GHG emissions estimates. In February 2017, the department provided the ANAO with a working draft of a consolidated inventory preparation project plan. This draft plan primarily consisted of direct extracts from the existing planning documentation and the latest National Inventory Report (in relation to Australia’s national inventory arrangements, inventory preparation, and data collection, processing and storage). Further, some key governance and operational information was absent, such as details regarding responsibility within the department for approving the inventory for publication and submission to the UNFCCC (currently recorded as ‘Secretary’s delegate’) and the role of the Minister for the Environment and Energy (the Minister) during the inventory preparation and reporting process (discussed below at paragraph 4.13). As the inventory approving authority, the ‘Secretary’s delegate’ should also approve the annual inventory project plan.

4.4 The refinement of the current draft inventory project plan (by reference to the department’s Project Plan template) would minimise the risk to future inventory quality and timeliness from the loss of corporate knowledge due to staff turnover.

Emissions projections

4.5 The department has established an effective planning framework for the preparation and reporting of Australia’s emissions projections. As part of the framework, the department has developed an overarching project plan for the 2016 emissions projections that outlined:

• key considerations for the calculation of the baseline and multiple sensitivity scenario projections (including links to the inventory estimates, assumptions regarding government policy considerations, and the projections period);
• primary data sources over the projections period (which, for some sub-sectors, can vary over time);
• projections data dependencies between emissions sectors (that is, where the activity data or emissions projections calculated for one sector is an input to the calculation of other sector’s activity data or emissions projections);
• consultation arrangements; and
• references to supporting documentation for work timelines, staff roles and responsibilities, and risk management (discussed below at paragraph 4.10).

4.6 In addition to the documentation referenced in the project plan, the department’s planning framework for emissions projections is appropriately supported by emissions sector plans⁵³, instructional guidance on preparing projections⁵⁴ and quality control and assurance documentation (examined earlier in Chapter 3). The effectiveness of the emissions projections planning documentation could be further improved by:

• clearly indicating: the officer responsible within the department for approving the projections; and the role of the Minister in the reporting and release of the projections (discussed below from paragraph 4.14); and
• the departmental delegate responsible for approving the projections approving the annual projections project plan.

Have risks to the preparation of inventory estimates and emissions projections been identified and mitigated where necessary?

Inventory estimates

4.7 The department’s risk management planning documentation for the preparation and reporting of inventory estimates is contained in the Business and Risk Management Plan 2016–17 of the National Inventory System and International Reporting branch. In this plan the department has outlined for each risk, its consequences, mitigation strategies and qualitative descriptions of the residual risk. However, the risk register does not accord with departmental risk management requirements as it:

• contains poorly defined risks by its inclusion of issues (that is, materialised risks) related to a reduction in financial and staff resources for 2016–17; and
• does not: assign risks to standardised risk categories; separate existing controls and risk treatments; identify when risk treatments will be finalised and the officer(s) responsible; and rate risks pre- and post-treatment (where required) using the departmental likelihood and consequence matrix.

4.8 Risk registers contained in previous Business and Risk Management Plans (for 2014–15 and 2015–16, which are identical) outlined risks that had been assessed in greater consistency with departmental requirements, albeit not using the departmental risk assessment template. These risks were rated pre-treatment only, were not grouped according to departmental standard categorisations, and did not list the due dates and responsible officers for treatments.

4.9 The 15 risks contained in the 2014–15 and 2015–16 risk registers (four rated high and 11 rated moderate) related to the (non-standard) risk categories of delivery, capability, knowledge and information, relationship management, external environment, and governance and accountability.⁵⁵ The risk assessments are incomplete for five risks (three rated high and two rated moderate) where the department has not identified its acceptance of the risks or the need for additional treatments (which would be expected for risks rated as ‘High’). Further, the department has not retained documentation to demonstrate its active management of risks between annual assessments and of the implementation of identified risk treatments.

Emissions projections

4.10 The risk register for the preparation and reporting of the December 2016 emissions projections identifies for each risk: risk impact; existing controls; and risk ratings pre- and post-treatment (where required). However, the risk register for projections also does not accord with departmental risk management requirements as it lacks risk sources; risk categories; and risk treatment implementation dates and the officer(s) responsible.

4.11 The 12 risks contained in the 2016 risk register have been rated as high (one risk), medium (seven risks) or low (four risks). Additional risk treatments have been identified for the sole high risk only. Many of the risks listed in the 2016 risk register are listed in a similar form in the inventory sector projection plans, with treatments (some in the form of existing controls) identified for risk. As was the case for inventory estimates, the department has not retained documentation to demonstrate its active management of risks between annual assessments and of the implementation of identified risk treatments.

Are effective arrangements in place to monitor the timeliness of the preparation and reporting of inventory estimates and emissions projections?

Inventory estimates

4.13 The UNFCCC requires Annex I countries to submit their National Inventory Report (including the annual GHG inventory data) by 15 April each year.⁵⁶ The department’s annual National Inventory Report preparation timeline and responsibilities document outlines staff responsibilities and a timetable for preparing the annual GHG inventory and timelines for the inventory’s preparation to meet the UNFCCC’s submission deadline. The department also used this document to monitor progress against the key deadlines for the inventory’s preparation and reporting, which enabled the department to submit the National Inventory Report 2014 (and supporting data), and answers to technical review queries, to the UNFCCC by the due dates. The Minister was informed prior to the impending submission of the original and revised National Inventory Report 2014 to the UNFCCC.

Emissions projections

4.14 The timing of the release of emissions projections is determined by the Minister, noting that Biennial Reports, which contain emissions projections, are required to be submitted to the UNFCCC by 1 January of every second year. At the commencement of every emissions projections recalculation or update, the department seeks and obtains the Minister’s agreement on the scope and timing of the projections’ release, and assigns tasks accordingly. The department maintains a list of tasks with deadlines and has monitored the progress of the various tasks effectively.

4.15 The 2015 projections release was originally envisaged to contain projections to 2030 and be published in advance of the UNFCCC Climate Change Conference in Paris in late-November 2015. However, the department informed the ANAO that at a meeting in mid-November 2015, the Minister and the departmental Secretary agreed that it would not be appropriate to project emissions to 2030 without examining alternative scenarios⁵⁷ that would have a material impact on the longer-term emissions projections. As alternative scenarios would not be complete by the Biennial Report submission deadline, it was agreed that the Government would progress the alternative scenarios for inclusion in a 2016 projections release. The final 2015 projections contained projections to 2020 only⁵⁸ and were published in late December 2015 (in advance of the UNFCCC submission deadline of 1 January 2016).

4.16 In relation to the 2016 projections report, the department had developed, and began working towards, an October 2016 completion of the 2016 projections report prior to the UNFCCC Climate Change Conference in Marrakech in November 2016. As the projections were developed, the department and the Minister’s Office agreed that incorporating newly available data in November 2016⁵⁹ and framing of the Government’s 2017 climate change policy review would take precedence over the achievement of the original timeframe. The 2016 projections were published in December 2016 as soon as the new information had been incorporated into the projections.

Have stakeholders been effectively engaged through the process of collecting source data and preparing and reporting of inventory estimates and emissions projections?

4.17 The primary stakeholders to the preparation and reporting of inventory estimates and emissions projections are the source data providers and users of the estimates and projections.⁶⁰ As noted earlier, the annual GHG inventory and emissions projections releases rely on a range of published and unpublished data from government and non-government sources obtained through long-standing agreements and cooperation. The source data providers include the:

• Clean Energy Regulator—in relation to:
  • its on-line portal of energy, industrial process and product use and waste sector data collected under the National Greenhouse and Energy Reporting Scheme (NGERS); and
  • contractual data related to abatement commitments and achievements under the Emissions Reduction Fund;
• Department of the Treasury—in relation to economic growth and gross domestic product forecasts to 2030;
• Department of Industry, Innovation and Science’s Office of the Chief Economist—in relation to commodity forecasts;
• Geoscience Australia—in relation to satellite imagery for the LULUCF sector; and
• state and territory government waste agencies—in relation to waste facility data.

4.18 The department has established effective working relationships with source data providers to obtain unpublished data relevant to the preparation of the inventory estimates and emissions projections in a timely manner. Further, as noted earlier, the department consulted effectively with its emissions projection’s source data providers to gain an understanding of the assumptions underpinning the source data and the rationale for trends over time.

4.19 The department’s engagement with inventory estimates and emissions projections users is limited primarily to external members of committees and groups established by the department to review the plans for, and/or the results from, estimates and projections updates.⁶¹ As well as forming part of the inventory’s quality assurance framework, these external stakeholders are able to provide feedback on the content and quality of the estimates and projections from a user’s perspective. Nevertheless, obtaining feedback on the inventory and emissions projections from a broader range of end users would be beneficial, particularly following the expanded publication of key projection input data, assumptions, formulas and methods (see Recommendation No.2).

4.20 Comments received by the ANAO from source data providers and committee/group members indicate their general satisfaction with the department’s engagement with them during the preparation and reporting of Australia’s GHG emissions estimates and projections. These stakeholders also indicated a high degree of satisfaction with the content and quality of Australia’s National Inventory Reports.⁶² While emissions projections were well-regarded by some stakeholders, suggestions for improvement included the timeliness of projections, greater consistency between projections releases, increased disaggregation of data (by including state/territory breakdowns), and increased transparency regarding the impact of government emissions abatement and mitigation measures.⁶³

Has the efficiency of the preparation and reporting of inventory estimates and emissions projections improved over time?

4.21 The department has not established key performance indicators to measure the efficiency of preparing and reporting emissions estimates and projections overall. In the absence of departmental indicators, the ANAO examined the efficiency of estimates and projections preparation and reporting by reference to available data on annual operational and capital expenditure in comparison to annual deliverables which, as a whole, have remained similar over recent years.

4.22 Departmental resource management documentation indicates that the operational expenditure (mostly in the form of staff salaries) of the branch responsible for inventory estimates over the period 2011–12 to 2015–16 has decreased year-on-year from $7.3 million to $5.3 million. Planned operational expenditure in 2016–17 is expected to further decrease to $4.3 million (a 41.3 percentage decrease over the period from 2011–12 to 2016–17). Over the same six-year period, capital expenditure (related to AGEIS and FullCAM enhancements) has remained relatively stable at between $650 000 and $750 000 per annum with the exception of 2012–13 when it was $1.9 million.⁶⁴

4.23 The department has significantly increased the efficiency of its preparation and reporting of inventory estimates, in part, by streamlining its business practices and reducing expenditure on contractors by, for example:

• automating the production of the annual Evaluation of Outcomes document (which measures performance against the inventory Quality Assurance / Quality Control Plan);
• automating aspects of the analysis of satellite imagery for the LULUCF sector; and
• renewing a key LULUCF contractor’s multi-year contract for the same annual cost as the previous contract, but increasing the quantity of deliverables by over 50 per cent.

4.24 Annual operational expenditure for the Projections Team over the period from 2013–14 to 2016–17 (planned) has ranged between $1.1 million and $1.4 million. Planned operational expenditure for 2016–17 is equal to the average annual expenditure of $1.3 million over this four-year period.⁶⁵ This broadly indicates that the efficiency of the Projections Team has remained relatively stable over the last four years.

Appendix 1 Entity response to proposed audit report

Auditor-General’s comment

While the Auditor-General is disappointed that the Department of the Environment and Energy breached its confidentiality obligations under the Auditor-General Act 1997, the Auditor-General is satisfied with the department’s actions outlined above to respond in this instance and to minimise the risk of future similar breaches.