Introduction

1.1 Since the mid-1980s public sector management frameworks have emphasised the importance of measuring program performance. While the frameworks have changed over the years, the fundamental aim has remained consistent—to be able to measure and assess the impact and outcomes of government programs, not just the inputs, activities and outputs. Some of the frameworks have also included a focus on program efficiency. This enables public sector management to regularly assess the impact and outcomes of programs, and realign resources where necessary, to achieve the required government outcomes on behalf of the Parliament and the public.

1.2 The current performance measurement and reporting requirements for Commonwealth entities (corporate and non-corporate) are established under the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and the accompanying Public Governance, Performance and Accountability Rule 2014 (PGPA Rule). The introduction of the PGPA Act was intended to establish a strong performance reporting system to demonstrate to the Parliament and the public that resources are being used effectively and efficiently.

1.3 These requirements, and any formal guidance issued by the Department of Finance (Finance), are collectively referred to as the Commonwealth performance framework (the framework). The framework is principles based and focuses on entities providing sufficient performance information to users, including operating context, to allow an assessment of progress against intended purposes. The aim is to provide users with a greater understanding of how entities have utilised resources, not just in producing outputs, but also the impact and efficiency in delivering outcomes. The introduction of the requirement for the publication of entity information on efficiency is a key part of the Commonwealth performance framework, and will typically be presented alongside effectiveness measures.⁶

1.4 The Auditor-General’s responsibilities, as set out in the Auditor-General Act 1997, include auditing the annual performance statements (performance statements) of Commonwealth entities in accordance with the PGPA Act. The PGPA Act does not require the Auditor-General to conduct annual audits of performance statements, unless requested by either the Minister for Finance or the responsible Minister. This means that the Parliament does not receive assurance, as a matter of course, on performance statements included in annual reports, as it does over financial statements, where an independent audit report is mandatory.

1.5 The Auditor-General has flagged to the Joint Committee of Public Accounts and Audit (JCPAA) and the Parliament, the intention to position the ANAO to conduct annual audits of performance statements. This would provide a similar level of assurance to the Parliament and the public as provided by mandatory annual audits of financial statements.⁷ The JCPAA has provided its support for this approach and in Report 469: Commonwealth Performance Framework, released on 7 December 2017, recommended amending the PGPA Act in this regard.

Commonwealth performance framework

1.6 The Commonwealth performance framework came into effect with the commencement of the PGPA Act. It has three inter-dependent elements—purposes, operating context and performance information—used to demonstrate the achievement of purposes.⁸ These elements are demonstrated below in Figure 1.1.

Figure 1.1: Elements of the Commonwealth performance framework

 

 

Source: The Department of Finance, 2017–2018 Corporate Plan Lessons Learned, November 2017, p. 2.

1.7 Portfolio Budget Statements (PBS), which set out planned financial performance, are also required to describe at a strategic level, the outcomes intended to be achieved with the funding appropriated by the Parliament. The performance measures presented in the PBS are required to be a strategically focused subset of those included in an entity’s corporate plan.⁹ Entities are also required to clearly map the measures from their PBS to their purposes, which are published in their corporate plans.

1.8 As the primary planning document¹⁰, the corporate plan sets out planned non-financial performance and provides the reader with an understanding of how an entity intends to measure and assess its actual performance. The alignment of the corporate plan and PBS provides readers an insight into the expenditure expected to achieve that performance.

1.9 Entities are required to publish corporate plans at the beginning of each reporting cycle. Corporate plans must set out entities’ strategies for achieving their purposes, and measuring progress. Entities are also required to prepare performance statements at the end of the reporting period. These are to be included in entities’ annual reports. Performance statements provide an assessment of the extent to which the entity has progressed in achieving its purposes, as set out in the corporate plan.¹¹ Key elements of the Commonwealth performance framework, and the broader Commonwealth Resource Management Framework, are set out in Figure 1.2.

Figure 1.2: Elements of the Commonwealth Resource Management Framework

 

 

Source: ANAO analysis of guidance from the Department of Finance, incorporating relevant legislation.

1.10 The elements of the framework are intended to improve the line of sight between the use of public resources and the results achieved by entities. The intended users of the PBS, corporate plan, and performance statements are the Parliament and the public. It is for this reason that an entity must carefully consider whether the level of performance information presented in these documents will meet those users’ needs. Guidance from the Department of Finance (Finance) advises that performance information can be categorised by how it communicates accountability, strategic, tactical, or management related information, as demonstrated in Figure 1.3.

Figure 1.3: Performance information hierarchy

 

 

Source: ANAO analysis of the Department of Finance, Resource Management Guide No.131: Developing Good Performance Information, April 2015, pp. 12–13, 30 & 45.

1.11 Accountability performance information demonstrates whether the use of public resources is making a difference and delivering on government objectives. Finance guidance makes it clear that this is the level of performance reporting that is the focus of the PGPA Act, and made public through corporate plans and performance statements. Performance reporting for accountability purposes is of most interest to the Parliament and the public¹², and should be balanced with the other levels of the performance information hierarchy.

1.12 Well-presented and easily interpreted accountability information is essential to enable governments to coordinate policy, clarify objectives, enhance transparency and accountability, improve service delivery, and keep the wider community informed. While strategic, tactical, and management performance information are also important, they should be used to support and advance accountability information, rather than replace it. Performance measures that address these lower levels of information, without sufficient connection to accountability information, may not be appropriate to include in the corporate plan, and consequently, the performance statements.¹³

1.13 As the key accountability documents under the Commonwealth performance framework, an entity’s PBS, corporate plan, and performance statements should provide the Parliament and the public with sufficient information to determine whether a measure addresses the accountability, strategic, tactical or management level of the performance information hierarchy. If a user is required to rely on an in-depth understanding of an entity, the basic information needs of the Parliament and public are not being met. In this circumstance, the performance measures and the PBS, corporate plan and/or performance statements should be reviewed.

1.14 In considering the balance of performance information presented in an entity’s corporate plan, the number of performance criteria presented against the purpose should also be considered. The ANAO conducted a review of the performance criteria presented in the 2017–18 Corporate Plans for 22 major Commonwealth entities.¹⁴ The intention of the review was to identify and compare the number of performance criteria presented across a number of entities’ corporate plans.

1.15 In compiling this information, the ANAO noted the varied approaches applied by each entity in designing their performance information. For example, some entities adopted a tiered approach, with an overarching performance measure framed as an objective, supported by multiple subordinate measures addressing the current year, and also the forward estimates period. In these circumstances, the ANAO totalled all criterion presented. The results of this review are presented in Figure 1.4 on the following page.

1.16 The wide-range in the number of performance criteria presented in 2017–18 Corporate Plans indicates that entities are embracing the flexibility provided by the framework to design performance information tailored to their circumstances. Finance guidance notes the quality of performance information should be emphasised over quantity, recommending a small set of measures that is sufficiently comprehensive to cover those factors that affect an activity’s performance.¹⁵ For those entities with greater than 30 performance criteria, it may be appropriate to confirm that an appropriate balance of performance information, against the different levels of the performance information hierarchy, is being achieved.

Figure 1.4: Performance criteria presented in 2017–18 Corporate Plans

 

 

Note: The number of performance criteria presented in the figure above were confirmed by each of the listed entities.

Source: ANAO analysis of 2017–18 Corporate Plans.

Roles and responsibilities

Minister for Finance and the Department of Finance

1.17 The Minister for Finance is the Minister responsible for administering the PGPA Act. The Finance Minister may, by legislative instrument, make rules prescribing matters required or permitted by the Act to be prescribed by the rules, including those relating to performance.¹⁶ On 25 April 2015, the Finance Minister amended the PGPA Rule to reflect the requirements for entities to publish corporate plans and annual performance statements from 2015–16.

1.18 As noted previously, under section 40 of the PGPA Act, the Minister for Finance or a responsible Minister may request the Auditor-General to examine and report on the annual performance statements of an entity. As noted in paragraph 1.5, the Auditor-General intends to position the ANAO to conduct annual audits of performance statements.

1.19 The Minister for Finance is supported by the Department of Finance (Finance). Finance is responsible for the whole-of-government administration of the Commonwealth performance framework and related legislation. Section 112 of the PGPA Act requires that an independent review of the operation of the PGPA Act and Rule be conducted as soon as practicable after 1 July 2017. The Minister for Finance, in consultation with the JCPAA, commissioned this review in October 2017.¹⁷ The review is expected to be finalised in early 2018. The objectives of the review are to:

• examine whether the operation of the PGPA Act and Rule is achieving the objects of the PGPA Act in a manner consistent with the guiding principles.
• identify legislative, policy or other changes or initiatives, to enhance public sector productivity, governance, performance and accountability arrangements covered by the PGPA Act.
• examine whether policy owners’ implementation of the PGPA Act and Rule has appropriately supported their operation in Commonwealth entities.¹⁸

Joint Committee of Public Accounts and Audit

1.20 On 17 October 2016 the JCPAA announced an inquiry focused on the Commonwealth performance framework and, in particular, the implementation of appropriate performance information by departments to strengthen accountability. The purpose of the inquiry was to improve the Commonwealth performance framework and ensure line of sight between the use of public resources and the outcomes achieved by Commonwealth entities—a long-term focus of the JCPAA. Public hearings for this inquiry were held on 23 November 2016 and 6 September 2017, and were based on ANAO Reports:

• No.58 2016–17 Implementation of the Annual Performance Statements Requirements 2015–16;
• No.6 2016–17 Corporate Planning in the Australian Public Sector; and
• No.31 2015–16 Administration of Higher Education Loan Program Debt.

1.21 On 6 December 2017 the JCPAA released Report 469: Commonwealth Performance Framework. The report included ten recommendations which are reproduced in Appendix 2. Five of those recommendations related to the committee’s consideration of ANAO Report No.58 2016–17. Recommendation 6 was that ‘The Australian government amend the PGPA Act to enable mandatory audits of performance statements by the Auditor-General …’.¹⁹ The Committee also referred this matter to the attention of the Independent Review of the PGPA Act. As noted in paragraph 1.5, the Auditor-General intends to position the ANAO to meet this remit if the PGPA Act is amended.

1.22 The report also included observations in regard to entity performance measurement and reporting relevant to this audit, including the need:

• to use a mix of quantitative and qualitative performance information, along with relevant contextual information and analysis, to focus on entity impacts and outcomes (reflecting the move away from key performance indicators based solely on measuring inputs and outputs);
• for narrative utilised as part of qualitative performance information to be evidence-based, reliable and robust;
• for further work on measurement methodologies for qualitative performance information, drawing on local and international research and practice in this area;
• for further collaborative work on measuring and articulating performance outcomes, to build consistency and maximise reporting efficiencies; and
• for methodologically robust attribution of entity activities to outcomes that makes accountabilities clear.

Responsible Ministers

1.23 Under the PGPA Act, a responsible Minister may:

• access the records kept about the performance of the entities within their portfolio (section 37); and
• request that the Auditor-General examine and report on the performance statements of entities within their portfolio (section 40).

1.24 A responsible Minister, who makes a request of the Auditor-General, is required to table the report in each House of Parliament with the accompanying performance statements as soon as practicable after the report is received.

Accountable Authorities

1.25 Accountable authorities are responsible for the implementation of the requirements of the Commonwealth performance framework in their entities. Part 2–3 of the PGPA Act—relating to planning, performance and accountability—sets out the requirements of accountable authorities. The requirements relevant to this audit include:

• preparing a corporate plan each reporting period that complies, is published, and is provided to the responsible Minister and Finance Minister in accordance with any requirements prescribed by the PGPA Rule;
• keeping records about the entity’s performance in accordance with any requirements prescribed by the Rule;
• measuring and assessing the entity’s performance and complying with any requirements prescribed by the Rule; and
• preparing annual performance statements about the entity’s performance that comply with any requirements prescribed by the Rule, and including these statements in the annual report.

Audit Committees

1.26 Audit committees are appointed by the accountable authority of an entity. The functions of an audit committee are prescribed by section 17 of the PGPA Rule, and must be set out by the accountable authority in a written charter. The required functions of an audit committee are detailed below.

Source: PGPA Rule.

1.27 In July 2017, Finance released Functions of audit committees: Reviewing the appropriateness of performance reporting, including for 2016–17 performance statements. The document provides additional guidance to audit committees to assist in discharging their functions under the PGPA Rule, in particular reviewing the appropriateness of an accountable authority’s performance reporting. The guidance reiterates the audit committee’s role and function, and makes explicit that audit committees are to provide a reasonable level of assurance as required by the PGPA Rule.

1.28 As presented in Appendix 2, the JCPAA Report 469 includes a recommendation that the PGPA Act and accompanying rules and guidance be amended, as necessary, to clarify that audit committee’s functions and charters must reflect their role in providing assurance of the appropriateness of performance reporting.

Requirements for annual performance statements

1.29 Performance statements under the PGPA Act are required to be published in entities’ annual reports. Specific requirements for the presentation of performance statements are outlined on the following page.

Source: PGPA Rule.

1.30 Section 37 of the PGPA Act sets out the requirement for Commonwealth entities to keep records that properly record and explain the entity’s non-financial performance as outlined below.

Source: PGPA Act.

Audit objective, rationale, criteria and scope

1.31 The objective of the audit was to continue to examine the progress of the implementation of the annual performance statements requirements under the PGPA Act and the PGPA Rule by the selected entities. The audit was also designed to:

• provide insights to entities more broadly, to encourage improved performance; and
• continue the development of the ANAO’s methodology to support the possible future implementation of annual audits of performance statements.

1.32 To form a view against the audit objective, the following high level criteria were adopted:

• the selected entities complied with the requirements of the PGPA Act and PGPA Rule;
• the performance criteria presented in the selected entities’ PBS, corporate plans, and performance statements were appropriate²⁰;
• the selected entities had effective supporting frameworks to develop, gather, assess, monitor, assure and report performance information; and
• sufficient records were retained to support the results reported by the selected entities against the performance criteria in the performance statements.

1.33 The audit involved assessment of the appropriateness (relevance, reliability and completeness) of the performance criteria, and the completeness and accuracy (fair presentation) of reporting. This was completed for either a subset, or all of the performance criteria presented by the selected entities in their 2016–17 Performance Statements.

1.34 Under the PGPA Act and Rule, an entity’s purpose may be their objectives, functions or role. This has led to variability in the language used by entities when labelling elements of their corporate plans, as demonstrated in Figure 1.5, and reduces inter-entity comparability for readers. This may be an area where further consideration or guidance by Finance may assist in providing additional clarity across entity performance reporting.

Figure 1.5: Comparison of corporate plan elements

 

 

Source: ANAO analysis.

1.35 The audit considered the performance criteria established by each of the selected entities to demonstrate progress against the following elements of their corporate plans:

• Employment’s performance criteria for ‘Outcome 1—[Employment services]’;
• Environment’s performance criteria for ‘Environment and Heritage Protection—Conserve, protect and sustainably manage Australia’s biodiversity and heritage’;
• Austrade’s performance criteria for all purposes; and
• the Sports Commission’s performance criteria for its purpose.

1.36 The audit involved reviewing:

• internal systems, processes, and procedures, including the governance and oversight put in place by entities to support their development of the annual performance statements;
• records, and interviews of staff of the selected entities; and
• the selected entities’ 2017–18 PBS and Corporate Plans to identify any further opportunities for improvements to its performance measurement and reporting that may be addressed in the 2017–18 Performance Statements, or 2018–19 PBS and Corporate Plans.

1.37 The audit also included reviewing Finance’s role in whole-of-government administration of the annual performance statements requirements, including:

• administering the PGPA Act and Rule;
• publishing resource management guides and supporting papers on selected areas of the framework;
• continuing to host communities of practice to provide a forum to share examples of better practice, and improving the feedback loop between framework design, implementation and results; and
• providing advice to entities as requested.

1.38 The audit was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $536,902.

1.39 The team members for this audit were Jennifer Hutchinson, Alicia Vaughan, Kara Ball, Taela Edwards and Michael White.

Other ANAO audit coverage

1.40 This performance audit is one of three follow-on audits identified in the ANAO’s current work program that address key aspects of the implementation of the PGPA Act. The other two follow-on audits are:

• Corporate planning in the Australian Public Sector. This performance audit commenced in August 2017 and is the third in a series of audits that is assessing progress in implementing the corporate planning requirement under the PGPA Act; and
• Management of Risk by Public Sector Entities. This performance audit would be the second in a proposed series, which would assess how effectively selected public sector entities manage risk, including compliance with the Commonwealth Risk Management Policy.

1.41 The most recent audits in this series are ANAO Report No.54 2016–17 Corporate Planning in the Australian Public Sector²¹, and ANAO Report No.6 2017–18 The Management of Risk by Public Sector Entities. Similar to the observations of ANAO Report No.58 2016–17, both of the audits concluded that further work is required by entities to fully address the core elements of the PGPA Act.²²

1.42 The ANAO’s continued focus in these areas is expected to assist in keeping the Parliament, the government, and the community informed on implementation of the resource, risk and performance management frameworks introduced by the PGPA Act and to provide insights to entities to encourage improved performance.

Have entities met the requirements of the Public Governance, Performance and Accountability Act 2013 and accompanying Rule?

2.1 Table 2.1 outlines the PGPA Act and Rule requirements for the presentation of annual performance statements. To determine whether entities met these requirements, the ANAO reviewed whether the entity had published performance statements that contained the basic elements set out in the PGPA Act and Rule (statements, results and analysis). This assessment is summarised in Table 2.1 below. All of the entities met the basic requirements to publish performance statements under section 39 of the PGPA Act. However, improvements to the quality of results and analysis presented in Employment’s, Environment’s and the Sports Commission’s performance statements are required to provide meaningful information to the Parliament and the public—one of the objects of the PGPA Act.²³

Source: ANAO analysis against PGPA Act and Rule requirements.

Statements

2.2 Each of the selected entities’ 2016–17 Performance Statements included the required statement by the accountable authority. The ANAO noted that Employment and the Sports Commission both included an additional statement that the performance statements were ‘based on properly maintained records’, which is above the minimum requirement set out above. Austrade’s accountable authority also noted that the statement was made ‘based on advice from Austrade management and the agency’s Audit and Risk Committee’.

Results

2.3 All four entities either addressed, or mostly addressed the requirement, under section 16F of the PGPA Rule, to provide the results of the measurement and assessment of their performance in achieving their purpose. Those entities requiring improvement to fully address the PGPA Rule were Employment, Environment and the Sports Commission. These entities did not fully meet the requirement that results are presented in accordance with the method set out in the corporate plan and PBS (Item 2: Results).

2.4 Finance guidance indicates that a corporate plan should include a description of performance measures, when they will be reported on, the data collection techniques to be used and any targets the performance measures will be assessed against.²⁴ Employment, Environment, and the Sports Commission did not disclose or clearly attribute data collection techniques or a method for assessment for all of their performance criteria in their corporate plans, or performance statements, and as a result could not fully meet this requirement. In addition, Environment presented results that did not address its performance criteria in some circumstances, while the Sports Commission did not present all performance criteria outlined in its corporate plan, or changed targets without adequately explaining the reasoning to the reader.

2.5 The consistency and completeness of the presentation of performance criteria and targets across the PBS, corporate plan and performance statements is important to establish a clear read. Guidance from the Department of Finance notes that entities should outline the reasoning for any changes to performance criteria and/or targets during the performance reporting cycle, in the corporate plan or performance statements.²⁵

Employment

2.6 Employment did not disclose in its 2016–17 Corporate Plan, or Performance Statements, the data collection techniques, or methodologies, intended to be used to present results against its performance criteria. An absence of this information makes it difficult for a reader to determine whether the result is an accurate reflection of the measure as described, and whether the result presents an unbiased view of performance.

Environment

2.7 While Environment identified selected data sources, systems and methods in the ‘Assessment’ section for each activity, these were not explicitly linked to individual performance criteria and targets. Consequently, it could not be determined whether the assessment information presented by Environment was complete, and the requirement under PGPA Rule, section 16F(1) Item 2: Results, was not fully addressed.

2.8 In addition, the reported result for approximately a third of the measures did not fully address the criterion/target. For example, the result for ‘Increase in the proportion of approved assessments that apply streamlining policy initiatives’ under the Environment Protection and Biodiversity Conservation Act 1999 was reported to be ‘Achieved’. However, in the commentary supporting the result, the total number of assessments, or the proportion that applied such initiatives was not disclosed²⁶, except to note that it was ‘the majority’. The commentary also did not specify whether an ‘increase’ had occurred. In this instance, Environment noted in the performance statements that it was improving internal reporting and data collection so that trends (such as an increase or decrease) can be ‘more meaningfully reported in future years’.

Austrade

2.9 While Austrade met the requirements for the presentation of its performance statements, the ANAO noted that four performance criteria were included in the 2016–17 PBS (May 2016) that were not repeated in the corporate plan (August 2016). The corporate plan is intended to be the primary planning document, and should contain a complete picture of an entity’s planned non-financial performance. Section 16E of the PGPA Rule requires an entity to include in its corporate plan ‘any measures, targets and assessments that will be used to measure and assess the entity’s performance for the purposes of preparing the entity’s annual performance statements under section 16F’. The approach adopted by Austrade, in presenting some performance criteria only in the PBS, does not meet this requirement.

2.10 In addition, Austrade’s 2016–17 Corporate Plan did not include targets for its performance criteria. The only exceptions were where a target was explicit in the criterion itself, such as ‘Double overnight visitor expenditure to between $115 billion $140 billion by 2020’. However, Austrade did present ‘forecasts’ of performance against the majority of performance criteria included in the 2016–17 PBS, which were then reported against in the 2016–17 Performance Statements.²⁷ In accordance with the Finance Secretary Direction²⁸, Austrade also included forecasts of performance against all, except one²⁹, of its 2016–17 performance criteria in the 2017–18 PBS. Discussion of the impact of the absence of targets on the appropriateness of Austrade’s performance criteria can be found in paragraph 2.70.

Sports Commission

2.11 For the majority of its performance criteria, the Sports Commission did not present the method of assessment, or accompanying parameters in the corporate plan, to enable a complete assessment against Item 2: Results. In addition, the results presented alongside some performance criteria and targets in the performance statements did not reflect the measure as described. For example, the result for ‘Increase in the percentage of Australians participating in organised sport’ [compared to 2015–16] presents the number of participants in 2016–17, rather than an assessment of whether this represented an increase from 2015–16.

2.12 The ANAO also noted differences between the performance criteria presented across the PBS, corporate plan, and performance statements, including instances where only a target was presented without the accompanying performance criterion. For example, in the PBS the target for the performance criteria ‘Improved organisational capability of NSOs’ is: ‘the organisational capability assessment of national sporting organisations shows an improvement across the four components of the organisational development tool compared to the baseline established in 2015–16’. However, the corporate plan presents the target as ‘Baselines established across the four components of the organisational development tool’, and is presented in the performance statements without the original performance criterion from the PBS and corporate plan. As results are required to be presented in accordance with the method set out in the corporate plan and PBS, all performance criteria and targets should have been presented in the Sports Commission’s performance statements to fully address the requirements of the PGPA Rule.

Analysis

2.13 The ANAO noted there are opportunities for Employment and the Sports Commission to increase the quality of analysis presented in the performance statements. All four entities would have also benefitted from presenting analysis that addressed the purpose directly. In most cases, the entities’ analysis was limited to the performance criteria without considering how they collectively demonstrated progress against the overall purpose. Environment’s analysis should also clearly acknowledge and consider the department’s specific contribution, as compared to those of private and state and territory government delivery partners.

2.14 As noted in Figure 1.1, the operating context of an entity (risk, capability, and environment) is a key driver of an entity achieving its purpose. There is an expectation that analysis in performance statements establishes a stronger connection with the operating context elements outlined in the corporate plan. For example, where risks are outlined in the corporate plan, they would be acknowledged and discussed in the context of their impact on an entity’s performance during that reporting period in the performance statements. By providing more detailed analysis in regard to these elements, users are provided with a deeper understanding of the cyclical nature of performance, and the effectiveness of an entity in anticipating and/or addressing matters affecting their performance during the reporting period.

Employment

2.15 As demonstrated in Figure 1.5, Employment’s corporate plan is structured to reflect its purpose, which is supported by outcomes and lower level objectives. Employment’s performance criteria have been set at the objective level and are accompanied by one overarching outcome measure. Employment’s performance statements present analysis to demonstrate progress at the objectives and outcome levels, however both sets of analysis are limited and do not adequately explain the key drivers of overall performance.

2.16 The analysis presented at the objective level provides some discussion of the results for the performance criterion within each objective, however does not necessarily address the results of the criteria collectively. For example, the analysis presented for ‘Objective 1—Job seekers find and keep a job’ discusses one of the five performance criteria, focusing on the criterion ‘Count of job placements’ target that was not met. There is no analysis of what contributed to the other targets being achieved, or how the group of measures should be considered together to determine progress against the objective or purpose. In addition, the Department’s explanation for not achieving the target was:

Although labour market conditions improved during 2016–17, employment growth remains below the annual growth rate for the past decade. This has limited the number of available job opportunities in 2016–17, the second year of the jobactive program.

2.17 While employment growth may contribute to the number of job placements, the performance statements would have benefited from discussion of the factors able to be influenced through jobactive specifically.

2.18 The analysis presented at the ‘Outcome 1’ level was also limited and did not provide an indication of Employment’s overall progress against its purpose. It described the core groups which are the subject of historically higher unemployment rates, and that the Department has implemented programs to support this, however it doesn’t adequately address the progress of those activities against the overall outcome.

Environment

2.19 Analysis is presented in Environment’s performance statements for each of its key activities (see Figure 1.5). Together, the analysis sections include discussion of operational factors, such as government policy decisions and market conditions that impacted on the delivery of these activities. However, the quality of analysis could have been improved by more clearly indicating how the department’s activities, and the other factors identified, may have affected the achievement of its purposes in the reporting period. This includes demonstrating the department’s specific contribution towards the reported result.

2.20 For example, reporting against the target for ‘Continued implementation of the Reef 2050 plan’, lists various milestones and achievements against the plan. However, the results and analysis do not demonstrate how the department’s activities contributed to these achievements. This is particularly important, as 11 other agencies, supported by delivery partners, are also responsible for the delivery of actions under the plan, including the Queensland Government and the Commonwealth Great Barrier Reef Marine Park Authority. Reporting the attribution of an entity’s activities towards an objective that relies on multiple stakeholders, whether inter or intra-governmental or private enterprise, is an area that requires greater focus by entities. This is also an area of the framework and guidance that may benefit from further consideration by the Department of Finance.

Sports Commission

2.21 The analysis presented in the Sports Commission’s performance statements is aligned to the four ‘strategic pillars’ supporting its purpose (refer to Figure 1.5). Each section provides a discussion of the results for the relevant performance criteria and includes activities that were undertaken during 2016–17, or underway for the following reporting period, that contributed to the Sports Commission’s performance. In two cases, the connection between the activities and achievements outlined in the analysis section to the results presented against the performance criteria was unclear. The Sports Commission would benefit from better establishing this connection to assist a reader to understand those matters influencing the Commission’s performance.

2.22 In addition, in some cases where a target had not been achieved, for example ‘90 per cent of targeted NSOs demonstrated satisfactory progress towards compliance with the Mandatory Sports Governance Principles’, there was limited analysis of what contributed to the target not being met. In this example, the Sports Commission did describe a number of activities or initiatives that appeared to be intended to improve this result, but as noted above, without making this relationship clear, a reader’s understanding is potentially limited.

Other annual reporting requirements

Plain English and clear design

2.23 The purpose of the Commonwealth performance framework is to enhance the transparency and accountability of the public sector.³⁰ As noted in Chapter 1, Finance guidance³¹ highlights the aim of the PGPA Act to improve the line of sight between what is intended and what is delivered. To support this aim, sections 17AC, 17BD and 28D of the PGPA Act include provisions for ‘Plain English and clear design’, in relation to commonwealth entities’ and companies’ annual reports, including:

• annual reports must be prepared having regard to the interests of the Parliament and any other persons who are interested in the annual report; and
• requiring information in the annual report to be relevant, reliable, concise, understandable and balanced, including through clear design and defining technical terms.

These requirements provide for clear interpretation of the annual report, including the performance statements, by users.

Environment

2.24 Finance guidance notes that ‘good performance reporting is not about the volume of data; it is about using quality data to support better decision-making and better assessment of performance’.³² While each of the three main sections of Environment’s 2016–17 Performance Statements (activity, performance criteria and results, and analysis) have a different focus depending on its purpose, a combination of activity descriptions, results and analysis are presented against all three. When combined with the volume of information presented against each criterion, and results being presented at the performance criterion and target level, Environment’s 2016–17 Performance Statements are not presented in a way that provides a clear view of the outcomes of its activities.

2.25 Environment adopted a tiered approach to the development of its 2016–17 performance criteria. Performance criteria were intended to be set at a higher level, which were then supported by more detailed targets. However, there was no clear differentiation or alignment between the criterion and targets in the 2016–17 Corporate Plan to determine how they were intended to be viewed together to demonstrate progress against the overall purpose. In some instances, a performance criterion was also repeated against multiple purposes, without any further information to determine how they were intended to be measured differently in each.

2.26 Using a performance criterion with multiple targets complementarily provides the opportunity to present information across each level of the performance information hierarchy. Where this approach is well-structured, a more comprehensive picture of performance can be achieved than would be possible with a single accountability/effectiveness measure. However, where this approach is not well structured, and an imbalance of lower-level information is presented, it is difficult for users to discern the actual progress of the entity. This is the case with Environment’s 2016–17 Performance Statements.

2.27 To enhance the line of sight between the corporate plan and performance statements, targets should be established as direct indicators of progress against performance criteria, rather than presenting them separately. Environment’s 2017–18 Corporate Plan has partly addressed this matter by mapping targets to specific performance criteria. This demonstrates how each target is intended to inform an assessment against the performance criterion to which it relates. The presentation of results and analysis against the revised structure in Environment’s 2017–18 Performance Statements will also need to be carefully considered, to ensure a clear line of sight.

Performance statements and the Annual Report

2.28 In the 2015–16 Annual Performance Statements Lessons Learned paper, Finance made the observation that the large size of some performance statements presents issues for the reader. It also noted that the intent of performance statements is to provide a concise, comprehensive and accurate picture of an entity’s performance. Finance recommends that, where the size of annual performance statements is considered to be an issue, consideration could be given to rationalising and reducing the number of performance criteria. Entities should review existing data collections and assessment methodologies to focus on information that is truly meaningful.³³

2.29 In considering the above, the ANAO totalled the number of performance criteria presented by each of the selected entities and compared it to the page span of their performance statements. The results of this comparison are provided in Table 2.2. Environment had the highest number of performance criteria (65) and also the longest performance statements (123 pages). The high number of performance criteria and targets, accompanied by the volume of results and analysis presented against each, contributed to the ANAO’s observations in the previous section regarding a lack of clear design and conciseness in Environment’s performance statements.

Table 2.2: Total number of performance criteria and performance statements’ pages

Source: ANAO analysis of the selected entities’ performance statements.

2.30 While the remaining entities’ performance statements appeared more concise in page span, the ANAO noted this was achieved by Employment and Austrade presenting, or directing readers to, additional performance information and analysis elsewhere in the annual report. For example, Employment’s annual report had a section titled ‘Our Performance’ spanning 63 pages that preceded the performance statements and presented more in-depth analysis and additional performance information.

2.31 Similarly, Austrade’s 16 page performance statements were presented within a section of the annual report titled ‘Report on performance’ (totalling 115 pages), and the performance criteria results directed readers to areas within the section, but outside of the performance statements, for further information and analysis. However, Austrade’s performance statements still presented sufficient information without referencing the additional information presented elsewhere in its annual report to fulfil the requirements of the PGA Act and Rule, as demonstrated in Table 2.1.

2.32 While this approach may be viewed by entities as a useful way to provide additional contextual information, the performance statements are intended to be the key accountability document to inform an assessment by the Parliament and the public of an entity’s performance. As a result, the performance statements should present all information and analysis necessary to inform that assessment. Finance guidance supports this view, noting that:

From the 2015–16 reporting period, annual performance statements will replace the report on performance section that is currently included in annual reports.³⁴

Did the entities’ corporate plans support performance measurement and reporting in the annual performance statements?

2.33 The key elements of the corporate plan that support performance measurement and reporting in the annual performance statements are an entity’s purpose/s and activities. Section 16E of the PGPA Rule requires that an entity’s corporate plan state the entity’s purposes over the next four years. The PGPA Act defines purpose/s as including the objectives, functions or role of an entity. Finance guidance notes that:

Well-expressed purpose statements make it clear who benefits from an entity’s activities, how they benefit and what is achieved when an entity successfully delivers its purposes. Essentially, purposes describe the value an entity seeks to create or preserve.³⁵

2.34 The aim of the purpose/s statement is to give context to the significant activities that the entity will pursue over the period covered by the plan, and should be stated in a relevant and concise manner. Finance has defined an activity as a distinct effort of an entity undertaken to achieve a specific result (fulfilling purpose/s). Activities should be aligned with the entity’s purpose/s, and be a focus for performance measurement and reporting.

Purpose

2.35 As demonstrated in Figure 1.5, the selected entities used different structures in the design of their purpose under the Commonwealth performance framework. The purpose expressed in the corporate plans of Employment, Environment, and Austrade demonstrate who will benefit from their respective activities, how they will benefit and the impact intended to be achieved in delivering against their purposes.

2.36 Employment’s purpose statement could have been more concise to better reflect the core elements of the purpose. The ANAO noted that in its 2016–17 Performance Statements, Employment removed some of the additional language used to reinforce the connection between its purpose and PBS outcomes, making the purpose more succinct.

2.37 Similarly, Environment’s purpose statement could be clearer. This may include using more measurable language to help users identify the intended impact, and clarify that the department often relies on other parties, including state departments, to assist in the achievement of national objectives.

2.38 The purpose expressed in the Sports Commission’s 2016–17 Corporate Plan demonstrates who will benefit from the Commission’s activities and how they will benefit. However, the intended impact of the Sports Commission could be better described in the purpose to reflect the outcome that is expected to be achieved.

2.39 The intended outcome of the Sports Commission’s activities, as described by the purpose, is strengthening Australian sport. However, this describes an intermediate output of activities rather than the intended impact on the population as a result of those outputs. Without adequately describing the intended impact, a reader cannot assess whether it is being achieved. By describing an output-focused purpose, the Sports Commission also limits the likelihood of designing measures that successfully demonstrate progress in achieving the intended impact.

Purposes and outcomes

2.40 As noted in paragraph 1.8, an entity’s corporate plan and PBS are intended to be complementary to establish alignment between the financial and non-financial performance of an entity, and facilitate an assessment of that performance by the Parliament and the public. Finance guidance notes that:

it does not assume that activities and purposes will necessarily equate to the ‘programs’ and ‘outcomes’ reported by Commonwealth entities in the PBS and other budget documents. However, to provide a clear line of sight between the allocation and use of public resources, entities will need to ensure that links can be made between the appropriations reported in the PBS and the performance information published in corporate plans and annual performance statements. How the links are made is left to the discretion of entities.³⁶

2.41 Establishing these links, and ensuring the funding and organisational structures established by an entity facilitate the development and collection of meaningful performance information and reporting, can be difficult. Examples of these difficulties are provided by Environment and Austrade’s corporate plans and performance reporting approaches.

Environment

2.42 The mapping of Environment’s PBS outcomes and programs structure to the purposes and activities presented in its corporate plan is complex, as demonstrated by Figure 2.1. This is particularly evident for the ‘Environment and Heritage’ purpose and activities, which span two outcomes and six programs. Where multiple programs address a single activity, the ANAO observed the department’s approach to the development of performance criteria was complex, and failed to coordinate a cohesive performance story, as previously noted in paragraphs 2.24–2.27.

Figure 2.1: Mapping of Environment’s 2016-17 Corporate Plan to PBS

 

 

Source: Reproduced from the Department of the Environment and Energy’s 2016–17 Corporate Plan, p.45.

Austrade

2.43 Entities are required to develop performance criteria for all programs presented under the outcomes in their PBS, which are also expected to align to an entity’s purpose in its corporate plan. Austrade presents two outcomes in its 2016–17 PBS:

• Contribute to Australia’s economic prosperity by promoting Australia’s export and other international economic interests through the provision of information, advice and services to business, associations, institutions and governments; and
• The protection and welfare of Australians abroad through timely and responsive consular and passport services in specific locations overseas.

2.44 Both outcomes are reflected in Austrade’s purpose presented in its 2016–17 Corporate Plan. While ‘Outcome 1’ represents Austrade’s responsibilities as set out in its legislation, ‘Outcome 2’ is a reflection of services provided by Austrade on behalf of the Department of Foreign Affairs and Trade (DFAT). This arrangement is governed under a Memorandum of Understanding between the two entities, and Austrade is funded directly via an appropriation to deliver these services. Austrade has developed performance criteria, as required, to reflect its delivery of services on behalf of DFAT.

2.45 As noted earlier in this report, the corporate plan is intended to serve as an accountability document to inform the Parliament and the public. While Austrade is funded to deliver these services, the performance measures are more indicative of the effectiveness of the management of a contractual arrangement, rather than representing a Commonwealth outcome in its own right. As a result, the level of accountability information presented by those measures is limited and may benefit from review in conjunction with DFAT and the Department of Finance. The way in which these types of arrangements are reported in the future is an area of the framework that may benefit from further consideration by the Department of Finance, particularly as collaboration across government continues to mature.

Activities

2.46 The ANAO assessed the activities listed by the selected entities in their 2016–17 Corporate Plans against the following criteria, sourced from Finance guidance³⁷:

• Are the activities readily identifiable?;
• Do the activities align with the entity’s purpose?;
• Do the activities clearly describe what actions the entity will undertake to achieve its purpose?; and
• Are the activities identified at an appropriate level?

2.47 Overall, all four of the entities’ activities presented in their corporate plans met, or mostly met, the assessment criteria. The ANAO has noted some minor improvements for Employment, Environment and the Sports Commission in the following paragraphs.

2.48 Activities should be aligned with an entity’s purposes, and be a focus for performance measurement and reporting. Employment described strategies to meet its objectives, which then contributed to the outcome and purpose. The positioning of the strategies and the language used to describe them denote that they are intended to be read as the activities of the department, but this could be clearer.

2.49 In addition, while it is clear from Employment’s strategies the actions it intends to undertake to contribute to achievement of its purpose, they have been presented as relevant only to 2016–17, rather than the entire four year period covered by the corporate plan. The PGPA Rule requires the corporate plan to address the entity’s purpose across the next four years, which would include the activities of the entity intended to meet that purpose. As presented, Employment’s corporate plan does not meet this requirement at the activity level. The ANAO notes Employment’s 2017–18 Corporate Plan has clearly presented its strategies across the four year horizon.

2.50 Some of Environment’s descriptions for the actions it intended to undertake could be made clearer to allow for more meaningful interpretation of the intended outcome and the specific nature of Environment’s intervention. In addition, while Environment’s performance statements mostly focus on significant activities, which contribute directly to the purpose, one-third of the delivery strategies (8 of 24) presented in the corporate plan describe minor or supporting activities which may be more appropriately monitored through internal planning and reporting.

2.51 Similarly, the Sports Commission’s activities were mostly presented at an appropriate level, with the exception of those under the ‘Organisational Excellence’ pillar. These focus on supporting, rather than significant, activities. The Sports Commission would benefit from reviewing the balance of significant activities, versus minor or internal management activities, presented in the corporate plan. The ANAO notes that the ‘Organisational Excellence’ pillar and accompanying activities were not presented in the Sports Commission’s 2017–18 Corporate Plan.

Did the entities’ performance criteria support the reporting of progress against the purposes?

Performance information hierarchy

2.52 As noted at paragraph 1.11, performance information in an entity’s corporate plan and performance statements should address the accountability level of the performance information hierarchy. This level of information is essential to enable government to coordinate policy, clarify objectives, enhance transparency and accountability, improve service delivery, and keep the wider community informed. While lower levels of performance information are also important, they should be used to support and advance accountability information.

2.53 Tactical and strategic information, which are focused on the activities and outputs that are intended to lead to fulfilment of an entity’s purpose, may be used to support higher level accountability information. Management information is an important input to determining whether an accountable authority is addressing their duties to govern an entity as set out in the PGPA Act.³⁸ However, there is limited benefit for the Parliament and public where this information is solely used to demonstrate performance at the purpose level, as the connection can be too remote.

2.54 Table 2.3 summarises the mixture of management, tactical, strategic and accountability level information presented by each entity’s performance criteria in their 2016–17 Performance Statements. Appendix 4 presents the ANAO’s assessment of each of the selected entities’ performance criteria considered by this report and the level of the hierarchy it was categorised as. As noted in paragraph 1.13, making an assessment of which level of the performance hierarchy is addressed by a performance criterion is influenced by the information presented in an entity’s corporate plan. This may lead to differences in the categorisations applied by the ANAO when further contextual information outside of the corporate plan is considered.

2.55 The Sports Commission was the only entity that did not present performance criteria that addressed the accountability level of the hierarchy and, of the four entities, had the highest proportion of criteria at the management level (31 per cent). Without an adequate balance of performance information, particularly at the higher levels of the performance hierarchy, the Parliament and the public are not able to adequately assess an entities’ progress in delivering its intended impact.

2.56 The remaining entities displayed a mixture of performance criteria across the different levels of the hierarchy. All of Employment’s performance criteria addressed the top two tiers of the hierarchy, and 73 per cent of Austrade’s were similarly focused. More than half of Environment’s performance criteria addressed the two lowest levels of the performance information hierarchy (management or tactical), and would benefit from a review to increase the proportion of strategic and/or accountability information provided.

Table 2.3: Assessment of the selected entities’ performance information hierarchy

Source: ANAO analysis, as provided to entities during the progress of the audit.

2.57 The 17 performance criteria that presented management level information were excluded from the ANAO’s assessment for appropriateness in the following section.

Appropriateness of entities’ performance criteria

2.58 The ANAO assessed the selected entities’ performance criteria for relevance, reliability and completeness. The basis for this assessment is drawn from the characteristics of ‘good’ performance information as defined by Finance.³⁹ The detailed criteria can be found at Appendix 3.

2.59 A summary of the ANAO’s assessment of whether the elements of appropriateness were suitably addressed by the selected entities’ performance criteria is presented in Table 2.4. The scale used to rate the performance criteria was:

• displayed all of the characteristics of the criterion (Yes);
• displayed most of the characteristics of the criterion (Mostly);
• displayed in part the characteristics of the criterion (Partly); and
• did not display the characteristics of the criterion (No).

2.60 The ANAO notes that the characteristics of the ‘relevant’ and ‘reliable’ criteria are linked, and observations made during an assessment against those characteristics may have contributed to reporting for more than one criterion.

Table 2.4: Summary of ANAO assessment of the appropriateness of the selected entities’ performance criteria

Source: ANAO analysis.

Relevant—benefit, focus, and understandable

2.61 In applying the ‘relevant’ criterion, the ANAO assessed whether each of the selected entities’ performance measures under review:

• clearly indicated who benefited and how they benefited from the entity’s activities;
• was focused on a specific aspect of the entity’s purpose/s and activity/ies, and the attribution of the result to the entity is clear; and
• was easily understandable.

2.62 Overall, the majority of the selected entities’ performance criteria were assessed as either demonstrating all, or most of, the characteristics of relevance. The summarised results of the ANAO’s assessment of the relevance of the selected entities’ performance criteria are presented in Table 2.5 below.

Table 2.5: Summary of ANAO assessment of the relevance of the selected entities’ performance criteria

Source: ANAO analysis.

2.63 Across the four entities, there were some common reasons for performance measures being assessed as mostly or partly relevant. Most commonly, measures were not understandable due to the inclusion of entity specific terms and/or technical language which were not adequately defined. This limited a readers’ ability to understand what was being measured. For example in Austrade’s corporate plan ‘investment outcomes’ are referenced in three separate criteria, and a fourth refers to ‘foreign direct investment outcomes’. Without defining these terms, readers are unable to determine what the criterion is intending to address, or whether they are measuring the same, or a different activity. Where a performance criterion was not understandable, in some cases, a reader’s ability to determine the benefit or beneficiary considered by the performance measure was also affected.

2.64 In other cases, although the measure was understandable, and the benefit and beneficiary clear, how the activity being measured contributed to the entity’s purpose, or the entity’s specific contribution to the activity, was not, affecting an assessment of the criterion’s ‘focus’. This was demonstrated by 12 of Environment’s measures. The description or language used by Environment in its measures was too broad in some cases to enable the reader to determine the alignment to the purpose, for example ‘meets its [Environment’s] obligations’. There were also examples where the measure was too narrow to provide an understanding of how it contributes significantly to achieving the purpose. This is demonstrated by the performance criterion: ‘All required assessments under Article 3.2 of the Ramsar Convention (change in the ecological character description of Ramsar wetlands) are completed consistent with obligations under the Convention’—referring to a specific article of a broader convention, without sufficient information to determine how it contributes significantly to the purpose, affects the focus of the criterion.

Reliable—measurable and free from bias

2.65 In applying the ‘reliable’ criterion the ANAO assessed whether each performance measure:

• was measurable, that is, it used and disclosed information sources and methodologies (including a basis or baseline for measurement or assessment, for example a target or benchmark) that were fit-for-purpose and verifiable; and
• was free from bias, allowing for clear interpretation and an objective basis for assessment of the results.

2.66 The majority of the entities’ performance criteria were commonly either mostly or only partly reliable, while a smaller proportion fully demonstrated the characteristics (seven), or not at all (ten). The summarised results of the ANAO’s assessment of the reliability of the selected entities’ performance criteria are presented in Table 2.6 below.

Table 2.6: Summary of ANAO assessment of the reliability of the selected entities’ performance criteria

Source: ANAO analysis.

2.67 In the case of Employment and the Sports Commission, the key matter that affected the reliability of their measures was the absence of a description of the methodology, data and/or information intended to be used to determine the results. Finance guidance notes entities are required to include this information in the planned performance section of their corporate plans.⁴⁰ It is important for a reader to be able to understand the basic methodology supporting a measure for them to determine whether it can be relied on. Specific commentary of the key matters affecting the reliability of some of Environment and Austrade’s performance criteria is provided below. An assessment of whether the methodologies supporting each of the selected entity’s performance criteria were suitable is presented in Chapter 3.

Environment

2.68 As noted in paragraph 2.7, the alignment of Environment’s performance measures to assessment methodologies presented in the corporate plan was unclear in most circumstances. In multiple cases, neither the performance criterion, nor the accompanying assessment information, indicated how results would be measured or the timeframe for measurement. These matters also affected an assessment of the potential for bias. For example, for the target ‘200 taxa revised or newly described’, the boundaries/parameters of when a revision would be made, or what constituted a revision was not clear. As a result, a reader could not determine what constituted a valid revision and whether the reported result was a fair representation of the department’s performance during the year. Providing a reference to the framework for revisions would assist the reader in understanding how the target would be assessed and improve a reader’s ability to assess for potential bias.

2.69 Undefined key terms also impacted the reliability of Environment’s performance reporting. Similarly, the absence of a baseline or target against which to measure success affects the reliability of a measure. This is demonstrated by the performance measure ‘Environmental watering improves the condition of water-dependent ecosystems in the Murray-Darling Basin.’ Without defining ‘environmental watering’ or providing a baseline for comparison or a target for achievement, the reader cannot determine what is being measured, and whether what is reported by the entity represents an improvement.

Austrade

2.70 As noted in paragraph 2.10, when reviewing Austrade’s performance criteria, the ANAO noted that no targets were presented alongside its criteria in the corporate plan. The only exceptions were where a target was explicit in the criterion itself.⁴¹ Finance guidance indicates that a corporate plan should include a description of performance measures, when they will be reported on, the data collection techniques to be used and any targets the performance measures will be assessed against. Without an accompanying target, a reader is unable to determine what is being measured, or what success looks like, affecting the reliability of the measure.

2.71 For example when considering the criterion: ‘The anticipated annual exports from investment outcomes facilitated’, a reader doesn’t know whether it’s the number, type, or the value of exports being measured. A reader also cannot determine the standard that is expected in order for the criteria to be achieved, leading to the potential for bias. This is demonstrated by comparing the results presented in Austrade’s 2015–16 and 2016–17 Performance Statements. In both years, Austrade reported ‘Criterion met’, even though there was a $68 million decline in anticipated exports between the two years.⁴² Austrade included additional information in its performance statements outlining the reasons for the fluctuation between years. Providing this information, including a target, in the corporate plan would assist a reader in determining the reliability of the measure through the primary planning document.

2.72 Austrade advised that its interpretation of Finance guidance was that there was no requirement for targets to be presented in the corporate plan, and that forecasts of performance against the performance criteria were instead presented in the 2016–17 PBS, in accordance with the Finance Secretary’s Direction issued on 24 February 2016. These forecasts were then published in Austrade’s 2016–17 Performance Statements as targets alongside the performance criteria. As noted in paragraph 2.9, section 16E of the PGPA Rule requires an entity’s corporate plan to include ‘any measures, targets and assessments that will be used to measure and assess the entity’s performance for the purpose of preparing the entity’s annual performance statements under section 16F’. This would mean, as a matter of course, that any target that has been, or is intended to be, published by an entity in its performance statements should appear in its corporate plan.

2.73 For 2017–18, Austrade presented four performance criteria, and accompanying targets, in its 2017–18 PBS, published in May 2017. This is in accordance with the Finance Secretary’s Direction issued on 3 March 2017, which notes that only one performance criterion per existing program is required to be published for the forthcoming budget year and forward estimates. This is compared to 12 criteria⁴³ presented without accompanying targets in Austrade’s 2017–18 Corporate Plan published in August 2017. As noted in paragraph 2.70 above, presenting the majority of performance criteria without an accompanying target in a Corporate Plan, limits the ability of the Parliament and the public to determine what is intended to be measured and what success looks like, and affects the reliability of Austrade’s performance criteria.

Completeness of the performance criteria in assessing the entities’ activities and purpose

2.74 The PGPA Act requires accountable authorities to govern their entity in way that promotes the proper use and management of public resources, which is defined by the Act to mean efficient, effective, economical and ethical, and to measure and assess the performance of the entity in achieving its purposes.⁴⁴ The Act also requires entities to provide meaningful information to the Parliament and the public to assist it in understanding how Commonwealth entities are performing, and how they are using the resources that have been entrusted to them.⁴⁵

2.75 This is fulfilled by entities meeting the requirements for corporate plans to include statements of how the entity will achieve its purposes, and how they will be measured and assessed, the results of which are reported in the entity’s performance statements. This would require the inclusion of relevant measures demonstrating the proper use of resources in achieving the entity’s purposes in the corporate plan.

2.76 Finance guidance outlines that the considerations that are critical to developing good performance information include using an understanding of an entity’s purposes to identify a set of measures that can demonstrate the extent to which those purposes and activities are being delivered effectively and efficiently. It is rare for a single measure to be able to adequately determine the effectiveness of an activity, and Finance guidance advises that good performance information will draw on multiple sources and the quality of performance information should be emphasised over quantity. The guidance recommends a small set of measures that is sufficiently comprehensive to cover those factors that affect an activity’s performance.

2.77 The guidance also notes that in some cases, effectiveness may not be measurable, due to cost or a lack of complete information. In these cases, other measures may be used as proxies for effectiveness. Entities are advised that in these circumstances, they should be clear on why effectiveness cannot be measured and how the proxy measures are suitable. Finance notes that activities often work on different timeframes, and information on one or more timeframes may not be available at the time of reporting. It is therefore recommended that the performance story told at a particular time reflect the outcome that can be reasonably expected from the relevant activities at that time.

2.78 In assessing the selected entities’ performance criteria for completeness, the ANAO considered whether the performance criteria present a basis for a collective and balanced assessment of the entity against its purpose. In particular, the ANAO considered whether the selected entities’ performance criteria:

• collectively address the entity’s purpose through the activities identified in the corporate plan (collective);
• provide a basis for assessment of the effectiveness and efficiency of the entity in fulfilling its purpose either directly or through the use of proxies (balanced);
• relied on a mixture of quantitative and qualitative data (balanced); and
• assess a mixture of short, medium and long-term objectives (balanced).⁴⁶

2.79 Across the four entities an area for improvement was increasing the use of effectiveness measures, or where appropriate making clear the intention to use input, activity and/or output measures as proxies.

2.80 Employment was the only entity to include a measure, that when considered over time, may be used to demonstrate its efficiency in achieving its purpose. As noted above, entities should identify a set of measures that demonstrate both effectiveness and efficiency. This is to support the Parliament and the public in their assessment of how well Commonwealth entities are performing, including how they are using the resources that have been entrusted to them.

2.89 In addition, for some of the entities, there was an imbalance between quantitative and qualitative measures. Entities are also not realising the full potential of the minimum four year horizon of the corporate plan by developing performance criteria that assess a mixture of short, medium and long-term objectives. Commonly, where targets were set by an entity, they were static across the four years of the corporate plan, and the analysis of results did not make clear the medium or longer term objective to be delivered.

Employment

2.90 Overall, Employment’s performance criteria were mostly complete, with improvements required to the balance of quantitative and qualitative performance criteria. While all of Employment’s performance measures could be related to an activity presented in the corporate plan, there were some that only partly addressed an activity, while other activities were not considered to be addressed by any of the performance criteria. Five of Employment’s measures appear to be designed to measure the effectiveness (delivery of impact against the purpose) of Employment’s activities. There is also a measure evaluating the cost of delivering an employment outcome, which when considered over time may provide the basis for measuring Employment’s efficiency. The remaining nine measures consider outputs.

2.91 Only two of Employment’s measures assessed report qualitative information, while the remainder focus on quantitative results. Improving the balance of quantitative and qualitative performance criteria would assist in providing readers with a more comprehensive understanding of the impacts of Employment’s activities, and its progress towards the overall purpose. Each of Employment’s performance criteria are reported against annually (short-term), and static targets have been set over the four years of the corporate plan.

Environment

2.92 Overall, it is unclear whether Environment’s performance measures provide a complete basis to assess the department’s progress in achieving its purpose. While the ANAO was able to map most of Environment’s delivery strategies to an accompanying performance criterion, it was unclear whether those presented a complete picture of performance.

2.93 Environment also did not clearly indicate how each performance criterion and target would be assessed to determine the intended measurement basis or type. As a result, the ANAO was unable to determine the basis for measurement and/or timeframe for assessment for more than half of the performance criteria assessed.⁴⁷ Of those that had sufficient information, only eight appeared designed to determine the effectiveness (delivery of impact against the purpose) of the department’s activities.

2.94 The remaining measures consider activities and outputs, and the ANAO noted that Environment did not present any efficiency measures under the ‘Environment and Heritage’ purpose. Environment has not signalled the intention for any indicators in its corporate plan or performance statements to be considered as proxies, in accordance with Finance guidance. Environment should consider whether the use of output and activity measures as proxies is deliberate and if so, explain this approach to assist users in understanding how future performance reporting will adequately address the purpose.

2.95 While there appeared to be a balance of quantitative and qualitative information in the performance criteria assessed, this would need to be reconsidered in light of those that could not be determined. Environment did not present performance criteria with distinct timeframes for achievement, and targets were mostly presented under the headings of ‘2016–17 and beyond’, and one under ‘2017–18 and beyond’. Environment should consider setting more distinct timeframes for its performance criteria and targets, to inform users of what information will be available in each reporting period, and to better demonstrate the intended progress against longer-term objectives.

Austrade

2.96 Austrade’s performance measures were mostly complete, however, incorporating efficiency measures in future reporting periods would improve their balance, as the current set do not provide the Parliament or the public with an understanding of Austrade’s efficiency in delivering its purpose. Austrade’s performance criteria aligned to the activities outlined in the corporate plan, providing a collective basis for assessing progress against the purpose. Ten of Austrade’s performance criteria measure effectiveness, while the remaining criteria measure outputs or activities. Austrade was also the only entity to clearly signal the intention for an output measure to be considered as a proxy for effectiveness.

2.97 Austrade’s group of performance criteria provide a balance of qualitative and quantitative information. As first noted in paragraph 2.10, no targets have been set by Austrade in its corporate plan against the performance criteria in the current year, or in the future periods covered by the corporate plan. As a result, all performance criteria were assessed as only addressing the short-term timeframe. Austrade would benefit from reviewing how to demonstrate to readers the expected timeframes of activities through the performance criteria by setting and publishing targets in the corporate plan, and also expected incremental progress over the forward years.

Sports Commission

2.98 None of the Sports Commission’s performance measures address effectiveness or efficiency. The observations noted in paragraph 2.39, regarding the Sports Commission’s output-focused purpose statement, may have contributed to the Sport Commission’s difficulty in identifying effectiveness measures. Only two of the performance criteria presented qualitative results. The Sports Commission would benefit from addressing this imbalance to provide readers with a more complete understanding the Sports Commission’s progress towards its overall purpose.

2.99 Each of the Sports Commission’s performance criteria have a static target established for the four years of the corporate plan, with the exception of ‘The AIS is a world leader in sports insights, services and facilities’ which only sets a target for 2016–17. As noted for the other entities, setting more specific timeframes for achievement provides the opportunity for an entity to demonstrate progress against short, medium and longer-term objectives.

Have entities made improvements to their proposed performance measurement and reporting for 2017–18?

2.100 The timing of this audit is intended to inform entities’ development of 2018–19 Corporate Plans and also improvements to the 2017–18 Performance Statements. To achieve this aim, the ANAO also reviewed the selected entities’ 2017–18 Corporate Plans, released August 2017, to determine whether improvements had already been made to proposed performance reporting.

Employment

2.101 As noted in paragraph 2.36, the ANAO observed the purpose presented in Employment’s 2016–17 Corporate Plan could have been more concise to better reflect the core elements of its purpose. In the 2016–17 Performance Statements, and 2017–18 Corporate Plan, Employment removed some of the additional language used to reinforce the connection between its purpose and PBS outcomes, which has made the purpose more succinct. The department’s outcomes, objectives and strategies are consistent with those presented in the 2016–17 Corporate Plan. No changes have been made to the performance measures or associated targets in the 2017–18 Corporate Plan.

2.102 During the course of the audit, the department advised that it had completed a review of the appropriateness of performance criteria presented in its corporate plan and PBS. This formed part of a broader review of the department’s business performance framework, approved by the department’s Executive group on 5 December 2016. The review, completed in June 2017, included recommendations focused on improving the department’s purpose statement, and the relevance and reliability of the department’s performance criteria. Employment advised that it intends to consider improvements to its business planning performance framework, including how it assesses the appropriateness of performance criteria during the 2018–19 performance reporting cycle.

Environment

2.103 In comparing the 2016–17 and 2017–18 Corporate Plans the ANAO noted limited changes to the presentation and design of Environment’s purpose, activities and performance criteria. Environment advised the ANAO that the department has commenced trialling a program logic model to inform divisional business planning and develop more meaningful measures in future reporting periods.

2.104 In July 2017, the department also established a Knowledge and Technology Division, to ‘focus on improving the Department’s information management, investment in research (creating and collecting data) and technology (providing access to data)’. The Division plans to support the delivery of the department’s priorities, including through:

• management, governance and oversight of information technology, data and records;
• expert advice on technology, data, science and environmental-economic accounts; and
• engagement with various stakeholders, including service providers, academia and line areas across the department.

Austrade

2.107 Austrade’s 2017–18 role, purposes, and activities are consistent with those presented in the 2016–17 Corporate Plan. Austrade has reduced the total number of performance criteria presented by consolidating eight measures into two, and removing four entirely. Minor wording changes have also been made to some performance criteria, and one new measure has been included. Overall the changes provide a smaller group of performance criteria, however, the key issues affecting the relevance, reliability and completeness of the criteria referred to in earlier sections of this report continue to apply.

Sports Commission

2.108 The Sports Commission’s 2017–18 Corporate Plan reflects some changes when compared to the 2016–17 Corporate Plan. The management level ‘Organisational Excellence’ strategic pillar and accompanying performance criteria have been removed, and the remaining pillars have been renamed but remain thematically consistent with those presented in the 2016–17 Corporate Plan. The total number of performance criteria has also been reduced from 16 to five, which has lowered the amount of management level information presented, however the completeness of the performance criteria remains an area for improvement. Positively, the ANAO also noted that the Sports Commission amended the wording for the following performance criterion presented in the 2016–17 Corporate Plan ‘Number of children participating in the Sporting Schools program’ to better reflect the results intended to be reported as ‘Number of student attendances in the Sporting Schools program’ in the 2017–18 Corporate Plan.

2.109 The Sports Commission noted that it had made limited changes to its 2017–18 Corporate Plan in anticipation of the National Sports Plan (NSP) and accompanying internal strategic review. The NSP was announced by the Minister for Sport on 22 May 2017, and one of its intentions is to inform policy objectives across the Australian sporting sector, of which the Sports Commission is a major participant. The Sports Commission has noted it will consider its performance measurement and reporting more broadly in light of changes to its strategic direction and framework while the NSP is being developed.

Did the entities’ processes inform the coordination and collation of the performance statements?

3.1 Corporate planning and performance reporting should be viewed by entities as a continuous process, rather than annual and distinct processes. Finance guidance notes that entities should design processes that support the complete cycle of planning, performance measurement and reporting. This would include the preparation and publication of the PBS, corporate plans, performance statements and annual reports, evaluating lessons learnt and incorporating these into the following cycle to contribute to continuous improvement.⁴⁸

3.2 In reviewing the processes implemented by the entities in developing their performance statements, the ANAO considered whether they reflected a whole-of-cycle approach to performance measurement and reporting. The ANAO also considered whether those processes were supported by clearly defined roles and responsibilities and adequate structures to facilitate management engagement and support.

3.3 The ANAO noted that each of the selected entities had engaged with the Department of Finance to seek feedback and/or guidance during 2016–17. This was through direct contact with representatives of the department, and/or through participation in Finance’s corporate planning and performance statements Communities of Practice.

Employment

3.4 Employment’s processes for the coordination and collation of its performance statements were clearly structured, including well-defined roles and responsibilities. The ANAO also observed effective management engagement through the monitoring and endorsement of the department’s performance measurement and reporting across the reporting period.

3.5 The 2016–17 corporate planning and performance statements process was centrally coordinated by a dedicated section and overseen by an Executive committee. The committee is co-chaired by Deputy Secretaries and comprises representatives at senior levels across the department, meets on a quarterly basis, and receives updates on progress against the department’s performance measurement and reporting requirements.

3.6 A paper containing the proposed approach to the development and publication of the department’s annual report, including the performance statements, was presented to the Executive committee and endorsed on 24 May 2017. The paper included a timeline with key milestones, and was informed by an evaluation of the 2015–16 Annual Report development process, completed in November 2016. A project plan was also developed which highlighted items such as timing constraints, related projects such as the PBS and corporate plan, milestones and responsible parties, roles and relevant stakeholders.

3.7 A formal structure for the submission and clearance of information for the annual report, and performance statements, was evident and also documented in the project plan. Coordinators were established at each level of the Employment cluster to coordinate the collection, collation and clearance of information that would support the annual report⁴⁹, including the performance statements. The Employment cluster determined the underlying processes supporting this activity, and received feedback on their submissions at each stage of this coordination process from the central coordination section.

3.8 The Employment cluster’s key decision-making forum is the Employment Steering Committee (ESC). The ESC is chaired by the Deputy Secretary Employment and consists of the group managers within the cluster. The ESC meets weekly to discuss the jobactive program policy, risk, delivery, performance, contract management and evaluation. One of the four ESC meetings each month is replaced with a Joint ESC and includes all Senior Executive Service branch managers and state managers.

3.9 Reports detailing progressive performance against the criteria set out in the PBS and corporate plan are provided to the ESC (or Joint ESC where appropriate) on a quarterly basis⁵⁰ and the Deputy Secretary receives a monthly update. The ANAO also observed that individual branches and groups also prepare more regular updates, that address ongoing performance monitoring at a more detailed level, for the Branch and Group Managers. This reporting is then used by those groups to inform the regular reporting to the ESC, which is coordinated by the Employment Services Reporting and Analysis branch within the Quality and Integrity Group.

Environment

3.10 The key elements of Environment’s processes to support the development, coordination and collation of the department’s performance measurement and reporting, are reproduced in Figure 3.1. Referred to as the activity performance planning, evaluation and reporting framework, it was designed to achieve the aim of continuous improvement.

Figure 3.1: Environment activity performance planning and reporting framework

 

 

Source: Reproduced from documentation provided by the Department of the Environment and Energy.

3.11 Activity performance plans are intended to be the primary performance planning document for Environment’s policies, programs and regulations. These plans then informed the performance criteria set out in the PBS and corporate plan. Underlying divisional, branch and section plans were then developed on this basis.

3.12 The biannual performance report outlines results against the performance measures set out in the PBS and corporate plan, and is used as a basis for preparing the department’s performance statements. It was designed to assist divisions in:

• preparing early;
• identifying measures that were at risk of not being met by 30 June;
• identifying improvements to the measures proposed to be included in the next activity performance plans, PBS and corporate plan; and
• improving the quality of results intended to be reported in the current year performance statements.

3.13 Environment’s 2016–17 corporate planning and performance statements process was centrally coordinated by a section within the department, and overseen by the Governance and Performance Committee. The committee comprised representatives at senior levels across the department, met five times across the reporting period and considered updates on the department’s progress against its performance measurement and reporting requirements, including the corporate plan, biannual performance report, and performance statements.

3.14 The ANAO did not identify any significant changes between the February 2017 biannual performance report and the 2016–17 Annual Performance Statements for the ‘Environment and Heritage’ purpose, and noted only minor changes to the 2017–18 Corporate Plan. While Divisions were responsible for developing measures and establishing suitable processes to support the collection and monitoring of results across the continuous performance reporting cycle described above, Environment’s central coordination unit developed and provided policy, guidance and templates to assist divisions. However, there was limited evidence of advice and feedback provided by the central coordination unit being considered or adopted by divisions. The ANAO noted that in some cases, if the advice and feedback had been actioned, issues outlined in this report would have been resolved, such as improving the meaningfulness of results presented in the performance statements.

Austrade

3.15 Austrade’s organisational structure during the 2016–17 performance reporting cycle reflected the groups represented by its purposes and presented in the corporate plan.⁵¹ The corporate planning and reporting section is responsible for the central monitoring and collation of performance information to be included in Austrade’s corporate plan and performance statements. While the central unit provided guidance and advice, each group was responsible for the development of its performance measures.

3.16 Management engagement was facilitated through an Executive group, comprising senior executives across the organisation. The Executive Group meets weekly and assists Austrade’s Chief Executive Officer by advising on organisational strategy, corporate governance, cross-organisational policy, management and accountability. The Executive Group monitors the status of key projects and considers performance reports against business plans, budget and key performance measures.

3.17 In March 2017, Austrade adapted existing quarterly management reports to a ‘traffic-light’ style of reporting against the consolidated performance criteria set out in the corporate plan. The traffic light reports set out the status, assurance strategy, area and executive member responsible for data, the data source and how the data will demonstrate performance for each performance measure. The report is compiled by the central coordination unit through liaising with each group. While the consolidated monitoring was implemented mid-way through the reporting cycle, it provides a foundation to identify ongoing improvements to Austrade’s performance measurement and reporting.

Sports Commission

3.18 Responsibility for the coordination and collation of the Sports Commissions’ performance measurement and reporting was held centrally. The level of the Sports Commission’s executive consideration and monitoring was commensurate with its size. Formal monitoring of the performance measures included in the corporate plan was conducted on a bi-monthly basis. Organisational performance ‘dashboard’ reports were presented to the Board in October and December 2016, and February, April and June 2017. These dashboards provided a progressive update of performance to date against those measures outlined in the corporate plan and intended to be included in the performance statements. These reports were compiled by the Governance section through coordinating with those sections with responsibility for the measures.

3.19 A similar approach was used for the year-end reporting to support the performance statements. The Governance section also referred to an Annual Report Procedure document which outlines the key processes, stakeholders and inputs required to facilitate the collation and publication of the annual report, including the performance statements. The coordination process largely occurred via email between the Governance section and other areas of the organisation and also included providing feedback on areas such as the performance criteria rationale (why, and how) and conducting ‘sense checks’ over the reported results.

Were systems and methodologies established to collect and report performance information?

3.20 In designing appropriate performance criteria, an important consideration for entities is the information that the entity will need to collect to report against those measures. Resource Management Guide No. 131: Developing good performance information, issued by the Department of Finance, sets out that entities should consider the following for information used to report performance:

• availability—is information readily accessible or feasible to access;
• suitability—is the information accurate, auditable and reliable;
• timeliness—the information be collected on a timescale relevant to the measure; and
• cost—the effort required to collect and analyse information should be commensurate with the benefit it provides.⁵²

3.21 One focus of this audit was whether the selected entities considered the above factors and established systems and methodologies to support the collection of performance information that could be relied upon as a complete and accurate reflection of the results. In assessing the approaches adopted by the selected entities, the ANAO considered the following:

• the method and frequency used to collect the data for each criterion and whether it was suitable;
• the known limitations of the data;
• how data is recorded and stored and what controls exist to ensure it is not inadvertently or purposely manipulated;
• how data is extracted from systems and what processes are applied to reach the reported result; and
• whether adequate documentation of policies and procedures to support consistent data outcomes exist.

3.22 The ANAO observed a number of different methods and approaches to identifying and collecting suitable information to support performance measurement and reporting. Entities would benefit from incorporating the identification of suitable information sources, and an understanding of how information will be collected, into the development of performance measures. These should also be clearly documented in policies and procedures to promote the consistency and accuracy of future reporting.

Data sources and collection methods

Employment

3.23 Of the selected entities, Employment was the most reliant on IT systems and tools to collect, collate, and extract information to report results. Given this reliance on systems data, the ANAO reviewed the effectiveness of key controls established by Employment to ensure data integrity.

3.24 The use of IT systems or applications, when not adequately controlled or monitored, can increase the risks posed to the accuracy and completeness of information being relied upon to produce reports. The existence of effective controls mitigates these risks. The ANAO focused on testing the following key controls:

• user access restriction to systems and any data warehouse systems;
• data input validation and quality assurance;
• monitoring and reporting of changes and/or deletion of records; and
• extraction and transformation of data.

3.25 Employment’s performance criteria for ‘Outcome 1’ rely on a combination of information sourced from a number of key management information systems.⁵³ The largest system is the Employment Business Intelligence Warehouse (EBIW). The EBIW provides a central environment for collating data to support Employment’s business functions and performance reporting from a number of other management information systems. These source systems are established to provide the validation required during data entry and business processing to ensure the completeness and accuracy of data subsequently loaded into the EBIW. Once loaded into the EBIW it is then able to be extracted and used by Employment to support further analysis and reporting across a number of business areas.

3.26 The complexity of the data collection and transformation applied following extraction from Employment’s management systems, including the EBIW, typically influence the tools used to ensure the completeness, accuracy and validity of data. The main tools used by Employment were the Statistical Analysis System (SAS) and MS Excel spreadsheets. Generally, SAS was used to perform complex analysis over larger datasets, and was subject to effective department wide controls, providing greater assurance of the integrity of data used for reporting results in the performance statements.

3.27 Spreadsheets were more commonly used for activities that were not complex in nature over smaller datasets; however, the ANAO did note examples of spreadsheets being used to support more complex data validation and transformations and were referred to as the source for performance reporting. The ANAO has previously noted the risks arising from using spreadsheets as a primary form of evidence. Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. This can make spreadsheets unreliable corporate data handling tools, as accidental or deliberate changes can be made to formulae and data, without there being a record of when, by who, and what change was made.⁵⁴

3.28 Overall, the ANAO’s assessment of the key controls within Employment’s systems supporting the performance criteria for ‘Outcome 1’ did not identify any significant control weaknesses that would affect the integrity of data extracted from those systems. However, the use of spreadsheets to support data validation and transformation following its extraction from some controlled systems increases the risk of inaccurate results being presented in Employment’s performance statements. Employment may consider expanding the use of its existing systems and technologies which are able to be effectively controlled and monitored, such as the EBIW, to limit the use of spreadsheets as a primary form of evidence in the future.

Environment

3.29 Environment’s approach to identifying data sources and collection methods, and the level of rigour applied, varied depending on the program area and the performance criteria or targets being assessed. In September 2017, the department prepared a summary of the data collection arrangements that had been used for the 2016–17 Performance Statements. Divisions were asked to identify the evidence base for each performance measure and provide any comments about the data source; frequency of measurement; any guidelines, benchmarks, or standards applied; quality assurance; and any limitations of the evidence base or assurance processes. The ANAO’s review of the responses identified that, for the majority of measures, the summary did not provide sufficient detail of the original data source or collection arrangements, but rather focused on the process for information to be reviewed after it had been collated.

3.30 A variety of data sources were used, ranging from reports submitted by grant recipients, through the department’s Monitoring, Evaluation and Reporting Improvement Tool (MERIT), to scientific monitoring conducted by the department itself. A number of results were also based on reporting by external parties, such as state governments. In its advice to the audit committee, the department reported that, for several of the measures, reporting was cross-referenced or reconciled with internal data. However, the department advised the ANAO in one instance that it generally relied on the accuracy of information provided by external reporting parties, or other external reporting mechanisms.

3.31 The department would benefit from reviewing existing policies and procedures for identifying, collecting and using suitable data for performance measurement and reporting. This would include confirming the understanding and consistent application of those policies and procedures by divisions. Establishing divisional procedures for identifying suitable data sources and collection methods at the activity planning stage may also assist information being available at the time of reporting.

3.32 For example, for two performance criterion, the reported result was ‘Data/information not available or incomplete’. An effective process to identify and document the methodologies to support the performance criteria would have identified this matter at an earlier stage. The measures could then have been reviewed to determine whether they should be amended or alternative information sources identified.

Austrade

3.33 Austrade relied on a small number of key information sources to support its performance statements: surveys; research and analysis completed by Tourism Research Australia⁵⁵; and a management information system. Overall, Austrade’s data sources and collection and reporting methods were well-managed.

3.34 Surveys were designed and completed by appropriately qualified personnel, including through engagement of a professional survey and statistics provider, with appropriate consideration of mitigation strategies for issues such as response bias and survey fatigue. Those that were published externally by Tourism Research Australia were also accompanied by methodologies, including sample sizes and confidence intervals. The key information system was supported by detailed processes and policy manuals to promote consistency and accuracy of the information recorded and extracted from the system.

Sports Commission

3.35 The Sports Commission used a range of information sources to support is performance reporting. Some were publically available through external bodies such as the International Olympic Committee’s medal tally, or published sporting body world championships results. Others were reported on the basis of internal reviews, or in some circumstances extracts from the Sports Commission’s financial management information system. Another key information source was the AusPlay survey. The survey is conducted by an external provider, and similarly to Austrade, the Sports Commission publishes the survey methodology and results through its website.

3.36 The initial development of the Sports Commission’s performance measures, and identifying and collecting suitable data, was assisted by adapting templates provided by the portfolio lead entity—the Department of Health. These templates provided a basis for the Sports Commission to identify, for each measure intended to be included in the PBS and corporate plan, the PBS program it related to, whether it was on an ongoing or time limited measure, quantitative or qualitative, and how frequently it would be measured. The template also set out the methodology and the data sources intended to be used to measure performance. Further discussion regarding the suitability of the methodologies supporting the measures can be found in the following sections.

Measurement methodologies

3.37 The development and documentation of measurement methodologies is important to support the collection of performance information that can be relied upon as a complete and accurate reflection of the results reported in the performance statements. In reviewing the selected entities’ methodologies supporting their performance criteria, the ANAO identified examples where the measurement methodology was not appropriate, or would have benefited from review, to provide the most appropriate method to reflect the measure as described.

3.38 In those cases where the methodology used by an entity does not reflect the measure as described, or support the result, the entity should consider:

• reviewing performance criteria so the result intended to be reported is clear to a reader; or
• establishing a methodology that leads to measurement of the performance criteria as described; and
• clearly documenting the methodology to support consistent and accurate reporting in future periods.

Employment

3.39 The level of procedural documentation and guidance of the methodologies supporting Employment’s performance criteria was variable. While it was clear from discussions with operational staff that there was a detailed understanding of the methodology that supported the reported result, these were not always formally documented. Given the department’s reliance on system-generated information and the requirement, in some cases, for these extracts to be analysed further through manual intervention, Employment should ensure it continues current efforts to document all methodologies and processes supporting external performance reporting. Operational guidance will promote consistency and accuracy of future reporting.

3.40 The methodologies supporting two of Employment’s performance criteria would also benefit from urgent review. These are summarised below:

• ‘Proportion of job seekers with mutual obligation requirements who are actively looking for work’—the wording of the criterion leads a reader to reasonably expect the result is reflective of the proportion of job seekers who are meeting their obligation to look for work. For example, submitting job applications, or attending an interview. However, the result reflects the proportion of job seekers that have a documented requirement to look for work, which is not necessarily the same; and
• ‘Proportion of jobactive organisations that meet the service delivery requirements’—the wording of the criterion leads a reader to reasonably expect the result is reflective of the proportion of jobactive providers complying with all of the their service delivery requirements. Rather, the result is the proportion of providers that met, or partially met, at least 80 per cent of requirements that were assessed during the period. Which requirements are assessed during a reporting period is at the discretion of a contract manager, and not all requirements are necessarily assessed in the 12 month reporting period. The same observation was made in ANAO Report No.4 2017–18 jobactive: Design and Monitoring, tabled in July 2017.⁵⁶

3.41 The ANAO also noted that the Survey of Employer Recruitment Experiences, which is used to determine the result for ‘Proportion of employers satisfied with the assistance provided by a jobactive organisation’ is designed to capture broader workplace observations, rather than specifically jobactive. In addition, the department advised that:

Many organisations deliver both jobactive and other employment services programs. It is not practical to ask employers to identify which service was utilised, so a decision was made to include all Government employment services as a close proxy for employer satisfaction with jobactive.

3.42 Of the 13 945 workplaces that were surveyed, 757 respondents were recorded as having used an employment services provider in the previous 12 months. Of these, 70 per cent were able to name at least one valid jobactive provider, or confirmed they had received employment services from a jobactive provider.⁵⁷ Of the remaining responses, approximately 20 per cent were unable to confirm the employment services they received were from a jobactive provider.⁵⁸ In these cases, the department advised that because the respondent subsequently confirmed that the service they received was free, it was recorded as an employment services provider and included in the total. The remaining 10 per cent of responses (74) related to other employment services providers such as Disability Employment Services or Transition to Work.

3.43 The department should make the assumptions underpinning the analysis of the Survey of Employer Recruitment Experiences clear in the corporate plan and performance statements. This would include explaining why the department is confident the result is representative of employer satisfaction with jobactive providers.

Environment

3.44 Similar to the observations made at paragraph 3.29 regarding Environment’s documentation of data sources and collection methods, the level of documentation for the methodology that would be used to determine and support the reported results in the performance statements was varied, and in most circumstances limited. This prevented the ANAO from making a complete assessment of the suitability of the information being used to support the reported results.

Sports Commission

3.45 The methodologies supporting seven of the Sports Commission’s performance criteria did not reflect the performance criteria as described. For example, the criterion ‘Number of children participating in the Sporting Schools program’ leads a reader to reasonably expect the result will reflect the number of children that took part in the program during the reporting period. However, the methodology to support the result captures the number of program attendances, leading to children being counted multiple times.

Austrade

3.46 The methodologies supporting Austrade’s performance criteria were largely suitable, with two assessed as requiring further consideration. The results presented against ‘Effective contribution to whole-of-government policy development using Austrade’s commercial perspectives on export, investment, international education and tourism’ describe a broad range of activities undertaken by Austrade across the reporting period. While an informative description of activities is provided, the absence of accompanying analysis that demonstrates why or how they represent an ‘effective contribution’ affects the ability of a reader to assess Austrade’s performance. Austrade would benefit from reviewing this approach in future reporting periods to ensure that the results reflect the measure as described.

3.47 In reviewing the methodology that supports the measure ‘Urgent passport issues are dealt with in a timely and responsive manner’, the ANAO noted that ‘timely and responsive’ was not defined, and no assessment of timeliness was made for urgent passport issues, leading to a 100 per cent result. In discussion with Austrade representatives, this was because there is no benchmark set within the memorandum of understanding with the Department of Foreign Affairs and Trade. Nevertheless, to provide a meaningful assessment of Austrade’s performance against this measure, there would be value in defining, and documenting, a timeframe that is intended to demonstrate success.

Were processes established to obtain assurance over the annual performance statements?

3.48 Section 16F of the PGPA Rule requires an accountable authority to certify that their performance statements accurately present the entity’s performance for the reporting period. An equivalent certification by the accountable authority is required by section 42 of the PGPA Act for the annual financial statements. It is important that an accountable authority has confidence that the entity has adequate processes in place to provide sufficient assurance to support these certifications.

3.49 The requirement for entities to prepare and publish their financial statements, including mandatory annual audits by the ANAO, has been in place for a number of decades. As a result, the financial statements preparation processes of Commonwealth entities are mature, and supported by established assurance frameworks. Entities may leverage from these frameworks and implement similar processes to provide an accountable authority with sufficient assurance to certify the performance statements. Relevant processes may include:

• documented procedures and processes supporting the performance reporting cycle;
• management reporting of progressive results to internal stakeholders such as the accountable authority and audit committee;
• effective direction of internal audit resources;
• management certifications supported by suitable quality assurance of results; and
• detailed audit committee review and endorsement.

3.50 Earlier sections of this chapter considered the entities’ processes and procedures, including progressive management reporting and monitoring to support the performance statements. The following sections focus on the entities’ use of internal audit, management certifications and audit committee involvement and endorsement, as assurance mechanisms for their accountable authority’s certification.

Internal audit

3.51 It is important for an entity to consider how their internal audit function might be utilised to support the accountable authority, and audit committee, in improving an entity’s performance measurement and reporting framework. This contributes to the confidence of an accountable authority in the appropriateness, and completeness and accuracy, of the entity’s performance reporting.

3.52 The ANAO noted that two of the four selected entities, Environment and Austrade, had undertaken an internal audit with specific consideration of elements of the entities’ performance measurement and reporting frameworks.⁵⁹ Environment’s audit considered the effectiveness of the department’s broader performance reporting framework, including the role of evaluation, while Austrade’s was designed to evaluate the processes supporting the performance reporting of a particular division within the entity. Both provide examples of how internal audit may be used to assess an entity’s performance measurement and reporting.

Environment

3.53 In October 2016 Environment’s audit committee endorsed the Internal Audit Work Plan, which included the audit of the department’s performance reporting. The objective of the audit was to assess whether the department was meeting the non-financial performance information reporting requirements under the PGPA Act and associated guidance, and to assess the role of evaluation in performance reporting. Environment presented the report to the committee in May 2017.

3.54 The audit focussed on the 2015–16 process and made a number of observations and recommendations consistent with those in this report, including:

• Clarify whether the Annual Performance Statements are meeting stakeholder needs;
• Update the department’s Evaluation Policy with clear definitions of performance reporting roles and responsibilities, requirements for divisional procedures, methodologies and record keeping and quality assurance; and
• Determine and document divisional procedures and methodologies for defining performance criteria, performance reporting and quality assurance.

3.55 As part of its response to these recommendations, management undertook to update the Evaluation Policy by March 2018, conduct a progressive, systematic review of its performance criteria and targets, and seek an IT solution to facilitate automated and integrated reporting in real-time.

Austrade

3.56 Austrade commenced an internal audit of its Investment Performance Management Framework during August 2017, as part of its 2017–18 Audit Work Plan. The internal audit was intended to ‘Assess the processes Austrade has designed and implemented to plan, monitor and report against the Investment KPIs outlined in its Portfolio Budget Statements and Corporate Plan’. This included reviewing a sample of records supporting the investment outcome results reported in Austrade’s 2016–17 Performance Statements.

3.57 The audit identified, prior to the finalisation of the performance statements, instances where there were insufficient records to support the reported outcomes. A review was successfully completed to re-validate the information in the system and support the reported results. The observations of the report, and the subsequent re-validation of records, provided additional information to the Audit and Risk Committee as to the completeness and accuracy of results presented against those performance criteria in the 2016–17 Performance Statements.

Management certifications

3.58 Written certifications provide a formalised model for the communication of assurance between an entity’s management and its accountable authority. However, the value of certifications are influenced by the robustness of the processes underpinning them, their completeness, and also the attention and/or action taken in resolving matters identified by those certifications.

3.59 Employment and Austrade adopted multi-level management certifications as part of assurance processes for the 2016–17 Performance Statements. Both entities’ assurance processes were considered to be effective, as supported by observations made in other sections of this report. The following section focuses on observations made in regard to Environment and the Sports Commission’s certification processes.

Environment

3.60 Environment’s management assurance model requires improvement, as evidenced by the matters raised in this report. Environment’s management certifications were provided by the First Assistant Secretaries (FASs) of each division, via standardised forms. Each form contained a checklist of key requirements relating to ‘performance assessment’, ‘performance measurement’ and ‘record keeping’. The form also included a certification statement for completion and signing. The form required FASs to disclose in this statement ‘relevant qualifications if required’, and was pre-populated with the following assertions:

• The results and analysis presented in the annual performance statements relate to the performance against the performance measures outlined in the Department of Environment and Energy Corporate Plan 2016-17 and in the Department’s Portfolio Budget Statements 2016-17; and
• In my opinion, and following a process to gain assurance, this component of the performance statements accurately presents the Department’s performance in the year ending 30 June 2017, and meets the requirements under sub-sections 37(1), 38(1), 39(1)(a) of the Public Governance, Performance and Accountability Act 2013.

3.61 Of the certification statements relevant to the performance criteria considered by this audit, none detailed qualifications or caveats in regard to the performance statements. However, the ANAO did note a checklist accompanying one certification included comments highlighting potential issues affecting the quality of data and data collection methods, including the implication that validation of data sources was not performed. These potential issues were not reflected in the ensuing certification statement, where no qualifications were noted.

3.62 In light of the matters raised in this report in regard to the quality of Environment’s performance measurement and reporting, the absence of further caveats, or consideration of those raised above, indicates a potential weakness in Environment’s certification processes. Environment may consider whether there is sufficient understanding among its management representatives of the Commonwealth performance framework to provide the level of assurance expected from the management certification processes.

Sports Commission

3.63 As a smaller organisation, the Sports Commission had a lesser number of key personnel that contributed to the performance statements and, as a result, did not use formalised certifications as part of its 2016–17 Performance Statements assurance processes. The performance statements were still subject to review, and there was an expectation that submissions forming part of the central coordination had been cleared by an appropriate level prior to approval by the Chief Executive Officer and the Board. However, these reviews and approvals were not consistently documented. There was limited evidence of advice being provided to the Board outlining what assurance mechanisms were in place to support the performance statements, except to note that ‘Annual report content had been sourced from the ASC Executive and relevant divisions staff’ and ‘developed in accordance with requirements released under the PGPA Act and associated PGPA Rule 2014’.

3.64 In smaller organisations, which tailor their processes to suit the entity’s circumstances, sufficient documentary evidence should be kept. This may include more detailed minutes of relevant discussions held at board or executive meetings, particularly in regard to how management is assured of the quality, and completeness and accuracy of the performance statements.

The role and function of audit committees and entity performance reporting

3.65 The accountable authority of a Commonwealth entity is required to ensure that the entity has an audit committee⁶⁰ and to determine by written charter the functions of this committee.⁶¹ The minimum functions of an audit committee are set out in the PGPA Rule and include reviewing the appropriateness of the accountable authority’s performance reporting (refer below, emphasis added).

3.66 The Department of Finance issued Functions of audit committees: Reviewing the appropriateness of performance reporting, including for 2016-17 annual performance statements in July 2017 ‘in response to a number of recent queries in relation to the 2016-17 annual performance statements’. The note reiterated the role and functions of audit committees under the PGPA Act and Rule in regard to the performance reporting of entities⁶², specifically that:

The role of an audit committee is to provide independent advice and assurance to the entity’s accountable authority. It is a requirement of s.17(2) of the PGPA Rule that the functions of an audit committee must include reviewing the appropriateness of the accountable authority’s financial reporting, performance reporting, system of risk oversight and management, and system of internal control. The PGPA Rule requires that the accountable authority must, by written charter, determine the functions of the audit committee (Rule s17(1)). This charter must include the requirements of s17(2) of the PGPA Rule, including the requirement in s17(2)(b) that functions of the audit committee must include reviewing the appropriateness of the accountable authority’s performance reporting.

In the case of performance reporting, an audit committee would reach a view on the appropriateness, of the entity’s performance reporting as a whole, with reference to the Commonwealth’s performance reporting framework, (including the relevant requirements, directions or guidance for entities corporate plans and PBS).⁶³

3.67 The JCPAA noted in its most recent report that it ‘regards audit committees as providing Commonwealth entities with an invaluable source of independent assurance and advice’. and recommended that:

The Australian Government amend, as necessary, the Public Governance, Performance and Accountability Act 2013, and accompanying rules and guidance, to clarify that the functions and charter of Commonwealth entity audit committees need to reflect their role in assurance of the appropriateness of performance reporting, as well as specifying that some members must have skills in performance measurement and reporting.

3.68 The ANAO noted that at the time of finalising the 2016–17 performance statements, none of the selected entities’ audit committee charters reflected the language set out in section 17 of the PGPA Rule. Specifically, the requirement that audit committees ‘review the appropriateness’ of the accountable authority’s performance reporting. While in some cases, the charter used comparable terms, such as ‘sound’, audit committee charters should be reviewed regularly to ensure alignment with the PGPA Rule. Employment subsequently updated its audit committee charter in October 2017 to more closely reflect section 17 of the PGPA Rule, including the requirement for its audit committee to ‘reach a view on the appropriateness of [the department’s] performance reporting as a whole’.

3.73 Finance’s guidance also made clear the expected level of assurance an audit committee is expected to provide:

A reasonable level of assurance on the appropriateness of an entity’s performance reporting is consistent with our expectations of the assurance provided by audit committees, as required in s17(2) of the PGPA Rule.

However, those audit committees that are unable to provide a reasonable level of assurance (for example, limited assurance) for the 2016–17 performance reporting cycle, should discuss the level of assurance they are able to provide with their accountable authority. This would include a discussion of what the audit committee proposes to do to provide a reasonable level of assurance to the accountable authority in the 2017–18 performance reporting cycle, and would include documenting this approach in their charter or workplan.

3.74 The level of assurance an audit committee is able to provide to an entity is influenced by the expertise and experience of its members, and how that is utilised in evaluating the design and implementation of an entity’s performance reporting framework. Audit committees are also typically able to influence and/or direct the work of internal audit and, as evidenced by the examples provided from paragraph 3.52, this may provide a valuable resource in contributing to an audit committee forming its view on the appropriateness of an entity’s performance reporting.

3.75 Employment, Austrade and the Sports Commission’s audit committees each provided a statement of reasonable assurance in regard to the appropriateness of the entity’s performance statements. Each of the three entities adopted a different approach, but all, at a minimum, had:

• involvement in considering, querying and/or providing advice to management on the performance measures presented in the entities’ 2016–17 PBS and Corporate Plans;
• received interim reporting of results against those measures. Some committees received this information more regularly than others, such as quarterly versus biannually;
• received briefings on processes supporting the completeness and accuracy of results presented in the annual performance statements; and
• received assurance from management representatives on the completeness and accuracy of results presented in the annual performance statements.

3.76 Environment’s audit committee advised the accountable authority that the level of assurance that could be provided was limited ‘without extensive resource-intensive audit activity’ and that it ‘had not become aware of any matter that would cause it to believe that the annual performance statements…had not been prepared in accordance with the Department’s performance reporting framework, the PGPA Act, the PGPA Rule and relevant resource management guides’. The ANAO did not observe evidence of Environment’s audit committee documenting its intended approach at that time to increase the level of assurance provided in the next reporting period as required by Finance guidance.

3.77 Entities and audit committees should not confuse providing reasonable assurance with confirming that the entity’s performance reporting is entirely appropriate. An audit committee may have identified an issue that affects the appropriateness of the accountable authority’s performance reporting and, while still providing reasonable assurance, have drawn it to the attention of the accountable authority in its advice.

3.78 The observations made in other sections of this report regarding the appropriateness of the selected entities’ measures, systems, methodologies, and record keeping indicate that there is still some way to go in the maturity of audit committees’ assurance and advice in this area.

Did the entities’ performance statements accurately present their performance?

3.79 An accountable authority is required to state, under section 16F of the PGPA Rule, that its performance statements accurately present the entity’s performance in the reporting period. This is reinforced by section 37 of the PGPA Act that sets out the requirement for Commonwealth entities to keep records that properly record and explain the entity’s non-financial performance. The ANAO reviewed records made available by each of the selected entities to record and explain the selected performance criteria results and analysis presented in the 2016–17 Performance Statements to determine whether they were an accurate representation of performance.

3.80 The results and analysis presented in Austrade’s and Employment’s performance statements accurately presented their performance, as supported by suitable records. Environment’s and the Sports Commission’s performance statements each contained results and analysis that could not be confirmed as an accurate representation of performance and were not supported by suitable records.

3.81 The ANAO noted that each of the selected entities used spreadsheets in some form to manually compile data, or review and filter data following its extraction from a management information system. The ANAO has previously noted the risks arising from using spreadsheets as a primary form of evidence. Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. This can make spreadsheets unreliable corporate data handling tools, as accidental or deliberate changes can be made to formulae and data, without there being a record of when, by who, and what change was made.⁶⁴

3.82 There were also instances where system generated reports, relied on by an entity to support a reported result, could not reproduce the same result when requested by the ANAO. In these instances, the entities had not retained system extracts reflecting the original result. The differences arose from the system design allowing entries to be submitted, edited or updated at a later date, and as a result, an extract applying the same parameters may produce a different result to what was reported. The ANAO acknowledges the need for systems to be updated to reflect changes in records, particularly in those circumstances where program guidelines provide the opportunity for valid claims to be made outside of a particular reference period. However, in these cases, a reporting extract should be retained in its original form and stored securely outside of the system so the requirements of the PGPA Act are fully met.

Environment

3.83 Nine of Environment’s performance criteria included results and/or analysis that contained information that could not be confirmed on the basis of records provided by Environment, or were confirmed as inaccurate. For example, ‘409 taxa were either reported as revised or newly described’ was the reported result for the target ‘200 taxa revised or newly described under the Australian Biological Resources Study’. The ANAO’s review of the spreadsheets provided by the department only identified 372 taxa revised or newly described. When the department was asked to confirm whether the records were complete, the ANAO was further advised that their re-performance could only identify 359.

3.84 Similarly, for the performance target ‘At least 64 researchers, including early career researchers, supported under the Australian Biological Resources Study’, which was reported as ‘Achieved’, the ANAO identified 59 researchers compared to the 136 reported. The ANAO was advised this was due to the records only showing those that had commenced in 2016–17, which were then added to the previous financial year (81) to reach the reported result. However, in applying this approach, the expected number of researchers in 2016–17 should have been 55, still less than the target of 64. When seeking to reconfirm the result, the department further advised that its records now identified 58. The department advised that the discrepancies for these results, and those referred to above, were due to the original source being a ‘living database, which is updated daily’.

3.85 In other instances, the department confirmed that the performance statements contained results that were inaccurate. For example, the performance criterion ‘Improved compliance with national environmental legislation’ included details of the number of allegations/enquiries of potential non-compliance, and those that resulted in compliance cases. The performance statements noted 476 allegations/enquiries were received during 2016–17, which resulted in 77 new compliance cases. When asked to provide the records to support these figures, the department advised that ‘Further assessment determined there was 468 allegations of or inquiries about potential non-compliance with Part 3 of the EPBC Act…and 81 new compliance cases over the time period’. The Department advised it will amend its annual report to correct this inaccuracy.

Sports Commission

3.86 Of the eleven performance criteria assessed, only one was identified as containing inaccurate information on the basis of records supplied by the Sports Commission. The results presented against the performance criterion ‘Australia’s performance at major international sporting events’ in the performance statements noted a total of 24 Paralympic world champions, however the records provided to the ANAO to support the result contained only 23. The Sports Commission confirmed this was an oversight.

3.87 For the target ‘Increased following on ASC social media platforms and increased media reach’, the Sports Commission was unable to provide records to support the baseline used to determine the reported increase in its Twitter following. The other reported increases were able to be confirmed by the ANAO. For the remaining performance criteria, the ANAO was able to confirm reported results as complete and accurate against the records supplied, however as noted in paragraph 3.45, the methodologies supporting seven were considered to lead to a result that did not reflect the measure as described.

Appendix 1 Entity responses

Department of Jobs and Small Business

 

 

 

 

Department of the Environment and Energy

 

 

Australian Trade and Investment Commission

 

 

Australian Sports Commission

 

 

Department of Finance

 

 

Appendix 2 JCPAA Report 469: Commonwealth Performance Framework—Recommendations

1. The recommendations presented in the Joint Committee of Public Accounts and Audit’s Report 469: Commonwealth Performance Framework are reproduced in the following box.

Appendix 3 Criteria for the assessment of the relevance, reliability and completeness of performance information

2. The following criteria were presented in ANAO Report No.58 2016–17 Implementation of the Annual Performance Statements Requirements 2015–16. The criteria reflect Finance’s guidance to support the Commonwealth performance framework.

Table A.1: Criteria for the assessment of the relevance, reliability and completeness of performance information

Source: ANAO Report No.58 2016–17 Implementation of the Annual Performance Statements Requirements 2015–16.

Appendix 4 Assessment of the selected entities’ performance criteria against the performance information hierarchy

3. The following tables present the ANAO’s assessment of each of the selected entities’ performance criteria considered during the audit, against the different levels of the performance information hierarchy set out in Tables A.2–A.5 below.

Table A.2: Assessment of Employment’s performance information hierarchy

Source: ANAO analysis.

Table A.3: Assessment of Environment’s performance information hierarchy

Note a: This target is presented as per the 2016–17 Performance Statements. The target was reported in full in the 2016¬–17 PBS, as a 2016–17 performance criterion, and truncated in the 2016–17 Corporate Plan where it was reported as a performance criterion for ‘2017-18 and beyond’.

Note b: Note that these targets were presented as a composite measure in the 2016–17 Corporate Plan, however, they have been reported separately in both the 2016–17 PBS and Annual Performance Statements.

Note c: Note that this target was first introduced in the Portfolio Additional Estimates Statements, in February 2017.

Source: ANAO analysis.

Table A.4: Assessment of Austrade’s performance information hierarchy

Note a: FTA refers to Free Trade Agreement, which are international treaties that reduce barriers to trade and investment.

Note b: EMDG refers to Export Market Development Grants. The EMDG scheme is a financial assistance program for aspiring and current exporters.

Source: ANAO analysis.

Table A.5: Assessment of the Sports Commission’s performance information hierarchy

Source: ANAO analysis.