The purpose of the ANAO is to improve public sector performance and support accountability and transparency in the Australian Government sector through independent reporting to the Parliament, the Executive and the public. This purpose is delivered under the Auditor-General’s mandate in accordance with the Auditor-General Act (the Act) and comprises assurance in respect of entities’ administration and use of public resources, through quality evidence-based audit services and objective reporting.

The Auditor-General is an independent officer of the Australian Parliament whose functions, as set out in the Act, include undertaking the following range of activities on Australian Government sector entities:

• auditing the financial statements of Commonwealth entities, Commonwealth companies and their subsidiaries;
• auditing annual performance statements of Commonwealth entities on request in accordance with the Public Governance, Performance and Accountability Act 2013 (PGPA Act);
• conducting performance audits, assurance reviews, or audits of the performance measures, of Commonwealth entities and Commonwealth companies and their subsidiaries;
• conducting performance audits of Commonwealth partners as described in section 18B of the Act;
• providing other audit services as required by other legislation or allowed under section 20 of the Act; and
• reporting directly to the Parliament on any matter or to a minister on any important matter.

The ANAO aims to deliver an integrated program of high-quality audits that are timely, cost-effective and conducted in accordance with the ANAO Auditing Standards. Through this program, the ANAO aims to meet the needs and expectations of stakeholders including the Parliament, the Government, audited entities and the community and to add value to public sector performance, through accountability and transparency. In 2016-17, the ANAO delivered 250 mandated financial statement audits, 58 performance audits, two limited assurance reviews and 47 other assurance reports. There were also 39 appearances before, and submissions to, parliamentary committees.¹

The Auditor-General is required under the Act to set auditing standards to be complied with when performing all of these functions, except for other audit services under section 20 and when reporting directly to the Parliament or a minister on any matter under section 25. To date, the auditing standards established by the Auditor-General incorporate the standards made by the Auditing and Assurance Standards Board as applied by the auditing profession in Australia (the AUASB standards).

The requirement in the Act for the Auditor-General to set auditing standards for their work avoids a potential conflict of interest. The AUASB is an Australian Government entity and as such is audited by the ANAO, as is the FRC, who have the ability to set the direction of the AUASB. If the ANAO was required to adopt AUASB standards set by an Australian Government entity there could be a real or perceived impact on the independence of the Auditor-General.

The AUASB standards for financial statement audits (ASAs) are consistent with the International Auditing and Assurance Standards Board (IAASB) standards (ISAs). These standards are designed to be sector-neutral and include some public sector audit considerations in the application guidance. However, these considerations and associated guidance are not extensive. As a result, the standards focus largely on performing financial statement audits in the private sector.

The AUASB standard for performance audits, ASAE 3500 Performance Engagements is an Australian standard with no IAASB equivalent, however it is issued by the AUASB under the international framework for standards on assurance engagements. While the ANAO Auditing Standards have historically referenced the AUASB standards this has been in the context that, until recent years, there was not a robust standard-setting framework for international public sector auditing standards. Since 2004, The International Organization of Supreme Audit Institutions (INTOSAI) has worked to build the capacities and structures needed to provide a widely recognised set of international professional standards for public-sector auditing. This work is continuing and the INTOSAI Framework of Professional Pronouncements is expected to be finalised in 2019.

The application of auditing standards provides a framework against which quality can be measured. The ANAO operates in a contestable environment, and is committed to maintaining a business model that has quality at its foundation and reflects contemporary better practices and delivers strong performance. The ANAO’s Quality Statement links the Quality Assurance Framework to the strategic objectives and priorities of the ANAO:

Audit quality is the provision of timely, accurate and relevant audits, performed independently in accordance with the Act, ANAO auditing standards and methodologies, which are valued by the Parliament. Delivering quality audits results in improved public sector performance through accountability and transparency.

The ANAO Quality Assurance Framework is based on the auditing standard ASQC 1 Quality Control for Firms that Perform Audits and Reviews of Financial Reports and Other Financial Information, Other Assurance Engagements and Related Services Engagements. Incorporated into the framework is a system of quality control and monitoring programs that provides assurance that the ANAO complies with applicable professional standards and relevant regulatory and legal requirements, and that all reports issued are appropriate in the circumstances.

The challenge in implementing the quality framework is addressing the other aspects of quality in the quality statement, being the need for audits to be relevant and conducted in a timely manner and, accordingly, valued by the Parliament. The framework for the IAASB and AUASB standards focuses on the auditor obtaining evidence to support a conclusion about the outcome of measurement or evaluation of a specified subject matter against suitable criteria. The objective is to enhance the confidence of intended users of the information being audited. Public sector auditing has a similar focus in terms of process but has an enhanced objective in providing users with information and independent and objective assessments concerning the stewardship and performance of government policies, programs or operations. The audit focuses on principles of transparency, accountability, governance and performance.

To address all aspects of quality articulated in our quality statement, the ANAO recognises that the quality assurance framework needs to be based on a clear quality strategy that is aligned with our risk management framework rather than focussing solely on compliance with auditing standards. The ANAO’s quality policies and monitoring programs are reviewed on an ongoing basis and benchmarked against audit offices in other jurisdictions to ensure we continue to implement better practice. The future focus of the ANAO quality framework includes the strengthening of quality reporting to the ANAO’s Executive, including reporting on audit quality indicators and external reviews of the framework and audit files. The objective of these initiatives is to ensure that the ANAO is transparent about its quality processes. This recognises that to meet its purpose and to have sustainable impact, the ANAO needs to not only undertake work of high quality but also transparently demonstrate to external stakeholders that its work is of a high quality.

Financial statements audit program

The purpose of a financial statements audit is to determine whether an entity's financial information is presented in accordance with the applicable financial reporting and regulatory framework. In performing the audit, the focus of the auditor is on obtaining sufficient and appropriate audit evidence to express an opinion as to whether the financial information is free from material misstatement due to fraud or error.

In performing financial statements audits, the ANAO applies the Australian Auditing Standards (ASAs) made by the AUASB under section 336 of the Corporations Act 2001. The AUASB adopts pronouncements issued by the IAASB in preparing Australian pronouncements. These standards are legally enforceable for audits of financial statements prepared under Part 2M.3 of the Corporations Act. Through the standards issued by the Accounting Professional and Ethical Standards Board (APESB), members of the Australian Professional Accounting Bodies are also compelled to comply with the requirements of AUASB Standards.

The reporting framework for the Australian Government's financial statements is based, in large part, on standards made by the Australian Accounting Standards Board (AASB). The AASB bases its accounting standards on the International Financial Reporting Standards (IFRS) issued by the International Accounting Standards Board. Because IFRS are designed primarily for use by private sector and for‐profit organisations, the AASB amends the IFRS to reflect significant transactions and events that are particularly prevalent in the public and not‐for‐profit private sectors. In doing so, it takes into account standards issued by the International Public Sector Accounting Standards Board.

The auditing standards issued by the AUASB are described in ASA 100 Preamble to AUASB Standards as being sector-neutral. The application and other explanatory material in some standards contain considerations for public sector entities; however the guidance provided in these sections is significantly less than that provided in accounting standards.

There are a number of challenges in applying the AUASB standards in the public sector.

Users and scope

The primary purpose of financial statements is to provide relevant, reliable information to users about a reporting entity's historical financial position and performance. In the public sector, the users of financial statements include Parliament, Ministers and the community. In the private sector, the users of financial statements are investors, regulators and financiers. Public expectations of audits in the public sector may determine the objectives of an audit in addition to the mandate under which the audit is performed. If the scope of the audit does not address additional objectives there will be an expectation gap between what users of the financial statements and the accompanying auditor's report think they are getting and what they are actually getting from an audit. Examples of this expectation gap may arise in respect of the objective of the auditor in assessing the internal control environment. Under the auditing standards the auditor does this as part of their assessment of risk relating to the preparation of the financial statements rather than to provide a separate opinion on the effectiveness of internal controls. In the public sector, the auditor may increase the scope of the audit to include additional work on the effectiveness of internal controls to address this expectation gap. This provides a challenge for the auditor in determining how to report the outcome of this additional work.

This challenge is articulated in ISSAI 1315 INTOSAI Practice Statement to ISA 315:

The objectives of a financial audit in the public sector are often broader than expressing an opinion whether the financial statements have been prepared, in all material respects, in accordance with the applicable financial reporting framework (i.e. the scope of the ISAs). The audit mandate, or obligations for public sector entities, arising from legislation, regulation, ministerial directives, government policy requirements, or resolutions of the legislature may result in additional objectives. These additional objectives may include audit and reporting responsibilities, for example, relating to reporting whether the public sector auditors found any instances of non-compliance with authorities including budgets and accountability frameworks, and/or reporting on the effectiveness of internal control. These additional objectives may lead public sector auditors to assess additional risks of material misstatement. However, even where there are no such additional objectives, there may be general public expectations in regard to public sector auditors' reporting of non-compliance with authorities or reporting on effectiveness of internal control. Therefore, public sector auditors keep such expectations in mind, and are alert to areas that may give rise to risks of non-compliance with authorities or risks relating to effectiveness of internal control when planning and performing the audit.

Example – audit of National Disability Insurance Agency

In respect of the ANAO's audit of the financial statements of the National Disability Insurance Agency (NDIA) for 2015-16 a significant audit finding was reported in ANAO Report No. 33 of 2016-2017 Audits of the Financial Statements of Australian Government Entities for the Period Ended 30 June 2016. Access to the National Disability Insurance Scheme (the Scheme) is regulated via NDIA's assessment and approval of individual applicants against eligibility criteria. Once approved as eligible for the Scheme, a participant plan is formulated and approved that outlines the reasonable and necessary support required by the participant. The ANAO reviewed the NDIA's progress towards implementing an assurance framework, including a compliance program, over the integrity of claims paid to both scheme participants and service providers. The review noted that there were no documented compliance activities for payments made directly to self–managed participants and that the review program for payments made to providers was based on a non–statistical sample methodology. This methodology does not allow results to be extrapolated across the population to estimate the potential rate of non–compliance within the Scheme. The ANAO also noted that there was insufficient documentary evidence to demonstrate quality assurance processes over the integrity of decisions made concerning provider registrations, participant identity or eligibility and participant plan approvals.

In addition, the ANAO conducted a performance audit with the objective of assessing the effectiveness of controls being implemented and/or developed by the NDIA to ensure that Scheme access decisions are consistent with legislative and other requirements². The audit concluded that the NDIA has implemented some controls to ensure that Scheme access decisions are consistent with legislative requirements, but these have been inconsistently applied. In addition, comprehensive quality and compliance arrangements have not been implemented to mitigate the risk of incorrect Scheme access decisions.

These findings did not impact the opinion on the financial statements of NDIA. This is because the AUASB standards focus on forming a conclusion in respect of compliance with the reporting framework. Payments made under the Scheme to ineligible participants do not represent a material misstatement because the payments have been made and therefore represent an expense which, in the absence of a compliance program to determine the identity of recipients, will not be recovered. Even if the entity is able to estimate the proportion of payments that are invalid, the accounting standards do not require that amount to be recorded as an asset and reduction of expense if the amount is not recoverable. For the amount to be recoverable the entity would need to determine the individuals to whom all invalid payments are made. However, users of financial statements could reasonably expect that the audit would identify invalid payments made under the Scheme and report these to the Parliament. As such, it is necessary for the ANAO to determine how best to report on such findings, whether that is through reports to Parliament or as a separate compliance or performance audit, in order to meet the purpose of improving public sector performance through accountability and transparency.

The INTOSAI framework, which for financial audit essentially picks up the IAASB standards, provides more guidance in this area although not a clear way for meeting the challenge.

ANAO's response to challenges

In response to this challenge the ANAO will consider how to more effectively report to meet the needs of users of financial statements. This will include reviewing the assertions addressed in performing financial statements audits to determine whether they remain appropriate in the public sector context. We will also consider how best to report the findings of weaknesses in internal controls and compliance programs to drive improvement and enhance accountability and transparency. The guidance provided by the INTOSAI framework to public sector auditors adopting ISAs through ISSAI Practice Notes may be useful in this regard. For example, ISSAI 1315 referred to earlier provides guidance on risk assessment in the public sector context, ISSAI 1320 Practice Note to ISA 320 discusses materiality in the public sector context with a focus on qualitative materiality factors and ISSAI 1260 Practice Note to ISA 260 provides a broader context to the auditor's consideration of laws and regulations in audits of financial statements and references ISSAI 4000 Compliance Audit Standard in respect of auditing in this broader context.

Materiality

When establishing the overall strategy for an audit, the auditor is required to determine materiality for the financial statements as a whole. This overall materiality level is based on professional judgment and the auditing standards do not prescribe quantitative requirements or guidelines for determining an appropriate amount. They do point to the definition of materiality in financial reporting frameworks as a frame of reference. ASA 320 Materiality in Planning and Performing an Audit notes in its application and explanatory material that for public sector audits, materiality may be influenced by the financial information needs of legislators and the public.

In determining materiality for financial statements audits, a percentage is often applied to a benchmark, such as profit, total assets, revenue or expenses. These benchmarks may be relevant to individual entities in the public sector, however qualitative factors and other benchmarks also need to be considered. When determining overall materiality for Whole-of-Government financial statements, applying percentages generally used by private sector audit firms to acceptable benchmarks results in levels of materiality that would not be considered acceptable by users of the financial statements. For example, applying common percentages and thresholds, materiality for the Australian Government Consolidated Financial Statements would result in a materiality level of approximately two billion dollars. This is not considered appropriate due to its magnitude and the expectations of users and, accordingly the ANAO adopts a significantly lower materiality level in performing this audit.  

Performance audit program

A performance audit is a review or examination of the operations of an Australian Government sector entity. A performance audit aims to provide the Parliament with assurance relating to the administration of entities, programs and projects, including where these activities involve a Commonwealth partner. Performance audits also encourage better administrative and management practices by identifying and alerting public sector leaders to areas of weakness and highlighting areas of innovation and good practice for adoption more broadly across the sector.

A performance audit can be entity-specific or involve examination of a common issue as part of a broader cross-entity audit involving a number of entities. Performance audits include an examination of one or more of:

• economy (minimising cost);
• efficiency (maximising the ratio of outputs to inputs);
• effectiveness (the extent to which intended outcomes are achieved);
• ethical use and management of public resources; and
• legislative and policy compliance.

Historically, the primary focus of ANAO performance audits was on the effectiveness of existing program strategies and their delivery. These 'effectiveness' audits examine entities' existing program and project strategies, and assess the extent to which they are implemented as intended. In that context, there was also a focus on legislative and policy compliance, for example in grants and procurement audits.

Success in delivering the ANAO mandate extends beyond evaluating implementation effectiveness in performance audits, to evaluating outcome effectiveness as well as strategy development, economy and efficiency. The quality of performance information is key to the ability to do this. Extending the performance audit work program also creates the challenge to design innovative audit methodologies to assess entities' performance which still meet the requirements of the performance audit standards.

The ANAO's performance audits are conducted under ASAE 3500 Performance Engagements (ASAE 3500). ASAE 3500 is an Australian auditing standard with no IAASB equivalent. This standard sits within the IAASB (and AUASB) framework for assurance engagements such that to meet the requirements of the standard the auditor is required to meet all the requirements of that framework and the umbrella standard ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information (ASAE 3000). This framework is somewhat restrictive in terms of the concepts that need to be included and adapted to a form of engagement which does not neatly fit into the framework developed for the private sector context.

Definition and objective

ASAE 3500 defines performance auditing with reference to the principles of economy, efficiency and effectiveness. The objective of a performance engagement under the standard is to evaluate the performance of an activity, with respect to economy, efficiency and/or effectiveness against the identified criteria. The definition and objective do not encompass the broader aims of performance auditing in the public sector of improving accountability and transparency. This represents a challenge in driving quality, as the ANAO has defined it, and in meeting our purpose. In addition, the requirements of the standard do not address a key challenge of auditing performance, being the availability of quality data. To assess performance against the criteria requires robust information and data about entity inputs, outputs, processes and results. In the absence of quality information and data the auditor may not be able to establish sufficient and appropriate evidence to conclude on economy, efficiency and outcome effectiveness.

Materiality

Materiality also provides a challenge in performance auditing, more so than in the financial audit context. As described earlier, materiality in financial statements audits is based on applying a percentage to a benchmark. Judgment is applied in determining appropriate benchmarks, applicable percentages to apply and the impact of qualitative factors. In the performance audit context, when, in many cases, it is qualitative attributes that are being assessed, the determination of materiality is highly judgmental and may be multi-layered. Materiality needs to be assessed in the context of the audit. That is, whether a matter is significant or important is relative to the context in which it is being considered, and to the intended users of the audit report. Materiality may require an assessment of social and political significance, compliance, accountability and transparency. This provides a challenge in providing sufficient and appropriate guidance for auditors, particularly in a framework that is more targeted to the private sector and quantitative determinations of materiality.

Level of assurance

The ANAO provides a reasonable assurance conclusion in its performance audits. This is the same level of assurance as that provided in financial statement audits and indicates a high, but not absolute, level of assurance. Financial statement audit methodologies are built around a reasonable assurance model such that it is clear what level of work is required to form a conclusion. By its nature performance auditing is subject to higher levels of judgment around the quality and quantity of evidence required to form a reasonable assurance conclusion, particularly when qualitative data is being assessed. This creates a further challenge in achieving a quality audit which requires sufficient appropriate audit evidence to support the reasonable assurance conclusion.

'Misstatements' and qualifications

Another challenge that arises in applying the performance audit standard is the concept of misstatement. In a financial statement audit, a misstatement is quantified and then assessed to determine if it is material to conclude as to whether the audit opinion should be qualified. In the performance audit context, misstatement refers to variations in performance when assessed against the established criteria. From the auditor's viewpoint, such variations are findings which are to be reported and may potentially result in recommendations for improvement. Under ASAE 3500, these findings are considered only in the context of the impact on the audit conclusion, which, under the standard, is required if the finding is considered material. A material variation would result in a conclusion that an activity did not perform, either in certain material respects or all material respects against the criteria. As noted in the discussion of materiality, assessing whether a variation is material can be challenging and requires significant judgment.

If a conclusion is qualified as a result of a material variation or the inability to obtain sufficient or appropriate audit evidence the result is a negative form of opinion. Although the recently revised standard allows the auditor to include additional information such as findings and recommendations, these are required to be clearly separated from the qualified conclusion. As such, the conclusion section of the report would only present the overall conclusion of the audit. Conclusions and findings in respect of individual criteria or audit questions would be required to be reported in separate sections of the audit report.

Reporting under this framework could be viewed as a barrier to achieving improvement in the sector, which is key to the ANAO's purpose and definition of audit quality. This challenge is emphasised when audits are performed in respect of new frameworks and requirements, such as the Public Governance, Performance and Accountability Act (PGPA Act) risk and performance frameworks.

ANAO's response to challenges

The ANAO is considering whether the INTOSAI standards framework provides a more relevant basis for our audit methodology. In particular we have considered ISSAI 300 Fundamental Principles of Performance Auditing (ISSAI 300) and ISSAI 3000 Standard for Performance Auditing (ISSAI 3000) as part of our performance audit manual review and our review of the application of the revised ASAE 3500.

ISSAI 300 defines performance auditing as an independent, objective and reliable examination of whether government undertakings, systems, operations, programmes, activities or organisations are operating in accordance with the principles of economy, efficiency and effectiveness and whether there is room for improvement. Performance auditing seeks to provide new information, analysis or insights and, where appropriate, recommendations for improvement.

ISSAI 300 notes that the main objective of performance auditing is constructively to promote economical, effective and efficient governance. It also contributes to accountability and transparency.

This definition and objective is more closely aligned with the ANAO's purpose and definition of audit quality. Our comparison of the revised ASAE 3500 and ISSAI 3000 revealed that ISSAI 3000 is a more useful standard than ASAE 3500 because of its language, structure, public-sector specific content and supporting guidance (both within the standard and in the supporting guidelines in ISSAI 3100 Guidelines on Central Concepts for Performance Auditing). ASAE 3500 is limited by the framework under which it is written so that the basic concepts of a performance audit such as objectives and criteria are described in complex terms and definitions which need to be unpacked to understand the requirements in practical terms. In addition, ISSAI 3000 contains specific requirements around the audit adding value to the sector, transparency of reporting, specific independence considerations and practical approaches to performing audits which are not contained in ASAE 3500.

The most noted area of difference is in the reporting requirements of the standards. Reports prepared under ISSAI 3100 would be consistent in presentation with current practice in the Australian public sector. ISSAI 3000 requires the reporting of assurance to be transparent, that is, the auditor is required to transparently communicate the audit objective(s), scope, methodology and data gathered and any significant limitations in the report. The standard also requires findings and recommendations to be reported as well as conclusions. Findings are not required to be presented as qualifications to the conclusion, rather it is recognised that formulating conclusions may require a significant measure of judgement and interpretation in order to answer the audit questions, due to the fact that audit evidence may be persuasive rather than conclusive. The need for precision should be weighed against what is reasonable, economical and relevant to the purpose. In addition, reporting is required to be balanced by including all relevant arguments in the discussion.

The revised ASAE 3500 includes a checklist of minimum content requirements, some of which are not currently included in performance audit reports. Examples are certain statements around the responsibilities of the responsible party (audited entity), auditor's responsibility and compliance with independence and ethical requirements. While these could easily be appended to the performance audit report, they are less relevant to users than matters not required as minimum content, such as findings, recommendations and data sources, which would be mandatorily included under ISSAI 3000.

In terms of materiality guidance, ASAE 3500 provides extensive general guidance on materiality as a concept, however this is not particularly focussed to the different users of performance audit reports compared to other audit reports. ISSAI 3000 includes specific focus on social and political aspects of the subject matter and value add when assessing materiality.

In considering the adoption of the revised ASAE 3500 as part of the ANAO's auditing standards, the ANAO is reviewing the need to include the requirements to obtain an understanding of internal controls relevant to the evaluation of the activity's performance against the identified criteria and the content of the assurance report. These requirements are not consistent with the audit and reporting approach of Supreme Audit Institutions. ISSAI 3000 does not include an equivalent requirement to the requirement to obtain an understanding of internal controls as this may not be essential in all performance audits. Further, the reporting requirements of ISSAI 3000 may be more appropriate for reporting on performance audit engagements in the public sector and are more consistent with the ANAO's current practice.

Further, the ANAO's Performance Audit Manual has been revised and includes additional guidance from ISSAI 3000 in the areas of materiality and reporting.

In addition to the change in standards applied to the preparation of performance audit reports, the ANAO also considers other challenges of reporting when considering the scope of its work. For example, it was noted earlier that reporting under a standards-based framework could be viewed as a barrier to achieving improvement in the sector, particularly when audits are performed in respect of new frameworks and requirements, such as the PGPA Act risk and performance frameworks. The ANAO's response to this challenge is to select and scope audits to drive improvement rather than focus on criticism of implementation. That is, it may be more appropriate to limit the scope of an audit where it is clear that implementation has not progressed to a desired level and would result in a qualified audit report. For example, in the review of the Major Projects Report³, an annual priority assurance review of selected major procurement projects in the Department of Defence, certain forecast information is excluded from the scope of the review due to the lack of systems from which to provide complete and accurate evidence, in a sufficiently timely manner to facilitate the review.

The performance audit Implementation of annual performance statements requirements 2015-2016, was the first examination by the ANAO of the implementation of the new Commonwealth performance statement requirements. The report sought to highlight better practice that would assist other Commonwealth entities and, as a result of a review of publicly available information, the ANAO chose entities to include in the audit who we thought would be able to demonstrate better practice. The audit report also includes a summary of key learnings that may be considered in preparing annual performance statements.

Other assurance activities

Other assurance activities conducted by the ANAO consist of audits by arrangement with an audited entity, assurance reviews and priority assurance reviews. Audits by arrangement are conducted in accordance with the requirements for financial statement audits. Assurance reviews and priority assurance reviews are conducted in accordance with ASAE 3000.

Assurance reviews

The conduct of assurance reviews by the ANAO under section 19A of the Act is generally less extensive than performance audits or financial statements audits, primarily in terms of the nature and scope of issues covered and the extent of evidence collected and analysed. An assurance review is initiated on the basis of information obtained in the course of performing an Auditor-General function or in response to requests from stakeholders, including parliamentarians, parliamentary committees, entities and members of the public. Such a review usually provides a limited assurance conclusion and provides the ability to produce more timely reports on issues of contemporary interest to the Parliament. Limited assurance is a lower level of assurance than reasonable assurance. Responses to requests for audits were previously known as limited scope assurance reviews and the findings of the review were usually communicated to the requestor by means of a letter. The ANAO has developed its assurance review program, with reports now prepared under the standards-based framework, and the review Malabar Headland: 2016 Lease between the Commonwealth of Australia and the New South Wales Rifle Association was the first such assurance review to be tabled in Parliament in June 2017. This was followed by Department of Agriculture and Water Resources' Assessment of New South Wales' Protection and use of Environmental Water under the National Partnership Agreement on Implementing Water Reform in the Murray-Darling Basin (Murray Darling Basin review) tabled in Parliament in November 2017.

Priority assurance reviews

The Auditor-General's mandate also provides for the conduct of priority assurance reviews, with the Joint Committee of Public Accounts and Audit (JCPAA) determining whether an assurance review is a priority assurance review. A significant assurance activity that is undertaken each year by the ANAO is the priority assurance review of major Department of Defence (Defence) equipment acquisition projects. The outcomes of this review are published annually in the Major Projects Report.

The challenge which arises from assurance and priority assurance reviews is in reporting, that is, how to report meaningfully to Parliament under the auditing standard framework, especially using the format of an audit report which is required to contain minimum requirements. The ANAO is responding to this challenge by preparing long-form audit reports which contain information which is relevant to users but not required by the standards. We are also considering options for the presentation of conclusions in a format which meets the requirements of the standards but is also easy to read and informative.

As an example, the Murray Darling Basin review report was comprised of:

• an independent assurance report prepared in accordance with the requirements of ASAE 3000; and
• an appendix which outlined detailed findings to provide additional context to the independent assurance report.

The drafting of the independent assurance report was challenging in terms of providing an overall qualified conclusion which users who are not familiar with this style of reporting would understand while still complying with the requirements of the standard. The outcome was a conclusion which referred to the bases for the qualified conclusion within the conclusion section but prior to the conclusion itself:

Based on the procedures I have performed and the evidence I have obtained, the following matters have come to my attention:

• the lack of specific, measurable deliverables, and outcome measures in the milestones and criteria for assessing the performance of NSW under the Murray-Darling Basin NPA represent significant weaknesses in the performance framework; and
• while the Department of Agriculture and Water Resources has followed agreed processes for monitoring performance, there was a lack of evidence and explanation to substantiate its positive assessment of NSW's progress under Milestone 81 of the Murray-Darling Basin NPA for 2015–16, in light of serious issues raised about the state's water regulation arrangements. Importantly, there was little in the Department of Agriculture and Water Resources' submission to the Minister for 2015–16 to suggest that there were risks that NSW was not delivering environmental water consistent with the Basin Plan. These factors have limited the effectiveness of Department of Agriculture and Water Resources' assessment.

Other than the possible effects of these matters, nothing has come to my attention that causes me to believe that the assessment undertaken by the Department of Agriculture and Water Resources for 2014–15 and 2015–16 has not provided a high level of assurance about the protection and use of environmental water in the Murray-Darling Basin, as evaluated against the criteria.

The addition of the Appendix was needed to provide relevant and useful information to users of the report, such as detailed findings for each if the identified criteria and some background to those findings.

ANAO quality assurance processes

As noted earlier, the ANAO Quality Assurance Framework is based on the auditing standard ASQC 1. The objective of the framework is to provide reasonable assurance that:

• the ANAO and its personnel comply with the ANAO auditing standards, relevant ethical requirements, and applicable legal and regulatory requirements; and
• reports issued by the ANAO or signing officers are appropriate in the circumstances.

Specifically, in respect of monitoring activities, ASQC 1 requires the ANAO to establish a monitoring process designed to provide it with reasonable assurance that the policies and procedures relating to the system of quality control are relevant, adequate, and operating effectively. The monitoring activities that the ANAO historically employed for this purpose consisted of annual quality reviews of completed financial and performance audit files. Annual reviews of completed files are performed internally or by contractors on behalf of the ANAO.

In line with the ANAO's objective of implementing better practice and increasing transparency in respect of its quality processes, the ANAO has reviewed its quality processes and benchmarked them against audit offices in other jurisdictions and private sector firms. A number of state and international audit offices⁴ perform in-flight file reviews in the financial audit and performance audit areas. As a result of this review, the ANAO has implemented a system of in-flight reviews of financial statement audit files which are performed by a Big 4 firm as contractor to the ANAO. The reviews are performed during the audit planning and fieldwork stages, as well as at audit completion with a view to identifying any quality issues prior to finalisation of the audit.

In addition the ANAO has agreed to external peer reviews of completed performance audit files. These peer reviews are performed by the New Zealand Office of the Auditor General under an arrangement for peer reviews on an alternating basis. In 2017, the report from this peer review process was made available on the ANAO website.

The ANAO is also in the process of engaging ASIC to review the ANAO Quality Framework and two completed financial statement audit files for the 2016-17 year. The objective of this review is to enhance the monitoring program by increasing the level of objective external scrutiny and also to increase transparency of our framework and audit processes. In respect of performance audit files, it is intended that other models of external review will be considered in the future to supplement the current peer review program.

Findings from all monitoring programs are reported to the ANAO's Executive Board of Management and Audit Committee. The ANAO has also established a Quality Committee which is responsible for reviewing the findings of external and internal reviews in relation to quality and monitoring the ANAO's progress in addressing these findings and recommendations. To increase transparency of the ANAO's quality processes, we are also considering external reporting on the operating effectiveness and efficiency of the Quality Framework. It is envisaged that this will involve reporting against a set of audit quality indicators which the ANAO is currently measuring and reporting on internally. Finally, the ANAO is reviewing the INTOSAI Performance Management Framework (PMF) to consider whether it may provide a more relevant framework for assessing our quality policies and processes than the current ASQC 1 framework. The PMF provides Supreme Audit Institutions with a benchmark for assessing their performance against established international good practice for external public auditing. Other jurisdictions, such as New Zealand have performed and published detailed assessments against this framework.

Concluding comments

The ANAO is committed to undertaking its role and achieving its purpose by being an organisation that: is independent and responsive; provides value adding audit services; and is confident in the delivery of audit services over future years. Critical to the ANAO's success in delivering on its purpose is the ability to anticipate changes occurring in its operating environment and to effectively respond to these changes through the audit program.

This commitment can present challenges when operating in a standards-based framework. It requires the ANAO to continuously review and improve its audit methodologies and reporting approaches and to evaluate available standards frameworks to ensure that the standards and frameworks implemented are the most appropriate for achieving our purpose.

In order for the Parliament, public sector entities and the public to have confidence in the ANAO's audit findings our work must be underpinned by a rigorous quality assurance process. The quality assurance framework needs to be based on a clear quality strategy that is aligned with our risk management framework and purpose rather than focussing solely on compliance with auditing standards.

The ANAO continues to explore options to provide its key client, the Parliament, and other stakeholders with additional insights into public sector administration. A specific focus is the development and publication of materials that draw together key learnings from across the ANAO's suite of activities, with the aim of assisting entities to improve their administration and delivery of outcomes.

With the expansion of the audit program to include the breadth of the Auditor-General's mandate, supported by flexible and innovative methodologies and quality assurance processes, and the sharing of insights from its work the ANAO will continue to achieve its purpose of improving public sector performance through accountability and transparency.