Introduction

1.1 The Australian Taxation Office (ATO) is responsible for the effective management of Australia’s taxation and superannuation systems. This includes responsibility for achieving confidence in these systems by ‘helping people understand their rights and obligations, improving ease of compliance and access to benefits, and managing non-compliance with the law.’⁵

1.2 One of the ATO’s strategic objectives is to provide the community with confidence in its administration of the tax and superannuation systems that support the collection of the right tax at the right time, for the wellbeing of all Australians.⁶ The ATO’s Corporate Plan 2022–23 identifies the tax gap as part of a strategic objective: ‘We build community confidence by sustainably reducing the tax gap and providing assurance across the tax, superannuation and registry systems.’⁷

The tax gap

1.3 The ATO defines the tax gap as ‘an estimate of the difference between the amount of tax the ATO collects and what we would have collected if every taxpayer was fully compliant with tax law.’⁸

1.4 The purpose of tax gap estimation is to measure what is ‘not directly observable’⁹ — revenue not collected due to taxpayers misreporting their true tax position:

• Due to misunderstanding their obligations;
• by choice; or
• by taking a tax position that differs from the ATO’s view of the law.¹⁰

1.5 Prior to 2012, there was no systematic program of tax gap estimation for any of the taxes administered by the ATO other than for the Goods and Services Tax (GST) and the Luxury Car Tax. The Joint Committee of Public Accounts and Audit¹¹ and the House of Representatives Standing Committee on Tax and Revenue¹² encouraged the ATO to publish its tax gap analysis on GST collections, and to continue to examine potential methodologies for tax gap estimates. The ATO published its first gap estimates (for the GST and Luxury Car Tax) in 2012.

1.6 The ATO currently calculates 21 tax gaps across three different groups: six transaction-based tax gaps (such as the GST or fuel excise); six administered program gaps (such as fuel tax credits or JobKeeper); and nine income-based tax gaps (such as those for individuals not in business, and large corporate groups).¹³ Figure 1.1 depicts the estimated total tax gap for 2019–20.

Figure 1.1: Estimated total tax gap for 2019–20

Source: ATO documentation.

1.7 The ATO defines individuals not in business as ‘Individuals with no business connections (including Wealthy Australians) but may have passive income (such as interest, dividends) and are not connected to a high wealth group with over $50 million in net assets.’

1.8 Figure 1.2 illustrates the way the ATO defines key elements of the tax gap estimate.

Figure 1.2: The tax gap framework^a

Note a: This graph is intended to explain concepts, not report results, and is not to scale.

Note b: Accounts for errors on tax returns that are not identified through the Random Enquiry Program (see paragraph 2.62).

Note c: Represents the unreported tax liability estimated through the Random Enquiry Program plus an estimate for people outside the system, after subtracting the value of amendments.

Note d: Tax liabilities that the Commissioner has assessed to be not legally recoverable or not economical to pursue.

Source: ANAO analysis of ATO documentation.

1.9 The tax gap estimate for individuals not in business has been calculated from the 2013–14 to the 2019–20 tax years. Table 1.1 outlines the ATO’s tax gap estimate for individuals not in business from 2014–15 to 2019–20. The ATO has an internal target of a net tax gap of 4.5 per cent. The ATO advised that this target was an aspirational internal target, and that reaching this target would depend on technological advances, environmental and economic factors, and policy changes.

Table 1.1: Tax gap estimates for individuals not in business

Note a: Amounts attributable to ATO compliance work.

Note b: Tax liabilities that the Commissioner has assessed to be not legally recoverable or not economical to pursue.

Note c: Represents the unreported tax liability estimated through the Random Enquiry Program plus an estimate for people outside the system, after subtracting the value of amendments.

Note d: Accounts for errors on tax returns that are not identified through the Random Enquiry Program (see paragraph 2.62).

Note e: Theoretical tax minus tax paid voluntarily.

Note f: Theoretical tax minus total tax paid.

Source: ATO documentation.

Previous audits and review

1.10 Auditor-General Report No. 36 1998–99 Pay As You Earn Taxation – Administration of Employer Responsibilities¹⁴ observed that the ATO had identified an area for improvement that, ‘sound methods are required to estimate the potential PAYE tax gap.’ (For more information on how this matter has been addressed see from paragraph 2.33)

1.11 Auditor-General Report No. 30 2005–06 The ATO’s Strategies to Address the Cash Economy¹⁵ noted that, at the time, the ATO did not attempt to calculate the tax gap attributed to undeclared cash transactions. (For more information on how this matter has been addressed see from paragraph 2.63)

1.12 Auditor-General Report No. 39 2013–14 Compliance Effectiveness Methodology¹⁶ examined the ATO’s Compliance Effectiveness Methodology, a measure that sought to optimise voluntary compliance. The audit report suggested top-down measurement of the effectiveness of collection of payable revenue could be accompanied by a bottom-up approach to form a high-level view of ATO effectiveness. The ATO’s approach to measuring the entire tax gap estimate combines both top-down¹⁷ and bottom-up¹⁸ approaches¹⁹, with the individuals tax gap being calculated by a bottom-up methodology.

1.13 Auditor-General Report No. 27 2015–16 Strategies and Activities to Address the Cash and Hidden Economy²⁰ noted that the ATO did not have a robust estimate of the taxation revenue foregone from the cash economy population and that publication of a robust estimate of the revenue at risk from the cash economy would contribute to increased transparency and enable the estimate to become a performance indicator for management of the cash economy risk over time. Further, the report stated that tax gap analysis could be used to measure the impact of activities on the cash economy, and to establish strategies and activities to address the cash economy. (For more information on how this matter has been addressed see from paragraph 2.63)

Rationale for undertaking the audit

1.14 In its 2021–22 annual report, the ATO estimated that the overall net tax gap²¹ for the Australian tax and superannuation system was $33.4 billion for 2019–20. Within this total, the estimated net tax gap for the taxpayer population of ‘Individuals not in business’ was the second largest gap in dollar terms — $9 billion for 2019–20²², meaning that for this gap, the ATO collected around 94.4 per cent of the tax revenue it would have collected if all taxpayers were fully compliant with tax law. This audit will provide Parliament with assurance that the ATO is appropriately managing the tax gap for individuals not in business.

Audit approach

Audit objective, criteria and scope

1.15 The objective of the audit was to assess the effectiveness of the Australian Taxation Office’s arrangements for identifying and reducing the income tax gap for individuals not in business.

1.16 The audit focused on the ATO’s calculation of the 2018–19 income tax gap for individuals not in business estimate. On 31 October 2022, the ATO released its income tax gap for individuals not in business estimate for 2019–20. Further comment on the 2019–20 estimate process is in paragraph 2.38.

1.17 To form a conclusion against the objective, the following criteria were adopted.

• Does the ATO use appropriate methods for identifying the tax gap?
• Has the ATO implemented risk-based compliance strategies for reducing the tax gap?
• Does the ATO assess the effectiveness of its compliance strategies for reducing the tax gap?

Audit methodology

1.18 The audit methodology included:

• review of ATO documentation such as strategies, plans, risk documents, meeting papers and minutes, reporting and internal briefings;
• meetings with ATO officers; and
• detailed technical walkthroughs of processes and procedures with ATO officers.

1.19 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $609,949.

1.20 The team members for this audit were Shane Armstrong, Connor McGlynn, Ella Young, Ben Thomson, Amanda Reynolds, Molly Wu, Nathan Singlachar, and David Tellis.

2.1 The ATO’s Corporate Plan 2022–23 identifies the tax gap as part of a strategic objective: ‘We build community confidence by sustainably reducing the tax gap and providing assurance across the tax, superannuation and registry systems.’²³ To achieve this, the ATO sought to design and implement a methodology to reliably and accurately estimate the tax gap for individuals not in business. An effective methodology should be well communicated to increase transparency²⁴ and improve perceived reliability of the tax gap estimate for individuals not in business.

2.2 The ATO describes its overall methodology for estimating the individuals not in business tax gap in the Individuals Not in Business Tax Gap Technical Guide (the Technical Guide), an internal document developed and updated annually by the ATO’s Client Engagement Group Services business line. A summary of the ATO’s full methodology is at Appendix 3. In brief, it involves:

• a Random Enquiry Program (REP), which entails selecting a random sample of tax returns from individuals not in business, verifying the returns through a data driven process or manually reviewing them to identify and adjust errors, and extrapolating the incidence of non-compliance and the mean value of adjustments across the wider individuals not in business population;
• processes to estimate the impact of factors not measurable through the REP, namely the impact of non-pursuable debt, of non-registrants or long-term non-lodgers, and of non-detection due to income misreporting, hidden wages and other issues; and
• calculations using the outputs of the above steps and other figures to arrive at the tax gap estimate (gross and net) and theoretical tax liability.

Is the ATO’s tax gap methodology well designed?

Expert advice and methodology coverage

2.3 To establish a methodology to calculate the tax gap for individuals not in business (the individuals tax gap), the ATO conducted external consultation between July 2014 and September 2015 (see Table 2.1) which addressed the measurement of multiple tax gaps. Examining the individuals tax gap as part of a broader process enabled the ATO to engage with a range of international experts in the field.

2.4 At this time, there was no single standard or better practice guide provided by the International Monetary Fund (IMF) or the Organisation for Economic Co-operation and Development (OECD) for the ATO to use as a model to develop its tax gap methodology.

2.5 An IMF report, United Kingdom: Technical Assistance Report – Assessment of HMRC’s Tax Gap Analysis²⁵, published in 2013 informed the ATO’s thinking during the design process. In 2021, six years after the ATO had developed and implemented its methodology, the IMF published The Revenue Administration Gap Analysis Program – An Analytical Framework for Personal Income Tax Gap Estimation, a technical manual which constituted a better practice guide. The ANAO analysed whether the ATO’s methodology was consistent with this guide. Further information on this analysis is available at paragraph 2.27.

Table 2.1: External consultation process

Note a: The ATO advised that its meeting with Treasury covered only the large corporate tax gap.

Source: ANAO analysis of ATO documentation.

2.6 The ATO also established a Tax Gap Advisory Panel (the Advisory Panel) in March 2014 consisting of three experts in the field.²⁶ The Advisory Panel’s purpose was to consider whether methodologies could be relied upon to produce a robust tax gap estimate, and were likely to be broadly accepted.

2.7 The members of the Advisory Panel later became members of the Tax Gap Expert Panel (the Expert Panel), which first met in its new capacity in November 2016 to provide further support to the ATO once the methodology had been established.²⁷ The role and continued function of the Expert Panel are considered from paragraph 2.65.

2.8 Between April and May 2015, the ATO engaged ‘the services of the ABS to advise on sample design’. The purpose of the procurement was to obtain ‘advice on sampling plan design – stratification dimensions (assumptions), extrapolation advice and calculation. Advice on sample size options and trade-offs. Explanation of reasons for any variation with international experience.’

2.9 The ATO also engaged the services of a US-based economics and statistics consultant, to conduct external peer review during the consultation process.

2.10 Table 2.2 outlines the aspects of the tax gap methodology about which the ATO engaged with external experts. The ATO covered each of the elements, ensuring adequate coverage of potential areas of contention.

Table 2.2: External consultation analysis

Source: ANAO analysis.

2.11 The ATO initially considered a potential sample size of 5000 random audits (covering both individuals and small business) over a period of four years.

2.12 Three of the four meetings with the ABS considered the issue. The ABS stated that 1250 audits annually for a total of 5000 over four years ‘sounded too low a number’. The ATO advised that a sample size of 2500 would constitute a snapshot of the overall health of the system, with a sample size of 5000 constituting an indicator of broad compliance movements over time. The US-based expert advised that a sample of 1250 random audits per year would not provide a good precision dollar point range, but would provide a reasonable estimate of rates of non-compliance.

2.13 At its meeting on 30 September 2015, the ATO Executive endorsed a pilot random enquiry program to review individual and small and medium business income tax returns.

2.14 The ATO settled on a three-year bundled sample of 1635 random audits (545 per year) as its methodology to form the basis of the annual tax gap estimate. The ATO stated in October 2022 that its sample size ‘[enables] the ATO to conduct a statistically reasonable sample size while balancing the broader Individuals compliance program’, and that ‘this was based on the original intended target of over 2000 random audits over a four year observation window.’ The purpose of bundling was to ‘increase the sample size’, and ‘assist in reducing the confidence interval on the estimate without the need for a larger annual sample.’

2.15 The ATO stated in its Technical Guide that the Expert Panel had:

highlighted that this bundling also causes an averaging effect creating a tension between how much is bundled and the need for insights on the trend. Given this advice, we have arrived at a three year bundling approach to try and minimise the averaging effect.

2.16 Expert Panel Minutes indicate that the Expert Panel was comfortable with this approach, with an action item generated to directly address the terminology around bundling. There is no evidence that sample size has been revaluated by the ATO since the methodology was established.

2.18 Stratification²⁸ of a sample was also recommended by the US-based expert, who stated that the ATO needed to consider how it sampled and areas to oversample, recommending that it would be best for the ATO to oversample in higher risk and higher impact areas using stratified sampling techniques.

2.19 In opting to oversample the rental tax strata by 10 per cent and undersample the zero tax strata by 10 per cent, the ATO stated in June 2022 that this: ‘was a judgement call based on the principle of over/undersampling based on where the tax gap is likely to manifest.’

2.20 The Expert Panel was briefed on sample design including the oversampling and undersampling of certain strata at meetings in September and November 2016. While the Expert Panel endorsed the individuals not in business tax gap estimate in February 2018 (which included sample size information), there is no evidence that the Expert Panel explicitly endorsed the ATO’s sampling approach in any of its meetings.

2.21 Analysis was performed on the minutes and meeting papers of meetings with experts to determine whether the ATO absorbed feedback into a single document. Feedback from all external meetings was incorporated with the exception of one meeting with the ABS, however the feedback from this meeting was similar to that provided in a previous meeting with the ABS on 10 March 2015.

Endorsement of the methodology

2.22 On 30 August 2016, the Advisory Panel provided its endorsement of the methodology to determine the tax gap for individuals not in business, endorsing it as ‘defensible and fit for publication’, noting ‘as with all tax gap estimates, we note that as further information becomes available and the approach matures, the estimates are likely to be subject to restatement and/or revision.’

Monitoring best practice and updating methodology

2.23 The ATO’s primary method of remaining up to date on developments in the field is its membership of the Advanced Tax Gap Analysis Community of Practice (known as the OECD Community of Practice). The OECD Community of Practice’s objectives are to share methodologies and best practice to estimate tax gaps more accurately, to share communication strategies for releasing gap estimates to the public, to provide opportunities for cross-border collaboration, and to consult with other relevant groups.

2.24 The ATO advised in May 2022, that to keep up to date on developments in tax gap methodology in academia that it subscribed to various tax research and policy distribution groups, attended academic conferences, and had attended and presented at an international tax gap conference. The ATO also advised in May 2022 that the ‘Expert Panel is also able to assist with staying current with relevant literature.’ There is no standing agenda item in the Expert Panel meeting papers to discuss developments in the field.

2.25 The ATO advised in August 2022 that the team responsible for the tax gap worked with business line leads to improve the tax gap estimate process, with the tax gap team meeting annually to discuss whether any improvements or additional work are required. The ATO provided one example of this occurring. The ATO considered how things had gone over the year, the role played by the Expert Panel, what to expect in the future, and identified the need to improve the reliability of the estimate by working more on ‘[People outside the system], non-detection uplift factors, non-pursuable debt.’

2.26 The ATO advised in May 2022 that the Expert Panel was ‘the avenue for revisions/changes to the tax gap methodology’ with any proposed changes to the methodology submitted as part of the process undertaken to obtain endorsement from the Expert Panel, but that the methodology ‘has remained essentially the same since the first publication’.

IMF best practice

2.27 As mentioned in paragraph 2.5, in 2021, the IMF published The Revenue Administration Gap Analysis Program – An Analytical Framework for Personal Income Tax Gap Estimation (the IMF guide) a technical manual which constituted a better practice guide.

2.28 To attain further assurance over the ATO’s methodology, ATO procedures were analysed against the IMF guide. The ATO’s procedures are largely aligned with the IMF guide, taking into account all components identified by the IMF as significant, reviewing and stratifying the sample selected, calculating net and gross gaps, analysing, reporting and using gap results, conducting regular monitoring and quality assurance of REP work, and fully articulating how sampling should be performed.

2.29 The following elements of the IMF guide have not been considered by the ATO in revising its methodology:

• The IMF suggests using a supplementary estimate or adjustment to cover low frequency but high value risk that a random audit program is unlikely to reliably capture. The ATO does not employ such a measure.
• The IMF also suggests having a clear policy on how and when statistics such as population data and uplifts are revised. The ATO does not have a policy on this matter.

Has the ATO’s tax gap methodology been well implemented?

2.31 To determine whether the ATO’s methodology has been well implemented, the ANAO focused upon the processes below, each of which plays a material role in producing the gap estimate.

2.32 The ATO’s process for generating the tax gap estimate for 2018–19 formed the basis of the audit team’s analysis. During the audit, the ATO published the tax gap estimate for 2019–20. There was no material change in the ATO’s procedures between the two years.

Extracting the sample for the Random Enquiry Program (REP)

2.33 The ATO’s methodology for estimating the individuals tax gap necessitates the carrying out of a range of steps depicted in Appendix 3.

2.34 The initial step in the ATO’s individuals tax gap methodology is to randomly select a sample of taxpayers whose returns will be reviewed as part of the REP. Responsibility for obtaining this selection is assigned to Smarter Data, the ATO’s specialist data and analytics team.

2.35 Smarter Data’s work in extracting the REP sample is guided by a Target Selection Rationale (TSR) produced annually. This document describes the process for drawing the sample and contains instructions on how to define, stratify²⁹ and exclude certain taxpayers³⁰ from the individuals not in business population.

2.36 A preliminary random sample is obtained, and tax return data for each taxpayer in the sample is provided to the relevant business line for quality assurance.

2.37 The final random sample of 545 taxpayers is extracted, and tax return data for each taxpayer in the sample is provided to the relevant business line. A reserve sample is also generated in case any taxpayer needs to be replaced during manual reviews for the REP.³¹

2.38 Smarter Data performs the extraction of the sample through the application of Structured Query Language (SQL) code. To assure that the extraction procedure matches the ATO’s documented process, the ANAO analysed the generation of the sample data for the 2018–19 tax gap estimate. The ANAO conducted an SQL code review, examined sampling results, inspected error rates and the use of reserve cases, and reviewed emails between Smarter Data and other business lines. The analysis found that, overall, the ATO produced credible sample data in accordance with TSR specifications. There were no methodological changes between the processes used to extract the sample for the 2018–19 and the most recent (2019–20) tax gap for individuals not in business.

Verifying taxpayers through data driven review

2.39 After obtaining the random sample, the ATO subjects the selected individuals to internal profiling, identifying those who have:

• simple tax affairs;
• a clear residency status;
• income that matches third party data on ATO systems; and
• only small amounts on their tax return that cannot be verified using data.

2.40 The ATO regards any taxpayer who fits these criteria to have an immaterial tax gap and refers to their tax return as ‘data verified’. To minimise compliance costs for taxpayers, the ATO excludes these individuals from having their returns investigated any further. The average number of cases ‘data verified’ is around 21 per cent.

Conducting REP manual reviews

2.41 Sampled individuals whose tax returns are not data verified are progressed to manual review. In the context of the REP, the purpose of a manual review is to determine the difference between a taxpayer’s actual tax liability under the law and the amount originally assessed. It involves the ATO:

• assessing all items of an individual’s return, including the correctness of their income reporting and their entitlement to deductions and offsets; and
• contacting the taxpayer or their agent to obtain substantiation of items on the return — where appropriate, third parties (banks, employers and rental property managers) are also contacted to verify claims.

2.42 The work of conducting the manual reviews is carried out by Case Actioning Officers (CAOs). The ATO advised in August 2022 that CAOs working on the REP are ‘experienced auditors selected from our workforce who are … usually skilled in either rental or work related expenses risks’ (two significant drivers of the individuals tax gap — see Figure 2.1). CAOs receive training on the procedural approach required for the REP.

2.43 The work of CAOs is supported by guidance documentation and templates. Guidance Notes for CAOs detail the steps for actioning cases and refer the reader to other guidance documentation and templates.

2.44 Manual reviews are conducted with the use of Siebel, the ATO’s Client Relationship Management system. The process of carrying out the REP case work as a review versus an audit is similar. The key differences between the two are that:

• in an audit, the ATO can amend a return without the taxpayer’s consent;
• penalties are not imposed for REP review cases; and
• audit cases are subject to formal objection rights, while review cases are not.

2.45 The work of CAOs is supervised by Team Leaders and Tax Technical Assessors (TTAs).

2.46 CAOs record their observations during manual reviews in the form of notes on Siebel and in template documents attached to Siebel. CAOs also capture all REP case results in an Access Database. Data extracted from this database underpins the calculation of the tax gap. Entries in the Access Database are reviewed by Team Leaders at the completion of a case.

2.47 The results of the REP and analysis of these outcomes are collated in a document known as a Close Out Report.

Quality assurance of REP manual reviews

2.48 The IMF Guide notes that ‘regular monitoring and quality assurance reviews can … facilitate higher standards of data recording’, and identifies ‘quality assurance audits, and especially data recording’ as important elements of a REP.

2.49 As mentioned at paragraphs 2.45 and 2.46, the manual reviews for the REP are supervised by Team Leaders and TTAs, who conduct quality control processes at various stages as CAOs carry out their work. In addition, there are quality control checks performed by staff from two separate technical areas within the ATO.

2.50 Since the 2014–15 individuals tax gap estimate, the ATO has also engaged a former Deputy President of the Administrative Appeals Tribunal to ‘review, and assess the correctness and consistency of, specified case work undertaken by case officers for the REP’. This external reviewer has analysed an average of 68 cases per year. Reports published by the reviewer have tracked the performance of the case work that underpins the REP year-to-year, with his report on the ATO’s manual reviews of 2017–18 tax returns observing that:

I have previously reviewed the program for the 2015, 2016 and 2017 income years and had seen marked improvement from one year to the next — not so much in the quality of the work I was reviewing (which was already of a high standard) but in the way it was being recorded.

2.51 The reviewer concluded:

I am also comfortably satisfied that, across the 70 cases reviewed, there has been consistency in the method of calculating the tax adjustments decided upon, and correct recording of those adjustments in the tax gap database.

2.52 The ATO’s employment of three levels of internal review, and having an external reviewer track the performance of case work, provides a fit-for-purpose framework to provide quality assurance over the conduct of the REP.

Methodological process to determine the final tax gap

2.53 Both the verified cases and the findings from the REP case work are ‘fed into’ a ‘methodological process … to determine the final tax gap’. This methodological process is laid out in the Technical Guide and has the steps described in Table 2.3.

Table 2.3: Methodological process for calculating individuals not in business tax gap

Note a: Non-pursuable debt refers to tax liabilities that the Commissioner of Taxation has assessed to be not legally recoverable or not economical to pursue.

Note b: The figure for amendments represents the amount recovered through the ATO’s broader compliance activities. It is generated early in the estimation process. To ascertain the figure, the ATO employs a ‘minimum client initiated’ approach that compares what a taxpayer would have voluntarily paid with their ultimate assessment.

Note c: Tax paid is the equivalent of tax reported minus the value of non-pursuable debt. Tax reported uses the ‘net tax’ definition in line with Taxation Statistics and other publications.

Source: ANAO analysis of ATO documentation.

2.54 Prior to putting the data from the sampled cases through the methodological process, the ATO applies its bundling approach. As discussed at paragraph 2.13, when working out the individuals tax gap for a given year, the ATO’s chosen method is to combine the random sample of 545 tax returns for that year with the random samples for two previous years, producing an overall sample of 1635 cases.

2.55 Notwithstanding the ATO’s chosen bundling approach, when a particular year’s tax gap estimate is first published, it is based only on the random samples for the two previous years (1090 cases). This is due to the timing of the REP manual reviews: when a gap estimate is being endorsed and prepared for initial publication, there are usually numerous REP cases from the latest year that have not yet been finalised. It is only when the ATO comes to calculate the individuals tax gap for a subsequent year — at which point it also refreshes previously published estimates — that it incorporates data from a full three-year sample.³²

2.56 Additionally, certain figures involved in estimating the individuals tax gap take substantial time to become final.³³ Accordingly, when the ATO initially estimates a gap, it uplifts or uses projections for these figures. The ATO then replaces these uplifts and projections with gradually more complete data through its process of refreshing earlier estimates.

Estimating unreported amounts for the verified sample (step 1b)

2.57 The purpose of step 1b is to establish a mean amendment and incidence rate of amendments for those individuals who were sampled as part of the REP but did not have their tax returns manually investigated because they were verified through data driven review (see from paragraph 2.39). The ATO advised in June 2022 that it carries out this step because even if such individuals have had their returns data verified, the ATO cannot confidently conclude that they have completely complied with their tax obligations. One reason for this is that the criteria for verifying tax returns means that verified taxpayers may still be entitled to and/or have claimed deductions.

2.58 To identify the mean amendment and incidence rate of amendments for the verified group, the ATO applies data from manually reviewed taxpayers who claimed less than $1000 in deductions.³⁴ The ATO considers these manually reviewed individuals to be representative of the verified group, and expects them to have similar errors in their tax returns.

Estimating for people outside the system (step 1e)

2.59 The ATO’s estimate for people outside the system (POTS) is intended to account for the impact of ‘non-registrants or long-term non-lodgers on the gap’. The estimate uses ABS population data and income tax returns lodged by individuals, and makes assumptions regarding people who may not need to lodge. The ATO combines this estimate with the results from the REP to identify the number of POTS who have potential tax obligations and the theoretical amounts owing, and apportions the population and amounts between individuals in and not in business.

2.60 In estimating the impact of POTS, the ATO assumes that ‘the incidence and relative magnitude of income non-compliance in the [REP] is representative of the incidence and magnitude of income non-compliance outside the system’. The ATO acknowledged in June 2022 that the nature of the assumptions used for the POTS estimate are subjective.

2.61 Along with the assumptions made in steps 1b and 1e, the ATO makes a series of other assumptions (see Appendix 4). The ATO’s Reliability Assessment requires the ‘sensitivity to underlying model, assumptions, judgement or expertise’, and ‘assessment of assumptions, judgement or expertise’ to be considered when deriving a Reliability Assessment (see from paragraph 2.79).

Estimating for errors not detected (step 2)

2.62 The Technical Guide states that ‘to derive a credible gap estimate’, the ATO must add estimates for non-detection to the base unreported tax liability ascertained through steps 1a–1e above. The ATO has identified ‘three factors that contribute to non-detection’ in the context of the individuals not in business gap:

• income misreporting;
• deductions and other issues; and
• the impact of hidden wages.

2.63 The ATO’s processes for estimating each of the above factors differs. To account for income misreporting, the ATO uses a standard uplift based on international research. For deductions and other issues, the ATO applies an uplift based on the findings from the external review of the REP manual reviews (see from paragraph 2.50), which it applies to the part of the unreported tax liability that can be attributed to non-income issues. The estimation of the impact of hidden wages draws upon data from the ATO’s pay as you go withholding gap estimate about the base tax impact of hidden wages. By making allowances for refunds, the ATO converts this base impact to the effective impact of hidden wages on individuals income tax, and then apportions this between the individuals not in business and other tax gaps.

ANAO verification of methodological process

2.64 The ATO provided the ANAO with input and output data for each step of the methodological process as conducted for the estimation of the 2018–19 individuals not in business tax gap. Following the ATO’s procedures as outlined in the Technical Guide and other relevant guidance, the ANAO replicated the process. It was found that that the ATO’s methodological process was repeatable and a result materially the same as the ATO’s was achieved.

Endorsement of the tax gap estimate

2.65 The Tax Gap Expert Panel Charter outlines the ATO’s expectations from the Expert Panel regarding endorsement of the tax gap estimate:

The Tax Gap program team will look to the Expert Panel for:

• Assessment and, as appropriate, endorsement of results for presentation to the ATO Executive.
• Support in terms of the ATO communications of the gap program credibility and reliability as well as contextualising the methodologies and results on an international basis.

2.66 While the Expert Panel Charter has been amended several times since its inception in 2016, these portions have remained consistent throughout.

2.67 The Expert Panel endorsed the 2018–19 tax gap estimate on 27 May 2021. When asked to provide evidence of the Expert Panel endorsing the latest technical guide, the ATO stated: ‘As there were no methodological changes between 2021 and 2022, and no changes to the reliability rating, the 2022 technical guide was not submitted for endorsement’.

2.68 The tax gap for individuals not in business is also reported in the ATO’s Annual Report. The ATO Executive approves the estimate as part of its process of approving the broader ATO Annual Report.

Has the ATO’s tax gap methodology been well-communicated?

Public reporting

2.69 The ATO publishes a summary version of the methodology used to measure the individuals tax gap on its website³⁵, the individuals tax gap estimate³⁶, and the most recent reliability estimate on both its website³⁷ and in its Annual Report.³⁸ This has taken place since 2018. The ATO started using the methodology detailed above in 2016 and started publishing these findings in 2018.

2.70 While the Technical Guide is not publicly available, the ATO publishes a summarised version of the Technical Guide online.³⁹ The ATO has a public-facing focus on; describing what the gap is, communicating what drives the gap, and the reliability of its work.

2.71 The ATO describes the tax gap on its website by providing an overview from the 2014–15 estimate to the 2019–20 estimate. The website summary is less descriptive than the Technical Guide and consolidates some steps, but is accurate in describing the methodology.

2.72 Key drivers for the individuals not in business tax gap are communicated as work related expenses, rentals, hidden (which includes hidden or undeclared income) and ‘other’, depicted in Figure 2.1. The ‘other’ category is not described.⁴⁰ The ATO caveats its work through the reliability assessment and support of the Expert Panel.

Figure 2.1: Drivers of the Individuals Not in Business Tax Gap, 2019–20

Source: ATO documentation.

2.73 The ATO applies reliability assessments to its tax gap estimates to support internal scrutiny of the methodology and of the level of confidence in the accuracy of the estimates themselves (see from paragraph 2.78). Beyond publishing a reliability score of up to 30 and the rating scale ranging from very low to very high, the assessment process and the Expert Panel review process are not communicated to the public. The reliability assessment criteria are not available through the individuals not in business tax gap webpage, though it is discoverable on the ATO website when the search function is used. The ATO publishes information on previous estimates.⁴¹ Tax gap data for the five years prior to the reporting year is published on the ATO website (see, for example Table 1.1). Reliability scores for prior tax gap releases are not available on the same webpage to enable the reader to determine the reliability of previous years estimates.⁴²

2.74 HMRC’s tax gap is an official statistic and the estimate is produced in line with the UK Code of Practice for Statistics, which states: ‘there is no point in publishing statistics that do not command confidence.’⁴³ The ATO did not publish the tax gap until it was considered ‘highly reliable’ in the 2015–16 tax year. This estimate was published in 2018. However, the data for earlier years, where the reliability of the estimate was assessed by the ATO to be lower, is available to the public without any context to explain its lower reliability. Publication of previous reliability assessments would increase understanding of how reliability has changed as the measurement process has developed.

2.75 The ATO’s 2021–22 Annual Report outlines its intent to improve transparency as a part of the strategic objective.⁴⁴ The publication of the tax gap estimate is important for transparency. The information which has been excluded, particularly relating to the trustworthiness of the estimate, may result in the public not having a full understanding of the reliability of the gap estimate.

Reliability Assessments

2.78 The reliability assessment process is a self-assessment framework used by ATO to examine its framework, methodology and internal processes, and to determine and communicate the credibility of its gap estimate to the public. In 2016, the ATO implemented ten measures which form the reliability assessment criteria.

2.79 The 10 criteria used in the current reliability assessment are depicted in Table 2.4.

Table 2.4: Reliability assessment criteria

Source: ATO documentation.

2.80 The reliability assessment criteria were established by the ATO, drawing on its own expertise and the IMF’s Assessment of HMRC’s Tax Gap Analysis. The ATO sought the Expert Panel’s feedback⁴⁵ on the reliability assessment criteria.

2.81 Criteria one to four have been specifically sourced from the criteria developed by the IMF for its review of HRMC Tax Gap Analysis. Criteria five reflects a condensed version of the IMF criteria while maintaining its intention. Criteria six to ten were designed by the ATO and reviewed by the Expert Panel. Based on feedback from the Expert Panel in September of 2018, changes were made to the reliability assessment criteria.

2.82 The rating scale for the ATO’s assessment reflects the assessment used in the IMF’s review. The IMF review focused on communicating effectiveness and reliability while the ATOs objective is to rate and track its reliability. To do this, the ATO represents its reliability on a zero to thirty scale.

2.83 The ATO’s Reliability Assessment Guide explains: ‘the assignment of scores against criterion is carried out in a thorough and collaborative approach between business lines, the Tax Gap Expert Panel, and the ATO Executive.’ The ATO advised in May 2022, ‘[t]he reliability ratings are a judgment based assessment, conducted by the tax gap team and recorded in the Technical Guide. Typically it is a conversation held with the EL1 stream lead, supporting Analysts and the Director.’

2.84 Meetings where the reliability assessment score is assigned are not minuted, no agendas are prepared and, as a result, there is no evidence of meetings occurring. While there is a methodology in place, as a consequence of this lack of documentation, there is no evidence to demonstrate how a reliability assessment score has been determined, or whether key assumptions (see paragraph 2.61) have been properly reviewed.

2.87 The Tax Gap Expert Panel Charter states the ‘program team will look to the expert panel for assessment and, as appropriate, endorsement.’ Endorsements should be recorded in the relevant meeting minutes and the technical guide. Endorsements are recorded in Table 2.5. The panel does not have any role in the allocation of ratings.

Table 2.5: Reliability ratings, allocation and endorsement 2013–14 to 2019–20

Source: ANAO analysis.

2.88 There is no evidence of the ATO executive explicitly endorsing the outcome of the reliability assessment process, however, the reliability assessment is included in the ATO Annual Report, which is signed off by the ATO Executive.

3.1 The ATO’s Corporate Plan 2022–23 identifies the tax gap as part of a strategic objective: ‘We build community confidence by sustainably reducing the tax gap and providing assurance across the tax, superannuation and registry systems.’⁴⁶ This requires the ATO to develop and implement compliance strategies to address identified tax risks to reducing the tax gap.

Does the ATO have sound processes for identifying and prioritising tax risks?

Processes for identifying and prioritising tax risks at the enterprise level

3.2 The two documents that govern the operation of the ATO’s risk management framework are the ATO’s Risk Management Guide, and the ATO’s Enterprise Risk Management Framework (ERMF). They constitute the basis for the way ATO staff should seek to identify and address risks.

3.3 The ATO’s Risk Management Guide states that the ‘[ERMF] and risk management processes apply to all levels of risk across the ATO.’ The ATO’s Risk Management Process is based on ISO 31000:2018⁴⁷, outlining the cyclical nature of the process, and is depicted in Figure 3.1.

Figure 3.1: The ATO’s Risk Management Process

Source: ATO documentation.

3.4 This framework states that the risk methodology is ‘the practical tool that should be used in all planning activities to ensure a positive and proactive approach to risk management is applied.’ The Risk Management Guide also notes that prioritisation of risk is an integral part of the risk assessment process.

3.5 The ATO’s Risk Management Guide requires staff to enter enterprise level risks (and business level risks where appropriate) on its Enterprise Risk Register (ERR). Further, the ERMF states: ‘The ATO will regularly monitor risk and adjust its strategies, goals and activities to reflect changing priorities resulting from emerging risks and the changing environment.’

3.6 When a newly developing or changing risk that may have a major impact on the ATO is identified, the ATO’s Risk Management Guide says that new or evolving risks ‘…can be entered into the enterprise risk register as emerging risks and updated over time…’ Four emerging risk assessments relating to individuals not in business were conducted as a result of the COVID-19 pandemic.

3.7 The ATO’s Risk Management Guide notes:

The purpose of monitoring and review is to assure and improve the quality and effectiveness of process, design, implementation and outcomes. In monitoring and reviewing, it is necessary to:

• review risk ratings whenever there are changes in the environment
• monitor risks through evaluation of the effectiveness of existing controls and treatment plans and ongoing observations of the risk environment.

3.8 Further information on monitoring and review is available from paragraph 4.12.

3.9 Guidance on risk management is made available to staff through internal risk management training courses, and risk management templates are provided on the ATO Intranet.

Oversight of enterprise level risks

3.10 Figure 3.2 illustrates the ATO’s governance structure around risk. Client Engagement Group Services is the group relevant to Individuals and Intermediaries.

Figure 3.2: Risk Governance Structure

Source: ATO’s Enterprise Risk Management Framework.

3.11 The ERMF outlines the ATO’s hierarchy of risks, those with primary involvement in the risk management process, and identifies risk owners:

• Strategic Risk — Accountability for strategic risks may be assigned to individual ATO Executive members;
• Enterprise Risk — Owned and managed at the SES level and overseen by the Enterprise Risk Management Committee; and
• Business Risk — ‘Owned and managed by a Band 1, EL2 or EL1 level staff member’.

3.12 The ATO’s Risk Management Guide addresses the issue of endorsement of risks: ‘The date of endorsement entered on the ERR is the date that the risk was endorsed by the risk owner. Evidence of risk endorsement may be attached to the ERR or retained within the business area.’ The ATO advised in September 2022 that its practice is to endorse risks through the approval of risk assessment documents.

3.13 Analysis of the six risk assessment documents prepared in respect of individuals not in business since 2018 indicated that all six were not endorsed by the risk owner, though five were endorsed instead by the risk manager or the risk steward, with the ATO considering the sixth to be ‘treated as endorsed’ by the Individuals Risk Sub-Committee. ATO documentation occasionally displayed confusion about who the owner of a risk actually was, incorrectly listing the person who was the risk manager or the risk steward in the owner’s place.

3.14 The Risk Management Guide also states that ‘Enterprise risks should be reviewed every twelve months as a minimum. Enterprise risks that are not within tolerance may need to be reviewed more frequently’. Analysis of risk review documents related to individuals not in business since 2018 indicated that the ATO reviewed risks in accordance with these required timeframes, and reviews were signed off by a risk manager or risk steward.

3.15 The ATO has also established an Individuals Risk Sub-Committee (the Sub-Committee), chaired by an Assistant Commissioner. The Sub-Committee is a strategic decision-making body for all work programs in the area of Individuals Client Experience. Its purpose is to ensure risks are effectively managed, and governance requirements are met.

3.16 The Sub-Committee can also make resource recommendations to ensure ATO investments appropriately target taxpayer behaviour risks that contribute to the individuals not in business tax gap, which are then sent to the Individuals and Intermediaries Executive Group for consideration and approval.

3.17 Analysis on the operation of the Individuals Risk Sub-Committee is available from paragraph 4.53.

Enterprise level risks relating to Individuals not in business

3.18 There are three business or enterprise level risks relating to individuals not in business that are managed by the Individuals and Intermediaries business line. These are depicted in Table 3.1.

Table 3.1: Individuals not in business risks and sub-risks

Note a: A treatment plan was not required as the risk was within tolerance.

Source: ANAO analysis of ATO documentation.

3.19 Each of the Individuals not in business risk assessments align to the ATO’s corporate plan objective to build community confidence by sustainably reducing the tax gap.

3.20 The tax gap estimate and findings from the REP are used to inform decision making around risk. The ATO stated that ‘outputs from the Tax Gap estimate include insights from the [REP] that provides opportunities to deepen our understanding of the behavioural drivers and errors that lead to adjustments made to returns’, and that ‘[t]he tax gap estimate is helpful in confirming that we have prioritised the management of certain risks such as work related expenses and net rental income as they continue to be the main risk drivers of the gap.’

3.21 For example, for the risk assessment for individuals fail to correctly report all their income, the strategy to achieve the objective to build community confidence by sustainably reducing the gap, seeks to reduce the tax gap by being streamlined, integrated, and data driven, and by improving the client experience. The insights contained in Close Out Reports (see paragraph 2.47) from previous REPs (specifically compliance behaviour, and revenue impacts) inform the content of the risk assessment.

3.22 To address the tax gap, the ATO employs compliance strategies.⁴⁸

Have compliance strategies been developed for identified tax risks?

Data and intelligence to identify tax risks

3.23 The ATO’s intelligence framework (Figure 3.3) outlines the types of information, data and analysis undertaken to inform risk assessments, risk reviews, emerging risk assessments, program logic documents, evaluations, and intelligence reports to reduce the individuals not in business tax gap. Most of the data the ATO uses is taken from lodged income tax returns, the outcomes of audits (which can examine returns lodged up to two years earlier), evaluations, and reporting.

Figure 3.3: ATO intelligence framework

Source: ATO documentation.

3.24 The ATO uses reporting and analysis from several sources to inform its work. It develops the Individuals Pipeline Report, and a Biannual Intelligence Report.

3.25 The annual Individuals Pipeline Report is ‘an end-to-end view of [the ATO’s] compliance interactions’ that seeks to ‘[identify] hotspots, [provide understanding of] drivers of client behaviour, and [to] draw attention to any unintended consequences of [ATO] strategies’, and is used to look at the impact of compliance strategies to address the individuals not in business tax gap from an end-to-end perspective.

3.26 Biannual Intelligence Reports are designed to ‘bring together insights from various sources’ to assist decision-makers ‘to improve risk detection, treatment of current and emerging risks and identify changes in the external environment.’ They outline overall performance against key risk areas, as well as key insights by risk area.

Monitoring individuals tax performance

3.27 The ATO sets thresholds at the label level in its Early Warning System (EWS).⁴⁹ The ATO’s formal definition of a label is ‘a discrete component of a tax form requiring data’. In the case of individuals not in business, a label is part of a tax return that contains data that should be declared for tax purposes.

3.28 Label level monitoring enables the ATO to monitor label movements (both volume and value) as the tax returns of individuals not in business are lodged. When a label-level threshold is exceeded, the Risk and Strategy Intelligence Director decides whether an intelligence scan (see Table 3.2) should be initiated. The ATO provided examples of EWS reporting driving further analysis through intelligence scans. An end of financial year report is also developed from EWS data, which examines key labels with significant movements and potential causes for these movements, predictions for label trends, and planned improvements for the next year.

3.29 The ATO has developed a EWS Tolerance Methodology which outlines how the ATO calculates its tolerances. This model uses several years’ worth of label-level data to develop a weekly moving tolerance. This tolerance is ‘based on a calculated trend the label is expected to be moving towards’, with the dispersion around the trend then calculated.

3.30 The ATO stated in July 2022 that this methodology:

allows us to better understand the natural movements of the labels and better understand when abnormal movements occur. It further allows us to shift our focus on labels that are out of tolerance and determine the causes. A 0.5% movement at a stagnant label is given much greater importance than a 5% movement at a variable label.

3.31 Table 3.2 outlines other monitoring sources used by the ATO.

Table 3.2: Monitoring sources

Source: ANAO analysis of ATO documentation.

Developing compliance strategies

3.32 The ATO undertakes a range of actions to address identified risks to the tax gap for individuals not in business. These may include controls, treatments, tactics, and strategies to both maintain and reduce tax risks. The ATO advised in May 2022 that activities to maintain and reduce gaps were intertwined, as prevention compliance strategies needed to be followed by strategies to ensure continued compliance. For the purposes of the audit, the term ‘compliance strategies’ has been used to encompass the range of terms used by to the ATO to address an identified tax risk. Figure 3.4 provides a visual representation of these terms.

Figure 3.4: Terms used to describe compliance strategies

Source: ATO documentation.

3.33 The ATO advised in May 2022 that the range of potential compliance strategies meant that different compliance strategies had different timeframes for implementation. For example, changing the content of written communication took less time to implement than an IT change. Data availability and quality also influence when a compliance strategy may be implemented. The ATO stated in May 2022 that, for example, while tax time commenced in July, property management data tended to come in around August to October, and that it took time to put the data into a useable format.⁵⁰ The ATO also advised that where it had higher quality data, it would intervene more quickly.

3.34 The ATO’s Measurement and Evaluation Guidelines (M and E Guidelines)⁵¹ outline the steps required to be undertaken when developing a compliance strategy to address a risk to the tax gap for individuals not in business. ATO staff are also provided with an Evaluation Checklist which outlines the 13 steps to be taken from understanding the risk through to presenting an evaluation report and feeding back any actionable intelligence insights. Steps one to six outline the development of a compliance strategy, with step seven being implementation of the compliance strategy, this is depicted in Table 3.3. Steps eight to 13 are addressed from paragraph 4.45.

Table 3.3: Compliance strategy development

Source: ATO documentation.

3.35 Where a risk to the tax gap for individuals not in business has been identified the ATO should develop a program logic, which ‘is a framework to understand how a project or strategy intends to produce specific outcomes and impacts. It maps the logical causal connections between goals, inputs, activities, outputs and outcomes.’ The program logic model is based on a model developed by the Organisation for Economic Co-operation and Development (OECD).⁵²

3.36 At a minimum, the ATO’s M and E Guidelines require staff to develop a program logic map, with a detailed program logic required for full evaluations and for major or high-profile treatments.

3.37 The ATO advised in July 2022 that program logics were a ‘crucial step in the evaluation process with each risk team having an overarching document for their area [which] covers the full gambit of their work across their risk’ and that ‘teams can create a Program Logic encompassing the different strategies/tactics for that individual risk.’

3.38 The Program Logic Model walks the user through the development of a program logic, asking questions to assist them in populating a six column Program Logic Map, which is a summary of the issue. For example, Table 3.4 outlines the program logic for the Work-Related Expenses (WRE) Risk.

3.39 Staff are guided in how to assemble a program logic through the provision of templates for program logics and program logic maps. A ‘self-paced’ training document and a guidance document are provided. Most program logics assessed by the ANAO conform to the structure required by the ATO. The ATO advised that training on program logic development took place online, generally on a team-by-team basis. The last training session was conducted in June 2022.

3.40 Regarding endorsement of program logics, the ATO’s M and E Guidelines state:

A treatment program logic map should be endorsed by the EL2 Risk Director. Overarching program logics for risk types should be endorsed by the Senior Risk Director, and the Assistant Commissioner should be informed.

3.41 Assessment of program logic documents indicated that there was no field for the document to be endorsed. The ATO provided several examples of program logics being endorsed by risk directors via email, and undertook to update templates to ensure endorsement is easier to identify in the future.

3.43 Following the development of a program logic, an appropriate methodology for compliance strategy evaluation should be developed, and an M and E Plan should be produced. Further information on M and E Plans is available from paragraph 4.12.

Register of compliance strategies

3.44 ATO guidance requires compliance strategies to address the tax gap for individuals not in business to be recorded on the Treatment Work Register (TWR).⁵³ The ATO’s M and E Guidelines require the development and review of a program logic and/or a program logic map (see paragraph 3.36), an M and E plan and evaluation/review. The key documents for each compliance strategy have not always been developed in a timely manner as outlined in ATO guidance or, in some cases, have not been developed at all.

3.45 The ATO’s M and E Guidelines require all plans to, at a minimum, be reviewed and endorsed by an evaluation manager. Of the 95 active and archived compliance strategies as at 11 November 2022, only 67 linked an M and E plan. 25, (or 37 per cent), of these M and E plans have no endorsements.

3.46 Of the 95 compliance strategies on the register, 10 did not contain a program logic or program logic map, which is not compliant with the M and E Guidelines.

3.47 Prior to 2022, the ATO used a different evaluation framework. The ATO stated in May 2022 that its evaluation framework is ‘fairly new’ meaning there have been some delays in the development of documents and migration into the TWR. The M and E Guidelines emphasise the need to develop and upload compliance strategy documentation as soon as possible.

Have selected compliance strategies been implemented as intended?

Selection of compliance strategies for testing

3.48 Thirteen individual compliance strategies were selected to assess whether compliance strategies to address the tax gap for individuals not in business were being implemented as intended. The selected compliance strategies were:

• Bulk Communications — 2021;
• Bulk Communications — 2022;
• Tax Agent Strategy (TAS) Deduction Prompter Campaign;
• Tax Agent Strategy (TAS) Monitoring Tax Time [TT] 2022;
• Rental Deductions Prompter 2020;
• Operational Analytics Rentals — Travel;
• Operational Analytics Rentals — Borrowing Expenses;
• Operational Analytics Rentals — Vacant Land;
• Cryptocurrency Discrepant Prompter — 2020;
• Cryptocurrency Discrepant Prompter — 2021;
• Cryptocurrency Discrepant Prompter — 2022;
• Refund Fraud & Assurance Expert Business Rules (EBRs) — excluding label 31b; and
• Omitted Income and Offsets (OIO) WinCAS Amendment — Employment.

3.49 An outline of these selected compliance strategies and the selection process is at Appendix 5.

Testing of selected compliance strategies

3.50 The M and E Guidelines were reviewed to determine the key documents required to develop and assess a compliance strategy. The selected compliance strategies were examined to determine whether the ATO effectively implemented its compliance strategy framework as intended. The findings of this analysis are contained in Table 3.5.

3.51 Within the selection of compliance strategies, there are some instances where the program logic bears little relevance to the compliance strategy. Where maps have not been developed for the compliance strategy, the ATO generally refers to an overarching map for the risk. This is conditionally permitted within the M and E requirements:

A program logic map is required for each risk’s different treatments (e.g. prompter letters, advisory letters, audits, etc.) and should be prepared at the outset of treatment development. If preferred, and where appropriate, similar treatments can be rolled-up into a single program logic map (risk team discretion).

3.52 There should be a unique program logic which addresses a risk’s treatment type. Often an overarching program logic refers only to the risk and will be applied to multiple treatment types. Three selected compliance strategies did not contain program logics or maps. This is depicted in Table 3.5.

3.53 The M and E Guidelines require an M and E Plan to be in place to enable a review or an evaluation: ‘[An M and E] plan is required for all evaluations and reviews. The plan can be derived from the relevant risks over-arching [M and E] plan and the treatment’s program.’ The ATO did not develop M and E Plans for all compliance strategies on the TWR. Of the 13 compliance strategies selected for review, nine included an M and E Plan as indicated in Table 3.5.⁵⁴ Without M and E Plans, the ATO is unable to outline goals, targets, resource needs and timelines for projects treating identified tax risks.

3.54 The ATO’s performance against document requirements for compliance strategies is also depicted in Table 3.5.

ATO implementation of selected compliance strategies

3.55 To establish whether the selected compliance strategies were being implemented in accordance with guidelines, the available program logics and program logic maps for the 10 of the 13 selected compliance strategies that contained an overarching or specific program logic were reviewed.⁵⁵

3.56 To assess the implementation of selected compliance strategies, an assessment of implementation against the program logic columns ‘inputs’ and ‘activities’ was conducted.

3.57 The inputs column of the program logic map seeks to outline what teams will be resourced, what data is utilised and what systems are applied.

3.58 The activities column of the program logic map seeks to outline some of the key steps undertaken by the ATO to implement and complete a compliance strategy. The level of detail provided for activities for selected compliance strategies ranges from a few general points to detailed descriptions of up to 12 steps to be undertaken.

3.59 There are some inconsistencies in document management and intended purpose of the evaluation framework between different risk groups responsible for their respective compliance strategies. For example, the Bulk Communications program logic was updated between the 2021 and 2022 compliance strategies, though, in the case of the three Cryptocurrency Prompter campaigns, program logics were not updated year-to-year to reflect on or implement lessons learnt.

3.60 Table 3.6 depicts how successful the ATO was in the implementation of inputs and activities.

Table 3.6: Assessment of the ATO’s implementation of compliance strategies

Note a: Operational Analytics Rentals programs were not developed in accordance with ATO guidance and, at the time that testing of selected compliance strategies commenced, a program logic had not been developed or uploaded to the Treatment Work Register.

Source: ANAO Analysis

3.61 It is not always possible to determine whether achievement against the ATO’s program logic can be demonstrated, as the maps are often not specific to the compliance strategy being addressed. The ATO conducted a self-assessment of its evaluation framework in May 2022, and the findings of this were consistent, stating: ‘ATO evaluation product suite exists but is not consistently applied.’ The program logics of the selected compliance strategies do not adequately describe actions to be taken. The framework outlines valuable information, but the implementation of this framework requires improvement to ensure planned activities are undertaken to assure ATO compliance strategies have an impact on the individuals tax gap.

Consistency between Program Logics and M and E Plans

3.62 As noted in paragraph 3.43, after a program logic is developed, an M and E Plan is produced. The M and E Plan Detailed Guide notes:

While the Program Logic document stands on its own, the [M and E Plan] stands together with the Program Logic document. The Program Logic informs the M and E Plan of the outcomes for measurement in determining if the strategies and tactics in use are effective in achieving the outcomes sought.

3.63 Of the 13 compliance strategies selected, four did not have a M and E Plan which is required, as outlined in ATO guidance, at the beginning of a program.

[An M and E Plan] should embed into strategy development before the actual implementation of activities; this will allow the evaluator to determine the data required to answer the agreed evaluation questions.

3.64 As a way of gaining assurance that the selected Program Logics informed the M and E Plans of outcomes for measurement, the content of the Outcomes column in each Program Logic were compared with the content of the related M and E Plan for each of the selected compliance strategies.

3.65 Inconsistencies were found across compliance strategies — some are fully aligned, while others vary. This is depicted in Table 3.7.

3.66 In some cases, M and E Plans do not adequately describe the content of program logics. For example, in the rental deductions compliance strategy, only two of the 11 desired outcomes in the program logic were communicated in their M and E plan.

Table 3.7: Assessment of ATO Program Logic desired outcomes being communicated in M and E Plans

Note a: Operational Analytics Rentals programs are not compliant with ATO guidance and a program logic and an M and E plan was not developed or uploaded to the Treatment Work Register.

Note b: Refund Fraud & Assurance Expert Business Rules (EBRs) — excluding 31b was not compliant with ATO guidance and an M and E plan was not developed.

Source: ANAO analysis.

Setting Targets and Benchmarks

3.67 Following a compliance strategy’s implementation, the M and E Plan Detailed Guide outlines the purpose and nature of the formal evaluation of the compliance strategy:

the evaluation should:

• be outcomes-focussed (i.e. what are the immediate outcomes we observe such as changes in attitudes, perceptions, and short term compliance behaviour?)
• note the relevant periods/years for evaluation
• seek to assess whether and the extent to which the strategy is achieving its goals, objectives, and its intended outcomes.

3.68 Analysis of selected M and E Plans enabled the ATO to assess ‘whether and the extent to which the strategy is achieving its goals, objectives, and intended outcomes’ was undertaken, with analysis split into establishing if the ATO could determine:

• whether a strategy was achieving its goals; and
• the extent to which a strategy was achieving its goals.

3.69 To answer if an evaluation could determine whether the strategy was achieving its goals, objectives and intended outcomes, analysis was undertaken to determine whether compliance strategy indicators logically flowed from the Key Evaluation Questions (KEQs).

3.70 For the nine selected compliance strategies with M and E Plans, the ATO has been largely successful in outlining measurable goals through its KEQs as depicted in the middle column of Table 3.8.

3.71 To answer whether the ATO could determine the extent to which a strategy was achieving its goals, another element of ATO guidance should be considered. The ‘Targets/benchmarks’ section of the Individuals and Intermediaries (IAI) ME Detailed Guide states:

What constitutes ‘success’? What should be the criteria, standards and/or targets/benchmarks for judging performance of the strategy?

Criteria refer to aspects of the strategy that are important to consider when deciding whether, and in what ways, it is a success or a failure, or when producing an overall judgment of performance.

Targets are particularly useful when there is a clear expectation of the standard to which a purpose fulfils (or an activity is undertaken), and comparison against such targets provides a clear and unambiguous result. They should reflect internal performance expectations, while also being informed by external benchmarks or trends from prior periods or activities. Baseline results, achieved before the activity or intervention begins, are a useful reference point…

If a target cannot establish or if the strategy is unable to establish baseline data at the outset of the work, explicit timelines for when these will establish as well as who is responsible for establishing them should be stated.

3.72 To answer if an evaluation could determine the extent to which a compliance strategy was achieving its goals, objectives, and intended outcomes, the KEQs were examined to determine whether targets or benchmarks were identified.

3.73 It was found that M and E Plans did not include targets or benchmarks based on previous year data as depicted in Table 3.8.

Table 3.8: Assessment of the ATO’s goals and targets

Note a: Operational Analytics Rentals programs and the Refund Fraud & Assurance program are not compliant with ATO guidance and an M and E Plan was not developed or uploaded to the Treatment Work Register.

Source: ANAO analysis.

3.74 While the ATO outlined some measurable goals in M and E Plans, these goals are usually⁵⁶ written to require a target, prompting the author to consider a percentage or dollar value. For example: ‘$ and % of total adjustments…’ or ‘# and % of clients who amend their target income tax returns…’ Rather than committing to a target, the indicator of success was left undetermined on 67 out of 68 occasions⁵⁷, meaning the ATO has not defined what would constitute success. Further, benchmarks have not been used in M and E Plans, making it difficult to drive improvement over time.

Assessing Outcomes

3.75 The ATO evaluates (or reviews) all finalised compliance strategies. Of the six finalised strategies evaluated, five were considered successful with the ATO stating that one ‘may not be a traditionally successful campaign’ in its evaluation document. This strategy was documented as not effective on the ATO’s TWR.

3.76 While the ATO considers what should be measured in its M and E Plans, almost always outlining instructions to calculate the percentage changed or dollar-value raised as a result of a compliance strategy, by not setting a target, the ATO is unable to define success. Without the inclusion of measurable goals or benchmarks, the ATO is unable to adequately evaluate the implementation of its compliance strategies.

4.1 Element nine of the Commonwealth Risk Management Policy provides that effective risk management ‘is a process of continuous improvement, requiring regular review and evaluation mechanisms.’⁵⁸

4.2 Step four of the ATO’s Enterprise Risk Management Framework: ‘How do we know the strategies are working and the risks are being managed effectively?’ seeks to address this element. It states that ‘assessing strategies and risk management activities helps us determine their effectiveness and enables us to make adjustments as needed so we stay on track to achieve the objectives.’

4.3 To effectively assess compliance strategies for reducing the individuals not in business tax gap, the ATO requires relevant data, and to assess and report its compliance strategies in a timely manner to inform subsequent activities.

Does the ATO use relevant data and intelligence for assessing effectiveness?

Use of Randomised Control Trials

4.4 The Measurement and Evaluation Guidelines (M and E Guidelines) embed the use of data in the evaluation process in the planning stage. The M and E Guidelines state that ‘evaluations and the use of control groups must be planned for at the outset of [compliance strategy] development or planning.’ The Risk and Strategy – Individuals – Evaluation Checklist requires the evaluation officer within the risk team to determine whether a randomised control trial (RCT) is an appropriate evaluation methodology to use for the compliance strategy. If it is determined to be appropriate, staff are required to use the RCT Decision Tool to calculate the size of the control group and determine the level of defensibility of the evaluation or review.

4.5 If a RCT is not determined to be appropriate, the evaluation officer must contact the Intelligence, Monitoring, Evaluation and Strategy Team for further advice, and ‘alternative defensible methods’ are considered.

4.6 The ATO advised in November 2022 that Operational Analytics and Expert Business Rules models, compliance strategies with small candidate pools, taxpayer behaviour considered ‘sufficiently egregious’ to require immediate treatment, and compliance strategies with complex and multi-faceted candidate pools were generally unsuitable for RCTs.

Sources of data and intelligence

4.7 The main sources of data and intelligence, analysis tools and resources used by the ATO to assess the effectiveness of compliance strategies to reduce the tax gap for individuals not in business are listed in Table 4.1.

Table 4.1: Data and intelligence sources, analysis tools and resources

Source: ATO documentation.

4.8 The ATO advised in August 2022 that other sources ‘that may occasionally be used’ include:

• intelligence capture forms to be completed by auditors when completing audits;
• informal intelligence received from auditors and others by email;
• complaints;
• communications team with information from the online ATO Community; and
• tip-offs and referrals through the Tax Integrity Centre and ATO Intelligence.

4.9 The ATO stated in December 2022 that it assures itself organisationally that its data and intelligence gathering services are fit-for-purpose by setting out guidance for staff in Chief Executive Instructions about principles, roles, and responsibilities.⁵⁹

4.10 The provision and curation of data involves Smarter Data providing data outputs based on the Target Selection Rationale (TSR) statement which is provided by the Individuals and Intermediaries business line.

4.11 Data provided by Smarter Data (see Table 4.1) is reviewed and tested by the business line to ensure the intent of the TSR is being met. This review and testing process involves checks back to core ATO systems to confirm the accuracy of information and to ensure alignment with the TSR.

Are compliance strategies assessed on a timely basis?

Reviewing and evaluating

4.12 The M and E Guidelines provide direction on the measurement and evaluation of compliance strategies to reduce the tax gap for individuals not in business: ‘All risk types should have an over-arching M and E Plan that explains how the effectiveness of risk management will be determined. This will be derived from that risk’s over-arching program logic.’

4.13 The M and E Guidelines require all business as usual compliance strategies to be ‘evaluated at least once to provide a baseline evaluation for future comparison.’ It also permits an evaluation to assess multiple related compliance strategies.

4.14 The ATO advised in November 2022 that evaluations of compliance strategies to maintain and reduce the individuals tax gap were generally undertaken by the team responsible for the risk, or on some occasions, a specialist evaluation team if an evaluation sits across multiple risks, requires a complex evaluation method to be applied, or if the evaluation involves newly acquired third party data.

4.15 The ATO advised in July 2022 that the specialist evaluation team oversees the evaluative work undertaken by risk teams, performing an assurance and governance role, with risk teams ‘[engaging] with the [specialist team] as necessary throughout the evaluative process.’

4.16 The specialist team is also able to draw on the expertise of the Client Evaluation Group (CEG) Evaluation Hub. The CEG Evaluation Hub was established in mid-2021 with a focus on ‘building the ATO’s evaluation culture, capacity and practice.’

4.17 The CEG Evaluation Hub developed an Evaluation Maturity Model which was used by Individuals Risk and Strategy staff to develop an evaluation roadmap which visually represented the maturity levels of inputs to evaluation. It tracked the current state of the individuals not in business evaluation function with a view to increasing maturity across five possible levels. The roadmap concluded that the ATO had fully achieved the first maturity level and was more than half way through implementing required changes to achieve the second maturity level.

4.18 In the case of reviews, the M and E Guidelines state that a ‘one-off review’ needs to be taken, but that ‘more frequent evaluation and review of [compliance strategies] can be undertaken where considered necessary [by the responsible officer] and resources permit.’

4.19 Staff are provided with templates to prepare reviews, and standard and detailed evaluations.

4.20 Prior to May 2020 the ATO reported outputs such as strike rates and audit yields⁶⁰, comparing them to prior years and trends. The ATO stated in July 2022 that ‘there was no analysis comparing the outcomes to a control group to determine causation effectiveness, showing that what occurred was because of the treatments and the treatments alone.’

Timelines for review and evaluation

4.21 The M and E Guidelines do not specify timelines for reviews or evaluation of strategies to reduce the tax gap for individuals not in business, however, the M and E Detailed Guide encourages staff to consider when they expect to see the compliance strategy start to show change, when the full effects are expected, and how often an evaluation will need to be conducted to establish sustained change.

4.22 The M and E Guidelines state:

All [BAU compliance strategies] must be evaluated at least once to provide a baseline evaluation for future comparison. Baseline evaluations for existing BAU treatments (i.e., currently in use projects, not legacy projects) should, where possible, be completed by 31 December 2022. We aim for at least 50% of baseline evaluations of 2021–22 treatment year projects (prioritised by the risk teams) to be completed by this date.⁶¹

4.23 To meet this objective, the ATO developed a plan on which evaluations and reviews should be undertaken and completed by 31 December 2022, outlining a timeframe and the reason for evaluation or review.

4.24 As at 6 December 2022, the ATO identified 176 compliance strategies as the population from which it would evaluate at least 50 per cent by 31 December 2022. The ATO advised on 2 January 2023 it had evaluated 82 of the identified compliance strategies (47 per cent). As at 9 January 2023, 54 evaluations were available on the TWR.⁶²

Analysis of selected evaluation timelines

4.25 M and E Plans for compliance strategies generally provide a timeline for program commencement, data collection and evaluation. A review of selected compliance strategies indicated that the ATO is inconsistent in meeting outlined evaluation timelines. Of the 13 selected compliance strategies, five were evaluated in the outlined timeline, four were not evaluated in the timeline and another four did not have M and E plans. This is depicted in Table 4.2.

Table 4.2: Timeliness of selected compliance strategy evaluations

Note a: Operational Analytics Rentals programs and the Refund Fraud & Assurance program are not compliant with ATO guidance and an M and E Plan was not developed or uploaded to the TWR.

Source: ANAO analysis of ATO documentation.

4.26 Parameters around evaluation dates and timelines to ensure compliance strategies are reviewed or evaluated in a meaningful timeframe would improve the overall treatment of risks. This is supported by ATO guidance, which states ‘[t]he evaluation or review should be completed as soon as possible after it becomes due, or the need is identified’.

4.27 In the case of the cryptocurrency prompter for 2020 and 2021, analysis indicates that data has not been evaluated in a timely manner, leading to an additional lag in the ATO implementing improvement, and informing strategy.

4.28 As a part of the Cryptocurrency Prompter 2020 program the ATO sent emails in November 2020 (see Figure 4.1). The evaluation was undertaken in June of 2021 and endorsed by a Risk Manager in August of 2021. The evaluation did not receive final endorsement from the relevant Director until after the emails for the Cryptocurrency Prompter campaigns for 2021 and 2022 were sent in March 2021 and February 2022 respectively. This meant that potential lessons learned from the 2020 compliance strategy were not able to be used in subsequent compliance strategies. The evaluation of the 2020 compliance strategy concluded it ‘may not be a traditionally successful campaign’.

4.29 The ATO advised in October 2022 that, for prompters, it ‘will look at the short term results after 90 days’. In the case of the 2020 Cryptocurrency Prompter, the first evidence of review was seven months later (see Figure 4.1). The evaluation was not initiated in accordance with the M and E Guidelines which requires that compliance strategies ‘be completed as soon as possible after it becomes due, or the need is identified.’ The ATO explained ‘it can take up to [two] years to properly evaluate a [compliance strategy].’ This timeline includes waiting for the taxpayer to submit their tax return in the following year, though this is required for compliance strategies which do not compare taxpayer claim year to year. The cryptocurrency prompter does not compare annual taxpayer claims. The evaluation for the Cryptocurrency Prompter 2020 program depicted in Figure 4.1 was finalised and endorsed after 16 months in March 2022. The campaign’s M and E plan indicates that the evaluation was due in April of 2021.

4.30 The Commonwealth Risk Management Policy states ‘[t]he effective management of risk is a process of continuous improvement, requiring regular review and evaluation mechanisms.’

4.31 There is no ATO guidance to ensure a compliance strategy for individuals not in business has been reviewed or evaluated before the next years compliance strategy commences to drive improvement. In the instance of the Cryptocurrency Prompter 2020, this meant the ATO was unable to use lessons learned to inform the two subsequent strategies.

Monitoring of compliance strategies

4.32 The ATO monitors the day-to-day performance of compliance strategies through the IAI Business Performance Dashboard. The dashboard enables staff to monitor the performance of key risk areas, enabling comparison of performance between anticipated and actual figures. The ATO advised in January 2023 that this tool was based on unassured data and was used to informally monitor performance.

4.33 The ATO uses weekly Pulse reports and fortnightly Heartbeat meetings to more formally monitor performance. Pulse reports ‘provide performance data to support our conversations in the Heartbeat meeting about delivery planning and decisions.’ The ATO advised in January 2023 that Pulse reports provided data, and not explanations as to why things had happened. Pulse reports include data on year to date planned versus actual results. Pulse reports are ‘drawn from unassured data.’ Assured data is then used to generate monthly reporting.

4.34 In January 2023 the ATO stated that the purpose of Heartbeat meetings was to ‘[focus] on issues impacting on the delivery of our audit programs and implementing team-level corrective action. Examples include shifting staff between programs, implementing process improvements and identifying new and emerging risks.’ The ATO advised in July 2022 that Heartbeat meetings ‘involve the Assistant Commissioners from the Risk and Strategy and Engagement and Assurance areas, along with their directors.’ The ATO was unable to provide meeting agendas or minutes of Heartbeat meetings, but provided meeting invitations. Although a brief agenda was attached to the meeting invitations, the ATO was unable to evidence the attendees at any recent Heartbeat meetings.

4.35 The ATO provided five Pulse reports. Two of these contained a list of six action items but neither contained information on who was responsible for the action item, or the due date for the action item. They stated: ‘[p]lease note that details such as responsible person and completion due dates were not captured in our last Heartbeat meeting and so could not be included in this report. This information will be included in future reports.’ The next Pulse report indicated that four of the six action items now included the required information.

4.36 The inadequate level of information around agenda items in the Pulse reports appears to be a direct consequence of the lack of minutes for Heartbeat meetings.

4.38 Tracking strike rates and average treatment effects assists the ATO in seeing the extent to which a compliance strategy has impacted a population for a selected period. Without a target or benchmark (see paragraph 3.77), it may be difficult for ATO officers to assess whether a compliance strategy is on track. The ATO does not track revenue for certain compliance strategies, including bulk communications, waiting until the next lodgment period has ended to compare the control and treatment groups.

Retaining lessons learned

4.39 The ATO has 461 archived compliance strategies for individuals and intermediaries. A compliance strategy may be archived due to emerging unintended outcomes or superseded by new methods to treat the risk.

4.40 The TWR now archives ceased activities that have been reviewed or evaluated, though, this has been newly implemented and only contains recently archived and ceased activities. Until early 2022, there was no method for archiving compliance strategies, meaning older strategies were dispersed across the risk group’s share drives which are inconsistently managed.

4.41 The ANAO conducted analysis into multiple risk group’s share drives. The ATO’s share drives have inconsistent filing structures and document names. There is no evidence of ceased activities being stored or documentation of lesson learned.

4.42 The ATO records some ceased activities in the biannual Risk Stories document. For each key risk⁶³, the responsible group reflects on the past six months, noting success and challenges and, among other things, responds to ‘what activities have we stopped?’ Sometimes the ATO will refer to finalising activities for the financial year or ceasing activities permanently. The description of ceased activities does not provide reasoning, evidence or lessons learned.

Does the ATO report on the effectiveness of its compliance strategies?

4.44 The ATO’s Risk Management Guide notes the importance of recording and reporting in the overall risk management process:

The purpose of recording and reporting in the risk management process is to:

• communicate risk management activities and outcomes across the organisation
• provide information for decision-making
• improve risk management activities
• assist interaction with stakeholders, including those with responsibility and accountability for risk management activities.

Any outcomes, findings and any other information obtained in performing the different steps of the risk management process (e.g. risk assessment, risk treatment) should be documented and reported through appropriate mechanisms.

4.45 Steps one to seven in the Evaluation Checklist were outlined from paragraph 3.34. Steps eight to 13, depicted in Table 4.3, outline how a compliance strategy should be monitored and evaluated, with outcomes communicated.

Table 4.3: Compliance strategy monitoring and evaluation

Source: ATO documentation.

Operational, managerial and stage gate reporting

4.46 Operational and stage gate reporting is ‘progress reporting in nature, normally required of larger projects as they proceed.’ The ATO was unable to provide examples of this report, explaining, ‘[i]n most cases, these reports are not required for the strategies and tactics piloted and run within IAI Risk and Strategy.’

Evaluation and review reports

4.47 Prior to the implementation of the new evaluation framework, the ATO advised in July 2022 that it analysed outputs such as strike rates, audit yields, and raised liabilities, and ‘reported outputs comparing them to prior years and trends.’ The ATO acknowledged that under this framework, there was ‘no analysis comparing the outcomes to a control group to determine causation effectiveness.’ The ATO reports that its new framework, which commenced at the beginning of 2022, aims to measure effectiveness through the use of control and comparison groups generally through the use of RCTs (see from paragraph 4.4).

4.48 The M and E Guidelines state that for each evaluation or review ‘[a] standard evaluation or review report …should be completed.’ For full evaluations, ‘[a] detailed evaluation report (in addition to the standard evaluation or review report)’ should also be completed.⁶⁴ The ATO usually uses or follows templates to create these reports.

4.49 The ANAO analysed all compliance strategies with the status of ‘Finalised’ in the TWR as at 30 November 2022. This analysis confirmed that all compliance strategies which required a detailed report had one uploaded to the TWR, and that all other compliance strategies had either a standard or detailed report uploaded to the TWR.

4.50 The M and E Guidelines note that evaluation and review reports require endorsement by a Risk Director. The ANAO analysed all compliance strategies with the status of ‘Finalised’ in the TWR as at 30 November 2022 that had a detailed report uploaded to the TWR, finding evidence that 11 out of 17 of the reports had been appropriately endorsed. The ATO subsequently advised in March 2023 that it considers the uploading of a report to the TWR as sufficient confirmation that it has been suitably endorsed.

Presentation of evaluation and review reports

4.51 The Evaluation Checklist requires the final evaluation report to be presented ‘to the relevant stakeholders.’ Presentation may be a formal presentation to the Senior Leadership Team (SLT)⁶⁵ and/or the Risk Sub-Committee.

4.52 Members of the SLT manage identified risks in the IAI business line. The ATO stated in November 2022 that risk teams present compliance strategy findings at SLT Meetings and Intelligence Debriefs (see paragraph 4.58). When evaluations are presented, the SLT may discuss opportunities for improvement to be incorporated into future compliance strategies. One example of this occurring was provided.

4.53 The Individuals Risk Sub-Committee co-chaired by the Assistant Commissioner of IAI (see also from paragraph 3.15) is a strategic decision-making body for individuals not in business. Its purpose is to ensure individuals tax gap risks are being adequately managed.

4.54 The Risk Sub-Committee is presented with the findings of evaluations and reviews through a risk call-over process. Risk call-overs are conducted biannually for each risk to the individuals tax gap according to an established work program. A presentation is provided on the overall risk and a document is provided that includes a section on measurement and evaluation of compliance strategies to reduce the Individuals tax gap.

4.55 The Risk Sub-Committee reports to the IAI Executive Group (see Figure 3.2). An overview of the Risk Sub-Committee meeting is presented to the IAI Executive Group. The outcomes of risk call-overs are reported, generally focusing on the resourcing required to address risks to the individuals tax gap.

High-level measures of effectiveness

Corporate measures

4.56 Table 4.4 outlines the corporate revenue measures the ATO reports, which ‘provide insights into the health of the tax system and the effectiveness of our administration.’ Aggregated figures for tax assured and Total Revenue Effects are reported on the ATO website and in its Annual Report.

Table 4.4: Corporate revenue measures

Note a: Australian Taxation Office, Overview, available from https://www.ato.gov.au/About-ATO/Research-and-statistics/In-detail/Tax-… [accessed 6 December 2022].

Source: ATO documentation.

Performance report

4.57 The ATO produces a Performance Report Summary on an annual basis, which reports against the four pillars of compliance: registration; lodgment; reporting and payment (which includes debt). It does not focus on the effects of individual compliance strategies, instead its purpose is to ‘[provide] a point of time snapshot of performance in the individuals not in business client experience.’ The last report was released in January of 2022 Reports are stored on the ATO intranet but do not appear to move beyond the Individuals area of the ATO.

Intelligence feedback

4.58 The ATO schedules intelligence debriefs, described as a:

Formal process which brings together staff from a number of areas including engagement and Assurance, Risk [and] Strategy, Review and Dispute Resolution (Objections) and Technical Leadership and Advice. The objective is to share experience, knowledge, insights and tap into the individual and collective expertise to collect, store and analyse information.

4.59 The ATO holds debrief meetings for the following risks: omitted income and health; internationals; investor portfolio; work related expenses; and refund integrity. The purpose of these meetings is not to assess outcomes but to share actionable insights from compliance strategy reviews and evaluations.

4.60 To ensure intelligence is provided at the executive level, the ATO provides monthly Executive reports to the Assistant Commissioner of Risk and Strategy which offer a ‘high level view of the activities undertaken with each portfolio’.

Appendix 1 Australian Taxation Office response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021–22 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• The ATO made continuous additions to the Treatment Work Register, reflective of its purpose as a live record (see from paragraph 3.44).
• As of July 2022, key risk documents were uploaded to an enterprise level risk register, some dating back to 2018.
• As of August 2022, the ATO made changes to an archived compliance strategy checklist.
• As of August 2022, after the sample was selected, the ATO linked and edited a previously existing program logic for a different compliance strategy to a suite of three selected compliance strategies. These compliance strategies did not have program logics attached on the Treatment Work Register at the time of selection. It was determined the program logic was not used by the ATO to implement the compliance strategies and that it would not be included in testing.

Appendix 3 Graphical representation of the Individuals Not in Business Tax Gap methodology

Appendix 4 Assumptions underpinning the tax gap methodology for individuals not in business

Table A.1: Assumptions made by the ATO in tax gap methodology for individuals not in business

Source: ANAO analysis of ATO documentation.

1. The following table shows the ATO’s results for each step of the methodological process between 2015–16 and 2018–19, as at the date of the calculation of the 2018–19 individuals not in business tax gap.⁶⁶

Table A.2: Results of methodological process 2015–16 to 2018–19

Note a: The figures in the table are rounded amounts.

Note b: The ATO’s criteria for deciding whether a tax return should be data verified (see from paragraph 2.39) means that no taxpayers in the rental stratum were included among the verified group.

Source: ATO documentation.

Appendix 5 Selection of compliance strategies for testing

1. With differences in risk prioritisation and the methods used by the ATO to treat identified risks to the tax gap for individuals not in business, the ANAO selected compliance strategies for testing. To ensure a relevant range of risks, treatment activities and defensibility of the compliance strategies were included, compliance strategies were selected based on key and emerging risks identified by the ATO. A selection matrix was developed to ensure full coverage across key and emerging risk areas. Compliance strategies were selected off the TWR as at 4 August 2022 to cover the following identified risk areas:

• Work Related Expenses; the ATO considers WRE to be the largest contributor to the Individuals gap;
• Rentals; the ATO evaluates that almost nine in ten taxpayers claimed rental related deductions incorrectly in their tax returns;
• Cryptocurrency; this is an increasingly risk and the ATO continues to build its strategy; and
• Tax Agents; the ATO works with tax agents to increase the compliance of individuals, through agents.

2. This selection matrix also enabled the ANAO to ensure full coverage across a range of compliance methods and the status of compliance strategies. These included:

• status of compliance strategy, for example finalised or in the evaluation phase;
• product type such as an email or letter to an individual or a tax agent, or an educational program for tax agents;
• ATO assessment of defensibility (see Table 3.3); and
• Risk Category (see Table 3.5).

Table A.3: Selected compliance strategies