Introduction

1. An agency’s accountability to the Parliament and the Australian public and the quality of its internal decision-making depend, to a large extent, on the quality of data collected by the agency and on its effective transformation into information contained in agency reports. The information in these reports may also be used to inform policy development, for resource allocation decisions and for other analytical purposes. The provision of accurate information forms part of the legal obligations of public sector entities, with the Public Service Act 1999¹ requiring that heads of agencies assist Ministers to provide factual information relating to the operation of the agency.

2. The Australian Taxation Office (ATO) collects approximately 91 per cent of revenue for the Australian Government², and produces reports that inform a range of strategic decisions across Commonwealth, state and territory government entities as well as private organisations. ATO data and statistics also contribute to the development of financial, social and economic policies. It is within this context that, for example, the Commonwealth Grants Commission uses data on the value of the goods and services tax when calculating the amount of revenue to be distributed to the states and territories each year.

3. The Information Technology (IT) systems and administrative processes used to capture, extract and report ATO data must be reliable and able to deliver accurate information that can be used confidently, internally and externally. IT systems are used in every phase of tax administration; the ATO has one of the largest and most complex IT operations in the Australian Public Service. It maintains over 750 systems, most of which are available 24 hours a day and across the ATO’s 65 sites nationally. The ATO’s IT systems are used by 24 700 employees, and process a daily average of 3.8 million transactions.

Change Program

4. Following the introduction of the Government's tax reforms in 2000³, the ATO reassessed its business systems and recognised the need to replace its ageing processing systems. It initiated, in 2002, an IT and business transformation change agenda, the Change Program.⁴ While the Change Program also included a range of business and IT reforms, a key original objective was to introduce a single Integrated Core Processing (ICP) system⁵ that would process the range of taxes, replacing all previous core processing systems.

5. The Change Program was closed in June 2010. It was described as one of the largest business change and information technology projects ever undertaken by the Australian public sector, and in effect transformed the way the ATO operates and delivers services to its clients.⁶ However, as noted by the Inspector-General of Taxation in 2010, notwithstanding the substantial changes brought forward by the Change Program, major work remained uncompleted.⁷ Most notably, the Business Activity Statement (BAS)⁸ and the Superannuation Guarantee components of the ICP system were not delivered.

6. As a result, a number of legacy core processing systems could not be retired and are still used to process key taxes. Data extraction, transfer, conversion and synchronisation must still be conducted between multiple systems. A number of systems had to be duplicated to accommodate the parallel operation of the new ICP system and the legacy systems. Importantly, the data generated by both systems requires the ATO’s enterprise data warehouse⁹ to be separated into two logical views—ICP system data is in the ‘new’ data warehouse and legacy systems data is in the ‘old’ data warehouse. From a reporting perspective, reports must be extracted and reconciled from all data sources—the legacy and ICP systems and the old and new data warehouses.

Reporting within the ATO

7. For the most part, the ATO does not have a centralised process for extracting data and generating reports: approximately 500 ATO business and data analysts, distributed across all business areas, have access to the ATO’s data warehouse for the purposes of reporting, analytics and risk and intelligence assessments. Reports are generated by all business areas, and while some of the data included in a report may be generated by the business area that produces the report, data from other business areas can also be utilised.

8. The ATO is developing an integrated framework for the reporting process known as the Enterprise Reporting Strategy. Initiated in 2006, the strategy was revised in 2010 and aims to deliver a centralised reporting capability (the reporting portal) that will produce quality-assured data for a standard suite of reports and business rules to govern the production and management of reports. The reporting portal is already operational and, while currently providing access to only a limited range of datasets, work is ongoing to expand the data available from the portal.

9. As part of the strategy, the ATO is also developing an enterprise policy to support the development of standardised reports across the agency. This policy, outlined in the Corporate Management Practice Statement Reporting in the Australian Taxation Office, sets out the principles underpinning the development of reliable and accurate reports, and defines reporting as the ‘process of providing information to decision-makers to support them in their work’. The policy states that ‘ATO personnel are expected to apply a consistent and repeatable approach for internal and external reporting’.¹⁰ The ATO advised that the policy has been effective across the ATO since May 2013.

Audit objective, criteria and scope

10. The objective of the audit was to assess the effectiveness of the ATO’s management and reporting of selected information relating to the goods and services tax and the fringe benefits tax.

11. The audit examined, for key ATO reported data, whether:

• management arrangements (including quality assurance) were appropriate and effective;
• IT controls were appropriate and effective; and
• data reported was complete, reliable and timely.

12. Given the large amount of data compiled and reported by the ATO and the size and complexity of the ATO’s IT systems, the audit examined a limited number of data items (11)¹¹ relating to the goods and services tax (GST) and the fringe benefits tax (FBT), reported in seven key ATO external and internal reports.¹²

13. The focus of the audit was on the ATO’s administrative processes and the reliability of ATO’s IT systems in extracting, collating and reporting data. The audit did not assess the accuracy of the estimation of revenue collections.

Overall conclusion

14. Information published by the ATO in publicly available reports informs the Parliament and the community about the performance of the ATO in relation to the administration of the taxation and superannuation systems. Information published internally allows managers to provide sound advice and to make informed decisions. ATO data can also be used in a broader policy development and research context. One of the common requirements of all users is the ability to trust the source of the information provided and to be confident that the results published are authoritative and accurate. The reliability of ATO reports depends on the adequacy of the ATO’s IT systems to capture, process and store information¹³, and on the practices adopted by staff to extract and collate the relevant data into reports.

15. The ATO’s reporting of the sample of GST and FBT data examined by the ANAO was generally reliable, indicating that users can be confident that the information provided is accurate. Of the 11 data items examined, it was possible to reproduce the exact figure contained in the selected GST and FBT reports in five instances, and to reproduce an indicative figure—close to but not exactly the same as that in the ATO’s published reports—in the other six instances.¹⁴ Further, six of the 11 data items were repeated across several reports, with one instance identified where the same data item was reported differently across two reports.

16. Reporting of GST and FBT data is supported by the effective capture, processing and storage of taxpayer information by IT systems. However, arrangements to extract data and transform it into collated reports lack consistency in processes and practices, formalisation and supporting documentation, and, as a consequence, do not support the production of standardised reports. While the ATO is aware of these shortcomings, and is taking steps to address them through the Enterprise Reporting Strategy, they continue to present risks to the integrity of the ATO’s reporting processes.

17. The lack of standardised and automated processes has required ATO staff to conduct extensive manual activities (or work arounds) to extract data and collate reports. These activities include gaining assurance that appropriate data is being used, that it is complete, up-to-date and accurate, and has been processed appropriately for the purposes of the particular report. In a context where approximately 300 data analysts extract and manipulate data and where reports are generated by reporting teams across most ATO work areas, there is substantial overlap and duplication of tasks, a lack of supporting documentation, and data quality control and assurance is inconsistent. Current reporting practices also rely on ad hoc, localised and often informal strategies that are not consistently efficient or robust. This increases the risk that some of the data published in ATO reports is not accurate, with potential consequences for users of that data.

18. In response to these known problems, in 2006 the ATO commenced the implementation of its Enterprise Reporting Strategy, which seeks to develop an integrated reporting process. The strategy aims to improve data reliability in reports through two key deliverables: a central reporting portal, producing assured reports; and a policy framework to guide and standardise the data extraction and report collation processes. The strategy had limited success during the first four years of implementation (2006–10), when it was part of the Change Program.¹⁵ Since then, the strategy has made considerable progress: a reporting portal is operational and policy documents were implemented in May 2013. However, the ATO has not measured progress against the long-term objectives of the strategy. In this light, there would be benefits in the ATO assessing, at regular intervals, the usage of the portal and the extent of compliance with the reporting policies.

19. The ANAO has made three recommendations to strengthen the management and reporting of ATO information. The recommendations complement the initiatives being implemented through the Enterprise Reporting Strategy, and are aimed at: maintaining up-to-date data in the data warehouse; documenting the development, testing, storage and quality control of data extraction queries; and measuring the achievements of the Enterprise Reporting Strategy against its long-term objectives.

Key findings

Lodgement and processing of taxpayer information (Chapter 2)

20. The ATO operates in a context where large volumes of information must be captured, processed, stored and transferred in a complete, reliable and timely manner. These tasks are further complicated by the parallel operation of two IT production environments—the legacy systems and the ICP system.

21. Overall, the ATO’s management of the lodgement and processing of taxpayer information in the ATO’s processing systems is carried out in a manner that is consistent with the ATO’s policies, practices and procedures. However, the ATO could take further steps to ensure that information that has been deleted in the ICP system is accurately and promptly reflected in the ATO’s data warehouse. Taking steps to monitor and resolve these data discrepancies will reduce the risk of publishing inaccurate information in ATO reports.

22. There is also scope to improve the efficiency of the transfer of data from lodgement to the processing systems. For example, manual intervention is required by ATO officers if a lodgement is identified by the automated edit and validation control check as being incomplete. While this practice is acceptable, a more efficient process would be for the ATO online forms to deny lodgement at the time of submission unless all required fields are completed. There would be benefit in the ATO identifying and implementing practices that improve the efficient processing of lodgements.

Reporting from the data warehouse (Chapter 3)

23. The ATO provides a comprehensive data service across the different business areas, which delivers information that aims to be complete and timely. However, the development, testing and storage of Structured Query Language (SQL¹⁶) scripts and SQL outputs is not formalised and appropriately managed. There is a heavy reliance on the expertise of a small group of data analysts, a lack of documentation of the SQL management process, and insufficient quality control of SQLs and SQL outputs.

24. Further, for six data items published in ATO flagship publications (such as the GST Annual Performance Administration Report), the ATO was only able to reproduce indicative, rather than exact, figures and statistical information. The inability to reproduce exact figures is attributed to several factors, however, the primary reason is that the business areas did not retain copies of the source data from which the data item was extracted. The ATO could take further steps to improve its practices to support the management of SQL outputs.

25. Business areas are responsible for the design, development and management of SQLs and their SQL libraries. Of concern is the absence of quality assurance processes being applied to the development, modification and testing of SQLs prior to use in the ATO data warehouse. Further, there is no evidence that preventative controls are in place to limit the accidental edit or deletion of SQLs from the SQL libraries. This highlights a significant risk for the ATO if business rules for the management of the ATO’s information are inconsistent or incorrectly applied. There would be benefit in the ATO monitoring the business practices for data analysis across the business areas, and employing effective software lifecycle management¹⁷ practices to improve the design, development and testing of SQLs. The policy documents that are part of the Enterprise Reporting Strategy (released in May 2013), have the potential to address some of the shortcomings relating to SQL management.

Processes for collating reports (Chapter 4)

26. Reporting practices in the ATO have evolved over time to respond to day-to-day business requirements and new policy initiatives. Current practice is that data is generated by a wide range of work areas, resulting in a complex network of data circulation across the ATO.

27. This practice is not unusual in an organisation of the ATO’s size, where IT systems and reporting processes have evolved in response to internal and external changes and pressures. It presents benefits: in that several work areas have access to the data warehouse and to extraction processes; and a level of cross-checking occurs when data is transferred from one branch to the next. However, the reporting process also presents some challenges: identifying the source of the data for the GST and FBT items was often difficult, and contradictory information was provided from different areas of the ATO. There was also no internal process to effectively identify the accountable and responsible business areas or ATO officers for each GST and FBT data item. Further, there is no consistent process to provide assurance that the data included in regular, ongoing ATO reports is reliable, and no formal mechanism to ensure that the same data presented in different reports is consistent.

28. The ANAO did not identify any significant data errors in the process of report collation in the 11 data items selected for the audit. However, in one instance, the same data item was reported differently across two reports.¹⁸ The complexity of the sourcing of the data, combined with the lack of formal quality assurance processes on the reliability of the data, present IT and management risks. These risks include a loss of data quality control, duplication of work, and lack of consistency in the data reported across different reports.

29. Taking into consideration the complex work environment in which reports must be collated, there would be benefit in the ATO reinforcing practices and procedures that provide assurance on the reliability of the data included in ATO reports. One step in that direction would be the more widespread use of a standard document to assist in providing such assurance. An example is the document developed by the Client Account Services business line that accompanies each data item to be included in public reports and in important internal reports. Information on the source, data extraction mechanism and method of calculation applied to obtain the data is recorded and provides stronger assurance on the quality of the data and facilitates the identification of the data source. This would be a practical approach to providing assurance about the integrity of data used for reporting purposes, pending the implementation of the new Enterprise Reporting Strategy framework.¹⁹

The Enterprise Reporting Strategy (Chapter 5)

30. As previously noted, the Change Program did not fully deliver on an ICP system, which meant that reporting practices became more complicated by the necessity for ‘hybrid’ reporting (requiring data extraction from both legacy and ICP systems).

31. The ATO has reviewed and documented its reporting processes and recognises the weaknesses of the current practices and the ensuing risks to report accuracy. To address these risks, the Enterprise Reporting Strategy, an ATO-wide initiative to improve data reliability in ATO reports, was developed and rolled-out from 2006. The strategy is characterised by two main deliverables: the development of a central reporting portal that will produce quality-assured reports; and the implementation of a set of guidelines and standards providing a framework for the production of reports.

32. The strategy has the potential to address most shortcomings currently observed in the ATO’s reporting practices and improve report reliability. The strategy, however, has been in progress since 2006, with achievements being slow during the first four years (2006–10). This lack of progress and initial impact has had the further negative outcome of diminishing the credibility of the strategy internally. Since 2010, when the strategy was reviewed and renewed, progress has been considerable, and the outputs delivered include:

• fourteen projects (delivering a number of datasets to different business lines) completed and another 19 planned²⁰; and
• a set of policy documents providing the framework for the production and delivery of assured reports in the ATO (released in May 2013, with compliance monitoring starting in January 2014).

33. The ATO has not measured the strategy’s progress against longer term objectives and, in particular, the level of usage of the reporting portal. The ATO was not able to demonstrate that, six years after the initiation of the strategy, the outputs delivered were contributing to the achievement of the strategy’s long-term objective of improving report reliability. Consequently, despite these outputs, there is a risk that the initiative will not change current reporting practices.

34. The ATO has included in its Annual Plan 2012–13 a key performance indicator relating to the coverage and use of the reporting portal. A review is also underway that will include the level of usage of the datasets available on the portal, and should be completed in June 2013. The ATO advised that it has been reviewing the ATO Annual Plan key performance indicators to determine priority for enterprise reporting through the portal.

35. The Certificate of Assurance process, as far as it relates to the implementation of the reporting policy, is expected to monitor compliance with the policy from January 2014. These initiatives will go some way to mitigating the risks of the reporting portal being under-used and of non‑compliance with the policy. Nevertheless, there would be benefit in the ATO: determining targets/milestones; developing interim performance indicators; and monitoring these indicators, to ensure that the strategy is on track to achieve its longer-term objective of improving the reliability of reports produced in the ATO.

Summary of agency response

36. The ATO provided the following summary response, with the formal response at Appendix 1:

The ATO welcomes this performance audit and considers the report supportive of our overall approach to improve administrative processes and the reliability of systems used to extract, collate and report data. In finding the ATO’s management and reporting of GST and FBT information to be generally reliable, the review identified a number of opportunities for improvement which will complement initiatives currently underway as part of the Enterprise Reporting Strategy. The ATO agrees with the three recommendations contained in the report.

The ATO also welcomes recognition in the report that the Enterprise Reporting Strategy is a long term approach and that significant progress has been made to develop a single data source for our reporting which will be used to support decision making and to measure the efficiency and effectiveness of the ATO.

Recommendations

Footnotes

[1]Public Service Act 1999, Sections 57 and 66.

[2] ANAO, Audit Report No. 51, 2011–12, Interim Phase of the Audits of the Financial Statements of Major General Government Sector Agencies for the Year Ending 30 June 2012, p. 216.

[3] Department of the Treasury, Tax Reform: Not a New Tax, a New Tax System, Overview, August 1998. Available at <http://archive.treasury.gov.au/documents/167/PDF/overview.pdf> [accessed 19 September 2012].

[4] ANAO, Audit Report No. 8 2009–10, The Australian Taxation Office's Implementation of the Change Program: a Strategic Overview. See also ATO, The Australian Tax Office Change Program, February 2011, Available at <http://www.ato.gov.au/content/downloads/CR00271687.pdf> [accessed 19 September 2012].

[5] The ATO ICP can be defined as an integrated suite of software to process tax returns. It includes a client register, an accounting sub system and a generic forms-processing sub system.

[6] Aquitaine Consulting, An Overview of the ATO’s Change Program 2002–2010, Canberra, 22 November 2010, p. 17. See also, for a description of the Change Program: Available at <http://www.ato.gov.au/corporate/content.aspx?menuid=0&doc=/content/00218609.htm&page=1&H1 > [accessed 1 May 2012].

[7] Inspector-General of Taxation, Review into ATO’s Change Program, December 2010, p. 113.

[8] Businesses use activity statements to report and pay a number of tax obligations, including the goods and services tax, pay as you go (PAYG) installments, PAYG withholding and fringe benefits tax.

[9] Because the data captured in the core processing systems is constantly being updated, a data warehouse is an electronic repository used to store data for ease of extraction, analysis and reporting.

[10] ATO, Reporting for the ATO, Corporate Management Practice Statement PM CS 2013/01, paragraphs 4 and 6.

[11] The 11 data items examined were, for the GST: value of GST net revenue cash collections, BAS refunds and GST debt (total/collectable); number of BAS lodgements; BAS refunds issued within 14 days; and active GST accounts/new registrations processed within 28 days. FBT data items included: FBT net cash collection, refunds and debt (collectable); value of employee contribution towards fringe benefits received; and number of FBT returns lodgement (overall and on-time).

[12] The seven reports examined were: the Commissioner of Taxation Annual Report 2011–12; Taxation Statistics 2009–10; the GST Administration Annual Performance 2011–12; the 2012–13 GST Revenue Product ‘HOTSA’(Health of the System Assessment); the GST Product Committee Report 2011–12; the FBT Revenue Product ‘HOTSA’ 2011–12; and the FBT Compliance Effectiveness Evaluation Report 2008–11.

[13] The ATO operates one of the largest and most complex computer systems in Australia, processing a daily average of 3.8 million transactions.

[14] The sample extracted figure was within five per cent of the reported figure for each of the six data items. The inability to reproduce exact figures was primarily due to: some business areas not retaining copies of the source data from which the data item was extracted; the dynamic nature by which data is updated daily in the ATO’s computer systems; and a lack of formal archiving policies surrounding the storage of source data identified in the extraction process.

[15] The Change Program included a ‘Reporting Solution’ that sought to deliver assured reports extracted from a single ICP system and a single data warehouse. As the Change Program was unable to fully deliver a single processing system, the transition period of ‘hybrid reporting’ during which reports would be extracted and reconciled from both sources—the legacy and ICP systems and the old and new data warehouses, had to be maintained into the future.

[16] SQL scripts are used to extract data from the data warehouse (see Glossary).

[17] Software lifecycle management—also referred as application lifecycle management—is a continuous process of managing the life of an application through effective governance, development and maintenance.

[18] The Commissioner of Taxation Annual Report 2011–12 indicates (p. 43) that 78.9 per cent of monthly BAS and 73.8 per cent of quarterly BAS were lodged on time; for the same period, the GST Administration Annual Performance Report indicates (p. 5) that in 74.2 per cent of monthly BAS and 73.9 per cent of quarterly BAS were lodged on time. The ATO advised that the variance is mostly explained by the difference in the time of data extraction between the two reports.

[19] The ATO advised that work has also recently commenced to implement a glossary of commonly used business terms and labels. While this project was at the scoping stage in May 2013, the ATO expects that the glossary would eventually include information on the source of data, extraction mechanisms and calculation methods.

[20] These projects were selected after consultation with business areas, and were largely aimed at areas where performance was of concern or where reporting gaps existed. They were focused on organisational-level performance measures, particularly service standards.