Introduction

1.1 On 20 May 2014, the National Archives of Australia (the Archives) recommended that the Attorney-General approve the development of a new policy for information management in Australian government agencies. It was intended that the policy would assist in the delivery of broader objectives for e-government and the digital economy, and support the implementation of effective digital information management throughout the Australian Government.

Development of the Digital Continuity 2020 policy

1.2 Development of the Digital Continuity 2020 policy commenced in August 2014.⁴ Internal documentation indicated that the policy would be the Archives’ ‘flagship’ policy, and that it should apply to all Australian Government entities. Development of the policy and associated implementation guidance continued throughout 2015, and the policy was formally launched on 27 October 2015 by the Director-General of the Archives and the Secretary of the Department of Finance.

The Digital Continuity 2020 policy

1.3 The Digital Continuity 2020 policy is a successor policy to the Australian Government’s Digital Transition policy, which was approved in July 2011. The objective of the Digital Transition policy was to move Australian Government entities from paper-based records to digital information and records management. The Digital Continuity 2020 policy seeks to: ensure that records and information is created, managed, and maintained digitally; and digital authorisation and approval processes are embedded into all business systems used by Australian Government entities to deliver services and undertake functions. The policy also requires entities to ensure that interoperable information, systems, and processes are developed and implemented that meet standards for short and long-term management.

1.4 The Archives issued the Digital Continuity 2020 policy under the Archives Act 1983, which authorises the Archives to issue standards for Commonwealth records, and to preserve and make accessible the archival resources of the Australian Government.⁵ The policy applies to all Australian Government entities, including Government Business Enterprises.

1.5 The purposes of the policy are:

• to support the Australian Government’s digital transformation initiatives;
• to enable the integration of information governance principles and practices into the work of agencies and their governance arrangements; and
• to promote a consistent approach to information governance across the Australian Government and within individual agencies.

1.6 The policy applies to government information, data and records, as well as systems, services and processes, including those created or delivered by third parties on behalf of Australian Government entities.

Principles

1.7 To achieve the objectives of the Digital Continuity 2020 policy there are three principles that entities are to embed by 31 December 2020, as outlined in Figure 1.1.

Figure 1.1: Principles of the Digital Continuity 2020 policy

 

 

Source: Digital Continuity 2020 policy.

Recommended actions

1.8 In order to embed the three principles, the policy includes 10 recommended actions with associated target dates for implementation, as outlined in Table 1.1.

Table 1.1: Recommended actions from the Digital Continuity 2020 policy

Source: Digital Continuity 2020 policy.

1.9 The implementation guidance⁶ accompanying the policy has a total of 29 targets, with due dates ranging from 30 June 2016 through to 31 December 2020. Seventeen of these targets were due by 31 December 2018. A full listing of the targets and associated due dates is provided at Appendix 2.

Responsible Authority

1.10 As the entity responsible for setting records and information management requirements, including all information created, used, or received as part of government business in digital and non-digital formats, the Archives is responsible for overseeing the implementation of the policy. This includes establishing effective administration and monitoring and evaluation arrangements to assist and encourage Australian Government entities to achieve the targets of the policy by 31 December 2020.

1.11 In November 2015, the Archives sought endorsement of the proposed policy through the Attorney-General, and advised that:

• the policy should have minimal budget or red tape impact on entities due to its five year implementation period (2015–2020);
• its design allows for implementation within normal budget cycles and procurement processes; and
• the policy will establish recommended best practice for all agencies.

1.12 The Attorney-General subsequently dispatched letters to the Prime Minister, the Treasurer and other Ministerial colleagues seeking, and obtaining, their endorsement and support to implement the policy within their respective portfolio agencies and departments.

Broader legislative and policy framework

1.13 The Digital Continuity policy sits within a broader framework of supporting standards, whole-of-government policies and strategies that Australian Government entities are to implement to meet legislative and regulatory requirements associated with information management. An overview of this framework is provided at Table 1.2.⁷

1.14 Detail on the Australian Government’s information management legislative, regulatory, and policy environment is provided at Appendix 3.

Rationale for undertaking the audit

1.15 Australian Government entities are legally required to manage information in a manner which properly records and explains their performance.⁸ Effective information management supports accountability and transparency, and enables informed decision making.⁹ The Australian Government’s transition to digital service delivery creates both opportunities and risks to effective information management. The Archives has an important responsibility to ensure that an appropriate framework is designed and applied to support this transition process, and the Digital Continuity 2020 policy is central to this in providing a whole-of-government approach to digital information governance. The final targets of the policy are due for implementation by 31 December 2020, and it is therefore timely to examine the extent to which entities have implemented the policy, and how effectively the Archives is administering and overseeing its implementation.

Audit approach

Audit objective, criteria and scope

1.16 The objective of this audit was to examine the extent to which selected Australian Government entities have implemented the Digital Continuity 2020 policy, and how effectively the National Archives of Australia is monitoring, assisting, and encouraging entities to meet the specified targets.

1.17 To form a conclusion against the audit objective the ANAO adopted three audit criteria:

• Has the National Archives of Australia established effective arrangements to administer the Digital Continuity 2020 policy?
• Has the National Archives of Australia implemented effective monitoring and evaluation arrangements?
• To what extent have selected Australian Government entities implemented the Digital Continuity 2020 policy?

1.18 The audit examined the administration, oversight, monitoring and reporting arrangements for the Digital Continuity 2020 policy, and the extent to which three selected Australian Government entities have implemented the policy.

Entities selected

1.19 The Attorney-General’s Department, the Civil Aviation Safety Authority and the Office of the Inspector-General of Intelligence and Security were selected by the ANAO to assess the extent to which the Digital Continuity 2020 policy has been implemented. The criteria used to select the entities is detailed at Appendix 4.

Audit methodology

1.20 Audit procedures included:

• examining documentation held by the Archives including briefing material, reports and advice, as well as administrative, quality assurance, monitoring and evaluation documentation;
• examining documentation held by the selected entities, particularly documentation detailing the information governance arrangements in place; and
• interviewing personnel from the Archives and the selected entities.

1.21 The audit was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $367,500. The team members for this audit were Joyce Knight, Nathan Callaway, Jessica Kanikula and Paul Bryant.

Have effective internal arrangements been established for the administration of the Digital Continuity 2020 policy by the National Archives of Australia?

2.1 In August 2015, the Archives identified a range of initiatives, including the development and release of tools, advice, training, and web content, that would be required to support entities across the Australian Government implement the Digital Continuity 2020 policy (detail on the products, advice and guidance material developed by the Archives is provided at paragraphs 2.11 to 2.24). Responsibility for these initiatives was added into the 2015–16 annual work plans for the responsible areas to be managed as business-as-usual activities.¹⁰

2.2 Between July 2015 and September 2016, information on the progress of these initiatives was included in branch reports. These reports were discussed at a monthly senior managers meeting involving the Director-General, Assistant Directors-General (Branch heads), and State and Territory Directors. The Executive Board, comprising the Director-General and Assistant Directors-General, were provided with formal updates on the progress of each branch against relevant corporate plan targets, however the progress of initiatives associated with the Digital Continuity 2020 policy were not consistently identified in the reporting provided.

2.3 In July 2016, the Archives recognised that the projects established to support implementation of the policy were not being managed as a coordinated program of work. In November 2016, a Digital Continuity 2020 Project Implementation Register (PIR) was established which identified 20 individual projects (and associated sub-projects) and linked each to the relevant ‘recommended action’¹¹ and ‘target’¹² of the Digital Continuity 2020 policy. The PIR also recorded a complexity rating¹³ and the current status of each project.¹⁴ A draft document titled ‘DC2020 Project Management Framework’ outlined that each project should have a project proposal or brief developed, however project documentation was only prepared for eight of the 20 projects, and the PIR has not been maintained since July 2017.

2.4 In March 2018, the Archives commenced an internal review to evaluate progress against the projects listed in the PIR and to identify any gaps and opportunities for further development. A final report from this review was not provided to management.

2.5 In August 2018, the projects established to support the implementation of the policy were incorporated into a high-level register provided to the Archives’ Project Management Committee, which is responsible for overseeing the administration and delivery of major projects across the Archives. However, the projects associated with the policy were not required to provide regular reports to this committee.

2.6 Since the launch of the Digital Continuity 2020 policy, the Archives has made several attempts to establish effective internal arrangements to coordinate the administration of the projects established to support and encourage entities implement the policy. However, appropriate program governance, oversight, and reporting arrangements were not maintained.

2.7 In January 2019, an internal audit commissioned by the Archives found that a project management approach had not been utilised to administer the implementation of the Digital Continuity 2020 policy. The internal audit subsequently recommended that the Archives should develop and implement a program of work to achieve the objectives of the policy, and any successor whole-of-government policies in the future, and Archives has agreed to implement the recommendation.

Did the National Archives of Australia develop products, advice, and guidance material that are fit for purpose?

Products, advice, and guidance material

2.11 The Digital Continuity 2020 policy states that the Archives will ‘develop advice, products and tools to support information governance, digital information management and interoperable information, systems and processes.’

2.12 The Archives’ Project Information Register (PIR) established a listing of projects, and associated products, that would be developed in this regard. While the PIR was not maintained after July 2017 (see paragraph 2.3), those projects from the PIR that have been completed and delivered¹⁵ have generated the following products to support the implementation of the policy:

• implementation guidance, and associated implementation material;
• an information management and data capabilities matrix¹⁶;
• a minimum metadata set¹⁷;
• a digital authorisation framework;
• a business systems assessment framework; and¹⁸
• an interoperability toolkit.¹⁹

Implementation guidance and associated implementation material

2.13 The development of implementation guidance and associated material commenced in January 2015, was released alongside the policy in October 2015, and is comprised of the following products.

• Implementation guidance — In October 2015, the Archives released the Digital Continuity 2020 – Agency Implementation and Pathways. This document breaks down the 10 recommended actions under the policy into smaller targets, with staggered implementation dates across the five year implementation period of the policy.
• Associated implementation material — the Archives has published advice to guide entities in establishing an information governance committee, and an information governance framework. The guidance material includes sample terms of reference and identifies what information should be included in key supporting documentation such as an information management strategy and associated policy.

Information management and data capabilities matrix

2.14 Initially, the Archives intended to develop a whole of government information management strategic workforce plan which would include professionalism and capability targets for entities. In June 2016, an agency professionalism working group was established which included representatives from across the Australian Government. As a result of engagement with the Australian Public Service Commission (APSC), Digital Transformation Agency (DTA) and external agencies through the working group, this approach was revised in June 2016.

2.15 The revised approach introduced two new targets to support entities meet the recommended action for skilled staff. The first target was added in November 2016, requiring all entities to establish a ‘Chief Information Governance Officer’ role by 31 December 2017. The second new target was introduced in September 2017 for all entities ‘to establish and implement a program of continuing professional development of information management staff to achieve professional recognition’ by 31 December 2018.

2.16 To support entities in undertaking work associated with the continuing professional development target, the Archives redesigned an existing tool, the digital information capability matrix²⁰, to identify the skills and knowledge that are needed to create and manage information and data effectively in order to meet business and accountability requirements.²¹ The resulting product is similar to the Australian Public Service (APS) work level standards, identifying the information management capabilities that ‘all staff’ and ‘information management staff’ should have, and mapping the capabilities to one of four proficiency levels. The revised information management and data capabilities matrix was released in May 2018, seven months prior to the due date for this target.

Minimum metadata set

2.17 The minimum metadata set identifies metadata properties essential for the management of business information created and used by Australian Government entities to inform decision making and deliver government services. The minimum metadata set was developed by the Archives based on the Australian Government Recordkeeping Metadata standard²², and provides guidance on the transfer of information to the Archives, or other entities, to support interoperability, accessibility and re-use (including auditability).

2.18 The minimum metadata set identifies nine properties required for the effective management of business information, and is made up of three components (see Figure 2.1).

Figure 2.1: Minimum metadata set

 

 

Source: National Archives of Australia, available from http://www.naa.gov.au/information-management/digital-transition-and-digital-continuity/information-is-interoperable/metadata/index.aspx [accessed 22 May 2019].

2.19 Development of the set commenced in March 2015, with the product released in February 2016.

2.20 Under the policy, entities are responsible for ensuring the requirements of the minimum metadata set are met by all new and existing business systems. The Archives engaged with the Digital Transformation Agency (DTA) in 2017 to explore incorporating the minimum metadata standards into whole-of-government Information and Communications Technology (ICT) procurement templates that were being developed.²³ In 2018, it was agreed that a generic reference to compliance with all relevant laws applying to the procurement process was sufficient to address the policy’s requirements, and that individual entities are responsible for determining how the minimum metadata set should be met when procuring new business systems.

Digital authorisations framework

2.21 The digital authorisations framework is a risk-based assessment tool designed to assist entities determine its digital approval requirements and select an appropriate digital approval method. Acceptable methods include email, action tracking, system workflows, and digital signatures. The Archives is also piloting an online digital authorisations tool.

2.22 Development of the digital authorisations framework commenced in June 2015, with entities invited to participate in a digital workflows working group in July 2016. The working group continued until mid-2017 and the digital authorisations framework was released in October 2017, two months prior to the due date of 31 December 2017 for the associated Digital Continuity 2020 policy target. ²⁴

Business systems assessment framework

2.23 The Archives developed a business systems assessment framework that entities can use to assess information management functionality. ²⁵ The framework provides entities with a structured approach to the assessment of information management functionality in business systems, based on the value of information and the associated level of risk.

2.24 The first policy target that refers to the use of the business system assessment framework requires entities to ensure that information management functional requirements are incorporated into any new business systems purchased after 31 December 2016.²⁶ Development of the business systems assessment framework commenced in April 2015, and the Archives released the framework in February 2016. In 2018, the Archives commenced work to refine the Business System Assessment Framework, and is currently piloting an online tool to simplify and streamline the business systems assessment process.

Entity views on Digital Continuity 2020 guidance

2.25 The selected entities advised that the products, advice and guidance material issued by the Archives to support the policy are comprehensive and valued, however expressed confusion in relation to the consistency of terminology within the guidance, and indicated that some material has not been consistently delivered in a timely fashion, with sufficient lead time for entity implementation.

Consistency of terminology

2.26 The Digital Continuity 2020 targets are interchangeably referred to as ‘recommended actions’, ‘targets’, and ‘interim targets’ within the Digital Continuity 2020 policy document, the Agency Implementation and Pathways document (implementation guidance), and the advice and guidance material published on the Archives website. There is also a mix of recommended actions listed in the policy and the targets detailed in the guidance provided within previous annual survey instruments, particularly the 2017 Digital Continuity Statements and the 2018 Check-Up Plus annual survey. As a result, entities have expressed confusion regarding the status of the actions of the policy, the targets in the implementation guidance, and whether the targets are mandatory, recommended or optional. Ensuring that clear and consistent language is used to communicate the mandatory requirements of legislation, regulation or policy was one of the key themes identified in the Belcher Review. ²⁷

Introduction of new targets

2.27 When the policy was launched in October 2015, the accompanying implementation guidance had a total of 25 ‘targets’ with links to the products and guidance material that had been developed and released by Archives to support entities achieve the targets and complete the recommended actions of the policy. In October 2017, and again in February 2019, the Archives revised the implementation guidance²⁸ adding a total of four new targets. The new targets state:

• entities are to establish a Chief Information Governance Officer (31 December 2017);
• entities are to establish and implement a program of continuing professional development of information management staff for professional recognition (31 December 2018);
• entities identify remaining analogue approval processes and evaluate against the Archives’ digital authorisations framework to implement fully digital authorisations and workflows (30 June 2019); and
• chief information governance officers or senior officers responsible for information governance individually join a professional association to support their continuing development (31 December 2019).

2.28 Three of the four targets added to the implementation guidance are mapped to principle one of the Digital Continuity 2020 policy, and are intended to support entities in meeting targets for skilled staff. When introducing the new professionalism related targets the Archives consulted with entities across the Australian government, including the Australian Public Service Commission (APSC) and the DTA prior to introducing the new targets requiring entities to establish a CIGO and a program of continuing professional development for information staff in 2016 through to 2017.

2.29 In January 2019, records managers across the Australian government sector were advised that the target to support the professionalisation of CIGO’s and senior officers responsible for information governance was being introduced. The new target requires ‘CIGO’s or senior officers responsible for information governance individually join a professional association to support their continuing development.’ This target is highly prescriptive and does not align with a principles based policy. During the course of the audit the selected entities advised that they were unsure why the target was introduced.

2.30 As part of the development of a wider stakeholder engagement and communication strategy, Recommendation no.2 (see paragraph 2.38) addresses the need for the Archives to ensure that stakeholders are appropriately consulted when introducing new or revised targets, and that mechanisms are established to ensure targets are clearly identified and consistently communicated as either mandatory, suggested, or optional.

Does the National Archives of Australia effectively engage with stakeholders to administer the Digital Continuity 2020 policy?

Stakeholder engagement and communication — strategy and planning

2.31 In July 2015, the Archives commenced the development of a communications plan for the Digital Continuity 2020 policy. In November 2016 the draft plan articulated: key communications objectives; target audiences; roles and responsibilities; risk and budget information; and key performance indicators, however the communications plan was not formally approved, and there is no evidence that stakeholder engagement matters were addressed in any other planning documents.

2.32 There is evidence that communication plans were developed for a small number of the individual projects initiated to support entities implement the policy, however the Archives does not have an approved communications or stakeholder engagement strategy to administer the implementation of the Digital Continuity 2020 policy as a coordinated program of work.²⁹

Stakeholder engagement and communication — in practice

2.33 In practice, the Archives engages with stakeholders using the following channels.

• Social media updates — where the Archives promotes events, and seeks community feedback on initiatives, including initiatives related to the Digital Continuity 2020 policy such as information awareness month.
• The Archives’ website — where the Archives releases supporting products, guidance and advice to assist and encourage entities to implement the targets of the Digital Continuity 2020 policy.
• The Government Agencies Information Network (GAIN) Australia³⁰ — GAIN holds forums, and issues regular e-bulletins. Forums are face-to-face meetings used to provide an avenue for information managers to share expertise and experience. The GAIN forums and e-bulletins are also used to provide updates on information management initiatives, standards, and policies including the Digital Continuity 2020 policy.³¹
• The Archives’ Agency Service Centre (ASC) — logs queries received in relation to all the functions of the Archives, including queries related to the implementation of Digital Continuity 2020 targets (and associated responses). Between 2017 and 2019, the ASC has logged 1208 queries, with 287 (24 per cent) related to Digital Continuity 2020 or a supporting product³², guidance material³³ or target.³⁴

Targeted support for stakeholders

2.34 The Digital Continuity 2020 policy states:

the Archives will use annual agency reports [surveys] and other information as part of performance monitoring. This includes identifying agencies that need assistance to complete the recommended actions. The Archives will work with these agencies to improve their digital information management.

2.35 The annual agency surveys are examined in detail in paragraphs 3.12 to 3.17. The Archives uses the results of the annual surveys to inform its annual planning cycle and support the identification of activities in business plans throughout the year. For 2016, the Archives used the results from the Check-Up Digital survey to identify entities that required further assistance. However, there is no evidence that action was subsequently taken to provide targeted assistance to the entities identified through this process.

2.36 The Archives has acknowledged that, to date, there has been no formal process to support entities that are experiencing difficulties in implementing digital information management practices and provide targeted assistance to implement the targets of the Digital Continuity 2020 policy. To address this gap, the Archives approved a program of work entitled ‘Join-the-Dots’ in July 2019.

2.37 The ‘Join-the-Dots’ initiative intends to analyse data from the annual surveys to identify and prioritise entities that require additional support to drive improvement. The Archives has completed an analysis of survey data from 2014 to 2018³⁵, identified the entities that require assistance, and is in the process of determining the most appropriate methods to deliver assistance and establish priorities. To assess the impact of the project, the Archives intends to analyse the annual survey results for the remaining duration of the Digital Continuity 2020 policy. The next survey has been released and is to be completed by entities by 30 September 2019, with analysis and reporting due to occur in the subsequent months.

Are risks to the implementation of the Digital Continuity 2020 policy being effectively identified, managed and reported?

2.40 In its 2015–16 corporate plan, the Archives identified the Digital Continuity 2020 policy as a key strategy to address risks relating to the Archives’ ability to provide leadership and continued support for digital information and records management capability across the Australian Government.

2.41 In October 2015, work to develop a risk register for the implementation of the policy commenced. This work was not completed.

2.42 In November 2016, the Archives established the project implementation register (PIR) to track the progress of the projects established to develop the products, advice, and guidance material to support entities meet the targets of the policy. Of the 20 individual projects listed in the PIR, project briefs and associated project management documentation have been located for eight projects, and these include risk assessment information. The risk assessments within the documentation identify the risk controls and a risk rating, however do not include a risk management action plan that identifies how the effectiveness of the risk controls will be assessed or tested. In accordance with the Archives’ April 2016 risk management framework and policy, the strategic and operational risks associated with the achievement and promotion of the Digital Continuity 2020 policy targets should have been formally assessed and regularly reviewed. However, there is no evidence that this occurred.

2.43 In 2017–18, the Archives’ strategic risk register included the risk that ‘Archives fails to impose records creation requirements and the minimum standards for digital Government records’, and identified the following risk mitigation strategies.

• Continuing to develop tools, strategies, guidance and standards to assist entities transition to digital information management.
• Continued achievement and promotion of Digital Continuity 2020 policy targets up to 2020.
• Implementation of a new policy to guide digital information management — to succeed the Digital Continuity 2020 policy.
• A new survey reporting tool to assess entities progress towards digital information management.

2.44 As at July 2019, there is no evidence of any activity to monitor and report on the effectiveness of these strategies in the subsequent period.

Has the National Archives of Australia designed appropriate monitoring and evaluation arrangements?

3.1 When undertaking planning processes for the implementation of a policy, better practice guidance³⁶ states that monitoring and evaluation arrangements should:

clearly define the objectives and outcomes of the policy that is being implemented, [and] determine what successful outcomes will look like and what evidence will be needed to demonstrate success. As this has planning implications, thinking needs to occur from the outset and ensure activities are fit-for-purpose.

Objectives, outcomes, and performance measures

3.2 The objectives of the Archives in relation to the Digital Continuity 2020 policy (policy) have been identified using various language in successive corporate plans since 2015–16.

• In 2015–16, the implementation of the policy was not specifically identified, rather it was captured under strategic priority three (of three), to ‘provide leadership and continued support for digital information and records management capability across the Australian Government.’ A single quantitative measure — a 95 per cent participation rate by entities in an annual survey on digital information maturity — was used to measure performance.
• In 2016–17, implementation of the policy was listed as a ‘delivery strategy’ under purpose three (of four) to ‘provide leadership on information management to the Australian Government’. Two qualitative measures relating to the development and delivery of guidance material and advice to support entities achieve the targets were included to measure performance. In addition, a quantitative measure aiming to have 90 per cent of entities managing information ‘digitally by default by 2020’ was also included.
• In 2017–18, implementation of the policy was listed underneath purpose one (of four) to ‘demonstrate leadership and best practice, to promote the creation, management and preservation of authentic, reliable and usable Commonwealth records.’ Four targets were identified. Australian Government entities transition to digital information management in accordance with the Digital Continuity 2020 Policy. Archives reports to the Minister (annually) and Prime Minister (2018) outlining entities’ progress towards digital transition, the support provided, and additional support needed, by the Archives to further drive improvement. A new survey reporting tool is issued to assess entities’ progress towards digital information management. Entities participate in annual survey reporting requirements. However, there were no specific qualitative or quantitative measures included within the plan to measure performance against these targets.

3.3 The latest corporate plan (2018–19 to 2021–22) identifies the policy as one of three central strategies the Archives is to implement to deliver its purpose and achieve its vision. The relevant strategy is ‘establish frameworks for best-practice management of Australian Government information and data by Australian Government agencies toward achievement of the Digital Continuity 2020 policy targets.’ The three activities that the Archives will undertake to achieve this strategy are:

• The development of standards, policies, guidance, information and services to assist entities adopt good information and records management practices and implement the Digital Continuity 2020 policy and targets.
• Survey entities and report on transition to digital information management.
• Report to the Minister and Prime Minister outlining entities progress towards digital transition, the support provided, and additional support needed, by the Archives to drive further improvement.

3.4 The measures to evaluate performance against this strategy are:

• the percentage of agencies completing the annual survey (with a target of 96 per cent) — measure one; and
• qualitative evaluation of progress towards Digital Continuity 2020 policy outcomes (to be measured using survey responses and case studies) — measure two.

3.5 As such, the Archives has not maintained consistent priorities, objectives and targets in relation to its work on the implementation of the policy.

Appropriateness of performance measures

3.6 To determine if the current performance measures for the Digital Continuity 2020 policy implementation are appropriate, they have been assessed against better practice principles in relation to relevance, reliability and adequacy³⁷, with the results summarised at Table 3.1.

Table 3.1: Appropriateness of the Archives’ performance measures for the Digital Continuity 2020 policy implementation

Legend: ◆ the measure fully aligns with the relevant principle ▲ the measure partly aligns with the relevant principle ■ the measure does not align with the relevant principle

Source: ANAO analysis of the Archives’ Digital Continuity 2020 performance information included in the 2018–19 corporate plan.

Relevance

3.7 Measure one is not relevant. The fact, or not, of a high participation rate in the annual survey does not enable the Archives or external users to assess: the extent to which entities are implementing the policy; the effectiveness of the standards, policies, guidance, information and services to support entities implement the policy; or to what extent the expected benefits of implementing the policy are being realised.

3.8 Measure two is relevant. It uses survey responses and case studies to evaluate entity progress towards policy outcomes, and has a stronger alignment to the strategic objectives associated with successful implementation of the policy.

Reliability

3.9 Measure one is reliable. It is a quantitative measure with a target that can be verified. The measure is simple to assess — percentage of entities that complete the survey, measured using data obtained from the Archives’ annual surveys. The corporate plan identifies the types of entities that are invited to complete the survey, with internal documentation clearly defining the parameters of the measure, identifying entities that are out of scope and specifying how survey responses are to be counted.

3.10 Measure two is partly reliable. Combining quantitative data (survey data) with qualitative data (case studies) can enrich the performance story³⁸, however, the Archives has not defined a methodology that will be used to collect information and determine how case studies will be selected for inclusion in the annual report to ensure that the case studies selected are free from bias.

Adequacy

3.11 The Archives’ performance measures have the potential to provide an adequate basis to assess its performance against the strategic objective to ‘establish frameworks for best-practice management of Australian Government agencies toward achievement of the Digital Continuity policy targets.’ However, the current measures do not enable users to assess how effectively the standards, policies, guidance, information and services developed by the Archives support entities to implement digital information management practices, and to what extent entities have implemented the targets of the Digital Continuity 2020 policy. Additionally, a methodology to select case studies for inclusion in the annual report has not been defined.

Quality of performance information

3.12 The Digital Continuity 2020 policy states that ‘the Archives will use annual agency reports [surveys] and other information as part of performance monitoring’.

3.13 Since the launch of the policy in October 2015, the Archives has utilised three different annual agency surveys, the:

• check-up Digital survey — 2016³⁹;
• digital continuity statement — 2017⁴⁰; and
• check-up Plus survey — 2018.⁴¹

3.14 The nature of these surveys and their relationship with the actions and targets of the policy are outlined at Table 3.2.

Table 3.2: The relationship between the annual surveys and the Digital Continuity 2020 policy

Note a: National Archives of Australia, ‘Information Management Standard,’ available from http://www.naa.gov.au/ information-management/information-management-standard/index.aspx [accessed 8 March 2019].

Note b. In 2016, the annual survey was aligned to the Digital Transition policy, and in 2018 the annual survey was aligned to the Information Management Standard launched in 2017.

Source: Analysis of documentation, surveys, survey reports.

3.15 The varying questions⁴², measures, and rating scales used to gauge performance since the launch of the policy has meant that the Archives has not collected consistent and comparable data sets across the three surveys to enable analysis of entity progress to implement the targets over the life of the policy. ⁴³

3.16 To address this, the Archives has reviewed the 46 questions from the 2018 survey and identified 11 questions as ‘critical statements’ associated with the Digital Continuity 2020 policy. For 2019, a shortened survey of 15 questions has been released, 11 of which are the critical statements.

3.17 For 2020, the Archives intends to conduct a broader survey, similar to that conducted in 2018, with the shorter 2019 style survey to be issued again in 2021. This approach is intended to reduce the administrative burden on entities, while maintaining a common core of questions to facilitate longitudinal analysis of the data and assess entity progress over multiple reporting periods.

Benchmark measures of success

3.18 In 2016, the Archives attempted to define ‘what success looks like’ in relation to whole-of-government take up of the Digital Continuity 2020 policy. In the 2016–17 corporate plan, the Archives selected the benchmark that ‘90 per cent of entities will manage information digitally by default in 2020’ for this purpose.

3.19 Internal briefings indicated an agreement that if an entity reported a level 3 maturity rating (‘defined’) or higher, the entity would be deemed to be ‘working digitally by default’.

3.20 It was subsequently identified that using this maturity scale to measure success was potentially inaccurate. According to the annual survey for 2016 the level 3 digital maturity rating is described as: ‘having implemented some digital business processes, systems, technologies and tools, as planned’. As such, the entities that may have only commenced or partially implemented the relevant strategies and plans to move to digital information management could be considered to be operating ‘digitally by default’.

3.21 It was subsequently discussed within Archives that a level 4 digital maturity rating of ‘managing’, defined as ‘my agency has systematically transformed many business processes, systems, technologies and tools to digital as planned’, was a more accurate measure of success. However, for reporting purposes, the level 3 digital maturity rating of ‘defined’ continued to be identified as the measure of success. In February 2019, the Archives advised entities that the ‘Join the Dots’ program (examined in paragraph 2.37) aims to support all agencies achieve a level 3 maturity⁴⁴ rating or above by the end of 2020.

3.22 Accordingly, the Archives has not established clear and consistent measures of success to evaluate and accurately report on the progress of entities to implement the targets of the Digital Continuity 2020 policy.

Do the monitoring and evaluation arrangements accurately assess the extent to which entities are meeting the targets of the Digital Continuity 2020 policy?

Current data on entity progress in the implementation of the Digital Continuity 2020 policy

3.28 The annual survey for 2018 was focused on assessing entity compliance with the Archives’ broader Information Management Standard, and as such is not structured in a way that enables a direct comparison of survey results with the targets of the Digital Continuity 2020 policy. The Archives has reviewed the 46 questions from the 2018 survey and identified 11 questions⁴⁵ as critical statements. An analysis has therefore been undertaken of the data associated with these 11 questions with the objective of gauging entity progress to implement the policy. The results of the analysis is detailed at Figure 3.1 to Figure 3.3.

Principle 1 — Information is valued

3.29 The survey results for the six questions that the Archives has mapped to Principle 1 of the Digital Continuity 2020 policy are examined in Figure 3.1 below.

Figure 3.1: Entity self-assessment of maturity against 2018 survey questions linked to Digital Continuity 2020 Principle 1 ‘Information is valued’

Note a: The question, does your agency have a formal governance mechanism with broad representation ensuring information management requirements are considered when making decisions? — had a different answer scheme to the other questions. There were four possible answers. To make the rating consistent across all questions, the audit team categorised responses to this questions as: Level 1 is a “No response”; Level 2 is “Partial – mechanism planned but not fully implemented or lacks maturity”; Level 3 is “Yes for ICT-related matters only”; Level 4 is excluded; and Level 5 is “Yes for all agency information management decisions”.

Source: Analysis of data from the 2018 Check-up Plus survey.

3.30 For the recommended actions associated with Principle 1 that were due by 31 December 2018, entities were to have established an information governance committee (recommended action 2) by 30 June 2016, and have an information governance framework (recommended action 3) in place by 31 December 2016. The survey results reflect that approximately two years after the associated due dates, 36 and 38 per cent of entities respectively have not implemented the two recommended actions. Recommended actions 4 and 5 are not due to be complete until 31 December 2020.

3.31 Overall, these results indicate that there is still significant progress required across Australian Government entities in order to meet the targets associated with Principle 1 of the policy.

Principle 2 — Information is managed digitally

3.32 The survey results for the two questions that the Archives has mapped to Principle 2 of the Digital Continuity 2020 policy are examined in Figure 3.2.

Figure 3.2: Entity self-assessment of maturity against 2018 survey questions linked to Digital Continuity 2020 Principle 2 ‘Information is managed digitally’

Source: Analysis of the results of the 2018 Check-up Plus survey.

3.33 Both of the recommended actions for Principle 2 are not due until 31 December 2020. The results indicate that 38 per cent of entities have assessed their maturity at level two or below in relation to the automation of business processes.⁴⁶ Performance in relation to the removal of paper from existing process / digitisation of authorisations appears more positive, with 57 per cent of entities reporting that they are at maturity level 4 or above.

Principle 3 — Information, systems, and processes are interoperable

3.34 The results for the three questions that the Archives has mapped to Principle 3 of the Digital Continuity 2020 policy are detailed at Figure 3.3.

Figure 3.3: Entity self-assessment of maturity against 2018 survey questions linked to Digital Continuity 2020 Principle 3 ‘Information, systems, and processes are interoperable’

Note a: For this question, there was an additional possible answer. The sixth option was ‘not measured’. To make the rating scale consistent across all questions for Principle 3, the audit team combined ‘not measured’ responses into Level 1.

Source: Analysis of the results of the 2018 Check-up Plus survey.

3.35 The recommended actions for Principle 3 are not due until 31 December 2020. However, for each question, between 39 and 51 per cent of entities have assessed their maturity at level 2 or below. This indicates that there is significant progress required across Australian Government entities in order to meet the targets associated with Principle 3.

Reporting on entity progress in meeting the Digital Continuity 2020 targets

3.36 Following the completion of the annual survey by entities, the Archives produces a de-identified whole-of-government report which it publishes on its website.⁴⁷ The report presents summary results against five information management components.⁴⁸ The Archives also produces individual reports for entities. These reports have been produced since 2016 and include the individual entity’s results and how these results compare to the whole-of-government population. Individual results are benchmarked against an average rating calculated using the self-reported results of all Australian Government entities that participated in the survey, however, the reports do not allow entities to compare progress against other entities with similar characteristics in either size or function. Similar surveys conducted by the APSC include the ability for entities to compare progress in this manner.⁴⁹

3.37 The Digital Continuity 2020 policy requires that the Archives report annually to the Minister (the Attorney-General) outlining ‘agencies progress towards digital transition, the support provided, and additional support needed, by the Archives to further drive improvement.’ The Archives submitted reports to the Attorney-General detailing the results for the 2016 and 2017 surveys respectively, and has published these reports on its website. The reports outline: key survey results; areas for attention; products and advice issued in the previous year; proposed actions for the coming year; and illustrate that moderate improvement has been achieved across the Australian government between 2014 and 2016. The 2018 report was delayed from October 2018 to March 2019 to allow for an analysis of the survey data, however was then further delayed due to the election and subsequent entry into the caretaker period. The 2018 report was scheduled to be issued in August 2019.

Processes to manage the accuracy and validity of reporting on entity progress

Participation rate

3.38 To date, the Archives has focused on maximising participation rates for the annual surveys across 160 Australian Government agencies. The Archives has a structured approach to tracking the responses of individual entities, following up entity progress in submitting survey responses, and has previously extended the survey period in order to maximise the response rate.

3.39 Since the launch of the policy, the Archives has had a target response rate of 95 per cent in 2016–17 and 2017–18, increasing to 96 per cent in 2018–19. These targets were met and exceeded in all three years with a 100 per cent response achieved in 2016–17 and 2017–18; and 97 per cent response rate in 2018–19.

Quality assurance

3.40 The quality assurance processes that the Archives has undertaken for each of the annual surveys are outlined at Table 3.3.

Table 3.3: Quality assurance activities on annual survey data

Source: Analysis of Archives’ documentation.

3.41 The survey process in each of the three years since the Digital Continuity 2020 policy was released has required that entities self-assess their progress to implement the recommended actions and associated targets of the policy. As part of the survey, the Archives requests that ‘agency heads’ approve their entity’s responses prior to final submission. The Archives relies on this process to assure the accuracy of the relevant submission.

3.42 Testing conducted on a sample of 30 (18 per cent) of the respondents to the 2016 survey found that 59 per cent of the checked responses had low or partial consistency.⁵⁰ This data forms the basis of reporting on whole-of-government progress in the implementation of the policy. As such, activities should have been undertaken to provide assurance on the accuracy of the 2017 and 2018 survey results.

Have the selected entities achieved the targets of the Digital Continuity 2020 policy?

4.1 The Digital Continuity 2020 – Agency Implementation and Pathways document (implementation guidance) lists the recommended actions under the Digital Continuity 2020 policy (policy), and breaks down these recommended actions into smaller ‘targets’ that entities should progressively implement. The implementation guidance lists a total of 29 targets, 17 of which were to have been implemented on or before 31 December 2018. This audit has assessed the progress that the Attorney-General’s Department (AGD), the Civil Aviation Safety Authority (CASA) and the Office of the Inspector-General of Intelligence and Security (IGIS) have made to implement these targets⁵¹, and the results are presented at a summary level in Table 4.1 below.

Table 4.1: Extent to which entities have implemented targets due prior to 31 December 2018 (as at July 2019)

Source: ANAO analysis of selected entities’ documentation against the Archives’ ‘Targets and Pathways’ document.

Note: Principle 1: Information is valued; Principle 2: Information is managed digitally; Principle 3: information, systems and processes are interoperable.

Principle 1 — information is valued

4.2 Principle 1 — Information is valued, identifies 13 targets that entities are to meet in order to implement the five recommended actions under this principle. Eight of the 13 targets were to have been implemented by 31 December 2018. The remaining five targets are due between September 2019 and December 2020. Table 4.2 lists the eight targets due by 31 December 2018, details the average digital maturity rating as self-reported by each of the selected entities in the 2018 Check-up Plus survey, and compares it with an assessment of the actual progress that the selected entities have made to achieve the targets associated with Principle 1 of the policy.

Table 4.2: Extent to which entities have implemented the targets of Digital Continuity 2020 policy Principle 1

Legend: ◆ fully implemented ▲ partially implemented ■ not implemented

Note a: The entity’s self-assessed average rating has been determined by mapping the recommended actions outlined in the policy to the 11 relevant questions in the Check-Up Plus 2018 annual survey (See Appendix 6). It uses a rating scale of 1 to 5, where Level 1 means rarely/never (not implemented), Level 2 means sometimes (partially implemented); Level 3 means often (partially implemented); Level 4 means usually and/or most of the time (fully implemented); and Level 5 means almost always or always (fully implemented).

Source: Analysis of department documentation.

4.3 The results of this analysis reflect that AGD has implemented all eight targets associated with Principle 1 which were due by 31 December 2018. CASA has implemented six targets, partially implemented one target, and has not implemented one target. IGIS has implemented seven targets, and partially implemented one target.

4.4 Given that the entity self-assessments were completed through the Check-up Plus survey in September 2018, and that a number of activities have been undertaken by entities to progress the implementation of the recommended actions and targets during the course of the audit⁵², the self-assessment ratings for AGD (4.2 out of 5), CASA (3.1) and IGIS (3.9) broadly align with actual progress.

Information governance reporting

4.5 All three of the selected entities submitted responses to the Archives annual agency surveys associated with digital information management between 2016 and 2018 (see paragraphs 3.12 to 3.16) for detail on the associated surveys conducted each year. Each survey response was approved by the relevant agency head. Therefore, all three entities have fully implemented this recommended action, and the associated targets.

Entities have an Information governance framework

4.6 The implementation guidance on the Archives’ website states that an information governance framework ‘is the legal, regulatory and business context within which information assets are created, used and managed.’ Entities are required to set out an approach and commitment to implementing an effective governance framework, and the controls required to maintain it.⁵³ The target associated with this recommended action was due 31 December 2016.

4.7 AGD has a documented information governance framework. The framework is supported by an information management policy, with additional supporting guidance hosted on the department’s intranet. As such, AGD has fully implemented this target.

4.8 CASA does not have a documented information governance framework. However, CASA does have an information management manual that details the legislative, regulatory and policy framework associated with its information governance. Following a 2015–16 internal audit, CASA developed an Enterprise Information Management Strategy that was issued in October 2017. The strategy states that CASA will identify and implement a program of work including:

• completing an information review;
• developing an enterprise information management model;
• developing and implementing an information governance framework; and
• achieving key milestones of the service delivery transformation project.

4.9 Work to implement the components of the strategy relevant to establishing an information governance framework commenced in January 2019 when CASA engaged an external contractor to review the current state of information governance, develop an information governance model, and an implementation plan to guide the development of information governance into the future. This work is not yet complete, and as such, CASA has partially implemented the recommended action and associated target.

4.10 IGIS has an information and records management policy that details: how records are to be created and maintained; procedures for carrying out information management functions; general records management practices; legislation; and a description of the operating environment. However, the policy applies to paper-based (hard-copy) files and information only. The policy has not been updated since 2014, however has been extended while changes to IGIS’ service delivery arrangements and operations are underway, including the transfer of IGIS into the Attorney-General’s portfolio⁵⁴, subsequent move to new premises, the purchase of licenses to install an Electronic Document Records Management System (EDRMS), and steps taken to establish digital information management practices. The existing framework will remain in place until planned changes to IGIS’ operating environment⁵⁵ to implement the recommendations of the 2017 Independent Intelligence Review have been completed.⁵⁶ Therefore, IGIS has partially implemented the recommended action and associated target.

Entities have an Information governance committee

4.11 The implementation guidance on the Archives’ website states that an information governance committee can be established as a board, a working group, or its responsibilities can be absorbed into an existing governance committee. The recommended action and associated target was due 30 June 2016.

4.12 AGD has utilised an existing governance committee — the Information, Communication and Technology Committee (ICTC) — for the purpose of information governance. The terms of reference for this committee state that the purpose of the ICTC is to provide:

• strategic direction for AGD’s ICT environment;
• strategic oversight and governance of the department’s ICT operations, strategies, information governance framework, policies and practices; and
• monitor the implementation of the ICT strategic plan and support compliance with the Digital Continuity 2020 policy.

4.13 AGD revised the terms of reference of the ICTC to include information management responsibilities in February 2016, and again in May 2018. Therefore, AGD has fully implemented the recommended action and associated target.

4.14 CASA has also utilised existing governance structures, however initially elected to split the information governance responsibilities across four committees:

• the Enablement and Capability Group⁵⁷;
• the Protective Security Subcommittee;
• the Service Delivery Transformation Program Board⁵⁸; and
• the Business Improvement Program Board.⁵⁹

4.15 CASA has now consolidated, the Service Delivery Transformation Program Board, and the Business Improvement Program Board into a single committee — the Business Improvement Program and Oversight Board. This board was established, and the first meeting held in April 2019. The terms of reference for this board include providing strategic direction and oversight of programs, projects, and business improvement initiatives aimed at improving the efficiency of operations and processes, including the enabling activities required to support successful implementation such as records and information management. As such, CASA has fully implemented the recommended action and associated target.

4.16 IGIS’ information governance arrangements are comprised of weekly senior staff meetings and monthly all staff meetings.

4.17 These arrangements are sufficient given the small size of the entity and co-location of staff. As such, IGIS has fully implemented this action and associated target. However, the size and jurisdictional responsibilities of IGIS as outlined in the 2018–19 corporate plan are planned to expand in line with recommendations from the 2017 Intelligence review. As such, there would be benefit in IGIS establishing fit-for-purpose governance arrangements, particularly as the management and security of information is a key priority for the entity.

Entities meet targets for skilled staff

4.18 The implementation guidance available on the Archives’ website states that agencies should increase support for their information practitioner’s development ‘to effectively deal with today’s dynamic digital environment, identifying that qualified and skilled information professionals are required.’ The targets associated with this action are intended to support entities meet targets for skilled staff, by:

• establishing a Chief Information Governance Officer (due 31 December 2017); and
• establishing and implementing a program of continuing professional development (due 31 December 2018).

Chief Information Governance Officer

4.19 AGD and CASA have established a Chief Information Governance Officer (CIGO) role. AGD has expanded the Chief Information Officer (CIO) role to include the responsibilities of the CIGO, whereas CASA has elected to split the responsibilities of the role across two existing positions — the CIO and the Branch Manager for Governance.⁶⁰ Therefore, AGD and CASA have fully implemented this target.

4.20 The responsibilities of the CIGO role in IGIS have been allocated to the Director of Enabling Services. For small or micro agencies such as IGIS, CIGO responsibilities can be included into the most senior information management role available. As such, IGIS has fully implemented this target.

Implementation of a program of continuing professional development for information management staff

4.21 As discussed at paragraph 2.16, the Archives released an information management and data capabilities matrix in May 2018 detailing the skills, knowledge, and information management experience required by all staff, information management professionals, and senior executives. This tool was launched to assist entities identify and prepare professional development strategies for their information management workforce.

4.22 AGD developed a continuing professional development strategy for information management staff in December 2018, and the strategy was approved in January 2019. AGD also established all-staff learning and development themes for 2018–19 that include digital and data skills. Therefore, AGD has fully implemented this target.

4.23 CASA has established a project plan to develop e-learning courses for information governance and records management including EDRMS specific courses. According to the project plan, information governance and records management courses were to be finalised and released by mid July 2019. In response to an internal audit, issued in May 2019, CASA stated that development of an information management curriculum to cover information governance and records management was still underway. Therefore, CASA has not implemented this target.

4.24 IGIS has not developed or established a formal program of continuing professional development for information management. However, continuing professional development requirements relating to information management have been included in the personal development agreements for the Director, and Assistant Director of Enabling Services. Given IGIS’ status as a micro agency, there is little value in establishing a formal program of continuing professional development for the limited number of staff with information management responsibilities. As such, IGIS has fully implemented this target.

Principle 2 — Information is managed digitally

4.25 Principle 2 — Information is managed digitally, identifies nine targets that entities are to meet in order to implement the two recommended actions of the policy. Five of the nine targets were to have been implemented by 31 December 2018. The remaining four targets are due between 30 June 2019 and 31 December 2020. Principle 2 identifies that entities are to manage their information digitally with business interactions, decisions, and authorisations recorded digitally, and migrate information in analogue to digital format where there is value for business. Table 4.3 lists the five targets that were due by 31 December 2018, details the average digital maturity rating as self-reported by each of the selected entities in the 2018 Check-up Plus survey, and compares this with an assessment of the actual progress that the selected entities have made to achieve the targets associated with Principle 2 of the policy.

Table 4.3: Entity progress to implement the targets of Digital Continuity 2020 policy - Principle 2

Legend: ◆ fully implemented ▲ partially implemented ■ not implemented

Note a: The entity’s self-assessed average rating has been determined by mapping the recommended actions outlined in the policy to the 11 relevant questions in the Check-Up Plus 2018 annual survey (see Appendix 6). It uses a rating scale of 1 to 5, where Level 1 means rarely/never (not implemented), Level 2 means sometimes (partially implemented); Level 3 means often (partially implemented); Level 4 means usually and/or most of the time (fully implemented); and Level 5 means almost always or always (fully implemented).

Source: Analysis of department documentation.

4.26 Of the five targets due by 31 December 2018, the results of this assessment reflect that AGD has implemented four targets and partially implemented one target. CASA has implemented three targets and partially implemented the remaining two, whereas IGIS has not implemented any of the targets. Overall, the average self-assessment ratings for AGD (3.0), CASA (2.5) and IGIS (1.5) broadly align with actual progress.

Entities work digitally, with business interactions, decisions, and authorisations recorded digitally

4.27 To achieve the targets of the policy entities are to: transform most paper-based business processes to digital; routinely make and record decisions using digital authorisations and workflows; have identified high value and long-term information assets by 31 December 2016; and identified all information assets, evaluated risk and management requirements, and implemented strategies to support implementation of the policy by 31 December 2018.

4.28 To assist agencies implement this target, the Archives released a digital authorisation framework (see paragraph 2.21), and a business systems assessment framework (see paragraph 2.23). The digital authorisation framework is a risk-based assessment tool for transforming analogue approval processes to fit-for-purpose digital approvals. The tool identifies four approval methods that can be applied to transfer paper-based processes to digital — email, action tracking, system workflows and digital signatures. The target associated with this recommended action was to have been implemented by 31 December 2017. The business system assessment framework provides entities with a structured approach to the assessment of information management functionality in business systems based on the value of information and the associated level of risk.

4.29 In April 2016, AGD identified 160 paper-based processes within the department. Of these paper-based processes, 117 required that the associated documents be digitised at the conclusion of the process. In August 2016, AGD released its information management policy which identified that emails are to be used to record business decisions and then transferred into the department’s EDRMS. AGD’s financial, human resource and case management systems have digital workflows embedded. AGD has identified that the 43 remaining paper-based processes will have digital workflows and authorisations embedded where practical as supporting business systems are upgraded or refreshed.

4.30 In March 2019, AGD determined that there was no requirement to complete an information stocktake to identify all information assets, stating that information assets and associated management requirements are identified and evaluated in accordance with agency specific and relevant general records authorities⁶¹, privacy impact assessments⁶², and business continuity and disaster recovery plans.⁶³ The strategy to support digital continuity is for information management requirements to be incorporated into the business system upgrades and/or redevelopment processes (see paragraph 4.56). Work to review the records authorities for AGD has been proposed as the current agency specific records authorities are out of date and do not reflect the full span of information types currently in use. As at June 2019, approval to commence the review and update the records authority has not been provided by AGD.

4.31 As outlined in the paragraphs above, AGD has:

• fully implemented the two targets associated with reducing reliance on paper processes, identifying and evaluating high-value information assets, and identifying strategies to support digital continuity;
• fully implemented the target associated with transforming most paper based business processes to digital processes; and
• partially implemented the target associated with the identification and evaluation of all information assets and implementation of strategies to support digital continuity.

4.32 CASA’s Electronic Transactions policy allows for electronic approvals and deems them the equivalent of a physical signature. The Electronic Transactions policy was due to be reviewed in 2016. However, the review did not take place and the policy does not align with the Digital Continuity 2020 policy. As part of a broader Service Delivery Transformation Project, CASA has commenced work to develop and implement digital authorisation and workflows into ICT systems, and has commenced a forms improvement project. This work is ongoing, and a recent internal audit identified instances where new hard copy files were being created, and of decisions being stored in personal drives. As such, CASA has partially implemented this target.

4.33 To identify information assets and strategies to support digital continuity, CASA has developed an information asset register, and completed a business assessment for eight high value business systems. The information asset register states that 99.5 per cent of CASA’s information assets are in digital format; and identifies the desired future state for each remaining information asset. As part of the Enterprise Information Management Strategy, CASA has engaged an external contractor to assist identify information assets and implement strategies to support digital continuity. While work has commenced, it is not yet complete, and has not been rolled into business as usual practices.

4.34 CASA has therefore:

• fully implemented the two targets associated with reducing reliance on paper, and identifying high-value information assets and strategies to support digital continuity;
• partially implemented the target to transform paper based processes to digital processes; and
• partially implemented the target to identify all information assets, evaluate risk, identify and implement strategies to support digital continuity.

4.35 IGIS has not developed or implemented digital authorisations and workflows into business processes. While emails may record decisions, the email record must then be printed and filed in accordance with IGIS’ information and records management policy, which states that ‘the electronic version of any document is regarded as an unreliable reference and must not be used for decision making’. As at June 2019, IGIS has procured licences for, and is in the process of building and installing an EDRMS, and finalising the design and functional specifications of a case management system. When completed, the installation of these two systems should allow IGIS to incorporate digital authorisations and workflows into some business processes. However, work to implement these systems is not yet complete and as such IGIS cannot be considered to have implemented this target.

4.36 In March 2019, IGIS conducted a stocktake of its hardcopy information assets as part of moving to new premises. As at June 2019, IGIS has procured licences for, and is in the process of building and installing an EDRMS, and finalising the design and functional specifications of a case management system. IGIS has stated that a digital transformation project is planned for the second half of 2019 and that it intends to establish a process whereby all new records will be managed digitally by the end of 2019.

4.37 IGIS handles classified and sensitive data on behalf of other agencies. The Protective Security Policy Framework (PSPF) states that material which is subject to an information security caveat must be handled in accordance with any special handling requirements imposed by the originator and caveat owner. Accordingly, there are instances where IGIS is subject to policy direction by an originating entity to retain particular information in hard copy only.⁶⁴ IGIS has indicated, therefore, that some paper-based records may not ever be suitable for digitisation.

4.38 Whilst noting this context, IGIS cannot be considered to have met the recommended action and associated target in relation to this principle of the policy.

Information in analogue format is migrated to digital format where there is value for the business

4.39 To implement this recommended action, entities are to have achieved targets in relation to digitally managing all records created after 1 January 2016.

4.40 AGD’s information management framework stipulates that all business records must be captured into the approved EDRMS, unless there is a specific exemption in place⁶⁵, and a digitisation guide has been created to support the migration of analogue documents into digital formats. In June 2017, AGD reported that 90 per cent of the department’s information is created and managed digitally, with the remaining 10 per cent of files created and managed physically. AGD has therefore fully implemented the target to appropriately manage records created in digital format.

4.41 CASA’s information management manual requires that staff capture and manage all business records in electronic format in a compliant recordkeeping system⁶⁶, and CASA has created a digitisation guide to support the migration of analogue documents into digital formats. In 2017, CASA developed an Enterprise Information Management Strategy, which sets out a program of work over three years (2017–2020) that includes ensuring that information in analogue formats is migrated to digital format where there is value for money. CASA has also commenced work to identify and catalogue physical records holdings. CASA has therefore fully implemented the target to appropriately manage records created in digital format.

4.42 All IGIS information assets are currently paper-based, with any records created digitally required to be printed and stored in hardcopy. IGIS’ information and records management policy states that ‘the electronic version of any document is regarded as an unreliable reference and must not be used for decision making’. As such, IGIS has not implemented this target.

Principle 3 — Information, systems and processes are interoperable

4.43 Principle 3 — Information, systems and processes are interoperable identifies seven targets that entities are to satisfy in order to implement the three recommended actions under this principle. Of the seven targets, four were to have been implemented by 31 December 2018. The remaining three targets are due to be implemented by 31 December 2020.

4.44 Under Principle 3, entities are to have interoperable information systems and processes that: meet standards for short and long-term management; improve information quality; and enable information accessibility, management, and reuse. The Archives have developed two tools to support entities achieve the associated targets due by 31 December 2018 and embed the principles of the policy:

• a business systems assessment framework⁶⁷; and
• a minimum metadata set.⁶⁸

4.45 Table 4.4 lists the four targets which were due by 31 December 2018, details the average digital maturity rating as self-reported by each of the selected entities in the Check-up Plus annual survey conducted in 2018, and compares it with an assessment of the actual progress that the selected entities have made to achieve the targets associated with Principle 3 of the policy.

Table 4.4: Recommended actions, associated targets, and target dates for Principle 3

Legend: ◆ fully implemented ▲ partially implemented ■ not implemented

Note a: The entity’s self-assessed average rating has been determined by mapping the recommended actions outlined in the policy to the 11 relevant questions in the Check-Up Plus 2018 annual survey (see Appendix 6). It uses a rating scale of 1 to 5, where Level 1 means rarely/never (not implemented), Level 2 means sometimes (partially implemented); Level 3 means often (partially implemented); Level 4 means usually and/or most of the time (fully implemented); and Level 5 means almost always or always (fully implemented).

Source: Digital Continuity 2020 policy, October 2015, p. 5.

4.46 Of the four targets due to be implemented by 31 December 2018, the results of this assessment reflect that AGD has fully implemented two targets, and partially implemented two targets. CASA has fully implemented one target and partially implemented three targets. IGIS has partially implemented three targets and not implemented one target. Overall, the average self-assessment ratings for AGD (3.0), CASA (2.25) and IGIS (2.0) broadly align with actual progress.

Information is managed based on format and metadata standards for information governance and interoperability

New business information systems

4.47 The first target for this recommended action states that all business systems procured after 31 December 2016 must meet minimum metadata standards, and should be evaluated against the Archives’ business systems assessment framework to meet functional requirements for information management. Digital Continuity 2020 guidance material states that for new systems the ability to capture the minimum metadata should be included as a requirement in procurement documentation.

4.48 Since 31 December 2016, AGD has purchased two new modules for an existing business system.⁶⁹ The procurement and solution architecture documentation did not state that the business system must meet the minimum metadata standards. In January 2019, AGD advised that checks are being built into governance arrangements to rectify this oversight. The Attorney-General’s Approved Technologies Committee (AGTAC) oversees the certification process to ensure that new IT equipment and software is assessed as compatible with existing infrastructure. However, the terms of reference for the AGTAC do not identify minimum metadata and information management functionality requirements as one of the criteria necessary to achieve AGTAC certification. As such, AGD has partially implemented this target. AGD should update the terms of reference for the AGTAC to ensure that minimum metadata standards and information management functionality requirements are addressed when procuring new business system applications, versions, or modules of software in the future.

4.49 CASA also procured two new business system after 31 December 2016, and did not confirm that the systems would meet the minimum metadata standards or evaluate the system against the Archives’ business system assessment framework. Subsequently, CASA updated the documentation it uses to set out the technical requirements for new ICT products. The update includes the requirement that new products be evaluated with reference to the Archives’ Business System Assessment Framework. Further, new products are required to comply with the metadata standards. This documentation is to be included with any procurement. As such, CASA has partially implemented this target.

4.50 In January 2019, IGIS procured licenses for an EDRMS and a case management system. As at July 2019, the specifications for the case management system are being confirmed. The procurement was managed by a separate government entity on IGIS’ behalf. The EDRMS meets the Archives’ minimum metadata and information management functional requirements. However, the system has not yet been installed, and the specifications for the case management system are still to be configured to ensure that the metadata standards are met and information management function requirements are embedded. As such, IGIS has partially implemented this target.

Existing business information systems

4.51 The second target under this recommended action applies to existing business systems and requires that entities’ business systems containing high-value and long-term information assets meet minimum metadata standards by 31 December 2017. The Archives have developed a minimum metadata set that identifies nine properties required for the effective management of business information (see paragraph 2.17).

4.52 In April 2016, AGD identified seven existing business systems containing high value and long-term information assets that required assessment to determine if the systems met the minimum metadata standards. From October through to December 2016, AGD completed assessments, identifying that the business systems met the minimum metadata standard and captured sufficient metadata to meet business and information management needs. As such, AGD has fully implemented this target.

4.53 As at June 2019, CASA has identified eight existing high-risk, high-volume business systems and conducted assessments of these. CASA has identified that these systems meet the Archives’ minimum metadata standard. Therefore, CASA has fully implemented this target.

4.54 IGIS’s business operations utilise email correspondence, Microsoft Word and Microsoft Excel spreadsheets stored on group drives, and an Access database to manage complaints. There is no evidence that metadata assessments have been undertaken of these existing systems, and in accordance with the IGIS’ information and records management policy, the output from the processes conducted using these systems is to be printed out and stored on the hard-copy file. As such, IGIS has not implemented this target.

All business systems meet functional requirements for information management

4.55 The second recommended action under Principle 3 requires that all business systems meet functional requirements for information management. The Archives have developed a business systems assessment framework that entities can use to assess information management functionality.⁷⁰ The framework recommends that entities develop a systems register to: identify business systems that create, capture, and store information; prioritise those systems that hold high risk, high-value business information; and develop and implement a system information management plan.

4.56 In 2018, AGD revised its approach and ceased using the business system assessment framework developed by the Archives to determine information functional requirements for all business systems. Instead, functional requirements for information management in relation to all business systems are addressed through the systems design process.⁷¹ However, information and records management requirements have not been consistently addressed in documentation submitted to, and reviewed by, the Design Authority. As such, AGD commenced work to adapt the relevant templates to incorporate records and information management requirements in July 2018, and the revised templates were released in October and November 2018. While AGD has developed processes to ensure that functional requirements for information management are evaluated as part of the systems design process, it has also identified that work to implement information management functionality into all business specific systems is not yet complete. Therefore, AGD has been assessed as having:

• fully implemented the target to evaluate the functional requirements for information management; and
• partially implemented the target to implement the functional requirements where necessary.

4.57 CASA has not yet completed an evaluation of all business systems. In 2017, CASA engaged an external contractor to begin undertaking this work as part of an enterprise information architecture review and a business systems assessment register has been developed as part of the first stage of this work. The full program of work is identified in the Enterprise Information Management Strategy, scheduled over a three year period, and due to be completed in 2020. CASA has commenced the work identified in the Enterprise Information Management Strategy and has partially implemented this target.

4.60 As discussed at paragraph 4.10, the EDRMS that IGIS is in the process of procuring and installing meets the Archives’ minimum metadata and information management functional requirements. However, the system has not yet been installed, and the specifications for the case management system are still to be configured. Corporate systems, such as payroll and human resources are provided by AGD, and AGD is responsible for ensuring that these systems meet functional requirements for information management. As such, IGIS has partially implemented this target. Recommendation no. 7 at paragraph 4.67 addresses the need for IGIS to establish a plan to address this and other elements of the Digital Continuity 2020 policy targets.

Have the selected entities established effective internal arrangements to monitor and report on progress against the targets of the Digital Continuity 2020 policy?

Internal monitoring and reporting

4.61 The Digital Continuity 2020 policy does not include a specific requirement for entities to monitor and report internally on progress towards achieving the policy. However, guidance issued by the Archives sets out that entities are expected to be reporting to their information governance committee or equivalent mechanism (see Appendix 7) on progress to implement whole-of-government information management initiatives (including the Digital Continuity 2020 policy).

Attorney-General’s Department

4.62 AGD’s Information Management Framework outlines that the effectiveness of the framework will be assessed using: annual reporting as required under the Digital Continuity 2020 policy; and in-progress reports to the department’s Information and Communications Technology Committee (ICTC).

4.63 In 2016, AGD expanded the terms of reference for the ICTC to include information governance. The ICTC is to receive reports on the status of activities initiated to implement the targets of the Digital Continuity 2020 policy every two months (February, April, June, August, October and December) or as required, and provide a formal report on activities to the Executive Board at least twice a year.

4.64 A review of the minutes and papers discussed at the ICT Committee meetings found that papers discussing the status of activities implemented to assist AGD in meeting the targets of the policy have been tabled and discussed at six meetings held between February 2016 and March 2019. The papers presented to the ICTC focused on reporting progress against the Digital Continuity 2020 targets and seeking endorsement, or approval of, initiatives to be implemented to meet upcoming targets.

Civil Aviation Safety Authority

4.65 CASA does not have an information governance framework, or a dedicated information governance committee. However, CASA does provide regular information management input into weekly executive manager reports, and provided high-level reports mapping progress against the targets of the Digital Continuity 2020 policy to the Enabling and Capability Group. Updates on progress against the targets of the Digital Continuity 2020 policy were tabled and discussed at two meetings held in December 2017 and January 2018. Where specific business improvement projects have been initiated that will assist CASA to achieve the targets of the policy, program updates were provided to either the Enablement and Capability Group, Service Delivery Transformation Program Board, or the Business Improvement Program Board. In April 2019, the Service Delivery Transformation Program Board and Business Improvement Program Board were consolidated into a single Business Improvement Program and Oversight Board. Reports are now provided to this body on projects implemented to progress the recommended actions and associated targets of the Digital Continuity 2020 policy.

Inspector-General of Intelligence and Security

4.66 IGIS has not established internal monitoring and reporting arrangements for the Digital Continuity 2020 policy. IGIS’s governance arrangements are outlined in its corporate plan and are comprised of weekly senior staff meetings, and monthly all staff meetings. The Digital Continuity 2020 policy, and progress against it, are not specifically discussed at these meetings, however projects related to the procurement and installation of an EDRMS and a case management system have been discussed regularly.

Appendix 1 Entity responses

 

 

 

 

ANAO comment on National Archives of Australia response

(a) The Digital Continuity 2020 policy statement published by the Archives in 2015 established the objective that Australian Government entities will have embedded the principles of the policy by 31 December 2020, and stated that the Archives:

• is responsible for leading the implementation of the policy;
• will collaborate with entities to develop advice, products and tools to support implementation of the policy principles; and
• will undertake performance monitoring to identify and work with entities that need assistance in implementing the policy.

The audit assessed the Archive’s performance against this objective, and its discharge of these responsibilities, and found that: the Archives arrangements to administer the Digital Continuity 2020 policy are limited in effectiveness (paragraph 8); that while the products, advice, and tools issued by the Archives to support entities are largely fit for purpose, there are material deficiencies (paragraph 12); that there are material deficiencies in performance monitoring (paragraphs 15 and 16); there are no formal processes to identify and work with entities that need assistance in implementing the policy (paragraph 13); and that Australian Government entities are unlikely to have embedded the principles of the policy by the 31 December 2020 target (paragraph 7).

On this basis, the audit has concluded that the Archives has been largely ineffective in monitoring, assisting and encouraging entities to meet the targets of the policy.

(b) The figure quoted in the above statement is based on the number of agencies that have self-assessed as having a digital maturity level of 3 or above in the Check-Up Digital and Check-up Plus surveys conducted in 2016 and 2018 respectively. As stated at paragraph 3.20, the Archives has itself identified that using this maturity scale to measure success is potentially inaccurate.

 

 

 

 

 

 

Appendix 2 Digital Continuity 2020 targets and pathways

Table A.1: Digital Continuity 2020 Agency Implementation targets and pathways

Source: National Archives of Australia, ‘Agency implementation targets and pathways,’ available from http://www.naa.gov.au/information-management/digital-transition-and-digital-continuity/agency-implementation-targets-pathways/index.aspx [accessed 10 April 2019].

Appendix 3 The information management legislative, regulatory, and policy environment

1. The legislative, regulatory, and policy environment that applies to information management spans multiple portfolio bodies, and independent bodies, is characterised by high levels of interdependency, and is comprised of:

• legislation, legislative instruments and standing orders⁷²;
• whole-of-government policies and strategies⁷³;
• records and information management standards and authorities⁷⁴; and
• subject matter guidance and advice.⁷⁵

2. The lead agencies responsible for overseeing compliance with the key pieces of legislation that detail how Australian Government agencies are to handle, protect, and manage information are listed below at Table A.2.

Table A.2: Lead agency and associated legislation that relates to information management

Source: ANAO analysis of legislation that guides the handling and management of information by Australian Government entities. Information available from http://www.naa.gov.au/information-management/information-governance/legislation-standards/index.aspx [accessed 10 April 2019].

3. The supporting standards, whole-of-government policies, and strategies that Australian Government entities are to implement (including the Digital Continuity 2020 policy) to meet legislative and regulatory requirements are detailed at Table A.3 below.^.76

Table A.3: Supporting standards and whole-of-government policies that apply to information management

Note a: The Archives has categorised whole-of-government strategies and policies as required practice. Required practices are practices that entities must be aware of, and implement to the level required as defined in the relevant strategy or policy.

Source: Analysis of the, ‘Legislation, policies, standards and advice,’ available the National Archives of Australia website: http://www.naa.gov.au/information-management/information-governance/legislation-standards/index.aspx [accessed 10 April 2019].

Appendix 4 Criteria used to select entities

1. An overview of the entities, and the criteria used to select them for audit coverage, is provided in Table A.4.

Table A.4: Selection of entities

Note a: The entity’s self-assessed digital maturity level has been based on their responses to the 2016 Check-up Digital survey.

Note b: When conducting inspections, inquiries and investigations into complaints, IGIS collects classified and sensitive data from the agencies involved. In accordance with the protective security policy framework classified and sensitive data is to be handled and accessed in a manner that complies with the originating entities instructions and requirements.

Source: Analysis of 2016 Check-up Digital responses.

Appendix 5 Criteria used to assess the appropriateness of performance information

Table A.5: Criteria to assess the appropriateness of performance information

Appendix 6 The average digital maturity assessments of the selected entities for 2018

Table A.6: Selected entities average digital maturity assessments for 2018

Source: Check-up PLUS Survey 2018.

Appendix 7 Internal monitoring and reporting guidance for information management

Table A.7: Archives’ guidance — internal monitoring and reporting

Source: The National Archives of Australia website: http://www.naa.gov.au/information-management/information-governance/ [accessed 10 April 2019].