Introduction

1.1 The value of Australia’s agricultural production is forecast to rise by six per cent to $85 billion in 2024–25.⁸ Australia exports approximately 72 per cent of the total value of agricultural, fisheries and forestry production.⁹ The Australian Government regulates the export of agricultural, fisheries and forestry products, issuing export documentation that verifies that the goods being exported meet both the Australian export requirements and the importing country’s requirements.¹⁰

1.2 The entity responsible for the regulation of the export of agricultural, fisheries and forestry products is the Department of Agriculture, Fisheries and Forestry.¹¹ The report refers to the entity as ‘the department’, unless distinction is required. During the period covered by the audit, the entity responsible for the regulation of the export of agricultural, fisheries and forestry products has been the: Department of Agriculture and Water Resources (from 29 May 2019 to 31 January 2020); Department of Agriculture, Water and the Environment (from 1 February 2020 to 30 June 2022); and Department of Agriculture, Fisheries and Forestry (from 1 July 2022 to date).

Digital reform of agricultural export systems

1.3 The department uses information and communications technology (ICT) systems to regulate and facilitate the export of agricultural, fisheries and forestry products and to issue export documentation, including the:

• Export Documentation System (EXDOC), which is used ‘to generate export documentation’ to ‘export prescribed primary produce from Australia’¹²;
• New Export Documentation System (NEXDOC), which is used to ‘generate export documentation’ and ‘introduces new features and enhances existing ones to streamline the export documentation process’¹³;
• Export Establishment Registration Database (ER), which is used to store information relating to registered establishments that prepare, store, handle and/or present for inspection prescribed goods for export¹⁴;
• Tracking Animal Certification for Export system (TRACE), which is used to manage ‘the application and approval processes for consignments of livestock and animal reproductive material exported from Australia’¹⁵;
• Plants Export Management System (PEMS), which is used to ‘capture and store information relating to the export of plants and plant products from Australia; including plant export Authorised Officer (AO) inspection and calibration results for product and transport units and all supporting documentation’¹⁶; and
• Manual of Importing Country Requirements (Micor), which is a resource that provides guidance for exporters on importing country requirements.¹⁷

Export Certification Modernisation and Digitisation

1.4 In the 2019–20 Mid-Year Economic and Fiscal Outlook, the Australian Government committed $29.2 million over four years to ‘streamline export processes by completing the delivery of a digital export certification management system, which will provide a modern and secure approach to assuring that produce meets importing country requirements’.¹⁸ This measure is referred to by the department as the Export Certification Modernisation and Digitisation (ECMOD) measure.

Busting Congestion for Agricultural Exporters package

1.5 In the 2020–21 Budget, the Australian Government committed $328.4 million over four years for a package of measures titled Busting Congestion for Agricultural Exporters. The package aimed to ‘transform Australia’s weak and outdated systems and processes into a cost-effective model to get products to export markets faster and more efficiently’ and ‘establish modern digital services, reduce regulatory cost and administration and improve interactions with export systems’.¹⁹

Digital Services to Take Farmers to Market

1.6 The Digital Services to Take Farmers to Markets measure, which was described as the ‘centrepiece of the reform package’, accounted for $222.2 million of this funding and was intended to:

modernise Australia’s agricultural export systems by slashing red-tape and improving regulation and service delivery for our producers and exporters. This measure will transition our systems online and provide a single portal for transactions between exporters and government, streamlining processes for exporters and helping them experience faster and more cost-effective services.²⁰

1.7 In March 2022, the department received an additional $127.4 million over four years ‘to continue and expand the Digital Services to Take Farmers to Market initiative to transform the delivery of Government agricultural export systems’.²¹

Agricultural exports

1.8 The gross value of Australian agricultural production has increased by 51 per cent in the last 20 years in real terms (adjusted for consumer price inflation), from approximately $62.2 billion in 2003–04 to $94.3 billion in 2022–23.²² When including fisheries and forestry, the total value of agricultural, fisheries and forestry production has increased by 46 per cent, from approximately $68.5 billion in 2003–04 to $100.1 billion in 2022–23.²³

1.9 In real terms, the value of agricultural exports has fluctuated between $44 billion and $80 billion annually since 2003–04. In the three years to 2019–20, Australia exported approximately 72 per cent of the total value of agricultural, fisheries and forestry production. The proportion of goods exported varied by commodity, with more than 80 per cent of rice and sugar being exported on average from 2017–18 to 2019–20, compared to less than 10 per cent of pig and poultry (Figure 1.1).²⁴

Figure 1.1: Share of agricultural production exported by sector

Note: The information presented is a three-year average from 2017–18 to 2019–20.

Source: ANAO representation of Australian Bureau of Agricultural and Resource Economics and Sciences (ABARES), ABARES Insights: Snapshot of Australian Agriculture 2024, March 2024, figure 6.

Rationale for undertaking the audit

1.10 The effective administration of the digital reform of agricultural export systems is intended to minimise disruption to exports and provide exporters with the benefits of faster, more reliable and cost-effective export services.

1.11 Past external reviews and ANAO performance audits of the department have found weaknesses in the department’s governance and culture, as well as its arrangements to manage its performance as a regulator.²⁵

1.12 Large-scale ICT improvement programs aimed at uplifting or replacing aging ICT systems are increasingly common across Australian Government entities. Recent audits of other ICT improvement programs have found weaknesses in monitoring and reporting on the program’s status and performance, which increases the risk that the program fails to deliver outcomes and limits effective measurement of benefits realisation.²⁶

1.13 This audit provides assurance to Parliament on the effectiveness of the department’s administration of the digital reform of the agricultural export systems.

Audit approach

Audit objective, criteria and scope

1.14 The objective of the audit was to assess the effectiveness of the department’s administration of the digital reform of the agricultural export systems.

1.15 To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria.

• Has the department established effective governance arrangements for the program?
• Is the department implementing the program effectively?
• Is the department managing change for the program effectively?

1.16 The Australian Government has been investing in the digital reform of the agricultural export systems through a series of measures (see paragraphs 1.4 to 1.7).

1.17 The audit focused on the department’s administration of the package of work approved by the Australian Government in October 2020 and relevant in-flight initiatives. This work is being delivered in three tranches, the first of which was scheduled to conclude at the end of 2022–23. The audit focused on the delivery of the first tranche (Tranche 1).

1.18 This package of work is funded by the Digital Services to Take Farmers to Market budget measure and builds on elements of work undertaken under previous measures, such as the delivery of a digital export certification management system. The report refers to this collectively as ‘the program’.

1.19 The audit did not examine:

• the effectiveness of individual initiatives or projects administered by the program;
• the delivery of digital initiatives that are not related to the export systems; or
• whole-of-government initiatives such as the Simplified Trade System.²⁷

Audit methodology

1.20 The audit methodology included:

• examining the department’s documentation, with a focus on documents that relate to the program’s governance, strategic planning, benefits realisation, financial management and monitoring and reporting;
• examining system incident logs and accreditation documentation;
• walkthroughs to demonstrate the system capabilities reported as having been achieved;
• reconciliation of contracts for the program with AusTender data;
• reconciliation of program financial records with departmental financial records; and
• meetings with relevant department staff.

1.21 The ANAO received no submissions from the public via the citizen contribution facility on the ANAO website.

1.22 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $622,700.

1.23 The team members for this audit were Casey Mazzarella, Jake Farquharson, Sky Lo, Michelle Penalurick, Talia Song, Dale Todd, Nathan Daley, Naveed Nisar, Jamie Lee and Corinne Horton.

2.1 The department identified the need for investment in the reform and ICT²⁸ modernisation of Australia’s agricultural export systems. The Commonwealth Digital and ICT Oversight Framework (IOF) includes the following key elements that support the government to manage digital and ICT-enabled investments:

• developing and presenting detailed business case(s) for consideration by the Australian Government to inform its decision on investment in the program of work proposed²⁹;
• establishing and maintaining appropriate governance and assurance arrangements³⁰; and
• establishing and maintaining effective risk management arrangements.³¹

Is the program supported by an appropriate business case?

First and second pass business case

2.2 Digital and ICT-enabled policy proposals with financial implications of $30 million or more are subject to the ICT Investment Approval Processes (IIAP). These proposals go through a staged government approval process. At each stage of approval, the agency is required to develop a business case. This ‘ensures that the Cabinet and its relevant committees have sufficient information about the proposal to make an informed investment decision’.³²

2.3 In October 2018, the department presented a first pass business case (1PBC) to the Australian Government for a package of reforms to modernise Australia’s agricultural trade. It outlined the need for digital reform, advising that international trade is becoming increasingly digital and that the volume of exports and industry participants as well as the number of export certificates issued by the department is increasing. It explained that importing country regulatory requirements are becoming more sophisticated, which increases the complexity of export certification and associated compliance activities. The 1PBC detailed the department’s technical environment, business problem, stakeholder impact and risks.

2.4 In July 2020, the department presented the second pass business case (2PBC) to the Australian Government for consideration. The 2PBC provided an update on the department’s operating environment and advised that the program will deliver 16 modern digital services from which three financial and three non-financial benefits would be realised.

2.5 The 2PBC proposed that the program would be delivered over six years from 2020–21, delivered in three ‘tranches’ with each tranche comprising two years. The 2PBC outlined packages of work that the program would deliver. These packages of work were referred to as ‘initiatives’ and were described as ‘outcomes-based’, with each initiative contributing to the development of one or more digital services.

2.6 In October 2020, the Australian Government agreed to proceed with Tranche 1, with options to proceed with the subsequent tranches to be considered in the 2022–23 Budget. The Australian Government approved funding for Tranche 1 over four years from 2020–21 to 2023–24. The Minister for Agriculture, Drought and Emergency Management announced that the Australian Government was investing in a ‘suite of reforms [that] will modernise Australia’s export systems by slashing red-tape and streamlining regulation and service delivery for our farmers’.³³ The Minister explained that this included:

$222.2 million over 4 years for digital services to take farmers to market. This will deliver a modern and reliable digital service to help farmers do business quickly and cost effectively – a single touchpoint for exporters that is available 24/7.³⁴

2.7 The package of work outlined in the 2PBC is funded by the Digital Services to Take Farmers to Market budget measure and builds on elements of work undertaken through previous measures, such as the delivery of a digital export certification management system. The report refers to this collectively as ‘the program’.

Business case addendum

2.8 In April 2021, the first mid-stage Department of Finance Gateway Review³⁵ of the program found that there was a disconnect between the work being delivered by the department and the program of work outlined in the 2PBC. It noted that:

The 2PBC included a Benefits Realisation Framework of investment outcomes, benefits resulting from these outcomes (that are both financial and non-financial), change priorities, key capabilities and key initiatives (outputs). This framework is appropriate however needs to reflect the pivot that the program has recently undertaken.

2.9 The Gateway Review noted that ‘a benefits realisation plan does not currently reflect this pivot or how this significant change to the program will be implemented’. The Gateway Review recommended that the department ‘develop a business case addendum to reflect the revised approach to deliver business value earlier’. Benefits realisation is discussed at paragraphs 4.26 to 4.51.

2.10 In October 2021, the department sought funding for future tranches of the program as part of a broader whole-of-government reform agenda to simplify Australia’s international trade. The department provided the Australian Government with a Business Case Addendum (BCA) to document the department’s implementation of the program.

2.11 The BCA included a program roadmap, which listed the initiatives that would be delivered by the program. The roadmap provided an indication of the start and end date for 63 initiatives, mapped across three tranches. Tranche 1 was scheduled from 2020–21 to 2022–23; Tranche 2 was scheduled from 2023–24 to 2024–25; and Tranche 3 was scheduled for 2025–26.

2.12 The program roadmap listed 31 initiatives that would commence in Tranche 1 (with 28 to be completed in Tranche 1 and three to be completed in Tranche 2); 31 that would commence in Tranche 2 (with 17 to be completed in Tranche 2 and 14 to be completed in Tranche 3); and one that would commence and be completed in Tranche 3.

2.13 In March 2024, the department advised the ANAO that ‘the initiatives described within the [BCA] program roadmap … are intended to replace those listed in the 2PBC’.

2.14 The BCA listed four outcome statements that described benefits arising from the completion of the program, including making industry’s export experience and the department’s regulation of exports easier and improving the quality of data and systems.

2.15 The BCA listed three financial benefits and one non-financial benefit that would be realised by the program. More information about the program’s benefits is at paragraphs 4.26 to 4.51.

2.16 The second mid-stage Department of Finance Gateway Review stated that its recommendation had been fully addressed:

The business case addendum has been developed and included as an attachment for the comeback to government.

…

The review team notes that the program has informed Government on the updated baseline of the TFTM Program within the existing timeframes, with some deliverables extending [into] Tranche 2.

Measurable outcomes

2.17 The 2PBC outlined the initiatives that would be delivered by the program and the 16 digital services that would be developed by this work. The 16 digital services were largely documented as measurable statements of capability, such as sign in once to access all functionalities; access to information is managed based on authorisation levels; and digital certifications by default, paper certification by exception.

2.18 The BCA listed 63 initiatives that would be delivered by the program and four outcome statements. The outcome statements described benefits that are anticipated to arise from the program. The BCA does not document how the outcome statements will be measured.

2.19 Measurable outcomes are an important element of effective oversight and accountability. Without measurable outcomes, the department’s ability to effectively monitor and report on the achievement of the program’s implementation (see paragraphs 4.54 to 4.75) is limited. This also limits the department’s ability to determine whether it is on track to deliver the program on time and within budget.

2.20 By focusing on the delivery of the program’s initiatives without reference to or consideration of how they will deliver the outcomes that the program has committed to deliver (digital services and outcome statements), there is a risk that the work being undertaken by the program may not effectively achieve the program outcomes or benefits.

Is the program supported by appropriate oversight?

Taking Farmers to Market Program

2.23 In October 2019, the department started drafting the Taking Farmers to Market Program Management Plan (Program Management Plan).

2.24 In February 2021, the Trade Reform Board (TRB) approved the operating model for digital trade initiatives in the portfolio, establishing the Taking Farmers to Market (TFTM) program to deliver the program of work funded by the Digital Services to Take Farmers to Market budget measure (the program).

2.25 In March 2021, the department commenced drafting the Taking Farmers to Market Governance Plan (Governance Plan).

2.26 In April 2021, the Program Management Plan was finalised and approved. In October 2021, the Governance Plan was approved.

2.27 The Program Management Plan states that the program is intended to deliver a ‘suite of contemporary and connected digital services for exporters, reducing the administrative burden by streamlining the multiple manual processes and reducing the associated effort for business and the department’. It explains that:

The program’s focus is the digital transformation of Australia’s agricultural export systems to help get agricultural products to market faster. It will deliver a modern and reliable digital service to help exporters do business quickly and cost effectively – a single touch point for exporters that is available 24/7.

2.28 From October 2021, work related to the development and delivery of a digital export certification management system (NEXDOC) and transition of commodities to NEXDOC was included in the program.³⁶

Governance structure

2.29 The Deputy Secretary of the Agricultural Trade Group is the Senior Responsible Officer (SRO) for the program. The SRO is accountable to the accountable authority³⁷ for the delivery of the agreed program outcomes and the realisation of program benefits.

2.30 The Program Management Plan and the Governance Plan document the program’s governance arrangements. Figure 2.1 illustrates the governance structure from June 2022 to May 2023. In May 2023, the program’s governance arrangements were restructured (this is discussed in more detail at paragraphs 2.32 to 2.34).

2.31 The program’s governance bodies are outlined in Table 2.1.

Figure 2.1: Program governance structure, June 2022 to May 2023

Source: Adapted from Governance Plan.

2.32 In March 2023, the TRB noted that the ‘Trade Reform Portfolio Office (TRPO) has been required to adjust its service offering since December 2022, in light of resource reductions due to the department’s difficult FY2022–23 financial position’.

2.33 In May 2023, the TRB agreed that it would be stood down and the Trade Reform Portfolio Office would cease providing ‘all governance and support services to the Trade Reform Portfolio of programs’. Support services such as the Communications Team and the Change Management Team are discussed from paragraphs 3.59 to 3.63 and in Table 4.1 respectively.

2.34 The TRB agreed that ‘all digital, and digitally connected projects [be] governed solely by the Executive Delivery Group’. Figure 2.2 illustrates the program’s governance structure from May 2023.

Figure 2.2: Program governance structure, from May 2023

Source: Adapted from Governance Plan.

Trade Reform Portfolio Office

2.35 The Trade Reform Portfolio Office (TRPO) was established in February 2021 to provide ‘support for strategy, investment, governance’ and manage ‘portfolio-level risks and dependencies that could compromise benefits realisation’. The TRPO ceased in May 2023.

Digital Results Office

2.36 The Digital Results Office (DRO) was established in February 2021 to ‘drive the planning and delivery work required to achieve program outcomes and benefits’. The DRO ‘supports program operations and ensures that the program remains strategically aligned to the broader portfolio’.

Program plans and strategies

2.37 In addition to the Program Management Plan and Governance Plan, the department has established several plans and strategies for the program, including:

• Sourcing Strategy (approved in January 2020);
• Risk and Issue Management Strategy (approved in September 2021);
• Assurance Strategy (approved in October 2021);
• Benefits Management Plan (approved in December 2021);
• Financial Management Plan (approved in December 2022);
• Reporting Plan (dated January 2023); and
• Risk Management Plan (approved in June 2023).

2.38 In March 2024, the department updated the program plans and strategies to remove references to the TRB.

2.39 The program is also currently using plans and strategies from the former Trade Reform Portfolio. For example:

• Trade Reform Communication Strategy (dated July 2022) (see paragraphs 3.59 to 3.61);
• Trade Reform Change Management Strategy (approved in September 2022) (see paragraphs 4.3 to 4.6); and
• Trade Reform Portfolio Benefits Management Strategy (dated September 2021) (paragraph 4.27).

2.40 These strategies relied on governance and other support structures and bodies that ended with the dissolution of the TRB and cessation of the TRPO in May 2023.

Oversight

2.41 The program’s governance frameworks and bodies have been established to support the SRO in ensuring that the program delivers the agreed outcomes and benefits. Governance bodies receive reporting regarding the status of the program’s initiatives (see paragraphs 4.59 to 4.75); risks and issues (see paragraphs 2.73 to 2.77); and the program budget (see paragraphs 3.40 to 3.43).

2.42 Program reporting is limited to short-term delivery goals and does not report on progress achieving the outcomes of the program as a whole, which the SRO is accountable to deliver. This may limit the SRO’s ability to effectively oversee the program’s progress and to make informed decisions regarding the direction of the program.

Assurance arrangements

2.43 In June 2021, the department engaged Terrace Services as a program assurer. Terrace Services³⁸ was contracted to:

• ‘develop a plan for assurance activities across the life of the program’;
• ‘in line with the agreed plan … conduct assurance activities and report findings and make recommendations to the SRO and Program Director’; and
• ‘at the discretion of the SRO and Program Director … undertake targeted assurance activities as may be required to address, for example, program risks or delivery concerns’.

2.44 In October 2021, the department established an Assurance Strategy for the program. The most recent version is dated January 2023. The Assurance Strategy outlines the assurance arrangements for the program (Table 2.2).

Table 2.2: Program assurance model

Source: Adapted from TFTM Assurance Strategy.

2.45 The program’s risk management arrangements are discussed at paragraphs 2.59 to 2.66 and the program’s plans and strategies are examined at paragraphs 2.37 to 2.40.

Assurance activities

Terrace Review

2.46 In December 2022, Terrace Services conducted a Product Teams Review (Terrace Review) for the program. It examined ‘whether TFTM’s operating model will deliver on intended commitments and realise intended benefits’, reviewing the program’s planning; leadership and coordination; delivery health; and benefits realisation.

2.47 The Terrace Review reported on ‘challenges associated with both the program’s novel delivery approach within DAFF and with its rapid expansion’, including:

• ‘Devolved approaches to scope definition, planning and coordination have resulted in challenges with dependency management and orchestration of activities across the program’;
• ‘Delivery, release and product uptake bottlenecks’; ‘unresolved dependencies’; and ‘conflicts between foundational capability and digital services technologies or implementations’; and
• ‘Cultural considerations have impacted the effective operation of delivery, governance and reporting practices that would typically be expected in a program of TFTM’s size’, with ‘delivery oversight and orchestration risks’ being introduced as a result of ‘organic and un-curated pursuit of product development’.

2.48 The Terrace Review found that the department’s approach ‘is not best suited for delivery of value for money and business benefits across the TFTM program’. It stated that ‘Top-down leadership, direction and monitoring is essential for accountability and effectiveness’.

2.49 The review made five recommendations, each of which included actions to implement the recommendation. All five recommendations are listed as ‘high priority’. The review recommended a total of 22 actions, of which 17 were listed as ‘high’ priority and five were listed as ‘medium’ priority. The EDG received reports on the program’s progress in completing the action items arising from the review (Table 2.3).

Table 2.3: Summary of completion status of Terrace Review action items

Source: ANAO analysis of departmental documents, as at January 2024.

Gateway Reviews

2.50 Department of Finance Gateway Reviews (Gateway Reviews) ‘examine programs/projects at key decision points during design, implementation and delivery’. The reviews are conducted by the Department of Finance and aim ‘to provide independent, timely advice and assurance to the SRO as the person responsible for delivering the program/project outcomes’.³⁹

2.51 Gateway Reviews provide a Delivery Confidence Assessment using a five-tiered rating system (green, green/amber, amber, amber/red, and red). The reviews also provide an assessment of key focus areas using a three-tiered rating system (green, amber and red).

2.52 As at May 2024, five Gateway Reviews have been conducted for the program: a first stage review in 2019 and four mid-stage reviews in 2021, 2022, 2023 and 2024. The Gateway Reviews rated the delivery confidence for the program as green/amber, amber, or amber/red with several key focus areas rated as amber in each review (Table 2.4).

Table 2.4: Summary of program Gateway Reviews

Note a: The Gateway Reviews use five delivery confidence assessment ratings: green, green/amber, amber, amber/red and red.

Note b: The Gateway Reviews use three key focus area ratings: green, amber and red. Green is defined as ‘There are no major outstanding issues in this Key Focus Area that at this stage appear to threaten delivery significantly’. Amber is defined as ‘There are issues in this Key Focus Area that require timely management attention.’ Red is defined as ‘There are significant issues in this Key Focus Area that may jeopardise the successful delivery of the program.’

Note c: The Gateway Reviews define green/amber as ‘Successful delivery of the program to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery.’

Note d: The Gateway Reviews define amber as ‘Successful delivery of the program to time, cost, quality standards and benefits realisation appears feasible but significant issues already exist requiring management attention. These need to be addressed promptly.’

Note e: The Gateway Reviews define amber/red as ‘Successful delivery of the program to time, cost, quality standards and benefits realisation is in doubt with major issues apparent in a number of key areas. Urgent action is needed to address these.’

Source: ANAO summary of findings of Gateway Reviews.

2.53 In May 2024, the department provided the April 2024 mid-stage Gateway Review. It found that ‘Successful delivery of the program to time, cost, quality standards and benefits realisation is in doubt with major issues apparent in a number of key areas’ and that ‘Urgent action is needed to address these.’ It explained that:

Since the last review the program has suffered serious delays and a reduction in scope, resources and program governance. These issues were mainly due to resource constraints imposed on the program during the agency’s financial crisis and consequent austerity measures.

…

Under difficult circumstances the program has delivered on most of the major technology elements and some business services in Tranche 1 and recovered from some of its significant resourcing pressures but there is still considerable doubt that Tranche 1 can be delivered by its revised deadline of June 2024.

…

Urgent action to progress the recommendations of the review including updating the minister on progress of the program, transitioning program approach and appointing a dedicated program director is needed.

2.54 The Gateway Reviews have made a total of 59 recommendations for the program to date. Recommendations are rated as ‘critical’, ‘essential’ or ‘recommended’.⁴⁰ Once a recommendation is made, an update on its implementation is provided in the next Gateway Review, including an assessment of implementation by the Gateway Review team (Table 2.5). After this, implementation is no longer tracked in Gateway reports.

Table 2.5: Number of fully addressed Gateway Review recommendations — as assessed by Gateway Reviews

Source: ANAO summary of Gateway Reviews.

2.55 In April 2024, the fourth mid-stage Gateway Review made 15 recommendations: five critical, six essential and four recommended.

Internal audit

2.56 In May 2023, the department conducted an internal audit of the implementation of Gateway and Digital and ICT Investments Review Recommendations. The internal audit assessed the department’s processes to manage, monitor, and report on its Gateway and Digital and ICT Investments Review activities and recommendations.

2.57 The internal audit found that ‘the department has effective governance arrangements to oversee the Gateway and Digital and ICT Investments Review activities at the program level’. It noted that ‘program teams have established processes to address the recommendations arising from reviews’ but that ‘these processes are project-dependant and inconsistently documented’.

2.58 The review made one recommendation and noted two business improvement opportunities. In June 2023, the department reported to its Audit and Risk Committee that the recommendation had been implemented.

Have appropriate risk management arrangements been implemented?

Risk and issue management

2.59 The department’s Enterprise Risk Management Framework and Policy (Enterprise Risk Framework) outlines the department’s arrangements to manage risk, including the department’s risk appetite and tolerance; risk matrix for assessing risk; and roles and responsibilities for risk management. The Enterprise Risk Framework states that the SRO ‘is ultimately responsible for ensuring that a program … meets its objectives and delivers the projected benefits and has the authority on how risks will be managed’.

2.60 The department has established risk and issue management arrangements for the program. The arrangements are documented in the Risk and Issue Management Strategy (Risk Management Strategy), approved in April 2021. The program’s arrangements align with the department’s Enterprise Risk Framework, using the same risk matrix to assess risks and risk appetite and tolerance statements.

2.61 The department has appointed a Risk Manager for the program. The Risk Manager is responsible for: assisting the Program Manager in the management of risks and issues; updating and maintaining the program risks and issues register; and developing program monthly reports, detailed risk reports, and risk summary reports. The Risk Manager provides support and guidance to program staff regarding the risk and issue management process. The Risk Manager also works with the Digital Results Office, Delivery Leads and Delivery Managers to ‘ensure risks and issues are actively managed and routinely reported’.

Risk management

2.62 The department’s Enterprise Risk Framework and the program’s Risk Management Strategy define risk as ‘The effect of uncertainty on objectives.’ They explain that ‘An effect can be positive, negative or both, and can address, create or result in opportunities or threats.’

2.63 The Risk Management Strategy outlines the program’s processes for assessing, treating, escalating and reporting risks. It defines three categories of risk for the program:

• program risks: ‘those which are likely to impact on the program objectives, realisation of benefits, and any individual risks at the initiative level that, if realised, will have a broader impact’;
• initiative level risks: ‘those that will have a life no longer than a quarter and are likely to impact on the initiative’s delivery commitments and Objective and Key Results’; and
• sprint⁴¹ risks: ‘those risks which are manageable within the sprint and likely to impact on the initiatives sprint goals’.

2.64 The Risk Management Strategy outlines the process for identifying, analysing, evaluating, and treating risks. It details how risks should be communicated and escalated, explaining that:

Staff at all levels must obtain appropriate and regular information about the management of risks within their area of accountability and control. Effective communication is critical to the identification of new and emerging risks and issues as well as understanding changes to existing risks. Appropriate and effective communication, recording and reporting of risk, facilitates effective risk-based decision-making.

2.65 Risks are recorded in Azure DevOps. As at February 2024, the program has documented 899 risks (627 closed and 272 open) in DevOps. Risks recorded in Azure DevOps are assigned an ID number. Each risk entry includes a risk title; description of the consequence and risk source; assessment of the current risk rating; assessment of the residual risk rating; description of controls (identifying each control’s owner); an assessment of each control’s effectiveness; and treatment decisions.

2.66 Of the 72 open risks at the ‘treat’ stage that are listed on the program’s risk register:

• 72 (100 per cent) have assessed the current risk rating;
• 59 (82 per cent) have assessed the residual risk rating;
• 41 (57 per cent) have identified a risk owner;
• 56 (78 per cent) have included at least one control;
• 43 (60 per cent) have assessed at least the first control’s effectiveness; and
• 53 (74 per cent) have documented a treatment decision.

Issue management

2.67 The Risk Management Strategy defines an issue as ‘an event that has occurred and is impacting Service Line and/or program objectives, scope, schedule, budget, quality and realisation of benefits’. It explains that ‘issues often constitute the realisation of identified risks’.

2.68 The Risk Management Strategy outlines the program’s processes for capturing, examining, treating and monitoring issues. Issues are recorded in Azure DevOps. As at February 2024, there were 551 issues (402 closed and 149 open) in DevOps. Issues recorded in Azure DevOps are assigned an ID number. Each issue entry includes an issue title, description, issue owner, ‘assigned to’, treatment options, and resolution.

2.69 Of the 149 open issues that were listed on the program’s issue register:

• 104 (70 per cent) have included a description of the issue;
• 37 (25 per cent) have identified the issue owner;
• 36 (24 per cent) have included treatment option(s); and
• 40 (27 per cent) have documented a resolution.

Risk management plan

2.70 In May 2023, the program established a Risk Management Plan, which was approved by the Head of Digital Trade Strategy and Initiatives. The Risk Management Strategy states that the Risk Management Plan ‘details the strategic risks for the program and the mitigations the program will employ to manage them’.

2.71 The Risk Management Plan documents the risk ID number, title, risk owner, risk manager and the category of risk. It assesses the current and post-treatment likelihood, consequence and risk rating. It details the risk sources, risk statement and consequences if the risk is realised. It lists the controls and control owners as well as assessing each control’s effectiveness. It also lists treatments, treatment owners and treatment due dates.

2.72 The Risk Management Plan and the program’s approach to risk management was presented to the department’s Audit and Risk Committee in September 2023.

Oversight of risks and issues

2.73 The Risk Management Strategy states that the program’s risks and issues will be managed by four governance levels: delivery managers and delivery leads (for medium to low risks, unless it impacts the program); the Program Delivery Meeting (PDM) (for initiative risks above medium); the Executive Delivery Group (EDG) (for severe and high risks); and the SRO (for severe and enterprise-level risks) ‘based on the risk rating and the risk/control owner’s delegation’.

2.74 The Risk Management Strategy states that the PDM is the ‘forum for risks and issues to be escalated within the program’. The PDM considers risks at its meetings, primarily through the Sprint Reports. The Sprint Reports identify initiative risks and issues. The PDM has a standing agenda item for program issues for all of its meetings. The PDM had a standing agenda item for program risks, which ceased from November 2022.

2.75 The EDG considers risks and issues at its meetings, primarily through the Program Status Reports (for more information about Program Status Reports, see paragraphs 4.68 to 4.71). The Program Status Reports list the program risks and issues, the risk ID number, rating and provides an update on the status of the risk.

2.76 As at November 2023, there was one open program risk with ‘treat’ status and a residual risk rating of high listed on the program’s risk register. This risk had been raised with the PDM and EDG and is included in the Risk Management Plan.

2.77 As at November 2023, of the 31 open program risks at the ‘treat’ stage that were listed on the program’s risk register with a residual risk rating of medium or higher, 18 (58 per cent) had been raised with the PDM, EDG or included in the Risk Management Plan. Examples of the 13 risks (42 per cent) that had not been raised include:

• key program decisions may not be made in a timely manner or without appropriate authority;
• the program may not maintain continuity of corporate knowledge and intellectual property;
• program incurs significant underspend against budgeted operational and capital expenditure; and
• financial records are incomplete (e.g. missing timesheet) and inaccurate (e.g. incorrect charging).

Shared risk

2.78 The department’s Enterprise Risk Framework and the program’s Risk Management Strategy define shared risk as:

A risk which extends beyond a single party which requires shared oversight and management. This may include other Commonwealth departments, State and territory governments, industry, community groups, international trading partners, groups, divisions, or another business area of the department.

2.79 The Risk Management Strategy states that ‘governance and oversight of shared risks will be dependent on internal and external stakeholders’. It explains that ‘staff should aim to utilise existing forums such as interdepartmental committees, portfolio committees or program governance boards to discuss, monitor and report on shared risks where appropriate’.

2.80 Of the 11 risks listed in the Risk Management Plan, no risks are classified as shared risks. Of the 80 open program risks at the ‘treat’ stage listed on the program’s risk register, no risks are classified as shared risks.

2.81 The digital reform of agricultural export systems involves and impacts the Australian export industry as well as other Commonwealth departments and agencies, especially those involved in the regulation of exports⁴² and international trade.⁴³ The department’s Enterprise Risk Framework and the program’s Risk Management Strategy require it to identify and manage program risks that extend beyond the department and require shared oversight and management.

3.1 The Australian Government has committed $349.6 million for the Digital Services to Take Farmers to Market budget measure over six years (2020–21 to 2025–26) for the department to deliver the digital reform of Australia’s agricultural export systems outlined in the program’s business cases.

3.2 The Commonwealth Digital and ICT⁴⁴ Oversight Framework (IOF) and Commonwealth Investment Framework include the following key elements that support the government to manage digital and ICT-enabled investments:

• detailed planning of what will be delivered, the delivery schedule and how each digital and ICT option could be acquired and delivered to deliver capability and benefits⁴⁵;
• detailed cost estimates based on rigorous planning of required ICT infrastructure, applications and support and then appropriately managing costs (including whole-of-life investment costs) ⁴⁶; and
• undertaking consultation to ensure effective implementation.⁴⁷

Is the program being delivered in accordance with implementation plan(s)?

3.3 The Business Case Addendum (BCA) included a program roadmap, which listed the initiatives that would be delivered by the program. The roadmap provided an indication of the start and end date for 63 initiatives, mapped across three tranches. Tranche 1 was scheduled from 2020–21 to 2022–23 and listed 31 initiatives that would commence in Tranche 1 (with 28 to be completed in Tranche 1 and three to be completed in Tranche 2).

3.4 In February 2022, the second mid-stage Gateway Review found that the program’s implementation planning documents were not linked:

It is not clear how the initiatives in the Tranche 1 program roadmap link with the items in the 12 months rolling plan and Quarterly Horizon plans. There must be a line of sight across these plans to provide assurance that these initiatives are being delivered.

3.5 The Gateway Review recommended that ‘as a priority the program should link the key program plans so that they can be rolled up based on a supportable level of detail’.

3.6 In May 2022, the program established a Tranche 1 Change Log. The Change Log assigned a number to each of the 31 Tranche 1 initiatives. The Tranche 1 Change Log was used to track changes to initiative titles and initiatives that had been consolidated, added or removed. Figure 3.1 shows the shift to the use of numbered initiatives in planning documents. The initiative numbers were used to identify and link initiatives across planning documents, such as quarterly plans and the Tranche 1 Plan. As at March 2024, the Tranche 1 Change Log listed 35 initiatives.

3.7 In February 2023, the third mid-stage Gateway Review stated that its recommendation had been fully addressed.

Figure 3.1: Timeline of program planning documents

Source: ANAO visualisation of departmental documents.

Quarterly plans

3.8 In July 2021, the Trade Reform Board (TRB) approved the first quarterly plan for the program, for Q1 2021–22. The program prepared quarterly plans for each quarter, from July 2021 to January 2024. The format of the quarterly plans changed each quarter between Q1 2021–22 and Q2 2022–23, after which the format stabilised for three quarters before the format of the quarterly plan changed. The format stabilised for Q1 2023–24 to Q3 2023–24.

3.9 For the three quarters from Q2 2022–23 to Q4 2022–23, each delivery team provided a one-page outline of the work that was anticipated to be delivered in that quarter. The outline of work identified the initiative number; listed the product manager and delivery manager; the Objectives and Key Results (OKRs) for the quarter; key dependencies; key risks; subject matter expertise (SME) requirements; ‘commodity and change impacts’; and relevant program milestones from the Tranche 1 Plan.

3.10 For the three quarters from Q1 2023–24 to Q3 2023–24, each delivery team provided a two or three-page outline of the work that was anticipated to be delivered that quarter. The outline of work identified the initiative number; listed the product manager and delivery manager; identified a quarterly objective; mapped a sprint plan; listed features the initiative would deliver; and listed key dependencies, key issues and risks.

3.11 For Q3 and Q4 2021–22, the quarterly plans established ‘Key commitments’ for the program for the quarter. From Q1 2022–23 to Q4 2022–23, the quarterly plans established ‘key milestones’ for initiatives. From Q1 2023–24, the quarterly plans established ‘features’ for initiatives.

Rolling roadmaps

3.12 The Q1 2021–22 and Q2 2021–22 quarterly plans included a ‘rolling 12-month view of the roadmap across our product teams’. It listed the capabilities or opportunities that ‘will be explored in the quarter’. The rolling roadmap noted that it would be ‘reviewed and iterated regularly and items in later quarters are likely to change’. The rolling roadmap was discontinued from April 2022 in response to work to address the second mid-stage Gateway Review (February 2022) recommendation (see paragraphs 3.4 to 3.7).

Tranche 1 Plan

3.13 In December 2022, the first Tranche 1 Plan for the program was presented to the TRB for approval. The Tranche 1 Plan states that it ‘provides a view on the scope and timeframes for delivery of Tranche 1’ for the program. The Tranche 1 Plan was updated in January 2023 and March 2023.

3.14 The Tranche 1 plans include a roadmap and ‘initiative one-pagers’. The roadmap shows the start and end dates of the initiatives across Tranches 1 and 2. The initiative one-pagers list the initiative number and name; initiative description; benefits; ‘user stories’; ‘definition of done’; initiative dependencies; and key milestones. The plan also lists the team leading delivery of the initiative and the product owner.

Tranche 1 end-state

3.15 The Tranche 1 plans include a section titled ‘definition of done/success criteria’ for each initiative, which outlines the end-state for the initiative. This was generally in the form of a list of capabilities that will be delivered by the initiative (e.g. users will be able to …; capability will be delivered when …; users can …).

3.16 In February 2023, the third mid-stage Gateway Review noted that ‘there isn’t a clear end-state for Tranche 1 and the program will need to take available budget into account when confirming what the end state is’. It recommended that the program ‘clarify and communicate end state for Tranche 1’.

3.17 In May 2023, the Executive Delivery Group (EDG) was advised that, in response to the recommendation, ‘Tranche 1 reconciliation with a clear definition of the end state will be undertaken at the completion of Tranche 1’, with an expected completion date of 31 December 2023. As at March 2024, the program had not established a Tranche 1 reconciliation with a clear definition of end-state.

3.18 In April 2024, the fourth mid-stage Gateway Review stated that its recommendation had been fully addressed:

The Tranche 1 Plan provides key milestones for delivery, and this was communicated. The program should keep Central Agencies and Government updated.

3.19 The fourth mid-stage Gateway Review made recommendations that the department ‘develop an integrated master schedule for the program’ and ‘re-confirm with business the definitions of done for the remaining Tranche 1 elements for endorsement by the EDG’.

3.20 Establishing an end-state for a package of work after its conclusion limits effective oversight and accountability. The department did not establish an end-state for Tranche 1 or document how its completion was anticipated to contribute to the delivery of the outcomes the program is committed to deliver prior to its original end-date of June 2023, or its amended end dates of December 2023 and March 2024.

Delivery of Tranche 1

3.23 Tranche 1 was scheduled to be delivered by the end of 2022–23. In August 2022, the TRB approved the extension of the Tranche 1 end-date to December 2023. In December 2023, the Senior Responsible Officer (SRO) approved a further extension of the Tranche 1 end-date to March 2024. In February 2024, the department advised the Department of Finance that the end-date for Tranche 1 had changed from December 2023 to March 2024 to ‘accommodate delivery of [the] remaining tranche 1 initiatives’.

3.24 In May 2024, the department advised the ANAO that the end-date for Tranche 1 had been extended to June 2024.

Completion of initiatives

3.25 In October 2021, 31 initiatives were planned to commence in Tranche 1 (with 28 to be completed in Tranche 1 and three to be completed in Tranche 2). In addition to the 31 initiatives that were planned to be delivered in Tranche 1, the department added four initiatives (one of which was listed for delivery in Tranche 2). In March 2024, the department advised the ANAO that, of the 35 initiatives in Tranche 1:

• six (17 per cent) had been delivered;
• 13 (37 per cent) had been partially delivered; and
• 16 (46 per cent) had been discontinued, consolidated into other initiatives, or were under development.

3.26 In February 2024, the department advised the minister that the program had ‘finalised the majority of its Tranche 1 commitments by the scheduled 31 December 2023 closure date’.

Is DAFF managing resources appropriately to deliver the program?

Program funding

3.27 In the 2020–21 Budget, the Australian Government committed $328.4 million over four years for a package of measures titled ‘Busting Congestion for Agricultural Exporters’. The Digital Services to Take Farmers to Markets budget measure, which was described as the ‘centrepiece of the reform package’, accounted for $222.2 million of this funding.⁴⁸ The implementation of this measure is referred to by the department as the Taking Farmers to Market (TFTM) program.

3.28 In the March 2022–23 Budget, the department received an additional $127.4 million for the Digital Services to Take Farmers to Markets budget measure allocated over four years. To date, the Australian Government has committed $349.6 million for the Digital Services to Take Farmers to Market budget measure over six years (2020–21 to 2025–26) (Table 3.1).

Table 3.1: Funding for Digital Services to Take Farmers to Market measure, 2020–21 to 2025–26

Source: ANAO analysis of budget papers.

Tranche 1

3.29 Funding for the Digital Services to Take Farmers to Market budget measure amounted to $199.9 million for 2020–21 to 2022–23. During this period, the department spent $166.2 million (Table 3.2).

Table 3.2: Funding for Digital Services to Take Farmers to Market measure compared with funding spent by the program, 2020–21 to 2022–23

Source: ANAO analysis of budget papers and departmental records.

3.30 In March 2024, the department advised that movement of funds occurred in 2020–21 to shift $17.6 million of the budgeted funds into the 2023–24 financial year. Figure 3.2 shows the funding allocated to the program by the department, compared with the funding that was spent.

Figure 3.2: Comparison of funding allocated to the program by the department compared with funding spent by the program, 2020–21 to 2022–23, $ million

Source: ANAO analysis of departmental records.

Reduction in program budget

3.31 In November 2022, the department’s Chief Finance Officer (CFO) advised the Executive Board (EB) that, ‘if [the department’s] current expenditure pattern continues, a significant overspend will be realised’. The EB directed the CFO to ‘confer with each Deputy Secretary undertaking a deep-dive into group budgets’.

3.32 In December 2022, the TRB agreed that ‘$6.7 million allocated to the TFTM program will be redirected to the department’s savings efforts’.⁴⁹ The TRB noted that ‘in turn, the TFTM program will need to adjust its work schedule for the remainder of Tranche 1 – in consultation with the SRO – to ensure it remains within its 2022–23 allocation’.

3.33 In January 2023, the department’s acting CFO reported that ‘the spending measures agreed by the Executive Board in November [2022] have not realised the required spending reductions’ and ‘recommended that the Executive Board consider the implementation of further austerity spending measures to reduce the forecast overspend’.

3.34 On 16 March 2023, the TRB noted that ‘at multiple points between January and March 2023’ work in a number of areas of the program ‘has ceased due to funding constraints’. On 20 March 2023, further reduction of funding was considered by the EB. The SRO submitted a paper to the EB that reported that ‘TFTM was almost at bare bones funding before this proposed cut’. The EB agreed that the Agricultural Trade Group, under which the program falls, would provide further savings of $1.15 million.

3.35 In May 2023, a paper submitted by the program to the TRB reported that ‘the program has undertaken extensive cost reduction measures to accommodate Departmental budget pressures’. The paper explained that ‘in the March period the FY OPEX forecast of $60M was reduced to $44.4M’ and that ‘$15.5M OPEX savings was surrendered to accommodate the government efficiency dividends ($1.3M) and Departmental savings mandated by the Executive Board ($14.3M)’. The paper reported that ‘this has resulted in the pausing of several initiatives and a reduction in contractor staffing’.

3.36 A timeline of these events is illustrated in Figure 3.3.

Figure 3.3: Timeline of program funding

Source: ANAO visualisation of departmental records.

3.37 In March 2024, the department advised the ANAO that ‘underspends from prior financial years are expected to be made available for the program to ensure delivery’. It explained that:

A paper has been submitted to the secretariat of the department’s financial governance body but not yet tabled seeking consideration of underspends to use in future financial years, pending the approval of [the Department of Finance].

3.38 In April 2024, the department’s Finance and Performance Committee (FPC) endorsed recommendations from the Digital Business Division seeking to access unspent operating funding in the TFTM program for use in 2024–25 and 2025–26.

3.39 In April 2024, the FPC reported to the EB that ‘At the appropriate time, the CFO will provide further advice to the Board on the amount required and this will be actioned through a formal operating loss request to the Finance Minister.’ It advised that the current underspend estimates include:

(a) unspent 2022–23 TFTM Tranche 1 funds of $12.204 million for use in 2024–25. The FPC was informed that there are no 2022-23 funds available in OPEX as all cash was used in the prior year, and that these funds would need to be sourced from the 2023-24 underspend.

(b) expected unspent 2023–24 TFTM Tranche 2 funds of approximately $13 million for use in 2025–26 to complete the deliverables of the Australian Export and Trade Support measure.

Financial reporting

3.40 In August 2021, the department started drafting the Taking Farmers to Market Financial Management Plan (the plan), which was approved by the Head of Digital Trade Strategy and Initiatives in December 2022.⁵⁰ The most recent version is dated January 2023.

3.41 The plan establishes roles and responsibilities for program participants and decision-makers. The SRO is responsible for approving the budget and the finance team in the Digital Results Office is responsible for activities such as financial reporting and recording and maintaining the program budget in the department’s Financial Management Information System. The Digital Results Office is discussed at paragraph 2.36.

3.42 The TRB was responsible for approving program budget allocations and adjustments, ensuring budget alignment with strategic priorities and approving the use of contingency funds. The plan was updated in December 2023 to reflect the dissolution of the TRB in May 2023. In March 2024, the department advised the ANAO that the EDG is responsible for program financial management activities, with the SRO being accountable for financial decisions.

3.43 The financial reporting accurately reflected the financial records in the department’s financial management system.

Resource planning and sourcing

3.44 In January 2020, the department established a Sourcing Strategy for the program, which was approved by the Head of Digital Trade Strategy and Initiatives. The Sourcing Strategy outlines the arrangements for recruiting resources for the program. The department established a ‘dedicated’ team within the Digital Results Office to ‘facilitate the procurement of the ICT specialists required by the program’. The team ‘[works] with the established streams to identify, source and engage resources’ and includes an ICT procurement specialist.

3.45 The strategy requires that the department approach the market for goods and services with an expected value of over $80,000, using panel arrangements where practical. The department advised that it primarily uses the Digital Transformation Agency’s Digital Marketplace to source staff through standing offers.

Contract reporting

3.46 The Commonwealth Procurement Rules (CPRs) establish contract reporting requirements, including the requirement for entities to use AusTender, the Australian Government’s procurement information system.

3.47 The CPRs require entities to report contracts and amendments on AusTender within 42 days of entering into or amending a contract if it is valued at or above the reporting threshold. Between 1 July 2019 and 30 June 2023, the department published 395 contract notices to AusTender related to the program. Over the same period, the department published 324 amendments to these contracts to AusTender.

3.48 Of the 395 contract notices:

• 370 (94 per cent) were published within 42 days of the reported start date;
• 25 (6 per cent) were published more than 42 days after the reported start date; and
• one was listed in DAFF’s contract register but had not been published to AusTender.

3.49 Of the 324 amendments:

• 10 (3 per cent) were published within 42 days of the reported start date; and
• 314 (97 per cent) were published more than 42 days after the reported start date.

3.50 In December 2023, the department advised that:

An issue with the configuration between the contract record for variations in TechnologyOne and AusTender has been identified. The incorrect Amendment Start Date for variations was being reported on AusTender. With this issue, the Amendment Start Date was being reflected as the initial contract start date; and not the date of effect of the variation.

3.51 The department stated that ‘moving forward, all contract variations will be reported correctly as this issue was fixed in November 2023 for new contract variations’.

Recruitment and retention of resources

3.52 In October 2020, the department estimated the types of resources required to implement the program and forecast the staffing requirements (Table 3.3).

3.53 The actual number of APS staff for the program was less than forecast, while the number of contractors was less than forecast in 2020–21 and exceeded the forecast in 2021–22 and 2022–23 (Table 3.3). In particular, in 2021–22 the department used 118 per cent more contractors than forecast.

Table 3.3: Forecast staff compared with departmental advice of actual staff, average staffing levels (ASL)

Note a: Staff numbers reported as at 30 June of each financial year.

Source: Departmental documents and advice.

3.54 In October 2020, the department noted that there was a risk that the department may not be able to source appropriate subject matter expertise to support the program. In July 2021, delivery teams started reporting lack of resources as an issue that may affect delivery.

3.55 In January 2023, it was reported to the EDG that departmental budget pressures had resulted in delays in renewing contracts and the standing down of contractors. This also affected the filling of vacant positions in delivery teams. The program’s resourcing challenges were noted by the third mid-stage Gateway Review in February 2023:

Maintaining the core capability with sufficient capacity for Tranche 2 is likely to be challenging. Long lead times to rebuild and ramp up the program will be inevitable if significant reductions are made to the contractor workforce.

3.56 In May 2023, the program reported that contractor staffing had been reduced ‘by over 50 FTE (~20%)’. It stated that ‘delivery has been further complicated with no Business Leads working on the program, with no ability to fund any, and no access to external support for strategy work’. The program explained that:

These changes present significant uncertainty for contractors who represent 70% of all program staff and in particular those stood down awaiting renewal.

There is a significant risk to the program without documented commitment and approval to fund T2, that contractors, will leave for roles with more certainty and this will only increase over the next 3 months, even with renewals being provided. This will impact current and yet to be planned activity and will need to [be] modelled into T2 timelines and commitments.

3.57 The department advised that, as at 31 December 2023, the program has 36 full time equivalent (FTE) APS staff and 217 contractors (headcount).

Have appropriate stakeholder consultation and communication arrangements been implemented?

3.58 In May 2021, the department undertook research to ‘gain a deep understanding of export processes and develop a high-level understanding of the needs and expectations of [its] clients’. The resulting Discovery Report stated that ‘a cross-section of the agricultural export industry were engaged in user research’ including ‘small, medium and large organisations, those involved at different stages of the exports journey, including freight forwarding and regulation’. The report noted that stakeholders involved in user research included:

• 2 meat and meat product exporters;
• 9 livestock Live Animal Exporters;
• 6 non-Livestock Live Animal Exporters;
• 7 plant (grain & horticulture) exporters;
• 4 Industry bodies (meat, plant & non-livestock LAE);
• 3 freight forwarders;
• 1 state regulator; and
• 5 regional DAWE⁵¹ offices.

Stakeholder communication arrangements

3.59 In December 2021, the department established the Agricultural Trade Reform Communication Strategy. The strategy aims to provide ‘the overarching framework for communications relating to the broad Agricultural Trade Reform program’. It states that the strategy ‘will be complemented by sub-strategies for individual projects as required, which will provide more specific key messages, audience definition and implementation plans’.

3.60 The strategy established arrangements, including a Trade Reform Communications Team, which was responsible for: developing and implementing the strategy; developing and managing the agricultural trade reform narrative; and leading trade reform specific internal and external communication activities.

3.61 Following the dissolution of the Trade Reform Portfolio Office in May 2023, the Trade Reform Communications Team is no longer in operation. At its final meeting, the TRB noted that ‘communications will be managed within each program and project’.

3.62 The program prepared a draft Taking Farmers to Market Stakeholder Engagement Strategy, which was last updated in March 2021. In March 2024, the department advised the ANAO that it is developing a TFTM Communications and Engagement Strategy. The initial draft was prepared in November 2023 and the most recent version is dated February 2024.

3.63 Some initiatives, such as those related to NEXDOC⁵² and the Digital Clearance Stream, have developed stakeholder engagement strategies and are undertaking communications and consultation activities.

NEXDOC Communications and Engagement Strategy

3.64 The NEXDOC Communications and Engagement Strategy was established in January 2017.⁵³ The most recent version was approved in March 2021. The strategy outlines the stakeholder management and change management arrangements for NEXDOC. Commodity-specific communication strategies were established for commodities that were being transitioned to NEXDOC: honey, dairy and eggs.

3.65 The department has published videos, user guides and factsheets on its website to support NEXDOC users.⁵⁴ It also uses its website to communicate updates regarding the transition of commodities onto NEXDOC. The department conducts virtual drop-in sessions for those commodities that have recently or will soon be transitioned to NEXDOC. The sessions are an opportunity for exporters to see a demonstration of the registration process and ask questions about the NEXDOC portal.⁵⁵

3.66 The department has also used its social media channels, such as Facebook, LinkedIn, and Twitter⁵⁶, to provide updates on program progress to stakeholders.

3.67 The program is not coordinating stakeholder engagement and communications activities that are being undertaken by program teams.

4.1 The Australian Public Service Change Framework establishes guidance regarding appropriately managing and supporting change.⁵⁷ The Protective Security Policy Framework outlines how to secure ICT⁵⁸ systems throughout all stages of their lifecycles, including ensuring that ICT systems are authorised to operate, in accordance with the department’s ICT change management procedures.⁵⁹

4.2 The Commonwealth Digital and ICT Oversight Framework (IOF) and Commonwealth Investment Framework includes the following key elements that support the government to manage digital and ICT-enabled investments:

• establishing arrangements to measure the benefits being realised by the program⁶⁰; and
• establishing arrangements to measure and report on the program’s performance.⁶¹

Have appropriate change management arrangements been implemented?

Change management arrangements

4.3 In September 2022, the department established the Agricultural Trade Reform Change Management Strategy (the strategy), which was approved by the Trade Reform Board (TRB). The strategy aims to ‘deliver a consistent, repeatable, scalable, and proportionate approach to change management across all trade reform initiatives, ensuring changes are delivered in a calm and planned way’.

4.4 The strategy requires that:

• projects complete an impact assessment to ‘determine the complexity and reach of their change, the level of change management activities required to support it, and whether assurances are required prior to roll-out’; and
• readiness assessments are completed for changes identified as medium or high impact, as the ‘final assurance step to test for, assess and demonstrate that a change is ready to be rolled out’.

4.5 Some of the arrangements established by the strategy, including the Trade Reform Change Management Team and oversight by the TRB, are no longer active following the dissolution of the TRB in May 2023 (Table 4.1). At its final meeting, the TRB noted that ‘change management will be the responsibility of program and project managers.’

4.6 The strategy states that it is not intended to apply to day-to-day or core business activities undertaken across the Agricultural Trade Group or system update changes that have no impact on a user’s behaviour or use of a product.

Table 4.1: Status of change management arrangements

Key: ◆ Arrangement is active. ▲ Arrangement is no longer active but being used as reference/guidance. ■ Arrangement is no longer active.

Note a: The Trade Reform Portfolio Change Management Playbook is examined at paragraph 4.14.

Source: ANAO analysis of departmental documents.

Completion of impact and readiness assessments

4.7 In February 2023, the program reported to the TRB that, as at 30 January 2023, impact assessments had been completed for 32 TFTM projects and products. At this time, 24 of the 32 TFTM impact assessments had been rated as medium or high impact. Readiness assessments were in place for two of the 24 medium or high impact projects and products (8 per cent).

4.8 In March 2024, the department established a change register for the program. The register lists 18 impact assessments, compared to the 32 projects and products that the program reported had completed impact assessments in February 2023. Approval dates of the items in the register range from 18 March 2022 to 7 December 2023.

4.9 The register has 35 entries, comprising 18 change impact assessments⁶²; 10 readiness assessments; four ‘Go live minutes’; and two other entries.⁶³ Of the 18 impact assessments listed on the register, seven (39 per cent) included evidence that the impact assessment had been completed. Of the 10 readiness assessments listed on the register, nine (90 per cent) included evidence that the readiness assessment had been completed.

4.10 Of the 18 impact assessment entries listed on the register, 11 (61 per cent) were assessed as ‘high’ or ‘medium’ impact. Of these, three (27 per cent) had completed readiness assessments.

4.11 The program is not completing impact assessments for all of its projects and is not completing readiness assessments for all projects with ‘medium’ and ‘high’ impact changes. By not completing impact and readiness assessments in accordance with the program’s change management arrangements, there is a risk that medium and high impact changes will not be effectively identified and managed.

Guidance and support

4.14 The department established the Trade Reform Portfolio Change Management Playbook, which was approved by the TRB in April 2022. The playbook provides guidance on change management, including: ‘defining the change’; identifying, understanding and engaging stakeholders; planning change; implementing change; and ‘celebrating and promoting success’.

4.15 The department established the Trade Reform Change Community of Interest as a practice-sharing group to ‘network, collaborate, and support members to deliver change activities in a planned and consistent way.’ The community of interest met 12 times (approximately monthly) from October 2021 to November 2022. In August 2023, the department advised the ANAO that the Trade Reform Change Community of Interest ‘ceased to be facilitated by the [Trade Reform Portfolio Office]’ in March 2023 and is not active.

Exports instructional material

4.16 In March 2021, the department established a policy for the development, publication and maintenance of exports instructional material.

4.17 In March 2024, the department advised the ANAO that the Export Regulatory Knowledge and Capability team provides quarterly instructional material reports to all export Senior Executives (SES) (from September 2022) and monthly instructional material status reporting to export area branch heads (from December 2022).

4.18 The reports provide a summary of the status of exports-related instructional material, including the number of published documents being amended, documents in draft, new documents and documents overdue for review. The most recent report provided to the ANAO by the department states that, as at June 2023, 669 of 998 (67 per cent) exports-related instructional material documents, which include instructional material for export systems and other export processes, were overdue for review.

4.19 System user guides have been developed for internal and external users for EXDOC, NEXDOC, ER, PEMS and TRACE (these systems are described at paragraph 1.3). The department has established 182 items of guidance for the export systems (EXDOC, NEXDOC, ER, PEMS, TRACE), with the oldest available guidance dated March 1996. Of the 182 items of guidance:

• 58 (32 per cent) have been updated at least once;
• 90 (49 per cent) were published within the last five years (since 2019); and
• 115 (63 per cent) were published or updated within the last five years (since 2019).

ICT change management

System accreditation

4.20 The Australian Government’s Protective Security Policy Framework states that ‘entities must only process, store or communicate information and data on an ICT system that the determining authority (or their delegate) has authorised to operate based on the acceptance of the residual security risks associated with its operation’.⁶⁴

4.21 Of the five agricultural export systems (EXDOC, NEXDOC, ER, PEMS and TRACE), three (60 per cent) have active authority to operate. The department has not documented whether the remaining two systems’ functionality would require an active authority to operate.

System change management

4.25 The department has centralised its incident and change management processes. Changes made to the department’s ICT environment including operating systems, databases, software, applications and programs are recorded in a single Enterprise IT Service Management (ITSM) tool. EXDOC, NEXDOC, ER, TRACE, and PEMS have been managed as part of the department’s enterprise-level change management process.

Have arrangements been established to measure and monitor the intended benefits of the program?

Development of program benefits

Benefits management arrangements

4.26 In October 2020, the department forecast that the program would achieve $50 million to $221 million per year in financial benefits, with the total benefits of the program forecast to be from $252 million to $1 billion from 2025–26 to 2029–30. The department established three categories of financial benefits.

• Under benefit 1, reduced industry burden, industry would save $14 million to $31 million from 2025–26 to 2029–30.
• Under benefit 2, increased staff productivity, the department would save $2.7 million to $7.3 million from 2025–26 to 2029–30.
• Under benefit 3, increased export value, $235 million to $1 billion of export value would be protected from 2025–26 to 2029–30.

4.27 In May 2021, the department established the Portfolio Benefits Management Strategy, which was approved by the TRB. The strategy outlined three financial benefits ‘to evidence delivery of positive value to stakeholders across all funding sources and to support achievement of the export strategy’⁶⁵:

• Benefit 1: reduced cost of compliance burden — for external customers;
• Benefit 2: reduced costs of regulation — for the department; and
• Benefit 3: market access gains and other economic benefits to industry — for external stakeholders.

4.28 In October 2021, the department revised the program’s financial benefits to align with the benefits described in the Portfolio Benefits Management Strategy. One non-financial benefit, improved user experience with agricultural export services (benefit 4), was also included.

4.29 In December 2021, the department established the Taking Farmers to Market Benefits Management Plan, which was approved by the Head of Digital Trade Strategy and Initiatives. The Benefits Management Plan outlines the benefits management arrangements for the program.

Baselines

4.30 In October 2020, the department documented baseline information for the program’s benefits. The baselines for benefits 1 and 3 did not document the current industry costs and market risks, respectively, on which the estimates for the baselines were based. The baseline for benefit 3 did not explain the connection between the benefit and the baseline measure. The department identified that there was a lack of departmental data to inform the baselines.

4.31 The baseline for benefit 4 was derived from several elements including:

• external engagement in 2016 and 2019;
• estimates of documents for export assessment that require re-submission; and
• the observation that export document assessment staff often work extended hours during peak export periods.

4.32 These elements did not establish a measurable evidence base for the baseline for benefit 4.

Measuring benefit realisation

4.33 The Taking Farmers to Market Benefits Management Plan requires that product managers and the Digital Results Office measure the benefits realised by the program. It establishes how benefits should be measured and the sources of data that should be used (Table 4.2).

Table 4.2: Benefits measurement arrangements

Source: TFTM Benefits Management Plan, as at January 2023; ANAO analysis.

4.34 The department is measuring the realisation of benefits 1 and 2 through estimated time taken on tasks and business running costs for the department and exporters. Within these categories, different areas are measured for different initiatives. Examples include:

• time taken across an entire process or for a specific component; and
• avoided costs; for example, no longer needing to courier physical export documentation overseas due to the implementation of electronic export documentation.

Market access gains and other economic benefits to industry

4.35 In October 2020, benefit 3 was forecast to generate between 93 and 96 per cent of the expected benefits for the program, valued between $235 million and $1 billion over five years. The department prepared three case studies to communicate this benefit.

4.36 In 2022, the department engaged Deloitte Australia⁶⁶ to conduct a review to ‘provide an evidence-based update for the third financial benefit’. The review found that the three case studies had ‘all failed to be verified’ and that ‘recent evidence and data has not validated the three baseline scenarios’.

4.37 The review made recommendations on next steps, including to:

• develop an estimated yearly benefit calculation for the impacts on trade from cyber incursions;
• implement benefit 3 calculation models described by the reviewer;
• identify and obtain time to market and market access data for baselining purposes; and
• conduct further research into the economic gravity model in the area of perishable goods.

4.38 The findings from the review were presented to the Executive Delivery Group (EDG) at its April 2023 meeting. The EDG noted the work underway to develop two new models for benefit 3. As at December 2023, the EDG had not endorsed replacement methods to measure benefit 3.

4.39 The program has reported to the TRB and EDG that benefits have been realised under benefit 3 (see paragraphs 4.44 to 4.46). A range of different approaches have been used to measure the benefits realised under benefit 3. Examples of approaches used include:

• calculating the difference in value of reducing the consequence of cyber-security related risks through database uplifts;
• estimating the reduction in costs by avoiding compensation claims by exporters on shipping handling charges due to missing messages; and
• calculating the effect of trade delays.

Improved user experience with agricultural export services

4.40 The Take Farmers to Markets Program: Client Experience Measurement Framework, endorsed on 30 June 2022, establishes questions to measure benefit 4. These questions are:

• How easy or difficult was it to use this service?
• From your perspective, how easy or difficult was it to understand what you needed to do?
• How supportive or unsupportive did you find us throughout the process?
• Please rate your overall experience today.

4.41 These questions may be asked during the alpha, beta and ‘go live’ stages of system development. The framework does not require user experience to be measured for all products or at specific times.

4.42 The department has conducted 20 user research surveys for the program incorporating user experience elements, including before the Client Experience Measurement Framework was established. Of the 20 surveys:

• two (10 per cent) covered all four questions;
• fifteen (75 per cent) covered between one and three questions; and
• three (15 per cent) did not cover any of the framework’s questions.

Oversight of benefits realisation

4.44 The program reports on forecast and realised financial benefits for the program in:

• quarterly reports to the TRB (from the quarter ending 30 September 2021 to the board’s cessation in May 2023);
• monthly reports to the EDG (from December 2022); and
• quarterly wave reports to the Digital Transformation Agency (DTA) (see paragraphs 4.78 to 4.80).

4.45 To inform quarterly benefits reports to the TRB, product managers reported benefits information to the Digital Results Office (DRO) and the DRO collated this information into a whole-of-program benefit view. The TFTM Benefits Management Plan states that the DRO will review the information provided by product teams and apply any necessary caveats to reports.

4.46 Despite the program not being forecast to realise benefits until 2022–23 (see Table 4.2) or 2025–26 (see paragraph 4.26), it has reported that benefits have been achieved from December 2021 (Table 4.3).

Table 4.3: Summary of TFTM benefits reporting

Note a: Benefits were reported according to benefit categories (benefits 1 to 3) prior to the report dated 30 June 2023.

Note b: These reports went by different titles including ‘TFTM status report’ and ‘Digital Trade Initiatives including TFTM program’. The reports were provided to the TRB and the EDG.

Note c: In Trade Reform Portfolio updates, benefits are reported cumulatively as at the date of reported figures. These reports were provided to the TRB while it was in operation.

Note d: The report does not clearly state the time period over which benefits have been realised.

Note e: Benefits in this report are reported cumulatively as at the date of reported figures.

Note f: Benefits in this report are reported year-to-date.

Note g: Benefits in this report are reported financial year-to-date.

Note h: Benefits in this report are reported cumulatively as at 31 December 2023. The report states that the program has realised $113,500,000 in benefits. The specific figures provided are $5,700,000 for benefit 1, $1,600,000 for benefit 2 and $54,500,000 for benefit 3.

Source: ANAO analysis of TRB and EDG papers.

4.47 Progress on achieving the program’s non-financial benefit was reported to the EDG and TRB in March and May 2023. The reports included the results of one user experience survey. No other progress towards the achievement of benefit 4 has been reported.

4.48 The department has not established baselines and methodologies for all of the program benefits, nor the evidence-base it is using to measure the benefits it is reporting it has achieved. The department is unable to demonstrate that its benefits reporting provides decision-makers with complete and accurate information on the realisation of financial benefits for the TFTM program.

Revision of forecast benefits

4.49 The department did not review or recalculate the forecast value of the program’s financial benefits (Table 4.2) following the changes made to the program’s benefits in October 2021 (see paragraph 4.28).

4.50 In October 2020, benefit 3 was valued at $235 million to $1 billion over five years and comprised between 93 and 96 per cent of the total forecast value of the financial benefits for the program. Without an evidence-based baseline or methodology for this benefit, the department cannot accurately forecast, measure or report on benefit realisation for between 93 and 96 per cent of the total anticipated value of benefits of the program. The department has not reviewed or recalculated the forecast value of the program’s financial benefits in response to the 2022 review of the methodology for benefit 3 (see paragraph 4.36).

4.51 In November 2023, the department informed the minister that ‘As of June 2023, an estimated $41.4 million in annual benefits has been delivered’. The department has not informed the minister that the approach used to forecast the value of benefit 3 has ‘failed to be verified’ or that the program is no longer expected to achieve benefit 3 as originally communicated to the Australian Government. As at March 2024, the department’s forecast is that the program will achieve $1.61 billion in cumulative benefits to 2029–30. $1.10 billion of this forecast is attributed to benefit 3.

Is the department effectively reporting on the progress of the program?

4.54 In December 2021, the department established the Taking Farmers to Market Program Reporting Plan (reporting plan), which was approved by the Head of Digital Trade Strategy and Initiatives.

4.55 The reporting plan states that it ‘sets up reporting mechanisms to ensure stakeholders are engaged appropriately and informed throughout the life of the program.’ It states that ‘It is key to maintaining visibility of the program, providing stakeholders with the information required to enable decision-making and adjustments to program implementation, and to track progress against delivery commitments.’

Ministerial reporting

4.56 The department provides a monthly dashboard report to the minister that contains status updates on budget measures across the department. The department reported on the program’s progress to the minister through the dashboard reports in November and December 2022 and February, March, April, June and July 2023.

4.57 The program’s status was rated green from November 2022 to March 2023 and amber from April 2023 to July 2023.⁶⁷ The department reported on program risks in reports from November 2022 to March 2023 and program issues from April 2023.

4.58 In March 2023, the department identified reductions to the program’s budget as a risk, reporting that ‘The OPEX budget for the program will be reduced due to a combination of government savings and a departmental dividend.’ In April 2023, the department reported that the reduction in the program’s budget ‘will impact planned [Tranche 1] outcomes and downstream benefits’ and that ‘Four initiatives have been ceased and a range of staff across all areas of the program have been stood down.’

Internal reporting

Executive Board

4.59 The program reporting plan stated that updates on the program will be provided to the Executive Board (EB) through portfolio reporting pathways on an as-required basis. In April 2023, the EB requested that the program provide a paper about the program’s risks, including an associated risk management plan. It also requested that the program provide a paper about the program’s progress every six months that includes information on the department’s digital progression, number of certifications, assurance processes and engagement with the Digital Services team.

4.60 In August 2023, the Head of Digital Trade Strategy and Initiatives presented a paper on the program’s risks as well as the program’s Risk Management Plan. The Head of Digital Trade Strategy and Initiatives also presented the first of the six-monthly program status updates, covering the period January 2023 to June 2023. The progress report included the outcomes of the February 2023 Gateway Review; key achievements over the past six months; current activity; and planned activity for the next six months. It did not include information on the number of certifications.

4.61 The progress report outlined the impact of budget reductions in March 2023, stating that ‘the program budget was reduced by $14.3 million OPEX by mandated departmental dividends and government savings requirements’. It stated that ‘Consequently, the program paused four initiatives deemed lower priority, ceased all engagements with consultancy firms, and reduced overall contractor numbers by ~50 FTE (representing around 20% of the total program workforce).’

Program governance bodies

4.62 The program’s governance bodies have been established to support the Senior Responsible Officer (SRO) in ensuring that the program delivers the agreed outcomes and benefits. The Trade Reform Board (TRB) and Executive Delivery Group (EDG) received reports regarding the program’s progress throughout their operation (more information on the TRB and EDG is available in Table 2.1).

Quarterly program status reports

4.63 From the establishment of the DRO in February 2021 to the dissolution of the TRB in May 2023, the program submitted program status reports to the TRB for six of the eight full quarters that occurred during the period and provided a partial status report for one of the quarters.⁶⁸ The partial status report comprised reporting on aspects of the program’s status throughout the portfolio update, such as information on the program finances and benefits realisation. This information was not a consolidated status report on the program.

4.64 The program status reports are high-level summaries that are one to three pages in length. The program status reports for Q2 and Q3 2021–22 included a traffic light system to communicate the program’s overall status as well as a narrative ‘program overview’. The program status was listed as amber for both of these reports.⁶⁹ The remaining program status reports provided a narrative ‘program overview’ but did not include a traffic light assessment of the program’s overall status.

4.65 All seven program status reports included a visualisation of the program’s ‘health’ comprising the number of initiatives with a reported status of red, amber or green.⁷⁰ The number of initiatives included in the visualisation varied between the reports, ranging from 31 in Q3 2021–22 to 18 in Q3 2022–23.

4.66 In addition to the program overview, the program status reports included updates on the program’s ‘key achievements’, benefits and finances. The reports for Q1 to Q4 2021–22 and Q1 to Q2 2022–23 included updates on risks and issues. The reports for Q2 to Q4 2021–22 and Q1 2022–23 included updates on ‘change and communication’.

4.67 The program status reports did not report on whether the program achieved what it had planned to achieve for the quarter. The program status reports do not explain how the key achievements delivered or contributed to the planned work, as outlined in the relevant quarterly plans or Tranche 1 Plan. The program status reports did not report on progress towards the delivery of the program as a whole.

Monthly program status reports

4.68 From November 2022, the EDG received monthly program status reports. The format of the program status reports is largely consistent from February 2023. The program status reports included a program overview, key achievements, program health, key issues and risks, finances, benefits and key dates.

4.69 The program health was reported as a visualisation comprising the number of initiatives with a reported status of red, amber or green.⁷¹ The number of initiatives included in the visualisation varied between the reports, ranging from 21 in the December 2022 report to seven in the November 2023 report.

4.70 The EDG was presented with monthly program status reports for all months from November 2022 to November 2023 except for February, April and September 2023.

4.71 The program status reports did not report on whether the program achieved what it had planned to achieve for the month. The program status reports do not explain how the key achievements delivered or contributed to the planned work, as outlined in the relevant quarterly plans or Tranche 1 Plan, or to the outcomes the program is committed to deliver.

Delivery reports

4.72 From November 2021 to September 2022 the EDG received program status updates through delivery reports. These reports were referred to as ‘paddock reports’ and ‘sprint reports’. The EDG was provided with an overview of the delivery reports, which commonly summarised the number of teams with a ‘delivery status’ or ‘OKR health’ of green, amber or red.⁷² The overview also commonly included the number of issues and risks reported by delivery teams and common and recurring areas of concern raised by teams.

Performance reports

4.73 The quarterly plans established performance measures for the quarter. These measures were most commonly in the form of objectives and key results (OKRs). Other measures, such as key commitments, milestones and features were also used. The program produced quarterly performance reports (Table 4.4), which largely reported against the performance measures established for the quarter.

Table 4.4: Program performance reporting

Note a: Objectives and key results (OKRs).

Note b: Harvey Balls are round ideograms used to visually communicate information. The performance report used Harvey Balls presented in one of five states: empty, one quarter full, half full, three quarters full or full to communicate the progress that had been achieved in the quarter. The performance reports did not include definitions for the Harvey Balls.

Note c: The blocks are an ideogram used to visually communicate information. The performance report used blocks in one of five states: empty, one block, two blocks, three blocks or full to communicate the progress that had been achieved in the quarter. The performance reports did not include definitions for the blocks.

Note d: Rating of high, medium or low. The performance reports did not include definitions for the confidence ratings.

Note e: Milestone status was reported using the following key: completed (green), one or two components delayed (off-green), half completed (yellow), limited progress (amber), no progress (red) and suspended (grey).

Note f: Feature status was reported using the following key: green (completed), amber (partially complete), red (no progress or significant issue), and grey (suspended).

Source: ANAO analysis of program performance reports.

4.74 Program status and performance reporting was limited to distinct periods of time (e.g. sprints⁷³, quarters, tranches) and short-term delivery goals within those periods. Reporting to governance bodies did not include reporting on progress in delivering the program as a whole or the program’s outcomes. The program’s plans and performance reports do not explain how the quarterly objectives and key results (or the work or achievements of the initiatives or teams) will or did deliver or contribute to the delivery of the outcomes the program is committed to deliver.

4.75 This limits the SRO’s ability to effectively monitor the progress of the program as a whole and to determine whether the program is on track to deliver its commitments on time and within budget. This also limits the SRO’s ability to make informed decisions regarding the allocation of funding and resources.

Digital Transformation Agency

4.78 The Digital Transformation Agency (DTA) requires Australian Government projects with a digital or ICT budget of $10 million or more to submit quarterly wave reports. This reporting is ‘for the purposes of providing advice to Government on the overall health and performance of the portfolio of ICT-enabled initiatives.’

4.79 The department has submitted quarterly wave reports to the DTA on the Digital Services to Take Farmers to Market (TFTM) and Export Certification Modernisation and Digitisation (ECMOD) budget measures. It has submitted reports on the TFTM program for all waves since October 2020.

4.80 The ANAO examined 10 wave reports submitted for the TFTM program covering the quarters ending 30 September 2020 to 30 June 2023. Of the 10 wave reports:

• information on the percentage of each benefit realised was requested by the DTA for eight reports. Of the eight reports, the department provided this information for all benefits in two reports (25 per cent) and did not provide this information for at least one of the benefits in six reports (75 per cent);
• information on the program’s staffing numbers was requested by the DTA for nine reports. Of the nine reports, the department provided this information in eight reports (89 per cent) and did not provide this information in one report (11 per cent); and
• information on the program’s spend to date was requested by the DTA for nine reports. Of the nine reports, the department provided this information in seven reports (78 per cent) and did not provide this information in two reports (22 per cent).

4.81 The department advised that it has not engaged with the DTA regarding its wave reporting and does not receive feedback on the reports and their content. The wave reports included specific achievements that the department had delivered in each quarter. The reported achievements are supported by evidence. The department also reported on program delivery confidence in the reports.

External reporting

4.82 The Public Governance, Performance and Accountability Act 2013 (PGPA Act) requires that accountable authorities promote the achievement of the purposes of their entity and prepare annual performance statements for the Australian Parliament that provide information about their entity’s performance in achieving its purposes.⁷⁴

4.83 Performance information is of most value if it is used to assess whether an entity’s purposes are being met and to provide a mechanism through which improvements can be made. In addition to improving performance, well-constructed and complete performance information can have an impact on improving productivity, policy and program implementation, integrity and innovation. Requirements for performance measurement and reporting are critical to accountability and the achievement of public sector entities’ purposes.⁷⁵

4.84 The department has been subject to the ANAO’s performance statements auditing process since 2020 and has continued to receive significant or moderate findings from the ANAO regarding its external reporting to the Parliament.

2021–22 Corporate plan

4.85 The department’s 2021–22 Corporate Plan outlined five performance measures relevant to the digital reform of the agricultural export systems.⁷⁶ The ANAO audited the department’s performance statements as part of the 2021–22 performance statements audit work program.

4.86 The department received significant or moderate findings in the areas of disclosure and presentation; preparation processes, including record keeping; methodology; and completeness and accuracy. The findings regarding performance measures relevant to the program are outlined in Table 4.5.

Table 4.5: Summary of findings made on relevant performance measures (2021–22)

Source: Summary of performance statements audit.⁷⁷

2022–23 Corporate Plan

4.87 The department’s 2022–23 Corporate Plan contained two performance measures relevant to the digital reform of the agricultural export systems.

• IG-04: Decrease in the number of point-of-entry failures of agricultural exports where the department’s actions have caused the failure.
• IG-05: Increase in the number of electronic certificates for export issued (moving to paperless trade).⁷⁸

4.88 The ANAO audited the department’s performance statements as part of the 2022–23 performance statements audit work program. The department received significant findings in the areas of enterprise-wide performance framework; completeness of performance information; reliable and trustworthy performance information; and preparation and record keeping processes. The findings regarding performance measures relevant to the program are outlined in Table 4.6.

Table 4.6: Summary of findings made on relevant performance measures (2022–23)

Source: Summary of performance statements audit.⁷⁹

2023–24 Corporate Plan

4.89 The department’s 2023–24 Corporate Plan contains two performance measures relevant to the digital reform of the agricultural export systems.

• IG-05: Number of international markets opened or improved by the department.
• IG-07: Increase in the number of electronic certificates issued for export (moving to paperless trade).⁸⁰

4.90 The department is included in the ANAO’s 2023–24 performance statements audit work program.

Appendix 1 Entity response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

Table A.1: Changes implemented during the course of the audit