Introduction

1.1 The First Principles Review: Creating One Defence⁴ was released publicly by the Minister for Defence in April 2015. The central theme of the review was that, for the Department of Defence (Defence) to achieve the outcomes the government expects efficiently and effectively, it needed to cease operating as a ‘federation of separate parts’ and become an integrated organisation.⁵ This ‘One Defence’ approach was expected to create ‘a more unified and integrated organisation that is more consistently linked to its strategy and clearly led by its centre.’⁶

1.2 One of the First Principles Review’s six key recommendations was to ‘fully implement an enterprise approach to the delivery of corporate and military enabling services to maximise their efficiency and effectiveness.’⁷ The review stated that:

There is general agreement about the nature of the problem. The current organisational model and processes are complicated, slow and inefficient in an environment which requires simplicity, greater agility and timely delivery. Waste, inefficiency and rework are palpable.⁸

…

Duplicated systems and processes reflect entrenched resistance to implementing businesslike approaches such as shared corporate services and the empowerment of single accountable officers in areas such as information management. According to the Chief Information Officer, Defence has over 2500 information and communication management applications including 300 financial applications.⁹

1.3 The Defence White Paper released on 25 February 2016 also identified issues related to Defence information and communications technology (ICT) systems, stating that underinvestment in ICT, and the lack of an enterprise-level strategy for Defence’s ICT requirements:

… has led to serious degradation across the information and communications capabilities of Defence. Key capabilities need urgent remediation, in particular to address the shortcomings of out-dated, and in some cases obsolete, systems that inhibit the conduct of day-to-day business within Defence, with overseas allies and partners, and with industry and the community more broadly.¹⁰

1.4 The 2016 Integrated Investment Program, released with the Defence White Paper, committed to enhancements to Defence’s ICT and business processes to support the implementation of First Principles Review recommendations, including by:

… standardising business processes to provide end-to-end visibility of Defence business through streamlined processes and a consolidated Defence Enterprise Resource Planning system that will improve core business functions, including force preparedness planning.¹¹

1.5 The 2016 Integrated Investment Program documented an ‘approximate investment value’ of $1–2 billion (out-turned) for an Enterprise Resource Planning System/Service, now known as the Enterprise Resource Planning (ERP) program, and a program timeframe of 2016–2025.¹²

1.6 The 2020 Force Structure Plan continued to emphasise the importance of business transformation, setting out that:

Driving improved business processes, enterprise information management, enterprise resource management and broader transformational reforms across Defence will be critical to future Defence effectiveness. The Government is committed to improving enterprise business processes by implementing reform across Defence to maximise business commonality and effectiveness.¹³

The Enterprise Resource Planning program

1.7 The ERP program involves the streamlining of Defence business processes associated with hundreds of separate Defence ICT applications into one SAP S/4HANA system¹⁴, with the intent of enabling better governance, faster processing and lower maintenance and support costs. Functional areas where applications and processes are in scope for the program are: finance, human resources, supply, maintenance, engineering, procurement and estate. The scale of the intended transformation is represented in Figure 1.1 below.

1.8 The ERP program is subject to the Australian Government’s first and second pass ICT investment approval process, administered by the Department of Finance.¹⁵ The government agreed in May 2017 to consider a series of second pass work packages for the program. This approach, involving a series of second pass submissions, reflects that the program is to be delivered through a number of tranches and that planning work was (and still is) ongoing.

1.9 A timeline of key program events to date is set out below at Figure 1.2.

Figure 1.2: Timeline of key program events to date

Source: ANAO analysis of Defence data.

First pass approval

1.10 The government provided first pass approval for the ERP program in May 2017. At first pass, the government agreed to Defence’s preferred option for the program¹⁶, which was to standardise and simplify end-to-end business processes, enabled by a new build SAP technology solution (see Appendix 3). The agreed program option would involve:

• transformation of Defence’s finance, logistics, procurement, engineering, maintenance and estate processes; and
• Defence engaging private sector partners, including systems integrators, to support delivery.

1.11 The Australian Government’s Digital Transformation Agency (DTA) supported the proposal, including the proposed iterative approach to design and planning. DTA noted that the proposal was consistent with best-practice design, delivery and assurance approaches. The government was also advised that the program risk rating was ‘high’ reflecting the size and complexity of the program.

1.12 The government agreed to a $58.9 million budget (out-turned based on parameters at the time) — including contingency of $10.7 million — from the Defence Integrated Investment Program provision for the ERP program of $1,002.2 million (excluding unfunded contingency of $221.8 million).¹⁷ The agreed funding was for the conduct of risk reduction studies, development of business cases for a series of second pass submissions, engagement of strategic partners, and the conduct of Offer Definition and Improvement Activities (ODIA) with potential systems integrators.

1.13 The indicative schedule for the overall program presented to the government included Initial Operating Capability (IOC)¹⁸ in February 2020 and Final Operating Capability (FOC)¹⁹ in 2026.

Second pass approval: Tranche 1

1.14 The second pass business case for Tranche 1 was approved by the government in June 2018, including $516.1 million (out-turned based on parameters at the time) from the Integrated Investment Program provision, to fund:

• the delivery of Tranche 1 ($257.9 million);
• ‘Prepare’ and ‘Explore’ phases for future tranches ($152.1 million); and
• sustainment over three years ($106.1 million).

1.15 DTA supported the proposal. DTA noted that the proposal would significantly uplift Defence’s internal capabilities, making it a more responsive and efficient organisation and enabling the development of additional functions on the resulting core platform, while mitigating a number of critical risks to its systems, most immediately Defence’s Military Integrated Logistics Information System (MILIS).

1.16 The government approved dates for IOC and FOC at second pass are set out in Table 1.1 below.

Table 1.1: Initial Operating Capability (IOC) and Final Operating Capability (FOC) presented at second pass^a

Note a: Defence advised the ANAO in March 2021 that the dates for IOC and FOC presented to the government at second pass were for the entire ERP capability (all tranches).

Source: Defence records.

1.17 The government approved the inclusion of the following Tranche 1 package of work:

• the human resources and finance foundations; and
• elements of logistics and land materiel capabilities, including the replacement of MILIS.

1.18 The second pass advice set out the delivery approach, including the program’s planned adoption of SAP Activate methodology (an iterative approach to planning, building and testing). The government was advised that SAP Activate is the common methodology globally for SAP implementation programs, and that all program partners were familiar with the methodology.²⁰

Second pass approval: Tranches 2 and 3

1.19 The second pass business case for Tranche 2 was approved by the government in December 2020. This allocated $250 million from the Integrated Investment Program provision for the delivery of a case management system which would replace 16 existing systems. The case management system was initially a stand-alone program, with its own Integrated Investment Program provision, and had received first pass government approval in December 2017.²¹

1.20 Tranche 3 is intended to deliver the balance of ERP capability through ten ‘Releases’, including force preparedness and planning, asset management, estate, portfolio and project management, supply chain, procurement, human resources, and finance. Defence advised the ANAO in July 2021 that

… to manage risk, the program currently plans to seek Government approval of the full ERP scope, schedule and cost in October 2021, with funds released to deliver Releases 1–5. A second funding release, for Releases 6–10, will be sought in 2023, subject to approval by Government.

Program management and budget

1.21 The ERP program has been managed within the Chief Information Officer Group (CIOG) by a contracted Senior Executive Service (SES) Band 2 Program Director, and an ADF 2 Star Business Lead.

1.22 The program budget breakdown for the financial years 2017–18 to 2022–23 is set out in Table 1.2 below.

Table 1.2: ERP program budget 2017–18 to 2022–23 ($ millions)^a

Note a: The program budget does not include the cost of Defence APS/ADF personnel. Defence advised the ANAO in 2018, in the context of the annual Major Projects Report, that its IT systems do not provide a direct mapping of personnel to projects, for the purpose of allocating personnel costs to projects.

Note b: The 2017–18 budget did not use the categories listed in the table. The most recent data for the 2020–21 financial year consolidated Contracted Services and Gate 1 into one category of commitment. The majority of the budget for 2017–18 was for contracted services ($29,347,496).

Note c: Numbers do not add up due to rounding.

Note d: The actual utilisation for 2017–18 also includes life to date spend up to 2017–18.

Source: ANAO analysis of Defence records.

Rationale for undertaking the audit

1.23 Effective delivery of key enabling services, including ICT, supports Defence in meeting its purpose: ‘Defend and protect Australia and advance its strategic interests’.²² The ERP program is a major ICT reform initiative of strategic importance to Defence, as it is intended to assist in the delivery of an integrated, enterprise approach to service delivery as set out in the First Principles Review: Creating One Defence.²³

1.24 This audit provides independent assurance to Parliament on Defence’s progress to date in implementing Tranche 1 of the ERP program, including governance, monitoring and reporting and the procurement and contract management of a systems integrator.

Audit objective, criteria and scope

1.25 The audit objective was to examine the effectiveness to date of Defence’s administration of the Enterprise Resource Planning (ERP) program, with a focus on ERP Tranche 1 activities.

1.26 To form a conclusion against the audit objective, the following high-level criteria were adopted:

• Has Defence established fit-for-purpose planning, governance, monitoring and reporting arrangements to support implementation of the ERP program?
• Has Defence conducted an effective procurement process for Tranche 1 of the ERP program that contributed to the achievement of a value for money outcome?
• Has Defence established fit-for-purpose contracting arrangements that support the achievement of Tranche 1 outcomes and its strategic priorities under the ERP program?

1.27 In terms of procurement processes and contracting arrangements, the audit scope focused on the contract established with the systems integrator (IBM). This contract is material to the successful delivery of Tranche 1 and represents more than 50 per cent of the total program budget from 2019–20 onward (See Table 1.2 above).

1.28 The audit scope did not include a review of: Tranche 2 and 3 planning and activities undertaken to date for those tranches; or the decision to use SAP for the ERP system.

Audit methodology

1.29 Audit procedures included:

• reviewing advice provided to government regarding the ERP program;
• reviewing Defence documentation related to ERP program planning, decision-making, the tender process, and monitoring and reporting of program performance; and
• discussions with Defence personnel and personnel contracted by Defence to deliver the ERP program.

1.30 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $498,822.

1.31 The team members for this audit were Kelly Williamson, James Woodward, Corne Labuschagne, Nate Wirihana, Tara Rutter and Sally Ramsey.

2.1 The ERP program involves the streamlining of Defence business processes associated with hundreds of separate Defence ICT applications into one system, to be applied by the whole of Defence, with the intent of enabling better governance, faster processing and lower maintenance and support costs. To support the effective implementation of the ERP program, fit-for-purpose planning, governance, monitoring and reporting arrangements, commensurate with the size and complexity of the program, need to be established.

2.2 This chapter examines whether Defence has:

• effectively planned for the delivery of Tranche 1 of the ERP program and provided appropriate advice to government following second pass;
• established a fit-for-purpose governance structure to support delivery of the ERP program, Tranche 1; and
• developed effective monitoring and reporting arrangements that provide senior leaders with adequate visibility of progress and emerging risks.

Did Defence effectively plan for the delivery of Tranche 1 of the ERP program and provide appropriate advice to government following second pass?

2.3 To establish whether Defence effectively planned for the delivery of Tranche 1 of the ERP Program, the ANAO examined:

• plans established by Defence to guide the delivery of Tranche 1;
• systems and processes established to identify and manage risks to the successful implementation of the ERP program;
• Defence’s management of recommendations from reviews of implementation; and
• Defence’s advice to government regarding the implementation of Tranche 1 and the program more broadly.

Tranche 1 program plan following second pass approval

2.4 Program management plans were developed in 2015 and 2017 to inform program development at first and second pass. A new program management plan (the plan) was developed by the systems integrator²⁴ and approved by the Program Director on 6 December 2019. The plan was updated and approved by the Program Director on 11 March 2021. These updates reflected:

• work done on future tranches;
• schedule changes agreed by Government; and
• governance changes, such as changes to the risk management approach and key program governance forums.

2.5 The plan sets out program objectives, outcomes, key success factors, high-level scope and schedule. It reflects planning that has taken place with the systems integrator since second pass. The high level scope approved by the government recognises the expectation that the ERP solution’s design will evolve during the ‘Explore’ phase (see Appendix 3).

2.6 There is a suite of planning documents which sits below the plan, including the Integrated Master Schedule, Tranche Implementation Plan, and Master Resource Schedule. These capture detailed planning and changes over time, and are reviewed quarterly as systems integrator deliverables.

2.7 The plan also breaks down Tranche 1 into two component parts:

Release 1A: An element of Tranche 1 Foundation Finance that will provide a central finance reporting capability based on the “to be” Chart of Accounts and Enterprise Structures. Release 1A will use the Tranche 1 core S/4HANA ERP system in conjunction with SAP Central Finance (CFIN). It will deliver a finance solution with enterprise structure and master data objects based on the new Defence Finance Group (DFG) design. Interfaces and mapping rules will be implemented to allow the extraction and loading of required ROMAN and BORIS data. Custom reports will be built to allow DFG to meet Internal and External reporting requirements using the new structure and master data design.

Release 1B: It will deliver a solution for Core Supply Chain Management and Maintenance [MILIS] across Defence. It will also deliver Engineering, Procurement, Finance and HR foundations to support this capability. This release will subsume capability in some legacy systems and interface to others.

2.8 The plan sets out a timeline for Tranche 1, including IOC and FOC dates for Release 1A and Release 1B. See Figure 2.1 below.

Figure 2.1: Tranche 1 timeline as set out in the March 2021 ERP program management plan^a

Note a: Ticks indicate completion of milestone. The Release 1B IOC and FOC dates in this figure reflect the agreed dates in the Release 1B official order with the systems integrator, discussed in paragraph 2.9 below.

Source: ANAO representation of Defence records.

2.9 Release 1A IOC and FOC have been achieved, as set out in the timeline in Figure 2.1. Release 1A FOC was originally planned for October 2020, and was delivered in December 2020. Deliverables and key milestones for IOC and FOC Release 1B were established as part of the Release 1B Realise and Deploy official order with the systems integrator (IBM), executed on 26 March 2021. The official order sets deliverable milestones for Release 1B IOC to be achieved in October 2022 and FOC in December 2022. Defence advised the ANAO that these delivery dates have been set for IBM to enable Defence to achieve the key dates agreed by government: IOC by Quarter 4 2022 and FOC by mid-2023 (see Table 2.1).

2.10 Requirements definition and process capture and mapping activities have been ongoing. Defence has advised that as at 26 April 2021, 836 persons across the Groups and Services have committed 7,695 workshop days, with a total commitment of 12,300 workshop days when ERP personnel are included. Workshops involved business representatives who assisted in confirming business processes and identifying gaps between business requirements and the SAP solution. Outputs have been formalised into business process documents, which are a systems integrator contract deliverable.

2.11 As discussed in the next section, there is evidence of ongoing planning to mitigate risks and address program challenges. This takes place through regular monitoring and review of the RADDICAL (Risk, Assumptions, Dependencies, Decisions, Issues, Changes, Actions, Lessons) registers maintained in the Enterprise Portfolio Management System (EPMS — an electronic system used for day-to-day program management).

Risk management

2.12 An August 2017 internal audit found that prior to first pass the program had not given sufficient priority to ensuring risks were effectively recognised and reported. The internal audit stated that assurance had been provided that a comprehensive risk management framework was planned to be delivered.

2.13 Subsequently, Defence developed a strategy to address the management of risks, assumptions, issues, dependencies and decisions in the program (known as ‘RAIDD’). The strategy set out the process for risk identification, analysis, reporting and management. Defence further developed this into an approach to manage benefits, risks, actions, dependencies, decisions, issues, change, communications, assumptions, and lessons (known as ‘B-RADDICCAL’), as set out in the program management plan approved in December 2019.

2.14 The ERP program has since moved to a model for managing risks, actions, dependencies, decisions, issues, change, assumptions, and lessons (known as ‘RADDICAL’), as set out in a draft 2020 risk and issue management strategy, the August 2020 ERP program handbook, and the March 2021 updated program management plan.²⁵ Registers for RADDICAL items are embedded in EPMS.

2.15 Defence has established a three-tiered risk management system for the ERP program. The three tiers are: stream²⁶, program and enterprise. Defence advised the ANAO that:

• ‘stream’ risks are reviewed weekly by the program management office;
• ‘program’ risks are reviewed monthly by the program executive team; and
• ‘enterprise’ risks are reviewed quarterly at a Groups and Services risk review session with ADF 1 Star officers, aligning with the monthly 1 Star group meeting.²⁷

2.16 As at 19 May 2021, there were 155 stream, 16 program, and 14 enterprise ‘in progress’ and ‘open’ risks recorded in the EPMS. Six stream and one enterprise risks in the register have a residual rating of ‘high’, and no risks have a residual rating of ‘extreme’.²⁸ Defence has advised that the Chief Information Officer Group operates under the Defence Enterprise Risk Appetite Statement. The Defence Enterprise Risk Appetite Statement states that:

Defence generally prefers to accept low to medium levels of risk, so far as reasonably practical. However, due to its complex operating environment, Defence may need to manage higher levels of risk.

2.17 Defence’s November 2020 internal audit, titled ‘ERP Release 1B Implementation Preparation Maturity Assessment’, recommended that the program consider key person risks within the program. In response to the internal audit, Defence committed to completing an assessment of key person dependencies and reporting on this to the ETB in February 2021. In April 2021 the program presented a paper to the ETB on its response to the audit recommendation. The ETB noted the paper, but requested that the program return in May with a clearer approach to demand and supply of ERP workforce requirements. On 12 May 2021, Defence added a program risk in EPMS related to key persons leaving the program, with an initial and residual rating of ‘medium’. In June 2021 the program returned to the ETB with a paper on how the internal audit recommendations had been addressed, including setting out its mitigation actions for key person risks. The ETB endorsed the actions the program had taken in response to the recommendations.

Management of recommendations from reviews

2.18 As well as the internal audits outlined above, the program has been subject to a number of Department of Finance Gateway reviews and an Independent Assurance Review, which have examined the adequacy of planning to date. Defence’s Audit and Risk Committee has determined that the program will be subject to further internal audits, ‘given its complexity and the effects on many parts of Defence’.

2.19 The Digital Transformation Agency’s comments at second pass (June 2018), while supportive of the program, highlighted the risks of change management, size and the ‘aggressive’ schedule, noting that adoption of SAP’s Model Defence Organisation (MDO)²⁹ and careful, responsive management and top-level support could assist in mitigating risks. At second pass, Defence’s stated reason for the ‘aggressive’ schedule for Tranche 1 was to mitigate the risks posed by MILIS, which at that time was considered to be not fully supportable beyond late 2020 and in need of urgent replacement.

2.20 A number of reviews had raised concerns regarding the planned schedule, including:

• A December 2017 Gateway review (Gateway reviews are discussed further at paragraph 2.48) recommended that Defence ‘mature dependency, project and schedule planning and adapt the schedule to include sufficient contingency for delays.’
• A September 2018 Independent Assurance Review considered whether the proposed phasing and rollout for the ERP program best met the needs and priorities of Defence and enabled effective management of risk. The review considered the overall program strategy, planning and resourcing, and found ‘significant issues and concerns’³⁰ which were ‘exacerbated by the current aggressive schedule to Tranche 1’. The review found that relief from the ‘aggressive’ program schedule would allow more time for business transformation, SAP product development and more detailed planning of Tranche 1 functionality. The review raised concerns that the program was shaped primarily by the pressing need to replace one element (MILIS) and would defer other core components of the ERP system, which would ‘necessarily involve complex interfaces and work-around and a level of additional risk, as yet unquantified’. At this stage, the program was waiting on the systems integrator procurement to progress planning.³¹
• A May 2019 Gateway review found that the above recommendation from the December 2017 Gateway review was not implemented, with Defence relying on the MDO to accelerate the schedule and de-risk delivery. Defence was aware that it would need to identify gaps in the MDO compared to Defence’s requirements, and that the MDO would need to be incrementally expanded to meet program outcomes.

2.21 Defence actions taken as a result of the 2018 Independent Assurance Review included: re-baselining the program schedule with the systems integrator once the procurement was complete; broadening the role of the responsible governance body (which is now called the Enterprise Transformation Board) to include key business domain owners; and integrating governance of the ERP and Enterprise Information Management programs.³²

Program re-baselining: 2019 and 2020

2.22 In late 2019 the program was re-baselined through engagement with senior leaders and the systems integrator, resulting in a planned delay in IOC from Quarter 4 2020 to mid-2022. Through this process, Defence confirmed the program scope and refined the implementation plan. Further, the imperative to replace MILIS — which was originally understood to be unsupportable beyond 2020 and which drove the ‘aggressive’ ERP schedule — was able to be reduced. Work was done to find options that would enable support to MILIS beyond 2020 and delay its decommissioning. Defence recognised that the delay in decommissioning MILIS would lead to an increase in maintenance and support costs. The cost of supporting MILIS was estimated by Defence to be $11,639,321 each year from 2021–22 to 2040–41 (in constant dollars, not out-turned dollars).

2.23 In late 2020 the program was further re-baselined, resulting in a further IOC delay to Quarter 4 2022.³³ The request to further delay IOC was initially agreed by Defence’s Enterprise Transformation Board (ETB) on 6 August 2020 (the board is discussed further from paragraph 2.38 below). The ETB had been presented with a proposal to implement: a change to how the ERP program finance scope was delivered; delivery of a ‘basic SAP Transport Management (TM) functionality to enable Cargo Visibility System (CVS) replacement’; and an extension for the ‘explore’ phase of Tranche 1B by three months and increased system testing during its ‘realise’ phase. The ETB was advised that the changes would: result in a three month IOC delay; would not change FOC; and would be delivered within the approved program budget.³⁴ After receiving the ETB’s agreement to implement the proposal, Defence signed a contract variation for $14,594,193.83 with the systems integrator on 30 November 2020.

Advice to government regarding changes to program planning

2.24 In November 2019, Defence advised the government that a delay to IOC³⁵ was required to reduce schedule risk from ‘high’ to ‘medium high’. This delay represented the re-baselining of the program that took place through the planning process with the systems integrator, and the identification of deficiencies in the initial program planning through reviews (discussed above).

2.25 Defence further advised the government that it had not yet assessed the impact on FOC, but that this advice would be provided in mid-2020.³⁶

2.26 The government agreed to the IOC change on 9 December 2019, noting that although it was affordable within the Integrated Investment Program provision³⁷, it would delay benefits in ICT sustainment (such as the potential costs associated with delaying the decommissioning of MILIS), workplace efficiencies, and improved information management capabilities.

2.27 In December 2020, Defence sought approval from government to further delay Tranche 1 IOC by three months.³⁸ The rationale for further IOC change was to keep schedule risk at ‘medium-high’ in response to a range of factors, including:

• the need for a better understanding of requirements and risks during rollout;
• the need to develop a strategy to optimise rollout (detailed rollout design was to take place in early 2021);
• to accommodate lessons learnt from Release 1A around additional test time needed;
• the emergence of ‘design challenges’; and
• a delay in a critical SAP software update.

2.28 The government considered and agreed to the request to delay IOC on 10 May 2021.

2.29 The advice provided to government in December 2020, updated by Defence in March 2021, was not complete, as it did not set out the action that Defence had already taken to initiate the program changes in anticipation of government agreeing to an IOC change. This included signing a contract variation for over $14.5 million with the systems integrator in November 2020. The variation included the items agreed by ETB in August 2020, which ETB had been advised would result in a three month delay to IOC (see paragraph 2.23). This included a design extension, with finalisation of R1B design moved from February 2021 to June 2021. Defence advised the ANAO in March 2021 that if government did not approve the IOC change, it would renegotiate the design extension in the systems integrator contract.

2.30 On 21 July 2021, Defence advised the ANAO that:

Defence does not agree that advice provided to government in December 2020 was not complete…

… The contract variation signed in November 2020 with the Systems Integrator was within the Program’s Government Approved Tranche 1 scope and Budget. From Defence’s perspective, contract changes of this nature do not require prior agreement from government.

2.31 While the contract variation was within the agreed scope and budget, it would impact on the schedule approved by government (including the IOC date) at that time. IOC and FOC dates have been approved by the government for this program. Table 2.1 below sets out the Tranche 1 timeframes as approved by government following Defence advice.

Table 2.1: Tranche 1 timeframe changes approved by government

Note a: At first and second pass, IOC and FOC dates were for the entire ERP capability, and not specific to Tranche 1. In the November 2019 update, IOC was for Tranche 1. From the May 2020 update (rows shaded), when Defence refers to IOC it relates to Tranche 1 Release 1B.

Note b: Government consideration, initially expected in February 2021, was delayed. The submission was updated in March 2021.

Source: Defence records.

Did Defence establish a fit-for-purpose governance structure to support delivery of the ERP program, Tranche 1?

2.32 This section examines the enterprise and program level governance arrangements in place for the ERP Program. The management of recommendations from internal audits that were completed in August 2017 and November 2020, and an Independent Assurance Review completed in September 2018 are also considered. The Department of Finance’s Gateway Review process has also been examined.

Enterprise-level governance arrangements

2.33 Key roles and responsibilities for the ERP program are assigned to senior officials with whole of Defence responsibilities — the Associate Secretary is the Capability Manager, and the Chief Information Officer is the Senior Responsible Officer.³⁹ These arrangements introduce a level of senior management responsibility commensurate with the whole of Defence ICT transformation to be delivered by the program.

2.34 Defence’s enterprise-level program governance arrangements involve a three tier structure, summarised in Table 2.2 below.

Table 2.2: Enterprise Resource Planning program enterprise-level governance structure

Source: Defence records.

2.35 High level program-specific governance is through the ETB chaired by the Associate Secretary.

2.36 The program is subject to Defence’s business as usual enterprise-level governance arrangements through the Enterprise Business Committee and the Defence Communications and Information Systems Committee.

2.37 In addition, there is involvement as needed by: the Defence Committee (DC), the most senior and primary executive committee within the Department of Defence, responsible for setting top-level organisational goals and driving delivery of Defence’s commitments to the government and the community; and the Investment Committee (IC), which is on the same tier as the EBC and is responsible for ensuring that Defence’s investment portfolio for military capability, estate and ICT meets the government’s requirements and the Chief of the Defence Force’s directive.

Enterprise Transformation Board

2.38 An SES Band 3/ADF 3 Star steering group for the ERP program was established in March 2018. It was renamed the Enterprise Transformation Board (ETB) in March 2020. The ETB was established after the December 2017 Gateway Review recommended that a sub-committee of EBC be formed to focus on the ERP program and provide guidance to EBC.⁴⁰

2.39 The ETB is chaired by the Associate Secretary. The committee meets monthly and its duties as set out in its charter are to:

• ensure clarity and support for the continuing organisational context for the programs⁴¹;
• review and support resolution of major issues and risks to support the progress of the programs against strategic objectives;
• actively champion the implementation of the required business transformational change;
• ensure identified Defence benefits are achieved from the investment;
• encourage active commitment and support of the program across Defence;
• advise and support the Program Directors/Managers; and
• oversee successful delivery and closure of the programs.

2.40 The membership of the earliest iteration of the SES Band 3/ADF 3 Star group comprised the Associate Secretary, Vice Chief of the Defence Force, and the Chief Information Officer. The September 2018 Independent Assurance Review⁴² of the ERP program recommended that membership be broadened to include key business domain owners. This recommendation was accepted and implemented. The following members were added: the Chief Finance Officer; Deputy Secretary Defence People Group; Deputy Secretary Estate and Infrastructure Group; Deputy Secretary Capability Acquisition and Sustainment Group; Deputy Secretary Chief of Joint Capabilities; First Assistant Secretary Contestability; and Deputy Director General Corporate Capability.

2.41 Concerns have been raised at the ETB regarding adequate representation from the Services (Navy, Army and Air Force) at the ETB, where business transformation decisions are being made. Further to this, a November 2020 Capability Manager Gate Review of the program identified that governance arrangements did not provide for adequate Group and Service representation. In response, in December 2020 the ETB directed the program to develop a terms of reference for a SES Band 2/ADF 2 Star Steering Committee. In February 2021, the ETB determined that rather than establish a new committee, the existing Defence Communications and Information Systems Committee should serve as the steering committee every alternate meeting. The ETB noted the importance of business representation at this meeting.

2.42 The first ERP-focussed Defence Communications and Information Systems Committee was held on 19 April 2021. The committee’s terms of reference were updated at this meeting to reflect that every second meeting would focus on the ERP program.

Program resourcing

2.43 The November 2020 internal audit included a recommendation that the program:

… identify the ongoing people capability requirements for ERP R1B Implementation. This should include seeking Enterprise Transformation Board agreement to the prioritisation of ERP R1B implementation activities and to obtain the commitment of essential resources from the Groups and Services.

2.44 In response to the internal audit recommendation, the program committed to returning to the ETB in February 2021 with a paper explaining the resourcing process and resource demand profile, seeking agreement for the ETB to prioritise ERP activities when making resourcing decisions.⁴³ In April 2021 the program presented a paper on its response to the audit recommendation to DCISC and the ETB. The ETB noted the paper and requested that the program return in May with a clearer approach to demand and supply of ERP workforce requirements. There was no May 2021 meeting. In June 2020 the program presented a further paper to ETB, setting out its approach to managing workforce requirements. The program asked the ETB to: endorse its actions in response to the audit recommendations; agree to its planned next steps — including returning to the ETB where resource constraint issues need to be escalated; and agree to the prioritisation of ERP implementation decisions when making resource decisions. The ETB endorsed the actions the program had taken, agreed to the next steps, and did not agree to the prioritisation of ERP implementation activities when making resource decisions.

Program assurance

2.45 In February 2021 the program presented a paper to the ETB, recommending that the ETB note assurance processes in place for the program. The ETB raised concerns with the model presented, and agreed to explore the engagement of an independent assurance professional to determine an appropriate assurance framework. In March 2021 the ETB Chair requested that the item be discussed again at the May 2021 meeting. In June 2021, the ETB discussed that a consulting firm, Proximity, had been engaged to conduct a review of the ETB. The ETB expect that the review will contribute to any future ERP assurance arrangement. The terms of reference for the review set out that it was expected to commence in late May 2021 and present its findings by mid-June 2021. On 21 July 2021 Defence advised the ANAO that the review was ongoing.

Program-level governance arrangements

2.46 Defence’s program level governance arrangements include a Program Management Office (PMO) and an internal program board. The November 2020 internal audit assigned a ‘medium’ maturity rating to the program governance arrangements, reporting that there were:

… sound governance mechanisms in place for decision making. While decisions are made by the ERP Program following consultation with the Groups and Services, accountability and responsibility for the decisions remains with the Program.

Program management office

2.47 Day-to-day management of the program is the responsibility of a dedicated PMO.⁴⁴ The PMO maintains EPMS, used to record and monitor all RADDICAL (risk, assumptions, dependencies, decisions, issues, changes, actions, lessons) items, and to generate reports.

Findings of Gateway reviews related to program level governance

2.48 As at November 2020, the program had five Gateway reviews.⁴⁵ Overall ratings as a result of the reviews were:

• 0/1 Gateway review of the ERP First Pass Business Case, provided to Defence on 11 December 2015, rated the program ‘Amber’⁴⁶;
• 2/3 Gateway review, provided to Defence on 15 December 2017, rated the program ‘Green/Amber’⁴⁷;
• Mid Stage Gateway review, provided to Defence on 17 May 2019, rated the program ‘Amber/Red’⁴⁸;
• Short Form Gateway review⁴⁹, provided to Defence on 4 March 2020, rated the program ‘Green/Amber’; and
• a second Mid Stage Gateway review, provided to Defence on 6 November 2020, rated the program ‘Green/Amber’.

2.49 The 15 December 2017 Gateway review noted that ‘balancing the needs of the Commonwealth and the commercial drivers of the partners will be critical to success’ and ‘a need to balance the role of solution delivery partners and key accountabilities within Defence Groups to ensure that the program is led and supported by Defence leaders’. The review recommended that Defence ‘establish an organisation structure appropriate to the delivery stage including appropriate staffing from ADF/APS in both leadership and decision-making roles to maintain the interests of the Commonwealth during 1st quarter 2018.’ The Department of Finance assessed this recommendation as partially implemented at the 17 May 2019 review, noting that an organisational structure had been implemented, but required revision. In response, the program noted that: it planned to have two program leads, with a program director (SES Band 2 level) focussed on the technical aspects and a business lead (SES Band 2/ADF 2 Star level) focussed on culture and change aspects; and it had commenced recruiting ADF/APS personnel in key positions in capability delivery teams, who would have private sector counterparts to lead and support their roles.

2.50 The structure described above had been implemented at the time of the ANAO’s audit fieldwork, and is illustrated in Figure 2.2 below. During the audit the program had:

• an ‘above the line’ SES Band 2 Program Director;
• an ADF 2 Star Head Business Transformation Lead;
• functional streams led by:
  • ADF/APS business leads;
  • ‘above the line’ delivery leads (SAP specialists); and
  • ‘below the line’ IBM capability leads (the contracted systems integrator).

2.51 Box 1 below summarises Defence’s use of ‘above the line’ and ’below the line’ descriptors for the program’s contractor workforce.⁵⁰

Internal Program Board

2.52 An ERP Internal Program Board was established in March 2019.⁵¹ The Board is chaired by the ERP Program Director and includes program executives, representatives from SAP and the systems integrator (IBM). Eight of the ten board members are contractors and two are APS/ADF.⁵² The board meets monthly. The ERP program management plan sets out that the board’s role is to:

• review monthly status reports;
• review key change management issues and risks;
• approve Program Change Requests⁵³ (as per delegation); and
• prepare for DCISC and ETB meetings.

2.53 The board met 16 times between May 2019 and April 2021. The board fulfilled the functions described above, with the exception of Program Change Request approval.⁵⁴ Defence advised the ANAO on 16 April 2021 that the Change Control Board was established in February 2020 to ‘progress and approve all Program Change Requests’.⁵⁵ As discussed in Box 2 on page 72, the program management plan was updated to reflect the Change Control Board’s role in March 2021. The March 2021 program management plan set out that the objective of the Change Control Board was to:

• ‘Review Project Change Requests’
• ‘Provide direction on Project Change Requests’
• ‘Decision making on Project Change Requests’
• ‘Approval / reject Project Change Requests’.

2.54 Terms of reference for the Change Control Board were approved by the CIO on 7 July 2021. The terms of reference state that the board is now ‘an advisory committee that reviews and provides guidance on Program Change Requests (PCRs) in order to provide endorsed recommendations to a Commonwealth decision-authority for approval.’ The Change Control Board’s role is further discussed in paragraphs 4.24–4.32.

Engagement with Defence business areas

2.55 The government was advised at second pass (May 2018) that key risks to the ERP program included organisational acceptance of new process, and integration with other information reform initiatives. Stakeholder engagement is a mechanism to mitigate these risks. Defence has developed a range of consultative forums to support ERP program governance and implementation.

2.56 In June 2017, Defence contracted Ernst and Young (EY) as an Organisational Change Manager (OCM) to assist with engagement with Defence business areas. The OCM’s role has involved development of stakeholder management and communications plans to support engagement.⁵⁶ Stakeholder engagement activities set out in the plans include three consultative forums (see Table 2.3 below).

2.57 Defence advised the ANAO in January 2021 that each forum has been implemented as planned. As set out in Table 2.3, the SES Band 1/ADF 1 Star Group is the highest level forum, intended to support governance through provision of advice to the ERP program, and support program implementation through resourcing. Once implemented, the SES Band 2/ADF 2 Star steering committee (see paragraph 2.41) is expected to ensure the program and the ETB are receiving strengthened support in these areas.

2.58 Although the forums have been implemented, the November 2020 internal audit found that the relationship between the governance committees and consultative forums could be more clearly defined. Defence advised the ANAO in May 2021 that it would commence work in mid-2021 to review, update and approve Terms of Reference documentation for ERP program committees and consultative groups to ensure relationships are more clearly defined.

Did Defence develop effective monitoring and reporting arrangements that provide senior leaders with adequate visibility of progress and emerging risks?

Internal monitoring and reporting

2.59 There is regular reporting on the ERP program’s progress to relevant senior leaders, including the CIO as the Senior Responsible Officer, and to enterprise-level Defence committees with a governance role in the program.

2.60 To support monitoring by Defence senior leaders, the following reports are provided:

• internal status reports; and
• the Chief Information Officer Group Quarterly Performance Report.

2.61 In addition, the CIO provides scheduled reports to the Defence Audit and Risk Committee on the status of the program. These reports were included in the committee’s forward work plan from 2020. The 31 March 2021 report included advice that ‘the ERP has progressed Tranche 1, received government approval for Tranche 2 (Case Management) and progressed planning for Tranche 3’.

Internal Status reports

2.62 The ERP program handbook sets out reporting arrangements to support program governance. These reporting arrangements are described in Table 2.4 below.

2.63 The following section examines the reports set out in Table 2.4. Reports were examined by the ANAO to determine whether the content was appropriate for supporting relevant decision-makers, including whether there was information on progress, risks and issues. The accuracy of the data used in the reports was not tested.

Weekly stream status reports

2.64 The ANAO examined examples of weekly stream status reports, including all 83 reports submitted in October 2020. The reports involve detailed stream level information, appropriate for the audience.

Weekly program status reports

2.65 The ANAO examined examples of weekly program status reports, including all nine reports submitted in September and October 2020. The reports involve a narrative update, along with a traffic light indicator on matters such as benefits management, scope, schedule, resource, budget, security, and dependencies. The reports provided management information to support the CIO in the role of Senior Responsible Officer.

Monthly program status reports

2.66 The ANAO examined the 18 monthly program status reports issued between October 2019 and April 2021. The reports examined provided: a timeline with key milestones, including IOC and FOC for Release 1A and Release 1B; a high level summary of current status; and a traffic light indicator of progress against schedule.

2.67 The ANAO observed that:

• the timeline was often illegible in the format included in the monthly report⁵⁷;
• information on workstreams tended to be technical, with prevalent use of acronyms. Senior leaders would require detailed knowledge of the deliverables to interpret the report⁵⁸; and
• there was scope for improved information to assist the reader to understand whether deliverables and milestones have been delivered on time (further discussed in paragraphs 4.21–4.23).

2.68 The monthly program status reports reviewed by the ANAO initially presented only enterprise and program risks that have a residual ‘high’ and ‘extreme’ rating after treatment of the risk. As there were no residual ‘high’ or ‘extreme’ enterprise or program risks between July 2020 and January 2021⁵⁹, there was limited visibility of enterprise and program risks to senior leaders. From November 2020, Defence commenced reporting on enterprise and program risks with a ‘high’ and ‘extreme’ risk rating before treatment. This provides improved visibility of the program risk profile to senior leaders.

2.69 In January 2020 the Enterprise Business Committee was provided with a list of ERP program strategic risks separate to the monthly report. The committee noted the approach, agreed to the ERP related enterprise-level risks under active management, and agreed that Group and Service representatives continue to develop and transition relevant risks into their risk management practices.

Monthly financial status reports

2.70 The ANAO examined all examples of monthly financial status reports provided to the ETB. Between September 2019 and April 2021, financial reports were presented at 11 out of 17 meetings. The reports provide a summary of issues, and track spending against the program budget. They provide visibility to senior leaders of key issues and what has been spent.

Chief Information Officer Group Quarterly Performance Reports

2.71 The Chief Information Officer Group quarterly performance report commenced in June 2019, and followed the format of the Capability and Sustainment Group’s (CASG) quarterly performance report.⁶⁰ The ANAO examined all six quarterly reports issued between June 2019 and September 2020. Each of these reports included the ERP program as a key ICT project, with information on risk (to capability, schedule and cost), progress against timeframes, and progress against budget presented. Defence advised the ANAO that recipients of the report include the Minister for Defence, the Secretary and Associate Secretary. The ERP program has used the narrative section of the quarterly report to present some additional information on risk.⁶¹

2.72 The report also includes categories for projects of interest and projects of concern.⁶² The ERP program has not been included in either category.

2.73 In the initial June 2019 quarterly performance report the ERP program was reported as ‘amber’ (some concerns, being managed, requires monitoring). In all subsequent reports the ERP program was reported as ‘green’ (on track to deliver approved scope). This included the September 2019 report, which presented the approved IOC date of 31 October 2020, and the forecast IOC date of 31 July 2022. At this stage, the revised (July 2022) IOC date was yet to be approved (see paragraphs 2.24–2.26 for further information on the change in IOC date). Quarterly reporting has been on hold since Quarter 4 2020. Defence advised the ANAO that a new ‘Capability Report’ is expected to commence in September 2021, with the aim of improving the timeliness and quality of advice to government.

Reporting to government

2.74 Defence also provides quarterly program updates to the Defence Minister and six monthly reports to the government. A number of quarterly reform priority reports have also been provided to government.

Quarterly program updates to the Minister

2.75 Quarterly reports were provided to the Defence Minister for: Quarter 3 2018; Quarters 1, 2, 3 and 4 2019; and Quarter 3 2020.⁶³ On 17 April 2020, the ETB agreed that quarterly updates to the Minister would not be required in the quarters where the six monthly reports for the government were produced, and that the Quarter 1 update was not required. For that reason, there were also no Quarter 2 or 4 2020 reports to the Minister. Defence has further advised that there was no Quarter 1 2021 update as there was a six monthly report waiting to be considered by government during Quarter 1, and a change in the Minister. The new Minister commenced 30 March 2021.

2.76 The reports provided a high level summary of status, which did not include financial information. The reports presented limited information on progress against timeframes.

Quarterly reform priority reports

2.77 In December 2019, the Minister for Defence provided a letter to the Prime Minister with an update on Defence’s reform agenda, setting out the lines of effort and initiatives expected to build on the implementation of the First Principles Review. This reporting included the ERP program, which was said to:

… provide an opportunity to eliminate complicated and unnecessary structures, processes, systems and tools, and to replace them with a single, trusted source of accurate, near real-time information.

2.78 The Defence Minister received five quarterly reports on Defence’s reform program from Quarter 3 2019 to Quarter 3 2020. The ERP program was included in these reports from Quarter 1 2020 (three reports). The reports that included the ERP program advised that the overall program was ‘on track’, and that some activities within the program were ‘off track’ or ‘at risk’.

2.79 Quarterly reform priority reporting ended in Quarter 3 2020, with the agreement of the Defence Minister. Defence advised the Minister that new reform reporting arrangements would be considered for the Lead the Way: Defence Transformation Strategy⁶⁴ launched on 27 November 2020. The ERP is not an area of focus in Lead the Way, although the strategy notes the importance of reforming processes ahead of future tranches of the ERP program to ‘obtain maximum value and benefit as early as possible’.

Six monthly reporting to government

2.80 At first pass, the government agreed that Defence would report to the Minister for Defence every four months, and provide routine reporting to the Secretaries’ Committee on National Security at key milestones. At second pass government agreed that Defence would report on progress quarterly to the Minister for Defence and six monthly to the government.

2.81 Six monthly reporting to government commenced in November 2019, with further reports provided in May and December 2020. The ERP program advised Defence senior leaders that the delay in commencing six monthly reporting was due to the timing of the 2019 federal election and changes to the Ministry.

2.82 Six monthly reporting includes a high level summary of progress, an attachment with further information on matters such as risk and schedule, and recommendations. In the three reports provided to the government as at December 2020, there was limited information on progress against budget.⁶⁵ The three reports provided a statement as to whether the program was on track against agreed timeframes. While the timelines provided as part of the reporting did not set out Release 1A or Release 1B IOC or FOC dates, some of this information was included in the report narrative.⁶⁶

2.83 The six monthly reports have included information on risk. Risk levels have largely remained stable, with the following exceptions:

• cost risk increased from ‘medium-high’ to ‘high’ in November 2019. The relevant report stated that the reason was the re-baselining of IOC from late 2020 to mid-2022; and
• schedule risk decreased from ‘high’ to ‘medium-high’ in November 2019. The reported reason was the re-baselining of IOC, with a further delay to IOC recommended by Defence in December 2020 to maintain a medium-high schedule risk.

2.84 Changes to the program’s risk profile over time are set out in Table 2.5 below.

Table 2.5: Risk changes since the second pass business case

Source: Defence records

2.85 The ANAO’s review of the risks reported to government compared to the risks reported internally (see paragraph 2.68) indicates that the risk profile presented to government is higher than the risk profile presented internally.

2.86 Defence advised the ANAO that it is aware that risk is not presented consistently in its internal reporting and its reporting to government, and advised that it had begun work on a solution. In May 2021 Defence provided evidence that it has included the risks reported to the government in EPMS, with links to corresponding program and stream level risks. Defence has also advised that these risks will be reviewed monthly as part of the regular monthly risk review sessions with ERP program senior leadership (discussed in paragraph 2.15) and directly extracted from EPMS for reports to the government. Defence should monitor its internal reporting and its reporting to government on ERP program risks, to ensure there is consistency in program risk reporting.

3.1 The second pass business case for Tranche 1 of the ERP program was approved by the government in June 2018. The approved funding for acquisition for Tranche 1 was $257.9 million (out-turned).⁶⁷ Tranche 1 activities included the procurement of a systems integrator. Systems integration services are material to the successful delivery of Tranche 1 and represent more than 50 per cent of the total program budget from 2019–20 onward.

3.2 This chapter examines Defence’s procurement process for Tranche 1, with a focus on the systems integrator procurement. The ANAO examined whether Defence:

• conducted an effective process to identify its requirements, risks and procurement approach for the systems integrator;
• conducted a tender process for the systems integrator that complied with relevant procurement requirements; and
• established appropriate probity and conflict of interest arrangements for the systems integrator procurement activity.

Did Defence conduct an effective process to identify its requirements, risks and procurement approach for the systems integrator for Tranche 1?

3.3 To assess whether Defence conducted an effective process to identify its requirements, risks and procurement approach for the systems integrator function in the context of Tranche 1, the ANAO reviewed Defence’s identification of:

• program level technical requirements and risks; and
• requirements, risks and approach for procuring a systems integrator.⁶⁸

3.4 Defence’s procurement of the systems integrator is discussed in paragraphs 3.24 to 3.57.

Initial identification of program-level technical and procurement requirements

3.5 Defence developed high-level requirements for the ERP program during the Program Definition Phase.⁶⁹ These were drawn from sources including the Defence ERP Strategy and Roadmap, other ERP implementations⁷⁰, the Defence Finance Systems Strategy, other Defence ICT programs that were planned or underway, and a modelling tool. Key documents developed by Defence that record the technical and procurement requirements identified for the ERP program include:

• two ‘Business and Technical Architecture’ documents which define and describe the ERP solution (the Program Definition Phase Business and Technical Architecture document, and Business and Technical Architecture document); and
• a Requirements Management Strategy which describes how the ERP program will collect, use and manage requirements to manage stakeholder needs and deliver program outcomes, and defines the role of requirements in supporting program decision-making.

3.6 These documents are discussed in more detail below.

Business and Technical Architecture documents

3.7 The Program Definition Phase Business and Technical Architecture document, dated 11 August 2015, was developed in the context of the pre-first pass business case Program Definition phase of the program. It states that it:

… defines and describes the Defence Enterprise Resource Planning (ERP) solution from a business and technical architecture perspective.

3.8 The document was developed based on the following inputs:

• business reference architectures developed between February 2014 and May 2015;
• stakeholder engagement across Defence;
• use of a modelling tool as a baseline from which to develop Defence’s standardised business processes;
• 14 process and functional requirements workshops held between 7 April and 11 May 2015 facilitated by an external consultancy (Deloitte), and 41 hours of stakeholder and subject matter expert consultation throughout Defence, to design and validate high level processes and associated requirements derived from the modelling tool baseline, and to identify related business requirements;
• a collection of non-functional requirements for business processes developed through four workshops held between 14 and 20 May 2015⁷¹; and
• six end-to-end processes and two other process groups that were developed to provide a view of how the ERP would enable business processes across Defence.

3.9 Process and functional requirements workshops resulted in the development and validation of 70 high level business requirements associated with business processes, the capture of a repository of 1260 detailed business requirements for logistics and finance, and the development and validation of 47 high level non-functional requirements. The Business and Technical Architecture set out that future solution design work would include the development and validation of detailed business process and associated business and non-functional requirements.

3.10 The high level requirements were endorsed in principle by the ERP Reference Group⁷² on 8 May 2015. High level non-functional requirements were endorsed by the ERP Reference Group in July 2015.

3.11 The Business and Technical Architecture document, finalised on 23 November 2017, was developed as part of the second pass business case, as well as a supporting document for the Request for Tender for a systems integrator. Its stated purposes were as follows:

This document defines and describes the Defence Enterprise Resource Planning (ERP) solution from a business and technical architecture perspective. It establishes the scope of the ERP solution, identifying the key information elements, applications, and technology considerations to deliver the target state. It incorporates work previously completed by the Solution Definition Stream during the ERP Program Definition Phase updated by Defence’s Strategic Partner based on changes that have occurred since the document was originally published.

3.12 The document further outlined 12 program outcomes (discussed in paragraph 4.50) and outcome characteristics that were to provide the strategic requirements for the ERP Business Architecture. It also stated that the high-level capabilities of the ERP solution were to be further developed to drive development of business and non-functional requirements.

The Requirements Management Strategy

3.13 The Requirements Management Strategy was finalised on 4 December 2017. The strategy set out the pre-first pass requirements baseline from which initial scope boundaries for the program were identified: Finance; Logistics; Engineering; Maintenance; and Estate. The strategy set out that the program would inherit high-level requirements from the SAP Model Defence Organisation (MDO, see Appendix 3), which Defence would expand on through:

• SAP’s development of the S/4HANA solution; and
• the identification of gaps and changes in the processes when the SAP MDO was compared with how Defence conducts its business.

3.14 The requirements were to be mappable to one or more of the 12 program outcomes including outcome characteristics.⁷³ Additionally, detailed requirements as needed in specific areas were to be developed during the Explore phase of Tranche 1.

3.15 Requirements were further refined through the tender process through Offer Definition and Improvement Activities (ODIAs) (discussed further below).

Identification of systems integrator procurement risks

3.16 Defence prepared a risk assessment specific to the systems integrator procurement as part of preparing an Endorsement to Proceed prior to the procurement. Once advised that an Endorsement to Proceed was not required⁷⁴, these risks were integrated into the program’s risk and issue registers and managed in those documents. The register evidences that assessment of systems integrator procurement risks began at least as early as December 2015. The Endorsement to Proceed risk register also outlined risk treatments for each risk, the status of these, and the residual post-treatment risk assessment. Additionally, the concept procurement plan outlined how the procurement and contracting method would mitigate certain risks relevant to achieving a successful systems integrator procurement.

3.17 Defence maintained program risk and issue registers during the procurement, dating between July 2016 and March 2017. The registers included risks and issues specific to the systems integrator procurement, evidencing risk assessment and management activity by the program in relation to the procurement. These risks and issues were substantially similar to those outlined in the risk register prepared for the Endorsement to Proceed.

3.18 The 2017 RAIDD (Risks, Assumptions, Issues, Dependencies and Decisions) risk management strategy outlined processes for identification, analysis, allocation and treatment of risk applicable to the procurement. Risks identified during the 2018 Independent Assurance Review resulted in a Post-Tender Submission ODIA process and a second tender evaluation for the systems integrator procurement.⁷⁵

Development of the procurement approach

3.19 The procurement approach for the systems integrator was developed and refined through several key program documents prior to and during the procurement, as set out below in Table 3.1 below. Initial development of the approach in 2015 was undertaken by external consultants (Deloitte) as a strategic partner, with work continued by the ERP Implementation Office.

3.20 Defence conducted market testing activities between December 2015 and January 2016 and held a market briefing on 17 December 2015. Defence published a follow-up AusTender notice on 21 December 2015, which included the slideshow from the 17 December 2015 briefing, a Market Briefing Information Pack for potential suppliers, and the answers to questions raised during the market briefing. On 13 January 2016 Defence published a Market Sounding Questionnaire on AusTender seeking insight and feedback from the market as it related to ideas and concepts described in the Market Briefing Information Pack. An August 2017 internal audit noted that there was no evidence that feedback from the Market Sounding Questionnaire had been incorporated into the development of the procurement approach.

3.21 The procurement approach chosen for the systems integrator was an open tender, involving an Invitation to Register, followed by a shortlisting process and a Request for Tender (RFT) process with the shortlisted vendors. Shortlisted vendors would engage in competitive ODIAs conducted in parallel (described as ‘joint solutioning’).

3.22 Defence planned to adopt a ‘long-term incremental contracting’ approach through a Deed of Standing Offer and work packages contracted under Official Orders.⁷⁶ Defence outlined in the System Service Concept Procurement Plan–Phase One that the rationale for long-term incremental contracting was based on:

a. Providing Defence with greater negotiation leverage due to the long term nature of the agreement without lock-in, while encouraging System Integrator(s) to offer the greatest value due to increased likelihood of securing future increments (capability drops).

b. Reducing ongoing procurement effort and complexity as primary and secondary System Integrator(s) are pre-qualified in contrast to Panel [Statement of Work] arrangements.

c. Effectively managing risk associated with acquiring the ERP system where the risk of acquiring the entire functionality in one step is considered unacceptable.

d. Enabling Defence to manage its risk exposure because its commitment is limited to the current set of capability drops.

e. Facilitating agile development and acquisition.

f. Aligning Defence’s acquisition and contracting process to the System Integrator(s)’s development processes.

3.23 The August 2017 internal audit found that the program’s proposed sourcing model was adequate, with governance measures and controls in place to ensure alignment with the Commonwealth Procurement Rules (CPRs) and the Defence Procurement Policy Manual.

Did Defence conduct a tender process for the Tranche 1 systems integrator that complied with relevant procurement requirements?

Tender process

3.24 Defence shortlisted potential vendors for the provision of systems integrator services through an open Invitation to Register (ITR) process conducted between May 2016 and April 2017. The ITR opened on 20 May 2016 and closed on 23 June 2016.

Invitation to Register Statement of Requirement

3.25 The Statement of Requirement included in the Invitation to Register (ITR) documents outlined that Defence intended to engage two systems integrators to deliver systems integration services using a consolidated, predominately SAP platform with integration to a limited number of ‘bolt-on’ applications where required. Defence stated that the scope of services would be refined during the Offer Definition and Improvement Activity stages of the Request for Tender selection process described in the Statement of Requirement. The proposed allocation of functional capability to three planned tranches was set out in attachments to the Statement of Requirement, with eight end-to-end processes to be enabled by the tranches.⁷⁷

3.26 Defence received six responses to the ITR, two of which were excluded due to non-compliances with the conditions of registration. The remaining four responses were assessed against evaluation criteria included in the ITR documents published on AusTender. The assessment process resulted in Accenture and IBM being shortlisted.⁷⁸

3.27 An initial RFT⁷⁹ was released on 31 July 2017 to the two shortlisted tenderers, following which three Offer Definition and Improvement Activities (ODIA) processes were conducted between 4 September and 27 October 2017. Program level and systems integrator requirements were further developed and refined through the ODIAs.

Refinement of requirements through Offer Definition and Improvement Activities (ODIAs)

3.28 ODIA workshops were attended by program staff, representatives of the shortlisted tenderers, Defence subject matter experts, the Organisational Change Management Partner (an external consultant, Ernst and Young), the Strategic Partner (an external consultant, KPMG Australia), as well as the Probity Advisor (Maddocks, a law firm) when necessary.⁸⁰

3.29 Through ODIA 1, Defence considered that it identified functional areas (transformation targets) that were fundamental to how Defence operates and had the potential to provide ‘extensive benefits’ if transformed. A report detailing ODIA outcomes was produced for each ODIA.

3.30 ODIA 2 included discussion of requirements in relation to rolling out Initial Operating Capability (IOC) and Final Operating Capability (FOC) and Defence’s requirements relating to transition and support. During ODIA 2, the ‘transformation targets’ were used to develop ‘outcome characteristics’ for the program, which when finalised would describe the end state that Defence expected the program to deliver. The outcome characteristics were to be used to: help develop Defence’s requirements; advise the process of identifying risks, benefits, business and organisational change; and develop the list of prerequisite tasks that Defence needed to action to enable the characteristics to be delivered. The ODIA 2 Report noted that the program intended to continue to engage with subject matter experts that attended the workshops as the characteristics were refined and broken into more precise requirements statements.

3.31 Prior to and in parallel with ODIA 2, Defence developed and refined a draft statement of requirement (scope of services) document, which was included in an updated RFT issued in October 2017.⁸¹ Responses were received on 22 November 2017.

3.32 At this stage it was envisioned that work packages would be divided between the two preferred tenderers on the basis of three tasking statements, as well as the Transformation Targets. The source evaluation report, finalised by the tender evaluation board on 19 January 2018, recommended that IBM be selected as preferred tenderer for Tasking Statement 1 and the Transformation Targets, IBM and Accenture be selected for Tasking Statement 2, and neither tenderer be selected at that time for Tasking Statement 3.⁸² Subsequently on 31 January 2018, at the request of the delegate (the ERP Program Manager), the tender evaluation board met with the delegate, who determined that neither tender be appointed for the Transformation Targets at that time.

3.33 On 26 March the Associate Secretary appointed the CIO as the new delegate for the systems integrator procurement. On 18 March 2018, the CIO asked the program to put additional scenarios to the tenderers to assist in making a decision on the appointment of a systems integrator to the program.⁸³ On the basis of the tenderer responses and advice received from the Program Manager on 23 May 2018, the new delegate determined on 31 May 2018 that Accenture be appointed to the program as systems integrator for Tasking Statement 1, with a decision on appointing a preferred tenderer(s) for Tasking Statements 2 and 3 and the Transformation Targets deferred to a later date.

3.34 On 17 July 2018 (prior to the appointment of a systems integrator) an Independent Assurance Review (2018 IAR) of the program as a whole commenced. The review stated that it was ‘an appropriate time, immediately following second pass approval and while essential contractors are being engaged and detailed plans are being developed, to review the overall Program strategy, planning and resourcing, to ensure that it is best placed to deliver successfully in accordance with Government decisions and Defence expectations.’ The systems integrator selection process was considered by the review. Engagement with the tenderers was suspended pending the outcome of the review. On 1 August 2018, the reviewers provided a minute to the acting Associate Secretary recommending all procurement decisions and recommendations to that date be set aside and that a lead negotiator, new Tender Evaluation Board and new delegate be appointed. The recommendation was based on:

• a ‘significant number of non-compliances, remaining uncertainties and areas requiring negotiation’ in the tender responses⁸⁴;
• ‘issues that have arisen during the [systems integrator] source selection process’ and ‘the varying judgements made about the preferred way ahead’; and
• the reviewers’ view that the source selection ‘would best be made on a more solid and informed basis than is available from the material received to date from the tenderers.’

3.35 Further, the IAR reviewers observed more generally that ‘there is a need to schedule a formal Assurance Board meeting, to address a number of elements of the Program’s organisation, strategy, resourcing and governance.’

3.36 On 27 August 2018 the acting Associate Secretary formally agreed⁸⁵ that the Chief Financial Officer (CFO) would be the new delegate and agreed to set aside all decisions and recommendations to date. The ERP Program Manager was tasked with actioning the recommendations.

Restarting the tender assessment

3.37 On 18 September 2018, the new delegate (CFO) determined that:

… neither tenderer’s offer, in its current form, is capable of being accepted without exposing the Commonwealth to unacceptable technical, delivery, financial and commercial risks and therefore neither offer currently represents value for money to the Commonwealth.

3.38 The delegate decided that the program would undertake a Post-Tender Submission (PTS) ODIA and parallel negotiations with the tenderers. PTS ODIA workshops were conducted between 26 November 2018 and 31 March 2019. The aims of the workshops were to:

a. address the solution issues identified through the evaluation of the RFT and drive down assumptions;

b. workshop the commercial non-compliances raised by IBM and Accenture in their 2018 tender responses and collaboratively develop contractual arrangements with the tenderers to reflect both Program evolution and solution workshop developments; and

c. workshop the pricing models to be applied across the Program and undertake a financial investigation of the price build up, including the build-up of labour rates.’

3.39 A Request for Updated Tender was released on 1 April 2019, closing on 30 April 2019.

3.40 A second source evaluation report, based on the updated tenders provided, was approved by the delegate on 4 June 2019. The report recommended that IBM be selected as the preferred tenderer for all tasking statements, but Accenture was not to be set aside at that stage, pending successful negotiations with IBM. IBM was considered to represent the best value for money based on its response against both the solution and financial evaluation criteria. There was no differentiation between tenderers against the commercial evaluation criteria.

3.41 A Deed of Standing Offer was signed with IBM on 19 July 2019, commencing on 19 July 2019 and expiring on 18 July 2025. On 19 July 2019 Defence signed an Official Order for the ‘Prepare’ and ‘Explore’ phases of Tranche 1 with IBM, valued at approximately $95.5 million, with the arrangement commencing on 22 July 2019. Defence outlined in its section 23 commitment approval documentation⁸⁶ a rationale for the value for money expected to be achieved. The rationale focussed on IBM’s experience in delivering similar programs; the skills of key people; how costs would be managed including the agreement for fixed price element and for benchmarking and open book pricing on other elements; as well as incremental work package contracting and Defence’s contractual right to terminate the agreement and engage a third party for the remainder of the program.

Defence Procurement Policy Manual Compliance

3.42 Defence was largely compliant with Defence Procurement Policy Manual requirements, with the exception of an identified instance of non-compliance with financial delegations and Defence policy designed to meet the requirements of the Public Governance, Performance and Accountability Act 2013 (PGPA Act): subsection 23(3) delegation; and subsection 23(1) delegation.⁸⁷

3.43 The Defence Procurement Policy Manual requires that these delegations are exercised in the following order, unless the procurement does not involve the commitment of relevant money in which case only the enter into an arrangement delegation is required:

• subsection 23(3) — commitment approval; and
• subsection 23(1) — enter into an arrangement.

3.44 The Defence official who signed the section 23 commitment approval documentation for Official Order 1 on 18 July 2019 was acting in the role of Program Manager ERP, and did not have the necessary delegation to approve the commitment of $95,413,351.37.

3.45 Defence advised the ANAO that between 10 July and 18 July 2019 (inclusive), the role of Program Manager ERP was an O6/EL2 equivalent position, with a standard Commitment Approval delegation of $5 million as per the Department of Defence Public Governance, Performance and Accountability Delegation (FINMAN 2). The purported exercise of the delegation was contrary to the Defence Procurement Policy Manual and the relevant version of Accountability Authority Instruction 15, which stated that Defence officials:

… must not exercise any of the financial delegations in FINMAN 2 unless they are a person holding, or are for the time being acting in, or have been directed to perform the duties of a position or class of positions listed in the relevant delegations schedule …

… must comply with the terms and conditions of the relevant delegations schedule in FINMAN 2 and … must not depart from any requirements, directions or limits set out in FINMAN 2.

3.46 The delegations issue was brought to Defence’s attention during the course of this audit. In response, Defence completed an investigation of the exercise of delegations in the ERP program. Defence advised the ANAO that no further instances of improper exercise of delegations have been identified.

3.47 The Defence official (the CIO) who subsequently entered into the financial arrangement with the systems integrator, did have the necessary delegation to both commit the Commonwealth to expenditure under subsection 23(3) and enter into an arrangement under subsection 23(1). However, FINMAN 2 sets out that the subsection 23(1) delegate must not enter into an arrangement that commits relevant money unless it has been approved by a subsection 23(3) commitment approver. Accordingly, the invalid exercise of the commitment approval delegation meant that the requirements of FINMAN2 and Accountable Authority Instruction 15 with respect to the ‘enter into an arrangement’ delegation were not satisfied.

3.48 An additional breach of Defence policy was identified during the course of the audit. FINMAN 2 sets out that a subsection 23(1) delegate must not enter into an arrangement that commits relevant money if the value of the arrangement exceeds the amount approved by the subsection 23(3) delegate. The commitment approval for IBM Official Order 3, signed on 19 June 2020, does not provide sufficient funds for the maximum contract value, with $263,963.07 less than the total contract price estimate of $4,591,282.17 approved, resulting in a breach of FINMAN 2 and Accountable Authority Instruction 15.

Commonwealth Procurement Rules compliance and AusTender reporting

3.49 Defence has largely complied with the mandatory requirements of the Commonwealth Procurement Rules (CPRs) during the systems integrator procurement process, with two exceptions.

3.50 The exceptions were: there was no evidence of consideration of the environmental sustainability of the proposed goods and services; and Official Order 3 was published on AusTender seven days after the required date.

Consideration of financial and non-financial costs

3.51 The CPRs require officials, when conducting a procurement, to consider the relevant financial and non-financial costs and benefits of each submission including, but not limited to the environmental sustainability of the proposed goods and services (CPR 4.5). There was no evidence of consideration of the environmental sustainability of the proposed goods and services.⁸⁸

3.52 In relation to the requirement to consider relevant financial and non-financial costs and benefits of tender submissions when assessing value for money, the ANAO sought advice from the Department of Finance (as the CPR policy owner), which advised that the following interpretation was consistent with the intention of the CPRs:

The statement of the section notes that “an official must consider the relevant financial and non-financial costs and benefits of each submission including, but not limited to…” and then proceeds to list the six conditions that are to be included. The important point is that officials are directed that they must consider these costs and benefits.

As they must be considered, it is a reasonable assumption that to comply with record keeping requirements of how value for money was considered and achieved (CPR 7.2 & 7.3.c) for a procurement, a documented consideration in some form should be kept even if to state that in a particular case a condition was not relevant in determining value for money.

3.53 Defence guidance reviewed by the ANAO did not include instructions for Defence officials regarding record keeping expectations for requirements that were considered but determined to not be relevant to the particular procurement. To ensure a complete record of decisions made, there is scope for Defence to improve its guidance for officials on how to document decisions about CPR requirements that are considered or determined to be not relevant to a particular procurement activity.

Reporting on AusTender

3.54 The Commonwealth Procurement Rules require that all contracts or amendments by non-corporate Commonwealth entities (such as Defence) at or above $10,000 be reported on AusTender within 42 days of entering into (or amending) a contract (CPR 7.18–19).

3.55 Official Order 3 was published on AusTender seven days after the required date. Defence advised the ANAO that this was due to a technical issue with the automatic upload of information to AusTender from ROMAN.⁸⁹ The program had completed all documentation to enable timely AusTender reporting.

3.56 Further, ANAO analysis of AusTender entries during the audit identified a number of additional issues with Defence’s AusTender reporting:

• dates were incorrect, with the start dates recorded in AusTender for Official Order 2 and variations 3, 4 and 6 to Official Order 1 preceding the execution dates. Defence advised the ANAO that the contract and amendment start dates in AusTender were erroneous;
• the total contract value for Official Order 1 in AusTender did not reflect all variations to date, with the value in AusTender being $254,908.24 higher than Defence’s records⁹⁰; and
• inaccurate values were reported, including the value of Official Order 3 reported on AusTender being $263,889.87 lower than the Official Order.

3.57 For the purpose of providing timely and accurate reporting on AusTender, Defence should consider reviewing its system(s) for uploading data to AusTender and undertake quality checking of reported data.⁹¹

Did Defence establish appropriate probity and conflict of interest arrangements for the Tranche 1 systems integrator procurement activity?

Probity advice

3.58 Defence appointed an external probity advisor (Maddocks, a law firm) for the ERP program on 15 December 2015, with the procurement taking place through the Defence Professional Services Standing Offer Panel. There are four contracts for these probity advice services disclosed on AusTender, dated 10 March 2016 to 30 April 2016⁹², 31 May 2016 to 30 September 2016, 25 July 2017 to 30 June 18, and 1 July 2018 to 30 June 2019⁹³, which cover the entire period that Defence undertook systems integrator tender processes.

3.59 There is evidence that the probity advisor conducted 23 probity briefings between December 2015 and May 2019. While attendance records cannot be located, Defence has advised that probity briefings were held for all evaluation team members. The advice and recommendations provided by the probity advisor during the procurement are recorded in the conflict of interest register discussed at paragraph 3.62.

Probity framework and plan

3.60 The ERP probity framework was developed by the probity advisor. It was issued in January 2016, revised in September 2018 and updated in 2020. The framework states that it applies to:

… personnel who may be involved in [the program] or who may have access to information relating to [the program], including: APS employees; Australian Defence Force personnel; and any Advisers or consultants (including sub-contractors) to Defence working on, considering, or providing services in relation to [the program].

3.61 A probity plan, intended to build on the probity principles in the framework in relation to specific issues, was first issued in June 2016 and revised in September 2017. The plan outlines communications protocols and business as usual protocols and also includes protocols for:

• tender evaluation;
• gifts and hospitality;
• offers of employment;
• business meetings and social functions; and
• confidentiality and information security.

Implementation of the probity framework and plan

3.62 A conflict of interest register and a contact register were established, in line with the 2016 probity framework requirement.⁹⁴

3.63 The 2018 version of the framework set out that all program personnel were to be provided with a copy of the probity framework. Personnel were required to:

• complete a confidentiality acknowledgement and a declaration of interests form; and
• return a signed copy of the probity framework acknowledging they have read and understood their obligations, roles and responsibilities under the framework.

3.64 The probity contact officer was responsible for maintaining a register of acknowledgements of the probity framework and a register of confidentiality agreements. Defence advised the ANAO that rather than requiring staff to return the signed probity framework, personnel were agreeing to the probity framework and confidentiality agreement by signing the conflict of interest declaration.⁹⁵ The change in practice was not reflected in the probity framework in place during the procurement process. The probity framework was further updated in October 2020 (during the course of this audit) and reflects the change. Personnel are now required to read, understand and comply with the framework, rather than sign and return it.

3.65 The ANAO’s review of the conflict of interest register showed that of 50 project personnel with decision-making roles, three personnel were not listed in the conflict of interest register. The three personnel were members of evaluation working groups and the tender evaluation steering group. During the course of this audit, Defence provided the ANAO with: statutory declarations from the two tender evaluation working group members stating that declarations were completed and no conflicts were declared; and a declaration of interest form from the tender evaluation steering group member, which listed interests which were not relevant to the ERP program.

3.66 Of the 50 project personnel with decision-making roles, ten registered a conflict. For nine of these there was a documented strategy to manage the conflict. There was no management strategy for the tenth, as the declared conflict of interest related to a different procurement, in which the individual was not involved.

Other probity issues

3.67 The ANAO’s review focussed on probity in the procurement process for the Tranche 1 systems integrator, as discussed above. During the course of the audit, a number of specific probity issues were identified which relate to the management of probity in the program more generally. These issues pertained to the management of: conflicts of interest; use of panel arrangements for this program; gifts and hospitality; and the use of official information.

Conflict of interest

Chief Information Officer

3.68 On 23 October 2019, there was questioning in the Senate Foreign Affairs, Defence and Trade Legislation Committee regarding an allegation that the son of Defence’s CIO worked at a company (Sinapse Pty Ltd) contracted by Defence.⁹⁶ The concern was that the CIO had a financial delegation for the letting of the contract. The CIO was present at the hearings and stated that although there had been no financial benefit for his son, ‘On reflection, yes. I think there was a chance of a perceived conflict of interest. I do acknowledge that’. When asked about the nature of the capability, product or service provided by Sinapse Pty Ltd, the CIO stated that:

It was a range of services, but predominantly with respect to an ability to undertake an independent assurance review around the ERP program, with particular respect to organisational change management plans.

3.69 The ANAO reviewed the contract signed by the delegate (the CIO) and Sinapse Pty Ltd on 19 December 2018. Under the contract, Sinapse was engaged to undertake ‘an independent expert assessment of the Enterprise Resources Planning Program Organisation Change Management approach’. Defence advised the ANAO that from 26 August 2019, Sinapse has also been engaged under the 2017 Ernst & Young Standing Offer (through three official orders) as an approved contractor for the purposes of ‘Change Leadership, Business Adoption and Stakeholder Management & Communications and Engagement work streams’.

Senior ERP program contractor

3.70 In September 2020, a potential improper use of position and conflict of interest issue in the ERP program was brought to the attention of Defence senior leaders. The issue involved an allegation that a senior ERP program contractor had discussed their partner’s employment situation with another contractor in the program. Subsequent to the discussion, the senior ERP contractor’s partner was employed by that contractor’s company. This resulted in an investigation by Defence’s Audit and Fraud Control Division. The investigation identified that there had been a breach of Defence policy by the contractor through a failure to identify and declare a perceived conflict of interest, as required by the contractor’s contract. The investigation found that the contractor’s conduct:

… can be described as naïve, technical breaches of Defence policy, however of a low level of seriousness and lacking intent or malice.

3.71 The investigation identified that the key contributor to the breach occurring was a lack of support through induction and training, and that there was a need to ensure contractors are educated to have a full understanding of Defence policy and governance requirements.

3.72 Subsequent to the investigation, the individual concerned completed a new conflict of interest declaration. However, this was not completed until five months after the investigation concluded, and the declaration had not been registered in the Conflict of Interest declaration log for the ERP program provided to the ANAO by Defence on 25 May 2021.

3.73 The ANAO also identified that this individual was a company director of a SAP consulting firm, and that the directorship had not been documented in Defence’s conflict of interest documentation provided to the ANAO. Defence advised the ANAO in July 2021 that it had taken action on this matter.

3.74 On 24 May 2021, Defence advised the ANAO that contractors to Defence did not have a conflict reporting requirement other than what was included in a contract and those related to specific projects. Defence further advised that the Chief Information Officer Group has developed a new contractor integrity framework. The framework was approved by the CIO on 1 July 2021 and issued as a Chief Information Officer Directive, applicable to all Chief Information Officer Group personnel. The framework requires contractors to complete:

• a ‘contractor conflicts deed’ at the commencement of the engagement with Defence;
• a ‘contractor conflicts declaration’ at the commencement of the engagement with Defence, at each anniversary of the commencement of the engagement, at any time there is a concern that a conflict of interest may, or has arisen, and where there is a material change in the engagement; and
• the ‘Contractor Conflicts Training Module’ within one month of the commencement of the engagement with Defence and then annually.

Use of panel arrangements

3.75 A panel arrangement is ‘the end result of a procurement process, where a number of suppliers are appointed through a contract or deed of standing offer.’⁹⁷ Where there is a panel arrangement, procurement can take place directly with any supplier on the panel.

3.76 ‘Above the line’ contractors in the ERP program⁹⁸ are largely recruited through labour hire firms included on Defence procurement panels. The ANAO identified instances of individuals being recommended for a role or roles in the ERP program by a program contractor they had previously worked with. The available evidence indicates that these individuals were subsequently matched by the program with labour hire firms appearing on Defence procurement panels, and subsequently engaged as contractors on the program through those firms. This approach reduces competition for roles in the program.⁹⁹ Paragraph 5.1 of the 2020 Commonwealth Procurement Rules states that:

Competition is a key element of the Australian Government’s procurement framework. Effective competition requires non-discrimination and the use of competitive procurement processes.

3.77 Auditor-General Report No. 4 2020–21 Establishment and Use of ICT Related Procurement Panels and Arrangements noted that:

Panels are a common procurement mechanism in the Australian Government sector and procurements from panels and similar arrangements are often perceived as requiring less time and effort, particularly when compared to the cost and time required to undertake an open approach to market. However, panels should not be used to eliminate competition. In addition … the CPRs state that procurements from standing offers (which most panels are) are not subject to the rules in Division 2 of the CPRs. As the Division 2 rules (which require high value procurements to be available to the open market unless certain conditions are met) no longer apply, buyers can approach a single provider from a panel and still comply with the CPRs. This occurs irrespective of the procurement value, provided the officials responsible for the procurement are satisfied, after reasonable enquires, that the procurement achieves a value for money outcome. It is essential to bear in mind that when using a panel, each use of the panel is a separate procurement and entities need to ensure they adopt processes that are not only technically compliant with the CPRs but are also consistent with the intent of the CPRs, which is to drive value for money through competition…¹⁰⁰

3.78 Defence advised the ANAO in July 2021 that:

Defence is undertaking a review into its procurement, commercial and financial practices within the Enterprise Resource Planning Program. This has identified areas for improvement and we will capture lessons learned that can be applied to other procurement/commercial and financial activities across the department.

Defence takes every opportunity to openly compete contracted work. Sometimes this requires approaching the market using alternative methods. In this instance, suitable candidates were located from across Australia with significant experience in the delivery of an extremely complex ERP Program. The category of SAP skills required are limited in the marketplace as it is a highly contested segment, with few suitably experienced resources to assist in the successful delivery of the program.

Undertaking this procurement approach means that the Department placed more weight on capability, than any other criteria, in the delivery of contracted services. The resources selected are highly regarded in their field of expertise. The procurement approach significantly reduced both delivery and financial risks. This is in line with the Commonwealth Procurement Rules, where both financial and non-financial benefits/risks must be taken into account in any Commonwealth purchasing decision.

Gifts and hospitality

3.79 Defence’s Financial Policy on Gifts and Benefits sets out that offers of hospitality can only be received:

… if it assists Defence to develop and maintain constructive relationships with stakeholders and does not give rise to either an actual, potential or perceived conflict of interest or compromises the reputation of the Australian Government, the APS or Defence.

3.80 As the ERP Program Management Office is responsible for managing contracts, accepting hospitality from a vendor can give rise to a conflict of interest. This risk arises for both Defence personnel and contracted program personnel. The policy requires gifts and benefits, including hospitality, to be recorded on the Defence Gifts and Benefits Register, excluding ‘working meals’.¹⁰¹ The policy does not require declarations of offers of hospitality that have been declined.

3.81 An extract of Defence’s Gifts and Benefits register, provided to the ANAO on 24 May 2021, documents eight instances of hospitality received by program personnel from vendors between August 2019 and February 2021.

3.82 The ANAO’s review of Defence records identified a further 10 instances of program personnel being invited to lunch, dinner or drinks by a vendor, on dates other than those recorded in the register. In respect to these 10 instances, Defence advised the ANAO that:

• in one instance a contractor provided hospitality estimated at under $100 for each member of Defence personnel in attendance. Permission was granted by the CIO, and there was an oversight in recording this event in the Defence Gifts and Benefits register;
• in one instance a contractor provided hospitality estimated at under $100 for each member in attendance. Defence advised that there was a misinterpretation of Defence policy, and personnel involved did not believe the instance required reporting in the Defence Gifts and Benefits register;
• in three instances no hospitality was offered or provided to Defence personnel, with individuals paying for themselves; and
• in five instances records do not confirm whether the event took place.

Use of official information

3.83 In the course of reviewing Defence documentation the ANAO identified a number of instances of senior program personnel sending Defence information relating to the ERP program from their Defence email accounts to external email accounts.

3.84 The ANAO analysed the email caches of senior ERP personnel and identified instances where information was not handled in accordance with Defence requirements.

• The ANAO identified 423 emails, up to the classification of ‘protected’, sent from the Defence accounts to non-defence email accounts.
  • Emails were sent to their personal, business and/or other government entity email accounts.
• The emails related to the monitoring, reporting and operations of the program, as well as some documents relating to technical architecture.
  • Three emails related to another Defence ICT program.
  • Most of the emails included attachments.
  • The misuse of security classifications in a number of emails enabled them to be sent to an external email account.

3.85 For the analysed senior ERP personnel, the ANAO identified 29,961 instances of program personnel transacting or being transacted with using external or personnel email accounts rather than their Defence email accounts.

3.86 The ANAO brought these matters to the attention of Defence’s Audit and Fraud Control Division in May 2021 and provided an update in June 2021. Defence advised the ANAO in July 2021 that it had taken follow-up action.

3.87 The probity issues discussed in this section relate to the management of probity in the program as a whole. Defence should review its probity arrangements for the ERP program and apply lessons learned to similar programs.

4.1 The First Principles Review: Creating One Defence was released publicly by the Minister for Defence in April 2015. One of the First Principles Review’s six key recommendations was to ‘fully implement an enterprise approach to the delivery of corporate and military enabling services to maximise their efficiency and effectiveness.’ Robust contract management arrangements are important in ensuring this strategic priority and ERP program objectives are achieved.

4.2 This chapter examines Defence’s contracting arrangements for Tranche 1 of the ERP program, with a focus on the systems integrator procurement and the achievement of program objectives and Defence strategic priorities. The ANAO examined whether:

• the systems integrator contract reflects Defence’s authorised negotiation outcomes;
• Defence has developed contractual arrangements for systems integrator services with clear milestones and performance expectations;
• Defence’s contract and program governance arrangements enable monitoring, management and reporting on the achievement of Tranche 1 outcomes; and
• Tranche 1 is contributing to ERP program objectives and Defence strategic priorities.

Does the systems integrator contract reflect Defence’s authorised negotiation outcomes?

4.3 The Contract Negotiation Directive for the systems integrator procurement was approved by Defence’s Chief Information Officer (CIO) on 11 June 2019. The directive authorised a lead negotiator (Ngamuru Advisory), whose services were procured from a Defence panel arrangement for negotiation services, to negotiate the appointment of a systems integrator to deliver the ERP program. The objective of the negotiation was to finalise, and recommend to the delegate for signature, a Deed of Standing Offer and an Official Order between the Commonwealth and IBM (Defence’s preferred tenderer for the systems integrator role) for the ‘Prepare’ and ‘Explore’ phases of Tranche 1. The intention was to enter into a Deed of Standing Offer and Official Order 1 by 1 July 2019.¹⁰²

4.4 A Negotiation Issues Matrix was included as an annex to the directive. This matrix divided 63 negotiation issues into three categories — commercial, financial and solution — with each characterised as minor, significant or critical.¹⁰³ IBM’s stated position and the Commonwealth’s negotiation objective and minimum fall-back position were set out for each issue.

4.5 Contract negotiations were conducted in accordance with the directive, with face-to-face negotiations taking place between 24 June 2019 and 5 July 2019. The negotiation outcomes against the Commonwealth position and the minimal fall-back position were set out in detail in a matrix included as an annex to a Contract Negotiation Report.

4.6 The ANAO reviewed the contract negotiation outcomes achieved for the 63 negotiation issues identified. Of the 63 issues set out in the directive and the Negotiation Issues Matrix, the negotiation team achieved Defence’s preferred position or better on 51 issues, and the minimum fall-back position for the remaining 12 issues. For eight of the 63 issues, the outcome achieved was clarification and agreement (for example, clarification of which partner had responsibility for data cleansing of legacy information technology applications). These matters did not require changes to be made to the key documents.

4.7 The negotiation team provided the Contract Negotiation Report to Defence’s CIO on 5 July 2019. The CIO was the Public Governance, Performance and Accountability Act 2013 (PGPA Act) section 23 delegate for entering into the arrangement. The report recommended a Deed of Standing Offer and Official Order between the Commonwealth and IBM. Contract negotiation outcomes were reflected in the Deed of Standing Offer and the Official Order. The report was endorsed by the CIO on 8 July 2019. The report stated that the negotiation team had updated the delegate on negotiations through regular briefings, and that the CIO had attended the Head Table sessions with the IBM leadership team.¹⁰⁴ Defence and IBM entered into the Deed of Standing Offer and the Official Order on 19 July 2019, 18 days after the intended date of 1 July 2019.

Has Defence developed contractual arrangements for systems integrator services with clear milestones and performance expectations?

4.8 As of 3 May 2021, the contractual arrangements entered into by Defence for systems integrator services comprised an overarching Deed of Standing Offer agreement and six Official Orders for each individual work package.

4.9 The six Official Orders, which were executed between July 2019 and March 2021, had an original combined value of $257,677,968.13. Their combined value as at 3 May 2021 was $274,511,082.94, reflecting a $16,833,114.81 (6.5 per cent) increase resulting from contract variations (see Table 4.1 below).

Table 4.1: Official Orders under the Deed of Standing Offer as at 3 May 2021

Note a: The difference between the original contract value and the value as at 26 March 2021 is the result of contract variations discussed later in this chapter.

Note b: Official Order 3 expires on 15 July 2022, the last milestone being due 7 July 2022.

Source: ANAO analysis of Official Orders and variations.

Contract milestones and deliverables

4.10 Of the six Official Orders established as of March 2021 and reviewed by the ANAO, each set out key milestones that IBM must meet, including for initial operating capability (IOC) and final operating capability (FOC), with clearly defined acceptance criteria. The orders also list deliverables and work products.¹⁰⁵ The due dates of deliverables and milestones are set according to the Integrated Master Schedule¹⁰⁶, which Defence has advised is submitted by IBM quarterly and reviewed by Defence.

4.11 The payment is fixed price, with payments linked to the completion of milestones. Payment of the fixed price component of each Official Order is contingent on acceptance of the relevant milestone set out in the relevant Official Order, with a payment schedule set out in each Official Order.

4.12 Across the six Official Orders there are 100 milestones, 52 of which were due before the end of April 2021. Defence data indicates that 51 of the milestones due have been approved. Of these, 40 (78 per cent) were approved late by Defence. On average they were 28 days late. Defence advised the ANAO that the date recorded as the milestone acceptance date is the date that a final acceptance certificate is provided to the systems integrator (IBM), rather than the date all deliverables in the milestone have been completed and approved by Defence. This means that some timeliness issues are due to the time taken to issue the formal acceptance certificate.

Contract performance expectations

4.13 The Deed of Standing Offer and the Official Orders include clear performance expectations, detailing the systems integrator scope and the Defence scope for each service under the contract. For example, the Official Orders set out required business outcomes to be achieved, applicable plans¹⁰⁷ and standards¹⁰⁸, a summary of services required and a statement of work outlining the scope of IBM’s work under the Official Order.

Do contract and program governance arrangements enable monitoring, management and reporting on the achievement of Tranche 1 outcomes?

Contract management meetings

4.14 Defence has established a commercial team within the ERP program to manage the systems integrator contract.¹⁰⁹ The commercial team holds fortnightly contract management meetings with IBM. These follow a consistent agenda, covering: deliverable status, milestone status and invoicing, issues impacting the Deed of Standing Offer or Official Orders, variations, contract notifications, government furnished equipment and dependency status, and obligations status.

Contract deliverable tracking and acceptance

4.15 The Program Management Office¹¹⁰ maintains a tracker for all ERP contract deliverables, including for the systems integrator contract.

4.16 As of 17 June 2021, there were 367 contract deliverables for the program with 308 deliverables relating to the contract with IBM for Tranches 1 and 2. The staff responsible for updating the tracker meet fortnightly with the commercial team to ensure the information in the tracker is up to date.

4.17 According to the data recorded in the tracker, IBM had 237 deliverables due to be provided to Defence by the end of April 2021. Of the 237 deliverables, 227 had been submitted, as follows:

• 134 deliverables (59 per cent) were submitted on time, on average three days early; and
• 93 deliverables (41 per cent) were not submitted on time, on average 16 days late.

4.18 The ANAO’s review of the deliverables tracker indicated that data entry errors¹¹¹ meant that more deliverables appeared overdue than was the case. There is a risk that data entry errors have reduced the accuracy of the tracker and related reporting. Defence advised the ANAO that it is reviewing the tracker to improve data quality.

4.19 The deliverables tracker also documents the approval of deliverables.¹¹² According to data in the tracker, of the 227 deliverables due by the end of April 2021 and submitted, 209 had been approved as of 17 June 2021. Of the 209:

• 83 deliverables (40 per cent) had been approved on time, on average four days early; and
• 126 deliverables (60 per cent) had been approved late, on average 12 days late.

4.20 Of the remaining 18 deliverables, thirteen had conditional approval, and five had yet to receive conditional approval or approval.

Reporting on achievement of deliverables

4.21 Achievements against IBM deliverables and milestones are reported monthly across the senior governance committees, through the ERP monthly report discussed in Table 2.4.

4.22 From October 2019 to April 2020, the report set out expected deliverable due dates, and provided a traffic light indicator to advise whether the deliverable was on track, with completed deliverables marked ‘complete’. Completion dates were not provided in the report.

4.23 From May 2020 to April 2021, the report provided deliverable approval dates, but not deliverable due dates or submission dates. To assist the users of this report (including users from senior governance committees) to monitor the timeliness of deliverables, the ERP program should consider including the due dates and submission dates for comparison.

Program change request process

4.24 A Program Change Request (PCR) is to be generated for any proposed program change, and a Change Control Board has been established as the governing body to progress and approve all PCRs (see Box 2 below).

4.25 As at 3 May 2021, the ERP program’s register of PCRs included 99 items, of which 55 were marked closed (that is, approved by the Change Control Board and the required action taken).¹¹³

• Of the 55 closed items, 41 involved a systems integrator contract change, resulting in 14 contract variations to Official Order 1, five variations to Official Order 2, one variation to Official Order 4, and one variation to Official Order 5.¹¹⁴
• The 14 items which did not require a contract change related to changes to items delivered by Defence or minor changes to systems integrator deliverables which could be absorbed without any change to cost or schedule.¹¹⁵

4.26 Ten of the 21 variations involved a change in value. The original Official Order 1 value was $95,413,351.37. As at 3 May 2021 the value was $111,662,084.50, a total increase of $16,248,733.13 (17 per cent). The largest single variation, in November 2020, was for $14,594,193.83, an increase of 15 per cent of the previous value. This variation related to extension of the design phase of the program, and additional scope items.

4.27 The original value of Official Order 2 was $11,723,439.64. As at 3 May 2021 the value was $12,307,821.32, an increase of $584,381.68 (five per cent) of the previous value.

Decision-making processes

4.28 The ANAO’s review of Defence processes for managing Program Change Requests (PCRs) indicates that the Change Control Board was substantively responsible for deciding whether to proceed with PCRs, and for their final resolution.¹¹⁶ The ANAO examined minutes from 31 Board meetings from February 2020 to May 2021. Decisions relating to PCRs were documented on 26 occasions, including that PCRs were to be:

• approved and proceed to a contract variation proposal;
• approved and relevant program documentation be updated;
• approved and endorsed for commercial assessment;
• endorsed for an impact assessment;
• cancelled/rejected; or
• put on hold.

4.29 The ETB, as the senior committee responsible for the program, has visibility of recent PCR decisions and pending PCRs through the monthly program report. Defence advised the ANAO on 16 April 2021 that the Program Director determines whether a PCR is to be escalated to the Defence Information Systems Committee (DCISC) or the ETB. No instances have been identified of escalation of PCRs to senior committees, other than the PCRs associated with the Finance Central Payments and Banking Change Key Design Decision. As discussed below at paragraph 4.34, this Key Design Decision was approved by ETB. Defence records indicate that after a PCR relating to a contract variation is approved at the Change Control Board¹¹⁷, a designated SES Band 1/ADF 1 Star financial and contract delegate (the Deputy Program Director) or the ADF 2 Star Head Business Transformation, is then responsible for deciding on the associated PGPA Act section 23 commitment approval. Contract variations have then been signed by Defence delegates.¹¹⁸

4.30 The ANAO’s review identified the risk of a positional authority issue resulting from Defence’s delegation arrangements, as the majority of delegations were exercised by staff subordinate to the SES Band 2 equivalent Program Director and the ADF 2 Star Head Business Transformation.¹¹⁹ While not inconsistent with Defence policy, a similar risk was identified in Auditor-General Report No.33 2015–16 Defence’s Management of Credit and other Transaction Cards, in which the ANAO noted that for review to work effectively:

… the reviewer must be in a position to exercise independent judgement … this means that they cannot be in a position which would constrain unreasonably their capacity to question transactions that appear inappropriate; for example, this may be difficult for a person junior to the cardholder … (paragraph 2.42).¹²⁰

4.31 The PCR process set out in the ERP program handbook during audit fieldwork is illustrated in Figure 4.1 below. This was the process followed in all but one case reviewed by the ANAO. This process was inconsistent with the approved 2019 program management plan, which set out that the Internal Program Board is responsible for approving PCRs. The program management plan was updated in March 2021, in the course of this audit, to align with the process set out in the handbook, setting out that the Change Control Board was responsible for approving PCRs.

4.32 At the time of audit field work there was no charter or terms of reference for the Change Control Board. As discussed in Box 2 above, the board’s functions were described briefly in the March 2021 version of the Program Management Plan, and terms of reference have been approved on 7 July 2021.

4.33 The ANAO also identified inconsistent advice in the program management plan regarding the approval process for Key Design Decisions, in particular, the instances where these should come to the ETB. The ERP program management plan in place during audit fieldwork, as well as the version updated in March 2021, indicate that Key Design Decisions are approved internally by the program, while the ETB charter indicates that the ETB approve Key Design Decisions unless there is a timeliness issue. During the course of the audit, Defence reviewed the Key Design Decision process to improve clarity on DCISC and ETB decision-making responsibilities. The new process, which sets out the role for DCISC and ETB, was approved by the ETB in February 2021. In May 2021 the program handbook was updated to include the approved process, stating that there is a requirement to obtain approval for Key Design Decisions above the Program Executive level. The program management plan could also be reviewed to reflect that Key Design Decisions are to be made above the program level.

4.34 Key Design Decisions and Program Change Requests (PCRs) can affect program scope, cost and schedule and can result in contract variations. While the program management plan sets out that the ETB is the decision-making authority for ERP scope, schedule and budget, in practice, Key Design Decisions and PCRs are largely approved at the program level. An exception includes a Key Design Decision, approved by the ETB in August 2020. This resulted in a contract variation of $14,594,193.83 (see paragraphs 2.23 and 4.26).

4.35 A strategic RACI (a document that sets out which stakeholders are responsible, accountable, consulted and informed) was approved by the ETB in August 2019. It sets out decision-thresholds for the program in areas of scope, schedule and funding, and when decisions should go to the Defence Investment Committee, Enterprise Business Committee, ETB, or DCISC. It also sets out when they can be approved at the program level. However, this information is not reflected in current program guidance materials, including the program management plan and handbook. Decision-making responsibilities should be clearly and consistently set out in program guidance documents, to support compliance with requirements.

Managing conflicts of interest in the program change request process

4.36 The ERP program involves a large number of contractors working across all parts of the program and at all levels of program decision-making.¹²¹ Defence has interspersed its officials across the ERP program to discharge its departmental responsibilities.¹²² For example, the Change Control Board has three departmental officials working alongside 11 contractors, including four IBM personnel, on program changes, including changes to the IBM contract. In four of the 26 Change Control Board meeting minutes examined by the ANAO, where decisions were made, there were no departmental staff present. The average level of Commonwealth representation at Change Control Board meetings was one departmental official present, with an average total attendance number of nine attendees across the examined meeting minutes. A low level of Commonwealth representation across a contractor-led program can create oversight risk for the responsible entity, which remains accountable for the proper use and management of public resources under the PGPA Act and for decision-making in the Commonwealth interest.

4.37 Further, Defence’s program change arrangements give rise to both real and perceived conflicts of interest, as contractors are involved in the department’s substantive decision-making processes relating to their contracts, including contract variations with financial consequences for the Commonwealth. As discussed in Box 2 above and at paragraphs 4.28–4.31, Defence has documented, and the board minutes indicate, that the Change Control Board is a decision-making and approval body for Program Change Requests (PCRs). The board had IBM representation at each of the 31 meetings reviewed by the ANAO. The meeting minutes document that PCRs relating to price-impacting variations to IBM’s contract were approved to progress to the next stage (commercial assessment or contract variation proposal) at six of the 31 meetings, resulting in four contract variations with a total value of $483,902.77. The relevant meeting minutes do not record any conflict of interest declarations being made by any participants. Further, the minutes do not document any participants recusing themselves from the Board’s deliberations and decision-making.

Contract governance arrangements

4.40 As discussed in paragraphs 2.50–2.51 and Box 1, Defence uses an ‘above the line/below the line’ model in the ERP program, which involves ‘above the line’ contractors in program administration roles alongside ADF and APS personnel. Defence’s contract administration process for the ERP program involves ‘above the line’ contractors submitting timesheets, which are approved by another program team member. The timesheets are then used by the ERP commercial team to validate the contractors’ invoices prior to payment.

4.41 In late 2019, the program identified a contractor whose personnel were invoicing the program for more days than those documented in their timesheets. The program engaged a strategic advisor to investigate the issue (Ngamuru Advisory Pty Ltd). The resultant May 2020 report included the findings of an internal financial investigation that the Commonwealth had been invoiced $51,769.39 above what was demonstrated in the relevant timesheets. The strategic advisor’s report found that the contractor believed there was an ‘informal working arrangement’ in place with the program. The report recommended various actions, including potential price adjustment. The contractor agreed to Defence’s proposed actions to remediate the issue.

4.42 Following identification of this issue, in June 2020 the ERP Deputy Program Director was provided with information on the program’s contract administration process. This included a table setting out which personnel were responsible for approving each contractor’s timesheets. This table demonstrated issues of positional authority¹²³, with three subordinate APS/ADF staff responsible for approving their manager’s timesheets, and two instances where the contractor and approver were in the same team and at the same level (see Table 4.2 below). Further issues related to whether there was adequate visibility of work performed and an ability to confirm timesheets, with 13 instances of the contractor and approver being in different teams.

Table 4.2: Contractor timesheet approvers as at June 2020

Source: ANAO analysis of Defence records.

4.43 The information provided in June 2020 to the ERP Deputy Program Director also indicated that some ‘above the line’ contractors were responsible for approving the timesheets of other ‘above the line’ contractors. The Deputy Program Director requested advice on ‘having contractors sign for contractors’. The APS commercial lead responded that ‘from a governance point of view, contractors are not making a commitment or spending Defence money when they sign a timesheet’. In July 2021 Defence advised the ANAO that ‘Defence acknowledges the advice provided by this official is incorrect. Defence does not support this advice and recognises the requirement that actions that lead to expenditure of Public Money can only be exercised by authorised Officials under the PGPA Act’. Defence further advised that it is ‘reviewing governance arrangements that may have led to a misunderstanding of this principle.’

Is Tranche 1 contributing to ERP program objectives and Defence strategic priorities?

Progress in delivering Tranche 1

4.46 An indicative schedule for the delivery of Tranche 1 was set out in the second pass business case provided to government. Defence subsequently refined Tranche 1 with the systems integrator — in accordance with the SAP prepare, explore, release and deploy methodology — and split Tranche 1 into Release 1A and Release 1B after second pass (see paragraph 2.7). To review progress in delivering Tranche 1, the ANAO examined 18 ERP monthly reports, 21 Enterprise Transformation Board (ETB) minutes, six quarterly reports to the Minister and three six monthly reports to the government for the period March 2019 to April 2021.

Release 1A

4.47 As set out in the initial Integrated Master Schedule for the program developed by the systems integrator, and approved by the ERP program director on 5 November 2019, Release 1A IOC was planned for 20 July 2020, and FOC was planned for 20 October 2020. The ETB was advised in the July 2020 ERP monthly report that Release 1A IOC was delivered on time on 20 July 2020. The ETB was advised in the November 2020 Monthly report that the Release 1A FOC was proposed for delay to 14 December 2020. The change in FOC was managed through a Program Change Request and approved by the program’s Change Control Board. The January 2021 monthly report set out that the Release 1A FOC milestone had been approved on 17 December 2020. Although FOC had been delayed, in Defence’s December 2020 six monthly update, the government was advised that Release 1A was delivered on schedule.

Release 1B

4.48 Delivery of Release 1B was underway as at May 2021.

4.49 In May 2020 Defence advised the government that IOC for Release 1B was planned for mid-2022 and FOC was planned for mid-2023. In December 2020, Defence returned to government and requested a further delay to IOC for Release 1B to Quarter 4 2022. The government considered and approved this request in May 2021. Release 1B FOC remained at mid-2023. As at March 2021, the program reported to Defence’s Audit and Risk Committee that it was nearing the end of the ‘Explore’ (design) phase, which was expected to be complete by June 2021.

Program Objectives

4.50 The program management plan (developed in 2019 and updated in 2021) includes 13 program objectives, and 12 program outcomes, set out in Table 4.3 below. Defence intends to track progress against these through the ERP benefits management system.

Table 4.3: Program objectives, outcomes and benefit areas, as set out in the March 2021 Program Management Plan

Note a: There is a fourteenth program objective set out in the second pass business case: ‘reduced training burden and enhanced ease of use.’ Defence advised this was omitted from the program management plan in error.

Source: Defence records.

Benefits management strategy

4.51 Defence finalised a benefits management strategy on 8 December 2017 to support the second pass business case. A new version of the strategy was approved by the ETB on 6 August 2020. The strategy’s purpose is to ensure that the program’s:

Benefits Management Methodology, including associated processes; roles and responsibilities; governance and approval mechanisms; tools and artefacts are clearly articulated and understood.

4.52 The strategy sets out the relationship between the program objectives, program outcomes and benefit areas.¹²⁴ The strategy defines benefits as:

… the measureable improvement from outputs and outcomes expected of a Program or Project.

4.53 A benefits management team has been established within the ERP program. Its role is to: provide benefits managements advice; develop the benefits management strategy; identify, plan and analyse benefits; support Defence Groups’ and Services’ measurement and realisation activities¹²⁵; report on measurement and realisation data; and hand over benefits management documentation and activities once the ERP end-to-end solution has been delivered. After benefits realisation plans are developed¹²⁶, six monthly Group and Service reporting to the benefits management team is planned. This is to be collated by the benefits management team into an ERP program benefits dashboard.¹²⁷

4.54 The program returned to the ETB in March 2021 with recommendations on benefit ownership and rebalancing. The ETB did not agree to the recommendations and requested that the program ensure alignment with the Defence Transformation Strategy, whole-of-government transformation and Government Enterprise Resource Planning. On 14 May 2021 Defence advised the ANAO that it was working with GovERP¹²⁸ and the Defence Transformation Office on this.

4.55 The benefits management team has developed a draft benefits map that traces program outcomes to benefit owners, end benefits, benefit areas and intermediate benefits. As at 21 July 2021, intermediate benefits were yet to be defined, and were undergoing review with the program and Defence Groups and Services. Defence advised the ANAO that it continues to ‘engage with Groups and Services to shape and define ERP intermediate benefits as they correlate to the eight end benefits referenced in the second pass business case.’ The strategy states that ERP benefits management is an iterative process, and that the approach will evolve over time. The strategy also states that benefits will not be realised until the change activities occur. As Tranche 1 Release 1B IOC is not expected until Quarter 4 2022, the program is not yet at the stage of benefits realisation.

4.56 The benefits realisation framework is intended to support Defence in measuring, monitoring and reporting on the extent to which outcomes have been achieved. While acknowledging that there is ongoing planning work occurring in the program, early establishment of a benefits realisation plan for each Tranche and for the program overall will better position Defence to collect the data necessary to demonstrate that outcomes have been delivered.

Strategic Priorities

Australian Industry Capability

4.59 At second pass, Defence undertook to achieve an estimated level of Australian industry involvement (also known as Australian industry capability (AIC)) of 80 per cent of program expenditure.

4.60 For the systems integration component of the program, the Deed of Standing Offer with IBM includes an AIC strategy. The strategy sets targets of a minimum 90 per cent local industry involvement and a minimum of 30 per cent small to medium enterprise involvement. IBM’s quarterly AIC progress report, covering December 2020 to March 2021, indicated that IBM was exceeding its AIC target, with an average of 99.93 per cent local industry involvement and 41.87 per cent small to medium enterprise involvement achieved to date.

4.61 Defence advised the ANAO on 21 July 2021 that:

To date Defence has achieved 75% Australian Industry Content (AIC) across all program expenditure. This is a cumulative total program expenditure to date and should not be taken as a projection of the total anticipated AIC component for the program on completion.

First Principles Review

4.62 As discussed in paragraph 1.2, one of the six key recommendations of the 2015 First Principles Review: Creating One Defence was to fully implement an enterprise approach to the delivery of corporate and military enabling services to maximise their efficiency and effectiveness. The 2016 Defence White Paper also identified issues related to Defence information and communications technology (ICT) systems resulting from underinvestment in ICT and the lack of an enterprise-level strategy for Defence’s ICT requirements. The 2016 Integrated Investment Program, released with the Defence White Paper, committed to enhancements to Defence’s ICT and business processes to support the implementation of First Principles Review recommendations, including by standardising business processes to provide end-to-end visibility of Defence business through streamlined processes and a consolidated Defence enterprise resource planning system intended to improve core business functions. The 2020 Force Structure Plan continued to emphasise the importance of business transformation in Defence, setting out that:

Driving improved business processes, enterprise information management, enterprise resource management and broader transformational reforms across Defence will be critical to future Defence effectiveness. The Government is committed to improving enterprise business processes by implementing reform across Defence to maximise business commonality and effectiveness.¹²⁹

4.63 At first and second pass, the business case prepared by Defence to inform the Minister for Defence’s submission to government set out that the ERP program would assist in addressing deficiencies identified in the First Principles Review, by standardising business processes.

4.64 As of May 2021, some six years after the First Principles Review identified the need to implement an enterprise approach to the delivery of enabling services, the ERP program remained at an early stage of implementation. Tranche 1A was delivered in December 2020 but Tranche 1B is the material activity of Tranche 1 and is yet to reach IOC. Tranche 2 has been approved and Tranche 3 is yet to be presented to government. Substantial work remains for Defence to fully implement the ERP program.

Appendix 1 Department of Defence response

 

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021–22 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

5. In respect to a range of issues identified by the ANAO in the course of the audit — particularly the issues reported in paragraphs 3.67–3.87 and 4.40–4.43 — Defence advised in its response to the audit report that:

Defence has taken actions to address the areas for improvement identified by the audit, including the implementation of a Contractor Integrity Framework [ANAO comment: see paragraph 3.74] and review of contractor on-boarding processes. Defence is conducting reviews of ERP Program operation and governance, with a particular focus on improving transparency in the selection of delegates [ANAO comment: see paragraph 4.30] and appropriate approval of timesheets [ANAO comment: see paragraphs 4.40–4.43]. In addition, mandatory training and periodic probity briefings have been introduced to ensure contractor awareness of their obligations.

…

Defence accepts the areas for improvement noted in the report. The matters identified have provoked immediate action in the interests of maintaining efficient, effective and secure delivery of the ERP Program. Where failings by individuals have been identified and verified [ANAO comment: see paragraph 3.86], steps have been taken, commensurate to the seriousness of the findings, to ensure no repeat of this behaviour. Defence has confidence that, irrespective of the individual actions, Defence’s systems are secure and value for money has been achieved through ERP Program delivery to date.

Defence agrees to implement all recommendations proposed in the report. These actions will ensure any shortcomings at the system level have been identified and appropriately addressed as a priority. Defence has initiated complete reviews of ERP Program operations and taken steps to vary delegations, increase oversight and reconsider the appropriate balance of contractors with APS/ADF personnel in key management positions.

Appendix 3 Approach to the use of the SAP application for the ERP program

1. At first pass on 22 May 2017, the Australian Government agreed to Defence’s preferred option for the program, which was to standardise and simplify end-to-end business processes, enabled by a new build SAP technology solution. Box 3 below provides further information on Defence’s approach to the use of the SAP application and program methodology in relation to the ERP program.

Appendix 4 ERP program schedule as at May 2021