Introduction

1.1 The Department of Defence (Defence) is required under its legislative and regulatory framework to provide comprehensive health care, including dental and other ancillary health care¹⁰, to members of the Australian Defence Force (ADF).¹¹ The Defence Regulation 2016, section 49, establishes that:

(1) The Commonwealth must arrange provision to a member of the Defence Force rendering continuous full time service of medical and dental treatment necessary to keep the member fit for the performance of the member’s duties.

(2) The provision of treatment under subsection (1) is not required to comply with a law of a State or Territory if it complies with a Defence Instruction.

(3) The Commonwealth must arrange for the supply of pharmaceuticals required for the provision of treatment under subsection (1), including arranging associated activities such as transport, storage and possession of the pharmaceuticals.

(4) The supply of pharmaceuticals (and associated activities) under subsection (3) is not required to comply with a law of a State or Territory if it complies with a Defence Instruction.

1.2 The Defence Health Manual is a key component of the policy framework¹² that applies to the delivery of health services to the ADF and sets out that:

Defence provides health care that is clinically necessary to keep Defence members medically and dentally fit to perform their military duties. Defence also provides health preparation for deployment; force optimisation; health support to operations; diagnosis and treatment of injury, illness or disability; and rehabilitation services to help Defence members return to duties or transition to civilian life.

Within the garrison setting, Defence members have access to the range of essential health services and entitlements equitable with those provided to the Australian community including general practice, dental services, mental health and occupational psychology services, hospital care, allied health care and specialist services.

Arrangements for the provision of health services

1.3 Health services are provided to approximately 60,000 permanent ADF members and approximately 25,000 reservists. Health services are provided day-to-day across 51 health centres and clinics located on Defence bases throughout Australia.¹³ Where an ADF member needs health care that cannot be provided on-base, ADF members are able to access a network of health care facilities and providers off-base. Access to off-base health care facilities and staffing of on-base facilities is acquired by Defence through the Australian Defence Force Health Services Contract (ADF health services contract).

1.4 On 28 June 2012, Defence awarded the ADF health services contract to Medibank Health Solutions Pty Ltd (Medibank). This contract was for an initial term of four years and was valued at $1.7 billion.¹⁴ Medibank was contracted to provide the following services.

• On-Base services — labour hire of health professionals and support staff across 67¹⁵ health centres and clinics around Australia.
• Off-Base services — including rehabilitation, optometry, other specialists, hospitals, and allied health services through the civilian community.
• Pathology services — a national network of pathology services.
• Imaging and radiology services — a national network of imaging and radiology services.
• The Health Hotline — a telephone based 24/7 nurse triage and referral service.

1.5 In February 2016, Defence extended its contract with Medibank to 31 October 2018. The two-year extension increased the total contract value to $2.7 billion.

1.6 In March 2017, Defence advised the Minister for Defence of its intention to conduct a procurement process and enter into a new contract for the delivery of health services by 1 November 2018. The contract with Medibank was due to expire on 31 October 2018 and there were no further options available to extend the contract.

1.7 Initial industry engagement was conducted between April and June 2017.¹⁶ In July 2017, Defence advised its Investment Committee that it had underestimated the time required to complete the procurement and that initial industry engagement had identified significant risks associated with the proposed timeframe to transition to new arrangements. In August 2017, Defence sought and obtained approval from the Minister for Defence and the Minister for Finance to issue a limited tender to Medibank to continue to provide health services to the ADF. Defence advised the Minister that:

the proposed extension satisfies two requirements under paragraph 10.3 (10.3b and 10.3d.iii) of the Commonwealth Procurement Rules in that unforeseen circumstances mean that health services to the ADF cannot be obtained in the allocated timeframe under open tender or prequalified tender, and the incumbent provider is the only reasonable option (in the timeframe) for delivery of the complex, bespoke clinical service delivery arrangement required.

1.8 Subsequently, a Deed of Variation was executed to extend the ADF health services contract with Medibank for a period of eight months (1 November 2018 to 30 June 2019) at an additional cost of $285 million.¹⁷

1.9 In September 2017, Defence issued an Invitation to Register Interest (ITR) to market. In January 2018, Defence issued a Request for Proposal (RFP) to three tenderers that had been shortlisted through the ITR process.¹⁸ In November 2018, Defence selected Bupa as the preferred tenderer. Negotiations were conducted between 22 November and 4 December 2018 and on 12 December 2018, Defence sought approval from the Minister for Defence and the Minister for Finance to award the ADF health services contract to Bupa. Defence advised the Ministers that:

The current ADFHS Contract with MHS has delivered benefits, including increased integration of health services and efficiencies in healthcare delivery. The next generation ADFHS Contract provides opportunities to further these benefits, including:

a) enhanced health service delivery with a robust continuous improvement and innovation process

b) where appropriate, a shift in focus, from health outputs to health outcomes, supported by a performance based contract and performance management framework

c) enhanced clinical leadership, with a focus on health literacy and preventative health

d) improved business intelligence through automation, data collection and analysis

e) improved commercial arrangements through specific additional contract mechanisms that promote cost containment and provide incentives for health services transformation.

1.10 On 14 January 2019, contract arrangements with the new service provider, Bupa Health Services Pty Ltd (Bupa or the contractor), were announced by the Minister for Defence. A revised ADF health services contract was signed on that day. The revised contract was to take effect from 1 July 2019, after a six-month transition-in period. The announcement stated that the arrangements would ‘support the delivery of a range of primary and specialist health services at both on-base health facilities and through a comprehensive network of off-base service providers’ for over 80,000 ADF members and reservists, and that under the new contract ‘ADF members will continue to receive the full scope of health services they currently receive.’¹⁹

1.11 When entering into the new arrangements, Defence advised ADF members and families that:

Whilst the range of health services procured under this Contract has not substantially changed, Joint Health Command has sought to continue to improve its delivery of health services to Defence personnel. Under the ADF Health Services Contract, Defence will see a greater use of data and analytics in health service delivery, the identification and minimisation of low value care, and an increased focus on continuous improvement and innovation.²⁰

1.12 Defence’s contract with Bupa is for an initial period of six years (2019–20 to 2024–25), and a maximum of 10 years. The contract objectives are set out in Box 1 below.

1.13 Under the contract, Bupa is to provide the full suite of health services necessary for ADF members to maintain an acceptable level of health, fitness and wellbeing. The services include:

• a health professional workforce²¹;
• access to medical advice, triage and referrals (including providing mental health risk assessments) 24 hours a day, seven days a week;
• access to a broad range of specialised services;
• an appointment and booking system and/or service;
• imaging and radiology services;
• pathology services; and
• occupational rehabilitation services.

1.14 In July 2019, the workforce engaged under the contract to provide health care in the garrison environment (on-base) was comprised of approximately 997 contracted health professionals. In June 2022, approximately 1,283 health professional and support staff were providing services on-base. The composition of the workforce in July 2019 and June 2022 is illustrated in Figure 1.1 below.

1.15 The quantity and type of health services delivered under the contract through the off-base network, between July 2019 and June 2022, are illustrated in Figure 1.2 below.

Figure 1.2: Health services delivered to ADF members through the off-base network — July 2019 to June 2022

Source: ANAO analysis.

Contract budget and expenditure

1.16 In December 2018, Defence advised the Ministers for Defence and Finance that the ADF health services contract was estimated to cost $3.4 billion (GST inclusive) over its initial six-year period and $6 billion (GST inclusive) over its maximum period of 10 years. Defence stated that:

The estimated total costs over the six-year term and potential extensions to 10 years within the extant ADFHS budget forecast are as follows:

a. Increase to the 2019-20 budget of 6.9 per cent (to $489.419 million)

b. Ongoing annual growth of approximately 4.6 per cent to adjust for contract indexation and anticipated volume increases (note: the projected contract price is forecasted to grow by approximately 3.75 per cent, which provides a margin for volume increase).

1.17 Defence also advised the Ministers that the cost estimates provided were indicative, as the ADF health services contract is demand driven, and that one driver of demand was Defence’s operational tempo. Defence stated that:

In the military context, there is a risk of significant changes in volume outside of predictions, such as increased operational tempo, which impacts demand.

1.18 In March 2022 Defence advised its internal Enterprise Business Committee, when seeking additional funds of $230.2 million for the contract, that a surge in demand for health support had been experienced due to the following.

• The 2019–20 black summer bushfires, which saw an additional 3,500 hours of un-rostered health professional hours delivered in support of Operation Bushfire Assist.
• The COVID-19 pandemic, which saw the release of contracted health professionals to assist vaccination delivery teams in aged care facilities and support Defence quarantine and testing activities.

1.19 As at October 2022, actual expenditure under the ADF health services contract totalled $1.8 billion (GST exclusive). The budget for the initial six-year period of the contract and actual expenditure under the contract as at 31 October 2022, is detailed in Table 1.1 below.

1.20 As shown in Table 1.1, actual expenditure under the contract exceeded the annual budget in 2019–20, 2020–21 and 2021–22, until additional funding of $230.2 million (GST exclusive) for 2021–22 and the forward estimates period (2022–23 to 2024–25) was approved by Defence’s Enterprise Business Committee in March 2022. As at October 2022, total expenditure incurred over the life of the contract was $44.6 million higher than the revised budget (that is, the budget inclusive of the additional funds approved in March 2022).

Administrative arrangements

1.21 Within Defence, responsibility for health services is shared between the Armed Services (Army, Navy, Air Force), Joint Operations Command (JOC) and Joint Health Command (JHC).²²

1.22 Located within Defence’s Joint Capabilities Group, JHC is responsible for the delivery of health services to enable ADF preparedness.²³ The Commander Joint Health is responsible for providing health and health related services in the garrison (on-base) environment and managing access to a contracted network of off-base health facilities and providers.

1.23 JHC is also responsible for administering the ADF health services contract. The contract identifies: the Commander Joint Health (CJHLTH) as the Commonwealth representative for contract management purposes; the Director-General Health Business and Plans (DGHBP) as the senior representative; and the Director Health Contracts²⁴ as the management representative.²⁵ The personnel occupying these positions are responsible for the management and administration of the ADF health services contract, including contract compliance, administration, reporting, invoicing and governance.

Rationale for undertaking the audit

1.24 Defence has a legal obligation to provide medical and dental services to ADF personnel who are providing continuous full-time service, and a capability imperative to maintain the health status of ADF personnel. Defence’s ADF health services contract is a key element of its support arrangements for the ADF.

1.25 This performance audit is part of an ongoing program of work that has examined aspects of Defence’s contract management and administration.²⁶ The audit was undertaken approximately half-way through the initial six-year period of the ADF health services contract²⁷, to provide independent assurance to Parliament on the effectiveness of Defence’s contract management.

Audit approach

Audit objective, criteria and scope

1.26 The objective of the audit was to assess whether Defence is managing its ADF Health Services Contract to achieve efficient and effective delivery of the contracted services.

1.27 To form a conclusion against the objective, the following high-level criteria were used.

• Has Defence established fit-for-purpose contract governance arrangements?
• Has Defence established fit-for-purpose performance monitoring, evaluation and reporting arrangements?
• Have services been delivered effectively against contracted requirements?
• Have the expected cost and service delivery efficiencies under the contract been realised?

1.28 To assess Defence’s contract management, the ANAO reviewed both the design and implementation of the contracted arrangements.

1.29 The audit focused on Defence’s arrangements to deliver its responsibilities under Defence Regulation 2016, subsection 49(1), through its management of the contract with Bupa Health Services Pty Ltd (Bupa) to deliver health services to the ADF.²⁸

1.30 The audit also followed-up on the moderate financial audit finding raised by the ANAO in 2020–21, relating to the governance of ADF health services, including invoicing and assurance arrangements for the contract.²⁹

1.31 The audit did not examine:

• the effectiveness of the procurement approach used to award the contract and the related value for money assessment;
• the effectiveness of the service delivery model selected by Defence to deliver health services to ADF personnel;
• other activities conducted by Joint Health Command that contribute to the delivery of Defence’s legal obligation to provide medical and dental services to ADF personnel;
• related procurements, including planned facility upgrades, JP2060 Phase 3 – the provision of deployable health units, or JP2060 Phase 4 – the replacement of the Defence electronic Health System (DeHS) with a Health Knowledge Management System; or
• the delivery of clinical services to ADF members.

Audit methodology

1.32 The audit methodology involved:

• examining relevant records and documents, including the ADF health services contract;
• the analysis of invoices submitted by the service provider;
• discussions with relevant Defence personnel; and
• review of a representation letter from the service provider.

1.33 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $682,300.

1.34 The team members for this audit were Joyce Knight, Elizabeth Wedgewood, Georgia Johnston, James Wright, Michael Brown, Dale Todd, Nathan Daley, Qing Xue, Sally Ramsey and Amy Willmott.

2.1 This chapter examines whether contract and financial governance arrangements have been established that support Defence to effectively monitor and manage contract delivery and payments. Effective and fit-for-purpose contract and financial governance arrangements support compliance with the requirements of the Public Governance, Performance and Accountability Act 2013 (the PGPA Act) and Defence’s Accountable Authority Instructions for proper resource use, the achievement of value for money, transparency, and accountability. Effective contract and financial governance arrangements also support the controls framework, the establishment of effective contractor relationships, and the achievement of contracted outcomes.

2.2 To assess whether Defence has established fit-for-purpose contract governance arrangements, the ANAO reviewed both the design and implementation of the arrangements.

Has Defence established fit-for-purpose contract governance arrangements?

2.3 The contract governance arrangements for the ADF health services contract are established through:

• a ‘Contract Management and Governance Framework’ (the Framework), which is part of the contract;
• arrangements for managing contract changes;
• arrangements for managing risk; and
• arrangements for financial management and claims for payment.

Contract management and governance framework

2.4 The contract includes an attachment entitled ‘Contract Management and Governance Framework’ (the Framework). The Framework sets out the following.

• The preferred relationship model, including executive commitment and engagement requirements.
• Contract governance arrangements, including: Defence’s management structure, roles and responsibilities of key positions, and key points of contact; the Contractor’s governance model; and communication requirements.
• How issues are to be resolved.
• The meetings and forums that Defence and the contractor are to participate in, including record keeping requirements.
• Contractor performance review requirements.
• Reporting requirements.
• Subcontractor management.

Relationship model and executive engagement

2.5 The Framework sets out the preferred relationship model including relationship principles, executive commitment and relationship, and behavioural and cooperation obligations. The Framework states that ‘senior executive commitment from both the Commonwealth and the Contractor (Bupa) is crucial to the success of the relationship’. The agreed interactions and the extent to which these interactions have occurred between July 2019 (contract commencement) and December 2022 are set out at Table 2.1.

Table 2.1: Contracted responsibilities for senior executive interaction

Note a: The contractor’s representatives were identified as the Chief Executive Officer (CEO), the Managing Director Health Services, and the General Manager ADF Health Services.

Note b: Defence records show that Bupa made several attempts to set this meeting up at the six-month point as required. On 20 August 2021, Defence advised that its representatives were available to meet on 9 September 2021 ‘subject to the COVID/Afghanistan situations’.

Source: ADF health services contract and ANAO analysis of Defence documentation.

2.6 In summary, Defence has not been able to demonstrate that the executive engagement requirements have been met. While there is documentation indicating that senior executive engagement has occurred, the agreed level and frequency of interaction has not occurred as outlined in the Framework and the key points of discussion and outcomes from these meetings have not been consistently documented.

Record keeping

2.7 In February 2023, Defence advised the ANAO that the Commander Joint Health (CJHLTH) and the senior executive of the contractor meet monthly, with outcomes of these meetings ‘back briefed to the Chief of Joint Capabilities (CJC) and that DGHBP [Director General Health Business and Plans] meets with the contractor fortnightly’. Defence further advised that ‘informal’ records of these meetings are retained, and back briefs are provided to the Director ADF Health Services Contract.

2.8 An informal approach to record keeping is not consistent with Defence’s records management policy or with whole-of-government guidance for contract management.³⁰ Failure to maintain appropriate records exposes Defence to risk where key points of discussion and any actions or outcomes are not recorded, particularly if disputes arise between parties to a contract.³¹

Contract governance arrangements

2.11 Contract governance arrangements outlined in the Framework include Defence’s management structure³², the contractor’s governance model, and communication requirements.

2.12 In December 2020, Defence advised the contractor of changes to Defence’s management structure, including roles that had been disestablished and additional roles to be added to the list of key points of contact in the Framework. Those changes were not agreed and formalised through a contract change proposal (CCP) (see paragraph 2.21), and as at December 2022, the Defence management structure outlined in the contract Framework did not reflect actual arrangements within Defence.

2.13 Changes have also been made to the contractor’s governance model and the meetings and forums identified in the Framework (see paragraphs 2.16 to 2.19). Some aspects of those arrangements have been formally adjusted through the provision of ‘notices’, as allowed for under the contract. For example, in October 2022 Defence advised the contractor that it would be taking over the secretariat functions for: meetings between the contractor’s executive, the Chief of Joint Capabilities (CJC) and Commander Joint Health (CJHLTH); the Program Governance Board; Garrison Management Committee; and the Contract Delivery Committee.³³ As at December 2022, the contractor’s governance model outlined in the contract was no longer up to date and a CCP had not been progressed to update the contract. Defence advised the ANAO in February 2023 that the CCP had been cancelled and changes to the secretariat function would be incorporated as part of the program of work being undertaken to amend and update the Contract Management and Governance Framework (see paragraph 2.19).

2.14 To support effective relationship management the Framework requires the development of a communication plan. A plan was initially developed by the contractor and approved by Defence for the six month transition-in period. That plan stated that a separate plan for the operational period (post 1 July 2019) ‘will be developed by the contractor, in partnership with Defence during Transition-In’. In August 2022, Defence directed the contractor to develop and submit a communications plan for Defence approval. A final draft plan was submitted to Defence on 31 December 2022. Defence advised the ANAO in February 2023 that it was reviewing the plan to provide feedback to the contractor.

Resolving issues

2.15 The Framework establishes escalation processes for managing issues, and states that issues are to be raised and resolved at the lowest level of management possible. The ANAO’s review indicates that in practice, issues are routinely escalated to the jointly chaired committees. This approach means that issues resolution may reside with the co-chairs of the committees.³⁴ Further, delay in the development of the communication plan means that guidance on the escalation of issues has not been in place since July 2019, when the communication plan for the transition-in period lapsed.

Meetings and forums

2.16 The Framework establishes requirements regarding: participation of the parties in various fora³⁵; the preparation and circulation of meeting agendas; and the recording of meeting minutes.

2.17 The ANAO examined the agenda packs and meeting records for five committees between July 2019 and November 2022, to assess their operation against the Framework.³⁶ These included the Program Governance Board (PGB), Garrison Management Committee (GMC), Contract Delivery Committee (CDC), Complaints and Clinical Incidents Meetings (CCIM), and Continuous Improvement and Innovation (CII) committee. The terms of reference (ToR) for each of these committees were developed by and are maintained by the contractor. The ToR identify the committee’s: purpose; term; composition and membership, including invitees; meeting administration; decision making; responsibilities, including for recordkeeping (the contractor); review requirements; and standing agenda items.

2.18 Changes made to meetings and forums listed in the Framework have been made outside of the contract change proposal process. As a result, the ToR for the five committees no longer align with the contracted requirements outlined in the Framework.³⁷ The changes include: disestablishment of committees and absorption of their functions into other committees³⁸; and amendments to the purpose, membership and composition of committees identified in the contractor’s governance model and the meeting schedule.³⁹ These changes were made outside of the mechanisms established in the contract and Defence policy guidance.⁴⁰

2.19 In April 2022, Defence commenced work to revise the contract governance arrangements. Defence advised the ANAO in February 2023 that an internal review of contract governance arrangements was underway to address the misalignment between actual governance arrangements and those outlined in the Framework.⁴¹ In April 2023, Defence advised the ANAO that the revisions to the Framework have been drafted and a CCP process to implement the new arrangements is expected to be completed in May 2023.

Contract management and administration

2.20 While a Contract Management and Governance Framework is in place, Defence has not implemented a contract management plan to administer the ADF health services contract.⁴² As at December 2022, work to develop and implement a contract management plan and related documentation — including a responsible, accountable, consulted and informed (RACI) matrix, a probity management plan, and a risk management and assurance plan — had not been completed. In February 2023, Defence advised the ANAO that the contract, risk, probity and assurance management plans and the RACI matrix had been developed but had not yet been implemented.

Arrangements to manage contract changes

Contract change proposals

2.21 The ADF health services contract establishes that it shall only be changed by Contract Change Proposal (CCP) and that either party may propose a change to the contract. The contract includes the template that is to be used and establishes that ‘unless otherwise agreed in writing, the Commonwealth representative shall approve or reject CCPs, giving reasons for such rejection.’⁴³

2.22 As at December 2022, 100 CCPs had been lodged since the contract was signed on 14 January 2019. Of the 100 CCPs, 79 have been approved and enacted, two have been rejected, nine have been cancelled, and 10 were in progress.

2.23 The ANAO examined the 79 approved CCPs and found that 48 involved material changes to the contract.⁴⁴ Box 2 below summarises the ANAO’s analysis of approved CCPs.

Note a: The 26 changes to the pricing tables include: the introduction of new rates for psychologists; new codes for telehealth, COVID related health services and prostheses; application of price indexation; increases to the recurring services management fee; and the application of new or amended Medicare Benefits Schedule (MBS) rates.

2.24 A number of contract adjustments have occurred without formal agreement to a CCP. The Defence Contract Management Handbook clearly identified, and highlighted, that this approach is ‘What not to do’ in relation to contract changes⁴⁵ and stated that:

Even where a contract sets out a contract change process, it is possible to vary the contract by conduct, for example, in an email or by simply acting as if the change has been made. In some cases, the conduct of contract management personnel may give rise to an ‘estoppel’, such that Defence will not be able to argue that a contract change has not taken place as a result of the course of action adopted by the parties.⁴⁶

2.25 Defence’s Contract Management Handbook was superseded by the Contract Management Framework (CMF) in July 2022. The CMF also states that ‘care should be given not to inadvertently make a change to the contract through verbal agreement or conduct — verbal changes, however informal, may be legally binding.’

2.26 Defence should ensure that in its management of the contract, all future contract changes comply with the contracted requirement for a CCP and have regard to the guidance in the CMF.

Guidance, oversight and assurance regarding contract change proposals

2.27 The Contract Management and Governance Framework of the ADF health services contract does not address how contract changes are to be managed. Defence has working level instructions to provide guidance to contract management personnel responsible for reviewing and processing CCPs. Defence has also drafted but not implemented a contract management plan that includes a reference to those working level instructions. The ANAO compared these instructions against the requirements of Defence’s Contract Management Handbook and the results are set out at paragraph 2.30.

2.28 While CCP summaries have been provided to the Contract Delivery Committee, Defence has not implemented a fit for purpose assessment and authorising framework for proposed contract changes. A fit for purpose framework would provide assurance to the authorised delegate that risks have been identified and appropriately mitigated prior to approving a CCP for implementation. This is particularly important for CCPs with significant implications for the contract, including those involving financial risk.

2.29 An example of an agreed CCP with significant implications is provided in case study 1 below.

2.30 In October 2022, Defence advised the ANAO that it maintains an internal work instruction to provide guidance to contract management personnel. The instruction outlines the procedures to be followed when processing CCPs. The ANAO’s review of the work instruction indicates that, as currently framed, it does not meet the requirements of Defence’s Contract Management Handbook (or the Contract Management Framework) as it does not provide guidance on how to:

• determine if the proposed CCP is required;
• assess the effect of the proposed CCP on service delivery and/or the contracted price;
• assess the potential effect of the proposed CCP on other significant terms and conditions of the contract, for example governance, risk or performance management frameworks;
• assess whether the CCP will transfer or create additional risk; and
• determine when subject matter (contractual, financial, or legal) advice on the proposed change is required.

2.31 To improve its management of risk, Defence should develop and implement an assessment and authorisation framework, supported by appropriate governance and assurance mechanisms, to oversee the handling of contract change proposals.

Arrangements for managing risk

Planning and reporting

2.34 Under the ADF health services contract the contractor is responsible for risk and issue management. The contractor is to prepare and submit a Risk Management Plan (RMP), conduct the risk management program, and manage issues in accordance with the approved RMP. The contractor is also responsible for implementing risk, issue, and opportunity management programs.

2.35 The June 2021 RMP submitted by the contractor to Defence was approved in August 2021. Updates made to the plan in December 2021 were accepted by Defence in March 2022.⁴⁷ Further updates and amendments were requested and made to the RMP between June and August 2022, however as at December 2022, a revised RMP had not been formally approved by Defence.

2.36 To aid Defence’s awareness of risks to contract delivery, the contractor’s quarterly Contract Status Report (CSR) is to include a report on risks and problems.⁴⁸ The ANAO reviewed the quarterly CSRs received by Defence for the period July 2019 to September 2022 and found that while they included a risk and a problem report as required, the reports have not consistently met the minimum requirements set out in the contract. For example, the progress of risk mitigation activities was not included in ten (77 per cent) of the risk reports included in the CSRs. In contrast, all 13 problem reports met the minimum requirements.

2.37 The ANAO also examined the risk registers included in the 13 CSRs. The risk registers: included a risk identification number; title and description of the risk; identified the risk owner; and identified the control measure in place. The risk registers did not consistently record the inherent and residual risk ratings or identify the risk treatment to be applied (tolerate, treat, transfer, terminate) where the control had not resulted in a reduction to the risk rating.

Risk oversight

2.38 Governance and risk oversight is provided by the jointly chaired (Defence-contractor) Garrison Management Committee (GMC) and Contract Delivery Committee (CDC). The Program Governance Board (PGB), which is chaired by Defence, also has a role in overseeing risk.

2.39 The roles of these committees and a summary of the ANAO’s review of their activities regarding risk is set out in Table 2.2 below.

Table 2.2: Committees with risk management responsibilities

Note a: Issues are also escalated from the off-base operations meeting, on-base operations meeting, finance governance meeting and occupational rehabilitation meeting.

Note b: The Contract Delivery Committee can escalate issues/decisions to the Garrison Management Committee.

Note c: Agenda papers and outcomes from the Program Governance Board meeting in September 2021 were sighted which confirm that the meeting was held. However, meeting minutes or an attendance, decision and action log for the September 2021 meeting could not be located by Defence.

Source: ANAO analysis.

2.40 The ANAO’s review of the minutes of GMC and CDC meetings indicated the following.

• Since 2021, the CDC has discussed risks and issues, and emerging or material changes to the risk profile, however these discussions have not been consistently documented.
• There is no evidence that the CDC or GMC have sought to assure themselves that the controls (mitigation activities) identified in the risk registers have been implemented and their effectiveness tested.
• There is no evidence that the CDC has sought to assure itself that the problems reported by the contractor have been resolved.

2.41 To improve Defence’s risk management arrangements for the ADF health services contract, there would be benefit in Defence reviewing its arrangements — including the approach adopted by relevant committees — for seeking assurance that the controls and mitigation activities identified to manage risk, and reported in the risk register, have been implemented and are effective.

Arrangements for financial management

2.43 Financial issues are discussed between Defence and the contractor at jointly chaired committees. These committees are discussed in Table 2.3.

Table 2.3: Governance committees with financial oversight responsibilities

Note a: Financial issues are escalated to the CDC from the co-chaired Finance Assurance Meeting, which was initially established in May 2021 as the Finance Governance Meeting and rebadged in November 2021. The Finance Assurance Meeting is not identified in the Contract Management and Governance Framework of the ADF health services contract, however it is identified in the governance model that is in place. The Terms of Reference for this committee state that the purpose of the committee is to define problems, agree actions, delegate and provide oversight of financial issues and escalate financial issues to the CDC.

Note b: The Business Rule Program was established in February 2021. Its purpose is to embed the Medicare Benefits Schedule and other pricing rules into the systems used by Bupa and enhance the accuracy of invoicing and billing passed on to Defence.

Note c: Agenda papers and outcomes from the Program Governance Board meeting in September 2021 were sighted which confirm that the meeting was held. However meeting minutes or an attendance, decision and action log for the September 2021 meeting could not be located by Defence.

Source: ANAO analysis of Defence documentation.

Financial reporting

2.44 To facilitate Defence’s financial management, the contractor is to include a finance report within the quarterly Contract Status Report (CSR).⁴⁹

2.45 The ANAO examined 13 CSRs covering the period July 2019 to September 2022. Each CSR included a finance report which provided a summary of financial activity for the quarter, invoicing and/or billing observations⁵⁰, compliance matters⁵¹, forecasts⁵², and other financial matters.

Financial levers

2.46 The contract includes a range of financial levers to support Defence to manage contractor performance. These include financial incentives and disincentives, as summarised in Table 2.4.

Table 2.4: Financial levers in the health services contract

Source: ANAO analysis.

2.47 As at June 2022, Defence had claimed liquidated damages totalling $1 million from the contractor.

2.48 Under the contract, ‘Periodic Cost Reviews’ (PCRs) are to be conducted to give effect to the gainshare commitments. In practice, gainshare has been calculated by the contractor and Defence has used its ‘commercial audit program’ to gather invoices and transaction data to review gainshare calculations. As at June 2022, $10.4 million of gains had been shared with Defence.

2.49 The PCRs are also used to inform the award term assessment process.⁵³ The aims of a PCR are to provide the information needed to enable the Commonwealth to:

• determine the allowable costs that the contractor has incurred in providing recurring services;
• determine the savings achieved by the contractor flowing from approved initiatives and otherwise;
• determine the likely allowable costs to be incurred by the contractor as part of any proposed award term extension;
• ensure that the contract represents, and will continue to represent throughout the proposed Award Term, value for money; and
• ensure that Defence receives the benefit of the gainshare commitments.

2.50 A PCR was conducted in 2021 as part of the award term assessment process (see paragraphs 3.35 to 3.37). In June 2022, Defence advised the contractor that the initial PCR had satisfactorily provided the information required for Defence to ‘assess the items and addressed the aims of the PCR as outlined in the Contract.’ As at December 2022, Defence was conducting a second PCR.

2.51 The contract also provides for benchmarking activities to be undertaken. Benchmarking is intended to support Defence to: determine if the service fees are competitive with the current market; and compare service fees against comparable charges and rates for services that are the same as, or similar to, the services being provided to Defence. In February 2023, Defence advised the ANAO that it was engaging with the contractor to finalise the selection of a benchmarking organisation, however the benchmarking activity had not yet commenced.

Has Defence established effective arrangements to manage claims for payment?

Arrangements for managing payments

2.52 Defence’s Director ADF Health Services Contract (EL2 level) has day-to-day responsibility for overseeing and managing all contract administration, including reporting, invoicing and governance aspects of the contract. To support the contract manager, Defence has established a dedicated business unit (ADFHSC – Financial Governance) to manage the invoicing aspects of the contract, process claims for payment, plan and conduct the ‘commercial audit’ program, and develop and submit the audit reports.

Claims for payment

2.53 Under the ADF health services contract, the contractor submits a consolidated monthly invoice. The invoice covers the provision of health services on-base by the contracted health workforce and health services that have been accessed through the off-base network. Over the period assessed by the ANAO (July 2019 to June 2022) the average value of the consolidated monthly invoice was $46 million. To support its claims for payment, the contractor is required to provide a Monthly Service Delivery Report (MSDR) alongside its monthly invoice.

2.54 The minimum requirements of the MSDR are detailed in the Contract Management and Governance Framework (the Framework). The MSDR is to include data regarding the provision of health services in the garrison environment (on-base) by the contracted workforce and the services accessed through the off-base network.

2.55 The ANAO’s review of monthly invoices submitted from July 2019 to June 2022 indicated the following.

• For health services provided on-base — the monthly invoice ranged between $13 million and $22 million and the supporting MSDR was comprised of between 17,258 and 28,747 line items.
• For health services accessed through the off-base network — the monthly invoice ranged between $1 million and $29 million and the supporting MSDR was comprised of between 14,977 and 163,938 line items.

2.56 The process used to accept, validate and pay the consolidated monthly invoice submitted by the contractor is outlined in a Commercial Governance Plan. The plan also outlines the processes used to validate charges made by subcontractors and service providers providing health services accessed by ADF members through the off-base network.⁵⁴

2.57 The Commercial Governance Plan states that its objective is:

to provide the financial delegate with assurance that the invoices received by Bupa are correctly rendered and in accordance with the relevant financial policies and procedures.

2.58 The plan also states that claims for payment are to be submitted electronically through Defence’s systems and that:

in line with the delegations identified in the contract management plan and Defence policy the Contract invoice is to be approved for payment by the Commonwealth Representative nominated in the Contract.

2.59 The plan states that a two-step process is to be used to approve invoices. This involves ‘a detailed error check upon receipt to ensure the supporting documentation equals the service package total and in summary the invoice total.’ The invoice may be approved for payment if the invoice amount and the value of the supporting information has been reconciled.

2.60 The ANAO’s review of invoice approval processes indicated that claims for payment are approved by the Assistant Director ADF Health Services Contract (Financial Governance). While the Assistant Director has the financial delegation required to approve claims for payment, the position is not identified in the Framework as a key contact and the responsibilities of this role have not been documented in a contract management plan (see paragraph 2.20).

Billing system

2.61 Under the contract, Bupa is required to deliver an automated and transparent billing system capable of identifying billing and other anomalies and ensuring that claims for payment have been calculated and are payable in accordance with the contract. Defence observed, during transition-in, that the billing system was not fully functional at the operative date of 1 July 2019.

Commercial audit program

2.62 Defence has developed and implemented a ‘commercial audit program’ for the purpose of validating the accuracy of the invoices and supporting information provided by the contractor with its claims for payment.⁵⁵

2.63 The policy and processes that apply to the commercial audit program are detailed in the Commercial Governance Plan. The plan was approved in October 2021 and states that the purpose of the ‘commercial audit’ program is to test that:

• the charges to the Commonwealth for the services delivered were in accordance with Defence’s requirements;
• were delivered in accordance with Defence’s requirements;
• were delivered to or for a Defence member;
• were appropriate;
• were charged in accordance with the Contract terms; and
• where applicable were consistent with the clinical and business rules of the health industry within Australia.

2.64 According to the plan, Defence is to conduct commercial audits on a quarterly basis. Defence completed four commercial audits in 2020–21 and four in 2021–22. As at December 2022, Defence was in the process of progressing the first commercial audit for 2022–23.⁵⁶

2.65 To conduct the commercial audits, Defence identifies a sample of transactions and selects a number of line items from the sample extracted for further testing.⁵⁷ For the eight commercial audits conducted between July 2019 and April 2022, Defence sampled 24,095 (0.72 per cent) of 3.4 million transactions, valued at $633 million, for the provision of services off-base. Of the 24,095 off-base transactions, 5,798 (24 per cent) were selected for detailed testing. To select the sample of transactions for detailed testing, Defence identifies key areas or categories of interest. For example, in March 2022, it was decided that the commercial audit would include a general invoice review and a targeted review on ‘duplicates’ and ‘hospital charging’.

2.66 The commercial audit program has identified a range of issues with the quality of data provided to Defence. Identified errors have included: the service provider name did not match the name on the referral; the speciality did not match; the ABN was incorrect; the Employee ID was not recorded; a Defence Approval Number (DAN) was not recorded; the referring health practitioner and health centre were incorrect; and the quantity, item number and description were incorrect. Error rates by category, as identified by the commercial audit program, are illustrated in Figure 2.1.

2.67 In June 2021 the ANAO observed, as part of its 2020–21 financial statements audit work, that for the period July 2019 to October 2020⁵⁸ Defence had found that ‘a significant percentage’ of provider invoices did not contain sufficient information to confirm that the invoice was for an authorised service provided to an eligible member.⁵⁹ The ANAO also:

• found issues with the timeliness of the completion, approval and management sign off, of the quarterly reviews conducted by Defence⁶⁰; and
• observed that the commercial audit program was limited to the provision of services through the off-base network and recommended that the assurance program be extended to include the provision of services on-base.

2.68 The ANAO recommended that Defence examine and strengthen the design of processes to provide assurance over the accuracy and validity of the health service payments. Given the issues noted in the review of off-base health services, the ANAO also recommended that the assurance activities be extended to include on-base service charges. The ANAO stated that the assurance processes should be completed in a timely manner and issues arising escalated to an appropriate level of management to ensure that issues can be dealt with promptly and recoveries initiated where required.⁶¹

2.69 Defence agreed to the ANAO recommendations. To address the findings from the ANAO’s 2020–21 financial statements audit work, Defence’s commercial audit program was extended to include an examination of on-base services. Between September 2021 and April 2022 Defence selected 5,823 transactions (1.12 per cent) valued at $3.3 million (1.79 per cent) of 521,629 transactions valued at $187 million for the provision of services on-base. These commercial audits identified risks and issues regarding: duplication of invoices for the provision of on-base services; total hours charged exceeding 24 hours for a single day and for a single contracted health professional; contracted health professional IDs linked to more than one individual; contracted health professionals linked to more than one ID; and contracted health professionals charged at higher rates outside of their registration (for example, an enrolled nurse being charged out as a registered nurse).⁶²

2.70 Work has been underway since February 2021 to address the findings from Defence’s commercial audit program and improve the quality of the invoicing data submitted to Defence. The improvements are to be delivered through the development and implementation of a suite of business rules within the contractor’s systems that are necessary to ensure that claims for payment comply with the requirements of the contract.⁶³

2.71 In March 2022, Defence was advised that the contractor had included additional fields in its data set to meet Defence’s requirements that the service had been provided to an eligible person. Defence also reported that the data fields had been added in June 2021 and that the September 2021 commercial audit had confirmed the remediation activities had been implemented. It was also noted that for the purposes of the commercial audits, Defence randomly samples approximately one per cent of the health invoices received by Bupa for the relevant period.

2.72 As part of the 2021–22 audit of Defence’s financial statements, the ANAO found that weaknesses around the governance of ADF health services identified in the 2020–21 audit remained unresolved. As at December 2022, work to develop and implement the business rules required to improve the billing system was ongoing.

Additional ANAO analysis of data quality

2.73 In the context of this performance audit, the ANAO reviewed the information recorded in 35 Monthly Service Delivery Reports (MSDRs) covering the period July 2019 to June 2022.⁶⁴ This information comprised four million transactions for the provision of health services through the off-base network (with a value of $744 million) and 711,360 transactions for services delivered on-base (valued at $547 million). The ANAO identified that 26 per cent of transactions contained at least one or more data quality issue. In summary, there were:

• 39,334 instances (one per cent) where the Employee ID had not been recorded;
• 137,877 instances (three per cent) where a Defence Approval Number had not been recorded⁶⁵;
• 254,821 instances (six per cent) where the referring health practitioner had not been recorded;
• 343,692 instances (nine per cent) where the referring health centre had not been recorded; and
• 607,612 instances (15 per cent) where the Service Provider Name, Number and/or ABN had not been recorded.⁶⁶

2.74 The ANAO tested the off-base transactions against a set of business rules that the ANAO confirmed with Defence should be in place. The ANAO’s analysis, and the errors identified by Defence’s commercial audit program, indicate that Defence does not have a fully effective control framework for payments and has paid invoices that were incomplete. Depending on the invoice, Defence was not informed of: whether the ADF member was approved to obtain the service; who provided the service; what service was provided; and where the service was provided. The payment of incomplete invoices limits Defence’s ability to provide assurance on the proper use of public resources for which it is responsible.

2.75 Key results from the ANAO analysis are summarised in Table 2.5.

Table 2.5: Data quality issues — Monthly Service Delivery Reports

Source: ANAO analysis of data provided by Defence.

2.76 The ANAO and Defence’s commercial audit program have identified data quality issues which indicate that there are weaknesses in the processes established to ensure that appropriate (complete and accurate) data is recorded and transferred to Defence.

2.77 The ANAO identified three combinations where blank or invalid fields increased the opportunity for fraudulent transactions to occur and constrained the ability of Defence to detect it. The three combinations were as follows.

• Where no Defence Approval Number (DAN), no EPID (Eligible Person ID), and no reason for not recording an EPID has been provided.
• Where no EPID, and no reason for not recording the EPID has been provided.
• Where multiple EPIDs are associated with a unique DAN.

2.78 A Defence Approval Number (DAN) is required to generate a referral to enable Eligible Personnel (EP) to access specialised health care, or health care that cannot be provided on-base. An Eligible Person ID (EPID) identifies the ADF member that the health care has been provided to. Where a DAN is not recorded, Defence is unable to confirm that the health services it has been invoiced for have been authorised.

2.79 A DAN, EPID and/or the reason for not recording an EPID is required to enable Defence to assure itself that only approved health services are being provided to EPs. Where this combination of fields is blank, Defence is unable to confirm that the health services provided were approved and is unable to identify the member that the services were delivered to. This combination of blank fields poses risks to Defence, including by constraining its ability to provide effective clinical follow-up. The ANAO identified 39,334 transactions where an EPID was not recorded, and where a reason for not recording one was not provided. The total value of these services was approximately $4 million. Of these 39,334 transactions, 10,927 (28 per cent) also had no DAN recorded. The total value of these services was approximately $1 million.

2.80 A DAN is intended to allow Defence to trace the quantity and categories of health services that have been provided to an EP. There should not be multiple EPIDs linked to one DAN. Excluding pathology services⁶⁷, the ANAO identified 452,556 unique DANs used for approving the delivery of health services, of which 3,305 (0.7 per cent) had more than one EPID linked to it. These 3,305 DANs were used to approve 55,257 transactions worth $20.2 million.

2.81 The National Archives of Australia identifies eight characteristics of data quality — accuracy, completeness, consistency, integrity, reasonability, timeliness, uniqueness, validity — and identifies that a good data quality strategy defines appropriate standards, requirements, and specifications for data.⁶⁸ Establishing mandatory data entry rules is important in ICT systems. Preventing the entry of invalid data supports accuracy, completeness, integrity and consistency of data and enhances the quality of data used to inform decision-making. The ongoing work described in paragraph 2.72, to develop and implement the business rules required to improve the billing system, should have regard to data quality requirements.

3.1 This chapter examines whether reporting and monitoring arrangements have been established that support effective contract management. Effective and fit-for-purpose contract reporting arrangements enable Defence to monitor and evaluate contractor performance in delivering the services and outcomes of the contract, while internal reporting arrangements enable oversight of Defence’s administration of the contract.

3.2 To assess whether Defence has established fit-for-purpose performance monitoring, reporting and evaluation arrangements, the ANAO reviewed both the design and implementation of the arrangements.

Has Defence established fit-for-purpose performance monitoring, evaluation and reporting arrangements?

Performance Management Framework

3.3 The ADF health services contract includes a Performance Management Framework (PMF) that sets out:

• performance measures;
• performance payments;
• performance reporting; and
• performance reviews and assessments.

3.4 The performance assessment process, set out in the PMF, is intended to measure and assess the contractor’s performance against the required outcomes. The required outcomes are:

• improved health outcomes for eligible personnel and value for the Commonwealth;
• a joint health effect that enables capability and provides care for eligible personnel; and
• a health effect that enables capability and provides administrative support to the Chief Medical Officer, Defence Force Recruitment.

3.5 There is an alignment between the required outcomes in the contract and the legislative responsibility of Defence to provide medical and health treatment for members of the Australian Defence Force (ADF). There is also an alignment between the performance measures used to determine whether an award term extension⁶⁹ will be granted and the objectives of the contract.

3.6 To achieve the required contract outcomes, seven contributory outcomes or Key Result Areas (KRAs) have been identified in the PMF. The seven KRAs are: readiness; wellbeing; patient experience; command experience; sustainability; cost efficacy; and behaviours.

3.7 There are four types of performance measures used to assess the contractor’s performance against the KRAs. These are: Enterprise KPIs (EKPIs); Contract KPIs (CKPIs); Strategic Performance Measures (SPMs); and Strategic Health Indicators (SHIs).

3.8 The performance measures and the relationship between the measures and the key result areas identified by Defence are illustrated below in Figure 3.1.

Figure 3.1: ADF health services contract — performance management hierarchy

Note: Enterprise KPIs (EKPIs) attract payment incentives and are intended to measure the contractor’s contribution to delivering improved health outcomes. Enterprise KPIs have been coloured yellow.

Note: Contract KPIs (CKPIs) affect the payment of the ‘At-Risk’ amount. Contract KPIs are intended to measure the contractor’s ability to provide a contracted health professional workforce to meet Defence’s long and short-term requirements. Contract KPIs have been coloured orange.

Note: Strategic Performance Measures (SPMs) are used to assess whether to award a contract term extension. Strategic Performance Measures have been coloured light blue.

Note: Strategic Health Indicators (SHIs) are used by the Commonwealth to validate KPIs and measure the performance of services not linked to a performance payment. Strategic Health Indicators have been coloured green.

Source: ADF health services contract.

3.9 The ANAO examined the measures and indicators, their purpose in the hierarchy, and whether they have been implemented. As discussed in Table 3.1 below, the Enterprise KPIs (EKPIs) and Strategic Health Indicators (SHIs) have been partly implemented. The Strategic Performance Measures (SPMs) have been largely implemented and the Contract KPIs (CKPIs) have been fully implemented.

Table 3.1: Performance measures and indicators

Note a: To measure improved population health outcomes, the contract identifies six population health metrics that will be used: dental care; degree of health literacy and capacity for self-care; two metrics associated with return to work; and back care.

Note b: The patient experience includes satisfaction and experience. The performance management framework for the contract states that a patient’s overall satisfaction is considered to be influenced by, but not limited to, timeliness, privacy, communication, ease of use, facilities, and perception of clinical quality.

Note c: The Defence command experience includes satisfaction and experience. The performance management framework for the contract states that command satisfaction is considered to be influenced by, but not limited to, the quality of clinical outcomes, communication, engagement, and reporting.

Note d: At October 2022, there were 17 full-time equivalent Medical Officer positions identified to provide support directly to Defence Force Recruitment.

Note e: The review periods for the Strategic Performance Measures vary. One SPM (SPM-04 Innovation) is to be reported on annually. Four SPMs (SPM-03 Health Steward, SPM-06 Cost Containment, SPM-07 Relationships and SPM-08 Best for Defence) are to be reported on bi-annually and three SPMs (SPM-01 Agility to support readiness, SPM-02 Commitment to patient satisfaction and SPM-05 Defence ready) are to be reported on quarterly.

Source: ANAO analysis of Defence documentation.

Review and evaluation of the performance measurement framework (PMF)

3.10 The PMF included a 12-month performance implementation period for the EKPIs. The performance implementation period was to be in place from the contract’s operative date (1 July 2019) to 30 June 2020. The objectives of the performance implementation period were to:

• validate the accuracy of the EKPIs as appropriate measures to assess the extent to which the contractor is contributing to the achievement of enterprise outcomes;
• confirm that the data collected and used to measure performance provides an accurate and valid measure;
• verify the processes for the collection of the data, measurement and reporting of EKPIs; and
• minimise the impact of unrepresentative performance discrepancies.

3.11 The PMF identifies six population health metrics that are to be used to assess EKPI-01. These are: dental care; degree of health literacy and capacity for self-care; two metrics associated with return to work; and back care. The 12 month performance implementation period (1 July 2019 to 30 June 2020) that applied to EKPIs was also to be used to establish a baseline for the six population health metrics. In June 2020, the contractor reported that the six population health metrics were yet to have a baseline established and that data from Defence was required to complete the activity. On 8 October 2020 the performance implementation period was extended to 31 December 2020 to allow more time to establish a baseline for the population health metrics and enable Defence to assess the contractor’s contribution to improved health outcomes as measured by EKPI-01. In February 2021, the contractor advised Defence of challenges in measuring the EKPI-01 metric as outlined in the contract. As discussed in Table 3.1, as at December 2022, EKPI-01 had not been implemented and Defence was unable to assess the extent to which the contractor has contributed to improved health outcomes. As at March 2023, work to implement EKPI-01 remained paused while Defence progressed a body of work commissioned in November 2021 to revise the PMF (see paragraphs 3.12 and 3.13).

3.12 In November 2021, Defence commissioned an external consultancy to undertake a review of the PMF.⁷⁰ The purpose of the review was to assess whether: the performance management framework was achieving required outcomes; whether the metrics appropriately incentivise the contractor to achieve the targets; and examine the position put forward by the contractor that a number of measures were not achievable and were considered ‘unfair’. In April 2022, the consultants provided Defence with the review findings and in August 2022, the review report was presented to Defence and the contractor at the Garrison Management Committee. The review found that the PMF:

• structure was not punitive;
• works as intended to ensure equitable service delivery across all facilities;
• rewards volume of care over quality of care;
• does not clearly relate to and incentivise achievement of contract objectives;
• includes performance thresholds that have not been consistently validated against service delivery requirements, and are highly sensitive to small changes in performance;
• reporting does not consistently provide the granularity required to inform operational decision making;
• incentivises the contractor to fill short-term absences even where not required; and
• potentially incentivises the contractor to fill roles with anyone with the appropriate credentials rather than selecting staff who are high performing and suitable for the Defence environment.

3.13 The review did not make any recommendations however it did identify opportunities for improvement. For example, the review found that that there would be benefit in clarifying the methodology and data sources used by Defence to assess the qualitative performance measures. Defence started a process in late 2022 to update and revise the PMF to address the issues identified by the reviewers. In February 2023 Defence advised the ANAO that a project to revise the PMF was underway.

Performance payments

3.14 The ADF health services contract includes a range of payments to support Defence manage contractor performance. The performance payments are detailed in Table 3.2 below.

Table 3.2: Performance–related payments

Source: ANAO analysis.

3.15 When the contractor has submitted claims for performance payments to Defence, Defence has verified the calculations are accurate and paid the invoice. As at October 2022, Defence had made incentive payments totalling $7 million and retained $18.8 million of the ‘at-risk’ amount.

3.16 Under the contract restrictions apply to claims for incentive payments. The restrictions are intended to ensure that such payments are not made where key requirements have not been met. For example, where the Commonwealth is entitled to claim liquidated damages or terminate the contract for default.

3.17 For the period examined by the ANAO (July 2019 to October 2022), performance payments were made in error by Defence. That is, performance payments were made in cases where contractual clauses restricting certain payments had been triggered. In July 2022, the ANAO advised Defence that it had identified $4.7 million in incentive payments had been made in error as the contractual clauses restricting certain payments had been triggered. In August 2022, another performance payment of $1.1 million was claimed and paid by Defence in error. Subsequently, Defence sought legal advice and commenced a review to identify all instances where contractual clauses restricting certain payments had been triggered and if other incentive payments had been made in error. Defence has also sought to recover the funds identified as paid in error, as a debt owed to the Commonwealth.

Performance reporting, reviews and assessments

Performance reporting

3.18 The contract requires the contractor to provide a monthly transactional report and a quarterly CSR. Each CSR is to include 14 components, including a performance measurement report intended to help Defence assess the contractor’s performance against the performance measures (see Figure 3.1).

3.19 The ANAO examined the 13 CSRs submitted between 1 July 2019 and 30 September 2022 and found that the performance measurement reports largely met the minimum standards established in the ADF health services contract.

Performance reviews and assessments

3.20 The contract sets out six types of performance reviews that are to be undertaken: periodic performance reviews; combined and/or individual services performance reviews; contract performance reviews; performance assessment reviews; and award term assessments, inclusive of periodic cost reviews.

Periodic performance reviews

3.21 Periodic performance reviews are to be conducted by the contractor. The contract states that periodic performance reviews are to be conducted in accordance with the approved Services Management Plan (SMP) and are to report on and review: the performance of services provided in the period just completed against the requirements of the contract, including the requirements of the Statement of Work; the estimated requirements for services in next and future periods; and any issues or risks that could affect the provision of the services in future periods.

3.22 The contract sets out that the frequency of these reviews is to be identified in accordance with the approved SMP; and that details of how the contractor proposes to conduct them are to be included in a communications plan. A communications plan was approved in March 2019 for the transition-in period as required under the contract (the contract was to take effect from 1 July 2019, after a six month transition-in period). However, a communications plan has not been in place since 1 July 2019 and the SMP does not refer to periodic reviews. In November 2022, Defence advised the contractor that the intent of the periodic performance reviews was being met through the provision of feedback on the quarterly CSRs.

Combined and individual services performance reviews

3.23 Combined services performance reviews are to be conducted by the contractor at intervals of no greater than three months unless otherwise agreed with the Commonwealth Representative. For individual services performance reviews, Defence may request at its discretion that the contractor participate in those reviews. The aim of both types of performance reviews is to provide an opportunity for both parties to discuss the provision of services, including any issues. The reviews are to consider matters that have been included in the CSRs and any other applicable reports.

3.24 While these aims are outlined in the contract, the mechanism through which the reviews are to be conducted are not identified. Additionally, there is no reference to a plan or guide, such as the SMP, that would otherwise set out how these contractual requirements should be met.

Contract performance reviews

3.25 Contract performance reviews are joint reviews, involving an appraisal by Defence of the contractor’s performance and an appraisal by the contractor of Defence’s performance. The aims of these reviews are to:

• consider matters reported in the CSRs, including problems, opportunities, risks and issues relating to the services; and
• facilitate the early identification and mitigation of any adverse effects on the contract of contractor or Commonwealth performance, and to deal with external changes impacting upon the contract.

3.26 The SMP, approved by Defence, identifies that joint contract performance reviews are to be undertaken every six months, or at a time as otherwise agreed. While Defence has not conducted the contract performance reviews as required under the contract, it has conducted a desk-top performance appraisal every six months since January 2020.

3.27 The ANAO examined the timeliness of Defence’s provision of appraisals to Bupa. Appraisals were not provided to Bupa for the January–June 2020 and July–December 2020 periods. The appraisal for January–June 2021 was provided in November 2021. The appraisal for July–December 2021 was provided in June 2022. The appraisal for January–June 2022 was provided in December 2022.

Performance assessment reviews

3.28 The contract includes two mechanisms to assess contractor performance against contracted requirements — performance assessment reviews and award term assessments.

3.29 The aims of performance assessment reviews are to enable Defence to:

• consider the contractor’s adjusted performance score for each KPI and performance against other applicable performance measures;
• provide an appraisal of the contractor’s performance;
• consider evidence provided by the contractor in support of any performance adjustment claims;
• determine the performance payments; and
• review the analysis of potential Initiatives and the progress of Approved Initiatives as part of the Continuous Improvement and Innovation (CII) program.

3.30 Defence has tracked the contractor’s adjusted performance score for each KPI, Strategic Performance Measure and Strategic Health Indicator since July 2019. Defence has also conducted a desk-top performance appraisal every six-months since January 2020. (See paragraphs 3.26 to 3.27.)

3.31 As at December 2022, Defence had not held a performance assessment review meeting to determine the performance payments or review the analysis of potential incentives and progress of Approved Initiatives as part of the CII program.

3.32 In the absence of performance assessment review meetings, Defence has done the following.

• When the contractor has submitted claims for performance payments to Defence, Defence has verified the calculations are accurate and paid the invoice (see paragraph 3.15).
• The contractor’s progress to implement Approved Initiatives was initially reported to the Garrison Management Committee (GMC) prior to the commencement of the Continuous Improvement and Innovation (CII) Committee in April 2021.⁷¹ Since April 2021, the progress of Approved Initiatives has been monitored by the CII committee, with updates provided to the GMC and a summary of progress provided to the PGB.

3.33 In October 2022, the contractor wrote to Defence requesting that the performance assessment review meetings be held. Defence responded in November 2022 and proposed that the Contract Delivery Committee (CDC) be used to meet the intent of the performance assessment review requirements outlined in the contract.

3.34 In February 2023, Defence advised the ANAO that it monitors performance payments monthly and has implemented an internal process to review the payment of potential incentives. The ANAO reviewed the supporting evidence provided and noted that while Defence has strengthened its assessment, review and approval activities, work to document, formally approve and implement the improved assessment and review processes had not yet been completed. In regard to the progress of Approved Initiatives, Defence advised the ANAO that the CII committee is responsible for monitoring progress.

Award Term Assessment

3.35 The contract expiry date is 30 June 2025, unless it is extended by an ‘Award Term’. Award term assessments are used to determine if an award term extension will be granted. The assessment process and criteria are set out in the contract. The criteria are as follows.

• The contractor has performed its obligations in a manner which satisfies the contract objectives.
• The contractor’s behaviours have positively contributed to its performance against the objectives of the contract and the performance requirements.
• The contractor’s performance has been assessed as being in performance band ‘A’ or ‘B’ for every review period.
• The contractor’s performance against the Enterprise KPIs and Other Performance Measures relevant to an award term is acceptable to the Commonwealth (Strategic Performance Measures).
• The outcome of a periodic cost review, conducted in accordance with the contract, is acceptable.
• The contractor has received no more than two ratings of ‘unsatisfactory’ in respect of the Continuous Improvement and Innovation (CII) Program at the Performance Assessment Reviews.
• The contractor has not been required to raise a remediation plan, or where one or more remediation plans have been required, the contractor has, in the opinion of the Commonwealth Representative, completed all the steps and activities required.

3.36 Since contract signature Defence has conducted one award term assessment and was in the process of conducting a second during this audit.⁷² The first assessment commenced in April 2021 and included a periodic cost review (PCR) that commenced in July 2021, upon receipt of the necessary financial data from the contractor.⁷³ The award term assessment used the desk-top appraisals conducted by Defence to inform Defence’s assessment of contractor performance against the performance measures.

3.37 Completion of the first award term assessment was due by 30 June 2022. In May 2022, Defence sought and obtained agreement from the contractor to amend the initial award term determination date to 30 September 2022. The award term assessment results were provided to the contractor in August 2022.

3.38 In summary, Defence considered that the contractor’s performance was ‘unsatisfactory’ and elected not to offer an award term extension, reducing the maximum duration of the contract to nine years. The contractor was advised that it was expected to develop and comply with a remediation plan to:

• lift performance against the contract KPIs, and ensure Defence has complete transparency of its subcontractor’s remediation plan;
• develop and implement ICT systems and process functionality enhancements, including the overall effectiveness of the iRBS;
• develop and implement a communication strategy to ensure timely management of emerging risks and issues including performance shortfalls; and
• address audit findings regarding the invoicing issues identified.

4.1 This chapter examines how Defence provides assurance that the delivery of health services meets contracted requirements, and whether the expected cost and service delivery efficiencies under the contract have been realised.

4.2 The effective delivery of contracted requirements and the achievement of expected efficiencies, supports the achievement of value for money as required by the Commonwealth Procurement Rules, and the proper use of public resources as required by the Public Governance, Performance and Accountability Act 2013 (PGPA Act).⁷⁴

Have services been delivered effectively against contracted requirements?

Health services

Health services provided to ADF personnel

4.3 As discussed in paragraph 1.3, access to off-base health care facilities and staffing of on-base facilities is acquired by Defence through the ADF health services contract.

4.4 Health services are provided to eligible Defence personnel, comprising approximately 60,000 permanent ADF members and approximately 25,000 reservists. Health services are provided day-to-day across 51 health centres and clinics located on Defence bases throughout Australia.⁷⁵ Where an ADF member needs health care that cannot be provided on-base, ADF members are able to access a network of health care facilities and providers off-base.

4.5 As discussed in paragraphs 1.14 and 1.15, a large and diverse health professional workforce is engaged under the contract to provide health services to ADF members on-base (see Figure 1.1) and a wide variety and large volume of health services are available to, and have been accessed by, ADF members through the off-base network (see Figure 1.2).

Health services provided to civilians

4.6 While the ANAO has not audited the delivery of clinical services under the ADF health services contract, Defence advised the ANAO that the contract had been used to provide health services to civilians as part of Operation COVID Assist, under Defence Assistance to the Civil Community (DACC) arrangements.

4.7 The ADF health services contract defines ‘Eligible Personnel’ as:

ADF personnel, reserve members serving on continuous full time service, reserve members serving on other than continuous full time service, and General Reserve–Special Reserve as detailed in the Defence Health Manual … and any other personnel approved by the Commonwealth.

4.8 In March 2021, Defence advised the contractor that it had declared ‘residents of aged care facilities’ as ‘Eligible Personnel’ and aged care facilities as ‘Defence places of work’ to enable its contracted health professionals to deliver COVID-19 vaccinations to those residents as part of DACC arrangements.⁷⁶

4.9 The Defence Health Manual states that a Defence member’s eligibility is associated with the member’s Service Category or Service Option.⁷⁷ While the focus is on permanent ADF members and reservists, the manual also outlines that ‘Defence health services provides care for civilians only when appropriate civilian resources are not readily available and until the patient can be safely transferred to a non-Defence health services provider.’ Notwithstanding this, the Defence Health Manual expressly does not apply to assistance provided under DACC arrangements.⁷⁸

4.10 In March 2023, the ANAO asked whether Defence had quantified the cost of providing health services to civilians as part of Operation COVID Assist. Defence advised the ANAO that the costs could not be quantified due to an inability to separate the work undertaken by the contracted workforce for Operation COVID Assist from business-as-usual tasks.⁷⁹ Defence further advised that ‘all ADF health services contract costs associated with directly supporting Operation COVID Assist, Op FLOOD Assist and Op BUSHFIRE Assist’ were absorbed by Joint Health Command and therefore Defence did not seek ‘no win/no loss’ funding.⁸⁰

4.11 In May 2023, the ANAO asked Defence for documentation confirming the contractual basis for the deployment of its contracted health professionals into aged care facilities, and to confirm the source of the policy authority supporting their deployment. Defence advised that it could not provide any further information as it had already provided the ANAO with the extent of the available contemporaneous evidence.⁸¹

4.12 Defence’s inability to confirm the source of policy authority for decisions taken as recently as 2021, limits its ability to support Parliamentary scrutiny and external review of its decision-making and administration. Other shortcomings in Defence’s record keeping arrangements for the ADF health services contract were discussed in paragraphs 2.7 to 2.8. Recommendation 1 in this audit report is that Defence ensure that all record keeping requirements are complied with.

Performance Management Framework

4.13 The performance assessment process set out in the Performance Management Framework (PMF) for the contract, is intended to measure and assess the contractor’s performance against the required outcomes. The PMF was reviewed in chapter 3.

4.14 As discussed in paragraph 3.9, the performance measures underpinning the assessment arrangements have been partly implemented, with the exception of the Contract KPIs (CKPIs) which have been fully implemented.

4.15 Without fully functioning performance assessment arrangements, Defence is not well placed to assure itself that services are being delivered effectively against the contracted requirements.

Arrangements to provide assurance that credentialing, training and security requirements have been met that support the effective delivery of services in the Defence environment

4.16 To support the effective delivery of the contracted services, the ADF health services contract establishes a range of requirements that are intended to ensure that the services are fit-for-purpose for the Defence environment. These include credentialing, training and security requirements.

4.17 The security requirements include accreditation requirements that apply to service providers engaged by the contractor to provide health services to ADF members. Other security requirements applying to the contractor and its subcontractors relate to the ICT systems used to collect, record and transfer health information. Under the contract, the ICT system used by the contractor to deliver health services to ADF members is required to be accredited. The accreditation is required as the ICT systems collects, records and transfers personally identifying information.

4.18 Defence’s arrangements to provide internal assurance that credentialing, training and security requirements have been met, are detailed in Table 4.1 below. In summary, assurance arrangements for credentialing are fully established, they are largely established for the training and security requirements.

Table 4.1: Assurance arrangements for contracted credentialing, training and security requirements

Note a: The contractor’s ICT system used to deliver health services to ADF members stores personally identifiable information.

Note b: The Annual Security Reports for 2020 and 2021 were provided to the Defence Industry Security Office (DISO). Copies of these reports were not provided to Joint Health Command, until requested in October 2022. The Annual Security Report for 2022 was provided to DISO and a copy was also provided to Joint Health Command.

Note c: As part of the 2020–21 Defence financial statements audit, the ANAO found that Defence was unable to provide evidence and assurance that personally identifiable information was being managed appropriately. In 2021–22, the finding remained unresolved.

Source: ANAO analysis of Defence documentation.

4.19 Table 4.1 indicates that while formal accreditation of the contractor’s ICT system was achieved in April 2023, the accreditation process was not completed in a timely manner. As outlined in the table, provisional accreditation was achieved in June 2019 and full accreditation was achieved in April 2023, almost four years later. As discussed in paragraph 4.17, the contractor’s ICT system collects, records and transfers the personal information of ADF members. Defence therefore requires assurance that the risks associated with contractor ICT systems are appropriately managed. The ANAO found, as part of its 2020–21 financial statements audit work, that Defence was unable to provide evidence and assurance that personally identifiable information was being managed appropriately. In 2021–22, the finding remained unresolved.

Have the expected cost and service delivery efficiencies under the contract been realised?

4.22 The objectives of the contract (see Box 1: of this audit report) include the following goals for cost and service delivery efficiency.

• ‘Collaborative delivery of episodes of care, that are effective and efficient’.
• ‘To achieve, over the Term, cost savings associated with the delivery of the Services, through the identification and implementation of initiatives, innovations and otherwise’.
• ‘To encourage the most efficient use of resources for the delivery of the Services and achievement of the Capability’.

Arrangements for achieving cost and service delivery efficiencies

4.23 The cost and service delivery efficiencies that Defence expected to achieve under the new ADF health services contract were identified in a benefits management plan and supporting benefits register, log and tracker. These documents were developed during the procurement phase of the contract, and identified the following strategic benefits that Defence hoped to achieve through implementation of the contract.

• Improved health outcomes.
• Greater workforce capability.
• Improved commercial acumen.
• Improved customer (patient and command) experience.
• Greater focus on ensuring the health preparedness of ADF personnel.

4.24 In October 2019, benefits management was handed over to five ‘action owners⁸²’ and was to be overseen by the Corporate Governance Board (CGB). The CGB last met in November 2021 and was replaced by the Executive Board in 2022.⁸³ In its review of the meeting minutes for the CGB and Executive Board, the ANAO found no evidence of progress reporting against the benefits identified in the benefits management plan.

Continuous improvement and innovation program

4.27 Under the contract, the contractor is required to develop, implement and refine, on an ongoing basis, a Continuous Improvement and Innovation Program (CII program) that:

• improves clinical outcomes (including the reduction of low value care where appropriate);
• enables the cost to the Commonwealth for the provision of services to be reduced;
• reviews process and procedures to ensure best practices are identified and implemented;
• promotes a culture of efficiency and cost consciousness between the Commonwealth and the contractor that actively seeks to achieve a reduction in the total cost to the Commonwealth for the provision of services; and
• improves the health literacy of eligible personnel and commanders.

4.28 The contractor is to conduct and manage the CII program in accordance with the approved Continuous Improvement and Innovation Plan (CII plan). The CII plan was initially approved in September 2019 and was last updated in September 2021.

4.29 As of December 2022, the CII program had identified 17 initiatives. These were approved for implementation by the Continuous Improvement and Innovation Committee (CII committee). The CII committee is a joint contractor–Defence committee that convened in April 2021. The initiatives that have been identified are summarised in Table 4.2.

Table 4.2: Continuous improvement and innovation initiatives approved for implementation as at December 2022

Note: The Continuous Improvement and Innovation (CII) Plan defines initiatives in the ‘pipeline’ as initiatives which have been approved but are yet to be implemented. The categories of initiatives identified in the opportunity register no longer align with the categories in the CII plan.

Source: Extracted from the ‘opportunities register’ provided to the Continuous Improvement and Innovation (CII) Committee at its December 2022 meeting.

4.30 According to the Performance Management Framework, Defence is to: review, on an annual basis, the contractor’s progress to implement the initiatives approved by the CII Committee; and provide a copy of the assessment to the contractor for inclusion in the CSR.

4.31 In February 2022, as part of a business case submitted to Defence’s Enterprise Business Committee seeking additional funds for the contract, the committee was advised that three continuous improvement and innovation initiatives were underway or had been implemented. These were: the purchase of four ADF dedicated low acuity mental health inpatient beds at the Darwin Private Clinic; the booking and payment of a GP appointment for all ADF members transitioning from Defence to enable a clinical handover; and a national telepsychiatry initiative to improve access to psychiatry services.

4.32 In April 2022, Defence assessed that the CII Program was starting to deliver improved clinical outcomes. The assessment also noted that the:

program was still in its infancy and noted concerns around the capacity of Bupa and Defence to implement the initiatives in a timely manner.

4.33 As at December 2022, three of the 17 initiatives identified through the CII program had been implemented. The remaining 14 initiatives were categorised as either ‘operationalised’, ‘approved, yet to be implemented’ or in the ‘pipeline’.⁸⁴

4.34 The approved initiatives in Table 4.2 (above) are recorded in an ‘opportunities register’ that was developed jointly by the contractor and Defence.⁸⁵ The ANAO examined the register and noted that the register does not enable Defence to identify:

• the link between the proposed initiative and the stated objectives of the CII program;
• a baseline or benchmark that an initiative is intended to improve; or
• how success will be measured.

4.35 In February 2023, Defence advised the ANAO that it acknowledged that:

• the link between an initiative and the objectives of the CII program is not included in the ‘opportunity register’; and
• the ‘opportunity register’ does not contain the expected timeframes, benefits or the baseline, that an initiative is intended to improve.

4.36 Defence also advised the ANAO of the following.

• Three criteria are used to prioritise initiatives. These are: alignment with the ADF Health Strategy, Value for Money, and Risk Reduction. This information is available in a separate register.
• A forward workplan, which identifies expected timeframes for implementation, was presented to the Program Governance Board in August 2022.
• Intended benefits are identified and managed at the individual initiative level.

Other mechanisms to track planned cost and service delivery efficiencies

4.37 As discussed in paragraphs 1.9 and 1.11, Defence expected the contract to deliver a number of benefits, including cost and service delivery efficiencies. In December 2018, when seeking approval to enter into the new contract, Defence advised the Ministers for Defence and Finance that the contract would deliver:

• enhanced health service delivery with a robust continuous improvement and innovation process;
• improved business intelligence through automation, data collection and analysis; and
• improved commercial arrangements through specific contract mechanisms that promote cost containment.

4.38 Defence also advised ADF members and families that:

Under the ADF Health Services Contract, Defence will see a greater use of data and analytics in health service delivery, the identification and minimisation of low value care, and an increased focus on continuous improvement and innovation.⁸⁶

4.39 The expected service delivery efficiencies were identified in the contract objectives (see paragraph 1.12) and the benefits management plan, register and log (see paragraph 4.23).

4.40 Mechanisms to achieve these objectives encompassed improved business intelligence, including improved data collection and analysis. Defence has also undertaken an internal review of cost drivers (see paragraph 4.43).

4.41 Improved business intelligence — through automation, data collection and analysis — was to be enabled and delivered by the contractor through its technical solution.⁸⁷ Issues regarding the functionality and usability of this technical solution have been raised by Defence since entering into the contract. In August 2022, Defence advised the contractor that it was to develop and implement ICT system and process enhancements to its technical solution, including the ‘intelligent Referral and Booking System’ (iRBS).

4.42 Data quality issues have also affected expectations regarding improved data collection and the use of analytics to achieve cost and service delivery efficiencies (see the discussion of data quality in paragraphs 2.73 to 2.81). One of the outcomes of the Award Term Assessment was a requirement that the contractor develop and implement a remediation plan to deliver ICT system and process enhancements for the booking referral process, reporting, and invoice processing. This was communicated to the contractor on 8 August 2022. On 11 October 2022, the contractor submitted a remediation plan which was not approved by Defence. On 11 November 2022, the contractor submitted a revised remediation plan which Defence ‘approved with minor defects’ in December 2022.

4.43 In October 2021, Defence completed an internal review of the costs incurred to date under the contract, to identify the factors contributing to a forecast overspend.⁸⁸ Defence assessed that the source of the overspend against the approved contract expenditure was a result of the following factors.

• Higher than initial estimated demand for contracted health professionals, including additional short-term (flex-fill) arrangements to meet ‘surge’ requirements.
• Allowances for non-priced off-base items were underestimated.
• Application of the Multiple Operation Rule in non-network hospitals did not appear to be occurring.
• An increased demand for several off-base service packages.
• The impact of unanticipated costs related to the COVID-19 pandemic.

4.44 The review observed that ‘there are opportunities for JHC [Joint Health Command] to address ongoing financial governance process and take a proactive approach in management of s23 [PGPA Act section 23] commitment versus forecast demand on an ongoing basis.’

4.45 The ANAO examined contract costs for July 2019 to June 2022 and identified the following.

• The growth in expenditure exceeded the one per cent growth in usage and estimated indexation allowed for in the contract.
• Key cost drivers have increased, particularly the on-base workforce, which grew from approximately 997 FTE in July 2019 to approximately 1,283 FTE in June 2022.
• The last reported cost per eligible person (CPEP) was $8,429 per eligible person. This was 11 per cent higher than the established benchmark of $7,540 in SPM-06, and was to be used to assess cost control.

4.46 In June 2021, Defence agreed to implement a price variation mechanism (see case study 1) in addition to annual indexation. In May 2022, Defence estimated that the additional costs associated with the implementation of the additional price variation mechanism could result in an additional two per cent increase to the annual cost of the contract. Assuming an average annual cost of $500 million⁸⁹, the introduction of this mechanism has exposed Defence to the risk of an additional cost increase of approximately $10 million per annum.

4.47 The budget for the contract was increased by $230.2 million in March 2022. As of December 2022, contract expenditure had exceeded the budget by approximately four per cent and the total estimated cost of the contract had increased from $3.4 billion to $3.6 billion over the initial six-year contract period.

4.48 In February 2023, Defence advised the ANAO that:

the ADF health services contract is demand driven and costs of services is directly linked to the demand for services by Defence and Defence members. Realisation of cost savings is sensitive to changes in the demand signal for clinically necessary care, both as set out in the Defence Health Manual and also in response to unanticipated events such as the COVID-19 pandemic.

4.49 As discussed in paragraphs 1.18 to 1.19, in March 2022 Defence advised its internal Enterprise Business Committee that a surge in demand for health support had been experienced due to:

• the 2019–20 black summer bushfires, which saw an additional 3,500 hours of un-rostered health professional hours delivered in support of Operation Bushfire Assist; and
• the COVID-19 pandemic, which saw the release of contracted health professionals to assist vaccination delivery teams in aged care facilities and support Defence quarantine and testing activities (see paragraph 4.6).

4.50 In summary, Defence has assessed that the realisation of cost savings was impacted by unanticipated events. However, it is unable to demonstrate that it is likely to achieve the cost and service delivery efficiencies as advised to the Ministers for Defence and Finance in December 2018 (see paragraph 1.9), or the service delivery efficiencies identified in the contract objectives (see paragraph 1.12) and the benefits management plan, register and log (see paragraph 4.23). This is because key arrangements and initiatives to drive and monitor benefits realisation have not been fully implemented.

Appendix 1 Entity response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2022–23 Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. The below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• Since February 2023, Defence and the contractor have been working on developing a revised contract management and governance framework. As at April 2023, this program of work was ongoing, see paragraph 2.19.
• As at February 2023, work to develop and implement a contract management plan and other artefacts, including a responsible, accountable, consulted and informed (RACI) matrix, probity management, risk management and assurance plan had been developed, but not yet implemented, see paragraph 2.20.
• In August 2022, Defence wrote to the contractor to advise that a communication strategy was to be developed to ensure timely management of emerging risks and issues including performance shortfalls under the contract, see paragraph 2.14.
• As at December 2022, Defence had commenced work to conduct a benchmarking activity. In February 2023, Defence advised the ANAO that it was engaging with the contractor to finalise the selection of a benchmarking organisation, however the benchmarking activity had not yet commenced, see paragraph 2.51.
• In August 2022, Defence sought legal advice regarding performance payments that were made in error and has subsequently sought to recover the funds as a debt owed to the Commonwealth, see paragraph 3.17.
• In November 2021, Defence commissioned an external consultancy firm (EY) to undertake a review of the performance management framework to assess whether: the framework was achieving required outcomes; whether the metrics appropriately incentivise the contractor to achieve the targets; and examine the position put forward by the contractor that a number of measures were not achievable and were considered ‘unfair’. As at October 2022, work to update and revise the performance management framework had commenced. In February 2023 Defence advised the ANAO that a project to revise the PMF was underway, see paragraph 3.13.
• In December 2022, Defence identified that the residual risk rating of ‘moderate’ was sufficient to recommend that the contractor’s ICT system be accredited. The process to formally accredit the system was completed in April 2023, see Table 4.1.