Background

1. The Australian Antarctic Division (AAD) within the Department of the Environment and Energy currently leases the Aurora Australis to ferry infrastructure, equipment, supplies and personnel to and from the stations in Australia’s Antarctic Territory. As the Aurora Australis is approaching the end of its usable life, a procurement process for a new icebreaker was undertaken by the AAD.

2. In September 2012, Ministers decided that a non-binding Request for Proposals process should be undertaken for concept design and to determine the associated costing for a new multi-purpose icebreaker. The Request for Proposals closed in May 2013. One shortlisted entity withdrew from the procurement process in December 2013. A Request for Tender was issued to two entities in July 2014, but one entity withdrew before the tender period closed. Evaluation of the single tender received in March 2015 was completed in July 2015. Formal contract negotiations commenced in October 2015 and the contract was signed in April 2016.

3. The announced total cost of the replacement icebreaker project in April 2016 was $1.912 billion over the next 34 years. This included $529 million for the design and construction of the replacement vessel, which is expected to arrive in Australia in mid-2020.

Audit objective and criteria

4. The objective of the audit was to assess whether the procurement by the Department of the Environment and Energy of a replacement Antarctic icebreaking vessel will provide value with public resources. To form a conclusion against the audit objective, the ANAO assessed whether:

• the decision to adopt a Design, Build, Operate and Maintain (DBOM) approach to procuring a new multi-purpose icebreaker was well informed;
• the procurement process has been well managed, including by generating sufficient competitive tension across each stage; and
• the procurement outcome will provide a whole-of-life value for money solution at an acceptable level of cost and risk.

Conclusion

5. While the Department of the Environment and Energy has contracted for a vessel to meet its expected needs, the procurement process was largely non-competitive with an outcome that is higher than the cost benchmarks established by the department, and significantly greater than the current chartering costs. The department therefore cannot demonstrate that its procurement is providing value with public resources.

6. Due to perceived time constraints and an associated need to fast-track the project, industry soundings were not undertaken to inform the selection of the procurement approach. This meant that the department focused its consideration on the merits of three possible procurement methods without attention to industry preferences. Subsequently, only one tender was received after the other two shortlisted entities did not bid, citing concerns in their withdrawal letters with the DBOM approach adopted by the department.

7. Consistent with the Commonwealth Procurement Rules, the department implemented a two stage approach to the market with competitive pressure intended to drive the achievement of a value for money outcome. The tender evaluation approach was also appropriate. However, the tender process took much longer than had been planned. Reflecting the various delays, the replacement icebreaker is expected to be available to commence AAD operations in October 2020, more than eight years after first pass approval for the project was granted. This is twice as long as was originally envisaged. In addition, only one tender was received due to two of the three entities shortlisted to tender withdrawing from the process. Some substantial changes to the contract provisions were made during contract negotiations. Of particular note was that the initial operate and maintain contract term was reduced to ten years (with four options of five years each); delivery was switched from Hobart to Europe, and the delivery date was extended into the following year.

8. The department has contracted for the design, construction, operation and maintenance of a vessel that is expected to meet the research and resupply needs of the Australian Antarctic Program for the next three decades, for a level of risk it has assessed as acceptable. The price that is to be paid by the department under the contractual arrangements is higher than the cost benchmarks established by the department, and significantly greater than the current chartering costs.

Supporting findings

9. The department had a preference for either a Public Private Partnership or a Design, Build, Operate and Maintain approach. The department saw a number of benefits from these two approaches, including that they would cover the full life cycle of a vessel under the auspices of the one contract and that it enables the entity that will be operating and maintaining the ship to be involved in the ship design and build. The consultancy advice that was obtained by the department was limited to considering three options, being the two the department preferred as well as Design and Construct.

10. Prior to releasing the Request for Proposals, there was no consultation with the shipbuilding industry on its appetite for participating under a DBOM procurement approach. The Request for Proposals process did not generate strong interest by industry in the DBOM procurement approach.

11. The Request for Proposals (RFP) evaluation activities were appropriate. The shortlisting excluded proposals that were assessed to not adequately meet the RFP requirements, or not offer value for money.

12. The subsequent Request for Tender (RFT) process prevented industry from submitting any alternatives to the department’s preferred DBOM approach. This resulted in two of the three shortlisted respondents withdrawing from the procurement process without submitting a tender as they were not prepared to submit a DBOM tender.

13. The planned tender evaluation approach was appropriate. Tender evaluation activities were undertaken in accordance with the Tender Evaluation Plan. However, delays were a feature of the procurement process. As a result, the replacement icebreaker vessel is now projected to be available for service in 2020, some five years later than was envisaged when the project commenced.

14. Consistent with the Commonwealth Procurement Rules, value for money was intended to be achieved through a competitive procurement process. The department planned for a process that would involve competition at each of the two stages (an open Request for Proposals followed by a pre-qualified Request for Tender). Six entities responded to the RFP, of which three were shortlisted to tender. Only one tender was received.

15. The tendered solution was assessed to satisfactorily meet each of the non-price evaluation criteria. While it was assessed to not fully meet any of those criteria, the department’s expectation is that the icebreaking research and resupply capability of the accepted tender will meet the requirements of Australia’s Antarctic Program for the next three decades.

16. In relation to the ongoing risks associated with the project, the department has concluded that the contract does not contain any contingent liability risk of occurrence greater than five per cent, or which would most probably result in expenditure of greater than $30 million if the event were to occur.

17. The department’s initial cost estimates were three to four times higher than the cost of the vessel it had identified as being comparable to the size of the ship it needed.

18. The ship capital and annual operating costs reported by the department were significantly higher than the estimates provided by RFP respondents. The department’s RFP contract principles allowed for a cost variation of up to 20 per cent between the RFP and the RFT, but the ship capital cost increased by 133 per cent and annual operating costs (excluding fuel) increased by 189 per cent between the RFP and RFT submissions. However, the Tender Evaluation Report made no comparisons to the information provided in the tenderer’s RFP response.

19. Recognising that there was a risk that only one tender may be received, the department obtained advice on strategies that would assist it to demonstrate that value for money had been obtained. The strategies to address that a single tender was received were only partially implemented and, due to a range of factors, were not fully effective in assuring that value for money is being received. Specifically:

• costs were not benchmarked before the tender response was received;
• the benchmarking inputs contained inaccuracies and included various items of questionable relevance;
• the target of verifying a minimum of 80 per cent of the tendered price during the ‘open book’ review was not achieved; and
• the tenderer refused to provide its pricing model for forensic accounting verification.

20. The cost of the acquisition exceeds all benchmark prices prepared for the department during the evaluation process. This situation was not accurately reflected in the Tender Evaluation Report. Further, the Tender Evaluation Report did not compare the costs of the replacement vessel to current charter costs. On an annualised basis, the contracted price is more than three times the 2015–16 charter cost for the Aurora Australis.

21. Contract negotiations were unsuccessful in reducing the price compared with that tendered.

Summary of response

22. The Department of the Environment and Energy provided formal comments on the proposed audit report, which are included at Appendix 1. The department also provided a summary response, as set out below. The summary response from KPMG (one of the department’s advisers) is also provided below. Formal comments received from recipients of the proposed audit report other than the department are included at Appendix 2.

Department’s summary response

The Department acknowledges that there is always room for improvement in a complex matter like the procurement of a bespoke Antarctic vessel. But the Department does not consider that the process that it ran was largely non-competitive and that the outcome was excessively higher than its benchmarks.¹

The Department contends that its cost estimates, modelling and benchmarking undertaken before, during and at the conclusion of the process were robust and of a high standard, especially those prepared by BMT and KPMG, and supported its decision making. The Department remains of the view that the cost investigation² and benchmarking processes³ that it put in place after the reduction to a single tenderer, a process supported by central agencies, allowed it to continue with the procurement and to achieve a competitive outcome.

In terms of demonstrating that the procurement is providing value with public resources, the Department would point to the comprehensive scrutiny and consideration given to the procurement and the cost modelling by Government agencies (including central agencies), by external, independent professional advisors on numerous occasions since 2011, and ultimately by the decision of the Cabinet to proceed with the acquisition of the icebreaker. In the Department’s view, this provides strong assurance around the procurement outcomes.

The Department considers that the procurement of the replacement Antarctic vessel has achieved a world-class scientific and logistical capability for the Government’s Antarctic program at an acceptable level of cost and risk.

ANAO comments on department’s summary response

1. See paragraphs 2.45 to 2.56 concerning competition during the Request for Tender stage and paragraphs 4.18 to 4.19 for a comparison of the contract prices to the cost benchmarks prepared for the department.

2. See paragraphs 4.20 to 4.25.

3. See paragraphs 4.14 to 4.19.

KPMG summary response

Whilst the loss of competitive tension late in the procurement process was regrettable, KPMG believes that the commercial advice provided to the Department (post-Request for Proposals through to contract finalisation) was reasonable and in-line with the intent of Commonwealth Procurement Rules. Furthermore, KPMG believes that value-for-money was achievable and was supported through the comprehensive benchmarking exercise, the open-book pricing review and the extensive negotiation support provided to the Australian Antarctic Division.

Introduction

1.1 The Australian Antarctic Division (AAD) within the Department of the Environment and Energy currently leases the Aurora Australis to ferry infrastructure, equipment, supplies and personnel to and from the stations in Australia’s Antarctic Territory. As the Aurora Australis is approaching the end of its usable life, a procurement process for a new icebreaker was undertaken by the department. The announced total cost of the replacement icebreaker project in April 2016 was $1.912 billion over the next 34 years, including some $529 million for the design and construction of the replacement vessel. The replacement vessel is expected to arrive in Australia in mid-2020.

1.2 The project is a key part of the Antarctic Modernisation Program, which commenced in 2011.

1.3 The AAD has several primary obligations. These include: administering the Australian Antarctic Territory and the Territory of Heard Island and McDonald Islands, and the conduct of Antarctic scientific research. The AAD is also responsible for the coordination of Australia’s scientific activities under the Australian Antarctic Program (through the Australian Science Strategic Plan) and the provision and management of logistics for the program. The AAD’s total 2015–16 budget was some $118 million.

Procurement approach and status

1.4 In September 2012, Ministers decided that a non-binding Request for Proposals process should be undertaken for concept design and to determine the associated costing for a new multi-purpose icebreaker. The Request for Proposals was released in January 2013¹, and closed in May 2013. Seven proposals were received from six entities, of which three were shortlisted to proceed to the Request for Tender stage. No respondents proposed an Australian build.

1.5 One shortlisted entity withdrew from the procurement process in December 2013, and a Request for Tender was issued to two entities in July 2014. One of the remaining two shortlisted entities withdrew before the tender period closed.

1.6 In March 2015, a single tender was received. Evaluation of the tender was completed in July 2015. Formal contract negotiations commenced in October 2015 and the contract was signed in April 2016.

Governance arrangements

1.7 To provide strategic guidance on the development of aviation and shipping options to modernise the Australian Antarctic Program, in March 2012 an interdepartmental committee chaired by the department was established. Members were provided by the Departments of: Prime Minister and Cabinet; Foreign Affairs and Trade; Defence; and Finance and Deregulation.

1.8 In addition, a steering committee was established by the department in November 2013 to oversee the project and advise the project sponsor (the chair was the AAD Director) regarding project matters. Its membership comprised the Program Sponsor (a departmental Deputy Secretary); AAD General Manager, Support Centre; AAD Chief Scientist; AAD General Manager, Strategies Branch; the AAD program and project managers; and departmental and external representatives² appointed by the Sponsor. The AAD was also required to provide monthly major project reports to the departmental executive (see paragraph 3.6).

Audit approach

1.9 The objective of the audit was to assess whether the procurement by the Department of the Environment and Energy of a replacement Antarctic icebreaking vessel will provide value with public resources. To form a conclusion against the audit objective, the following high level criteria were adopted:

• Was the decision to adopt a design, build, operate and maintain approach to procuring a new multi-purpose icebreaker well informed?
• Has the procurement process been well managed, including by generating sufficient competitive tension across each stage?
• Will the procurement outcome provide a whole-of-life value for money solution at an acceptable level of cost and risk?

1.10 The scope of the audit included all aspects of the procurement process, starting from the original decision to commence the replacement of the Aurora Australis, through to the evaluation of the tender that was received and the signing of the contact with DMS Maritime Pty Ltd (now Serco Defence) on 28 April 2016. The procurement of other associated infrastructure was not audited.

1.11 The audit was conducted in accordance with ANAO auditing standards at a cost to the ANAO of approximately $325 771.

1.12 The team members for this audit were David Spedding and Brian Boyd.

Was there a comprehensive and objective consideration of options?

2.1 In November 2016, the department advised the ANAO, as background, that:

The Department has been a charterer of ships for most of its history. Unlike the Department of Defence, the Department of Immigration and Border Protection and the CSIRO, the Department does not own, operate or maintain ships. In this context, the Department’s approach from the outset was to identify the procurement model that would best serve its future interests and Australia’s Antarctic program. In determining this, consultant advice was sought to provide context around possible options.

Initial preference for PPP

2.2 From early in the life of the project, the department expressed that a Public Private Partnership (PPP) was its preferred procurement model. By December 2011, the department had identified PPP, DBOM and ‘government funded and operated, industry built and maintained’ as the three options to consider in preparing the first and second pass approval submissions.

2.3 How and why the department chose these three options, and the rationale for not examining the merits of other approaches³, was not documented at that time.

2.4 BMT Design & Technology Pty Ltd was engaged by the department in July 2011 via a pre-qualified tender request under the Defence Material Organisation Support Services Panel to develop icebreaker ship procurement documentation.⁴ In April 2012, the department directed BMT to ‘cast their PPP justification in light of the guidelines’. To assist, BMT obtained a proposal from PwC⁵ in May 2012.

2.5 Due to a subsequently identified conflict of interest, PwC withdrew and was replaced by Ernst & Young. In early June 2012, Ernst & Young provided a draft report that ranked PPP as the highest preference.⁶ The Minister was advised in late-June 2012 that the department proposed recommending a PPP.

Change to preference for DBOM

2.6 By late-May 2012, the department had changed its preferred procurement method to DBOM when it identified that a PPP posed unacceptably high schedule risks. This was due to the expectation that assembling an appropriate consortium would cause delays of 12-18 months. In February 2017, the department advised the ANAO that:

The chosen procurement model provided the Department with an opportunity to cover the full life cycle of a vessel under the auspices of the one contract. This was determined to be an effective means of managing risk where the Department has never been a ship owner, operator, designer or builder. A further rationale for this approach is that it enables the entity that will be operating and maintaining the ship to be involved in the ship design and build, enabling it to better manage the maintenance and operational risk, to the benefit of the Commonwealth. This approach was informed by the lessons learned from contracting and managing the current vessel, the Aurora Australis.

2.7 An inter-departmental committee Project Steering Group meeting held on 7 June 2012 was advised that the consultants were preparing a procurement strategy with the preferred approach being DBOM. As it transpired, the Ernst & Young report finalised in August 2012 continued to rank the PPP model as highest preference. The report stated that it was ‘limited to a preliminary assessment’ of the three procurement options short-listed by the department.⁷

KPMG procurement options analysis

2.8 KPMG was engaged by the department in October 2013 to provide business support services to the project.⁸ Its proposal recognised that the RFP process (involving a DBOM approach) had already been conducted and that a DBOM model had already been selected for the procurement. KPMG’s proposal stated that its services include developing and finalising a procurement strategy ‘based on the DBOM procurement approach’.

2.9 In December 2013, amendments were requested by the department to the draft strategy prepared by KPMG to support the recommendation to continue with DBOM, including:

• stating that a RFT process on design and construct only ‘is not sustainable and would adversely impact on industry’;
• adding that there are ‘compliance concerns’ over a ‘change in procurement process midstream’ notwithstanding that the RFP had specifically (consistent with the September 2012 decision of Ministers) provided capacity to revert to a design and construct approach;
• stating that a well-managed DBOM process ‘will result in a value for money outcome’; and
• referring to RFP responses to demonstrate the market has guided the appropriateness of DBOM.

2.10 In February 2017, KPMG advised the ANAO that:

The changes made to the initial draft of KPMG’s Procurement Strategy of December 2013 were to provide clarification and were consistent with KPMG’s views.

2.11 The final procurement strategy recommended that DBOM be used. It further recommended that the option for design and construct no longer be considered as an alternative contracting methodology. Consistent with this recommendation, the RFT issued in July 2014 allowed for a DBOM only and precluded submission of a design and construct option.

2.12 In November 2016, the department acknowledged to the ANAO that the RFT ‘did not allow alternative tenders. This took into account recommendations from KPMG and consultation with the Probity Advisor’.

Analysis of procurement options assessments

2.13 The scoring methodology that underpinned the procurement options selections was sensitive to small changes in the ‘importance’ attributed to the eight ‘procurement objectives’ considered in the model. The department was aware of this when it checked the Ernst & Young scores in August 2012, noting that a change in the importance rating on just one criteria (‘budget certainty’) changes the overall ranking whereby DBOM becomes the preferred option, but closely followed by the equally scored PPP and D&C options.

2.14 KPMG adopted the same scoring methodology in December 2013. Of particular note was the value for money score of zero awarded to the design and construct option by KPMG. As illustrated in Appendix 3, this was the only score of zero awarded to any option against any criterion by Ernst & Young, the department or KPMG. The score of zero indicated that a design and construct approach would be ‘ineffective’ in providing value for money. In February 2017, KPMG commented to the ANAO that:

• it ‘continues to stand by the view expressed in its Procurement Strategy of December 2013, namely that ’the lack of focus on operations, maintenance and life cycle in the design and construct model as seriously impairing the likely value for money of this option’ and hence that the design and construct model would be ineffective in providing value for money’;
• empirical research ‘indicates that design and construct and similar contractual forms generally result in both cost and time overruns’⁹; and
• even if it had used the same score for value for money as Ernst & Young, the design and construct option would have remained in third place.

2.15 The Commonwealth Procurement Rules outline that value for money in procurement requires competition to be encouraged. In this respect, also of note was that KPMG scored DBOM and D&C as having an equally high level of market interest. This was not supported by the results of the RFP process. In particular, respondents to the RFP were required to provide details of up to five contracts completed within the last five years of broadly comparable contracts in relation to the design and construction of a polar icebreaking vessel or comparable vessel, and operation and maintenance of a multipurpose vessel. Of the three shortlisted respondents, DMS included one DBOM example. This involved 15 vessels (such as harbour tugs and barges) the Australian Navy is acquiring under a DBOM approach, with a value ranging between $2.1 million and $69.9 million per vessel.

2.16 The RFT process that followed confirmed that there was little market interest in DBOM. In particular, two of the three shortlisted entities did not tender, each of whom had design partners with considerable icebreaker experience, citing concerns in their withdrawal letters with the department’s DBOM approach. Each indicated that they would have preferred a different procurement approach (a managing contractor model, which was not one of the three options considered by the department).

Was there adequate engagement with industry in deciding upon the procurement approach?

2.17 The department recognised in October 2011 that it would need to complete a procurement options analysis. This was consistent with May 2011 advice from a consultant (Mortimer)¹⁰ to the department that:

Many choices are available for both the acquisition and operation of a new icebreaker for the AAD. These options should be fully explored in conjunction with industry and other authorities to enable an informed judgement to be made on the preferred means of procurement and operation of the vessel. [ANAO emphasis added]

2.18 Other consultants (BMT, PwC and Ernst & Young) had all similarly emphasised to the department the importance of conducting ‘market soundings’ with industry regarding selection of the preferred procurement option. Obtaining a sound understanding of relevant industry markets before making procurement strategy decisions is also advocated in the PPP Guidelines.

2.19 On 9 July 2012, the department decided that, as a PPP was no longer the preferred option and had been replaced with DBOM, ‘a market sounding would not be required’. Earlier on the same day the department received advice from a legal services panel member that:

For such large-scale procurements it would usually be a Request for Expression of Interest process to see who is in the market and interested in participating, and from there create a group of possible providers to issue either an RFP or RFT.

2.20 In late July 2012, the department drafted a Request for Information (RFI) to seek information from potential icebreaker designers, builders and operators. The stated purpose of the RFI included the following:

The AAD currently considers that the preferred procurement option for a potential new icebreaker to support the Australian Antarctic Program is a Design, Build, Operate and Maintain (DBOM).

The AAD seeks information as to whether industry is able to support the DBOM model, and also what other models industry is able to support.

2.21 The RFI was not released to industry. The reasons for not proceeding with the RFI were not recorded. In November 2016, the department advised the ANAO that:

This matter was discussed and decided on but no record is available to document the decision. The RFI was overtaken by the non-binding RFP process.

2.22 In mid-August 2012, the AAD Director was advised by AAD staff that a market sounding process was likely to take several months and that this would adversely impact on the planned deliverables to such an extent that it was not viable. He was also advised that:

DBOM is an established procurement option and there are established service providers in the market. A recent similar ship procurement sought interest in a DBOM arrangement and received responses.

2.23 CSIRO had not adopted a DBOM approach but had used a design and construct contract for the Investigator. This information was not included in the advice to the Director. Further, the department has not been able to provide evidence to the ANAO that DBOM is a commonly used and supported procurement option in the shipping industry. The ANAO’s research did not reveal that DBOM is a procurement approach commonly used in the shipbuilding industry.¹¹

2.24 The AAD Director accepted the recommendation that DBOM be selected as the preferred procurement option. Importantly, he also accepted that it would be prudent to include in any RFP/RFT the capacity to revert to a design and construct arrangement under certain specified circumstances. These recommendations were also subsequently accepted by Government during the first pass approval.¹²

2.25 In November 2016, the department commented to the ANAO that:

There are a range of factors (including perceived time pressure due to P&O signalling in early 2012 the unavailability of the RSV Aurora Australis beyond 2015, the BMT review of the life extension survey, and P&O signalling in July 2012 the retirement of RSV Aurora Australis in 2017), that introduced an urgency which influenced decision making around a market sounding process.

2.26 Notwithstanding these factors, consultancy advice to the department had emphasised the importance of early engagement with industry in deciding upon the procurement approach. Obtaining a sound understanding of relevant industry markets before making procurement strategy decisions is also advocated in the PPP Guidelines. Further, industry engagement should have been recognised as more, rather than less, important given the department had a preference to adopt a procurement approach that is not demonstrably common in the shipbuilding industry.

What did the Request for Proposals process reveal about industry’s appetite for a DBOM approach?

2.27 In September 2012, the Government decided that the RFP issued by the department retain the capacity to revert to a design and construct arrangement, with Australian Government funding during construction, and operation and maintenance of the ship via a long term contract.

2.28 The department was aware that under a DBOM it may not get a response involving an Australian build¹³ ‘if the current players in the Aus(tralian) construction arena cannot find an ‘operate’ partner’. The department was also concerned that unless the RFP mandated that a conforming DBOM was required, it may end up with a significant amount of wide-ranging alternative solutions that it would then need to evaluate. The RFP that was subsequently issued provided that a conforming DBOM proposal must be submitted before any alternative proposal (including a design and construct) would be considered. The Minister was not advised of this requirement.

2.29 It was therefore not surprising that no design and construct proposals, and no Australian-build proposals, were elicited by the RFP process.¹⁴

2.30 In August 2013, the Minister was advised that:

The market has clearly demonstrated the capacity to deliver DBOM. Responses to the RFP have provided broad support for a DBOM procurement model, although a number of issues were raised by one respondent in relation to the scope of the risk transfer proposed in the Contract Principles and the potential impact on contract pricing. There has been no information provided through the RFP process that would warrant consideration of a different procurement approach. In response to the RFP, the market has clearly demonstrated a capacity to deliver DBOM. The three submissions shortlisted are from strong, viable candidates and the Evaluation Board considers that this will produce strong competition in any subsequent RFT based on a DBOM procurement model.

P&O response to the RFP

2.31 P&O was the ‘one respondent’ referred to in the advice to the Minister. The advice to the Minister understated P&O’s concerns with the RFP and the proposed contract principles.

2.32 P&O had noted that the RFP and the proposed contract principles had some requirements that were beyond the normal practices of the commercial maritime industry. Inability to set a fixed price contract was flagged as a risk, and amongst other matters P&O proposed:

• adopting various risk reduction strategies to reduce its compliance uncertainty risks;
• increasing its uncertainty margins and flow-on costs to the department;
• not to provide parent company guarantees;
• not to accept unlimited liability, which it said would be difficult and costly to insure and that these costs would ultimately be borne by the department; and
• that in line with normal ship building and ship management contracts, it would not be liable to the department for consequential loss.

2.33 In its RFP cover letter, P&O stated that it was:

open to alternative solutions for the provision of the new AAD icebreaker, including an ownership arrangement. Such arrangement could be based upon a similar contract model to the existing charter of the Aurora Australis which has been proven in over 20-years successful service.

Babcock response to the RFP and subsequent withdrawal from the proposed RFT

2.34 Babcock was also one of the shortlisted respondents. In contrast to the department’s advice to the Minister that no information was provided that would warrant consideration of a different approach, and that the three shortlisted respondents would provide strong competition under a DBOM arrangement, Babcock’s response to the RFP stated that:

We do not believe that DBOM represents best value for money for the client. In our experience, where we have seen DBOM contracts, the allocation of all risks to the contractor is ultimately reflected in the price. Following the initial contracted period, the client has re-tendered the contract, due to the excessive profit being made by the contractor because the perceived DBOM risks fail to materialise. We recommend that the AAD contracts for the Design and Build separately to the Operate and Maintain.

We are not able to provide a fully compliant DBOM proposal. Despite our best efforts, we have not been able to secure the commitment of a designer, builder, operator or maintainer to enter into the DBOM prime contract, nor to form a DBOM consortium or joint venture, principally due to the liability and risk allocation outlined in the RFP. We have proposed alternative contract conditions to guide the AAD for any future RFT around liability, risk allocation and Intellectual Property.

Our conclusion is that the DBOM risk allocation does not represent value for money for the AAD. The risk allocation is the driver of the premium that AAD would need to consider for the acquisition of the icebreaker through the DBOM model. We have provided an indicative price for DBOM to illustrate the effect. In our experience, we have seen a number of government vessels procured in this manner; in almost all cases, the customer has re-tendered the O&M contract at the earliest opportunity due to the high level of profit being made by the DBOM contractor, following the failure of risks to materialise as priced.¹⁵

2.35 Babcock advised the department in early December 2013 that it was withdrawing from the icebreaker procurement process. Specifically, Babcock wrote:

Many thanks for your response to our clarification question. We understand that following a competitive Request For Proposal pre-selection of bidders that it is not acceptable to the AAD for us to introduce a 3rd party to prime our solution; the opportunity for the involvement of such a 3rd party having passed with the closure of the RFP process.

As we indicated in our non-compliant proposal,¹⁶ we would not be able to submit a response to a subsequent RFT for a contract with a single legal entity for DBOM, due to the associated risk profile. We also indicated that we would continue to explore ways in which we could respond; we have now exhausted those discussions. These discussions have reinforced our conclusion that DBOM does not represent value for money to the AAD, attracting a 26-29% premium above a separate DB & OM contracting model.

Regrettably, I am therefore writing to inform you that as of the date of this letter we are withdrawing from the AAD’s Ice Breaking Shipping Capability procurement process; we will not be responding to any future tender.

2.36 The department did not reassess the ongoing viability of DBOM in light of Babcock’s withdrawal.

2.37 In numerous subsequent briefings over the next several months, the Minister was not informed of Babcock’s withdrawal notwithstanding the implications of this withdrawal on the competitiveness of the procurement process. The withdrawal was not communicated to the Minister until 24 July 2015, following a series of media articles criticising the icebreaker procurement. That brief did not mention Babcock’s concerns with the DBOM procurement model. Rather, the Minister was advised that:

This withdrawal of Babcock was primarily a result of their desire to source and nominate an entirely new entity to replace them as the Prime Contractor of their consortium for the subsequent Tender phase. After receiving the necessary legal and probity advice, the Department concluded that the proposed change by Babcock would not be acceptable in accordance with the Commonwealth Procurement Rules.

Other responses to the Request for Proposals

2.38 Respondent C acknowledged the department’s preference for a single DBOM contract and stated ‘that this can be facilitated’. However, it proposed changes under which ‘the cost benefits to AAD are likely to be substantial’. It proposed two separate contracts, one for D&B and the other for O&M, which it considered ‘should be the preferred position of the AAD’ and ‘lead to more transparency and competitive costing to the AAD’. Respondent C wanted to further discuss these options with the department ‘in an effort … to identify the most cost effective utilisation of public funds’. The department did not convey these views to the Minister.

2.39 Respondent D’s response indicated that it did not accept many of the proposed terms and conditions, which it considered were unreasonable, not relevant or required discussion at the next phase.

2.40 Respondent E also indicated it could not comply or would only partially comply with a number of conditions.

2.41 Further, although KPMG concluded that DMS’ acceptance of the DBOM model was unqualified, the response from DMS indicated that it would not accept unlimited liability and did not accept that the price for operational services would be fixed during the contract term. Both KPMG and DMS (now Serco Defence) commented on this issue, as follows:

• KPMG: ‘In its response to the RFP on the principles of any future contract, DMS either indicated its compliance or it acknowledged every principle apart from these two, and in both cases its response was reasonable in the circumstances and consistent with the overall DBOM model.’
• Serco Defence: ‘Given the scope and timeframe of the contract, it was advised that, until operational testing it was not possible to accurately cost the services to allow an appropriate fixed price.’

2.42 In discussing sources of risk relating to the procurement strategy, the department’s Risk Assessment and Treatment Plan for the project acknowledged that ‘The Department did not do any significant market sounding prior to the RFP’. In November 2016, the department also advised the ANAO that:

The Department acknowledges that conducting market soundings prior to RFP may have further informed the procurement approach; further increased the interest; and/ or further increased the quality of some of the RFP responses submitted by the six Australian entities. However, this outcome was not inevitable.

Was the evaluation and shortlisting of responses to the Request for Proposals appropriate?

2.43 Six respondents submitted proposals by the closing date and one of the respondents also submitted an alternative proposal. All proposals were considered compliant for evaluation. In a comparative assessment against the designated evaluation criteria, only three of the proposals were assessed by the department to represent value for money. Each of the three proposals was ranked equally as ‘the differing approach by each of these submissions made any final direct comparison challenging’.

2.44 The three proposals that were not shortlisted were assessed to each have low technical worth¹⁷; with one also assessed as high risk and the remaining two as severe risk. The alternative proposal did not demonstrate that it was more beneficial to the department; did not have equivalent or higher technical worth; had a higher price (approximately 24 per cent); and was assessed as not offering greater value for money compared to the other proposals recommended for shortlisting.

Did the Request for Tender process result in a sufficient number of competing proposals being received?

2.45 The RFT prevented shortlisted entities from submitting an alternative tender that altered the DBOM procurement model proposed in the draft contract and statement of work.

2.46 When seeking approval to release the RFT, the department did not advise Ministers that only DBOM proposals would be accepted.

Industry feedback on DBOM following RFT

2.47 In November and December 2014, P&O advised the department that it was not in a position to accept the full design and construction risk as outlined in the RFT Draft Contract. In essence P&O advised that the multiplier effect on the duplication of risk under the DBOM model would result in an increase in the design and build price ‘in the order of $50 million’ compared to a design and build under a managing contractor project delivery model.

2.48 P&O formally withdrew from the RFT process on 23 December 2014, stating that:

As we have communicated previously we believe there are substantial flaws in the procurement philosophy that will not allow P&O Maritime to deliver a competitive and value for money proposal. We have sought to mitigate or relax some of the main areas of concern around the DBOM model and appreciate the constructive position of the AAD but at this juncture it is clear that P&O Maritime is unable to continue this engagement.

2.49 The Minister was provided with a copy of the P&O withdrawal letter. He was advised that ‘P&O’s reason for withdrawal is based on its assessment that it is unable to deliver a competitive offer based on the Department’s preferred procurement model of Design, Build Operate and Maintain (DBOM)’. The Minister was not advised of P&O’s view that the DBOM approach would result in an increased price in the order of $50 million.

2.50 The department advised DMS of P&O’s withdrawal within hours of receiving formal notification. It also sought to prevent P&O disclosing that it had withdrawn from the tender process without departmental approval, an approach that P&O subsequently successfully challenged with the department. In mid-January 2015, P&O commented publicly that the ‘unusual’ procurement model is ‘better suited to large infrastructure like highways’, and that it ‘doesn’t lead to value for money, it just adds up costs’.

2.51 Following the withdrawal of P&O, a report was prepared by KPMG that supported the department’s ‘firm opinion’ that continuing the current process with only a single bidder would still deliver a value for money outcome (discussed further at paragraph 4.13). The department did not ask KPMG to consider whether other procurement options may provide better value for money.

2.52 Further, KPMG stated that ‘As one tenderer appears to understand and be willing to bid on the DBOM model as outlined by the Commonwealth, this approach still has validity and should not be reviewed in light of one tenderer withdrawing’.

Other opportunities to reassess the procurement approach

2.53 The department did not assess the ongoing validity of the DBOM approach when the sole tender was received in early March 2015 (revealing the tendered price, which was the first opportunity to assess whether the tender did or did not represent value for money), and subsequently when the tender evaluation was completed in July 2015. There was also no reassessment when subsequent contract negotiations became protracted and the RFT validity period expired.

2.54 In response to a recommendation from a contracted adviser to the AAD Director¹⁸, a consultant’s paper on design and construct options was prepared in January 2016. However, this paper was not submitted to the Contract Negotiation Review Board.

Post-tender evaluation advice to Minister

2.55 In late July 2015, the department briefed the Minister that it had undertaken ‘an extensive examination of options’ and ‘extensive consultation’ with industry which resulted in the decision to adopt the DBOM model. However, evidence of consultation with the shipping industry is scant.

2.56 This briefing led the Minister to a concern that the current process could not be abandoned because ‘under the terms set out for the process by the ALP government, significant compensation would have been due.’ The Minister requested that ‘more details be provided’. However, the department did not respond to the Minister’s request with a written briefing.

Was the evaluation approach appropriate?

3.1 Governance arrangements for tender evaluation were appropriate for the scale and complexity of the procurement. This included a hierarchy that involved various evaluation working groups whose work was then considered by a Tender Evaluation Team oversighted by a Tender Evaluation Board. The Board was chaired by the AAD Chief Scientist, with membership by the AAD’s General Manager, Operations and Support Branch; the department’s Assistant Secretary, Business and Ministerial Services Branch; CSIRO’s Director, Strategy and Development; and the Director General, Maritime Acquisition, Department of Defence.

3.2 A Tender Evaluation Plan (TEP) was developed and approved in July 2014, prior to the RFT being issued. The TEP was amended in December 2014, following review by the department’s probity adviser. It was further amended in February 2015 to reflect that there would only be one tenderer.

3.3 The TEP provided appropriate guidance to officers and advisers involved in the evaluation of tenders. It covered the receipt and registration of tenders, the conduct of the various phases of evaluation work, the evaluation timetable and the approach to rating tenders.

3.4 The tender was evaluated against the five criteria (which comprised 23 factors to be taken into account across the criteria¹⁹) published in the RFT, and in accordance with the process set out in the TEP. The records of the tender evaluation process outlined the basis for the assessment conclusions and ratings awarded against each of the evaluation criteria and sub-criteria.

Was the procurement delivered on-time?

3.5 In mid-2010 the department advised its Minister that a replacement icebreaker would be in operation by 2015, in time for the then expected withdrawal from service of the Aurora Australis at the end of its serviceable life.

3.6 As shown in Table 3.1, there have been significant delays in the delivery of the project.²⁰ Despite the ongoing delays to the project, all of the department’s monthly major project reports from October 2013 through to May 2016 (the most recent report available at the time of ANAO audit fieldwork) reported that the project was ‘on track’ with a ‘green light’ status.²¹

3.7 In April 2014, the Minister advised his colleagues that ‘The scale, complexity and cost of this (replacement icebreaker procurement) activity was significantly underestimated by my department.’ Around that time, the department also advised the Minister of a two year extension of the project timeframe, acknowledging that:

It is now recognised that the timing relating to the activities intended to be completed in 2012-13 was ambitious and compressed. The depth and scale of the tasks relating to the development of the RFT [subsequently released a further three months later on 25 July 2014] has proven far more complex than first envisaged.

3.8 The RFP outlined that in-service operation of the new Icebreaker was intended to commence on 1 October 2017 from Hobart, Australia. The Request for Tender outlined that final acceptance was required in October 2019.

3.9 The contract signed on 28 April 2016 (four months later than had been planned when the RFT was issued) specified that final acceptance would now occur in the Netherlands rather than Hobart (with the contractor to then deliver the vessel to Hobart ‘as soon as reasonably practical’). Final acceptance was to occur 48 months after the contract commencement date, with the ship being available to commence AAD operations at the start of the season in October 2020.

Table 3.1: Summary of reported project timelines (as revised periodically during project implementation)

Source: ANAO analysis of departmental records.

RFP and RFT extensions

3.10 The original periods set for industry responses to both the RFP and the RFT were extended by the department. They were increased by 29 and 73²² days respectively following requests by industry during the bid periods. These extensions were granted for valid reasons (primarily difficulties industry was experiencing with adapting to the unaccustomed DBOM model) and did not materially contribute to the overall delays experienced in the project.

Single bidder impact on evaluation approach

3.11 The changed approach adopted for the evaluation of the sole tender introduced some additional processes and requirements, particularly in relation to analysis of the tendered price. The initial schedule was six weeks for this activity. The changes added eight weeks to the timeline.

Contract negotiation extensions

3.12 In July 2015, the contract negotiation phase was forecast to conclude by December 2015. The tender submitted by DMS had a 12 month validity period, expiring on 2 March 2016. On this date the department advised DMS that its status as preferred tendered had been extended until 24 March 2016. No further extensions were issued covering the period from 25 March 2016, notwithstanding that the contract was not signed until 28 April 2016.

Price impact of delays

3.13 In March 2016, the department was advised by DMS that the price incorporated into its tender for its ship building subcontractor (Damen) expired in late-November 2015.

3.14 By mid-April 2016, the department was advised that the Damen final price offer had increased by Euro 16.434 million ($24 million) from the original price of Euro 260.0 million, and that this offer would expire on 1 May 2016. In February 2017, Serco Defence advised the ANAO that:

the increase [of Euro 16.434 million] was attributed to the final negotiated position following adjustments in the Statement of Work. These adjustments were primarily within the Function and Performance Specification which added additional complexity to the vessel.²³

Was sufficient competitive tension generated and maintained?

3.15 Achieving value for money is the core rule of the Commonwealth Procurement Rules. These rules outline that achieving value for money requires the consideration of the financial and non-financial costs and benefits associated with a procurement. The rules also outline that value for money in procurement requires competition to be encouraged. The department planned for competition to occur through a two stage process involving:

• a non-binding Request for Proposals (RFP) process that commenced in January 2013. The RFP documentation informed interested parties that the RFP process would be used to develop a shortlist of respondents who would, subject to Government funding and approvals, be invited to respond to a Request for Tender; and
• a pre-qualified Request for Tender (RFT) that was issued in July 2014. The RFT documentation outlined that the aim of the RFT was to select a preferred tenderer from the shortlisted RFP respondents. To achieve this, the department planned an evaluation process that would allow it to assess and compare the merits of competing tenders and then rank the tenders based on specified value for money considerations so as to identify the preferred tenderer to progress to contract negotiation.

3.16 At the time the procurement process was being conducted there had been a decline in international shipbuilding activity. This should have assisted the department to obtain a healthy level of competition in the procurement process.

3.17 The RFP effectively required that consortiums be created. The department was aware that the relatively short timeframes it set would also impact on the ability of industry to arrange consortiums and to reserve required hull space in shipyards.

3.18 Six entities responded to the RFP, and three were shortlisted to tender. One withdrew before the RFT was issued (see paragraph 2.34).

3.19 Inviting only two firms to submit a tender generated less competition than would have been in place had the planned requests to three entities proceeded. The May 2014 Gateway review (Gate 2) stated that it was ‘Imperative that both prospective tenderers remain engaged to ensure competitiveness’. It also warned the department that one tenderer may withdraw, but the department considered that this risk was ‘unlikely’.

3.20 Given the timing, in that bids were not yet due to be received, all competitive tension was effectively removed when DMS was advised of P&O’s withdrawal.

3.21 Negotiations were led by the Modernisation Programme Manager (an AAD Branch Head). In February 2017, KPMG advised the ANAO that negotiations were supported by experienced negotiators from both KPMG and the department’s legal advisors.

3.22 The Negotiation Team²⁴ was oversighted by the Contract Negotiation Review Board.²⁵ The department also did not seek information from the Department of Defence on possible suitable candidates to provide expert assistance with negotiation of the contract.²⁶

3.23 By the time the contract was signed, substantial changes had been made in many of its provisions, including reducing the initial operate and maintain contract term to ten years (with four options of five years each)²⁷; switching delivery from Hobart to Europe, and extending the delivery date into the following year. The department had also made concessions that significantly reduced the tenderer’s liabilities and effectively passed various risks back to the Commonwealth. Further, the department did not achieve its targeted level of contract negotiation savings (see paragraph 4.33). In February 2017, KPMG advised the ANAO that:

In KPMG’s opinion, the signed contract was the result of a number of compromises between the draft contract in the RFT and DMS’s responses to that draft in its proposal, and generally was much closer to the Department’s proposed position that to that of DMS.

Is the accepted solution expected to meet the specified requirements?

4.1 A five point rating scale (of A to E) was adopted for the four non-price criteria. A rating of ‘D’ represented the lowest level of acceptable achievement of the requirements specified in the RFT.²⁸ The tender had been assessed to have met the four non-price evaluation criteria. It was also assessed to have not fully met any of those criteria.²⁹ Rather, it was rated ‘B’ or ‘C’ against each of the criteria. Specifically, it had been assessed as having:

• ‘good or sound achievement’ (a rating of ‘B’) against the following two criteria:
  • nature of the tendered design and build solution and the extent to which it meets the statement of work; and
  • demonstrated capability and capacity of the tenderer to deliver the proposed DBOM solution; and
• ‘minimal but still acceptable achievement’ against the following two criteria:
  • nature of the tendered operation and maintenance solution and extent to which it meets the statement of work; and
  • extent to which the tenderer conforms to the commercial and legal requirements set out in the RFT.

4.2 At the conclusion of the procurement process, the department advised the Minister that the tendered design solution would fully meet Australia’s Antarctic shipping requirements until 2050. The tendered solution was for a 156 metre icebreaker with full load displacement of 23 800 tonnes. Other key features include:

• dimensions suitable for access to specified berths and terminals, in addition to the ability to access confined waters including at Mawson Station;
• compliance with the specified seakeeping requirements for safe operation in the Southern Ocean;
• a propulsion and manoeuvring system that was based on the AAD’s requirements for dynamic positioning, low speed loitering for science operations, economical cruising and operation in icebreaking conditions with the operational speeds balanced against requirements for a low acoustic noise vessel;
• icebreaking capacity that included a guaranteed level of performance at continuous speed of three knots in ice with thickness of 1.65 metres; and
• accommodation for up to 117 special personnel plus up to 32 crew.

4.3 The Contract Negotiation Report prepared at the conclusion of contract negotiations outlined that the ‘negotiation process has resulted in DMS and Damen being able to address or eliminate a range of complexities; and to progress the Vessel design to a high level of maturity, with an enhanced science platform.’ Nevertheless, it was also recognised that:

• the contractor and key subcontractor are relatively inexperienced in the design, construction and delivery of a multi-purpose research supply icebreaker (which was seen as being mitigated by factors including the demonstrated performance of the subcontractor in delivering a wide range of one-off bespoke vessel designs, including ice capable and scientific research vessels);
• the overall vessel design is bespoke, includes a high degree of complexity in terms of the integration of a large number of individual capabilities (in particular various scientific research capabilities) into a single platform design, and was considered to be at the upper end of the spectrum in terms of vessel size taking into account the defined operating profile; and
• throughout the contract negotiation process a reasonable portion of the originally allocated project contingency was used to address shortcomings in the tendered solution, a situation which places additional pressure on the remaining contingency.

Are the residual risks at an acceptable level?

4.4 At the conclusion of contract negotiations, the department identified approximately 20 risks for ongoing monitoring and treatment. The Secretary was advised that ‘the Negotiation Team has satisfied itself that the project risks are manageable and are not unreasonable given the nature and scale of the project’. The types of risks and the department’s assessed risk ratings are summarised in Table 4.1.

Table 4.1: Summary of residual risks (with mitigations in place) identified by the department

Source: ANAO analysis of AAD records.

4.5 The department concluded that the contract does not contain any contingent liability risk of occurrence greater than five per cent, or which would most probably result in expenditure of greater than $30 million if the event were to occur. It also concluded that the overall level of risk associated with the project remains ‘MEDIUM RISK as per the Tender Evaluation Report’.

Were soundly based initial cost estimates established by the department?

4.6 Although the department knew that the icebreaker that was most closely comparable to the vessel it proposed to acquire (the South African Agulhas II) had cost approximately $150 million in 2012, it presented cost estimates to Government that were three to four times this amount.³⁰ A complicated model developed by its consultants that the department knew overestimated costs was used to support these estimates.³¹ Refined estimates were expected following receipt of RFP responses.³²

4.7 Departmental records raise questions about whether the vessel being acquired is larger than the AAD really needed.³³ In addition, after receiving the RFP responses, the department increased the passenger capacity of the new ship from 80 to 116 passengers before the RFT was issued. No evidence to support this increase was evident in the department’s records.

Did the department accurately assess and report the RFP costing information received?

4.8 In August 2013, the Government was advised that the design and build of the new ship would cost $400.5 million (including a 20 per cent contingency). This was based on the highest of the cost estimates provided by the three short-listed respondents, notwithstanding that this respondent (Babcock) advised the department that its estimate was $264 million and that it does not represent value for money. Based on the $264 million Babcock submitted to the department, the actual contingency appears to be 51.7 per cent, significantly more than had been advised to Ministers by the department.

4.9 Following a change in government at the 2013 election, the department’s estimate was raised to $467.2 million in advice provided in April 2014. No evidence to support this increase was evident in the department’s records.

4.10 Not surprisingly given the proposed vessels were only at the early concept stage, the annual operating cost estimates provided by respondents were incomplete and meaningful life-cycle costs could not be robustly calculated. However, ship operating costs equivalent to $42.5 million per annum were advised to government in August 2013 (increased to $43.8 million in April 2014), significantly higher than the $24.95 million estimate in the Babcock proposal.

Were tendered prices compared to the cost estimates in the RFP response?

4.11 The Tender Evaluation Report made no comparisons to the information provided in the tenderer’s RFP response.³⁴ Importantly, the department’s RFP contract principles allowed for a cost variation of up to 20 per cent between the RFP and the RFT. The Evaluation Work Group report that addressed pricing and cost information and related responses outlined that:

It is important to note that the prices provided in Schedule 5 were broadly indicative rather than definitive, and the evaluation report has considered materiality in undertaking the analysis. In addition, the contract principles allowed for a cost variation of 20 per cent between the RFP and the next stage.

4.12 However, Table 4.2 shows that increases in costs and crew numbers significantly exceeded this threshold. In February 2017, the department commented to the ANAO as follows:

The ANAO has focused on the difference between the prices tendered in the Request for Proposal phase and the final contracted price for the ship. This does not take into consideration the evolution of the Department’s requirements between the Request for Proposal and Request for Tender phase, which is an expected part of this type of approach to the market. For example, the increase in the passenger numbers mentioned in the report, to match that of the Aurora Australis, was a result of the decision of Government to discontinue the aviation-related aspects of the earlier approach. In addition, the price comparison does not adequately account for the fluctuations in the exchange rate over the period of the procurement process (there was a 10 per cent depreciation in the Australian dollar against the euro and the United States dollar between 2012 and 2016).

Table 4.2: DMS cost and crew number changes from RFP to RFT

Note: Costs and crew numbers shown were subsequently varied in the final contract.

Source: ANAO analysis of departmental records (including DMS’ RFP and RFT responses).

Were the planned strategies to address only a single tender being received effectively implemented?

4.13 In December 2014, the department requested KPMG to prepare a contingency planning brief to provide a high level assessment of the risk of one of the two remaining shortlisted entities withdrawing from the process prior to tender submission, and provide an overview of the risk mitigation options available to the department. In terms of risk mitigation, the department and its advisers considered that value for money could still be publicly demonstrated where only one tenderer remained by:

• developing a benchmark cost model before the Request For Tender closes;
• developing a detailed pricing model to compare the tenderer’s response when received;
• requesting ‘open book’ pricing; and
• conducting forensic accounting verification of the tenderer’s pricing model.

Were cost benchmarks established before the tender response was received?

4.14 The department’s contingency planning brief prepared in early December 2014 identified that a benchmark cost model would be developed ‘in the period before the RFT closes’. An RFT addendum dated 13 February 2015 advised the tenderer that the department would conduct a cost investigation of the tendered price. However, the tenderer was not advised that the department would be conducting any benchmarking. Notwithstanding, the Tender Evaluation Plan was amended on 27 February 2015 to include the cost investigation and also to enable the Evaluation Team to request, if required, that a benchmarking exercise be conducted to compare prices for core components of the tendered solution to other comparable vessel builds.

4.15 As KPMG’s benchmarking report was finalised some four months after the RFT closed, a perception that it may have been influenced by the tender is unavoidable. Further, in March 2015, KPMG advised the department that preparing its benchmarking model ‘minimises the risk of the Department of Finance and Deregulation producing alternative pricings and declining the value for money assessment’.

Are the benchmarking results well-founded?

4.16 The KPMG benchmarking report presented various ranges and expected values based on different methodologies. This left it open to the department to choose which figure to rely on. In February 2017, KPMG advised the ANAO that:

The Independent Cost Estimate estimated a range of between $304.2 million and $454.5 million with an expected value of $379.4 million

4.17 Other issues identified by the ANAO with the benchmarking results included:

• the method used to convert ship costs to 2015 Australian dollars. For example, a sample ship examined by the ANAO that cost Euro $116.2 million in April 2012 ($A149 million at April 2012 exchange rates, increasing to $A159.29 million in 2015 dollars after allowing for inflation) was shown as $225 797 830, a figure 42 per cent higher. The method KPMG chose to use was to convert all costs to United States dollars, then to escalate this figure using OECD Total Producer Price Index Manufacturing data to update prices to 2014 United States dollars, and then convert those costs into 2015 Australian dollars using OECD data on 2014 exchange rates;
• not all of the vessels in the benchmarking cohort were icebreakers, with the benchmarking report stating that the vessels included ‘had to be built for an icebreaking role or be ice strengthened, be civilian in nature and have data available on actual cost of completion’³⁵;
• reflecting that the benchmarking relied upon publicly available information, there is no evidence that a range of ‘additional costs’ that KPMG ‘deemed’ were not included in the reported ship costs were, in fact, not included. These additions (which were based on a February 2008 study of tankers ranging from 80 000 tonnes to 110 000 tonnes built in Croatian shipyards between 1996 and 2005) increased the benchmark figures derived using the regression formula by $79.8 million to $89.3 million (24 to 36 per cent) and increased the benchmark figure derived using the ‘Independent Cost Estimate’ model by $75.6 million (28 per cent);
• the analysis used as ‘an important reference point’ the estimated cost of the Aurora Borealis (which has not been built as, with an estimated cost in excess of $1 billion, it was considered too expensive); and
• the report included estimated crew numbers³⁶ of between 34.7 and 49.3, with an expected value of 42.0, compared to 29 crew specified in the signed contract and 24 crew specified in the tenderer’s RFP response.³⁷ In February 2017, KPMG advised the ANAO that:

The independent analysis was undertaken without access to the RFP, contract and other project data. Based on the limited access to relevant crew data the assessed crew was predicted to fall between 34 and 49.

4.18 The benchmarking report outlined that ‘a value for the cost of the total build program is estimated to fall between $304.2m and $454.5m, with an expected value of $379.4m.’ The benchmarking report adjusted the price tendered by DMS ($454.29 million, that is, $210 000 below the upper benchmark range) to remove ‘unallowable escalation and pre-bid charges’ of $8.41 million, and added in the estimated cost of minimum fuel required after delivery of the vessel (which had been omitted from the tendered price). On this basis, the benchmarking report concluded that:

The adjusted tendered prices for the Design and Build exceeds the broad parametric cost estimate by $67.3m but is within the upper estimate by $7.8m.

4.19 However, the report also recognised that there were 29 price risks that would need to be addressed during negotiations. The price further increased during contract negotiations such that the contracted prices for the new icebreaker exceed all benchmarks (see Table 4.3). Specifically, at the time the contract was entered into, the department estimated that:

• vessel design and build would cost $488.19 million (excluding GST), which is $108.79 million (29 per cent) above the ‘expected value’ reported by KPMG, and $33.69 million (7 per cent) above the top of the benchmark range reported by KPMG; and
• operation and maintenance would cost $A285.45 million (excluding GST, fuel and other pass-through costs) in the first ten years (an average of $28.5 million per year, which is $1.0 million per annum above the KPMG benchmark).

Table 4.3: Summary of KPMG benchmarking analysis results

Note a: Estimate based on various ships’ light weight tonnages using a regression formula developed by KPMG.

Note b: See discussion regarding ‘additional costs’ assumed by KPMG at third dot point of paragraph 4.17.

Note c: Based on a confederated model developed by KPMG using SEER for Hardware (an off the shelf parametric modelling tool) and Microsoft Excel.

Note d: Recalculated estimate following changes to the model requested by the department. These included: replacing the modelled labour costs for the ship hull elements with the actual labour rate submitted by the bidder; extracting profit from all individual elements and applying that as a single factor; and adding a five per cent contingency.

Source: ANAO analysis of KPMG Benchmarking Analysis Report.

Was the ‘open book’ review effective?

4.20 KPMG’s fee proposal outlined that, to be in a position to make a statement on the ‘reasonableness’ of the bid pricing it would need to validate 80 per cent of the tenderer’s price to independent supporting documentation. KPMG reported that it sought to check 81.2 per cent of the total tendered price and was able to check 71.9 per cent. KPMG’s report to the department outlined that:

There were a number of significant items for which the Tenderer did not provide supporting evidence and as a result were not able to be checked.

4.21 Where items were not able to be traced to documentation, this was noted in the KPMG report as a price risk. The KPMG report identified 29 ‘price risks’ for negotiation of lower prices, along with two items that were not permitted under the contract ($4.9 million to recover the cost of bid preparation and $3.5 million for unallowed price escalation). However, there was no clear line of sight between the identification of these price risks and unpermitted items in the KPMG report and the ultimate outcome of the contract negotiations.

4.22 The Tender Evaluation Report prepared by the department stated that 71.9 per cent of the tendered price had been verified³⁸ but did not address whether, having regard to the target of 80 per cent, this was sufficient. The Minister was advised that 80 per cent of the tendered price had been verified.

4.23 The review revealed that the tenderer had increased its project management fees by $53.1 million following P&O’s withdrawal (up from $27.5 million to $80.6 million in the submitted tender). KPMG’s report referred to representations from the tenderer that the increase reflected a lack of understanding by the consultant engaged by the tenderer to review its RFT pricing that a DBOM approach was being adopted.³⁹ However, the tenderer’s consultant’s report had explicitly recognised that the procurement involved ‘design, build, deliver, operate and maintain a new vessel.’ In addition, February 2017 advice to the ANAO from Serco Defence advised that:

The AAD pricing model in the RFP was quite basic, whereas the price model in the RFT required inclusion of many price elements not included in the RFP in a rigid and complex format.

4.24 A reduction of $14.3 million⁴⁰ to the $80.6 million project management fee was subsequently negotiated. However, the project management fee remained well above the $27.5 million referred to in paragraph 4.23.

Was the tenderer’s pricing model verified and assessed as appropriate?

4.25 The tenderer refused to provide its pricing model, instead attending a workshop with the department and its advisers to explain how the model works.

Did the Tender Evaluation Report accurately reflect the pricing findings?

4.26 The Tender Evaluation Report prepared by the department stated that:

The Tenderer proposed a price for D&B activities of $458.4 million. While there were a large number of items within this price that are in line with expectations or which could be agreed to the supporting documentation provided by the Tenderer, the overall price is at the upper end of what would be expected based on KPMG’s open book evaluation and independent cost analysis.

4.27 The Tender Evaluation Report did not identify the KPMG benchmark figures and did not state that a tendered price of $458.4 million⁴¹ was:

• $3.9 million above KPMG’s upper estimate; and
• $79.0 million above the ‘expected value’ calculated by KPMG.

4.28 The Tender Evaluation Report prepared by the department stated that the tendered Operate and Maintain costs were $883.0 million over 30 years (or $29.4 million per annum⁴²), excluding the pass-through costs of fuel and insurance. Similar to what was said in relation to the design and build costs, it stated that those costs were ‘at the upper end of what could be expected’ when they were actually above the KPMG benchmark figure of $27.5 million. In this respect, the KPMG benchmarking report had stated that the ‘adjusted tendered price is $2.0m higher than the expected value’ of $27.5 million.⁴³

4.29 The Tender Evaluation Report did not seek to explain how prices that were at (in reality above) ‘the upper end’ could still have been assessed by the department as representing value for money given that the tender had not been assessed as fully meeting any of the evaluation criteria.

Were tendered prices compared to the costs of chartering the Aurora Australis?

4.30 The Tender Evaluation Report made no comparisons to the current charter costs, which totalled approximately $18.1 million (excluding GST) in 2015–16.⁴⁴ Based on the contracted prices for the design and build of the vessel and its annual average operational and maintenance costs, the ANAO calculated that the comparative cost of Commonwealth ownership of the new icebreaker is approximately $59.25 million per annum.⁴⁵

4.31 In February 2017, the department commented to the ANAO as follows:

Operating cost comparisons between the Aurora Australis and the new icebreaker are misleading. The new vessel is designed to meet the anticipated needs of the Australian Antarctic Program for the next three decades, and is therefore larger and more capable than the Aurora Australis. In addition, compliance with global ship design regulations has driven an increase in complexity and size of these types of vessels and as a result an increase in cost. The Aurora Australis was initially designed to work as part of a two-ship operating model and did so until mid-way through the last decade. The new ship has to satisfy the full requirements of resupply and research over extended seasons, without additional vessel support.⁴⁶

Were the targeted contract negotiation savings realised?

4.32 The Tender Evaluation Report stated that the total cost of ownership was $1 322 million. This comprised $458.4 million for the design and construction phase, and $863.6 million for a 20 year operate and maintain phase.

4.33 In October 2015, the department advised the Minister that, through contract negotiations, savings of $20 million were achievable for the design and build phase, with further savings of $4.7 million per annum achievable during the operate and maintain phase.

4.34 The targets the department set were low in comparison to the overpricing and unallowed items identified by KPMG. Although many items were not quantified, those that were totalled in excess of $116 million for the design and build phase.

4.35 Some savings were achieved but overall the contract price increased.⁴⁷ This was largely due to design and schedule changes. Additional costs will also be incurred for life extension works and extending the charter of the Aurora Australis. In February 2017, the department commented to the ANAO that:

The department negotiated cost savings of $14.3 million⁴⁸ to the tendered fixed-price for the ship. These savings went towards fully costed design improvements obtained by the department through workshops with the tenderer supported by a transparently documented costing process. The Department also negotiated significant savings in the 10 year operate and maintain phase⁴⁹, and contract variations which were to the benefit of the Commonwealth.

4.36 At the conclusion of negotiations in April 2016 the Minister was advised that the ship was now scheduled to arrive in June 2020 (rather than October 2019), the contract price had increased and that work on refinement of the design ‘has been progressing ahead of contract signing to minimise delays but has added to the contract price’.

4.37 The contracted design and build phase price was denominated in two currencies ($A59.95 million and Euro 276.43 million) to be paid in 10 payments across 20 milestones. At the time the contract was entered into, the department estimated that vessel design and build would cost $488.19 million (excluding GST).

4.38 Operation and maintenance of the vessel involves monthly service payments comprised of five components:

• a monthly base payment for up to 200 days of availability within the annual operating period;
• a daily service payment for additional days of availability above 200 days within the annual operating period;
• a maintenance payment to reflect forecast maintenance;
• a variable payment based on the extent to which the actual season activity exceeds the baseline level of activity; and
• a pass-through payment for certain costs (in particular, fuel).

4.39 The service payments involve various formulae as well as four currencies. At the time the contract was entered into, the department estimated that operation and maintenance would cost $A429.69 million (excluding GST) in the first ten years. The department further estimated that the four options of five years each would cost a total of $A939.57 million (excluding GST) over the 20 year period.⁵⁰

Appendix 1 Entity response

ANAO Comments:

1. See paragraphs 2.45 to 2.56 concerning competition during the Request for Tender stage and paragraphs 4.18 to 4.19 for a comparison of the contract prices to the cost benchmarks prepared for the department.

2. See paragraphs 4.6 to 4.7.

3. See paragraphs 2.27 to 2.42.

4. See paragraphs 4.32 to 4.36.

5. See paragraphs 4.20 to 4.25.

6. See paragraphs 4.14 to 4.19.

Appendix 2 Other formal comments on the proposed report

Appendix 3 Procurement options assessment scores

Note a: The importance ratings had weightings of High = 3, Medium = 2 and Low = 1.

Weighted scores were calculated by multiplying the assessment score by the Importance rating.

Source: ANAO analysis of departmental documentation and consultants’ reports.

Appendix 4 RFP Capability Matrix

Note: The RFT was based on point #8 in the matrix. By way of comparison, the Aurora Australis has an icebreaking capacity of 1.23 metres and a cargo capacity of 800 tonnes dry cargo capacity plus 1 100 000 litres of bulk fuel.

Source: ANAO analysis of the department’s Request for Proposals and other departmental records.