Introduction

1.1 Geoscience Australia is a non-corporate Commonwealth entity within the Industry, Science and Resources portfolio. The Chief Executive Officer is the accountable authority.

1.2 The purpose of Geoscience Australia is to inform government, industry and community decisions on the economic, social and environmental management of the nation’s natural resources through enabling access to geoscientific and spatial information.¹

1.3 In Geoscience Australia’s 2023–24 Corporate Plan, the Southern Positioning Augmentation Network (SouthPAN) is part of the broader Positioning Australia initiative that is part of the ‘creating a location-enabled Australia’ strategic priority. The relevant objective of the priority is to deliver ‘an accurate and reliable national positioning capability’, including ‘access to precise positioning services across Australia and its maritime zones’.

The Southern Positioning Augmentation Network

1.4 A Satellite-Based Augmentation System (SBAS) is a technology that delivers augmented Global Navigation Satellite System (GNSS) positioning. There are two user categories — aviation and non-aviation.

1.5 SBAS for aviation is an International Civil Aviation Organization certified safety-critical system² that provides wide coverage of enhanced GNSS by transmitting augmentation information from geostationary satellites.³ The purpose is to improve the accuracy, integrity, availability and continuity of GNSS signals for aircraft navigation.⁴ SBAS for non-aviation supports a range of GNSS applications that focus on high-accuracy positioning solutions primarily for industry.

1.6 SouthPAN is an SBAS that will deliver international open standard signals for positioning. This will be achieved by combining and correcting data from the European Galileo GNSS, the USA’s Global Positioning System (GPS) and dedicated GNSS reference stations⁵ across Australia, New Zealand, the Pacific, Antarctica, Asia and Africa which will be uploaded as signals to a geostationary satellite for transmission to end-users.

1.7 SouthPAN will support three end-user services provided as signals⁶, each with an internet service. It has been stated publicly that these services will provide access to real-time location data to an accuracy of 10cm across Australia and its maritime regions. Any person who has access to a device that is SBAS enabled, such as a mobile phone, will be able to receive the signals.

1.8 The economic benefit of this satellite technology is linked to innovation in automation solutions across a range of sectors and when fully operational in 2028, is expected to also deliver safety-critical capability.

The SBAS trial and procurement for SouthPAN

1.9 A national network for positioning, navigation and timing (PNT) data is regarded by Geoscience Australia as ‘critical to Australia maintaining its competitive advantage’ for industries that use this data. A market solution has not emerged in Australia, a trend mirrored internationally with countries investing in a range of government funded PNT infrastructure since the 1990s.

1.10 The satellite technology of an SBAS can deliver positioning accuracy to 10cm without the need for mobile phone coverage, making it a solution for regional and remote Australia and instances when mobile coverage is not available. Australia’s position on the globe also affords an advantage over other countries in relation to observing global PNT data. This provides greater commercial and research opportunities for Australia.

1.11 Following a conversation at a conference in 2016 between a Geoscience Australia official and a representative from Lockheed Martin Space Corporation, Geoscience Australia received an offer by letter in July 2016 to test an SBAS in the Asia-Pacific region. The offer (which expired on 31 December 2016) was for an international consortium led by Lockheed Martin Space Corporation to absorb just under half the expected costs by waiving $10 million for access to a satellite payload.⁷ In October 2016, a different technology company advised its stakeholders by email (including the same Geoscience Australia official) that it had won the contract for a South Korean SBAS and that it remained ‘committed to the campaign to bring SBAS to Australia and New Zealand’.

1.12 In the Mid-Year Economic and Fiscal Outlook in December 2016, funding of $12 million over three years was provided to take up the Lockheed Martin Space Corporation offer to trial the benefits of high-accuracy satellite positioning technology for Australian industry. The New Zealand Government joined the project, providing an additional $2 million to extend the trial to New Zealand.

1.13 The SBAS trial or test-bed⁸ was delivered by the three commercial organisations that comprised the international consortium through contracts that were entered into in January 2017. The test-bed was delivered through: Lockheed Martin Space Corporation’s (LMC) uplink station⁹; GMV Innovating Solutions’ (GMV) software; Inmarsat’s 4F1 satellite; and Geoscience Australia’s existing Continuously Operating Reference Stations. No public procurement process was undertaken.

1.14 The SBAS trial, commencing in October 2017, involved 27 demonstrator projects across 10 industry sectors. During the SBAS trial, Geoscience Australia released four approaches to market (between 26 February 2018 and 27 August 2019) that sought information in relation to ‘identified areas of program risk’ for delivering an SBAS for Australia and New Zealand.

1.15 The decision to proceed with the SBAS was announced in the Federal Budget on 8 May 2018. The National Positioning Infrastructure Capability (NPIC) and the SBAS programs were announced as part of the Australian Government’s Australian Technology and Science Growth Plan with funding provided through the then National Digital Economy Strategy. The NPIC program was established with $64 million over four years and $11.7 million ongoing with the SBAS program receiving $160.9 million over four years and $39.2 million ongoing from 2022–23 for the purpose of building world leading positioning capability.

1.16 At the end of the demonstrator projects in January 2019, an economic benefits study based on the test-bed results was undertaken. This report of the study along with a test-bed overview and technical report were published in August 2019.^{10 11 12} In February 2019, the contract with Inmarsat was extended to 31 January 2024 at a cost of $19.6 million.¹³

1.17 The SBAS project (now called SouthPAN) is led by a partnership between Geoscience Australia and Toitū Te Whenua Land Information New Zealand (LINZ). The partnership between the countries was announced following the annual Australia-New Zealand Leaders’ Meeting on 28 February 2020, by the Prime Ministers of Australia and New Zealand under the Agreement Relating to Science, Research and Innovation Cooperation.¹⁴ Under a Joint Procurement Agreement, the parties agreed that Geoscience Australia would be responsible for undertaking the procurement with LINZ retaining a decision-making role. New Zealand also made a financial contribution to the procurement process and the contract.

1.18 The procurement commenced in March 2020 and was designed as an open and competitive tender, with a single approach to market for products and services to establish and maintain SouthPAN as a service. The transmission of the three test signals continued until the contracts with LMC and GMV ended on 31 July 2020. Due to the bids from the tenderers exceeding the available funding, between November 2020 and March 2021 Geoscience Australia undertook an ‘offer definition and improvement’ process followed by a ‘best and final offer’ process against a reduced scope for the request for tender. The bids again exceeded the available funding.

1.19 In October 2021 contract negotiations commenced with the preferred tenderer, Lockheed Martin Australia Pty Ltd (Lockheed Martin Australia). In the Mid-Year Economic and Fiscal Outlook in December 2021, additional funding of $521.8 million over 20 years from 2021–22 was provided towards the procurement to deliver the Australian Government’s publicly announced commitment to provide 10cm accuracy through SouthPAN across Australia and its maritime zones. On 13 September 2022 the contract for $1.18 billion was signed. On 26 September 2022, the test-bed was reactivated and early open services became available. The services that SouthPAN is expected to deliver are to become progressively available with a fully operational and certified safety-critical system expected to be in place by 2028.

Rationale for undertaking the audit

1.20 On 16 September 2022 the Australian Government announced a $1.18 billion contract over 19 years with Lockheed Martin Australia Pty Ltd to deliver SouthPAN. In 2019, it was estimated the services SouthPAN delivers would be worth $6.2 billion to the Australian economy over the next 30 years.

1.21 The audit provides assurance to the Parliament whether Geoscience Australia undertook an effective procurement that was compliant with the Commonwealth Procurement Rules and has resulted in the potential to achieve value for money over the life of the contract.

Audit approach

Audit objective, criteria and scope

1.22 The audit objective was to assess the effectiveness of Geoscience Australia’s 2020 to 2022 procurement of SouthPAN.

1.23 To form a conclusion against the objective, the ANAO examined:

• Did Geoscience Australia deliver a compliant procurement process that will achieve value for money?
• Did Geoscience Australia put in place fit-for-purpose arrangements to effectively manage the SouthPAN contract?
• Did Geoscience Australia put in place fit-for-purpose arrangements to effectively assess the performance of SouthPAN?

1.24 The audit assessed whether the 2020 to 2022 procurement process complied with Commonwealth Procurement Rules (CPRs) and has the potential to deliver value for money over the life of the contract. It also assessed the effectiveness of Geoscience Australia’s contract and performance management of Lockheed Martin Australia to deliver SouthPAN. The audit scope did not include:

• the procurement of various satellite payload/navigation services;
• the expression of interest process from landowners to host GNSS reference station sites;
• an analysis of the services provided through SouthPAN;
• the quality or technical support of the services provided through SouthPAN;
• user satisfaction with the services; or
• the direct arrangements that exist between Lockheed Martin Australia and its subcontractors, beyond how Geoscience Australia maintains control of such arrangements.

Audit methodology

1.25 The audit involved:

• examining Geoscience Australia’s records;
• conducting system walkthroughs; and
• meetings with relevant staff and contractors.

1.26 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $399,689.

1.27 The team members for this audit were Rowena Thomson, Sean Brindle, Ben Thomson, Caitlin Williams, Renina Boyd and Michelle Page.

2.1 Following a trial of Satellite-Based Augmentation System (SBAS) services, Geoscience Australia commenced a procurement process in 2020 to deliver positioning services with decimetre accuracy and high integrity positioning data.¹⁵ Officials from non-corporate Commonwealth entities are required to undertake procurements in accordance with the CPRs and, in particular, achieve value for money.

Did the 2020 to 2022 SouthPAN procurement comply with the CPRs?

Governance arrangements for the procurement

2.2 The Tender Evaluation Plan for the procurement was approved by the Chief Executive Officer (CEO) of Geoscience Australia on 21 July 2020. The Tender Evaluation Plan outlined the roles and responsibilities of the delegate (the CEO), the Tender Evaluation Steering Committee and the Tender Evaluation Board. The governance structure is provided in Figure 2.1.

Figure 2.1: Governance structure for the SouthPAN procurement

Source: Geoscience Australia’s Tender Evaluation Plan.

2.3 The role of the Tender Evaluation Steering Committee was to oversee the governance of the request for tender process. It comprised four senior executive service officers — two from Geoscience Australia and two from Toitū Te Whenua Land Information New Zealand (LINZ). An additional member from the Department of Defence joined the committee in October 2021. Substantive changes to the Tender Evaluation Plan, such as changes to the board or steering committee membership, required approval by the CEO. This occurred on 20 October 2021.

2.4 The role of the Tender Evaluation Board was to prepare a draft Tender Evaluation Report for endorsement by the Tender Evaluation Steering Committee and then prepare a final Tender Evaluation Report for decision by the CEO. It comprised the Chair (Geoscience Australia), a Deputy Chair (LINZ) and two evaluators (Geoscience Australia and LINZ), all of whom were executive level officers. The Tender Evaluation Board was supported by probity, legal, commercial, cyber security, technical and project management advisers as required.

2.5 There were two other governance groups outside the procurement process — the Joint Governance Board of Management and the Positioning Program Board.

• The Joint Board of Management was established in November 2019 to coordinate the planning and scheduling of the implementation of SouthPAN between Australia and New Zealand. In September 2022 it was replaced by the Joint Governance Board of Management. It is responsible for: developing bilateral agreements; endorsing technical requirements; agreeing cost sharing arrangements; the communication and implementation strategies; major procurement recommendations; changes to scope, cost or timing; monitoring program risk and the realisation of program benefits; and providing program assurance. There are currently five members — one each from Geoscience Australia and LINZ, and three independent members.¹⁶ The chair position alternates between Australia and New Zealand. The Geoscience Australia and LINZ members report to the head of their respective agencies.
• The Positioning Program Board provided (and continues to provide) strategic guidance, advice and oversight of Geoscience Australia’s Positioning Program¹⁷ within the context of Geoscience Australia’s National Positioning Infrastructure Branch activities. It also provides direction on management of high or extreme risks and decision making, as well as endorsing or not endorsing suggested resolutions, that may affect progress of the Positioning Program.

2.6 The members of the Tender Evaluation Board were different to those of the Tender Evaluation Steering Committee. The Chair of the Tender Evaluation Board reported to the CEO through the Tender Evaluation Steering Committee. During the planning of the procurement and up until the Tender Evaluation Plan, the Chair of the Tender Evaluation Board was also a member of the Joint Board of Management. Two members of the Tender Evaluation Steering Committee (including the Chair) were also members of the Joint Board of Management throughout the procurement.

The procurement process

2.7 Prior to the request for tender, there were four requests for information released on 26 February 2018, 17 January 2019, 3 May 2019, and 27 August 2019. Based on the information received from these, the request for tender invited suppliers to build, operate and maintain SouthPAN.

2.8 Before identifying whether a single stage open tender was appropriate, a range of options for the procurement were assessed by Geoscience Australia, including: a panel arrangement or multi-use list through an open tender; leveraging a whole-of-government arrangement or an existing government contract; a limited tender; funded prototypes developed by short-listed tenderers to be coordinated by Geoscience Australia; and a multi-stage open tender.

2.9 The Senior Responsible Officer, who was also the Chair of the Positioning Program Board (see paragraph 2.5), approved the Procurement Strategy on 1 August 2019, noting that: ‘The option of single stage Open Tender for seeking a proven technical solution delivered by a service provider with a proven track record of establishing and sustaining similar capabilities in recent years is recommended as the preferred option’.

Requests for information

2.10 Geoscience Australia undertook the requests for information to ‘reduce the risk of failing to fully understand what needs to be done or how it will be done’. The goal was to enter into a contract arrangement with a single service provider ‘to deliver and sustain a turn-key solution as an end-to-end managed service — “SBAS as a service”’. It was regarded by Geoscience Australia as a complex system that was required to be ‘designed to meet global performance requirements under Australia and New Zealand’s unique service area and space weather conditions’.

2.11 On 27 August 2019, Geoscience Australia published a fourth request for information. It closed on 5 November 2019 and the resulting Market Analysis Report was approved on 19 December 2019. It outlined that the market analysis validated the single stage open tender approach as the ‘most effective way of achieving value for money for Geoscience Australia and Land Information New Zealand’.

2.12 Potential suppliers gave feedback on: potential solutions and services; service delivery performance; service delivery timelines; contractual arrangements; and refined indicative pricing with seven potential suppliers giving presentations to Geoscience Australia. Geoscience Australia assessed that four of the seven potential suppliers proposed solutions that closely aligned with the project requirements. Geoscience Australia concluded that there was a competitive market sector with the capacity to deliver an SBAS and maintain it. At the same time, Geoscience Australia noted that while the four potential suppliers that proposed closely aligned solutions had established and built an SBAS, and two others were able to provide satellite payload services, none had yet run an ongoing SBAS service.

2.13 The Market Analysis Report documented that the potential suppliers identified additional risk associated with: a lack of clarity around software integration; the tight timeframe to deliver set by Geoscience Australia; and the challenges associated with certifying an SBAS service for aviation. One potential supplier also raised the concern that another supplier of SBAS products had an advantage over the others, due to a small number of project requirements and ‘emphasised the importance of making requirements supplier/technology agnostic’. This feedback was considered by Geoscience Australia in reviewing the requirements.

The request for tender

2.14 On 11 March 2020 the CEO approved the request for tender for release. In the email to the CEO requesting approval, the Joint Board of Management advised that the request for tender built on the test-bed ‘to specify a full operational SBAS’ and that ‘this experience has proved highly valuable in the development of system requirements’. It also noted that feedback to the SBAS Function and Performance Specification request for information had been incorporated into ‘very mature system specifications’ which had ‘substantially reduced the implementation risk of the program’.

2.15 On 13 March 2020, Geoscience Australia published a request for tender on AusTender for the Southern Positioning Augmentation Network (SouthPAN). Between 25 March 2020 and 19 July 2020, fourteen addendums were issued that provided more information to tenderers. At the closing date of 22 July 2020¹⁸, responses were received from Lockheed Martin Australia Pty Ltd (Lockheed Martin Australia) and two others.

2.16 The request for tender documentation outlined that the purpose of the procurement was to develop and maintain a positioning augmentation network to improve the capability of the positioning already provided by the GPS and Galileo satellite constellations. The expectation was for improved positioning capability to benefit user applications in the agriculture, construction, resources and utilities as well as decimetre accuracy (with a 40 minute convergence time), and sub-metre level accuracy (in real time) for the aviation and road transport sectors. Services to be delivered included: open, internet and safety-of-life.¹⁹

2.17 The Tender Evaluation Plan outlined the five stages of the evaluation:

• Stage 1 — compliance screening;
• Stage 2 — detailed evaluation against the evaluation criteria;
• Stage 3 — risk assessment and verification activities (which could also be conducted as part of other evaluation stages);
• Stage 4 — value for money assessment; and
• Stage 5 — the Tender Evaluation Report.

2.18 The report on the compliance screening included: the tenders received; details of any late tenders; the completeness of the tenders; compliance of the tenders with the minimum content and format requirements or conditions for participation; and any recommendations regarding tenders assessed as non-compliant. The report recommended that Lockheed Martin Australia and another tenderer progress to Stage 2, which the Tender Evaluation Steering Committee approved on 5 August 2020. The third tenderer was not recommended to progress as the tender was not compliant.

2.19 The nine evaluation criteria for the procurement and the weightings for each are outlined in Table 2.1. Geoscience Australia advised that there were ‘several workshops during the week of 2 March 2020 to finalise the weightings for the evaluation criteria, attended by the Director, Project Manager, Senior Engineer, and Procurement Officer who used their professional experience and judgement — including comparing alternate models — to establish the weightings’.

2.20 For the weighted criteria, the methodology was a comparative qualitative rating on a scale out of 100.²⁰ For criteria six to nine an order of merit was established. Pricing was assessed against four ratings: unacceptable; not favourable; acceptable; and favourable. Ten categories²¹ of risk were assessed against a likelihood scale of five (rare to almost certain) and degree of impact, also on a scale of five (minor to catastrophic), to produce an overall level of risk. Under the approved Tender Evaluation Plan each member of the Evaluation Tender Board was required to make an individual qualitative assessment against the criteria. Agreement by consensus followed in relation to an overall assessment for each evaluation criteria for each tenderer.

Table 2.1: The weighted and non-weighted evaluation criteria

Source: ANAO analysis of the Request for Tender documentation.

2.21 Geoscience Australia had an evaluation template for the Tender Evaluation Board members to assess the tenderers. A colour-code rating was applied with green for ‘outstanding’, ‘very good’, or ‘good’; amber for ‘fair’, ‘acceptable’, or ‘marginal’; and red for ‘poor’, ‘very poor’, or ‘unacceptable’. There were also free text fields for each evaluator to identify the strengths, weaknesses and risks.

2.22 Grey Advantage Consulting reviewed the criteria evaluation templates for Geoscience Australia and noted as at 18 June 2020 that the templates were ‘overly detailed and not easily linked to the procurement objectives described in the Procurement Plan or to the high level evaluation criteria. This will make scoring of the high level evaluation criteria extremely difficult. There is a risk that the audit trail provided by the Tender Responses to the Requirements will not be sufficient to evidence achievement of the procurement objectives and demonstrate value for money. Neither the requirements or the evaluation criteria have been mapped to the procurement objectives.’

The best and final offer process

2.23 At the end of October 2020, evaluation on pricing commenced. Both tenderers that progressed to Stage 2 were in excess of the approved budget. This resulted in Geoscience Australia moving to an offer definition and improvement process²², followed by inviting the tenderers to submit their best and final offers in December 2020. The requirement for the provision of government-owned ground uplink stations was removed in an effort to reduce costs.

• The original SBAS model assumed the Global Navigation Satellite Systems (GNSS) ground reference stations and the uplink stations would be owned by the Commonwealth. The uplink stations generate navigation signals and transmit those signals to the satellite payload. To receive and verify the data for the signal, uplink stations house central processing facilities²³, antennas (satellite dishes) and network (telecommunications) infrastructure.²⁴ Geoscience Australia advised the Australian Government that the alternative of leasing industry owned uplink stations would result in a saving of $105.2 million over 15 years and minimise delays.

2.24 Prior to the offer definition and improvement process, Lockheed Martin Australia proposed that two of the five major system technical reviews that would form part of the eventual contract be conducted prior to the awarding of the contract. Geoscience Australia sought legal advice from MinterEllison, which stated in an email dated 17 December 2020: ‘We understand that the tenderer has proposed that by undertaking early works, it will be better placed to meet the proposed timetable for delivery of the operating system into service and it can reduce the contingency in its price accordingly’. Geoscience Australia did not seek advice from the other participant in the offer definition and improvement process as to whether it would like to put forward a similar proposal.

• Geoscience Australia undertook a risk review in relation to the Lockheed Martin Australia proposal. SHOAL Group identified that the risks included: a perceived bias towards the tenderer; the criteria and requirements for the system technical reviews were not yet agreed; and it may result in a significant price increase (or withdrawal) due to unexpected project risks being identified. Following Lockheed Martin Australia being selected as the preferred tenderer, an agreement was executed on 7 December 2021.²⁵ The mitigation was for a number of contract documents to be agreed between the parties before the system reviews were undertaken.

2.25 In the letter inviting tenderers to provide their ‘best and final offer’ the amount available was identified as $750 million. The tenderers were requested to split the pricing of their best and final offers into separate capabilities so Geoscience Australia could better understand the service component costs. The best and final offers, while less than the original bids, were still 52.9 per cent above the available budget.²⁶

• The satellite payload component of both the original bids was deemed by Geoscience Australia to be excessively expensive²⁷ so in July 2021, the CEO approved removing the two payloads²⁸ from the procurement and undertaking separate procurement processes for these. This approach was identified by Geoscience Australia to potentially save an estimated $100 million.

2.26 The best and final offer period closed on 15 March 2021 and the CEO was briefed that despite the removal of the two satellite payloads, both bids were again significantly above the allocated budget and proceeding to contract award was not possible. This resulted in the CEO approving continuation of the request for tender while Geoscience Australia made a submission to the Australian Government in the December 2021 Mid-Year Economic and Fiscal Outlook for additional funding of $521.8 million, which was received.

• The separate procurement for the first satellite payload was released to the market between 1 October 2021 and 17 December 2021. There were no respondents. Geoscience Australia briefed the CEO about the need for revisions to the requirements following engagement with the market and the preferred tenderer²⁹ for SouthPAN. An amendment to the request for tender was released and the procurement resulted in Inmarsat being awarded the contract on 1 May 2023. The procurement for the second payload is expected to be completed in 2024. The expected saving of $100 million for separating these procurements from the SouthPAN procurement is unlikely to be achieved.

Compliance with the CPRs

2.27 The procurement commenced in March 2020 and the ANAO assessment of the procurement was against the April 2019 version of the CPRs in the period up to the release of updated CPRs in December 2020. For procurement activities that occurred after December 2020, the updated CPRs were applied. Table 2.2 summarises Geoscience Australia’s compliance.

Table 2.2: Procurement of SouthPAN CPRs compliance

Key: ◆ Geoscience Australia’s SouthPAN procurement complied with the CPRs

● Geoscience Australia’s SouthPAN procurement largely complied with the CPRs

▲ Geoscience Australia’s SouthPAN procurement partly complied with the CPRs

■ Geoscience Australia’s SouthPAN procurement did not comply with the CPRs

Note a: Depending on the timing of procurement activities, assessment was against the April 2019 or December 2020 CPRs.

Note b: Ethical behaviour in this context includes conflicts of interest, equitable treatment of potential suppliers and tenderers, probity advice, gifts or benefits, use of public resources, complying with relevant legislation and the handling of complaints.

Note c: The additional rules in Division 2 relate to conditions for limited tender, request documentation, specifications, conditions for participation, minimum time limits, late submissions, receipt and opening of submissions and awarding of contracts. ‘Minimum time limits’ in this context refers to the timeframes published on AusTender relating to applicant submissions and responses to the various stages of the procurement.

Source: ANAO analysis of CPRs compliance.

Value for money

The value for money framework

2.28 The CEO approved Tender Evaluation Plans³⁰ set out that the value for money assessment would consider, including but not limited to:

• the quality and fitness for purpose of the proposal (technical worth against the scored and weighted evaluation criteria);
• financial assessment of the pricing information;
• the extent to which the proposed Indigenous Participation Plan met the mandatory minimum requirements;
• the extent to which potential benefit to the Australian economy may be achieved;
• the extent to which the proposal met the broader outcomes of works and services undertaken in NZ; and
• any other risks that the entity considers relevant.

2.29 The Tender Evaluation Report states that the Tender Evaluation Board undertook a value for money assessment which considered:

• the objectives of the procurement;
• the scores for each of the five weighted evaluation criteria;
• the orders of merit for the IPP and Economic Benefit evaluation criteria;
• the price, including the schedule of payments and abatement regime;
• the risk profile;
• non-contract costs, including consideration of all costs, fees, allowances and charges associated with the implementation and completion of the obligations set out in the Draft Contract; and
• the combination of Contract and non-contract costs referred to as the Total Cost of Ownership.

2.30 Geoscience Australia advised that that reference in the Tender Evaluation Report to consideration being given to the objectives of the procurement was ‘an editorial inconsistency in the production of the report’. Geoscience Australia stated ‘at no stage were the “objectives of the procurement” used to evaluate tenders’ and that ‘the evaluation described in the Tender Evaluation Report fully meets the intent of the Tender Evaluation Plan’. The variances in the approach between what was planned and what was in the report warranted the Tender Evaluation Plan being updated by the CEO as the approving officer (see paragraph 2.3). This did not occur.

2.31 A presentation to the Tender Evaluation Steering Committee in July 2021 provided an overview of the Tender Evaluation Board’s value for money assessment for both tenderers and their recommendation. The value for money slides simply described the weighted criteria and the non-weighted criteria assessments (without explicit reference to the value for money assessment) as did the value for money section in the Tender Evaluation Report. The briefing and the Tender Evaluation Report were not clear or transparent as to how the value for money assessment linked to the procurement objectives.

Table 2.3: Tender Evaluation Board value for money overview presented to the Tender Evaluation Steering Committee

Note a: A rating for risk was not documented in numerical form.

Source: ANAO analysis of Geoscience Australia Tender Evaluation Steering Committee briefing.

The value for money assessment

2.32 The value for money assessment (Stage 4) was conducted on 22 July 2021 after the best and final offer process at a meeting of the Tender Evaluation Board. The purpose was for the Board to reach consensus on all the criteria and decide if any additional narrative was required for the 31 March 2021 draft of the Tender Evaluation Report. The minutes recorded that the assessment of Lockheed Martin Australia and the other tenderer was discussed in the context of the best and final offer submissions. The final Tender Evaluation Report (approved by the CEO on 16 September 2021) and the Tender Evaluation Board’s assessment were reviewed to determine how the value for money assessment was documented.

2.33 Two of the weighted criteria ratings were varied following the best and final offer process: Criterion 3 — Functional and Performance Specification; and Criterion 4 — Demonstrated Experience.

2.34 The other tenderer was reduced by five points for Criterion 3 as Lockheed Martin Australia:

• had a more compact site footprint;
• addressed the durability of stations in the hot and remote locations;
• included improved redundancies of uplink facilities and provided more information and data supported claims around achievable system performance; and
• the site architecture supported better system performance (accuracy and availability provided by long-distance sites in Thailand and South Africa).

It was also noted that the other tenderer ‘had some compliance issues with the minimum service area requirement, particularly in relation to northern New Zealand and northwest Australia’.

2.35 For Criterion 4, Lockheed Martin Australia was reduced by two points as: ‘[Lockheed Martin Australia] brings significant experienced staff meeting the minimum requirement but does not have organisational experience with certifying SBAS’ while the other tenderer had ‘significant and current experience in developing other SBAS and the certification process’.

2.36 Following the request for tender, the other tenderer was ranked ahead on Criterion 6 (Indigenous Participation Plan). Lockheed Martin Australia requested an exemption for this criterion, which it did not receive. In its best and final offer submission, Lockheed Martin Australia updated its tender to include a plan to recruit and train at least two Indigenous personnel to work at its Uralla uplink facility. The Tender Evaluation Board meeting minutes stated that ‘both the tenderers are equivalent’. The Tender Evaluation Board stated that Lockheed Martin Australia had ‘a detailed plan’ compared to the other tenderer which had ‘a generic plan’ to attain at least four per cent full-time equivalent Indigenous employees or sub-contractors through the capital works set-up and maintenance phases.³¹

2.37 As both tenderers proposed undertaking much of the work outside Australia, the Tender Evaluation Report noted that this necessarily limited the economic benefits to infrastructure development, operation and maintenance for both tenderers. The primary reason given for Lockheed Martin Australia ranking ahead of the other tenderer for Criterion 7 (Economic Benefit) was because it had a ‘superior plan for the commercial exploitation of the system outside of Australia and New Zealand when compared to [the other tenderer]’.³² The minutes from the meeting on 16 July 2021 note that a LINZ member of the Tender Evaluation Board queried whether a tied order of merit was acceptable if there was no clear basis on which to separate the tenderers. The Chair said that while the other tenderer had improved on the ability to on-sell the services during the offer and definition improvement process, that the Chair ‘was comfortable that [Lockheed Martin Australia] provided a greater degree of confidence that they would create a business opportunity and this would produce an economic benefit’. The summary for these sections in the Tender Evaluation Report stated that Lockheed Martin Australia had a ‘well thought out plan’ compared to ‘a plan’ for the other tenderer.

2.38 Despite the value for money framework stating the Tender Evaluation Board would consider the order of merit for the Indigenous Participation Plan and Economic Benefit evaluation criteria (see paragraph 2.29), in the value for money section of the Tender Evaluation Report it was stated: ‘Whilst the Tender Evaluation Board considered each Tenderer’s response to the non-weighted Indigenous Participation Plan and Economic Benefit Evaluation Criteria, it ultimately concluded they did not affect the overall value-for-money’.

2.39 A Deloitte Australia draft pricing analysis³³ dated 9 April 2021 informed the Tender Evaluation Board’s evaluation of the pricing schedules submitted in response to the best and final offer process. The pricing analysis stated that the other tenderer’s bid had a variance of 73 per cent more than Lockheed Martin Australia for the first year. In relation to Lockheed Martin Australia, the Tender Evaluation Report stated it was a strength that the pricing schedule ‘ramps up over the first three years of the contract so expenditure is deferred until late in the establishment phase’. The other tenderer’s proposal included greater up-front costs in the first year, with these decreasing over time. This was identified as a weakness in the Tender Evaluation Report. Geoscience Australia advised that the guidance to tenderers was to price according to effort each year.

2.40 In the pricing analysis, the other tenderer’s bid was significantly less than Lockheed Martin Australia’s: ‘[Lockheed Martin Australia’s] total project cost is consistently around 5%-15% more expensive than that of [the other tenderer] across the incremental option and all proposed alternatives, with an average variance of $82.94m or 9.54%’. This was identified in the analysis as a weakness for Lockheed Martin Australia but was not included in the Tender Evaluation Report. The pricing analysis also stated that the other tenderer’s ‘pricing alternatives are less transparent compared to [Lockheed Martin Australia’s] alternatives options’ and that it was ‘noticeably harder to comprehend [the other tenderer’s] set of assumptions’.

2.41 The meeting minutes record for Criterion 8 (Price) state that the Tender Evaluation Board ‘agreed that price is not a differentiating factor’ as both ‘proposed solutions were favourable and the price difference was negligible over the life of the project’, both were ‘within budget and within five per cent of each other’ and that the other tender had provided ‘the slighter cheaper solution’. The Tender Evaluation Report included a comparison on Total Cost of Ownership (the combined contract and non-contract costs) resulting in the bids being within 4.3 per cent of each other.

2.42 The other tenderer ranked first against pricing, which was Criterion 8, with a total cost projection less than Lockheed Martin Australia. The other tenderer also had no ‘tender validity extension’ cost, whereas Lockheed Martin Australia specified that the price would increase by $19.26 million if the contract was awarded after 1 January 2022. The Tender Evaluation Report stated: ‘the Tender Evaluation Board presumes this cost to be sunk and is included in [Lockheed Martin Australia’s] price’, but this was not stated to be a weakness.

2.43 Lockheed Martin Australia was deemed to have an overall lower risk profile (Criterion 9), however, both tenderers were described in the Tender Evaluation Report as having an acceptable level of risk. Of the 10 categories, Lockheed Martin Australia was rated as having low risk for three and very low risk for seven. The other tenderer was rated as having medium risk for three, low risk for two and very low risk for five.

2.44 In determining the risk profiles of the tenderers, the Tender Evaluation Board included the following considerations.

• It was likely a contract ‘…could be negotiated more quickly with Lockheed Martin Australia when compared to [the other tenderer] due to the significance of non-compliances with the contract and technical requirements’ (see paragraphs 2.46 to 2.50).³⁴
• Negotiation with the other tenderer would likely result in an increase in the Total Cost of Ownership³⁵, due to an increase in the contract price due to invalid assumptions by the other tenderer, or an increase in non-contract costs should the scope of the procurement be reduced to make the assumptions valid (see paragraph 2.51 to 2.54).
• Provision of an additional contingency amount of $44.695 million was needed against the establishment phase of the contract to clarify scope and mitigate risk for the other tenderer. The statement in the Tender Evaluation Report was that scope clarification and risk mitigation would result in a) an increase in price following negotiations; b) contract changes following contract award; c) organisational effort from Geoscience Australia and LINZ to deliver the full scope of the project; or d) a combination of the above (see paragraphs 2.55 to 2.56).

2.45 These three considerations were assessed against the supporting evidence in the Tender Evaluation Report.

The significance of non-compliances with the contract and technical requirements

2.46 The Tender Evaluation Report states that in relation to comparing the contract non-compliance between the two tenderers they were both rated good and that ‘Overall, [Lockheed Martin Australia’s] Tender is marginally more compliant and shifts less risk to Geoscience Australia but the differential between the Tenders remains small’. For commercial compliance, there were no ‘overt commercial risks or concerns’ for either tenderer. For legal compliance, non-compliances were grouped by significance. Table 2.4 outlines the assessment.

Table 2.4: Legal non-compliance assessment

Source: ANAO analysis of the Tender Evaluation Report.

2.47 The details of the non-compliances were grouped into 24 themes (which reflected terms in the contract) and assessed. Table 2.5 outlines the risk ratings against these.

Table 2.5: Legal non-compliance risk rating

Note a: For one theme, the other tenderer received a risk rating, there was no risk identified for this theme for Lockheed Martin Australia.

Source: ANAO analysis of the Tender Evaluation Report.

2.48 For the technical compliance assessment, Lockheed Martin Australia was assessed as being 95.8 per cent compliant with the Statement of Requirements and the other tenderer was found to be 99.6 per cent compliant. For the function and performance specifications Lockheed Martin Australia was 73.3 per cent compliant and the other tenderer was 96 per cent compliant. Table 2.6 outlines the percentages of compliance, partial compliance and non-compliance for the technical assessment.

Table 2.6: Technical compliance assessment

Source: ANAO analysis of the Tender Evaluation Report.

2.49 Table 2.7 provides the risk rating associated with the technical compliance assessment.

Table 2.7: Technical compliance risk rating

Source: ANAO analysis of the Tender Evaluation Report.

2.50 The information relating to contract non-compliance and technical compliance, as laid-out in the Tender Evaluation Report, did not facilitate ease of comparison between the two tenderers. It also did not document how the compliance and non-compliance related to the risk categorisation (see paragraph 2.20)

An increase in the Total Cost of Ownership due to invalid assumptions

2.51 The pricing section in the Tender Evaluation Report in relation to the other tenderer stated ‘The Tenderer has provided a long list of pricing assumptions that are relatively complex and it is therefore difficult to interpret to gain a clear understanding of what is included and excluded from the price.’ Table 2.8 outlines the assumptions documented in the Deloitte Australia draft pricing analysis. Lockheed Martin Australia had a total of 41 assumptions and the other tenderer had a total of 42. These were categorised into three assumption types: timing related; pricing related but unrelated to timing; and technical.

Table 2.8: Number of tenderer pricing assumptions by category

Source: ANAO analysis of the draft external pricing analysis.

2.52 Key assumptions that were identified by Deloitte Australia for the other tenderer were:

• assumptions that directly affected price required Geoscience Australia and LINZ to confirm which aspects of the solution would be relevant in the first 1–2 years of the project;
• assumptions for the full project and sustainment costs were relevant for the whole project, but determining which capabilities would be exercised by Geoscience Australia and LINZ would determine the scope of the sustainment phase;
• the offer was valid for three months longer than Lockheed Martin Australia’s offer;
• assumptions presumed cost and timeline would be borne by Geoscience Australia and LINZ if GNSS references sites were not available at least four months before the site acceptance date;
• assumptions were linear and relied on assumptions preceding it, therefore choosing a subset of capabilities was more difficult to plan compared to Lockheed Martin Australia’s offer; and
• the assumptions were more numerous and complex overall.

2.53 On 12 April 2021 Deloitte Australia advised Geoscience Australia that it did not include any of the technical assumptions in its analysis as it did not have the engineering expertise to assess the cost implications of them. It stated that ‘…the [Tender Evaluation Board] might be able to flag any technical assumptions that directly drive pricing…’ and ‘If none are found to materially impact pricing, then no update would be needed’. The Deloitte Australia draft analysis was not finalised. Geoscience Australia advised that the document was in draft form as ‘Given the importance of the content, [Geoscience Australia] worked directly in face-to-face discussions with [the contractor] to ensure the Tender Evaluation Board understood the advice.’

2.54 The value for money section in the Tender Evaluation Report stated ‘Lockheed Martin Australia had a lower level of technical risk by demonstrating a better understanding of the scope of works, and by making less assumptions about the delineation of scope between the contractor and customer’. This statement is made in the context of risk, not price (or Total Cost of Ownership). Further, while the value for money section summarises that Lockheed Martin Australia was assessed ahead of the other tenderer on project management (Criterion 1), service management (Criterion 2), function and performance (Criterion 3) resource management and personnel (Criterion 5) in the context of assumptions, it did not state how this assessment was connected to the total cost of ownership outcome or the risk assessment.³⁶

Provision of an additional contingency amount of $44.695 million

2.55 The Tender Evaluation Board applied a contingency amount of $44.695 million³⁷ as risk mitigation against the other tenderer’s assumptions. As there was no information in the Tender Evaluation Report other than that the contingency had been applied (see paragraph 2.44), information was sought on how the amount of $44.695 million increase to the establishment phase of the contract for the other tenderer was quantified. Geoscience Australia advised ‘The aggregated risks informed the [Tender Evaluation Board’s] decision to apply the risk adjustment of $44.695m’ and ‘The figure of $44.695m was not proposed as an increase to [the other tenderer’s] tendered price, it was an internal contingency adjustment reflecting the scope uncertainty and higher risk assessed on the other tender. The contingency adjustment was to ensure that an adequate budget was available on a total cost of ownership basis. It did not affect the ranking of the tenders on price or the overall outcome.’

2.56 This amount was included as part of the other tenderer’s total costs in the Tender Evaluation Report, which reduced the price differential between the tenderers. The minutes of the Tender Evaluation Board meeting on 22 July 2021 recorded that the purpose of putting a price on the potential cost of incorrect assumptions made by the other tenderer was to ‘inform management’. While the inclusion of Lockheed Martin Australia’s tender validity extension cost of $19.26 million in the total cost had a footnote to the relevant table in the Tender Evaluation Report, the contingency amount of $44.695 million against the other tenderer did not.

Demonstrating achievement of value for money

2.57 Part 6 of the CPRs relates to efficient, effective, economical and ethical procurement. Ethical is described as having a range of characteristics including consistency. Part 4 of the CPRs relates to value for money. Achieving value for money is the core rule of the CPRs. Procurements should facilitate accountable and transparent decision making.

2.58 Geoscience Australia had a framework for undertaking a value for money assessment and undertook the assessment. However, the inconsistencies in both the process and the reporting of it meant it was unable to be determined if Geoscience Australia correctly identified that Lockheed Martin Australia’s tender was better value for money than the other tenderer, or that either of them would deliver value for money.

Efficient, effective, economic and ethical

Probity matters

2.61 In March 2019, Geoscience Australia contracted Maddocks, a commercial law firm, to act as the external probity advisor for the procurement. Maddocks developed a probity framework for the procurement which detailed that all project personnel must complete conflict of interest and confidentiality documentation. The probity framework included a form for Australian Public Service (APS) employees and non-APS employees to declare conflict of interests and forms for APS employees to acknowledge their obligations regarding confidentiality, and for non-APS employees to complete a confidentiality undertaking via deed poll.

2.62 Geoscience Australia appointed an internal probity officer (referred to as the Contact Officer) who was an APS officer in the reporting line (for the purposes of the procurement) to the Tender Evaluation Board (see Figure 2.1).³⁸ The template for the conflict of interest declaration form in the Probity Framework stated that the person making the declaration was to immediately notify the Chair of the Tender Evaluation Board (in their usual role in the Positioning Australia branch) and the internal probity officer regarding any conflict of interest that may arise during the procurement.

2.63 Geoscience Australia created and maintained a probity register which contained conflict of interest and confidentiality registers for procurement personnel from both Australia and New Zealand. The probity register contained 332 conflict of interest declarations and 321 confidentiality acknowledgement/undertakings. Sixteen of the conflict of interest declarations were related to other procurements. Of the remaining 316 conflict of interest forms, 36 contained declarations of a potential conflict relating to 30 people. Five of those did not document mitigating action by Geoscience Australia’s probity officer, who was required under the probity framework to take action to avoid or manage any identified actual, apparent or potential conflict of interest that arose.

2.64 ANAO analysis of the register sought to match the names in both the conflict of interest and confidentiality sections. The results identified 28 names in the confidentiality register not recorded in the conflict of interest register and eight names in the conflict of interest register not recorded in the confidentiality register. An analysis of the individual forms for the members of the governance arrangements (Figure 2.1) found one of Geoscience Australia’s staff had submitted signed, but not witnessed, confidentiality forms.

2.66 The Chair of the Tender Evaluation Board initially declared nil conflicts in August 2019. In October 2019, Geoscience Australia outlined in an email to Maddocks that LINZ had raised the issue of the impartiality of the Chair of the Tender Evaluation Board. In an email LINZ stated the Chair should ‘be a commercial advisor who can lead moderations and not be unduly influenced by past experience with suppliers, as is the norm in evaluation panels’ and ‘The Chair needs to be someone who is impartial to the outcome, and not had any previous involvement with suppliers, and be able to Chair moderations and not have their own view/bias’.

2.67 Maddocks responded to Geoscience Australia stating ‘We understand that LINZ is (or was) concerned that there may be a perception that [the individual] has a conflict of interest in circumstances where [they have] pre-existing relationships with some potential suppliers. This is not an unreasonable concern. There is likely to be an at least perceived conflict of interest as tenderers, external scrutineers and the public more broadly may consider that [the individual’s] experience and relationships may influence [their] ability to assess some potential suppliers’ tenders impartially’. The email also stated ‘We understand that since your email [Geoscience Australia] no longer proposes having [the individual] as the Evaluation Panel Chair’.

2.68 The individual advised that they believed they should be the Chair as they had the technical knowledge for the procurement. In the same email (October 2019) from Maddocks it was noted that ‘it would have been acceptable, from a probity perspective, for [the individual] to be appointed as the Chair’ provided the person ‘was fully aware of, and complied with, [their] obligations as set out in the SBAS Probity Framework; and [Geoscience Australia] took appropriate measures to address any potential or perceived conflict of interest on [the individual’s] part’. The individual was agreed as Chair by the CEO when the Tender Evaluation Plan was approved (see paragraph 2.2).

2.69 In the same email Maddocks had further stated ‘it is not possible to completely ‘remove’ a perception risk. However, it is possible to address the perception risks stemming from [the individual’s] perceived conflict of interest in the following ways…’. This included probity briefings, robust documentation of decisions to withstand scrutiny, independent monitoring/observation of all Tender Evaluation Board meetings and assessments by Maddocks (as the external probity advisor), having at least five members of the Tender Evaluation Board and the Chair not being the delegate, as ways to address this perception risk.

2.70 On receipt of the tender responses, the Chair of the Tender Evaluation Board declared their status from 2009 to 2017 as the Positioning Program Manager at the Cooperative Research Centre for Spatial Information (CRCSI) which became Frontier SI in 2018 (see paragraph 2.92). Geoscience Australia advised the ANAO that this role supported the CEO of Geoscience Australia as the Chair of the CRCSI Program Board. The declaration of the Chair of the Tender Evaluation Board also included their interactions with tendering organisations as project manager of the test bed, and then as project manager, in the early stages of the SouthPAN procurement. The Deputy Chair of the Tender Evaluation Board who was a LINZ employee did so at the same time, declaring a business relationship with Lockheed Martin, GMV, and Zeta Associates due to the SBAS trial (see Table 3.1) and that they had been ‘taken out for dinner by Lockheed Martin/GMV/Zeta on several occasions at conferences in 2018’. Maddocks was contacted by the internal probity officer for advice regarding this declaration and was advised that ‘any attempts by organisations (but particularly Tenderers) to communicate with [the individual] need to be declined, file noted and referred to you as the Contact Officer’. It was noted in the register ‘Maddocks notified of declaration and agreed that it’s of no concern. [The individual] has been made aware of their obligations under the Probity Framework, no further action required.’

2.71 The Compliance Screening Report (Stage 1 of the evaluation process, see paragraph 2.17) noted the inherent conflict of interest issues related to the existing relationship between the international consortium members delivering the SBAS trial and Geoscience Australia staff responsible for the procurement. The mitigation strategy outlined in the Compliance Screening Report, which was approved by the Tender Steering Committee on 5 August 2020, stated that all correspondence between Lockheed Martin Corporation and the Chair of Tender Evaluation Board had been lodged with Geoscience Australia’s internal probity officer. It also stated that information sessions held prior to the procurement which were chaired by [the Chair of the Tender Evaluation Board] had been conducted ‘under the direct supervision of a [Geoscience Australia] probity officer at all times’.

2.72 The identification of potential, perceived or actual conflict of interest should, in the first instance, lead agencies to seek to eliminate the conflict of interest. When this is not possible the Department of Finance recommends effective management strategies be implemented.³⁹

2.73 Geoscience Australia did not effectively manage the perceived conflict of interest in relation to an incumbent provider tendering for a new contract. It did not have: robust documentation of decisions to withstand scrutiny; independent monitoring/observation of all Tender Evaluation Board meetings and assessments by Maddocks (as the external probity advisor); and at least five members on the Tender Evaluation Board. It also did not address the perceived power imbalance of appointing an APS officer in the reporting line to the Tender Evaluation Board.

The procurement outcome

2.75 On the basis of compliance issues with the minimum service area requirement, the other tenderer’s rating for Functional and Performance Specification (Criterion 3) was reduced by five points (see paragraph 2.34). For the Demonstrated Experience (Criterion 4) rating, Lockheed Martin Australia was reduced by two points due to referee responses highlighting less organisational experience in delivering a certified SBAS (see paragraph 2.35). Table 2.9 outlines the final ratings.

Table 2.9: Tenderer results for the weighted criteria

Source: ANAO analysis of the Tender Evaluation Report and process.

2.76 In relation to the non-weighted criteria, the Tender Evaluation Board agreed that price was not a differentiating factor between the tenderers and overall, the Lockheed Martin Australia solution held ‘significant lower risk than the other tenderer’s solution’. In relation to Indigenous participation, the minutes state that ‘both the tenderers are equivalent’. The criterion of Economic Benefit was not included in the minutes of the Tender Evaluation Board’s second value for money discussion. Table 2.10 outlines the final ratings.

Table 2.10: Tenderer results for the non-weighted criteria

Note a: A rating for risk was not documented in numerical form.

Source: ANAO analysis of the Tender Evaluation Report and process.

2.77 Based on the value for money assessment, the Tender Evaluation Board recommended to the Tender Evaluation Steering Committee and the CEO, that Lockheed Martin Australia represented the best value for money at an acceptable level of risk and that the other tenderer represented value for money (but not the best value for money) at an acceptable level of risk. It recommended that Lockheed Martin Australia move to the contract negotiation stage but that the other tenderer should be set aside and only engaged should contract negotiations not succeed with Lockheed Martin Australia.

The complaint

2.78 Geoscience Australia reported that there was one complaint related to the procurement. The complainant, the other tenderer, as an unsuccessful tenderer was provided a debriefing session with Geoscience Australia on 18 October 2022. The other tenderer was not satisfied with the explanations provided by Geoscience Australia and subsequently outlined four areas of complaint in a letter dated 6 December 2022.

2.79 On 12 December 2022 Geoscience Australia emailed the other tenderer to confirm that it had received the complaint and its internal complaint handling processes had commenced. On 23 December 2022 Geoscience Australia informed the other tenderer that an independent review of the complaint was to be undertaken and the review findings would be assessed by an official who was the delegate of the CEO. Geoscience Australia emailed the other tenderer an additional three times to update the progress of the review. The third email dated 17 March 2023 reported that the review of the complaint had been completed by the Department of Defence and was being assessed by the delegate of the CEO.

2.80 The scope of the review undertaken by the Department of Defence addressed one of the four concerns listed by the other tenderer in its letter of complaint, which related to the CPR requirement (10.14) that all potential suppliers that are participating at the time be informed of the changes to the specifications and be provided with adequate time to modify and re-lodge their submissions. The other tenderer had been advised of changes on 23 June 2022 but the Department of Defence noted this related to Geoscience Australia querying whether this would impact the validity period of the other tenderer’s offer (to 30 June 2022) not as an invitation for it to resubmit its proposal in line with the amendments.

2.81 The review found that as the contract had not been awarded at the time of the amendment of specifications, the procurement was still being conducted. Further, as Geoscience Australia reserved the right to enter into negotiations with the other tenderer should negotiations with the preferred tenderer not succeed, this meant the other tenderer remained a potential supplier. However, it noted that there would not have been a change to the overall outcome of the tender evaluation process had the other tenderer been afforded the opportunity to resubmit its submission, as it related to price and the other tenderer had already been ranked ahead of Lockheed Martin Australia on price. The review also made recommendations for the three other concerns that were not in-scope for the review.

2.82 On 20 June 2023, Geoscience Australia wrote to the other tenderer and advised that ‘the [Defence] report was not based on a sufficiently comprehensive investigation and that a more detailed investigation and report should be undertaken’. Maddocks undertook this investigation and found that three of the concerns did not fall under Commonwealth Procurement Rule 6.9 (Judicial Review) as reasons for complaint. For the in-scope concern, the investigation concluded that Geoscience Australia had not contravened rule 10.14 of the CPRs as the other tenderer was not ‘participating’ in the procurement at that time as it had been advised it was not a preferred tenderer.

Will the outcome of the 2020 to 2022 SouthPAN procurement achieve value for money?

The contract negotiations

2.83 The Tender Evaluation Report was approved by the CEO on 17 September 2021 and the Contract Negotiation Plan on 8 October 2021. Both of these documents had been endorsed by the Tender Evaluation Steering Committee (see Figure 2.1).

2.84 The negotiation team included three officials from Geoscience Australia, two officials from LINZ and nine external advisors. The Chair and Deputy Chair were also the Chair and Deputy Chair of the Tender Evaluation Board. As outlined in the Contract Negotiation Report, meetings were held twice weekly from 11 October 2021 to 16 August 2022 with two in-person sessions. The Contract Negotiation Plan outlined that negotiations were expected to conclude with contract award by 31 March 2022. No reason was provided in the Contract Negotiation Report or the briefing to the CEO regarding the additional five months taken for the negotiations. Geoscience Australia advised that Lockheed Martin Australia was not willing to negotiate in the first instance.

2.85 Geoscience Australia did not undertake parallel negotiations with the other tenderer during this time. Geoscience Australia advised that the Tender Evaluation Report recommended to the CEO that this should only occur if negotiations with Lockheed Martin Australia broke down. It further advised that Geoscience Australia was not resourced to undertake parallel negotiations. The request for tender documentation stated ‘…Geoscience Australia may enter into negotiations with any one or more Tenderers’.

2.86 The objective of the negotiations was to formally agree to finalise and recommend for signature, a draft contract for the establishment of a prime contractor for SouthPAN. The Contract Negotiation Report provided a summary of the negotiation outcomes for issues identified as significant or very material. The issues were categorised into pricing, technical, and contracting/legal matters.⁴⁰ In relation to pricing, the Commonwealth negotiation team had authority to negotiate on cost (less than one per cent of the total, or $14 million), quality, schedule, and risk.

2.87 The result of negotiations was that the negotiation team achieved at least the minimum fallback position for each of the issues set out in the Contract Negotiation Plan except for an increase in pricing. During negotiations, Lockheed Martin Australia sought to increase its price after the expiry of the best and final offer period due to worldwide cost pressures from the COVID-19 pandemic and the war in Ukraine, asking for an additional $69 million. The contract negotiation team agreed to an increase of $32 million, which the CEO later approved.

2.88 In planning the procurement, Geoscience Australia sought advice from the Australian Government Solicitor (AGS) in relation to liability considerations of SBAS services. This included contractual liability with respect to end-users. As the service is being provided free of charge, Geoscience Australia is not entering into any contract with end-users, however the AGS advised that should SBAS services be extended to third parties through contractual arrangements (for example, mobile phone application providers) that Geoscience Australia should ensure contractual provisions are in place that address liability risks. For example, including clauses that would prevent the third party making claims on behalf of end-users.

2.89 Lockheed Martin Australia advised Geoscience Australia during the contract negotiations that it is not currently possible to obtain any insurance for non-aviation end-user liabilities, and for aviation events a maximum of USD$500 million per annum in insurance is available. Lockheed Martin Australia agreed to be liable for an amount of AUD$60 million for both aviation and non-aviation events, as well as providing an additional USD$500 million in insurance coverage for aviation events. On 13 September 2022 the CEO approved acceptance of liability⁴¹ above AUD$60 million for non-aviation events and above the combined amount of AUD$60 million and USD$500 million for aviation events, effectively transferring this risk to the Commonwealth.⁴²

2.90 The contract provides for third party and commercial use of the system by the contractor, subject to Geoscience Australia and LINZ approval. The Contract Management Plan notes that the terms of commercial use would be separately negotiated.

The SBAS trial economic benefits analysis report

2.92 In 2017 Geoscience Australia and LINZ contracted the Cooperative Research Centre for Spatial Information (now Frontier SI) to run an SBAS test-bed. Frontier SI noted the project was relatively short, with signals available for testing between October 2017 and January 2019. There were two objectives: user testing by selected industry sectors⁴³ in Australia and New Zealand; and an economic benefits study of the SBAS technology.

2.93 The technical report outlined that the three signals⁴⁴ performed as expected under good conditions, with the following factors adversely affecting positioning performance:

• obstructions from buildings, trees, and machinery (on construction and mining sites) which limit the number of GNSS satellites that can be observed, and in some cases blocking the correctional signal from the SBAS satellite;
• the quality of the receiver — consumer grade receivers and antenna hardware gave no extra benefit, but the use of mid-range equipment (cost between $2,000 and $3,000) gave significant accuracy improvements; and
• there were a number of unplanned signal outages/disruptions that varied in length from a few hours, through to days and as long as a month, with the test-bed taken offline with little or no notice.

2.94 Frontier SI commissioned Ernst and Young to assess the economic benefits to Australia and New Zealand.⁴⁵ The SBAS Trial Economic Benefits Analysis Report (the report) was delivered in June 2019 and estimated the economic benefit across the 10 industry sectors to be $7.6 billion. The estimated benefits to industry were calculated in Australian dollars and projected out 30 years. To take into account the falling purchasing power, or the reducing value of the Australian dollar due to the effects of inflation, the report modelling discounted the estimated returns by 6.5 per cent.⁴⁶ The report outlined that 82 per cent, or $6.2 billion, of benefits were anticipated to flow to Australia with 18 per cent, or $1.4 billion, to New Zealand.

2.95 The report outlined that 44 economic benefits from quantitative data across nine sectors and 30 economic benefits from qualitative data across 10 sectors were identified. These benefits were included in the report and describe various applications of the technology. Examples include: virtual fencing for strip grazing of herds; tracking feeding zones for pasture management; precision spraying of crops; personnel safety on mine and construction sites through accurate location tracking of workers and vehicles/heavy machinery; safer navigation for cargo ships and the ability to track container movements; advanced train management systems; track surveys; regulatory vehicle speed determination; and cooperative intelligent transport systems.

2.96 The estimations of the future economic benefits rely on an assessment of the uptake of the SBAS services. The report stated that a ‘challenge associated with transformative technologies is attempting to understand uptake rates’. To assist in understanding the rate of adoption by sectors (and incorporate into the modelling) Ernst and Young consulted sector experts and undertook research of similar transformative technologies to create the uptake (or new technology adoption) curves. The report did not include sector specific uptake curves (or the data underpinning them), nor did Ernst and Young provide this information to Geoscience Australia. In an internal briefing to the Positioning Program Board (see paragraph 2.5) on 16 July 2021, it was noted that the project management office ‘spoke to [Ernst and Young] and no additional information can be provided due to non-disclosure agreements signed with stakeholders who provided input information into the analysis. [Ernst and Young] confirmed stakeholders were cautious about providing information and would not agree to provide underlying data’.

2.97 Twenty-seven industry participants from the 10 industry sectors were successful as part of an expression of interest to be part of the test-bed. Each participant tested the SBAS signals and evaluated their own economic benefits. The report states ‘Each Demonstrator Project was different. For example, some Demonstrator Projects started testing as early as October 2017 or as late as February 2019 and project duration ranged from a few months to 12 months. Some projects carried out extensive testing, while others carried out enough testing to be able to demonstrate the applications relevant to the economic benefits discussed in this report’.

2.98 As outlined in Figure 2.2, the greatest number of participants were from the agriculture sector, with a total of seven. Road and spatial were next with four each, maritime had three while the remainder had two or one. The participants represented private sector, public sector and tertiary education organisations. The report did not address issues of the representative or generalisable appropriateness of the participant projects.

Figure 2.2: Number of participants by industry sector

Source: ANAO analysis of the SBAS Trial Economic Benefits Analysis Report.

2.99 The report outlined the analytical approach to assessing the economic benefits was to:

• quantify the benefits anticipated for each of the 27 industry participant projects;
• scale-up the benefits identified in each project to the sector level (and apportion to Australia and New Zealand); and
• sum all the benefits to provide an estimate of the potential economic benefits.

2.100 The report did not identify any assessment of the costs that may offset the benefits, such as negative impacts to service reliability, including:

• the end-of-life⁴⁷ Inmarsat 4F1 satellite used by SouthPAN;
• the downtime related to solar activity or other external threats; or
• the competition and disruption from current and future commercial operators in Australia providing SBAS services.

2.101 Ernst and Young included a number of disclaimers in the report. This included that: it had been prepared for Frontier SI for the purpose of release into the public domain for informational purposes only; that Ernst and Young ‘disclaims all liability in relation to any other party who seeks to rely upon the Economic Benefits Report or any of its contents’; and that Ernst and Young ‘makes no representations as to the appropriateness, accuracy or completeness of the Economic Benefits Report for any other party’s purposes’.

2.102 The report did not address the sample size of each sector (see Figure 2.2) and how representative this was of each sector. It also did not include a cost-benefit analysis. These limitations were not communicated to the Australian Government when funding decisions were made.

3.1 The Department of Finance (Finance) notes that ‘good contract management is an essential component in achieving value for money for Australian Government procurements’.⁴⁸ To ensure SouthPAN is implemented as expected and attains full operating capability by late 2028, it is critical that Geoscience Australia manages the contract effectively.

Is there a fit-for-purpose contract management framework in place?

The contract

3.2 The contract with Lockheed Martin Australia was executed on 13 September 2022 and commenced on 20 September 2022. It expires on the 19^th anniversary of the commencement date. Geoscience Australia advised that the 19 years represents the four-year build program plus the 15 year life-cycle of a satellite. Under the Master Delivery Program outlined in the contract, the approval for safety-critical services for the first satellite is due on 22 May 2028. Approval for safety-critical services for the second satellite is due on 8 October 2029.

3.3 The Chief Executive Officer (CEO) approved the financial commitment of the contract costs as being $1.280 billion including capital, CPI and foreign exchange (assuming forecast CPI and foreign exchange rates) excluding GST.⁴⁹ The CEO was briefed that the value of $1.183 billion would be published on AusTender and used in all public communications as ‘AusTender stipulates the value is published excluding indexation but with GST’.

3.4 Finance’s guidance on reporting contract amounts states ‘For each contract reported, the relevant entity must report the total value of the initial term of the contract (including GST where applicable). This does not take into account the value of any options, extensions, renewals, or other mechanisms that may be exercised at a future date to increase the value of the contract’. There is no reference to not including indexation or CPI in the guidance. The reference to Finance’s guidance was provided by the SouthPAN internal probity advisor but was not acted upon by Geoscience Australia.

3.5 Figure 3.1 outlines the projected cost of the contract as at 30 June 2023 against the procurement pricing bid of Lockheed Martin Australia (see paragraph 2.39). It includes payments for achievement of milestones (see paragraph 3.38), the monthly service fee, pass through costs⁵⁰ and the build of the Global Navigation Satellite Systems (GNSS) reference stations.

Figure 3.1: Current projected costa of the SouthPAN contract as at 30 June 2023 compared with Lockheed Martin Australia’s procurement pricing bidb

Note a: Includes foreign exchange, CPI and GST.

Note b: Includes foreign exchange and CPI and excludes GST.

Source: ANAO analysis of Geoscience Australia data.

3.6 The contract states that Lockheed Martin Australia is a wholly-owned subsidiary of Lockheed Martin Corporation (LMC), which is a publicly listed company in the United States of America (USA). LMC is subcontracted by Lockheed Martin Australia to deliver the activities with the work undertaken in the USA. LMC has subcontracted Zeta Associates (also a wholly-owned subsidiary of LMC based in the USA which reports to LMC’s space systems business area) and GMV, a Spanish company. Table 3.1 outlines the arrangements.

3.7 Should a dispute arise that is not able to be resolved through the dispute resolution representatives, then the parties have recourse to the Australian Centre for International Commercial Arbitration under the International Arbitration Act 1974 (Cth).

Table 3.1: Contractor and sub-contractor structure

Note a: Activities means all things the contractor is, or may be, required to do to comply with and complete its obligations arising from or in connection with the project documents, including performing the work and providing the services.

Source: ANAO analysis of the contract with Lockheed Martin Australia.

3.8 Under the contract, the role of Lockheed Martin Australia is primarily to coordinate and manage SouthPAN so that the system is interoperable and in accordance with project documents, all laws, best industry practices, and all applicable standards. The subcontract between Lockheed Martin Corporation and GMV was signed on 7 December 2022 and the Key Subcontractor Direct Deed was executed by Lockheed Martin Corporation, Lockheed Martin Australia and Zeta Associates on 5 December 2022.

3.9 The contract does not include the accuracy requirement of 10cm, as has been referred to in public communications, but rather the program specific policy objective of the provision of 15cm accuracy positioning. Geoscience Australia advised that its ‘public communication on SouthPAN performance describes indicative accuracy in simplified terms’ and that ‘SouthPAN signal performance is expected to achieve accuracies of ‘down to 10cm’ (or better) for a percentage of time, which is characterised for technical purposes through the 15cm 2 sigma precision specification’.⁵¹

3.10 As the contract value was more than $500 million, Lockheed Martin Australia was required to prepare an Australian Industry Participation (AIP) Plan.⁵² The AIP Plan is part of the contract and outlines how Australian industry will be given a full, fair and reasonable opportunity to participate in Australian Government funded projects.⁵³ Opportunities are required to be published on a nominated website, which was the Industry Capability Network. As at October 2023, there were a range of ‘work packages’ advertised by Lockheed Martin Space on this website.⁵⁴

3.11 With the award of the contract, Geoscience Australia and LINZ agreed the terms of an establishment and sustainment ‘on-supply’ agreement. This agreement was executed on 13 September 2022 and sets out how SouthPAN will be implemented and operated in New Zealand. Geoscience Australia will on-supply the services provided by Lockheed Martin Australia to LINZ. While the procurement process was led by Geoscience Australia, the implementation of SouthPAN is done jointly by the two entities, noting Lockheed Martin Australia is only contracted to Geoscience Australia.

The contract management plan

3.12 Finance’s Contract Management Guide⁵⁵ provides guidance regarding the four phases of the procurement life cycle. The phases include activities and considerations that assist effective contract management. The ANAO assessed whether Geoscience Australia’s Contract Management Plan followed this guidance as it relates to the planning and start-up phases only, noting SouthPAN is in the start-up phase.

3.13 The SouthPAN project is a high value and highly complex procurement. Geoscience Australia has developed a range of plans (in addition to its Contract Management Plan) to help it effectively deal with the greater risk due to the complexity. In addition to the Contract Management Plan which was approved on 14 April 2023 (with an updated version approved on 10 August 2023), the assessment also included the:

• SouthPAN Risk Management Plan;
• Stakeholder Engagement and Communication Plan;
• Project Management Plan; and
• Security Certification and Accreditation Plan.

3.14 Geoscience Australia’s Contract Management Plan complied with Finance’s Contract Management Guide.

The risk management plan

Risk management during the procurement

3.15 An internal audit undertaken in March 2020⁵⁶ noted that there were risk registers in place for the Satellite-Based Augmentation System (SBAS) and National Positioning Infrastructure Capability (NPIC) programs but that Geoscience Australia had not comprehensively identified risks to project-specific objectives, deliverables and outcomes. An example provided in the internal audit report was the lack of a risk analysis of risk events that could impact on achieving the accuracy and integrity of SBAS (SouthPAN) signals, noting that this ‘impacts the safe use of this data by the aviation industry’.

3.16 The Risk Management Plan for the procurement was approved by the Joint Board of Management on 11 June 2020. It outlined that risk likelihood and consequences would be assigned based on Geoscience Australia’s corporate risk matrix as built into the corporate risk register template. The standard risk categories were security, work, health and safety, finance/resources, reputation, compliance and program delivery. Geoscience Australia also identified specific risk categories for the procurement, which were cost/budget, schedule, stakeholders, technical, environmental, operational safety, and the Australia–New Zealand partnership. For both sets of risk, tolerances and consequences were identified in the risk management plan.

3.17 In July 2020, Geoscience Australia undertook a risk assessment for the procurement and identified 30 risks. As at July 2021, Geoscience Australia’s SouthPAN risk register contained eight active risks. None of the eight risks were from the procurement risk assessment.

3.18 The approved Risk Potential Assessment Tool⁵⁷ that Geoscience Australia submitted to Finance, rated the SouthPAN project as medium risk both before and after mitigation. Projects with: a total estimated cost of $30 million or more for procurement or infrastructure; a total of $30 million or more including an Information and Communication Technology component of at least $10 million; or programs with a total estimated cost of over $50 million that have a risk rating of medium or above may be subject, on the recommendation of Finance, to the Gateway Review Process.⁵⁸ SouthPAN did not undergo a Gateway Review Process.

3.19 Geoscience Australia advised that it met with Finance several times and held the view that the SouthPAN project was not subject to the Gateway Review Process as it was ‘not an Information Technology procurement’. Geoscience Australia did not have minutes of the meetings with Finance. The Department of Industry, Science and Resources which assisted Geoscience Australia with the submission to the Australian Government, advised that minutes of meetings with Finance were not taken. No comment on the risk assessment was made by Finance in its briefing to the Australian Government on the proposal.

3.20 In August 2022, an internal review found that while the risk approach had been adequate for the procurement to date, it had not identified issues or opportunities associated with implementation and ‘is cast within a risk management framework that is not designed to deal with projects of the nature of SouthPAN’. Geoscience Australia advised that it ‘struggled with the procurement risk assessment process as Geoscience Australia’s risk framework was not scalable to such a large and complex procurement’. The internal review recommended Geoscience Australia develop a fit-for-purpose approach to SouthPAN risks and issues management, with agreed risk appetite and escalation triggers. Finance states that ‘an entity’s risk management approach must be regularly reviewed’.⁵⁹

Risk management approach to the contract

3.22 The minutes of the October 2022 Joint Governance Board of Management meeting state that there had been significant work on the risk register. In December 2022, the Joint Governance Board of Management: approved the Risk and Opportunity Management Plan (which replaced the Risk Management Plan); noted the SouthPAN integrated risk register had been updated to reflect project risks, issues and opportunities; and accepted that certain strategic risks would be referred to the Joint Governance Board of Management.

• The Risk and Opportunity Management Plan outlines roles and responsibilities, the risk management process and tools. It also documents the roles of the risk owner and risk manager and how risk will be escalated within Geoscience Australia. Geoscience Australia uses the Risk and Opportunity Management Plan to monitor and review identified and emerging risks and opportunities. For risks, there are eight information requirements, which are to be mapped to priority, likelihood, consequence and treatment option. For risks being treated, a status must be assigned. The cost of the risk and the potential impact on the project overall is also required. For opportunities, there are eight sub-components for identifying opportunities and supporting options for decision.
• The SouthPAN integrated risk register contains risks, issues and opportunities. It also contains sub-registers relating to components of the project not covered by the SouthPAN contract. Each risk has been assigned a risk level, a treatment action plan, a risk status before and after mitigation and the board/committee responsible for the risk level.

3.23 The Technical Director Committee met with the Operational Management Committee (see Figure 3.2) to discuss and action a range of matters between October 2022 and June 2023. During the same period, the Joint Governance Board of Management provided six directives to the Committees in response to matters that were escalated to them.

Is the contract being managed effectively?

Administrative arrangements for contract management

3.24 The governance structure for SouthPAN business-as-usual is set-out in Figure 3.2. Since the SouthPAN contract with Lockheed Martin Australia was signed on 13 September 2022, the Joint Governance Board of Management, Operational Management Committee, and Technical Director Committee meetings have followed a schedule as described by their respective Terms of Reference.

Figure 3.2: SouthPAN governance structure

Source: ANAO Analysis.

3.25 The SouthPAN Delivery Manager is responsible for leading the team that will deliver SouthPAN. The role is supported by the Technical Director Committee and the Operational Management Committee. The Technical Director Committee is accountable for directing, managing and overseeing the implementation. The Operational Management Committee is responsible at the strategic and operational levels and reports to the Joint Governance Board of Management.

3.26 The three support structures that report to the SouthPAN Delivery Manager cover engineering, contract management and project support. The engineering section is accountable for the integrity of SouthPAN and the services it provides. Figure 3.3 outlines areas of responsibilities.

Figure 3.3: SouthPAN systems engineering section functions

Source: Geoscience Australia’s Systems Engineering Management Plan.

3.27 The Contract Manager role is responsible for day-to-day management of the SouthPAN contract administration, including coordination and reporting on contract status and progress activities. The project support section undertakes project planning, execution, assessment, and control (including reporting). It also supports quality, transition, stakeholder engagement, risk and communications management.

Managing the contract

3.28 The SouthPAN Delivery Manager is responsible for managing the contract. It includes provisions for technical reviews to verify that Lockheed Martin Australia has achieved milestones, can claim payment and is adequately prepared for the following phase of work. It also includes provision for quality audits to verify that Lockheed Martin Australia is delivering on contractual obligations. The contract also includes remediation mechanisms, including defect rectifications and warranty claims.

3.29 Under the contract, the Contract Status Report is the principal reporting product required each month. It provides updates on progress, planned activities, problems, risks and issues. The contract states that the Contract Status Report is subordinate to the Project Management Plan and the AIP Plan. The Contract Status Report is inter-related to the Contract Master Schedule and all other reporting required under the contract. The contract outlines twelve pieces of information that the Contract Status Report must include. A Contract Status Report has been delivered and presented each month since December 2022 to December 2023. All requirements for reporting were met.

3.30 With respect to Lockheed Martin Australia’s plan for the employment of at least two Indigenous employees at the Uralla facility (see paragraph 2.36), Geoscience Australia advised that Lockheed Martin Australia had commenced recruitment for one network engineer at Uralla, most likely to be sourced from Melbourne, and ‘that Lockheed Martin Australia is not required under the Contract to report how many Indigenous Australians are hired and have provided no information to-date’. Under the contract, Lockheed Martin Australia must submit a written report⁶⁰ to Geoscience Australia on its compliance with its reasonable endeavours to purchase from Indigenous enterprises and its employment of Indigenous Australians. This report was provided by Lockheed Martin Australia in the June 2023 Contract Status Report, which noted that the individual recruited at Uralla station for SouthPAN identifies as an Indigenous resource and that sub-contractors have been directed to engage with Indigenous enterprises.

3.31 In tandem with the contract progress meeting, where the Contract Status Report is discussed, engineering meetings occur. While separate, the meetings generally occur on or around the same day. Under the contract, Lockheed Martin Australia is required to convene both these meetings. There were monthly meeting records for both the contract progress meetings and SouthPAN engineering meetings which commenced in December 2022 and February 2023, respectively. Geoscience Australia was actively involved in the presentation of the required reporting at the contract progress meeting that the ANAO attended, and challenged Lockheed Martin Australia and GMV on several points for follow-up.

3.32 The contract sets out that Lockheed Martin Australia is to establish, coordinate and participate in five contract working groups. These are the interface, site installation, systems engineering, system safety and security working groups. The five contract working groups generally meet every one to two months.

3.33 Geoscience Australia, LINZ, Lockheed Martin Australia, Lockheed Martin Space Systems, Zeta Associates and GMV are members of the contract working groups. Airservices Australia and Airways NZ are members of the interface and system safety working groups. The Australian Civil Aviation Safety Authority and the New Zealand Civil Aviation Authority are also members of the system safety working group.

3.34 The government-only meetings that support the ‘SBAS for aviation’ component are the technical interchange meetings held weekly and the 6-Party meetings which are held once or twice per year. Geoscience Australia, LINZ, Airservices Australia, Airways New Zealand, the Australian Civil Aviation Safety Authority and the New Zealand Civil Aviation Authority attend these. The technical interchange meetings are to coordinate aviation certification activities and the 6-Party meetings discuss how SouthPAN will be adopted in the aviation industry.

Reporting and payments

3.35 Under the contract, Lockheed Martin Australia’s performance is measured against 45 key performance indicators (KPIs) and must provide reports on those KPIs in the monthly Contract Status Report (see paragraph 3.29). Thirty-nine KPIs are measured by a pass or fail and eight KPIs are subject to a scaled performance measure. If a KPI is not met, the contract requires four actions by Lockheed Martin Australia and an abatement may apply. Abatement amounts can be applied monthly or annually.The contract allows Geoscience Australia to reduce the monthly abatement amount with sole and absolute discretion.

3.36 For KPIs which require the system to be operational for 12 months to accumulate a data set sufficiently large to calculate the KPIs accurately, no abatement is applied during that time. However, in the first month that the performance data for the full reporting period (i.e. 12 months) is available, the abatement will be applied against the full data set and applied retrospectively to what has been paid for that service over the immediate preceding 11 months.

3.37 There are two payment types under the contract, the first being the contract price which is comprised of 20 Milestone Payments. The second is the monthly service payment based on the 45 KPIs. Under the contract, both payments are subject to indexation, escalation, adjustment or revision in specific circumstances. Abatements may occur if the quantum and quality of the services provided do not meet that set out in the performance section of the contract. The abatement amount is applied as outlined in Table 3.2.

Table 3.2: How contract payments are impacted by performance

Source: ANAO analysis of the contract with Lockheed Martin Australia.

3.38 The contract has 20 milestones which are scheduled to be delivered between 2022–23 and 2034–35. The first three milestones have been delivered and as at June 2023 the project is on schedule.

Figure 3.4: Milestone delivery by year

Source: ANAO analysis of the contract with Lockheed Martin Australia.

3.39 There are a range of tools that are used by Geoscience Australia to manage the delivery of milestones, including the SouthPAN integrated master schedule, the data item review register and specific sub-project trackers.

3.40 To support SouthPAN remaining technically current over the 19 years of the contract, the last four milestones (17 to 20) facilitate a ‘mid-life technical refresh’ commencing 9.5 years from the contract start date through to 12.5 years from the contract start date. The purpose of the technical refreshes are to:

• ensure that items at the end of their useful life cycle are replaced so that the system remains optimised throughout its life cycle;
• allow advances in technology to be incorporated to support growth, evolution and obsolescence risks, issues, and opportunities; and
• allow the achievement of performance requirements through an appropriate balance of item reliability, availability, maintainability and life cycle cost.

3.41 Receipt of required documents and reports is formalised and each deliverable aligns with document due dates enabling Geoscience Australia to follow-up if items are late. When there was a delay of two months by Lockheed Martin Australia in relation to the third milestone, Geoscience Australia managed this according to the contract. There is a configuration management library with clear revision controls. The configuration management library also has a full audit trail available for review. Obsolete documentation is not deleted but retained in an archive.

3.42 Geoscience Australia has an invoicing flowchart that is consistent with the contract and sets out how reporting flows through to payment. This process was adhered to for the three milestone payments and the six monthly service payments paid in the 2022–23 financial year. All payment certificates issued by Geoscience Australia included advice to Lockheed Martin Australia that payments may subsequently be subject to abatement in line with the contract. For the 2022–23 financial year, all payments were made on time therefore no penalty interest was applied.

Workforce planning

3.43 As the contract is for 19 years, it is important Geoscience Australia has effective workforce planning in place.⁶¹ Geoscience Australia has developed documentation to support project team staff in their management of the SouthPAN contract and its deliverables.

3.44 An internal review (see paragraph 3.20) of the SouthPAN project was undertaken in August 2022. The objective of the review was to assess the project’s ‘status, outlook and readiness to complete Stage 2 – Procurement, and progress onto Stage 3 – Execution.’ The report included the following findings specific to project management and resourcing:

• the project team was experiencing capacity and capability shortfalls;
• the level of maturity of documentation being less than would be expected at this stage, with some key documents not yet delivered; and
• a project of this size, complexity and nature coincided with a lack of clarity in the team’s roles, responsibilities and authority in regard to decision making.

3.45 The internal review recommendations linked to the project management and resourcing findings included to develop a workforce plan, induction package and develop a continuous recruitment model.

3.46 Between October 2022 and March 2023, the monthly meetings of the Joint Governance Board of Management resulted in:

• acknowledgement of the need for a resourcing plan;
• directing the Operational Management Committee to report on progress of the implementation of the internal review recommendations;
• noting the staffing issues, current staffing levels and details of staff vacancies; and
• receiving an update on the development a workforce plan, induction package and management of the resourcing levels of the project team.

3.47 As at June 2023, Geoscience Australia has developed a resourcing plan which includes an induction package. It has also developed a hierarchical structure in the project team with the aim of supporting retention of corporate knowledge for the project’s key roles heading into the sustainment phase of the project.

SouthPAN security reviews

3.48 System security was assessed in the context of external data, such as the privacy of end-users, open data and the implications of using real-time location services. Geoscience Australia has a range of security services in place as part of its gateway protection and managed service arrangements. Geoscience Australia also works closely with the Australian Cyber Security Centre. A range of security reviews have been completed successfully for the current services, with additional reviews to be conducted as more services come online.

3.49 A Privacy Impact Assessment was performed for SouthPAN during planning in December 2020. It identified that as SouthPAN would be an open and free service, where users are not required to register and no personal information is collected or handled as part of providing SouthPAN services. Due to being a broadcast service only, SouthPAN complies with the Privacy Act 1988 (Cth).

3.50 Geoscience Australia’s Chief Operating Officer granted SouthPAN authority to operate for a period of two years from 8 September 2022. Lockheed Martin Australia owns the uplink facilities and is building all the GNSS reference stations (except for two which will be positioned in Antarctica). Once built, if the ground stations do not attain authority to operate, milestone payments can be withheld under the contract.

3.51 Geoscience Australia advised the Australian Government in September 2021 that funding had been allocated for the Security Construction and Equipment Committee to audit the uplink and GNSS reference stations. The Security Construction and Equipment Committee is an inter-departmental committee that evaluates security equipment for use by Australian Government departments and agencies.⁶² Geoscience Australia advised that it will appoint ‘a Security Construction and Equipment Committee (SCEC) consultant to fulfil the role of SCEC Agent defined in the Statement of Requirements. The SCEC consultant [will provide] reports of its findings to [Geoscience Australia], LINZ and the Contractor. The physical security audits will commence late 2024 (September) and [are] scheduled to be finalised in mid-2026’.

4.1 Effective performance monitoring and reporting provides a clear line of sight between planned and actual performance, while evaluation can help to confirm that expected results are being achieved. To ensure the expected economic benefits to Australia are realised, Geoscience Australia needs to ensure that the services SouthPAN delivers meet agreed performance standards.

Is there a fit-for-purpose performance management framework in place?

The contract performance framework

4.2 The SouthPAN contract addresses performance matters in three schedules relating to milestones, payments and requirements. The general and industry objectives, as well as the three service types, and the benefit realisation approach accurately reflect the request for tender documentation.

4.3 Under the contract, Lockheed Martin Australia’s performance is measured against 45 key performance indicators (KPIs). Thirty-five of the KPIs relate to 10 end-user services (horizontal and vertical accuracy, availability, continuity, integrity, containment and/or latency) while the other 10 relate to five support services to be provided to Geoscience Australia. In relation to corporate reporting, the KPIs under the contract will support the corporate outputs performance measure of: ‘availability of positioning services for precise positioning as percentage of time’ (see Table 4.1).

4.4 The contract includes 10 program specific policy objectives of SouthPAN and states achievement of the objectives will be met by the provision of the three service types⁶³ and measured by:

• monitoring cost and schedule tolerances;
• an assessment of uptake in users, services and technology enabled by SouthPAN in the years following establishment; and
• achieving planned objectives.

4.5 Geoscience Australia advised that as the services SouthPAN provides are one-way signals (such as a radio station or TV signal), it is not able to identify who is using the service, which service they are using, where they are using it, when they are using it or for what purpose. This means Geoscience Australia will be unable to assess the uptake in users, services and technology enabled by SouthPAN. Geoscience Australia also has no methodology to measure achievement of the following objectives in the contract:

• positively contributing to the aviation, maritime, road, rail, agriculture, construction, resource and utility sectors, providing a capability to match what is available in the northern hemisphere;
• positively impacting all users of satellite positioning, particularly citizens in regional and remote areas without mobile phone coverage; with the provision of 15cm accuracy positioning;
• improving the safety and efficiency of aviation, maritime navigation, rail operations, roads; and
• improving mining and resources operations and benefiting the agriculture sector.

Performance data

4.6 The current SouthPAN services are from the original test-bed system, with operational capability providing L1 SBAS, Dual Frequency Multiple Constellation (DFMC) SBAS and Precise Point Positioning (PPP) signals. Thirty-nine existing Continuously Operating Reference Stations from the National Positioning Infrastructure Capability (NPIC) program are used for L1 SBAS accuracy, 49 Continuously Operating Reference Stations for DFMC accuracy and 10 for PPP accuracy. As implementation progresses, SouthPAN will primarily rely on 35 new Global Navigation Satellite Systems (GNSS) reference stations being built under the contract that will provide safety-critical capability. Figure 4.1 provides the indicative locations of the new stations.

Figure 4.1: SouthPAN’s 35 dedicated GNSS reference stations (indicative)

Source: Geoscience Australia.

4.7 For current KPI performance measurement (see paragraph 4.3), data is collated from the real-time GNSS data streams from the Continuously Operating Reference Stations and the PositioNZ Global Positioning System data service which provides data from continuously recording GPS receivers located around New Zealand and the Chatham Islands. The SBAS signal in space and data over the internet are also used.

4.8 The Systems Engineering Management Plan sets out quality register and quality assurance guidelines for the project. It also incorporates a KPI validation toolset, to validate that the contractor continues to meet service performance KPIs. Geoscience Australia advised that as ‘SouthPAN is currently providing early Open Services, Geoscience Australia/LINZ are [in] the process of building and validating our own service verification tool set and developing a baseline’.

Internal performance reporting

4.9 Internal reporting is supported by the project support section. Agreement by the Technical Director Committee (refer to Figure 3.2 earlier in the report) to progress updates is provided in weekly meetings. These meetings are not minuted. Terms of Reference for the Technical Director Committee that were provided to the Operational Management Committee for noting at the May 2023 meeting outline that there is a standing agenda item of a project update and discussion of the schedule. The SouthPAN Project Status Report was first presented to the Operational Management Committee in the January 2023 meeting and includes information on project status, operational status, the schedule, achievements and planned outcomes, significant issues, risk and budget. It has since been provided at each monthly meeting to October 2023.

4.10 The Joint Governance Board of Management (see Figure 3.2) receives the SouthPAN Project Status Report in dashboard form at its quarterly meetings. It was first provided to the board in December 2022, where the dashboard format was endorsed.

4.11 Geoscience Australia has developed a single measure for reporting on the SouthPAN expected economic benefits of $6.2 billion to the Australian economy over the next 30 years called estimated direct economic benefits as a result of SouthPAN signals ($ millions, present value). It commenced on 26 September 2022, when the SouthPAN signals began under the contract. Geoscience Australia has an endorsed Benefits Management Plan which states that reporting against the measure will: ‘Be performed by Project teams already monitoring leading indicators monthly as part of their project management’ and ‘Report estimated benefits achieved annually to Branch Head and governance boards through [Geoscience Australia’s] annual performance reporting’; and ‘The results can be used in annual report, media releases, [Geoscience Australia’s] website, Senate Estimates briefs, updates to the Secretary and Minister etc’.

4.12 As the SBAS Trial Economic Benefits Analysis Report projected benefits over 30 years and the contract is for 19 years, the Benefits Management Plan states that it is assumed that the benefits achieved will be consistent across each year commencing 2023–24,⁶⁴ but the $12 million per annum benefits for aviation will not commence until the safety-of-life certification has been achieved. This estimates that $203 million of benefits per year will be achieved from 2023–24, increasing to $215 million from 2028–29.

4.13 Geoscience Australia’s methodology further assumes that if 95 per cent availability of SouthPAN services is not achieved in a given month, then economic benefits from SouthPAN will be reduced. To account for the loss of service, for each month where availability is not achieved, it is assumed 10 per cent of the new benefits for that month would not be realised. This has been simplified to $2 million of new annual benefits being lost.

4.14 Geoscience Australia advised that as the SBAS Trial Economic Benefits Analysis Report provided no uptake data, the assumptions were identified for the purpose of using a linear shaped uptake curve and to delay the benefits in the aviation sector until safety-of-life services are certified.

4.15 The adapted methodology does not address the issue that the SBAS Trial Economics Benefits Analysis Report assessed the services provided by the test-bed, not the new system which is still being built and is contractually required to provide accuracy to 15cm (2 sigma precision specification) not 10cm (see paragraph 3.9).

Is the performance of SouthPAN being managed effectively?

Monitoring performance

Complaints from end-users

4.17 The Explanatory Statements for the Industry Research and Development (National Positioning Infrastructure Capability Program) Instrument 2018⁶⁵ and the Industry Research and Development (Satellite-Based Augmentation System Program) Instrument 2018⁶⁶, state that people who have complaints about the programs, which includes the provision of Positioning, Navigation and Timing data, have recourse to Geoscience Australia. The Explanatory Statements set out that Geoscience Australia will investigate any complaints about the program in accordance with its complaints policy and procedures.

4.18 Aside from Geoscience Australia’s standard complaints process, which is outlined on its website⁶⁷, Geoscience Australia advised that there is no specific complaints process or procedure currently in place and that no complaints from end-users in relation to SouthPAN services have been received. Geoscience Australia also advised that it had received calls during the outage of the Inmarsat 4F1 satellite in April 2023, which impacted services across Australia, but did not have a record of those calls or how they were handled.

4.19 The Commonwealth Ombudsman’s Better Practice Complaint Handling Guide states that expectations of complaint handling systems have grown. Complaints are also a valuable source of information regarding the performance of services experienced by end-users.

Stakeholder engagement

4.22 A key risk advised to the Australian Government was that the economic benefits of SouthPAN may not be fully realised if uptake was low. Geoscience Australia advised it would be proactively communicating with end users to facilitate awareness and encourage the use of SouthPAN services.

4.23 Geoscience Australia has an Engagement and Communication Strategy (and an implementation plan) that was approved by the Operational Management Committee on 30 November 2022 before being provided to the Joint Governance Board of Management for noting. In the out-of-session minutes, the board requested that a short update be provided to each meeting through the SouthPAN Status Report. The SouthPAN Status Report was recorded in the 20 March 2023, 13 June 2023 and 13 October 2023 meeting minutes.

4.24 The Engagement and Communication Strategy sets out four objectives for communication and engagement:

• promote awareness of SouthPAN and its applications to encourage uptake of the services by a broad range of potential users⁶⁸;
• increase and maintain stakeholder support for the objectives of SouthPAN;
• ensure accuracy of information is disseminated through public channels; and
• build a positive reputation through the delivery of timely, responsive, accessible channels for comments and information about SouthPAN.

4.25 The Engagement and Communication Strategy outlines a stakeholder analysis⁶⁹, with a corresponding categorised⁷⁰ approach to communication engagement. Performance against the Engagement and Communication Strategy will be assessed by two performance measures.

• The first measure is to increase awareness and understanding of SouthPAN measured by: data about the number of media articles; the number of positive media articles; increased SouthPAN webpage visits; increased SouthPAN queries; social media engagement above 3.5 per cent; and strong attendance at events.
• The second measure is to enhance and encourage users to be adopters of SouthPAN, measured by: an increase of devices with access to SouthPAN; increased use of SouthPAN services; increased SouthPAN informational downloads; government departments, peak bodies and associations communicate to their industries; new products and services using SouthPAN appearing in the market; case studies of end-users; and estimation of achievement of economic benefits.

4.26 The Department of Finance notes that performance measures must be reliable and verifiable.⁷¹ Three data sources will not accurately measure the second measure to enhance and encouraged users to be adopters of SouthPAN. Mobile phones are one type of device that can provide access to SouthPAN services but an increase in the purchase of mobile phones cannot be linked to the use of SouthPAN. Geoscience Australia has advised there is no way to track the end-use of the services, this means an increased use of SouthPAN services cannot be measured. The Benefits Management Plan will estimate potential economic benefits, but due to the limitations of the methodology (see paragraphs 4.11 to 4.16), this is unlikely to be an accurate estimation of actual economic benefits.

4.27 The implementation plan for the Communications and Engagement Strategy specifies that industry engagement will be prioritised in order of sectors of the economy estimated to gain the most economic benefits (see paragraph 2.95), as well as the external stakeholder analysis undertaken. Frontier SI has been engaged to provide technical support to industry, research, government organisations (including Geoscience Australia) and the general public using SBAS-enabled receivers, as well as liaison with hardware manufacturers. By 30 June 2024, Frontier SI has also been contracted to deliver six case studies to demonstrate how SouthPAN is being used.

4.28 Geoscience Australia has a Positioning News subscription service⁷² which is a periodic newsletter that includes updates on the Positioning Australia program, including SouthPAN.

Reporting performance

External reporting

4.29 The purpose of Geoscience Australia includes informing government, industry and community decisions on the economic, social and environmental management of the nation’s natural resources through enabling access to geoscientific and spatial information.⁷³ In Geoscience Australia’s 2023–24 Corporate Plan, SouthPAN sits, as part of Positioning Australia under strategic priority⁷⁴ five — creating a location-enabled Australia.

4.30 The NPIC program and SBAS (SouthPAN) were prescribed by the minister under the Industry Research and Development Act 1986 in 2018 to support the Australian Government’s Australian Technology and Science Growth Plan. Prescribing the programs facilitated funding for the programs and was required for the exercise of the Parliament’s legislative communications power in relation to postal, telegraphic, telephonic and other like services.

4.31 NPIC requires access to the mobile phone network. This means in areas with mobile phone coverage, NPIC can deliver 3-5cm accuracy. SouthPAN has the potential to deliver 10cm accuracy irrespective of mobile phone coverage. NPIC and SouthPAN are complementary programs. While they can operate independently and provide different capabilities, SouthPAN services currently rely on existing Continuously Operating Reference Stations (which are part of the NPIC program) to function.⁷⁵ Until dedicated GNSS reference stations⁷⁶ are built for SouthPAN, the services to 10cm accuracy are delivered by both NPIC and SouthPAN.

4.32 In Geoscience Australia’s 2022–23 Corporate Plan, separate performance measures were developed for NPIC and SouthPAN although this was not stated. The measures were: ‘Operate the infrastructure and systems to enable 3–5 cm accurate positioning services in areas with mobile phone coverage’ for NPIC and ‘build the infrastructure and systems to deliver trusted and 10cm accuracy positioning service across Australia and its maritime zones’ for SouthPAN (see paragraph 3.9 on the contractual requirement of 15cm accuracy). The 2022–23 Corporate Plan also had a 10 year target of deliver positioning accuracy of 10cm across Australia and enable 3–5cm accuracy in areas with mobile phone coverage, adding at least $6.2 billion to the Australian economy over the next 30 years.

4.33 Geoscience Australia has removed this target from its 2023–24 Corporate Plan and replaced the two performance measures above with one measure. The new measure relates to the broader Positioning Australia initiative (which includes NPIC and SouthPAN) and is outlined in Table 4.1. The new measure is described in the 2023–24 Corporate Plan as a quantitative effectiveness measure of two key activities, being enhance knowledge and understanding and quality advice and public access.

Table 4.1: 2023–24 Corporate performance measure that includes ‘SouthPAN’

Source: ANAO analysis of Geoscience Australia’s 2023–24 Corporate Plan.

4.34 The description of SouthPAN in Geoscience Australia’s corporate documents to date (except for the 2023–24 Corporate Plan and 2022–23 Annual Report) incorrectly characterised SouthPAN as enabling 3–5 cm accurate positioning services in areas with mobile phone coverage. This relates to NPIC, not SouthPAN.⁷⁷ It is not clear how the new measure and the target of SouthPAN services being available 95 per cent of the time will demonstrate achievement of (or a contribution to) enhanced knowledge and understanding, and quality advice.

Evaluation

4.35 A program evaluation strategy for the Positioning Australia initiative which included SouthPAN, was planned to be completed in June 2019 but in an internal audit report in March 2020 noted this had not occurred. The Positioning Australia Monitoring and Evaluation Strategy (the strategy) was endorsed by the Positioning Program Board (see paragraph 2.5) on 2 December 2020.

4.36 SouthPAN was included in the strategy as part of the broader program, which was informed by a program logic⁷⁸ and the development of eight key performance indicators. Updates were provided to the Positioning Program Board in April and November in 2021 and April and June in 2022. Over that time the measures for assessing the KPIs for seven of the KPIs (two of which were specific to SouthPAN), were agreed or in development by Geoscience Australia, with work undertaken through the test-bed on the specific KPI in relation to economic benefits realisation.

4.37 In September 2022, Project Services International undertook an assessment of the benefits realisation work. It found that most of the KPIs were ‘output related, rather than reflective of the outcomes of the program’ with only one KPI reflecting the ‘“economic, social and environmental benefits to Australia”, albeit worded too broadly to be measurable’. The report recommended Geoscience Australia close out the current monitoring and evaluation reporting and rely on the SBAS Trial Economic Benefits Analysis Report (see paragraphs 2.92 to 2.102) for SouthPAN. On 12 December 2022, the Positioning Program Board (see paragraph 2.5) endorsed the Benefits Management Plan which implemented the recommendation.

4.38 The Benefits Management Plan also sets out that the benefits for SouthPAN will be realised through: early open services being available in 2022–23; safety-of-life certification occurring in 2028–29; and 95 per cent availability of SouthPAN services. While these outputs align to the Corporate Plan target that support the relevant performance measure, relying on a modified version of the test-bed benefits analysis (see paragraphs 4.11 to 4.16) to report on adding at least $6.2 billion to the Australian economy over the next 30 years will not be accurate.

4.39 Further, the Benefits Management Plan does not align with the contract which states that benefits realisation is to be measured by: achieving planned objectives; monitoring cost and schedule tolerances; and an assessment of uptake in users, services and technology enabled by SouthPAN in the years following establishment. The contract facilitates monitoring of performance through outputs, however there is no mechanism in the contract to measure achievement of the planned objectives or assessing the uptake in users, services and technology enabled by SouthPAN in the years following establishment.

4.40 As at October 2023 Geoscience Australia does not have any other evaluation strategy or plan in place, although the Positioning Program Board was asked to note on 25 August 2023, that a long term plan for assessing benefits realised will be developed. Geoscience Australia advised that ‘The measurement of the benefits of the program are difficult and expensive given [SouthPAN is] open and freely available, therefore usage cannot be monitored through the system.’

4.41 In relation to the contract requirement of 15cm, Geoscience Australia advised ‘For technical purposes, signal performance is described in terms of uncertainty. Positioning measurements occur as varying values over time. An accuracy requirement is not a hard limit on the maximum error that could occur, but rather a metric that indicates the level of performance the user will experience as a range.’

4.42 There is a gap in monitoring performance and evaluation for Geoscience Australia relating to gaining verification over the accuracy of the signals on the ground, in time and place.

Appendix 1 Entity response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlined to Geoscience Australia the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• As of 10 August 2023, there was an updated approved Contract Management Plan (see paragraph 3.13) which includes a new section on managing conflicts of interest.
• Geoscience Australia removed reference to the 10 year target to ‘deliver positioning accuracy of 10cm across Australia and enable 3–5cm accuracy in areas with mobile phone coverage, adding at least $6.2b to the Australian economy over the next 30 years’ (see paragraph 4.33) in its 2023-24 Corporate Plan.