Introduction

1.1 The Entrepreneurs’ Programme was announced as part of the 2014–15 Budget with funding of $484.2 million. It was launched on 1 July 2014. The program forms part of the Australian Government’s industry policy outlined in the Industry Innovation and Science Agenda, released on 14 October 2014, and the National Innovation and Science Agenda, announced on 7 December 2015. The Department of Industry, Science, Energy and Resources (DISER or the department) is responsible for administering the program.³

1.2 On 27 September 2019, the department issued a Request for Tender (RFT) for the engagement of delivery partners to deliver expert business advisory and facilitation services for the Entrepreneurs’ Programme. The estimated value of the procurement was $182 million for the maximum contract term of five years. The department expected it would engage approximately 10 delivery partners able to employ a minimum of five specified personnel. At the time, the program was being delivered by providers appointed from a procurement process conducted in 2014–15. Most of those contracts expired on 30 June 2020.⁴ The new delivery partners engaged through this RFT process were to commence services from 1 July 2020.

1.3 The development of the RFT was informed by a ‘redesign project’. Five of the ten incumbent delivery partners (formerly named ‘industry partners’) were engaged to assist with the redesign to draw on their ‘deep knowledge of customer needs/program experience.’ Three of the incumbents engaged to contribute to the redesign project were subsequently awarded contracts as delivery partners.⁵ As a result of the project, the department decided on an operating model focused on the outcomes of ‘growth, innovation, and commercialisation’. The outcomes are supported by four program elements: business management, innovation connections, incubator support and accelerating commercialisation.⁶ Grants are available to businesses under the business management, innovation connections, and accelerating commercialisation services of the Entrepreneurs’ Programme.

1.4 In the RFT, the department sought:

• a single delivery partner to provide commercialisation services to customers nationally;
• a single delivery partner to provide innovation and incubator support services nationally; and
• one or more delivery partners to provide expert business advisory and facilitation services in nominated geographic areas on a state and/or regional basis.

1.5 Fifty-five responses were received to the RFT, of which 10 were from incumbent providers (18 per cent). Two responses did not satisfy the conditions for participation and/or the minimum content and format requirements. The remaining 53 progressed for evaluation.

1.6 Seven tenderers (one to deliver commercialisation services nationally; one to deliver incubation support and innovation connection services nationally; and five to deliver advisory and facilitation services in nominated geographic areas) were awarded contracts for a term of three years from 1 July 2020 to 30 June 2023, with two one-year extension options. Five of the seven tenderers (71 per cent) awarded contracts were incumbent providers.

1.7 Under the contracts, performance is to be assessed and managed by the department using a combination of quality, volume and outcome measures to evaluate each delivery partner’s contribution to the overall performance of the program against its objectives. Performance measurement is a combination of the number of customers, the outcomes achieved and the impact for the customer.

Rationale for undertaking the audit

1.8 This procurement is a key element for the delivery of a services and grants program that is part of the National Innovation and Science Agenda, the design of which was examined in Auditor-General Report No.10 of 2017–18. The procurement was DISER’s largest value procurement in 2019–20 and represented 37 per cent of the total value of contracts entered into by DISER and reported on AusTender for that year. The audit will provide assurance to the Parliament about whether value for money was demonstrably achieved and whether services have been provided under the contracts to date.

Audit approach

Audit objective, criteria and scope

1.9 The audit objective was to examine whether the design and conduct of the procurement process for delivery partners for the Entrepreneurs’ Programme complied with the Commonwealth Procurement Rules, and whether the signed contracts are being appropriately managed.

1.10 To form a conclusion against this objective, the following high-level criteria were applied.

• Did the procurement process demonstrate achievement of value for money?
• Are the contracts being managed appropriately to achieve the objectives of the procurement?

1.11 The audit examined the design and conduct of the 2019–20 procurement process for delivery partners for the Entrepreneurs’ Programme, and the management of the contracts for all delivery partners since services commenced in July 2020. The audit scope did not include:

• the 2014–15 procurement process for industry partners and the awarded contracts; and
• the development of program guidelines and the evaluation and award of grant funding under the Entrepreneurs’ Programme.

Audit methodology

1.12 The audit methodology included examination and analysis of DISER records; and meetings with relevant DISER staff.

1.13 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $507,000.

1.14 The team members for this audit were Sean Neubeck, Jocelyn Watts, Joshua Carruthers and Brian Boyd.

2.1 To assess whether the procurement demonstrated achievement of value for money, the ANAO examined whether an open and competitive procurement process was conducted that complied with the CPRs.⁷ This reflects that competition is a key element of the Australian Government’s procurement framework set out in the CPRs.⁸ Generally, the more competitive the procurement process, the better placed an entity is to demonstrate that it has achieved value for money. Competition encourages respondents to submit more efficient, effective and economical proposals. It also ensures that the purchasing entity has access to comparative services and rates, placing it in an informed position when evaluating the responses. Openness in procurement involves giving suppliers fair and equitable access to opportunities to compete for work while maintaining transparency and integrity of process.

Were relevant evaluation criteria included in request documentation to enable the proper identification, assessment and comparison of submissions?

2.2 The CPRs require that evaluation criteria be included in request documentation to enable proper identification, assessment and comparison of submissions on a fair, common and appropriately transparent basis.⁹ Request documentation must include a complete description of evaluation criteria to be considered in assessing submissions and, if applicable to the evaluation, the relative importance of those criteria.¹⁰

2.3 The RFT documentation outlined that tenders would be assessed on the basis of value for money consistent with Commonwealth procurement policies, through the application of evaluation criteria. The RFT included six evaluation criteria to be applied for the purposes of evaluation, two of which were weighted and the remaining were not weighted. The RFT stated that the criteria were not in order of importance.¹¹ The criteria were:

• organisational capability (70 per cent weighting);
• promotion and marketing and industry knowledge (30 per cent weighting);
• price (not weighted);
• risk (not weighted);
• corporate and financial viability (not weighted); and
• Commonwealth policies (not weighted).¹²

Tender evaluation plan

2.4 An evaluation plan should be established before market responses are sought.¹³ Drafting the evaluation plan and RFT together helps to avoid any inconsistencies between the evaluation plan and requirements specified in RFT documentation, and to ensure that the evaluation methodology proposed in the plan is consistent with the RFT.¹⁴ An appropriately drafted and implemented evaluation plan assists to demonstrate that tender submissions have been objectively and fairly evaluated in accordance with the RFT.

2.5 In March 2022, the Department of Science, Industry and Resources (DISER or the department) advised the ANAO that:

The Tender Evaluation Plan was largely completed as a working document when the RFT was published, and was finalised and approved prior to the evaluation process commencing.

2.6 The ANAO’s analysis of the contemporaneous records of the conduct of the procurement did not support the department’s advice that the evaluation plan had been largely completed at the time the RFT was published. The draft plan in circulation at the time the RFT was published was based on the plan used for the previous tender for delivery partners and included various comments from the probity adviser and departmental officers. It also could not be considered a largely complete document, given the draft at the time the RFT was published did not reflect key aspects of the approach later employed for the evaluation. In particular, the following aspects were missing.

• The draft plan did not identify who would conduct the evaluation:
  • the draft plan stated that a six member Tender Assessment Panel (TAP) would be employed whereas the evaluation of tenders was undertaken by a three member Panel, notwithstanding that a three person TAP had been established by the department prior to the RFT being published (see paragraph 2.16);
  • this draft version of the evaluation plan also did not reflect that four ‘internal experts’ would play a role in the conduct of the evaluations (while the use of ‘internal experts’ was identified in the final version of the evaluation plan that was attached to the tender evaluation report their role was not defined in that document).
• The draft plan did not reflect the evaluation criteria set out in the RFT:
  • the draft evaluation plan at the time the RFT was published set out five weighted evaluation criteria being service delivery (25 per cent), key personnel (35 per cent), project management (10 per cent), industry knowledge (15 per cent) and promotion and marketing (15 per cent); whereas
  • six criteria had been published in the RFT, of which two had been weighted — organisational capability (70 per cent weighting), promotion and marketing and industry knowledge (30 per cent weighting), price, risk, corporate and financial viability, and Commonwealth policies.

2.7 The evaluation plan¹⁵ for the procurement process for delivery partners for the Entrepreneurs’ Programme was:

• not signed off and approved before the RFT was issued on 27 September 2019 or before tender submissions were opened on 28 November 2019;
• not signed off and agreed by all TAP members;
• not signed off and approved by the probity adviser; and
• not at any time signed off and approved by the delegate for the procurement.

2.8 The draft plan was provided to the TAP to be signed off after stage 2 technical evaluations were completed on 19 December 2019. As evidence that the plan had been signed, DISER provided the ANAO in October 2021 with a scanned signature page including the TAP Chair’s signature from 23 December 2019. One member of the TAP had provided agreement to the terms and conditions of the plan by email on 19 December 2019. No evidence that the remaining member signed the plan was recorded.

2.9 The draft plan was also intended to be approved by the procurement delegate and by the probity adviser. Signatures by the delegate and probity adviser were not recorded.

2.10 While the RFT stated that the evaluation criteria were not listed in order of importance, the department’s tender evaluation plan set out a multi-staged process where tender submissions were required to meet the first two weighted evaluation criteria to be shortlisted for further evaluation against the remaining criteria.

2.11 The ‘multi-stage procurement method’ is one of the procurement methods identified in the CPRs.¹⁶ DISER did not inform tenderers that the procurement involved a staged approach. Specifically, the RFT did not outline to potential tenderers that a threshold would be applied and that there would be a limit¹⁷ on the number of tender submissions that would proceed for further evaluation against the remaining criteria. This was important information for tenderers to know to inform their decision about whether to participate in the tender process.¹⁸ The contributions received by the ANAO to the Citizens Contribution facility for this audit included two from tenderers that clearly evidenced the tenderers were unaware that there had been a shortlisting process employed based solely on evaluation against the first two criteria listed in the RFT.

2.12 The ANAO compared the RFT to other approaches to market for four procurements¹⁹ the ANAO has audited where shortlisting was employed (going back to 2004). Compared with the approach to market documentation for those procurements, the Entrepreneurs’ Programme RFT was significantly deficient. All the other procurements had:

• clearly stepped out how shortlisting would be employed, including clearly identifying the criteria for shortlisting as well as those for full evaluation; and
• where relevant, also clearly outlined to the market the Commonwealth’s expectations in terms of how many respondents were likely to be shortlisted.

Tender evaluation team

2.15 Guidance to entities is that the procurement team should be selected carefully and early, preferably before the RFT is released, and probity training should be provided to the team as soon as they are identified, not when the evaluation is about to start.²⁰

2.16 DISER established the TAP comprising a chairperson and two members prior to issuing the RFT. A probity adviser had also been engaged in advance of the RFT’s release (see paragraph 2.71 below). Neither the panel of internal experts nor the pricing assessment team had been established prior to commencement of evaluation.²¹ The role of the internal experts was not defined in the tender evaluation plan. The pricing assessment team was to:

• actively participate in briefings on the RFT and the evaluation process;
• analyse the whole-of-life contract cost for each tender submission at stage 3 of the evaluation process; and
• provide a report to the TAP containing the price ranking and analysis and risk assessment of pricing elements for consideration as part of the stage 5 value for money assessment.

2.17 A probity briefing was provided to the Entrepreneurs’ Programme branch on 13 September 2019 in advance of an Entrepreneurs’ Programme Annual Forum. The forum was attended by the incumbent delivery partners and facilitators. DISER has not retained a record of the briefing, including an attendance list. The ANAO’s analysis of records indicates that key personnel, including the TAP Chair, were not present at the probity briefing provided in advance of the Forum.

2.18 A probity briefing to the TAP and other members of the procurement team was provided by the probity adviser on 28 November 2019, the day after the RFT closing date. Three of the four internal panel experts did not attend the briefing. The pricing assessment team had not been established at the time of the briefing.

2.19 DMV Consulting was engaged to perform the stage 3 pricing evaluation. The procurement process for the engagement, which was not consistent with the CPRs, was as follows:

• on 31 October 2019, the probity adviser provided the procurement team with recommendations for financial and pricing analysts;
• on 11 November 2019, delegate approval was provided to engage a financial analyst to do a cost comparison and financial health check of shortlisted tender responses;
• the request for approval outlined that, subject to delegate approval, the department would approach four businesses to begin discussions about the estimated costs (the CPRs state that ‘the expected value of a procurement must be estimated before a decision on the procurement method is made’)²²;
• the department requested a quote from each of the four businesses on 15 November 2019;
• the department received two quotes for comparison. The comparison was made on the basis that the shortlist would be of 15 tenderers;
• on 9 December 2019, the department sought a revised quote from DMV Consulting based on the department’s own estimates of time required to conduct the financial viability reviews and cost comparison of pricing schedules. The department advised DMV Consulting that it would be engaged under the department’s panel arrangement for the supply of Accounting, Audit, and Related Professional Services. Neither the cost estimate nor the procurement method was identified in the original procurement process approval; and
• on 16 December 2019, the delegate provided stage 2 approval to engage DMV Consulting to provide financial analysis services for the delivery partner tender as well as another tender process being undertaken for the Entrepreneurs’ Programme. The total approved contract value was $120,428 with the award of this contract reported by the department on AusTender on 24 December 2019.

2.20 The total amount paid to DMV Consulting was $56,823.²³

Were the successful candidates those assessed as providing the best value for money?

Registration and Screening

2.21 The CPRs require request documentation to include any conditions for participation and any minimum content and format requirements.²⁴ Further consideration must be given only to submissions that meet minimum content and format requirements.²⁵

2.22 Fifty-five tender responses were received. The initial screening concluded that two tender responses did not satisfy the conditions for participation and/or the minimum content and format requirements. There were 13 tender responses for which additional information was sought to satisfy the conditions for participation and minimum content and format requirements. Of those 13 tender responses, 11 were progressed to stage 2 evaluations before the additional information was received. This meant that DISER did not comply with the CPR requirement outlined in paragraph 2.21.

2.23 As part of the screening process, the tender evaluation plan allowed for tender submissions to be excluded, with supporting legal advice, if the tender response was ‘clearly uncompetitive’ when compared with others received. The TAP received probity advice that Darwin Innovation Hub’s tender submission should be excluded on the basis of being non-competitive and that ‘bid repair’ was required to address non-compliance with the minimum content and format requirements.

2.24 The TAP later decided to shortlist Darwin Innovation Hub’s tender submission after a panel member reassessed the original rating and the remaining members agreed to shortlist. The records of the TAP’s decision to shortlist were inadequate to explain the decision that was taken.

Application of evaluation criteria

2.25 The TAP did not evaluate tender submissions in accordance with the published evaluation criteria and methodology outlined in the RFT. Figure 2.1 below shows the extent to which tender submissions were evaluated against each of the criteria.

Figure 2.1: Application of evaluation criteria

Note: This graph includes 53 tenderers progressed to the stage 2 technical evaluations. Two tenderers were excluded from further consideration because they did not meet minimum content and format requirements (refer to paragraph 2.22).

Source: ANAO analysis of DISER records.

Criteria 1 and 2 — ‘organisational capability’ and ‘promotion and marketing and industry knowledge’

2.26 The first two weighted criteria were applied as part of the stage 2 technical evaluations. These were conducted by the TAP from 3 December to 18 December 2019 (refer to Figure 2.2 below). The TAP was provided with an individual ‘Stage 2 Technical Worth Assessment Form’ by the probity adviser²⁶ on 1 December 2019, two days before technical evaluations commenced. At stage 2, the TAP was to perform a comparative assessment of the ability and relative merit of each tender response against the objective comparative assessment rating scale in the tender evaluation plan.

2.27 While all 53 tender submissions that progressed to stage 2 were evaluated against the ‘organisational capability’ criterion, six tender submissions did not receive an evaluation against the ‘promotion and marketing and industry knowledge’ criterion.

2.28 In contrast to the RFT, which stated that the evaluation criteria were not in order of importance, DISER applied the first two weighted criteria as a threshold and shortlisted tender submissions following the stage 2 technical evaluation against the first two criteria.

2.29 The 14 shortlisted tender responses were provided to the internal panel experts for further evaluation. The tender responses were divided among the internal panel experts according to each assessor’s knowledge of the outcomes and/or their experience within relevant and applicable policy. The TAP provided the internal panel experts with a set of specific questions as well as questions relevant to each shortlisted tender response. Internal panel experts were not required to allocate a score or comment against the full evaluation criteria contained in the RFT, only those relevant to the internal expert’s expertise.

Criterion 3 — Price

2.30 The shortlisting meant that of the 53 tender submissions that were progressed for evaluation, 14 (26 per cent) received evaluation against the price criterion. As set out in Appendix 6, a number of tenderers have complained about the conduct of the tender process, including the extent to which the successful candidate was more expensive than the price they offered.

2.31 As outlined at paragraph 2.19, DMV Consulting was engaged to conduct the pricing evaluation. The initial stage 3 pricing assessment involved a comparison of the prices tendered by each of the 14 shortlisted tenderers. The tender submissions were separated according to the three outcomes (growth, innovation, and commercialisation) and ranked from ‘best’ to ‘least best’ based on the ‘tendered price per facilitator’.

2.32 A significant error was made in the conduct of the pricing evaluation, which resulted in Darwin Innovation Hub’s tender response being ranked as the best tenderer for the growth outcome.²⁷ Had Darwin Innovation Hub’s price per facilitator been calculated correctly it would have ranked seventh of the ten tenderers assessed for price in the growth outcome. In March 2022, DISER advised the ANAO that it:

was not aware of the error with DMV’s assessment of the Darwin Innovation Hub (DiH) pricing until it was raised by the ANAO. This error did not have a material impact on the overall pricing, or value for money assessment.²⁸

Criterion 4 — Risk

2.33 According to the tender evaluation plan, the evaluation of risk was to form part of the overall value for money assessment at stage 5. Risks were to be identified throughout the evaluation process, including during the technical and financial evaluations, and were to be formally assessed and taken into account in the rating during the overall best value for money considerations in stage 5.

2.34 The TAP did not perform a formal risk assessment which encompassed risks identified throughout the process for any of the tender responses. While aspects of the risk criterion were considered throughout the process, evaluation of risk was limited to:

• compliance checks at stage 2 that tender submissions included the information requested in the RFT. While the template used at stage 2 included a section for TAP members to record evaluation criterion risks identified through the evaluation process, this section was not completed for any of the assessments;
• limited analysis of risks as part of the value for money assessment of the 14 shortlisted tenderers, which was largely based on the tenderers’ interview responses;
• consideration of price risk in relation to three tender responses for the commercialisation outcome; and
• consideration of the level of compliance with the draft contract of the seven tenderers identified as the recommended tenderers in the evaluation report.²⁹

Criterion 5 — Corporate and financial viability

2.35 DISER reduced the shortlist to 10 tender submissions following interviews with the 14 shortlisted tenderers and an initial ‘value for money assessment’ based on the initial stage 3 pricing and the results of the internal panel expert evaluations.

2.36 Evaluation against the corporate and financial viability criterion was conducted by DMV Consulting and was limited to the reduced shortlist of 10 tender submissions. DMV Consulting provided individual reports for each of the three outcomes (growth, innovation, and commercialisation), and assigned a risk rating for each of the tender submissions against the corporate and financial viability criterion.

2.37 DMV Consulting proposed mitigation strategies for three tenderers:

• Darwin Innovation Hub — tenderer to obtain a bank guarantee from Paspalis family;
• Respondent 53 — tenderer to provide evidence of liquidity prior to contract signing; and
• i4 Connect Pty Ltd (i4 Connect) — tenderer to provide evidence of liquidity prior to contract signing.

2.38 In March 2022, in response to the ANAO’s request for information regarding the mitigation strategies, DISER advised the ANAO that:

A bank guarantee was not obtained from the Paspalis Family Trust as the TAP determined they were sufficiently assured of DiH’s financial position.

Evidence of i4 Connect’s liquidity was obtained prior to contract signing. DISER emailed i4 Connect on 11 February 2020 with a series of questions relating to pricing. This included questions on their current financial position, ability to pay debts and whether there were bank guarantees or alternative approaches to deal with the issues of there being few substantial assets.

i4 Connect’s responses were provided to the Pricing Analyst and the Probity Advisor. Both indicated the responses were satisfactory and the TAP was satisfied with this additional information.

2.39 The DMV Consulting report that proposed the mitigation strategies was prepared after receiving i4 Connect’s responses and was provided to the department on 17 February 2020. DMV Consulting’s report stated that ‘whilst the Tenderer has indicated that they have negotiated loans to assist with short-term liquidity, these would need to be sighted prior to contract signing.’

Criterion 6 — Commonwealth policies

2.40 The CPRs require officials to assess the economic benefit of a procurement to the Australian economy as part of determining the value for money. This assessment requires officials to gather appropriate information on economic benefits as part of the decision-making process and document how economic benefit has been considered as part of the overall value for money assessment.³⁰

2.41 DISER did not comply with this requirement. Evaluation against the Commonwealth policies criterion allowed for this requirement to be met, as well as consideration of other relevant Commonwealth procurement connected policies. Consideration of the criterion was restricted to a compliance check at stage 2 that tender submissions included the information requested in the RFT. The TAP did not document an assessment against the evaluation criteria, including how economic benefit was considered, as part of the overall value for money assessment for any of the tender responses.

Technical scoring and application of weightings

2.42 As part of stage 2, each TAP member was asked to record their assessment of each tender response and rate them (on a scale of 0 to 5) against the weighted criteria (‘organisational capability’ and ‘promotion and marketing and industry knowledge’). After completing individual assessments, the TAP was to meet to reach a consensus on the proper word description based on the rating scale and the consequent supporting score for each evaluation criterion. Weightings were then to be applied to each score for each of the evaluation criteria in accordance with the weightings specified in the RFT.

2.43 The TAP did not employ a consistent and transparent process in evaluating each of the tender submissions. The process was adversely impacted by the following.

• The number of evaluations, such that:
  • only four (eight per cent) tender responses received an evaluation from all three TAP members;
  • 26 (49 per cent) tender responses received an evaluation from two TAP members; and
  • 23 (42 per cent) tender responses received an evaluation from only one TAP member.
• The technical scoring of tender submissions only being applied incidentally to producing the shortlist of tender responses, such that the TAP did not apply a transparent process. In particular:
  • the individual TAP members scored responses inconsistently (for example, one assessor gave an overall score while the other two did not);
  • there is no record of the TAP meeting to moderate and agree a final score for each submission;
  • while overall scores were recorded in a ‘tracking’ document, how those scores were determined was not transparent and the scores recorded in the tender evaluation report are inconsistent with the tracking document;
  • there is no evidence of weightings having been applied; and
  • the TAP did not produce a ranking of each tender response as an outcome of the technical evaluation, or apply a consistent threshold for shortlisting such that 16 of the 30 tender submissions rated as ‘acceptable’ or better were not shortlisted (see paragraph 2.49).

2.44 An important probity step when adopting an ‘individual scoring’ followed by a ‘consensus scoring’ method of assessment is to ensure all members have had sufficient time to conduct their individual scoring prior to convening to undertake the consensus scoring process.³¹ The ANAO analysed the conduct of the assessment process in the context of the extent of information tenderers were required to submit in response to the RFT (for example, all six criteria were required to be addressed), the number of tenders the department expected to receive, and the planned approach of individual TAP members conducting separate assessments prior to undertaking consensus scoring.

2.45 Figure 2.2 outlines the timing of evaluation assessments over the two week evaluation period. A significant proportion of assessments were undertaken late in the two week period and there was an uneven distribution of assessments conducted by individual TAP members, with the assessor independent to the program undertaking the fewest assessments. To expedite the evaluation process, part way through the technical evaluation period, the TAP Chair (in consultation with the probity adviser) decided that a division of the tender responses among the TAP was required. The reasoning provided in the tender evaluation report was that this was a result of the large number of responses received. This rationale was at odds with:

• the RFT indicating that approximately 10 delivery partners would be engaged;
• the market analysis recorded in DISER’s procurement plan that had indicated that the number of responses received was similar to that expected; and
• the 2015 Entrepreneurs’ Programme Industry Partner tender where more than 50 responses had been received.

Figure 2.2: Timing of evaluation assessments

Source: ANAO analysis of DISER records.

Shortlisting

2.46 Following the stage 2 evaluations against the first two criteria, DISER shortlisted 14 tender submissions for further evaluation against the remaining criteria. This meant that of the 53 tender submissions that were identified as having met the conditions for participation and the minimum content and format requirements, 39 tender submissions (74 per cent) received no further evaluation against the remaining criteria.

2.47 As outlined at paragraphs 2.3 and 2.10, the RFT stated that the criteria were not in any order of importance. The department relied on a standard clause included in the RFT that ‘[t]he Department may use its discretion to shortlist a sub-set of tenderers at any time during the RFT process’. This clause did not provide potential tenderers with a transparent indication of the extent to which the department would shortlist tender submissions such that a significant majority (74 per cent) would not be evaluated against any criteria other than the first two. Further, DISER did not transparently advise unsuccessful tenderers that shortlisting had occurred based on the first two criteria, with two unsuccessful tenderers (which were not shortlisted) told by the department in April 2020 that:

All tender submissions were assessed against each of the evaluation criteria contained in the RFT through a comparative rating assessment process overseen by an independent probity adviser.

2.48 In April 2022, DISER advised the ANAO that ‘the email sent to unsuccessful tenderers should not have stated that the tenders had been evaluated against all evaluation criteria’, and further advised the ANAO in May 2022 that it would provide the correct advice to those unsuccessful tenderers.

2.49 DISER advised the ANAO in March 2022 that ‘it shortlisted tenders that demonstrated the capability to deliver the requested services’ and that ‘if the tenderer could not provide the required level of technically capability, [it] did not progress’. At odds with this advice to the ANAO, is that while the tender evaluation report outlined that there were a total of 30 tenders rated as ‘acceptable’ or better against the first two criteria:

• 14 tenderers (47 per cent) of those rated as ‘acceptable’ or better³² were shortlisted; and
• 16 tenderers (53 per cent) rated as ‘acceptable’ against those criteria were not shortlisted.

2.50 Further, in respect to the shortlisting approach:

• only three of eleven tenders (27 per cent) from non-incumbent tenderers³³ (each for the growth outcome with none for the innovation outcome) were shortlisted; and
• of the three non-incumbents shortlisted for growth, two were for Northern Territory and one was for Western Australia, meaning that in the remaining five states/territories there was no competition from non-incumbents.

2.51 Following an initial ‘value for money’ assessment and interview responses, the TAP reduced the shortlist of tender submissions further to ten. The remaining tender submissions proceeded to the final value for money assessment stage, which included an evaluation against the corporate and financial viability criterion and further comparative assessments of price. Of the eight included in the final shortlist for the growth and innovation outcomes³⁴, only one was a non-incumbent (Darwin Innovation Hub), which was successful and awarded a contract.

2.52 As noted at paragraphs 2.34 and 2.41 above, the TAP did not properly conduct an evaluation against either of the risk or Commonwealth policies criteria.

2.53 In April 2022, DISER advised the ANAO that:

The RFT could have been clearer in the TAP’s intention to shortlist based on the capacity to deliver the required services. The RFT should not have contained the words “criteria not in order of importance” This was an unintentional error noting the RFT was approved by the Probity Adviser.

Value for money assessment and recommendations

2.54 After reducing the shortlist to 10 tender submissions, the TAP conducted the overall value for money assessment. As stated in paragraphs 1.2 and 1.4, the RFT advised potential tenderers that the department was expecting to engage up to 10 delivery partners, including:

• a single delivery partner to provide commercialisation services to customers nationally (in relation to the commercialisation outcome);
• a single delivery partner to provide innovation and incubator support services nationally (in relation to the innovation outcome); and
• one or more delivery partners to provide expert business advisory and facilitation services in nominated geographic areas on a state and/or regional basis (in relation to the growth outcome).

2.55 The tender evaluation plan required the TAP to rank the tender submissions on the basis of the technical merit of each response, the associated risks and the cost of the solution for each response. The TAP did not do this. Rather, the TAP ranked the remaining ten tender submissions in ‘bands’ as ‘competitive and recommended’, ‘competitive and not recommended’ or ‘uncompetitive’ against each of the outcomes (commercialisation, innovation, and growth). This meant that DISER was not able to transparently demonstrate that the successful candidates were recommended based on their merits as determined by the evaluation against each of the six specified criteria.

Commercialisation

2.56 Two tender responses offering commercialisation services were shortlisted for comparative assessment: Respondent 53 and i4 Connect. Table 2.1 shows DISER’s ranking of each tenderer in respect of price and risk ratings for ‘corporate’ and ‘price’.

Table 2.1: DISER ranking of commercialisation tender responses

Source: ANAO analysis of DISER records.

2.57 While Respondent 53 was identified as the first ranking tenderer, the TAP identified i4 Connect as the recommended tenderer.

2.58 The analysis presented in the tender evaluation report in favour of selecting i4 Connect did not satisfactorily address the significant price difference and there was bias evident in the analysis to support the decision to recommend i4 Connect over Respondent 53.

• The TAP gave considerable weight to ‘inherent risks’ in Respondent 53’s bid regarding the financial and corporate viability of a consortium with limited financial history, and governance structure. While risks were acknowledged in the analysis of i4 Connect, these were outweighed by the TAP’s confidence in the management team.
• Notwithstanding that ‘price risk’ was not evaluated for any of the other outcomes, the TAP gave considerable weight to the price risk associated with Respondent 53’s bid.
• The strengths of Respondent 53’s bid identified by the TAP in its value for money assessment were not presented in the analysis in the tender evaluation report.
• The tender evaluation report included statements recycled from the 2014–15 procurement’s tender evaluation report in regard to a different tenderer, which indicates it was not a complete and genuine attempt to support the recommendation.

Innovation

2.59 Two tender responses, both from incumbent providers, for innovation services were shortlisted for a comparative assessment: Respondent 1 and Commonwealth Scientific and Industrial Research Organisation (CSIRO). The tender evaluation report concluded that there was ‘minimal difference between the two tender submissions on the basis of both price and corporate and financial viability’. On a price per facilitator basis, CSIRO’s price was marginally lower.

2.60 CSIRO was rated as both ‘competitive’ and ‘recommended’ while Respondent 1’s was rated as ‘competitive’ and ‘not recommended’.

Growth

2.61 In relation to the growth outcome, the RFT stated that the department was ‘seeking to engage one or more delivery partners to provide expert business advisory and facilitation services in nominated geographic areas on a state and/or regional basis’. According to the tender evaluation report, the TAP identified that ‘placed-based knowledge, relationships and local networks for delivery services was pivotal’. As a result, the TAP decided that a single provider for growth services would not be recommended as the preferred tenderer.

2.62 Two of the shortlisted tenderers (Australian Industry Group (Ai Group) and Deloitte Consulting Pty Ltd (Deloitte)) both tendered for national service delivery, and one other (NSW Business Chamber Ltd (NSWBC)) tendered for services across the designated states and regions of the eastern seaboard, as well as the Northern Territory.³⁵ The TAP approached each of the three tenderers regarding their tendered delivery model to confirm that they were prepared to revise their tender responses to deliver services on a state and/or regional basis. To accommodate this approach, the TAP also requested additional pricing information.

2.63 DMV Consulting was engaged to conduct a comparative price assessment of the seven shortlisted tender responses for growth in relation to the following regions:

• Queensland;
• New South Wales;
• Victoria and Tasmania;
• South Australia;
• Northern Territory; and
• Western Australia.

2.64 The assessment compared the total prices of three or more tender responses in each of these regions and ranked them accordingly. The tenderers that submitted bids for national services — Ai Group and Deloitte — featured in the comparative assessments for all six regions, while NSWBC featured in three of the six regions (Queensland, New South Wales, and Victoria and Tasmania). NSWBC’s tender submission also included the Northern Territory in its service coverage but was not included in the comparative assessment for the Northern Territory.

2.65 The pricing information used for comparative assessment in DMV Consulting’s report was unreliable. The report did not include a transparent methodology for how it calculated the adjusted prices of the tender submissions of Ai Group, Deloitte and NSWBC in accordance with the baseline FTE used in each region. The report also contained errors in relation to the pricing for Darwin Innovation Hub’s tender submission, including the error in the initial pricing analysis report, discussed in paragraph 2.32, and stated the total adjusted tender price as being $4,384,799 GST inclusive which was actually the GST exclusive amount, as shown in Table 2.2. This meant that the total price was represented as being $0.44 million less than the actual tendered price.

2.66 Table 2.2 highlights the TAP’s first ranked tenderers for each region based on the comparative assessment of price.

Table 2.2: Outcome of comparative analysis for ‘growth’

Note a: Darwin Innovation Hub’s total adjusted tender price was reported in the table incorrectly. The correct amount was $4,823,280.

Note b: Darwin Innovation Hub’s total price per facilitator (FTE) based on the correct total adjusted tender price is $964,656.

Source: ANAO analysis of DISER records.

2.67 Table 2.3 shows the TAP’s final selection of recommended candidates for growth services. The ANAO observed the following inconsistencies with the selections.

• Opportunities were provided to Ai Group, Deloitte and NSWBC to revise or improve tender submissions with alternative delivery models to what they had initially proposed, along with additional pricing information in support of those alternative delivery models.
• While Deloitte — an incumbent provider — was not identified as the best candidate in any particular region, DISER tailored arrangements to select Deloitte to provide growth services in Queensland on the basis that it was able to provide additional ‘national specialist roles’. The RFT did not include a requirement for ‘national’ specialist roles.³⁶
• Darwin Innovation Hub’s tender submission was originally regarded as both non-compliant and non-competitive before the TAP decided it would shortlist the submission (see paragraphs 2.23 and 2.24). An error was made in the initial pricing assessments which lead to it initially being ranked as the best price for growth and subsequently being shortlisted for the final value for money assessment. A further error in the comparative assessment of price in the Northern Territory meant that the difference in price with the next ranked tender submission was overstated.

Table 2.3: Final selection for ‘growth’

Source: ANAO analysis of DISER records.

Were procurements conducted ethically, including identifying and managing any conflicts of interest and transparently managing incumbency advantage?

2.70 The CPRs require that procuring entities act ethically throughout the procurement, including by recognising and dealing with actual, potential and perceived conflicts of interest and by dealing equitably with potential suppliers, tenderers and suppliers.³⁷

Probity adviser

2.71 In August 2019, DISER appointed Mills Oakley as the probity adviser for the RFT to advise on key documentation for the procurement process, conduct probity briefings for procurement personnel and advise on ad hoc probity and process issues.³⁸ While the estimated cost of the engagement was $76,316, the contract notice published on AusTender has a contract value of $116,316.³⁹ The ANAO’s analysis of DISER records indicates that the total expenditure on probity services provided by Mills Oakley for the tender was $151,019.⁴⁰

Engagement of the probity adviser

2.72 DISER sourced the probity services through a departmental legal services panel arrangement. Three quotes were obtained from different law firms on the panel and evaluated based on each provider’s capacity and capability, the price offered and any identified risks. The engagement of Mills Oakley as probity adviser did not transparently demonstrate value for money and lacked probity due to the evaluation:

• criteria applied by DISER not being included in the request documentation⁴¹;
• of each candidate’s capability unfairly advantaged Mills Oakley because the deciding factor was its previous experience with the Entrepreneurs’ Programme (Mills Oakley was the probity adviser on the previous procurement of industry partners for the Entrepreneurs’ Programme and was directly approached in June 2019 to provide probity advice on the redesign of the program — refer to paragraph 2.92 below); and
• of each provider’s price unfairly advantaged Mills Oakley because, while another candidate offered a lower estimated price, ‘the Mills Oakley quote showed their greater relevant experience which indicated that they had a better grasp of what work was required of them.’

2.73 Engaging the same probity adviser on an ongoing or serial basis over several related or unrelated issues increases self-interest and familiarity risks that may threaten the actual or perceived independence of the practitioner.⁴² DISER’s focus on the benefits of Mills Oakley’s previous experience in its evaluation also failed to consider whether this experience could adversely affect the independence of its probity advice.

Scope of probity services

2.74 Probity experts should be independent and free from all conflicts of interest and have sound knowledge of all relevant government policies and procedures.⁴³ DISER engaged Mills Oakley to provide advice on probity and process issues.⁴⁴ As noted in paragraph 2.71, the cost of the probity services was double that estimated by the department when it undertook the procurement. DISER correspondence with the probity adviser indicates that the increased cost reflects:

a number of additional iterations of the RFT (including detailed review and drafting and amendment of the evaluation criteria, schedules, and response attachments) and other work additional to [the probity adviser’s] original scope such as the provision of a draft tender evaluation plan and detailed drafting work on the RFT and response schedules between them and connections with them and evaluation criteria.

2.75 At the request of the department, Mills Oakley had involvement in procurement work including:

• drafting planning documents including the probity plan and tender evaluation plan;
• drafting the RFT documentation, including redrafting the evaluation criteria and response schedules;
• drafting the draft form of contract attached to the RFT;
• drafting one of the evaluation templates used by the TAP;
• recommending financial analysts to undertake pricing and value for money assessments; and
• participating as a procurement team member in meetings with a tenderer during the value for money evaluation phase of the tender process.

2.76 DISER’s decision to source probity and procurement process advice from the same adviser adversely affected the independence of the probity advice. For example, in September 2019, Mills Oakley advised the delegate that ‘on the basis of the review criteria and investigations contained in this letter, we are satisfied that the RFT and draft form contract meet the probity and procurement requirements.’ This was not appropriate because the same team from Mills Oakley was responsible for developing the RFT and providing the interim probity report.

Probity plan and protocols

2.79 The probity plan for the procurement was approved by the delegate in September 2019.

2.80 The probity plan formed ‘a key part of assurance to delegates, suppliers and the Commonwealth that the procurement process for the program is conducted in a manner that is fair, equitable and defensible.’ The probity plan applied to ‘key persons’ involved in the procurement, including members of the assessment panel and other persons nominated by the procurement manager. Probity protocols were also issued to members of the procurement team, providing detailed instruction on reporting probity issues, assessment and record keeping requirements, information management, declaration of conflicts of interest and communications with potential tenderers.

2.81 The probity protocols established by the probity plan were not fully implemented⁴⁵, and had limited effect on the conduct of the procurement. Key deficiencies in implementation related to the management of conflicts of interest (refer to paragraph 2.82 to 2.86 below) and maintenance of a complete probity register. In addition, while probity briefings were undertaken for the procurement team, key personnel involved in the procurement did not attend them (refer to paragraphs 2.17 and 2.18 above).

Conflicts of interest management

2.82 DISER’s probity plan for the procurement stated that a conflict of interest arises in any circumstance in which an official’s public duty is (or is seen to be) in conflict with their private interest.

2.83 Department of Finance guidance to entities on ethics and probity in procurement states that ‘persons involved in the tender process, including contractors such as legal, commercial or probity experts, should make a written declaration of any actual, potential or perceived conflicts of interests prior to taking part in the process’.⁴⁶ Consistent with this guidance, the probity protocols required members of the procurement team to declare actual or potential conflicts of interest in relation to the procurement. The department has not retained on file all conflict of interest declarations, including that of the external financial analyst responsible for preparing pricing evaluations and corporate and financial viability checks of shortlisted tenderers, and not all members of the procurement team signed conflict of interest declarations, including:

• the procurement delegate⁴⁷; and
• internal legal and procurement advisers who formed part of the specialist advice and support team.

2.84 In March 2022, DISER advised the ANAO that in its view:

Requiring a specific conflict of interest declaration from any person, even those with a tenuous and superficial link to the procurement process is overly bureaucratic, administratively burdensome and does not appropriately manage risk.⁴⁸

2.85 Not all conflicts of interest were identified. For example, one internal expert did not declare his (known) employment relationship of nine years with one of the tenderers — a breach of the probity protocols — and, as a result, no steps were taken to manage this conflict.

2.86 DISER’s approach to management of conflicts of interest for the procurement and compliance with probity protocols was largely passive. No management actions were put in place to avoid or mitigate identified conflicts of interest. This was reflected in a number of missed opportunities to avoid or effectively manage potential conflict of interest concerns for procurement team members who held contract management positions in the department.

Incumbency advantage

2.87 The CPRs state that all potential suppliers to government must be treated equitably based on their commercial, legal, technical and financial abilities.⁴⁹ When an incumbent provider competes for new work, it may have (or be perceived to have) certain advantages, such as understanding of an agency’s needs, established relationships with agency staff, and knowledge that is not available to other potential suppliers.⁵⁰

Composition of the TAP

2.88 DISER advised the ANAO in October 2021 that a key probity control to manage incumbency risks was the composition of the TAP. One member of the TAP was external to the Entrepreneurs’ Programme (from within the AusIndustry division of the department) and the other two members were employed in the Entrepreneurs’ Programme branch. Further, Figure 2.2 above demonstrates that the number of technical evaluations was not divided equally between panel members.

• Assessor 1 (internal to the Entrepreneurs’ Programme) completed 34 stage 2 technical assessments (39 per cent of assessments completed).
• Assessor 2 (internal to the Entrepreneurs’ Programme) completed 44 stage 2 technical assessments (51 per cent of assessments completed).
• Assessor 3 (external to the Entrepreneurs’ Programme) completed 9 stage 2 technical assessments (10 per cent of assessments completed).

2.89 While the design of the TAP was intended to mitigate incumbency risks, the unequal participation of assessors in stage 2 technical evaluations demonstrates that the effectiveness of this control was minimal.

Probity controls for the redesign project

2.90 Incumbent providers may be consulted on technical, operational or other information to be used in a procurement process. There are significant risks with this approach that require management, as an incumbent may (or be perceived to) gain insight from the information collected, giving it an unfair advantage.⁵¹

2.91 Prior to commencing the RFT, DISER undertook a redesign project with the stated aim to improve the operation of the program, including the delivery mechanism and the structure of the program. DISER engaged five of the ten existing industry partners to assist with the redesign to draw on their ‘deep knowledge of customer needs/program experience.’⁵² In response to a request from the ANAO for advice on the contractual arrangements for engaging five of the incumbent providers to assist with the redesign, and how much they were paid for this assistance, in March 2022 DISER advised the ANAO that:

DISER did not enter into new contracts with providers for the redesign project. DISER accessed its existing arrangements with the supplier and paid for their services under those arrangements.⁵³

2.92 DISER engaged Mills Oakley to provide probity advice on actual and perceived conflicts of interest involved in industry partners participating in the redesign project, who may later tender for services in a resultant procurement process.⁵⁴ While Mills Oakley noted that engagement of industry partners was acceptable from a probity and legal perspective, strong probity protocols should be implemented to provide ‘sufficient probity and practical protection for the department, the process, the individuals and their employers in any future EP procurement process’. Table 2.4 sets out the probity adviser’s recommended probity measures for the redesign project and the extent to which these probity controls were implemented.

Table 2.4: Implementation of probity controls for the redesign project

Note a: While the RFT was originally due to close on 28 October 2019, the closing date was extended three times during the period tender submissions were open. Complaints were received by the Minister’s Office about the procurement process, which were provided to the procurement delegate. For example, one complaint stated that the tender application is ‘extremely paper work heavy’ and includes ‘requests for a lot of superfluous and theoretical requirements.’ ANAO analysis of DISER records indicates that the procurement delegate agreed to the second extension of the timeframe because of ‘the degree of agitation by the [Minister’s] Office and their view of the possibility of complaints’. The procurement delegate approved the third extension due to the ‘strong market interest’, number of requests for clarification (approximately 200) and complaints received by the Minister’s Office.

Source: ANAO analysis of DISER records.

2.93 Table 2.4 illustrates that some key probity controls for the redesign process were not implemented or eroded during the redesign process. For example, Mills Oakley recommended including provisions in the confidentiality deed polls prohibiting industry partner personnel involved in the redesign project from assisting in the preparation of the tender response. DISER removed these provisions, despite the heightened risk. Removal of these provisions occurred at the request of industry partners, who raised concerns that these provisions would unfairly disadvantage their tender preparation.

Breach of probity for the redesign project

2.94 The redesign project was put on hold before the release of the RFT in September 2019. While DISER had planned for the redesign project to continue through the RFT, Mills Oakley advised the department that to mitigate potential probity risks the redesign project should be paused.

2.95 In a briefing to the Deputy Secretary, DISER officials stated that ‘following legal advice regarding probity issues, the redesign project has been put on hold until the RFT process is complete’. In contrast, the ANAO’s examination of DISER email records indicates that the redesign project was put on hold because of a breach of the information barriers put in place. The breach was that advice was provided to those incumbent providers involved in the redesign that effectively meant that the work they had performed had been completed and the delivery model decided, rather than being simply concept development work to present options for the department to consider and make a decision later on. This breach provided those incumbent providers with information about the tender process before the RFT was released.⁵⁵

2.96 The breach occurred on 30 August 2019 and a meeting with the probity adviser to obtain ‘urgent’ advice was held the next Monday, 2 September 2019. No record of the nature of the probity issue and how it was handled was entered in the probity register.

2.97 The probity advice recommended putting the project on hold until after completion of tender evaluation and delivery partners were selected. Nevertheless, the redesign project recommenced in December 2019, including consultation with one of the incumbent providers — Deloitte — three months before the delivery partners were selected. While an email to Deloitte stated that resumption of the process was based on ‘probity advice’, written advice was not evident from DISER’s records with the department advising the ANAO in March 2022 that this occurred via a phone call with a record of the call not made. The re-engagement of Deloitte was also not recorded in the probity register.

Were appropriate procurement records maintained?

2.100 The CPRs state that officials must maintain a level of documentation commensurate with the scale, scope and risk of each procurement. The documentation should provide accurate and concise information on the requirement for the procurement; the process that was followed; how value for money was considered and achieved; relevant approvals; and relevant decisions and the basis of those decisions.⁵⁶ As described in paragraph 1.8, the Entrepreneurs’ Programme delivery partner procurement was the largest value procurement for DISER in 2019–20, representing 37 per cent of contracts entered into by DISER.

2.101 The requirement for the procurement was recorded in the stage 1 approval endorsed by the delegate in September 2019 and the tender evaluation report. Relevant approvals were endorsed by the delegate through the department’s financial management system, and the process that was followed was set out in the tender evaluation report to the delegate.

2.102 The ANAO observed the following deficiencies in the department’s record management practices for the procurement.

• Key documentation relating to the procurement was not signed or approved by the TAP or delegate (refer to paragraphs 2.8 and 2.9).
• While the tender evaluation report was signed by the TAP members, dates were not recorded. The procurement delegate signed and dated the tender evaluation report but did not indicate whether the report was ‘approved’ or ‘not approved’.
• Incorrect or misleading statements were included in the tender evaluation report to the delegate (refer to Appendix 4 for further details).⁵⁷
• There are no records (such as minutes) of the TAP’s meetings. DISER officials provided notes in emails as evidence of TAP deliberations.
• Records of individual evaluations by TAP members are incomplete (refer to paragraph 2.25 to 2.41). Two records of individual evaluations are blank.
• There are insufficient records regarding the probity of the procurement process. For example, the probity register is not a complete record of all occurrences concerned with the probity of the procurement process (refer to Appendix 5 for more information).
• Key procurement records were not stored in the secure procurement library. During the course of the audit, DISER added approximately 200 records to the procurement library including conflict of interest declarations, correspondence with the probity adviser and TAP deliberations.

Was the procurement approach open, fair and non-discriminatory?

2.105 The CPRs state that procurements should encourage competition and outline that effective competition requires non-discrimination and the use of competitive procurement processes.⁵⁸

Open and effective competition

2.106 As described in paragraphs 2.28 and 2.46 above, DISER shortlisted tender submissions following the stage 2 technical evaluation against the first two criteria⁵⁹ with 14 of the 30 tenders assessed as ‘acceptable’ or better against those two criteria shortlisted and 16 of the tenders assessed as ‘acceptable’ or better against those two criteria not shortlisted. Shortlisting did not provide for open and effective competition to be maintained.

2.107 Figure 2.3 illustrates the extent to which shortlisting limited competition at key stages of the procurement for each of the three outcomes (growth, commercialisation, and innovation). As noted in paragraph 2.47, the shortlisting process had a significant bearing on the likelihood of tenderers being successful such that the department should have communicated in the RFT the extent to which shortlisting would be applied.

Figure 2.3: Number of tenderers at key stages of the procurement

Source: ANAO analysis of DISER records.

2.108 Figure 2.4 illustrates that the department’s approach to the procurement resulted in incumbent providers receiving greater consideration. While 20 per cent of tenders submitted for the growth and innovation outcomes were from incumbent providers, tender submissions from incumbents represented 73 per cent of shortlisted submissions.⁶⁰ In contrast, three submissions (27 per cent) from non-incumbent tenderers for growth and innovation were shortlisted, and one (Darwin Innovation Hub) was awarded a contract. The other non-incumbent that was successful — i4 Connect — was for the commercialisation outcome, for which there were no incumbent delivery partners.

Figure 2.4: Proportion of incumbent providers at key stages of the procurement

Note: There were 51 tenders submitted for the growth and innovation outcomes; 11 tenders shortlisted for the growth and innovation outcomes; and six contracts awarded for the growth and innovation outcomes.

Note: This figure does not include tenders submitted exclusively for the commercialisation outcome. Commercialisation services were not provided by a delivery partner under the previous Entrepreneurs’ Programme. No incumbent providers submitted a tender for commercialisation.

Source: ANAO analysis of DISER records.

2.109 The consequences of DISER’s shortlisting were that:

• three submissions (27 per cent) from non-incumbent providers for growth (three) and innovation (zero) were shortlisted;
• of the three non-incumbents shortlisted for growth, two were for services in Northern Territory and one was for services in Western Australia;
• in New South Wales/Australian Capital Territory, South Australia, Tasmania, Queensland, and Victoria there was no competition from non-incumbents; and
• after further shortlisting there was one non-incumbent — Darwin Innovation Hub — remaining for the evaluation against the ‘corporate and financial viability’ criterion and for the final value for money assessment.

2.110 The CPRs state that to:

ensure that SMEs can engage in fair competition for Australian Government business, officials should apply procurement practices that do not unfairly discriminate against SMEs and provide appropriate opportunities for SMEs to compete.⁶¹

2.111 ANAO analysis indicates that the department’s approach to evaluation did not include appropriate consideration of Commonwealth policies relating to SMEs (refer to paragraphs 2.40 to 2.41). The result of the evaluation was that:

• while 81 per cent of tenders submitted were from SMEs, 20 per cent (9 of 44) of those were shortlisted with 15 of the 16 tenderers (94 per cent) assessed as ‘acceptable’ or better against the two criteria used for shortlisting that were not shortlisted being from SMEs;
• 19 per cent of tenders submitted were from non-SMEs, and 50 per cent of those (5 of 10) were shortlisted; and
• three of the seven contracts awarded were to tenderers identified as SMEs.⁶²

Fair and equitable treatment

2.112 Conducting a procurement process without due regard to probity and fair dealing may leave it open to potential challenges or complaints.⁶³

2.113 Consistent with the ANAO’s analysis that the procurement did not demonstrate open and effective competition, and that tender submissions were not evaluated transparently in accordance with the published criteria and methodology, the ANAO identified a number of complaints from tenderers. Seven complaints from five tenderers were identified by the ANAO, either made to DISER or to the ANAO through the citizens contribution facility on the ANAO website (see Appendix 6 for details of the complaints and the extent to which each tenderer’s submission was evaluated against the evaluation criteria). The concerns raised in the complaints were that:

• tender prices had not been properly considered in the evaluation and that value for money was not demonstrated;
• tender submissions were not evaluated fairly and equitably such that SMEs were able to compete;
• the information requirements for the tender submission were substantial, requiring significant time and costs to prepare, and the feedback provided by the department did not provide confidence that the evaluation was an impartial assessment of the merits of the submission as a whole; and
• the process of awarding the contract lacked probity.

2.114 Following the rejection of a submission or the award of a contract, officials must promptly inform affected tenderers of the decision.⁶⁴ All unsuccessful tenderers were advised on 30 March 2020, including the two tenderers which were assessed as not meeting the minimum content and format requirements on 11 December 2019⁶⁵ and the 39 tenderers which were not shortlisted on 17 December 2019.

2.115 After receiving the department’s notification of the outcome, two unsuccessful tenderers (Respondents 2 and 44) raised concerns with the department regarding its handling of ‘competitive neutrality’ in the procurement given that the successful tenderer, CSIRO, is a Commonwealth entity. In its response to the tenderers, the department advised that:

All tender submissions were assessed against each of the evaluation criteria contained in the RFT through a comparative rating assessment process overseen by an independent probity adviser. The assessment process focused on capability and capacity to deliver the services first, before a pricing analysis was conducted. The pricing analysis and the subsequent value for money determination, which included bringing together the price assessment, assessment of risk and all other unweighted evaluation criteria determined that the CSIRO response complied with competitive neutrality requirements including those relating to pricing.

2.116 DISER’s advice to the tenderers was misleading and did not reflect the evaluation process that was conducted. Inconsistent with the advice that all tenderers were assessed against each of the evaluation criteria:

• a complete evaluation of Respondent 2’s tender submission was not conducted. Only two of the three TAP members conducted any assessment and those assessments only addressed two of the six criteria. The submission was rated ‘technically suitable’ against the first two criteria but after further consideration by the TAP the submission was not shortlisted as it was deemed ‘uncompetitive’ when compared to other submissions for the innovation outcome;
• a complete evaluation of Respondent 44’s tender submission was not conducted. Only one of the three TAP members conducted any assessment. The assessment only addressed two of the six criteria. The submission was rated ‘technically unsuitable’ and ‘uncompetitive’ against the first two criteria and was not shortlisted for further consideration; and
• neither submission was evaluated against the price, risk, corporate and financial viability and Commonwealth policies criteria.

2.117 The ANAO also observed inconsistent treatment of tenderers throughout the procurement process, such as:

• providing only 14 tenderers of the 30 tender submissions rated as ‘acceptable’ or above to the ‘internal panel experts’ for further technical evaluation (see paragraph 2.29);
• providing opportunities for select tenderers to improve or revise tender submissions (see paragraph 2.67);
• meeting with one tenderer (i4 Connect) to discuss its response to the department’s request for additional information (see paragraph 2.75) when this opportunity was not provided to another tenderer that had been asked for additional information (and which was not awarded a contract); and
• tailoring its approach to a particular tenderer (see paragraph 2.67).

3.1 To assess whether the contracts are being appropriately managed, the ANAO examined the department’s reporting of the contracts and variations, the appropriateness of the department’s contract management framework and approach to securing delivery of contract deliverables, and whether the contracts include clear performance requirements and a means to effectively measure performance against the requirements.

Have relevant contracts and amendments been accurately reported on AusTender within 42 days of a contract being entered into or amended?

3.2 The Commonwealth Procurement Rules (CPRs) establish reporting requirements for all relevant entities that enter or amend a contract value at or above the reporting threshold.⁶⁶ DISER (a non-corporate Commonwealth entity) has a reporting threshold of $10,000 with a reporting deadline within 42 days of entering or amending a contract.

3.3 Contracts with the successful tenderers were signed in March 2020. The department reported each of the contracts accurately within 42 days of being entered into. Since contract signature, there has been a total of 19 contract variations across the seven delivery partner contracts (see paragraphs 3.31 to 3.42). Three of the variations were below the $10,000 reporting threshold. Table 3.1 below outlines the department’s compliance with the reporting requirements.

Is there an appropriate contract management framework in place?

3.4 The delivery and payment of services and the ongoing management of the contract are important elements in achieving the objectives of the procurement.67 An appropriate contract management framework supports the effective management of procurement risks and the achievement of procurement objectives.68 Its purpose is to provide a clear and standardised approach to managing and administering contracts.

Contract administration

3.5 DISER’s contract management framework for the Entrepreneurs’ Programme comprises the delivery partner contracts, policy documents which relate to delivery partner performance, promotion and marketing, and business intelligence, and standard operating procedure documents.

3.6 The Australian Government Contract Management Guide requires contract management plans for complex and strategic contracts.⁶⁹ DISER’s procurement toolkit contains a contract management template for complex and/or high value contracts. Notwithstanding that the delivery partner contracts were DISER’s largest value procurement in 2019–20 and represented 37 per cent of the total value of contracts entered into by DISER and reported on AusTender for that year, the department has not developed contract management plans for the delivery partner contracts. Contract management plans are useful tools for managing risks to the success of the contract, and for ensuring that what is negotiated as value for money is achieved.⁷⁰

3.9 The department’s internal guidance also states that contract managers are responsible for assessing the risks identified during ‘the planning stages of [the] procurement process, including those that were captured in [the] risk management plan or contract management plan.’ The department has not maintained risk management or contract management plans for the delivery partner contracts.

3.10 The Australian Government Contract Management Guide states that ‘proper systems for identifying, gathering and recording relevant information will help you to manage your contract effectively and provide a clear audit and accountability trail.’⁷¹ The department has not had these proper systems to support appropriate management, including:

• the department does not maintain consolidated contracts reflecting all contract variations made;
• there is no clearly articulated process for acceptance of delivery partner quarterly reporting (refer to paragraphs 3.28 and 3.29); and
• an appropriate performance framework with clearly defined performance requirements, and a means to effectively measure performance against the expectations is not in place (refer to paragraphs 3.46 to 3.62).

Monitoring of delivery partner conflicts of interest

3.11 The delivery partner contracts require delivery partners to notify DISER ‘immediately’ in writing when a conflict of interest arises. The notification must make full disclosure of the conflict and the delivery partner must take such st eps DISER requires to resolve or deal with the conflict. Delivery partners are required to notify DISER of all actual, potential or perceived conflicts of interest of their personnel including their employees, facilitators or contractors.

3.12 DISER has not developed systematic processes or guidance for delivery partners to comply with their contractual obligations.⁷² DISER advised the ANAO in February 2022 that:

delivery partners…must identify to the department any actual, perceived or potential conflicts of interest they believe will or may arise during the delivery of the program and, where a conflict of interest is identified, specify how that conflict will be addressed and monitored to ensure that it does not compromise program outcomes.

3.13 In contrast, the ANAO’s analysis of DISER records indicates that delivery partners have not consistently complied with their contractual obligation to ‘immediately’ notify DISER of conflicts of interest, and that DISER has failed to take appropriate action when delivery partners do not meet their contractual obligations, including when actual conflicts occur. For example, a facilitator for the strengthening business program (employed by NSWBC) submitted an expression of interest for a grant under the incubator support initiative. NSWBC did not notify the department of this potential conflict of interest in accordance with its contractual obligations, but rather this potential conflict was brought to the attention of the department by a facilitator for the incubator support program. DISER took appropriate action to eliminate the conflict of interest, with the outcome being that the facilitator withdrew the application for the grant.

3.14 On another occasion, DISER did not take appropriate action when dealing with an actual conflict of interest. A member of i4 Connect’s management team placed an offer for investment in a company receiving facilitation services from i4 Connect and applying for an accelerating commercialisation grant under the Entrepreneurs’ Programme.⁷³ This conflict of interest was disclosed when the grant application was assessed by the Entrepreneurs’ Programme Committee in June 2021, nine months after the company commenced its engagement with i4 Connect.

3.15 The department sought legal advice which confirmed that this constituted an ‘actual conflict of interest’. Although the conflict of interest was not resolved, the program delegate endorsed the declaration of interest without requiring any management actions to mitigate the conflict of interest other than that i4 Connect be reminded of its contractual obligations in using information they may be privy to in their jobs for personal benefit. The company was awarded a $398,350 accelerating commercialisation grant (the full grant amount sought).

3.16 The department’s internal legal area has identified that conflicts of interest in the Entrepreneurs’ Programme are dealt with inconsistently and on an ad-hoc basis and that ‘proactive management of probity risk leads to better outcomes for both the department and for the people to whom the risk relates.’

Payment of delivery partners

3.19 The delivery partner contracts are ‘fee for services’ arrangements where DISER pays six of the seven delivery partners quarterly for services delivered. One delivery partner is paid monthly for services delivered. The ANAO’s analysis of DISER records indicates that total cumulative expenditure for the delivery partner contracts between 1 July 2020 and 31 October 2021 is $70.8 million, comprising of:

• $62.5 million of regular service charge payments;
• $8 million of service establishment payments paid to six of seven delivery partners upon engagement of 50 per cent of the required specified personnel prior to services commencement⁷⁴; and
• $278,216 of pre-service payments paid to all delivery partners on completion of the induction for specified personnel prior to services commencement.

3.20 DISER has not established robust service verification procedures for the delivery partner contracts. Internal departmental guidance on making payments states that on receipt of an invoice, the receiving official must ‘obtain assurance that goods and services have been received.’ Service verifications were not recorded consistently, with 15 per cent of invoices analysed by the ANAO not having evidence of services verification, resulting in $11.4 million being paid without evidence that services had been delivered. While on the remaining occasions verifications have been recorded, it is also not clear how services are verified prior to payment of invoices (refer to paragraphs 3.28 and 3.29 for more information).

Contract deliverables

3.21 Table 3.2 sets out the contractual deliverables and the milestone dates that the delivery partners are required to meet.

Table 3.2: Deliverables

Source: ANAO analysis of delivery partner contracts.

Specified Personnel

3.22 DISER required engagement of all specified personnel by the services commencement date of 1 July 2020.⁷⁵ The ANAO’s analysis of DISER records indicates that five of seven delivery partners did not engage the required number of specified personnel by this date. While this requirement was not met, there were no deductions made from the first quarterly payments. In March 2022, DISER advised the ANAO that it ‘exercised discretion when processing first quarterly payments. The number of facilitators were considered sufficient to support service delivery’. In its advice, the department noted that the only option available in the contract to vary payments was ‘to suspend payment due to the contractor not complying with its obligations under the contract’.

Work and communications plan

3.23 The draft work and communications plan is developed by the delivery partner and details how the delivery partner will deliver the services, including activities, targets and qualitative measures. Once the department accepts the work and communications plan the delivery partner must perform the services in accordance with the plan. Table 3.3 outlines delivery partner compliance with work and communications plan contractual deadlines.

Table 3.3: Delivery partner compliance with work and communications plan deadline

Source: ANAO analysis of delivery partner work and communications plans.

3.24 Table 3.3 illustrates that no delivery partner has complied with the contractual deadline for this deliverable, and that performance deteriorated further in 2021–22 compared with 2020–21. For example, in financial year 2020–21 the average delay for submission across all delivery partners was 94 days (more than three months) and up to four months late, and in financial year 2021–22 the average delay was 163 days (more than five months) and up to eight months late. This situation reinforces the risks to service outcomes where contract managers do not, through their management of the contract, reinforce the importance of meeting requirements from the commencement of the provision of services, as identified in a recent ANAO audit report.⁷⁶

3.25 As stated in paragraph 3.23, once the work plan is accepted by the department, the delivery partner must perform the services in accordance with the plan. The delay means that the services for the financial year have not been agreed at the beginning of the financial year. This is indicative of the lack of an appropriate contract management framework being employed by the department.

Quarterly management report

3.26 The quarterly management report contains updates on the delivery partner’s year-to-date performance against the work and communications plan. The report must also provide information on current and emerging trends in the industry sectors and jurisdictions covered by the delivery partner. Table 3.4 sets out the timeliness of delivery partner quarterly reporting and the relationship between quarterly invoice payment and receipt of the quarterly report.

Table 3.4: Timeliness of delivery partner reporting

Note a: DISER pays i4 Connect on a monthly basis (refer to paragraph 3.19), which is not dependent on the submission of the quarterly management report or any other deliverable.

Source: ANAO analysis of DISER quarterly reports and invoices.

3.27 Table 3.4 indicates that there are varying levels of compliance with quarterly report submission timeliness. Fifty-one per cent of the quarterly reports submitted were outside of the contractually required deadline with an average delay of 15 days. Two delivery partners — Ai Group and Deloitte — have submitted all quarterly reports within the contractual deadline. In contrast, Darwin Innovation Hub has not submitted any quarterly reports within the contractual deadline, with an average delay of 18 days.

3.28 The delivery partner contracts state that payment of invoices is subject to the department’s acceptance of the quarterly management report.⁷⁷ Table 3.4 highlights that the department has not applied the terms of the contract, with 60 per cent of invoices recorded as having been verified and processed for payment without submission of the quarterly reports by the delivery partners.

3.29 The absence of a clearly documented acceptance process of this key deliverable is another significant flaw in the department’s administration of the delivery partner contracts. DISER has not retained appropriate records to demonstrate that this acceptance occurs on a quarterly basis for all delivery partners. In February 2022, DISER advised the ANAO that ‘acceptance of the reports will be provided via email and/or discussed as part of regular…meetings with delivery partners. In response to an ANAO request for a copy of emails to delivery partners confirming formal acceptance of quarterly reports in accordance with the contract, the department provided in May 2022 two emails sent internally within the department but not to the delivery partners.

Contract variations

3.30 When contract variations amount to a significant change to the contract or significantly vary the scope of the contract, consideration should be given to whether⁷⁸:

• other potential suppliers may have responded differently to the amended contract scope in the tendering process which may have produced a different value for money outcome; or
• the variation may compromise the original procurement’s value for money assessment.

3.31 As at 18 February 2021, there has been a total of 19 variations to the delivery partner contracts since the original contracts were signed in March 2020. All delivery partner contracts have increased in value, except for the CSIRO contract due to the removal of the incubator support services (see paragraphs 3.41 to 3.42 below). The overall cost of the contracts has increased by $19.4 million (13 per cent).

3.32 The contract variations have varied the scope of the services to be provided, including:

• the introduction of two new program elements, ‘strengthening business’ and ‘strengthening northern Australia business’; and
• the removal of a program element, ‘incubator support’, aligned to the ‘innovation’ outcome.

New program elements

3.33 Since the contracts were signed, DISER has introduced two new program elements to the Entrepreneurs’ Programme, ‘strengthening business’ and ‘strengthening northern Australia business’, with facilitation services to be provided by existing delivery partners.

3.34 Strengthening business was announced as part of the Australian Government’s bushfire recovery package for small business on 11 March 2020 and included 21 new facilitators to be engaged by existing delivery partners at an expected cost of $12.8 million.⁷⁹

3.35 Eight of the 19 (42 per cent) contract variations relate to the introduction of the strengthening business program element.

3.36 On 6 May 2021, the Australian Government announced the Northern Australia Development Program (NADP).⁸⁰ ‘Strengthening northern Australia business’ is implementing advisory services in support of the NADP. In January and February 2022, DISER varied the contracts with Deloitte, Ai Group and Darwin Innovation Hub to include an additional 11 new facilitators at an expected cost of $5.5 million for the strengthening northern Australia program element.

3.37 The new program elements were not part of the original procurement and did not form part of the original value for money considerations that lead to the decision to engage the delivery partners.

3.38 The delivery partners, Ai Group, NSWBC and Business SA, were selected to engage the facilitators for strengthening business on the basis that they provided ‘growth’ services in the bushfire affected regions of New South Wales, South Australia and Victoria. The contract with Deloitte was varied in September 2020, to provide three ‘digital solutions specialists’ at a cost of $1.8 million for the strengthening business program element.

3.39 Deloitte, Ai Group and Darwin Innovation Hub were selected to deliver strengthening northern Australia business services due to their geographic coverage in providing growth services, track record on business advisory services and expertise. The request for delegate approval also recorded ‘urgency’ as the justification for the decision to approach the existing delivery partners as a separate standalone procurement would have added months to the timeframe, which would reduce the effectiveness of this approach. This is notwithstanding that Government approval for the program was provided in May 2021 and the contract variations were not signed until late January 2022 and early February 2022 (eight months after Government approval).

3.40 Reflecting the general nature of the contracts, and their lack of performance measures, service levels and targets (see paragraphs 3.51 to 3.56), the contract variations did not include any specific requirements for the strengthening business or strengthening northern Australia business services. The variations to the contracts were to increase the number of specified personnel (facilitators) and the contract value.

Incubator support services

3.41 As discussed in paragraph 2.60, CSIRO was awarded the contract to deliver innovation and incubator support services. Incubator support services was included as a program element in the RFT but discontinued in May 2021 as part of a savings measure announced in the 2021–22 federal budget. The savings achieved by early closure of the program was approximately $9.8 million over the forward estimates.

3.42 To partially achieve this savings measure, the CSIRO contract was varied in June 2021 to remove the two facilitators associated with delivery of the incubator support services. Removal of these two facilitator positions reduced the total contract value from $17.3m to $16.2m.

Approvals

3.43 Section 23 of the Public Governance, Performance and Accountability Act 2013 states that accountable authorities of a department may enter into, vary or administer a contract. DISER’s accountable authority instructions (AAIs) state that the delegate must not vary an arrangement unless it is within the scope of their delegation. DISER has not retained a copy of the necessary section 23 approvals for 10 of the 19 contract variations signed to date.

Do the contracts set clear performance requirements, and a means to effectively measure performance against the requirements?

3.46 Performance management is key to delivering value for money and involves ensuring that goods or services are delivered as required under the contract.⁸¹ To assess whether the contracts set clear performance requirements, and a means to effectively measure performance against the requirements, the ANAO examined:

• the request documentation, including the draft contract;
• the signed delivery partner contracts; and
• the relevant policy documents referred to in the contracts.

3.47 The draft contract developed and released as part of the Request for Tender (RFT) on 27 September 2019, advised that the performance management regime had not been fully settled, and contained an indicative performance management regime which stated that:

quantitative and qualitative measures and targets to assess performance will be developed by the department in consultation with prospective delivery partners during the contract negotiation stage.

Procurement of consultant to develop a performance management framework

3.48 DISER engaged consultants, Bevington Group, to produce a performance framework to support contract negotiations with successful tenderers at a cost of $56,760. The procurement did not comply with the CPRs and did not transparently demonstrate value for money.

• DISER identified the consultant through a ‘Consultancy and Business Services’ panel, but that panel had expired, and instead employed a limited tender procurement method.
• In justifying the limited tender procurement approach, the department cited a ‘short timeframe’ as its justification. While ‘reasons of extreme urgency brought about by events unforeseen⁸²’ is a condition for limited tender, the ANAO’s analysis is that the short timeframes were not brought about by unforeseeable events given the RFT had foreshadowed that the performance framework would need to be developed.
• DISER did not estimate the expected value of the procurement before deciding on the procurement method.
• Evidence of the approval to approach the market or the original request for quote was not retained.
• Quotes were not requested from multiple providers.
• DISER did not consider the costs in its value for money justification, and only focused on the short timeframe and Bevington’s prior program knowledge.

3.49 In December 2019, the department accepted delivery of the new performance framework from Bevington. This was despite departmental officers recording that they were not satisfied with the work completed by Bevington. The framework was changed by the department before contract negotiations commenced to align it more closely with the outcomes of the redesign work. In March 2022, DISER advised the ANAO that:

Bevington was engaged as an expert adviser to assist the department to consider performance frameworks and outcome assessment. The work Bevington completed informed the final performance framework and a range of ways the program could consider measuring activities, efficiency and outcomes.

DISER proactively contract managed the engagement with Bevington. There were some delays with delivery of key activities under the contract due to Bevington’s availability. However, these deliverables were rectified and managed to the department’s satisfaction.

Contracted service requirements

3.50 The delivery partner contracts require three types of services to be provided:

• business facilitation and advisory services by specified personal (facilitators);
• promotion and marketing to promote the Entrepreneurs’ Programme; and
• provision of market and business intelligence to the department.

3.51 The contracts set out the general requirements in relation to the provision of those services for all delivery partners. While ‘service level’ is a defined term in the contract⁸³, the contracts do not contain specific service levels that each delivery partner must achieve or exceed.

3.52 The contracts refer to external supporting documents that provide further guidance on performance expectations for each of the three services, including:

• performance and capability policy;
• communications and marketing guidelines; and
• market and business intelligence framework.

Performance management

3.53 The Australian Government Contract Management Guide states that performance management involves:

• performance measurement: collecting accurate and timely data of supplier performance;
• performance assessment: deciding on whether the supplier’s performance meets the standards contained in the contract; and
• performance adjustment: understanding what constitutes good performance, correcting areas of underperformance, amending the contract requirements to meet changing needs.⁸⁴

3.54 The delivery partner contracts state that

performance will be assessed and managed by the department using a combination of quality, volume and outcome measures to evaluate each delivery partner’s contribution to the overall performance of the program and its objectives.

3.55 The contracts do not contain any performance measures to assess delivery partner performance.

3.56 The contracts state that the process for performance management will be detailed in the performance and capability policy. The policy contains a mix of qualitative and quantitative measures and ‘expected behaviours’ the delivery partners will be assessed against. No targets are set in the policy against the performance measures and no process is set out to assess performance.

3.57 As outlined in paragraph 3.23, delivery partners must perform services in accordance with the work and communications plan, which should include ‘targets and qualitative measures.’ The ANAO’s analysis of the 2020–21 work and communications plans identified a clear disconnect between the performance and capability policy and the work and communications plans. None of the delivery partner work and communications plans include targets for all measures contained in the performance and capability policy.⁸⁵

3.58 While there is a lack of an effective performance management framework in place, DISER does maintain oversight of delivery partner and facilitator performance through portfolio management tool dashboards.⁸⁶ The portfolio management tool is intended to provide an overview of:

• services and how they have progressed;
• collaboration activity such as referrals and uptake of specialist services
• client demographics such as ‘full-time equivalent’, ‘revenue’, ‘export’ and ‘research and development’ activity;
• outcomes being achieved in actions implemented, their impact and business performance; and
• implementation activity — where client capability is developed.

3.59 The department holds quarterly reviews of dashboards with delivery partners.

Performance adjustment

3.60 There are benefits from linking contract payments to the achievement of key performance indicators.⁸⁷ The draft contract included a provision by which DISER could reduce quarterly payments for failure to meet contracted service levels. The signed contracts do not contain this provision or a similar provision which provides for adjustment of payment in relation to service levels. None of the successful tenderers had requested that this clause be removed. Rather, its removal was initiated by DISER. In February 2022, DISER advised the ANAO that the clause

was removed from the draft contract (released as part of the RFT). This was replaced with the Performance and Capability Policy approach that was referenced in the final contracts issued to all Delivery Partners.

3.61 The performance and capability policy does not contain a provision that allows for adjustment of payments. The only contractual rights available to the department to manage underperformance are rectification notices, withholding of payment until underperformance is addressed, reducing the scope of services or rescinding the contract.

3.62 Without targets or any basis for comparison, the department is unable to assess whether delivery partner performance is satisfactory and is also unable to demonstrate that the contracts are delivering against the expectations that informed the decision to select the successful tenderers.

Appendix 1 Entity response

ANAO comments on Mills Oakley response

1. Paragraphs 2.74 and 2.75 sets out the work undertaken by Mills Oakley in addition to probity advice.
2. The ANAO’s finding, set out in paragraph 2.76, was that DISER’s decision to source probity and procurement process advice from the same adviser adversely affected the independence of the probity advice.
3. In finalising the report, while the ANAO considered Mills Oakley’s suggested editorial amendments to the proposed report, no amendments were made to paragraphs 2.75 and 2.76 as the ANAO concluded that the wording accurately reflects the contemporaneous evidence of the conduct of the procurement of delivery partners for the Entrepreneurs’ Programme.

ANAO comments on DMV Consulting response

4. The DMV Consulting report stated in its ‘Pricing Evaluation Approach’ that ‘The Pricing Assessment process involved an analysis of all information provided by each Tenderer in Attachment 10 of the submitted Tender documents. A review was also completed of their overall tender submission to gain an understanding of the relationship between the prices quoted and their overall approach to the delivery of services, particularly in regard to programme management and travel assumptions.’ The Darwin Innovation Hub tender response set out that the tenderer would be providing 5 Full Time Equivalent (FTE) personnel through 11 individuals with a table in the tender response identifying those personnel and the contribution each person would make to the overall 5 FTE resources being tendered.
5. Refer to paragraph 2.32 in the audit report in relation to the impact of the pricing evaluation error.
6. The audit approach reflects that the audit was of the Department of Industry, Science, Energy and Resources pursuant to the Auditor-General Act 1997 (the Act) and that, while section 18B of the Act enables the Auditor-General to audit the performance of Commonwealth partners such as DMV Consulting, this audit did not employ that power. Consistent with the ANAO’s usual practice as well as principles of transparency and accountability, the ANAO report has named the consultants employed by the department in conducting this procurement. As required by the Commonwealth Procurement Rules (CPRs), the department has also publicly reported the names of the consultants it engaged on AusTender.

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021–22 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. During the course of the audit, the ANAO did not observe any change in the Department of Industry, Science, Energy and Resources’ approach to the procurement of delivery partners for the Entrepreneurs’ Programme.

Appendix 3 Evaluation criteria in the RFT

Source: DISER Request for Tender documentation.

Appendix 4 Errors identified by the ANAO in the department’s tender evaluation report

Source: ANAO analysis of departmental records.

Appendix 5 Probity plan requirements not fully implemented

Source: ANAO analysis of DISER records.

Appendix 6 Tenderer complaints about the procurement

Source: ANAO analysis of DISER records and submissions made to the ANAO.

Appendix 7 Non-compliances with the Commonwealth Procurement Rules

Source: ANAO analysis of DISER records.

Appendix 8 Non-compliance with ethical requirements

1. The Australian Parliament has established requirements in the Public Governance, Performance and Accountability Act 2013 (PGPA Act) including to require the Commonwealth and Commonwealth entities to use and manage public resources properly (section 5). The accountable authority for an entity responsible for relevant money has a duty under section 15 of the PGPA Act to promote the proper use of the money for which the accountable authority is responsible. ‘Proper’, when used in relation to the use or management of public resources, means efficient, effective, economical and ethical (section 8).

2. The Department of Finance PGPA Glossary defines ethical as:

the extent to which the proposed use of public resources is consistent with the core beliefs and values of society. Where a person behaves in an ethical manner it could be expected that a person in a similar situation would undertake a similar course of action. For the approval of proposed commitments of relevant money, an ethical use of resources involves managing conflicts of interests, and approving the commitment based on the facts without being influenced by personal bias. Ethical considerations must be balanced with whether the use will also be efficient, effective and economical.

3. The Australian Parliament has also established, through the Public Service Act 1999 (PS Act), the APS Values set out in section 10. Subsection 10(2) states that ‘The APS demonstrates leadership, is trustworthy, and acts with integrity, in all that it does’. The APS Commissioner has made directions under the PS Act including in subsection 16(f) requiring accountability of APS members by ‘being able to demonstrate clearly that resources have been used efficiently, effectively, economically and ethically’. A mandatory code of conduct is set out in section 13 of the PS Act for APS employees.

4. PGPA Act requirements, including ethical requirements, directly inform key public sector resource management frameworks for specific Australian public sector activities addressed through performance audits. These frameworks contain ethical requirements specific to the activity they regulate. For government procurement, ethical expectations are set out in paragraphs 6.1–6.8 of the Commonwealth Procurement Rules (CPRs), made by the Minister for Finance under the PGPA Act. For the ANAO, in conducting performance audits of procurement activities in entities subject to the PS Act, compliance with the CPRs is assessed against the background of the requirements of the PS Act.

5. In conducting performance audits of entities, the ANAO obtains evidence to inform an assessment of whether the audited entity executes its activities in accordance with the requirement to promote proper use of public money. Findings may be made as to whether the use or management of public money was efficient, effective, economical and ethical. In forming an overall conclusion in a performance audit, the ANAO may also form a view on whether the entity’s activities have been executed in accordance with both compliance with the Rules framework and the intent of that framework, including the requirements of the PS Act for the APS (the entity) to act with integrity in all that it does.

6. Where ANAO findings or a conclusion are made as to whether the use or management of public resources by the entity has been ethical, it is a matter for an accountable authority to assess whether the audit findings in the particular case reflect the broader posture of the entity or relate to individual APS staff conduct.

Source: ANAO analysis of DISER records.

Appendix 9 Portfolio management tool dashboard extract