Introduction

The Regional Land Partnerships program

1.1 The Regional Land Partnerships (RLP) program is a major component of the National Landcare Program Phase Two.³ The RLP program is providing $450 million over five years from 2018–19 to 2022–23 for services that contribute to the achievement of six outcomes under Australian Government responsibilities (see Table 1.1).

Table 1.1: Regional Land Partnerships program five-year outcomes

Note a: Ramsar sites refer to wetlands protected under the Convention on Wetlands of International Importance (the Ramsar Convention). Australia has 65 Ramsar sites covering more than 8.3 million hectares.

Note b: The Threatened Species Strategy outlines key action areas and target measures to halt the decline of Australia’s threatened species and support their recovery.

Note c: The EPBC Act identifies threatened species as: extinct; extinct in the wild; critically endangered; endangered; vulnerable; or conservation dependent.

Note d: World Heritage Lists are maintained by the United Nations Educational, Scientific and Cultural Organization (UNESCO). Australia has 20 properties on the World Heritage List.

Note e: The EPBC Act lists threatened ecological communities under the categories: critically endangered; endangered; or vulnerable.

Source: Department of Agriculture, Water and the Environment.

1.2 The RLP program is funded through the Natural Heritage Trust of Australia Account (NHT Account), established under the Natural Heritage Trust of Australia Act 1997 (NHT Act).⁴ The Natural Heritage Ministerial Board, which comprises the Environment Minister and the Agriculture Minister, supports the design and delivery of the program. The Department of Agriculture, Water and the Environment (DAWE) administers the program, supports the Board, and delivers program governance, risk management, and financial reporting in accordance with the NHT Act.⁵

1.3 The Joint Executive Committee (JEC) provided oversight of the RLP program from August 2017.⁶ Membership comprised nine executive representatives from the Department of the Environment and Energy (DoEE) and the Department of Agriculture and Water Resources (DAWR). In March 2020, following the Machinery of Government changes that merged the two departments into DAWE, the JEC was superseded by the RLP Management Committee. The new committee retained membership from environment and agriculture divisions of DAWE and met until July 2020.⁷ Ongoing management and oversight of the RLP program is currently provided by the First Assistant Secretary, Biodiversity Conservation Division.

1.4 RLP program services are being delivered in 54 management units across Australia.⁸ Each management unit represents a catchment area or geographical region. Management units are based on existing natural resource management (NRM) regions, which were established in 2002–03 under the Natural Heritage Trust program (see Figure 1.1). Each region is overseen by an NRM organisation, which are regional bodies responsible for the delivery of various regional NRM initiatives, including the RLP program.

The procurement approach

1.5 Previous NRM initiatives, including Phase One of the National Landcare Program, were primarily delivered through grants programs.⁹ Parliamentary reviews and ANAO audits of these initiatives raised findings relating to lack of meaningful performance information, difficulties in demonstrating value for money, and poorly defined and coordinated targets and priorities.¹⁰

1.6 In response to the findings of these reviews and audits, the Australian Government proposed an open procurement approach to deliver the RLP program. This approach was intended to provide greater clarity and transparency on purchases, reporting of services delivered, and project outcomes.¹¹

1.7 The procurement of service providers for the RLP program involved five tenders conducted between December 2017 and June 2019. At the conclusion of the procurement process, DAWE had executed 50 services agreements to deliver 225 projects in 54 management units.¹²

1.8 The contractual arrangement for the RLP program comprises an overarching services agreement with each service provider, with work orders separately executed under the agreement for each project.

• Services agreements specify the ‘core services’ that are to be delivered by the service providers for the duration of the program for each management unit. These include NRM planning for the management unit, community engagement activities, and delivery and monitoring of projects.
• Project work orders specify project-specific requirements, including project service deliverables for each project. Examples of deliverables include number of hectares treated for weeds, number of community engagement workshops, and kilometres of fencing installed.

Regional Land Partnerships projects

1.9 Of the 225 projects funded under the RLP program, 159 are environment projects (addressing Outcomes 1, 2, 3 and 4) and 66 are agriculture projects (addressing Outcomes 5 and 6). Twenty-two projects have been completed as at 9 September 2021, comprising:

• 17 projects in 2019 (five of which were agriculture projects);
• three projects in 2020; and
• two projects in June 2021 (one agriculture project).

1.10 RLP projects cover a range of activities that support achievement of the six program outcomes. For example: controlling pests such as weeds and feral animals; revegetating habitats; establishing breeding sites and populations; increasing water and soil quality; and increasing understanding of sustainable land management practices.

1.11 Service providers nominate one of the six RLP program outcomes towards which their project will principally contribute. These are called the primary outcomes (see Figure 1.2).

Figure 1.2: Regional Land Partnerships projects by primary outcome

Source: ANAO analysis of RLP data.

1.12 DAWE reported that as at 30 June 2020, the total contract value of the 225 projects was $448.6 million.¹³ As at 30 June 2021, $263.4 million (59 per cent) has been spent on services delivered under the contracts.

1.13 A budget of $49.2 million over 2018–19 to 2022–23 was allocated from the NHT Account to the departments for the administration of the RLP program, with:

• $39 million allocated to DoEE, comprising Average Staffing Level (ASL) funding of $32.3 million and professional services funding of $6.7 million; and
• $10.2 million allocated to DAWR, comprising ASL funding of $4 million and professional services funding of $6.2 million.

Rationale for undertaking the audit

1.14 Effective management of Australia’s environment and natural resources is important to ensure conservation of nationally and internationally significant species, ecological communities, and heritage values. Sustainable land management practices support increased agricultural productivity with reduced environmental impact.

1.15 Previous Auditor-General reports examining the delivery of NRM activities have identified issues such as the clarity of assessment and selection processes, the implementation of risk-based compliance strategies, and performance monitoring.¹⁴ This audit examined DAWE’s effectiveness in implementing the RLP program. It provides assurance over the RLP program’s value for money, governance and monitoring arrangements, and the delivery of intended outcomes.

Audit approach

Audit objective, criteria and scope

1.16 The objective of the audit was to examine DAWE’s effectiveness in implementing the RLP program.

1.17 To form a conclusion against this objective, the following high-level criteria were adopted:

• Did the procurement processes support the achievement of value for money?
• Does the department effectively manage service provider performance?
• Does the department effectively monitor achievement of program outcomes?

1.18 The audit focused on DAWE’s administration of the RLP program. The audit did not examine the outcome of National Landcare Program Phase One or National Landcare Program Phase Two’s other sub-programs.¹⁵

Audit methodology

1.19 The audit methodology included:

• examination of DAWE documentation;
• assessment of DAWE processes; and
• interviews with relevant DAWE staff.

1.20 The ANAO received 18 submissions from the public via the citizen contribution facility on the ANAO website. Key points from the submissions are summarised in Appendix 4.

1.21 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $356,500.

1.22 The team members for this audit were Jennifer Myles, Se Eun Lee, Aden Pulford, Michael White and Corinne Horton.

2.1 Paragraph 4.4 of the Commonwealth Procurement Rules (CPRs) states that ‘officials responsible for a procurement must be satisfied, after reasonable enquiries, that the procurement achieves a value for money outcome’.¹⁶

2.2 To assess whether the procurement processes met this requirement, the ANAO examined whether:

• the approved projects align with the program outcomes;
• the procurements complied with the CPRs;
• the procurement processes were conducted in accordance with the tender evaluation plans; and
• procurement risks were managed appropriately.

Do the approved projects align with the program outcomes?

The procurement processes

2.3 The procurement of service providers for the RLP program involved five tenders conducted between December 2017 and June 2019.¹⁷ The initial open tender received 44 submissions that complied with the tender requirements.¹⁸ The submissions comprised one response for each management unit, with one respondent applying for multiple management units as a consortium. Of the 44 responses, 39 tenders were assessed to be suitable, covering all states and territories except for Queensland and two of three management units in Tasmania.

2.4 As the initial tender did not achieve its intended purpose of procuring a service provider for each of the 56 management units, four subsequent tenders were undertaken (see Table 2.1).

2.5 Figure 2.1 outlines the timeline of the procurements.

Figure 2.1: Timeline of the procurements

 

 

Source: ANAO analysis of DAWE records.

2.6 At the conclusion of the five procurements, DAWE had contracted a service provider for 54 of the 56 management units. DAWE did not contract service providers for the Torres Strait and Marine management units. The National Landcare Program website notes that ‘separate arrangements are in place for the Torres Strait and Marine management units to support natural resource and environment projects, recognising the unique circumstances characterised by these areas.’¹⁹

2.7 DAWE used similar assessment and selection processes for the five procurements. This process is illustrated at Figure 2.2. The key participants in the tender evaluation process included:

• NHT delegate — responsible for the approval of the procurement plan, screening report, and draft and final tender evaluation reports;
• RLP Tender Steering Committee — provided general oversight and quality assurance of the tender evaluation process and the project allocation process; and
• tender evaluation panels — responsible for evaluation and scoring of tenders against the evaluation criteria.

2.8 The tender evaluation process was conducted concurrently with the project allocation process, which identified the initial list of projects to commence under the RLP program (see paragraphs 2.12–2.20).

Figure 2.2: Tender evaluation and project allocation process

 

 

Note a: The figure illustrates stage one of the project allocation process. There were two subsequent stages that resulted in more projects being procured. See paragraph 2.15.

Note b: The tender evaluation criteria were: knowledge of natural resource management; capability and capacity to deliver core services; capability and capacity to deliver project services; experience; price; and risk.

Source: ANAO summary of DAWE processes.

Program logic

2.9 In June 2018, DAWE developed a high-level program logic for each of the six program outcomes as part of the RLP Evaluation Plan.²⁰ The RLP Evaluation Plan states that the program logics were developed to support consistent alignment of all projects and services with the program outcomes, from the tender process to the reporting and evaluation activities.

2.10 The program logics provide guidance on:

• core services and project services that are expected to contribute to the achievement of each program outcome; and
• monitoring data required to demonstrate progress towards achievement of program outcomes (Figure 2.3).²¹

Figure 2.3: Program logic structure

 

 

Note a: Outcome 5 refers to ‘Condition indicators’ and Outcome 6 refers to ‘Adaptation indicators’.

Source: ANAO analysis of RLP program logic.

2.11 The ANAO assessed the program logics to determine whether there was a logical progression from project services to the achievement of program outcomes. All program logics were assessed as supporting the achievement of each outcome. The program logics for Outcomes 1 to 4 incorporate information relating to threats and treatments outlined in relevant plans and strategies such as recovery plans. The linkages and assumptions in the program logics for Outcomes 5 and 6 support the adoption of improved land management practices and adaptation.

Alignment of projects with program outcomes

2.12 During the procurement, tenderers were required to submit, for at least one environment outcome and one agriculture outcome:

• detailed project designs for projects proposed to commence in year one of the program; and
• high-level project proposals for projects proposed to commence in years two to five of the program.

2.13 A project allocation process based on these project designs was conducted parallel to the tender evaluation process (see Figure 2.2). The key participants in the project allocation process included:

• RLP Tender Steering Committee — which provided general oversight and quality assurance of the tender evaluation process and the project allocation process;
• technical advisers — responsible for assessing project designs and project proposals to ensure that they are technically and scientifically sound;
• project assessors — assessed and scored project designs submitted by tenderers against the project evaluation criteria;
• National Allocation Review Team (NART) — reviewed the projects as assessed by the project assessors, with a view to recommending projects that represent value for money at a whole of program level; and
• project allocation coordinators — assisted project assessors and the NART to complete their work within the timeframe.

2.14 The aim of the project allocation process was to identify preferred projects to be contracted, taking into account:

• the suitability of projects with reference to the relevant Commonwealth plans and strategies, such as recovery plans or heritage strategies;
• the merit of individual project designs; and
• the broader objectives of the RLP program.

2.15 The project allocation process involved three stages (see Table 2.2).

Table 2.2: Three stages in project allocation

Source: ANAO summary of DAWE process.

2.16 All three stages involved merit assessment against the following project evaluation criteria:

• technical merit and methodology (including consideration of how the project will contribute to primary outcome²² and investment priority²³);
• contribution to secondary outcomes and investment priorities;
• community engagement and partnerships; and
• Indigenous participation.

2.17 Project assessors scored the projects against each criterion and rated each project based on project risk and value for money.

2.18 The final scores and ratings, along with a list of 114 recommended top-scoring projects in each management unit, were provided to the NART. NART reviewed the project designs to determine which combination of projects was best suited to achieving the intended outcomes and provide value for money at a national scale. National factors considered by the NART include whether the proposed combination of projects:

• were within the yearly and overall budget for RLP;
• broadly aligned with the indicative split of funding between core services and the six outcomes; and
• allowed for distribution of funds between management units to support viability of service providers.

2.19 In Stage 1 of the project allocation process, the NART review resulted in changes to the list of recommended projects for 67 projects. NART’s recommendations were reviewed by the Steering Committee, who suggested further changes to eight projects to reduce transition issues likely to result from significant change in funding levels from previous NRM programs.

2.20 NART provided its final recommendation to the delegate in a draft project allocation report. The draft project allocation reports were approved by the delegate prior to the contract negotiation stage, and then finalised for approval prior to the execution of project work orders.

Project logic alignment with program outcomes

2.21 Service providers were required to develop project plans for projects selected during the project allocation process. These project plans included a project logic, which outlined how the services to be delivered under the project would contribute to program outcomes. The plans were attached to the project work orders and became part of the services agreements after approval.

2.22 The ANAO assessed a sample of 39 project logics out of 225 projects for alignment with the program outcomes.²⁴ While the presentation of outputs, outcomes and monitoring indicators varied considerably, all project logics tested aligned with the program outcomes. Alignment was further supported by additional material included in project plans.²⁵

Did the procurements comply with the Commonwealth Procurement Rules?

2.23 The CPRs identify mandatory requirements with which Commonwealth officials must comply when undertaking procurement activities. DAWE has an internal procurement policy that outlines requirements for staff when applying the CPRs. The ANAO examined whether DAWE complied with all mandatory requirements under the CPRs.

Compliance with the Commonwealth Procurement Rules

2.24 DAWE developed a procurement plan for each of the five procurement processes. The procurement plans outlined the procurement scope, estimated value, and procurement methodology. A probity plan (see paragraph 2.29) and a risk assessment (see paragraphs 2.44–2.51) were included in each procurement plan. The procurement plans were approved by the delegate prior to the release of the tender documents.

2.25 Approaches to market met the relevant content and timing requirements under the CPRs.²⁶ The tender evaluation processes were conducted in accordance with the tender evaluation plans developed for each procurement. This is examined in more detail at paragraphs 2.35–2.39.

2.26 The CPRs outline specific circumstances in which an entity can conduct a limited, rather than open, tender process.²⁷ DAWE received advice from legal and probity advisers prior to conducting the Queensland Limited Tender. The procurement plan documented the rationale for conducting a limited tender, which complies with the CPRs.

2.27 The ANAO identified five instances of non-compliance with the mandatory requirements under the CPRs (see Table 2.3).

Table 2.3: Non-compliance with Commonwealth Procurement Rules

Note a: Paragraph numbers refer to the version of the CPRs that commenced on 1 January 2018, which was in force when the Initial Tender was released. The CPRs have been updated three times since: on 1 January 2019, 20 April 2019, and 1 December 2020 (current).

Note b: The omissions did not affect the outcome of the evaluation and tenders for both management units were successful.

Note c: Both tenderers were unsuccessful. It is unknown whether the outcome would have been affected by the allowance of the hyperlinked material for the second tenderer.

Source: ANAO analysis of tender evaluation documentation.

Management of conflicts of interest

2.28 The CPRs require that procuring entities act ethically throughout the procurement, including by recognising and dealing with actual, potential and perceived conflicts of interest.²⁸

2.29 DAWE established an appropriate probity framework for the procurements. A departmental probity contact officer and an external probity adviser were appointed for each procurement process. A probity plan was prepared for each procurement that required:

• any personnel involved in the procurement to declare conflicts; and
• any conflicts declared to be referred to the departmental probity contact officer for advice.²⁹

2.30 A probity register was established for each procurement to track participants’ compliance with probity obligations.

2.31 DAWE developed a combined probity register for the Initial Tender and the Queensland Limited Tender that included 278 personnel. The register did not clearly identify the role of each person and in which tender they participated. Therefore, the ANAO compiled a list of key personnel derived from tender evaluation documents and communications with DAWE for the purpose of analysis outlined in Table 2.4.

2.32 Separate registers were maintained to manage probity forms for the Southern Queensland, North West Tasmania and South Tasmania tenders. The ANAO assessed whether the listed participants’ conflict of interest declarations were appropriately documented for all five procurements (see Table 2.4).

Table 2.4: Conflict of interest declarations

Note a: The ANAO compiled a list of key personnel comprising the delegate, project sponsors, RLP Steering Committee members, RLP Steering Committee secretariat, tender evaluation panellists, and technical advisers.

Source: ANAO analysis of DAWE records.

2.33 In four of the five procurement processes, DAWE did not document conflict of interest declarations for all participants, and there is no documented management review of or response to declared conflicts. ANAO’s analysis indicates that declared interests largely related to general contact with the NRM organisations or the landcare sector in the course of business-as-usual work in the department.

2.34 For future procurements, DAWE should ensure that declared conflicts of interest are managed in accordance with the CPRs and the probity plans.

Was the process conducted in accordance with the tender evaluation plan?

2.35 DAWE developed a tender evaluation plan for each of the five procurements. The tender evaluation process is illustrated in Figure 2.2.

Tender screening

2.36 The tender evaluation plans required that the tenders be screened to ensure that they were received by the closing time and met the minimum content and format requirements.

2.37 For each of the five procurements, screening reports were prepared and approved by the delegate. These reports outlined the receipting process undertaken, which complied with the tender evaluation plans.

Tender evaluation

2.38 Tender evaluation reports were prepared and approved for each of the five procurements. The evaluations were conducted in accordance with the stated evaluation criteria, which comprised:

• knowledge of natural resource management;
• capability and capacity to deliver core services;
• capability and capacity to deliver project services;
• experience;
• price (value for money); and
• risk.

2.39 The tender evaluation plans required that tender evaluation panels consider clause 19.7 of the conditions of tender. Clause 19.7 relates to minimum standards for Indigenous employment and subcontracting. Of the five procurement processes, only the Southern Queensland Tender evaluation panel documented its consideration of this requirement.

Value for money assessment

2.40 The RLP tender requirements specified that the service provider must be able to deliver both environment and agriculture project services during the term of the services agreement. Throughout the procurement process, only one compliant tender response was received for each management unit, all from the existing NRM organisations.³⁰

2.41 To evaluate value for money in the absence of competing tenders, tender evaluation panels compared pricing for each tender with jurisdictional and national average pricing collated by Synergy Group Australia. This assessment informed the draft tender evaluation report.

2.42 Value for money was further supported through the contract negotiation stage. Contract negotiation plans were prepared for each of the 50 preferred tenderers, based on DAWE’s calculation of reasonable labour costs for the relevant full-time equivalent positions expected to deliver the services. Negotiations were conducted with all preferred tenderers and succeeded in reducing the total cost of core services by $97 million (45 per cent) compared to original tender offers, allowing additional funds for project services.

Have procurement risks been managed appropriately?

2.43 Paragraph 8.2 of the CPRs requires entities to ‘establish processes for the identification, analysis, allocation and treatment of risk when conducting a procurement’.³¹

Procurement risk assessments

2.44 The RLP risk assessment was first approved in August 2017 and focused on risks relating to program design and the procurement process. The risk assessment was addressed by the Joint Executive Committee (JEC)³² in eight of 22 meetings held between July 2017 to January 2019. In January 2019, the JEC directed that:

• the risk assessment be revised to address program implementation risks such as financial viability of service providers; and
• monitoring of risks be made a standing agenda item in JEC meetings.

2.45 A separate risk assessment was also undertaken for each of the five procurements. These were included in the procurement plans, which were approved by the delegate prior to the release of the tenders.

Initial Tender and the Queensland tenders

2.46 The Initial Tender, the Queensland Limited Tender, and the Southern Queensland Tender each identified four procurement risks comprising:

• the tender not resulting in a service provider for the relevant management units;
• an unsuitable provider or providers being contracted for the management units;
• the inability to obtain services to deliver against priority outcomes or areas within management units; and
• the failure to maintain appropriate probity controls.

2.47 The ANAO assessed whether the risk treatments identified to reduce risk ratings across the three procurement risk assessments were appropriately implemented. Of the 48 risk treatments examined:

• 44 (92 per cent) were appropriately implemented;
• two (four per cent) were assessed as partly implemented — relating to creating more deliverables for core services, including plans to measure performance (see paragraph 3.32); and
• two (four per cent) were not implemented — relating to restricting access to tender documentation (see Table 2.3).

2.48 The risks were not reviewed and updated as the treatments were implemented and procurements progressed, as required under the relevant risk management guidelines.

Tasmania tenders

2.49 The two Tasmania tenders each identified 14 risks, with additional risks drawn from the broader program-level risk assessment.

2.50 The risk assessments for the Tasmania tenders contained risk treatment due dates of September 2018 — three months before the first Tasmania tender (for North West NRM region) was released. One of the risks for the Tasmania tenders also contained a reference to a deadline of October 2018, two months before the first Tasmania tender was released. This indicates that not all fields sourced from other risk assessments were updated before being incorporated into the procurement risk assessment template.

2.51 DAWE should ensure that each risk assessment is appropriately adapted to manage the risks relating to relevant procurements.

3.1 Effective management of service provider performance helps to ensure services are delivered in accordance with contracted requirements, which supports achievement of program outcomes.

3.2 To assess whether DAWE³³ effectively manages service provider performance, the ANAO examined whether:

• program requirements are adequately specified in the services agreements;
• effective processes are in place to assess service providers deliverables;
• service provider reporting is reviewed for accuracy; and
• project risks are effectively managed.

Are program requirements adequately specified in the services agreements?

3.3 Clearly identifying program requirements in services agreements allows DAWE to communicate relevant obligations to the service providers and supports effective performance management.

Execution of services agreements

3.4 DAWE has a services agreement with each of the 50 service providers selected to deliver services under the RLP program through the tender process described in Chapter 2 (see Appendix 3).

3.5 The services agreements provide for project work orders to be prepared by service providers if requested by DAWE. Once executed, the project work orders form part of the services agreement. Project work orders specify project-specific requirements, including project service deliverables.³⁴

3.6 The ANAO identified seven key elements that support effective performance management.³⁵ All RLP program services agreements incorporate the seven elements (see Table 3.1).

Table 3.1: Key elements of the services agreements

Source: ANAO analysis.

3.7 All 50 services agreements have been signed, witnessed, and dated by the service providers and the appropriate Australian Government delegate as specified under the Natural Heritage Trust delegation instruments.

Variations to the services agreements

3.8 In late 2019 to early 2020, the services agreements were amended via deeds of variation.³⁶ This was principally to provide an option for project work orders and additional services work orders entered into by DAWE and service providers to form separate contracts, rather than part of the services agreement.³⁷ This arrangement allows DAWE to procure additional projects and services from these service providers for programs other than the RLP program.³⁸

3.9 The variations process was appropriately documented, with:

• an explanatory table outlining the proposed changes and reasons for those changes provided to the service providers; and
• a sign-off of the process by external legal advisers.

3.10 Thirteen out of 18 submissions to the audit commented positively on this variation, stating that it provided more stability by funding additional projects and offset some of the increased costs of tendering (see Appendix 4).

3.11 Two submissions to the audit from non-RLP service provider organisations commented that this approach lacked transparency and competition.

Are effective processes in place to assess service provider deliverables?

3.12 Effective processes to assess service provider deliverables enable DAWE to monitor service providers’ contractual obligations and support achievement of program outcomes.

3.13 Service providers are required to deliver core service deliverables under the services agreement, and project service deliverables under the relevant project work orders. Service providers invoice DAWE for core and project services delivered. All invoices for core services and project services are required to be accompanied by:

• a report submitted in Monitoring, Evaluation, Reporting and Improvement Tool (MERIT) stating that invoiced services have been delivered³⁹; and
• a statutory declaration stating that the service provider has performed all of the services included in the invoice.

3.14 The ANAO examined whether DAWE has effective processes to assess the completion of these deliverables prior to processing payments.

Core service deliverables

3.15 Under the services agreements, service providers are required to deliver the following core services:

• maintain the currency of natural resource management (NRM) planning and the prioritisation of management actions;
• support the community, including landcare, Indigenous communities, and industry to participate in the delivery of projects;
• undertake communications;
• coordinate delivery of projects and monitoring, evaluation, reporting, and improvement activities;
• undertake Regional Agriculture Landcare Facilitator (RALF) services;
• develop project designs and project proposals; and
• maintain a productive, cooperative, and ongoing relationship with DAWE.

3.16 To guide the delivery of these core service requirements, service providers are required to submit core service plans within specified timeframes (see Table 3.2).

Table 3.2: Core service plans required under the services agreements

Note a: Unless agreed otherwise with DAWE.

Note b: Commencement date refers to the date the services agreement commences. Of the 50 agreements in place, 47 commenced in July or August 2018. Three commenced in January, April, and August 2019 respectively.

Note c: These plans were excluded from further assessment in the audit as they were not due to be received until the end of the audit fieldwork.

Source: ANAO analysis of requirements under the services agreement.

3.17 ANAO analysis of submitted core service plans indicate that a majority of service providers have submitted the required documents, with an average of 64 per cent submitted by the submission deadline. Plans that were submitted late were on average 42 days late, with two plans submitted over a year after the submission deadline. One plan is outstanding as at September 2021.

Process for assessing core service plans

3.18 DAWE’s Project Manager’s Handbook (the handbook) outlines the review and approval process for submitted core service plans. The handbook specifies that core service plans must be assessed using a linked template and approved by the director of the relevant project team.

3.19 As at September 2021, of the 487 plans submitted by service providers, DAWE has assessed and approved 408 plans (84 per cent). Table 3.3 outlines the number of days taken to approve the plans from their date of submission.

Table 3.3: Time taken to approve core service plans as at September 2021

Note a: This is calculated from date of submission. DAWE informed the ANAO that delays in approving these plans have been due to competing priorities, such as the 2019–20 bushfires, that required urgent reallocation of resources.

Note b: RALF work plans are required to be submitted annually. The timeframe in the table relates to 2020–21 RALF work plans. Some dates were unavailable as the relevant documents were not dated.

Source: ANAO analysis.

3.20 Service providers are expected to follow and implement the submitted plans pending feedback or approval from DAWE. Risks relating to the implementation of unapproved plans have not been documented, although it was raised by the Department of Agriculture and Water Resources’ (DAWR) audit committee in November 2018 in the context of RALF work plans. As this issue was not raised as a formal recommendation, there was no subsequent follow-up from the audit committee.

3.21 DAWE should implement a process to ensure that assessment and approval of core service plans is completed in a timely manner. Setting an internal target timeframe for assessment and approval can assist in monitoring and managing the process. If delays are expected in this process, risks of such delays should be documented to ensure that any impacts to the delivery of projects and program outcomes can be managed.

Project service deliverables

3.22 Project service deliverables for each project are specified in the project work orders executed under the services agreements. Examples of project service deliverables include number of hectares treated for weeds, number of community engagement workshops, and kilometres of fencing installed.

Process for assessing project service deliverables

3.23 Service providers submit an invoice to DAWE outlining the number of deliverables they have completed in the relevant invoicing period. Clause 27.5 of the services agreement requires that service providers ensure that they have sufficient evidence of service delivery at the time they make a claim for payment.

3.24 DAWE has established different processes for service providers when submitting invoices for completed project service deliverables.⁴⁰ For environment projects, service providers must identify available evidence that demonstrates delivery of services but are not required to submit it unless requested by DAWE — for example, as part of formal assurance activities. For agriculture projects, service providers must upload specific evidence items as detailed in their project work orders in MERIT every time they submit an invoice (see Table 3.4).⁴¹

3.25 Prior to September 2020, DAWE checked whether all specified evidence for invoiced agriculture project deliverables was submitted and aligned with the project work orders, before approving payment. This requirement was incrementally reduced from September 2020 to July 2021, when the delegate approved the removal of the requirement to check for submission of evidence.

3.26 Agriculture and environment projects now follow the same invoice review process, where invoice details are checked against the costs specified in project work orders and number of deliverables reported as delivered in MERIT reports. The Project Manager’s Handbook advises project managers to ‘use their judgement’ when approving invoices for payment and check whether the MERIT report ‘makes sense’, but there is ‘no requirement to interrogate the evidence provided’.⁴²

3.27 DAWE informed the ANAO that this reduction was to align its processes with the new assurance framework that commenced in June 2021. In its brief to the delegate, DAWE explained that:

the proposed approach aims to be more preventative, aiming to manage the sources of the risk to decrease the likelihood of the risk occurring. This would include educating Service Providers prior to an invoicing period, rather than during or after the invoicing period. Assurance during the invoicing period would be managed through [Quarterly Assurance Checks] and Audits, as a detective assurance activity.

3.28 The RLP Assurance Manual commenced in June 2021 and operationalises the assurance framework. The manual states that the project managers’ role goes beyond educational activities and includes reviewing and verifying evidence prior to processing project payments. There is a need to clarify internal processes, including in relevant guidance documents, to ensure that there is a clear alignment between the intended approach and the implemented practice.

3.29 Since July 2021, there is no requirement outside of the assurance process for DAWE to verify any evidence prior to invoice payment. DAWE has accepted the risk that inaccurate, incomplete or poorly implemented deliverables are invoiced and paid for, at the rate identified in their assurance processes. Assurance activities are examined at paragraphs 3.46–3.61.

Is service provider reporting reviewed for accuracy?

3.30 Accurate reporting of progress against project milestones enables DAWE to monitor service provider performance, and measure and report on the achievement of program outcomes. Service providers are required to report regularly on their progress towards delivering core and project services via MERIT. The ANAO examined whether DAWE has effective processes in place to review service provider reporting for accuracy.

Core services reporting requirements

3.31 Under the services agreements, service providers are required to submit the following progress reports for core services in MERIT (see Table 3.5).

Table 3.5: Core services reporting requirements

Source: ANAO analysis of requirements under the services agreement.

Reviewing core services reports

3.32 Core services do not have units of delivery that must be achieved, or specific performance measures to report against. Activities specified in the core service plans, if any⁴³, are not assessed or monitored for completion by DAWE. Core services payments are processed based on statements from the service provider that they have met the core services requirements under the services agreements during the relevant invoicing period.⁴⁴

3.33 There is limited departmental oversight over what core services were delivered, how the core services funding was spent, and whether core services requirements were met prior to processing the core service payments. Elements of core service delivery are checked as part of the contract assurance process (see paragraphs 3.49–3.54), which occurs after the payments have been made and for a selection of service providers.

Reviewing annual core services reports

3.34 At the end of each financial year, service providers are required to submit the annual core services report to outline their delivery of core services for the past 12 months. These reports largely rely on service providers to self-disclose any issues or risks to core service delivery.⁴⁵ No evidence is required to be submitted by the service providers to demonstrate their delivery of core services, and DAWE does not have assurance over the accuracy of information provided.⁴⁶

3.35 Review of the annual core services reports for ‘key achievements’ in the delivery of core services as reported by service providers show a range of responses. Some responses included general organisational activities, such as appointing a board member during an annual general meeting, ‘supporting schools’, and ‘exercising skill, care and diligence’. It is unclear how some of these achievements relate to the delivery of core services or contribute to RLP program outcomes.

3.36 There would be benefit in DAWE developing specific performance measures for service providers to report against to evidence their delivery of core services. This would allow for improved contract management and ensure that value for money is being delivered with core services funding.

Project services reporting requirements

3.37 The services agreement requires service providers to submit the following project service reports in MERIT (see Table 3.6).

Table 3.6: Project services reporting requirements

Note a: DAWE’s outcomes monitoring and reporting is examined in Chapter 4.

Source: ANAO analysis of requirements under the services agreement.

Reviewing project services reports

3.38 Project services reports are submitted with each invoice and outline the project service deliverables completed during the reporting period. On receipt of an invoice, DAWE checks whether the value of units reported in the MERIT project services report matches the amount invoiced prior to approving payment.

3.39 As outlined in paragraphs 3.23–3.29, any evidence submitted by service providers are not verified by project managers prior to processing payments. The process relies on the accuracy of service providers’ reporting and the execution of statutory declarations, as well as any additional checks performed during the quarterly assurance check process by the RLP Contracts Assurance Section (see paragraphs 3.55–3.61).

Reviewing annual project services reports

3.40 At the end of the financial year, service providers submit an annual project services report, outlining their project delivery activities for the past 12 months. DAWE assesses these reports to determine how the service provider is performing under the project work order to achieve RLP program outcomes.

3.41 There is a reliance on the accuracy of service providers’ reporting in this assessment, including for service providers to self-disclose any emerging risks or issues to the delivery of project services and the achievement of project outcomes.

COVID-19 arrangements

3.42 In April 2020, the Minister for the Environment approved a revised payment arrangement to support the ongoing viability of regional service providers during the COVID-19 pandemic. The arrangement enabled service providers to invoice for the remaining balance of contracted funds for 2019–20 in quarter 4, even if they had not provided the full scope of services required under contract. Service providers were instead able to deliver a replacement service of ‘Maintaining organisational capacity’ by:

• retaining service provider personnel, key project partners and subcontractors that are essential to delivery of the services; and
• ensuring that these persons and entities are ready and able to resume delivery of services once the impact of the COVID-19 pandemic has eased.

3.43 Service providers who wished to take up this arrangement were required to give an undertaking that they will deliver on the full scope of services at a future time ‘to the extent reasonably possible, without seeking any further payment for doing so.’⁴⁷ Service providers have until the end of the program (June 2023) to deliver the services impacted by COVID-19.

3.44 If any services that had been delayed are unable to be delivered, the service providers are required to identify the necessary changes to the project plan, project logic, and the service charges specified in the project work order. The handbook states that DAWE project teams should progress a change order to reflect these changes if the proposed changes do not adversely affect the project achieving its intended outcomes. The handbook does not provide guidance on steps to take if the proposed changes are assessed to have an adverse impact on the achievement of project outcomes. DAWE informed the ANAO that the delegate is briefed on each individual case for a change order and makes decisions based on the evidence presented.

3.45 According to DAWE reports, as at the end of December 2020, 44 service providers requested access to the COVID-19 replacement service for 118 RLP projects (out of 50 service providers and 225 projects), totalling $7.9 million.⁴⁸ DAWE advised the ANAO that more than 50 per cent of projects have since delivered the services that had been delayed by COVID-19.

Contract assurance activities

3.46 The RLP assurance framework was first developed in May 2018⁴⁹ and details the assurance approach and planned assurance activities. A summary of the assurance activities and ANAO’s assessment of their implementation are outlined in Appendix 5.

3.47 The RLP Contracts Assurance Section was established by the Department of the Environment and Energy (DoEE) in March 2019 to implement the RLP assurance framework. The RLP Contracts Assurance Section’s primary assurance functions are to undertake audits of service providers and perform quarterly assurance checks over a selection of invoices. Agriculture projects were included in quarterly assurance checks from October 2020.

3.48 All service providers have been examined at least once through an audit or a quarterly assurance check.

Audits

3.49 DAWE implemented an audit program of RLP service providers and projects in late 2019, with the first audit reports delivered in December 2019. As at September 2021, audits of 11 service providers have been undertaken.

3.50 The scope of the audits is informed by the service providers’ obligations under the services agreements and include:

• core service requirements — including evidence that 20 per cent of total funding is directed to small on-ground community activities, and four per cent is directed to Indigenous employment or procurement;
• common requirements — such as appropriate governance and financial procedures; and
• a selection of project services — examining evidence supporting delivery of invoiced project services.

3.51 Audits involve a desktop review of service providers’ documents, as well as a site visit to inspect project sites.⁵⁰ The findings raised in the audit reports are summarised in Table 3.7.

Table 3.7: Summary of audit findings

Note a: Clause 16.9 outlines the terms and conditions that subcontracts must contain.

Note b: As at September 2021, DAWE has not established a process to recover the overpayments.

Note c: Clause 44.3 relates to service provider obligations to notify the department regarding WHS matters, including suspected and actual breaches, incidents and other events.

Source: ANAO analysis of DAWE audit findings.

3.52 For each finding, audit reports may make a direction, a recommendation, or both. Service providers must comply with the directions. Recommendations are suggestions for improvements. Within 15 business days of receiving the final audit report, service providers must provide DAWE with an outline of key actions and a schedule to implement the directions identified in the report.

3.53 DAWE monitors the service providers’ implementation of directions via an audit directions register. The register shows that, across the 11 audits undertaken as at September 2021:

• 23 directions were made, with five of these completed; and
• 31 recommendations were made, with seven completed.

3.54 DAWE’s monitoring of these actions have not been well documented, making it difficult to trace what actions were taken in response to the directions and when these actions were completed. Not all directions or recommendations marked ‘completed’ on the register explain what actions were taken by the service provider to address the relevant finding. Further, although the register tracks the number of days taken to complete the audit process, it does not outline the expected timeframes for service providers to address the relevant directions. It is also unclear when the status of the actions were last updated as any notes made by the departmental officers are not dated.

Quarterly assurance checks

3.55 The purpose of a quarterly assurance check is to provide assurance that selected project services have been delivered as invoiced. It is a desktop exercise where the project managers and officers from the RLP Contracts Assurance Section review documentary evidence of service delivery submitted by service providers.⁵¹

3.56 As at the end of March 2021, quarterly assurance checks have been conducted over 60 projects out of 207 multi-year projects⁵² (see paragraph 4.34), examining 293 invoiced services out of 2999 services contracted under the program.⁵³ Of the 60 quarterly assurance checks, 48 (80 per cent) did not provide sufficient evidence that invoiced services were delivered on initial request. Eight (13 per cent) required three or more follow-ups (Figure 3.1).

Figure 3.1: Quarterly assurance checks – number of follow-up requests

Note: The figure excludes one project for which relevant information was not available.

Source: ANAO analysis.

3.57 As a result of the quarterly assurance checks, 16 of the 60 projects (27 per cent) required revision to their invoice or MERIT report to align with available evidence. Nine (15 per cent) had outstanding evidence issues at the end of the quarterly assurance check.⁵⁴

3.58 Quarterly assurance checks are undertaken once invoices for the quarter are received but before they are processed for payment. A satisfactory completion of the quarterly assurance check is required before invoices are processed for payment. DAWE aims to complete the quarterly assurance checks within 14 days to enable a timely processing of the payment for the quarter.

3.59 Of the 60 quarterly assurance checks undertaken, five were completed within 14 days after receiving the initial invoice. Forty quarterly assurance checks took 31 days or longer, with three taking more than 91 days to finalise (Figure 3.2). On average, a quarterly assurance check took 42.7 days to finalise. The delays were primarily due to the service provider not providing, or being unable to provide, adequate evidence on initial request.

Figure 3.2: Number of days taken to finalise quarterly assurance checks after receipt of invoice

Note: The figure excludes two projects for which relevant information was not available.

Source: ANAO analysis.

3.60 Project teams are responsible for any identified follow-up actions after the quarterly assurance check is completed. It is not always clear whether these follow-up actions are implemented — for instance, records of relationship management meetings do not reflect ongoing discussions of compliance (see paragraphs 3.62–3.63). DAWE should improve documentation on what follow-up actions were taken and when, and ensure these records are captured consistently.

3.61 As outlined in paragraphs 3.23–3.29, DAWE relies on contract assurance activities to verify that there is evidence to support the delivery of services as invoiced. However, DAWE has not addressed the high incidence of incorrect invoicing and insufficient evidence to support completion of project services, with repeated follow-ups required throughout the assurance process. Although most of the identified non-compliance issues stemmed from inadequate record-keeping by the service providers, there have been instances of service providers invoicing twice for the same service, invoicing for services not delivered, and being unable to evidence delivery as invoiced. Reliance on inaccurate reporting of outputs may also affect the measurement of program outcomes.

Relationship management meetings

3.62 Service providers are required to submit a relationship management plan that specifies a schedule of regular contact between the service provider and DAWE on matters of significance. The frequency of formal meetings ranges from monthly to once every six months.

3.63 The Project Manager’s Handbook indicates that relationship management meetings are one of the key mechanisms through which project managers facilitate management of project risks, compliance and contractual requirements. The handbook requires records of formal relationship management meetings to be maintained by the project managers. However, the ANAO found these records to be inconsistent, with notes recorded in varying levels of detail and in different formats for each management unit. Records of meetings were not kept for one service provider, and only partial records are available for 10 service providers.

Are project risks effectively managed?

3.66 Managing project risks supports timely and on-budget delivery of project milestones and achievement of program outcomes.

Service providers’ risk assessments

3.67 Service providers are required to develop risk assessments and WHS plans for each project. These provide an overview of project delivery risks, such as unavailability of subcontractors, working outdoors in a remote area, or extreme weather events. These are reviewed and approved by DAWE.

3.68 Service providers are required to update these as needed. These risk assessments are not used to inform DAWE’s contract management activities.

Risk assessments by the Contracts Assurance Section

3.69 The RLP Contracts Assurance Section undertook a risk assessment of each service provider and each environment project to inform their assurance activities in June 2019.⁵⁵ Key risk criteria considered include: project duration and location; whether the entity is statutory or non-statutory; quality of project design; and any known issues or key dependencies. The assessment was updated in September 2020 and covers 48 service providers and 137 of 159 environment projects.⁵⁶

3.70 In early 2021, the RLP Contracts Assurance Section developed an assessment based on financial risk, with a focus on four services that have a higher risk of non-compliance due to lack of clearly defined deliverables.⁵⁷ This assessment ranks service providers based on the relative cost of an individual project service compared to:

• all other services;
• other services of the same type; and
• unit costs of services that use the same target measure.

3.71 DAWE informed the ANAO that the financial risk assessment was developed to better target their assurance activities, as it now considered that the weighting placed on some of the criteria in the June 2019 risk assessment was too high.

3.72 The financial risk assessment is limited in scope and does not reflect other relevant risk criteria, such as the quality of the service provider’s reporting and engagement with DAWE. The lack of a comprehensive and up-to-date risk assessment limits DAWE’s ability to demonstrate that it is using its resources to target those with the highest risks of non-compliance.

Risks for project management

3.73 DAWE project managers have not undertaken assessment of project risks to inform their project management processes.

3.74 There is limited evidence that — outside of consultation during the assurance process — project managers are considering risks and adapting their project management approaches based on their day-to-day interactions with the service providers.⁵⁸ Internal risk discussions are not consistently documented. As a result, current project management activities, such as level of project oversight, reporting to executives, and engagement with service providers, are not informed by risk.

3.75 In July 2021, a risk management guide for agriculture projects was developed and approved. The guide instructs agriculture project managers to score each project based on the quality of the service provider’s invoicing and reporting practices, as well as their communications with DAWE. These quarterly scores would be added up to calculate an annual risk assessment score to establish if the project poses a high, medium, or low risk.

3.76 The guide does not apply to environment projects. As at September 2021, a risk assessment process for environment projects has not been developed. DAWE has acknowledged this as an area for improvement in the risk management processes.

3.77 A consistent approach to project risk management is important to ensure that program resources, including for assurance activities, are appropriately allocated and project-level risks can be aggregated to inform risks to program delivery.

4.1 At the outset of the program, DAWE⁵⁹ developed an evaluation plan for each of the six program outcomes. The evaluation plans consist of three key elements (see Table 4.1).

Table 4.1: Key elements of the Regional Land Partnerships program evaluation plans

Source: RLP Evaluation Plan, 2018.

4.2 To assess whether DAWE effectively monitors achievement of program outcomes, the ANAO examined whether:

• baseline data for all projects has been established;
• program risks are effectively managed; and
• progress towards achievement of program outcomes is being measured and reported.

Has baseline data for all projects been established?

4.3 Baseline data provides a reference point from which improvement can be measured for individual projects. The RLP program evaluation plan indicates that a baseline measure should be established at project commencement and measured after two or three years to detect changes. All RLP projects have commenced, and all baselines should have now been established.⁶⁰

Identifying baseline data

4.4 The ANAO reviewed 224 RLP project work orders⁶¹ to determine whether all projects had identified a baseline methodology. All projects include a narrative description of how the baseline will be established in the monitoring methodology section. Of the 224 projects:

• 168 (75 per cent) include ‘collecting or synthesising baseline data’ as a project service;
• 45 (20 per cent) include other project services that may be used to establish a baseline but are not identified as baseline collection services (for example, fauna survey or soil testing); and
• 11 (five per cent) do not include project services that relate to establishing a baseline.

4.5 Project services are invoiced and reported in the Monitoring, Evaluation, Reporting, and Improvement Tool (MERIT) when completed. Only projects that invoice the service ‘collecting or synthesising baseline data’ can be readily identified as having a baseline.

4.6 Of the 168 projects that include ‘collecting or synthesising baseline data’ as a project service, 105 indicate the service will be delivered multiple times throughout the life of the project. Given that baseline data is to be established at project commencement, there is a risk that services scheduled in later years of the project are incorrectly identified. On 6 September 2021, DAWE confirmed that details relating to the baseline data contained in MERIT are unclear and that some of the services identified as ‘collecting or synthesising baseline data’ scheduled and invoiced in later years may represent monitoring against the baseline.

4.7 The inconsistent recording of baseline data services in project work orders and subsequent invoicing and reporting means that DAWE cannot accurately identify which projects have established a baseline, or how many baseline data sets have been reported as delivered.

Reporting against baseline data

4.8 Service providers were asked to report on any changes detected against their baseline data in the recently completed Outcomes Report 1. There were 200 reports submitted to 9 September 2021, reporting against 990 short-term outcome statements (see paragraphs 4.32–4.33). Service providers reported that they were able to quantify the level of achievement against a baseline for:

• all short-term outcomes for 128 projects (64 per cent);
• at least one short-term outcome for 192 projects (96 per cent); and
• no short-term outcomes for eight projects (four per cent).

4.9 DAWE has not assessed the risks arising from 72 projects with one or more outcome statements reported as being unable to be measured against a baseline.

4.10 DAWE engaged Hadron Group to assess the effectiveness of the RLP program and contribute to the mid-term review. As part of this engagement, raw datasets for 35 outcomes statements across 11 projects were assessed for consistency with service provider reports. The Hadron Group’s October 2021 report indicates that 86 per cent of self-reported short-term outcomes statement achievements were based on evidence. Inconsistencies found in 14 per cent were often the result of insufficient data or subjective reporting.

Are program risks effectively managed?

4.13 Effective management of program risks requires collaboration between areas with responsibility for project management, program management and assurance. It enables early identification, prioritisation and treatment of risks and supports delivery of program outcomes.

Oversight of program risks

4.14 The Joint Executive Committee (JEC), with representatives from the Department of the Environment and Energy (DoEE) and the Department of Agriculture and Water Resources (DAWR), initially oversaw the implementation of the RLP program, including managing program risks. Risk became a standing JEC meeting agenda item in May 2019.

4.15 Following the Machinery of Government (MoG) changes in February 2020, the JEC was superseded by the RLP Management Committee. The RLP Management Committee met in March, April, and June 2020. The minutes for each meeting reflect discussion on the risk assessment and document any changes made.

4.16 In July 2020, the RLP Management Committee was disbanded. Ongoing management of the program, including management of risks, was consolidated within the Program Delivery branch in the Biodiversity Conservation Division of DAWE.

4.17 The RLP Implementation Working Group, comprising representatives from the relevant areas of the Program Delivery branch, was established in July 2020 to manage operational matters relating to program implementation, including risk.

4.18 Risks are reported to the First Assistant Secretary of the Biodiversity Conservation Division on a quarterly basis as part of the RLP Quarterly Executive Report.

Risk management process

4.19 There is no documented RLP risk management framework or procedure to guide the program’s risk management process.

4.20 The first RLP program risk assessment was developed by DoEE in consultation with DAWR in August 2017, using guidelines and procedures detailed in DoEE’s risk management framework. It initially focussed on risks relating to program design and procurement. In January 2019, following the release of the final tender, it was revised to focus on program implementation risks.

4.21 The RLP Implementation Working Group is responsible for managing the RLP program risk assessment. DAWE informed the ANAO that the RLP Implementation Working Group does not maintain formal minutes and actions are recorded in the RLP issues and decisions register. The RLP issues and decisions register does not record any decisions relating to the program risk assessment.

4.22 The RLP program risk assessment is a spreadsheet which includes a date of risk analysis field that has not been consistently updated. For example, the risk assessment attached to the RLP Management Committee Meeting of 17 June 2020 shows the date of risk analysis as October 2019.

4.23 This makes it difficult to determine when risks were last reviewed or updated. It is also difficult for DAWE to demonstrate that risks are being actively managed in accordance with organisational risk appetite and risk management policies and guidelines.

Changes to risk ratings

4.24 Risk ratings for the identified RLP program risks remained unchanged between January 2019 and January 2021, when three risks ratings were updated (see Table 4.2).

Table 4.2: Changes to risk ratings in January 2021

Source: ANAO analysis of RLP risk assessments.

4.25 The Quarterly Executive Report for October to December 2020 provides a summary of changes made to the risk assessment, including completed risk treatments, updated risk controls, and new sources of risks. However, the summary does not mention that changes were made to overall risk ratings for risk numbers 1, 7 and 13 in the risk assessment attached to the report.⁶² There is no documented rationale for making the changes to these three risks.

4.26 Changes to overall risk ratings should be supported by a record of consultations undertaken and a documented rationale. This should include the reasons for the change and decisions on whether any action should be taken.

4.27 Paragraphs 3.73 to 3.77 also raise the issue of a lack of consultation and documentation of decisions on project risk management. A consistent, well-documented approach to risk management that is informed by consultation with relevant personnel would improve DAWE’s management of risks to RLP program delivery.

Is progress towards achievement of program outcomes being measured and reported?

Monitoring program outcomes

4.30 There are three levels of outcomes that contribute to the RLP program (refer to Figure 2.3).

• Short-term outcomes are delivered over one to three years. All projects have short-term outcomes.
• Medium-term outcomes are delivered over three to five years. Projects of three to five years duration have medium-term outcomes.
• Long-term outcomes are expected to be achieved in 10 to 20 years, after completion of the RLP program.

4.31 DAWE relies on a range of reporting by service providers to monitor progress towards achievement of project-level outcomes (see Tables 3.5 and 3.6). These reports are intended to inform DAWE’s reporting on progress towards achieving program outcomes.

Short-term outcome reporting

4.32 Service providers specify in their project plans one or more short-term outcome statement(s) that indicate what outcomes the project is expecting to achieve by delivering on project services — for example, by June 2021, the habitat of a threatened species will be improved through the removal of weeds across 500 hectares.

4.33 Service providers report against the achievement of short-term outcome statements in the Outcomes Report 1 (see Table 3.6).

4.34 As outlined in paragraph 2.15, through the project allocation process, DAWE selected short-term projects to commence in year one of the program (one-year projects), and long-term projects for years two to five of the program (multi-year projects). Different outcomes reporting arrangements were put in place for one-year and multi-year projects.

Short-term outcomes reporting for one-year projects

4.35 There are 17 one-year projects that were completed in 2019. All have submitted the Outcomes Report 1. These projects reported against 64 short-term outcome statements in the project work orders as being fully achieved, partially achieved or not achieved (see Table 4.3).

Table 4.3: Outcomes Report 1 results for one-year projects

Note a: Outcome could not be measured in the project timeframe.

Source: ANAO analysis of Outcomes Report 1.

Short-term outcomes reporting for multi-year projects

4.36 In 2020, a new reporting arrangement was established for the remaining 207 multi-year projects with a completion date of 2020 or later. These were due to be submitted by service providers by 29 July 2021. The report asks service providers whether the level of achievement for each short-term outcome statement in the project work order can be quantified against four evaluation criteria (see Table 4.4).

Table 4.4: Regional Land Partnership program evaluation criteria

Source: RLP Evaluation Plan, 2018.

4.37 As at 9 September 2021, 200 projects, out of 207 multi-year projects, have submitted their Outcomes Report 1. The 200 projects reported against 990 short-term outcome statements. Of these, service providers indicated that they could quantify their level of achievement against 848 (86 per cent) outcomes statements (Figure 4.1).

Figure 4.1: Outcomes Report 1 results for multi-year projects — as at 9 September 2021

Source: ANAO analysis of Outcomes Reports 1.

Medium-term outcome monitoring

4.38 Medium-term outcome results will be submitted by service providers for each project at the end of the program in 2023 through the Outcomes Report 2 (see Table 3.6). This data will form the basis of DAWE’s end-of-program evaluation.

4.39 Outcomes Report 1 for multi-year projects asked service providers to determine the extent to which they have progressed towards the medium-term project outcome. Of the 200 submitted reports:

• 99 (50 per cent) indicated that they have had significant progression;
• 83 (42 per cent) had satisfactory progression;
• 17 (nine per cent) had partial progression; and
• one (one per cent) stated that they have had no progression towards the medium-term project outcome.

Long-term outcome monitoring

4.40 In March 2019, DAWE engaged Griffith University to help develop a framework for long-term monitoring for the RLP program. In July 2019, DAWE established a Long-term Monitoring Program (LTMP) Steering Group, made up of executive level members from across the department, to guide and provide advice on the development of the framework and other aspects of the LTMP.

4.41 The long-term monitoring framework was delivered in February 2020 and endorsed by the LTMP Steering Group.⁶³ The proposed framework ‘aims to enhance ecological monitoring and evaluation at a RLP project level as well as promoting a culture of robust long-term ecological monitoring and evaluation for Natural Resource Management in Australia more broadly’.

4.42 DAWE informed the ANAO that the framework has been used to underpin a series of projects to help evaluate the current RLP program, including the outcomes reporting process, and to build tools and data collection protocols to support future Australian Government Natural Resource Management programs. These include:

• working with the Terrestrial Ecosystem Research Network to develop monitoring standards and ensure collection of consistent biodiversity data⁶⁴;
• developing techniques and tools to track the trajectory of feral animals;
• establishing best practice monitoring methods for species included in the Threatened Species Strategy;
• supporting collaboration between RLP service providers and other relevant conservation management groups to develop consistent data collection and reporting; and
• supporting RLP service providers to engage with the Commonwealth Scientific and Industrial Research Organisation (CSIRO) to ensure consistent monitoring of bushfire recovery.

Evaluation and reporting

4.43 DAWE’s reporting obligations are specified in the RLP Evaluation Plan.

Table 4.5: DAWE reporting requirements

Source: RLP Evaluation Plan, 2018.

Annual progress report

4.44 DAWE has not produced an annual progress report. A draft RLP Achievements Snapshot report has been developed focussing on outputs delivered in the first two years of the program.

4.45 DAWE informed the ANAO that the snapshot report will be updated to cover years one to three of the program and released publicly following completion of the RLP mid-term evaluation report.

Mid-term evaluation

4.46 The mid-term evaluation will draw on the service providers’ Outcomes Report 1, which were due to be submitted for all multi-year projects by 29 July 2021. An internal report is expected to be finalised in November 2021. A summary of the report will be included in the public snapshot report and high-level recommendations will be included in the National Landcare Program Phase Two Review.

4.47 Two submissions to the audit from the public noted the lack of public reporting provided by DAWE and advocated for more transparent and accessible reporting of program performance.

Appendix 1 Department of Agriculture, Water and the Environment response

 

 

 

 

Appendix 2 Improvements observed by the Australian National Audit Office

1. The existence of independent external audit, and the accompanying potential for scrutiny can promote improved performance. Improvements in administrative and management practices can occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021–22 Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions, and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• The ANAO identified a project involving the translocation of an endangered species that did not consider a section of a Commonwealth recovery plan cautioning against undertaking translocations of that species. DAWE undertook a review into the project to satisfy itself that the project will not breach the recovery plan.
• Tender submissions are required to be treated as confidential before and after a contract is awarded. During the course of the audit, DAWE was made aware that the tender submissions stored on their records management system allowed open access to departmental staff. DAWE has since restricted the relevant folders to specified personnel.
• DAWE is developing a report that compares the number of planned services against the number of services delivered according to data in MERIT. When reviewing the figures provided by DAWE, the ANAO noted that there are some outliers in the project services data — for example, where number of units delivered far exceeded the total number to be delivered. DAWE undertook further investigations into those project services to determine reasons for the discrepancies. As a result of the investigation, DAWE informed the ANAO that it had implemented processes to correct the overestimates and ensure data integrity going forward.

Appendix 3 List of Regional Land Partnerships service providers

Note a: The NSW Local Land Services is one legal entity with multiple regional offices.

Source: DAWE.

Appendix 4 Summary of public submissions

Source: ANAO summary of public submissions.

Appendix 5 Regional Land Partnerships assurance activities

Key: ◆ Appropriately implemented; ▲ Partly implemented; ■ Not implemented

Source: ANAO analysis of activities outlined in the RLP assurance framework.