Introduction

Australia's fisheries are managed either by the Commonwealth, by the States/Northern Territory or through a joint authority involving two or more of these parties. Generally, Commonwealth law applies from the three nautical mile limit of State/Territory coastal waters to the limit of the Australian Fishing Zone.¹ The right to access and catch fish for commercial purposes in Commonwealth-managed fisheries is controlled by the Australian Fisheries Management Authority (AFMA) through fishing concessions, such as statutory fishing rights and fishing permits.

The Securing our Fishing Future package

On 23 November 2005, the former Minister for Fisheries, Forestry and Conservation (the then Minister) announced the Securing our Fishing Future package. The package was intended to address factors affecting the sustainability of the Commonwealth's fish stocks and the profitability of the fishing industry. These factors included overfishing², excess fishing capacity and economic and market pressures. Its primary component was a structural adjustment package. The Minister also advised that the other element of the package would be new management measures to halt overfishing and give overfished stocks a chance to recover.³

The $220 million one-off structural adjustment package, which was delivered by the Department of Agriculture, Fisheries and Forestry (DAFF), was to:

• buy back up to $149 million in fishing concessions within and across Commonwealth-managed fisheries;
• provide $50 million in grant assistance to individuals, businesses and companies that had been affected by the impacts of the buyback and/or by the new management measures; and
• provide $21 million to AFMA (over three years) to offset the expected reduction in levies received from fishing concession holders ($15 million) and improve its science, compliance and data collection ($6 million).

The buyback program

The objectives of the Fishing Concession Buyback (the ‘buyback' program also known as the Business Exit Assistance program) were to:

• facilitate business exit from the Australian fishing industry;
• remove Commonwealth fishing concessions, especially from targeted fisheries, within a capped budget;
• remove significant fishing capacity within and across Commonwealth fisheries to contribute towards the recovery of overfished fisheries and to assist their management on a more ecologically sustainable basis; and
• remove certain fishing concessions and related fishing effort displaced by a proposed marine protected area in the south-east marine region.⁴

The buyback offered the holders of eligible concessions the opportunity to rationalise their business or to exit the industry. The voluntary tender process primarily targeted concessions in the following four fisheries where stocks were overfished or at risk of being overfished:

• Bass Strait Central Zone Scallop Fishery (BSCZSF);
• Eastern Tuna and Billfish Fishery (ETBF);
• Northern Prawn Fishery (NPF); and
• Southern and Eastern Scalefish and Shark Fishery (SESSF).

The RFT process

The first Request for Tender (RFT) was released on 30 March 2006 and was sent to holders of eligible fishing concessions. One or more tenders could be submitted in each round and tenders could be linked to other concession holders' tenders if, for example, fishers were in partnership together. Each tender was to indicate the concession(s) offered, the total price asked and, in round one, the price asked for each concession. Tenders for round one closed on 22 June 2006 and 551 tenders were received. A second RFT was released on 19 October 2006 with tenders closing on 23 November 2006. DAFF received 324 tenders in the second round.

Assistance to complete the RFT was available from DAFF's helpdesk and website and from the RFT contact officer. DAFF also held public information sessions and tenderers could seek funding for professional advice (such as accounting and legal) through the Business Advice Assistance (BAA) program.⁵

Evaluating and selecting tenders for purchase

DAFF established a dedicated Fisheries Adjustment Taskforce in its Fisheries and Forestry Division to develop and administer the buyback program. It also provided secretariat support to the evaluation panel that was appointed to oversee the evaluation process and recommend successful tenders to the program's delegate.

Tenders were screened by DAFF for compliance with the RFT's conditions and for errors, and eligible tenders were then evaluated against the evaluation methodology. ABARE translated the evaluation methodology into a series of mathematical programs (the algorithm) that could be solved using the Generalised Algebraic Modelling System.⁶ By using an algorithm, DAFF did not need to set maximum prices for each type of concession purchased. Using the total prices asked and the concessions offered by tenderers, the algorithm analysed the tenders to identify the maximum number of concessions that could be purchased across the fisheries with the funds available (the ‘optimal' bundle). Through this process, DAFF sought to achieve value for money for the program.

To find the optimal bundle, the algorithm assigned an ‘effective' price to each concession. If the tender contained one concession then the effective price would equal the total price asked. However, if multiple concessions were offered, the algorithm could assign multiple combinations of effective prices that equalled the total price asked.

Each tenderer received the total tender price asked but the effective prices ‘paid' by the algorithm for each concession in a tender could be very different to the individual prices asked.⁷

Tenders selected by the algorithm were recommended by the evaluation panel and approved by the decision maker (a senior DAFF officer). The then Minister announced the results of the: round one evaluation on 15 September 2006; and round two evaluation on 22 December 2006. In total, DAFF purchased 292 tenders for a total cost of $148.7 million. The prices paid per tender ranged from $6000 to $9.6 million. More than 550 fishing concessions (approximately one-third of the total number of concessions available) were purchased. The targeted number of concessions in the SESSF, ETBF and NPF were exceeded but the target set for the BSCZSF was not met.⁸ Only 22 of the 122 BSCZSF packages (18 per cent) sought were purchased.

Independent assessment and audit of the buyback

In June 2008, the Minister for Agriculture, Fisheries and Forestry announced that Ernst & Young would undertake an audit of the buyback program. This audit honoured an election commitment made prior to the 2007 federal government election to review whether unsuccessful BSCZSF fishers ‘were rejected on the basis of good public policy and value for money considerations'.⁹ Ernst & Young's audit examined DAFF's administration of the tender evaluation processes for the buyback of BSCZSF fishing concessions to identify if the tenders were evaluated appropriately and consistently in accordance with the RFT's evaluation criteria.¹⁰ Ernst & Young did not identify any inconsistencies with the rules set out in the evaluation plan and the methodology or in the application of the algorithm.

Audit objectives and scope

The objective of the ANAO's audit was to examine the effectiveness of DAFF's implementation and administration of the buyback of fishing concessions under the Securing our Fishing Future structural adjustment package.

In undertaking this audit of the buyback program, particular emphasis was given to the following elements:

• participating in the buyback;
• evaluating tenders; and
• finalising the buyback program.

The $21 million provided to AFMA under the structural adjustment package (refer to paragraph three) was not examined in this audit. The ANAO has audited the department's administration of the industry and community assistance grant programs in Audit Report No.39 2008–09, Administration of the Securing our Fishing Future Structural Adjustment Package Assistance Programs, which has been tabled in conjunction with this report.

Overall conclusion

The Securing our Fishing Future package sought to improve the sustainability of the Commonwealth's fish stocks and the profitability of the fishing industry by reducing fishing capacity, particularly in the target fisheries. The voluntary buyback program offered eligible concession holders the opportunity to exit the industry or to rationalise their businesses. The longer term benefits of the buyback are to be supported by the improved fishery management measures being introduced by AFMA.

AFF successfully completed the buyback of fishing concessions and exceeded the reduction targets established for three of the four target fisheries. The target was not achieved in the BSCZSF primarily because the prices asked for the concessions being offered in this fishery were considerably more than the department was prepared to pay.

To evaluate the 875 tenders received, DAFF translated the evaluation methodology into a series of mathematical programs (the algorithm). Given the complexity of the evaluation process and the large number of tenders involved, this was an innovative and practical approach. The algorithm selected the maximum number of concessions that could be purchased within the available budget, which DAFF had defined as the benchmark for achieving value for money.

In circumstances where an algorithm is used as the basis for decisions in tender selection processes, it is important that effective assurance processes are put in place to provide confidence that the results generated are accurate and soundly based. For the buyback program, there were no formal quality assurance processes in place during the development and running of the algorithm to provide assurance that it was operating correctly. This was particularly important as manual steps were incorporated into the process.

The algorithm was applied across all fisheries and no inconsistencies were identified by Ernst & Young and the ANAO. However, the evaluation process was not transparent as the selections made could not be compared or interrogated easily by the evaluation panel, the decision maker or the tenderers. There was also a lack of documentation to support key elements of the evaluation process. In particular, the testing of the algorithm and the briefings provided to the evaluation panel and decision maker on the evaluation results. Minutes were also not kept for the meeting where the panel reviewed and discussed the evaluation results in round one.

The decision maker was briefed on the process followed, the evaluation methodology and the results generated by the algorithm, a list of tender numbers for purchase and the total price asked for each tender. Despite assurances being given by ABARE that the results were correct, errors resulted when the algorithm's settings were adjusted to accommodate time pressures in round one. The tenders were re evaluated after DAFF had finalised both rounds and three additional tenders were selected.

The buyback's communication strategy did not effectively manage the lack of transparency surrounding the process for stakeholders. In the absence of any information to the contrary, fishers assumed that the prices asked for individual concessions in successful tenders were paid. In reality, because of the way the algorithm evaluated the tenders, this was not necessarily the case even though the total price asked was always paid. Providing more detailed information about how the algorithm would evaluate and select successful tenders would have gone some way to addressing stakeholders' expectations.

Should this approach be adopted for any future evaluations, the process would be strengthened and greater assurance given for the outcomes if key risks were better managed. The key risks for the buyback program were the lack of quality assurance processes, the transparency of the evaluation process and managing stakeholders' expectations. Although DAFF advised that it had assessed the risks associated with using the algorithm, the assessment and mitigation strategies were not documented. The ANAO has made one recommendation that incorporates the lessons learnt from the buyback.

It is too early to determine the extent to which the buyback program has impacted the sustainability of fish stocks and the profitability of the fishing industry. The full effect of the buyback program will also be influenced by the effectiveness of the management measures being instituted by AFMA. DAFF has engaged ABARE to assess the impact of the buyback program. It is currently examining survey data and its first report will be released in July 2009.

Key findings by Chapter

Participating in the buyback (Chapter Two)

DAFF developed and implemented in both rounds a multi faceted communication strategy for the structural adjustment package. The strategy outlined the intended audience for the buyback, the key messages to be promulgated and the channels that would be used to distribute information. The RFT and its addenda provided potential tenderers with clear instructions and a reasonable basis for understanding the legal responsibilities of submitting a tender. DAFF provided many avenues to obtain advice and assistance.

For some potential tenderers, their key questions, which centred on the pricing of concessions and how the tenders would be evaluated, remained unanswered. Potential tenderers were not advised that an algorithm would be used to evaluate tenders and DAFF did not release round one price information publicly prior to round two, although it had advised in the RFT that it may.

DAFF's communication strategy would have been more effective if it had explained to tenderers that an algorithm would be used to evaluate tenders and provided the reasons for this approach. There would also have been merit in offering some insight as to how the algorithm would evaluate the tenders based on the total tender price asked, fishery budgets and reduction targets and not the prices asked for individual concessions. In its audit of the buyback program, Ernst & Young also found that clearer guidance could have been provided to fishers regarding how the department was going to assess value for money.

Evaluating tenders (Chapter Three)

Evaluation plan

An evaluation plan was developed for each round, which documented the evaluation process and methodology. Each plan was comprehensive and supported the systematic evaluation of tenders. The methodology documented the number of fishing concessions that DAFF sought to purchase in each target fishery and the budgets available to achieve each target. Although the overall approved budget for the program ($149.5 million) did not change, DAFF revised the budget allocations.

DAFF advised that revisions to the budget were made in consultation with the then Minister. However, the department was unable to provide documentation to support the decisions to give additional funds to the BSCZSF or to redistribute the MPA allocation. Further, the calculations underpinning the initial budget and the quantum of funds provided to the non-target fisheries and the BSCZSF in the revised budget could also not be provided. The final budget was signed off as part of the evaluation plan.

Indicative prices set for round two

DAFF incorporated ‘indicative' prices—the maximum price that DAFF was prepared to pay for each type of concession—into the evaluation methodology for round two. With the exception of the ‘indicative' price set for BSCZSF packages, indicative prices were based on information collected from successful tenders in round one. DAFF was unable to provide documentation to support the rationale for setting the indicative price for the BSCZSF packages at $110 000. The then Minister agreed with this price and was aware that potential tenderers would not be advised of the indicative price.

Developing the algorithm

Developing the algorithm was an iterative process, which involved: ABARE analysing synthetic data generated using random prices to help understand the operation of the evaluation methodology; refining data inputs; and running the algorithm with test data. DAFF was unable to provide documentation to support the tests undertaken, revisions made as a result of this testing or discussions between the panel and ABARE during the development of the algorithm. DAFF also advised that it assessed the risks associated with using the algorithm. However, it did not document this assessment or any strategies to mitigate the identified risks.

Ernst & Young examined the extent to which the algorithm complied with the evaluation procedures stated in the evaluation plan and methodology. Scenarios were used to test particular aspects of the specifications contained within the RFT and the evaluation methodology. No inconsistencies were identified in the application of the algorithm or with the rules set out in the evaluation plan and methodology. The ANAO's review of the algorithm did not find evidence to the contrary.

The 10 per cent method was also reviewed as its application required an ABARE officer to manually intervene in the process by adjusting the code to instruct the algorithm what to do next. Although the adjustments reflected the evaluation methodology, testing undertaken as part of the audit identified that there was limited documentation available to support these manual steps. When the manual steps were replicated, no inconsistencies were found. However, this does not necessarily mean that the manual processes carried out in the actual evaluations were performed similarly.

Although the need to test the algorithm had been discussed, there were no formal quality review processes in place even though introducing manual processes as part of the 10 per cent method increased the potential for error. There would have been merit in the evaluation panel and/or the decision maker signing off the algorithm to confirm that adequate testing had been undertaken and there was confidence that the algorithm would perform the evaluation as expected.

Process for evaluating tenders

DAFF effectively screened the tenders received and implemented processes to manage the extensive errors in the tenders. DAFF advised that, of the 875 tender received across two rounds, in excess of 400 tenders contained errors or required clarifications. The high error rate in round one extended the evaluation timeframe by six weeks.

Consideration of the results by the panel

Contrary to the probity arrangements put in place for the tender evaluation, no minutes were kept for the meeting (held over two days) where the panel received and discussed the evaluation results from round one. DAFF advised that the outcomes of this meeting were captured in the round one evaluation report. The members of the evaluation panel advised that, although not part of the formal process, they undertook ‘spot checks' to confirm that the results of single tenders were ‘as expected' and for the multiple fishery tenders, the results ‘looked reasonable'. The chair of the evaluation panel advised that queries were raised and satisfactory answers provided by the DAFF contact officer. However, there was no documentation to support these checks and the evaluation report does not record any checking processes or the queries that were raised.

Data integrity checking
Running the algorithm was time consuming. Further, managing the errors identified in the round one screening process reduced the time available to run the algorithm. To help the algorithm to run more quickly, ABARE adjusted the algorithm's settings. The adjustment of the settings and the potential risk this created was not documented and the panel and the decision maker were not advised during the evaluation process. A data integrity check in January 2007 identified errors in the results of the round one evaluation, which were caused by the adjustments ABARE had made to the algorithm. Round one tenders were re-evaluated with the algorithm set correctly. This re evaluation identified three tenders that should have been selected and were not, and one tender that had been purchased and should not have been. DAFF subsequently purchased the concessions offered in the three additional tenders.

Was value for money achieved?

For the buyback, DAFF defined value for money as the maximum number of concessions that could be purchased with the available funding. Within the parameters set by the evaluation plan, the department purchased the concessions identified as offering the best value for money by the algorithm. However, when the round one tenders were re-evaluated, DAFF identified one tender ($50 000) that had been purchased and was not value for money.

It was also uncertain whether value for money was achieved in round one when additional concessions were purchased in fisheries where reduction targets had already been met. Particularly, as the price paid for these concessions was up to 10 per cent more than the highest price previously paid (the 10 per cent method). For example, almost $14 million was spent on purchasing additional concessions in the ETBF. There was no analysis of whether purchasing these concessions offered better value than purchasing concessions in the other target fisheries or in round two.

Finalising the buyback program (Chapter Four)

DAFF has publicly reported the number of fishing concessions that were removed through the buyback program. DAFF has not reported publicly on the amounts (average prices/ range of prices) paid to tenderers or the extent to which the buyback's policy objectives have been achieved.

It is too early to determine the extent to which the buyback program has impacted the sustainability of fish stocks and the profitability of the fishing industry. The full impact of the buyback program will also be influenced by the effectiveness of the management measures being instituted by AFMA. In 2006–07, DAFF engaged ABARE to collect benchmark data for a later assessment of the full impact of the structural adjustment package. ABARE is currently examining survey data collected for three of the four target fisheries. It was not possible to examine the other target fishery, the BSCZSF, in the same way as it has been closed to all fishing since 2006.¹¹ ABARE is expected to report on the impact of the buyback program in July 2009. This work will also provide a basis for measuring any future impact of the buyback on the successful fisheries.

Summary of DAFF's response to the audit

The ANAO provided a copy of the proposed report to DAFF. In addition, the ANAO provided an extract of the relevant sections of the proposed report to all members of the evaluation panel, for comment.

A response was received from DAFF. A copy of DAFF's full response is included in Appendix One. DAFF also provided the following summary comments:

The department accepts the ANAO's conclusion that the buyback was successfully completed and that appropriate advice and assistance were provided to applicants. DAFF also notes the ANAO's conclusions that the evaluation of tenders was comprehensive, with the use of an algorithm in the evaluation being an innovative and practical approach. DAFF acknowledges that there were limitations in the documentation surrounding the development and implementation of the evaluation process and agrees with the ANAO's recommendation regarding enhancing the transparency and accountability of any future evaluation processes.

Footnotes

¹ The Australian Fishing Zone (AFZ) extends to 200 nautical miles from the Australian coastline and includes the waters surrounding Australia's external territories, such as Christmas Island in the Indian Ocean and Heard and McDonald Islands in the Antarctic.

² Overfishing refers to the amount of fishing and where a fish stock is experiencing too much fishing. As defined in: Bureau of Rural Sciences, Larcombe, J. and McLoughlin, K. (eds), Fishery Status Reports 2006: Status of Fish Stocks Managed by the Australian Government, Canberra, 2007, p. 2.

³ On 14 December 2005, AFMA was formally directed by the then Minister (under s.91 of the Fisheries Administration Act 1991(Cth)) to implement these measures.

⁴ In May 2006, the then Department of the Environment and Heritage (DEH), which is currently known as the Department of the Environment, Water, Heritage and the Arts (DEWHA), announced 13 Marine Protected Areas (MPAs) in the south-east marine region which stretches from the far south coast of New South Wales, around Tasmania and Victoria and west to Kangaroo Island (South Australia).

⁵ The Business Advice Assistance (BAA) program was administered by DAFF as part of the structural adjustment package. A grant of up to $1500 was available to each applicant.

⁶ The Generalised Algebraic Modelling System is an off-the-shelf software package designed for modelling linear, nonlinear and mixed integer optimisation problems and particularly suited to large, complex problems. A summary of how the algorithm worked is provided in Appendix Two.

⁷ Within the context of the buyback program, the price ‘paid' is the value allocated by the algorithm to each concession offered within a tender. Appendix Three illustrates how the algorithm distributed the total prices asked across the concessions offered in three successful tenders.

⁸ The panel sought the following reductions in the total number of concessions in each fishery: a 30 per cent reduction in the ETBF; a 25 per cent reduction in the NPF; an approximately 50 per cent reduction in the SESSF and an 80 per cent reduction in the BSCZSF.

⁹ The Honourable Tony Burke MP, Rudd Govt. to hold early review of Bass Strait scallop licence buyout, DAFF08/071B, 2 June 2008.

¹⁰ Available from, <http://www.daff.gov.au/fisheries/domestic/independent_audit_of_the_voluntary_ surrender_of_fishing _concessions> [accessed 24 October 2008].

¹¹ The former Minister for Fisheries, Forestry and Conservation formally closed the Bass Strait Central Zone Scallop Fishery (BSCZSF) from 2006 for a minimum of three years.