Introduction

1. Performance audits conducted by the Australian National Audit Office (ANAO)—often referred to as ‘external’ audits—involve the independent and objective assessment of the administration of Australian Government agency programs, policies, projects or activities. Audits initiated by agencies using resources under their control—known as ‘internal’ audits—fulfil a complementary role, providing assurance to agency management on the effectiveness of the internal control environment and identifying opportunities for performance improvement.¹

2. Whether part of an external or internal program of review, audits contribute to better public administration by identifying opportunities for improved agency management and administration, and may include recommendations. Audit recommendations highlight actions arising from an audit report that are expected to improve agency performance when implemented, and generally address risks to an agency’s delivery of its outcomes.

3. A key focus of audit activity is the consideration of the efficiency and effectiveness of agency administration, and consistency with legislative and policy settings—providing an independent assessment of the ‘proper use’ of public resources anticipated by the Australian Government’s financial management framework.² ANAO performance audits and an agency’s internal audits can assist agency Chief Executives to deliver on government program objectives and fulfil their ‘special responsibilities’ under the financial management framework, to manage agency affairs in a way that promotes the proper use of public resources.³

4. A further ‘special responsibility’ of Chief Executives under the financial framework is the requirement to establish and maintain an audit committee. Audit committees have an important role in the internal governance framework of agencies subject to the Financial Management and Accountability Act 1997 (FMA Act)⁴, and the Financial Management and Accountability Regulations 1997 (FMA Regulations) specify a range of functions for such committees, several of which relate specifically to internal and external audit.⁵ Accordingly, while audit committees do not undertake management responsibilities and are not a substitute for management controls and accountabilities, they have an important role in assisting the Chief Executive to ensure that the agency derives the anticipated benefits from internal audit activity and responds appropriately to the findings and recommendations of external audits.

5. The appropriate and timely implementation of recommendations that are agreed by an agency is an important part of realising the full benefits of an audit. A better practice approach to the implementation and oversight of audit recommendations exhibits: a clear process and timelines; the clear allocation of roles and responsibilities at each stage in the process; and appropriate regard by agency leaders and managers to the key principles for effective implementation. The key elements of a better practice approach are outlined in Table S1.

Table S1 Elements of better practice in implementing audit recommendations

Notes:  (A) The roles and responsibilities of stakeholders may vary depending on the stage in the implementation process.
            (B) The key principles are drawn from the Australian National Audit Office and the Department of the Prime Minister and Cabinet Better Practice Guide—Implementation of Programme and Policy Initiatives: Making implementation matter, October 2006, Canberra.

6. A better practice approach recognises that the general principles applying to the implementation of program and policy initiatives—such as planning, stakeholder management, monitoring and review—have direct relevance to the implementation of audit recommendations.⁶ More specifically, an effective agency system for implementing audit recommendations will exhibit:

• A clear process for assigning responsibility for implementing a recommendation to the appropriate line area within an agency, including identifying a responsible senior official, and establishing an agreed timeframe within which action should be completed (agreed between the actioning area and internal audit area, with input from the audit committee if considered appropriate). The responsible official should provide regular updates to an agency’s internal audit function on progress in implementing the recommendation.
• An active role for internal audit in maintaining an appropriate process for monitoring the status of implementing agreed audit recommendations, and reporting this status to the agency’s audit committee. Where internal audit is not satisfied with progress in implementing a recommendation, the matter should be escalated to senior line management, and if not effectively resolved, reported to the audit committee.
  • Internal audit may also have a role in reviewing the actions taken by the agency in response to a recommendation, commensurate with the level of risk posed to the agency by the issue the recommendation addresses.
• An audit committee that, on behalf of the agency’s Chief Executive, monitors management’s implementation of audit recommendations. The committee should also keep the agency Chief Executive informed on progress in implementing recommendations that are difficult to implement or are overdue.⁷

7. The Department of Defence has a well established internal audit function (Audit and Fraud Control Division) that is led by the Chief Audit Executive. Among other responsibilities, the division conducts internal audits and monitors the implementation of audit recommendations from both ANAO performance audits and internal audits. The division operates a dedicated information system, the Audit Recommendations Management System (ARMS), which provides a repository to track progress and record action taken in response to audit recommendations.

8. The department’s audit committee—the Defence Audit and Risk Committee (DARC)—has multiple responsibilities, amongst which are monitoring the implementation of audit recommendations and advising the Secretary of Defence and Chief of the Defence Force (CDF) on significant issues identified in audits.

9. The Defence Materiel Organisation (DMO) also has an audit committee (a financial framework requirement, because DMO is an FMA Act agency)⁸ and an internal audit function. While there are some parallel processes across the department and DMO, the department’s audit function has final authority for closing DMO-related recommendations arising in ANAO and departmental audit reports. DMO is responsible for monitoring and closing recommendations in DMO internal audits.⁹

10. The implementation of specific audit recommendations is the day-to-day responsibility of Defence (the department and DMO) management¹⁰, and is undertaken by the responsible Defence line areas (known as Defence Groups).

11. Consistent with its scale, complexity and financial materiality¹¹, the Defence organisation has been the subject of almost 200 internal (163) and external (33) audits, including ANAO financial statement audits, that made at least one recommendation in the last five financial years. Since July 2010, 968 recommendations from such audits have been managed by Defence.¹² Of these, 71 recommendations resulted from ANAO performance audits.

Audit objective, criteria and scope

12. The objective of the audit was to assess the effectiveness of Defence’s monitoring of the implementation of ANAO and internal audit recommendations. This involved an examination of Defence’s system and procedures for monitoring and reporting on the implementation of audit recommendations, including the roles of Defence’s internal audit function, the Defence audit committee and the use of the ARMS database. The audit also examined a sample of ANAO and internal audit recommendations—reported as being complete by Defence—to assess the extent to which these recommendations had been implemented by Defence.

13. To conclude against the audit objective, the audit examined whether:

• Defence’s procedures and system for monitoring the implementation of audit recommendations provided adequate visibility and assurance on the status of audit recommendations to the Department of Defence’s internal audit function and the Defence Audit and Risk Committee.
• Defence’s procedures and system for monitoring the implementation of audit recommendations facilitated the adequate implementation of recommendations in a timely manner.

Overall conclusion

14. External performance audits conducted by the ANAO, and internal audits initiated by Defence, contribute to the improvement of management and administrative practices so that better outcomes may be achieved. Audits of Defence’s administration commonly address risks to the successful delivery of Defence outcomes and identify opportunities for improvement, generally through the use of recommendations. In combination, external and internal audit activity represents a significant investment of public resources, the benefit of which is only fully realised through the careful review of audit conclusions, and the timely implementation of agreed audit recommendations.

15. In the Australian Government context, where responsibility for addressing audit conclusions and recommendations resides at the agency level, implementation relies on organisational leadership supported by timely management action and an effective system of oversight within each agency.

16. Defence’s system for implementing audit recommendations exhibits many positive elements, such as having a clear process for assigning responsibility, and systematic monitoring and reporting on progress by Defence internal audit. There is also a generally clear allocation of roles and responsibilities within Defence. However, there are weaknesses in Defence’s system with respect to following-up on slow implementation, escalating to senior management where there are continuing difficulties, and achieving closure on the implementation of recommendations within specified timeframes.

17. At a process level, Defence’s system and procedures for monitoring and reporting on implementation provide adequate visibility and assurance on the status of audit recommendations to the department’s and DMO’s internal audit functions, and alert the departmental and DMO audit committees on overdue audit recommendations. However, regardless of the source of a recommendation (either ANAO or internal audit), or the priority assigned to its implementation by Defence, the timeliness of implementation is a significant problem. The average time taken by Defence to complete recommendations examined by the ANAO was approximately 400 days, which was on average 175 days later than the original estimated completion date agreed between the Defence Group responsible for implementing the recommendation and Defence internal audit. Additionally, approximately half of the 28 ANAO recommendations examined in this audit (all of which were reported as implemented by Defence) were assessed as not being adequately implemented. These outcomes indicate that monitoring and reporting are a necessary but not sufficient condition for achieving the timely and adequate implementation of audit recommendations.

18. Once agreed, audit recommendations become a management responsibility, and an effective system to implement recommendations will feature collective ownership within the agency and an action orientation which promotes timely and adequate management activity. In this respect the implementation of audit recommendations is similar to an agency’s successful delivery of its other business, relying heavily on agency leadership and the active support of those charged with implementation (agency management) and its oversight (internal audit and the audit committees).

19. The monitoring of audit recommendations has not been one of the Defence Audit and Risk Committee’s (DARC’s) stated priorities, although it is a requirement under its charter. Further, the DARC has not provided an annual report to the Secretary and CDF, which is also a requirement under its charter, and advice on audit recommendations has not formed part of the written reports the DARC provides to the Secretary and CDF after committee meetings.¹³ While acknowledging the competing demands on the DARC flowing from the scale and complexity of the Defence organisation, this approach has provided very little time for the consideration of audit conclusions and recommendations.

20. There is scope for the DARC to review the reports it receives on the status of overdue audit recommendations, to provide a stronger focus on the department’s performance in implementing recommendations, and as a basis for alerting the Secretary and CDF of organisational risks that may arise from the failure to implement recommendations. A targeted approach would enable the Secretary and CDF to focus on key risks for Defence.

21. Further, there are no consequences for responsible officers and Defence Groups for not implementing recommendations in a timely manner. For instance, in July 2012 the DARC wrote to six Group Heads seeking an update on recommendations overdue by more than 150 days. The DARC received no response to three of these six letters, and the DARC did not follow up this matter. While the DARC has the option to invite Group Heads responsible for overdue audit recommendations to attend a DARC meeting to provide further information, that is not the DARC’s practice. To provide additional support to Defence leadership, the DARC should prepare an annual written report to the Secretary and the CDF on its operation and activities, as provided for in its charter. This report should include information on the overall effectiveness of Defence Groups in implementing audit recommendations, and Defence’s monitoring and reporting arrangements.

22. By not implementing agreed audit recommendations in a timely manner, Defence is foregoing opportunities to enhance its performance. This does not reflect well on those managers who have the responsibility to act on the agreed recommendations and deliver stronger outcomes for Defence. The DARC also has an important charter responsibility in encouraging a stronger focus on this aspect of the department’s administration. To improve Defence’s management and implementation of audit recommendations, the ANAO has made two recommendations aimed at reinforcing management responsibilities and accountabilities for the implementation of audit recommendations.

23. As a consequence of this audit, Defence informed ANAO in February 2013 that:

Defence has initiated action to elevate the importance of the timely implementation of audit recommendations and to hold responsible officers to account for not implementing recommendations in a timely manner. Commencing 18 February 2013, the Chief Audit Executive will be providing to the Defence Committee on a regular basis, details of all overdue audit recommendations (both ANAO and Defence Internal Audit recommendations) as well as data on all outstanding recommendations.¹⁴

Key findings by Chapter

Governance Arrangements (Chapter 2)

24. Defence management is responsible for the implementation of recommendations to which Defence has agreed, including determining the appropriate actions to be undertaken to adequately implement the recommendations. To support management, and oversight overall progress, Defence has introduced clear governance arrangements for managing audit recommendations. These arrangements are based on: monitoring and reporting by the departmental internal audit function with support from the Defence Group responsible for implementation; an electronic database (the Audit Recommendations Management System—ARMS) that tracks progress and records action taken in response to audit recommendations; and reporting to and consideration by Defence’s audit committee.¹⁵

25. Individual Defence Groups are required to manage the implementation of recommendations for which they have been assigned responsibility, with the timeframe for completion agreed between the Group and Defence internal audit. A senior officer within a Group is clearly assigned responsibility for implementing a recommendation, reflecting better practice.

26. In 2010, an ANAO report examined the implementation of 15 ANAO performance audit recommendations and identified that, although all these recommendations were marked as complete in ARMS by the responsible Defence Groups, only four had been fully implemented by Defence.¹⁶ In response to this audit, Defence informed the ANAO that in order to address this issue, Defence Groups would no longer have authority for closing ANAO recommendations on ARMS, and that Defence Audit Branch (part of Audit and Fraud Control Division) would take responsibility for closing ANAO recommendations based on evidence provided by the Group. However, this revised procedure was not promulgated and implemented by Defence until January 2012.

Defence internal audit undertakes quality assurance reviews of implemented recommendations

27. Effective monitoring and review involves assessing the quality of action taken to implement a recommendation. Audit Branch undertakes quality assurance reviews of all ANAO performance audit recommendations, all high priority internal audit recommendations, and five per cent of other internal audit recommendations marked as complete in ARMS. However, quality-assuring the implementation of ANAO performance audit recommendations has resource implications for internal audit, and involves judgement in assessing the implementation status. Internal audit’s role provides additional assurance to the Chief Audit Executive and the DARC that recommendations have been implemented as intended and claimed by the relevant Defence Group.

28. In reviewing the implementation of ANAO performance audit recommendations, a number were identified by the ANAO as not being implemented adequately although some of these recommendations had been assessed as implemented by the quality assurance review process. For these recommendations, implementation was considered by the ANAO to have been only partially complete, or was considered insufficient (either the action taken did not address the recommendation, or there is no evidence that action was undertaken).

The Audit Recommendations Management System has some limitations

29. An effective information management system can underpin efforts to systematically monitor and report on progress in implementing recommendations. ARMS is a database that assists Audit Branch to track Defence’s progress in implementing recommendations, and provides the basis for reporting statistical data to the DARC. Although recently updated in July 2012, the longer-term future of ARMS in its current form remains unclear. Vendor support for the version of the application software underpinning ARMS ceased in April 2010, and proposals to replace audit management systems have not been successful because of funding constraints and the priority assigned to it by the Defence Chief Information Officer Group (CIOG).

30. During audit fieldwork, the ANAO analysed the ARMS database and interviewed users from Defence Groups about their experience in using ARMS. Defence Groups indicated that the useability of ARMS had improved with recent updates, but noted limitations with the user interface and reporting function. The ANAO found that the comments field on recommendations was updated infrequently, which often reflected that slow progress was being made in implementing a recommendation. There was also inconsistent attachment of supporting documents on closed recommendations, such as sign-off documentation from authorised officers.

There is scope for the DARC to have more involvement in following up overdue audit recommendations

31. The DARC is provided with aggregate statistics on all open ANAO and internal audit recommendations, as well as detailed statistical data on Defence’s progress in implementing audit recommendations that are overdue by more than 60 days. The DARC has responsibilities and priorities beyond the consideration of audit matters, reflecting the scale and complexity of the Defence organisation. Currently, very limited time has been allocated for consideration by the DARC of audit recommendations. The status of recommendations, particularly those that are proving difficult or which have been slow to implement, is not a part of the DARC’s written reports to the Secretary and CDF (these reports advise the Secretary and CDF on DARC business, and are provided when the Chair of the DARC is unable to verbally update the Secretary and CDF following a DARC meeting). Further, the DARC does not provide an annual report to the Secretary and CDF, notwithstanding that it is a requirement under its charter.¹⁷

32. In July 2012, the DARC wrote to six Group Heads seeking an update on recommendations overdue by more than 150 days. The DARC received no response to three of these six letters, and the DARC did not follow up this matter. These letters did not lead to an increased focus from the DARC on overdue recommendations. The DARC also has the option of following up overdue recommendations with Groups by inviting responsible Group Heads with overdue recommendations to meet with the Committee, although this approach has never been employed.

33. A prominent role in monitoring and following up recommendations by an agency audit committee indicates the importance the Chief Executive places on the implementation of recommendations. To achieve the full benefit of audit recommendations, the ANAO has recommended that Defence reinforce managers’ responsibilities for implementing agreed recommendations; and the DARC bring to the attention of the Secretary and CDF, on an exception basis, any recommendations of particular concern that have not been implemented.

Implementation of Audit Recommendations (Chapter 3)

34. There were 143 audits listed in the ARMS database as at 26 July 2012, covering ANAO audits (both financial statement and performance), Defence internal audits, and DMO internal audits. The July 2012 version of the database contains recommendations which are currently open, or were completed after 1 July 2010. These 143 audits include a total of 968 individually managed recommendations. Of these audits, 70 (49 per cent) are marked as having all their recommendations completed. As at 26 July 2012, there were 299 active recommendations of which 166 (56 per cent) were overdue.

The ANAO’s sample testing highlighted problems with timeliness, and some issues with the extent of implementation

35. Defence’s system exhibits many key elements of better practice, such as a clear allocation of roles and responsibilities amongst stakeholders. However, this has not led to the implementation of recommendations in a timely manner. For audits in 2007–08 and 2009–10, the average time taken to implement a recommendation was 275 days. This included an average delay of 88 days compared to Defence’s original estimated timeframe for completion. Only 34 per cent of recommendations in these two financial years were completed within the specified timeframe. On average, all recommendations that are currently outstanding are 400 days old. Of these, those that have exceeded their estimated completion date (that is, are overdue) are on average over 500 days old.¹⁸

36. For the sample of 52 Defence internal audits that were marked as having all recommendations completed on ARMS, authorisation to close these recommendations—from a nominated senior responsible officer within a Defence Group—was not always attached. For 19 of these audits authorisation was attached for all recommendations; for 15 of these audits authorisation was attached for some recommendations; and for 18 audits no authorisation was attached for any recommendation.

37. The ANAO also examined 48 Defence internal audit recommendations from 12 of the 52 audits in the sample. Approximately 80 per cent of the 48 recommendations were adequately implemented, based on evidence available in ARMS. However, the 12 audits reviewed by the ANAO are not representative of all the Defence internal audits with closed recommendations in ARMS. Approximately one-third of the total audits did not contain sufficient evidence to allow the ANAO to form a conclusion on the implementation of any of their recommendations. Further, only eight of these 48 recommendations were completed within Defence’s estimated completion timeframe, with the average time taken to implement a recommendation being 429 days.

38. For ANAO performance audits, the ANAO examined all recommendations that were marked as completed in ARMS from July 2009 to July 2012—a total of 28 recommendations from nine audit reports. Sixteen of the recommendations reviewed were assessed by the ANAO as being adequately implemented; six were assessed as being partially implemented; and six were assessed as either not being implemented sufficiently or not having any evidence implementation available.¹⁹ Timeliness was also a problem for the implementation of ANAO recommendations: the average time taken to implement these recommendations was 354 days, with only 10 per cent completed within Defence’s nominated timeframe.

Delays in implementation mean the full benefits of recommendations are not being realised

39. Interviews with various Defence personnel, supported by the ANAO’s examination of the ARMS database, highlighted a wide range of potential causes for the late or non-implementation of recommendations. These causes highlighted insufficient regard to many of the key principles for effective implementation outlined in Table S1, particularly planning, risk management and resourcing. Prominent causes identified were: the priorities and workloads of the staff responsible for implementation; recommendations being ‘overtaken by events’ or considered not possible to implement (for example when an internal audit recommendation relates to funding); frequent turnover of personnel; and a lack of meaningful consequences for those responsible when a recommendation is not implemented.

40. The DARC is required to provide an annual report to the Secretary and CDF on its operation and activities, including information on audit reports and recommendations. However, it has not been the DARC’s practice to provide such a report. The DARC should provide additional support to Defence leadership, through an annual report to the Secretary and the CDF, including advice on the overall effectiveness of Defence’s system for implementing audit recommendations. Providing high level visibility of Defence’s effectiveness in implementing recommendations would further reinforce management responsibilities and accountabilities for implementation.

41. Audit activity represents a significant investment of public resources, and the appropriate and timely implementation of agreed recommendations is an important part of realising the full benefits of an audit. Approximately 95 per cent of all ANAO performance audit recommendations are agreed in full by audited entities, following a process of formal consideration. It is expected that entities will only agree to recommendations where they accept the practical benefit of taking the corrective action contained in a recommendation, and that once agreed to, the recommendations will be implemented in a timely manner with sufficient management vigour and executive oversight.

Agency response

Defence acknowledges the findings contained in the audit report on Defence’s Implementation of Audit Recommendations, and agrees to the two Recommendations. Defence appreciates the value of the audit process and continually seeks opportunities for improvement.

Defence understands that the timely implementation of audit recommendations is important for realising the full benefits of an audit and for enhancing performance. Defence welcomes the Recommendations made by the ANAO which are aimed at reinforcing management responsibilities and accountabilities for the implementation of audit recommendations, and has already commenced progress to further ensure that the importance of the timely implementation of audit recommendations is better understood across the Department.

The Chief Audit Executive will work with the Group Heads and Service Chiefs to provide to the Defence Committee, on a regular basis, advice on the status of audit recommendations. Defence considers this additional level of reporting is an effective method of reinforcing managers’ responsibilities for implementing agreed audit recommendations and for bringing to the attention of the Secretary and Chief of the Defence Force, recommendations of particular concern.

 Recommendations

Footnotes

[1]     The principles relating to audits and the implementation of audit recommendations discussed in this report are generally applicable to all Australian Government entities. In this audit, the term ‘agency’ is generally used, because both the Department of Defence and the Defence Materiel Organisation are Australian Government agencies subject to the requirements of the Financial Management and Accountability Act 1997 (FMA Act).

[2]     Section 44 of the FMA Act defines ‘proper use’ as ‘efficient, effective, economical and ethical use that is not inconsistent with the policies of the Commonwealth’.

[3]     Part 7 of the FMA Act sets out the ‘special responsibilities’ of agency Chief Executives, which include a requirement under section 44 to ‘manage the affairs of the Agency in a way that promotes proper use of the Commonwealth resources for which the Chief Executive is responsible’.

[4]     Section 46 of the FMA Act requires an agency Chief Executive to establish and maintain an audit committee with functions that include: helping the agency to comply with obligations under the Act, the Regulations and Finance Minister’s Orders; and providing a forum for communication between the Chief Executive, the senior managers of the agency, the internal auditors of the agency and the Auditor-General.

The Directors of a Commonwealth authority must also establish and maintain an audit committee, in accordance with the Commonwealth Authorities and Companies Act 1997 and the Commonwealth Authorities and Companies Regulations 1997.

[5]     The functions of audit committees are set out in FMA Regulation 22C. They include: advising the Chief Executive about the audit plans of the agency; coordinating work programs relating to internal and external audits; reviewing the adequacy of the agency’s response to audit reports; and reviewing the content of audit reports with a view to advising the Chief Executive on good practice, significant matters of concern and opportunities for improvement.

[6]     The 2006 Better Practice Guide, Implementation of Programme and Policy Initiatives, emphasises the importance of leadership and effective support in the implementation process, and contains a number of checklists to assist agency Chief Executives, which have more general applicability for implementation activities.

[7]     For further information on better practice for internal audit and audit committees, see the following ANAO Better Practice Guides: Public Sector Internal Audit: An Investment in Assurance and Business Improvement, September 2012; and Public Sector Audit Committees: Independent assurance and advice for Chief Executives and Boards, August 2011.

[8]     DMO is listed as a prescribed agency under the FMA Act. For an agency to be prescribed for the purposes of the FMA Act, the basic premise is that it is legally or administratively independent at a level that justifies financial autonomy.

[9]     DMO conducts its own small program of internal audits, separate to the larger program of audits conducted by the department’s audit area (Audit and Fraud Control Division), which audits across the Defence organisation, including DMO.

[10]     The Defence portfolio consists of a number of component organisations that together are responsible for supporting the defence of Australia and its national interests. The three most significant bodies are: the Department of Defence, the Australian Defence Force (the ADF) and the Defence Materiel Organisation. In practice, these three bodies have to work together closely and are broadly regarded as one organisation simply known as Defence or the Australian Defence Organisation. The Defence portfolio also contains the Department of Veterans’ Affairs and associated bodies. However, the Department of Veterans’ Affairs is administered separately to Defence and is not included in the terms ‘Defence’ or ‘Defence organisation’ or in the scope of this audit.

[11]     The Defence organisation manages over $30 billion in funding and employs over 102 000 civilian and ADF personnel. Its business and assets are geographically dispersed and supported by complex logistic and administrative support systems, and the organisation undertakes substantial procurement activity.

[12]     Recommendations that require action from different Groups are divided into multiple individual recommendations on ARMS, and are managed as separate recommendations. Hence, the actual number of unique recommendations would be less than 968.

[13]     Defence informed ANAO in February 2013 that written reports on each DARC meeting are provided when the Chair is unable to provide the Secretary and CDF with a verbal update.

[14]     For the full actions taken by Defence in response to Recommendation No.1, see paragraph 2.91.

[15]     As discussed earlier, DMO has its own internal audit area that reports to its own audit committee—the Materiel Audit and Risk Committee (MARC).

[16]     ANAO Audit Report No.24 2009–10 Procurement of Explosive Ordnance for the Australian Defence Force.

[17]     Defence informed ANAO in February 2013 that:
                   Defence considers that such a report is likely to become a summation of the advice progressively provided to the SEC and CDF.

[18]     Outstanding recommendations include those that may have previously had an extension to their estimated completion date. Overdue recommendations include those that may have previously had an extension and then exceeded this revised timeframe. Estimated completion dates are usually set for between six and 12 months after the completion of the relevant audit.

[19]           The ANAO categorised the implementation of recommendations into three groups. Adequate—the action taken met the intent of the recommendation, and sufficient evidence was provided to demonstrate the action taken. Partial—where the action taken was less extensive than expected by the ANAO (the action either fell short of the intent of the recommendation, or only addressed some of the intended issues), or where Defence may have established a process or procedure to address an issue, however the specific action noted in the recommendation has not been done (this could also be categorised as ‘pre-emptive closure'). Insufficient/no evidence—either where there is no indication from evidence or comments that action has been undertaken, or the action taken does not address a recommendation at all.