Background

1. Performance audits play an important role in stimulating improvements in the administration and management practices of public sector organisations. Australian National Audit Office (ANAO) performance audits involve the independent and objective assessment of the administration of Australian Government entity programs, policies, projects or activities, including identifying any significant risks to the successful delivery of relevant outcomes. Performance audits initiated by entities with resources under their control—known as ‘internal’ audits—fulfil a complementary role, providing assurance to agency management on the effectiveness of the internal control environment and identifying opportunities for performance improvement.¹ The appropriate and timely implementation of audit recommendations agreed by management is an important part of realising the full benefit of an audit.²

2. The Department of Veterans’ Affairs (Veterans’ Affairs) is responsible for providing a range of programs of care, compensation, income support and commemoration for the veteran and defence force communities and their families. In 2015–16, the department will administer $11.8 billion of Commonwealth funds.³ Between 1 July 2011 and 31 December 2014, the ANAO completed five performance audits relating to Veterans’ Affairs and made 12 recommendations. Over the same period, the department has completed 60 internal performance audits which included over 300 recommendations.

Audit objective and criteria

3. The audit objective was to assess the effectiveness of the Department of Veterans’ Affairs monitoring and implementation of ANAO and internal performance audit recommendations. The audit continues a series of ANAO performance audits in recent years on the implementation of audit recommendations by Australian Government entities.⁴

4. To conclude against the audit objective, the audit examined whether the department’s arrangements for monitoring the implementation of performance audit recommendations:

• provided adequate visibility and assurance to management regarding the status of recommendations, with appropriate involvement by the audit committee and the internal audit function; and
• facilitated the appropriate implementation of ANAO and internal audit recommendations in a timely manner.

Conclusion

5. The Department of Veterans’ Affairs internal arrangements for monitoring the implementation of performance audit recommendations performed well against the above audit criteria. The department has fully or partially implemented all the performance audit recommendations in the ANAO sample.⁵

6. There is room for improvement in some aspects of the existing quarterly reporting process used to inform the department’s Audit and Integrity sub-committee, and the ANAO has made a recommendation in this regard. The recommendation is intended to assist the sub-committee in monitoring the risks to departmental operations pending implementation of open recommendations. It would also enable the sub-committee to have greater confidence that recommendations proposed for closure had in fact been fully implemented.

Supporting findings

Governance arrangements

7. Governance arrangements for monitoring the implementation of performance audit recommendations are well-developed. They facilitate the provision of reasonable assurance to the Secretary as the department’s accountable authority. In particular, clear responsibilities and reporting arrangements have been implemented to guide the work of the audit committee and relevant sub-committee in monitoring implementation.

8. Consideration of how audit recommendations should be implemented was a formal part of the department’s internal audit process. In terms of ANAO recommendations, there was not the same formal process. The department took a number of different approaches to planning the implementation of ANAO recommendations. Adopting an approach similar to the existing internal audit process would improve departmental planning and advisory processes for ANAO recommendations.

9. Supporting departmental systems, including the audit database and quarterly reporting processes, were appropriate for monitoring the progress of relevant recommendations against agreed implementation timeframes. Quarterly reporting on ‘open’ recommendations—that is, where implementation is not complete—would be improved by including clear information on the management of risks by business areas pending implementation. Likewise, reporting on recommendations before their closure could be improved by ensuring that the summary reports address all relevant actions taken to fully implement the recommendations. The ANAO has made a recommendation on these matters.

10. There also remain opportunities to enhance the operations of the audit committee and sub-committee in respect to monitoring the implementation of recommendations. In particular, there would be benefit in developing guidelines, in consultation with the sub-committee, setting out when a management representative should be invited to a meeting of the sub-committee or audit committee to discuss implementation delays.

Implementation of recommendations

11. Seventy one per cent of the performance audit recommendations sampled by the ANAO had been adequately implemented by the department, with 29 per cent partially implemented. This result was slightly better than the average of the eight entities previously audited by the ANAO.

12. On average, the department took 311.3 days to implement ANAO audit recommendations and an average of 216.6 days to implement internal audit recommendations. This is superior to the implementation times of other Commonwealth entities examined by the ANAO in 2013–14 and 2014–15.

13. For both ANAO and internal audits, around 75 per cent of recommendations were fully implemented after the original estimated time. Of the ANAO recommendations, 50 per cent were ‘overdue’ by less than three months and 25 per cent by more than three months. For internal audit recommendations, the equivalent figures were 43 per cent and 34 per cent respectively.

Recommendation

Summary of entity response

14. The Department of Veterans’ Affairs summary response to the proposed report is provided below, while its full response is at Appendix 1.

The Department of Veterans’ Affairs was pleased with the results of the audit and thanks the Australian National Audit Office for the opportunity to respond to the issues raised.

The recommendation from this audit will be used to bring about improvements in audit processes.

Introduction

1.1 Performance audits play an important role in stimulating improvements in the administration and management of public sector entities as well as providing independent assurance to Parliament on the administration of programs. Recommendations in audit reports highlight actions that are expected to improve entity performance when implemented. The appropriate and timely implementation of audit recommendations agreed by management is an important part of realising the full benefit of an audit.⁶

1.2 Primary responsibility for implementing agreed audit recommendations generally lies with senior managers in the business area of the entity that was subject to the audit. Successful implementation of audit recommendations requires strong senior management oversight and implementation planning to set clear responsibilities and timeframes for addressing the required action.⁷ Implementation planning should involve key stakeholders, including the internal audit function.

1.3 Audit committees, through their position in an entity’s governance framework⁸, also have an important role in monitoring the implementation of recommendations. Audit committees assist the accountable authority to ensure that the anticipated benefits of audit reports are realised, through the effective and timely implementation of recommendations. The audit committee’s role in maintaining oversight of implementation is supported by the entity’s internal audit function, including by monitoring and providing advice on management’s progress in implementing recommendations.

1.4 This audit examines the Department of Veterans’ Affairs (Veterans’ Affairs or the department) systems for monitoring the implementation of recommendations arising from ANAO performance audits—often referred to as ‘external’ audits—and the department’s internal performance audits.

Veterans’ Affairs framework for managing recommendations

1.5 As required by the Public Governance, Performance and Accountability Rule 2014, the functions of the department’s Audit and Risk Committee (the audit committee), are set out in a written Charter, approved by the Secretary in July 2014. Under the Charter, the audit committee is to provide independent assurance to the Secretary in relation to both internal and ANAO audit matters, including the monitoring of management’s implementation of audit recommendations and advising the Secretary on action to be taken on ‘significant issues’ raised in relevant audit reports.

1.6 The Charter also provides for the establishment of sub-committees to assist the audit committee in carrying out its responsibilities. In relation to internal and ANAO performance audits, this function is performed by the Audit and Integrity sub-committee (the sub-committee), whose Chair is also an independent member of the audit committee.

1.7 Under the sub-committee’s terms of reference, it is to provide independent assurance to the audit committee on audit matters, by monitoring management’s implementation of audit recommendations and advising the audit committee on any action to be taken on ‘significant issues’ raised in relevant audit reports. In consequence, the sub-committee undertakes the monitoring function, providing a brief status report on the progress of the implementation of audit recommendations at each audit committee meeting. The status report, amongst other issues, seeks the audit committee’s endorsement of the sub-committee’s proposals for the ‘closure’ of recommendations on the basis they have been implemented.

1.8 In relation to their responsibilities for monitoring the implementation of audit recommendations, both the audit committee and sub-committee are supported by departmental officials (the ‘Assurance Team’) from the Legal Services, Assurance and Deregulation Branch. The department’s contracted internal auditors⁹ also work with the Assurance Team to support the monitoring effort. The monitoring process is underpinned by a system of quarterly reporting to the Assurance Team by departmental business areas.

1.9 Figure 1.1 illustrates Veterans’ Affairs framework for managing audit recommendations.

Figure 1.1: Department of Veterans’ Affairs framework for managing audit recommendations

Source: ANAO analysis of Veterans’ Affairs information.

Parliamentary interest and previous audits

1.10 In recent years, the Joint Committee of Public Accounts and Audit (the JCPAA) and other parliamentary committees have expressed interest in the performance of Australian Government entities in relation to implementing audit recommendations. For example, the JCPAA noted that:

… the purpose of internal and external auditing is to identify weaknesses and better enable an organisation to address risk. The benefits of this work are undermined if agencies do not institutionalise robust monitoring, implementation, reporting and oversight mechanisms … the Committee continues to support the strategic use of ‘follow-up audits’, as part of the ongoing process of improving agency performance.¹⁰

1.11 In response to this feedback, the ANAO has conducted a series of performance audits on the implementation of audit recommendations. In the past three years, the ANAO has completed the following audits:

• ANAO Audit Report No.8 2014–15 Implementation of Audit Recommendations which examined the Department of Health’s implementation of ANAO performance audit and internal audit recommendations;
• ANAO Audit Report No.34 2013–14 Implementation of ANAO Performance Audit Recommendations which was a cross-agency audit that examined the implementation of ANAO performance audit recommendations by two agencies: the Department of Human Services; and the Department of Agriculture;
• ANAO Audit Report No.25 2012–13 Defence’s Implementation of Audit Recommendations which examined the Department of Defence’s implementation of ANAO performance audit and internal audit recommendations; and
• ANAO Audit Report No.53 2012–13 Agencies’ Implementation of Performance Audit Recommendations which was a cross-agency audit that examined the implementation of ANAO performance audit recommendations by four agencies: the Department of Education, Employment and Workplace Relations; the Department of Families, Housing, Community Services and Indigenous Affairs; the Department of Finance and Deregulation; and the Department of Infrastructure and Transport.

1.12 This series of ANAO performance audits has highlighted that a structured and planned approach to the oversight and implementation of audit recommendations assists entities to manage timeliness, completeness and adequacy of implementation, by allowing progress to be clearly targeted and monitored.¹¹

Audit objective, criteria and scope

1.13 The audit objective was to assess the effectiveness of the Department of Veterans’ Affairs monitoring and implementation of ANAO and internal performance audit recommendations.

1.14 To conclude against the audit objective, the audit examined whether the department’s arrangements for monitoring the implementation of performance audit recommendations:

• provided adequate visibility and assurance to management regarding the status of recommendations, with appropriate involvement by the audit committee and the internal audit function; and
• facilitated the appropriate implementation of ANAO and internal audit recommendations in a timely manner.

1.15 To assess the timeliness of Veterans’ Affairs implementation of the recommendations, the ANAO examined all 12 ANAO recommendations and 236 internal audit recommendations from performance audit reports finalised between 1 July 2011 and 31 December 2014 that were recorded as implemented (closed) by the department as at 30 June 2015.

1.16 The 12 ANAO performance audit recommendations¹² and a sample of 12 closed internal audit recommendations were also analysed in detail to assess the adequacy of implementation. The result of this analysis was compared against the results for the eight other Commonwealth entities examined in this series of ANAO audits.

1.17 In conducting the audit, the ANAO met with the Chairs of the audit committee and sub-committee, the Veterans’ Affairs Chief Audit Executive, Assurance Team staff, and senior members of the contracted internal auditors. The ANAO also reviewed key documentation related to the monitoring of audit recommendations, including audit committee and sub-committee papers, the department’s audit database, and documentation provided to evidence the implementation of a sample of closed ANAO and internal audit recommendations.

1.18 The audit was conducted in accordance with ANAO auditing standards at a cost to the ANAO of approximately $230 000.

Introduction

2.1 Governance refers to the arrangements and practices which enable an organisation to set direction and manage its operations to achieve expected outcomes and discharge its accountability obligations. Veterans’ Affairs governance arrangements for managing the implementation of internal and ANAO performance audit recommendations are supported by the department’s Legal Services, Assurance and Deregulation Branch—particularly the Branch’s Assurance Team and the contracted internal auditors. The department’s Audit and Integrity sub-committee (the sub-committee) has direct oversight of the progress made by the department in implementing audit recommendations. The sub-committee reports to the Audit and Risk Committee (the audit committee), which has ultimate responsibility for providing independent assurance to the Secretary on audit matters.

Have clear responsibilities and reporting arrangements been established for monitoring implementation?

2.2 Audit committees do not displace or change the management and accountability arrangements within entities, but are intended to enhance the existing governance framework, risk management practices and control environment, by providing independent assurance and advice.¹³

2.3 Figure 2.1 shows the responsibilities of the Veterans’ Affairs audit committee in relation to internal and external (ANAO) audits, including monitoring the implementation of both internal and ANAO audit recommendations. The scope of the audit committee’s responsibilities in relation to such audits is consistent with the requirements set out in section 17 of the Public Governance, Performance and Accountability Rule. The sub-committee’s responsibilities, which are set out under its terms of reference, largely mirror those of the audit committee. The three-member sub-committee reports to the audit committee and is chaired by an independent external member of the audit committee.¹⁴

Figure 2.1: Audit and Risk Committee responsibilities

Source: Department of Veterans’ Affairs, Audit and Risk Committee Charter.

2.4 The sub-committee generally meets quarterly, with an update of the progress of the implementation of audit recommendations as a standing item at each meeting. The sub-committee considers a report provided by the Assurance Team summarising the status of each open recommendation.¹⁵ The report includes proposals for closing recommendations on the basis that implementation is complete, as well as applications for a formal extension of implementation timeframes. The audit committee’s meetings are held quarterly¹⁶, usually about two weeks after the sub-committee meeting. At the audit committee, a standing item is a report from the sub-committee, which includes a summary of the sub-committee’s proposals regarding relevant audit recommendations and a request that the audit committee endorse those proposals.¹⁷

2.5 Consistent with its Charter, the audit committee provides a short written annual report to the Secretary. The 2014 report observed that one of the key themes identified from internal audit reports was that ‘implementation of agreed audit recommendations is variable and often impacted by changes in responsible officers’. The report concluded that ‘overall [Veterans’ Affairs] open audit recommendations are being adequately managed.’¹⁸ The audit committee’s Charter also requires it to provide a report to the Secretary after each committee meeting on significant issues and outcomes. In order to meet this requirement, the Chair of the audit committee advised the ANAO that he meets with the Secretary within two weeks of every audit committee meeting. Following audit committee meetings, the department’s Executive Management Board also receives a briefing on implementation of recommendations. Such briefings focus on recommendations where implementation has been delayed, or where actions by the business area¹⁹ may not adequately address the intent of the recommendation.

Do departmental processes for managing audits include planning for implementation of audit recommendations?

2.6 Previous ANAO performance audits have highlighted that the successful implementation of audit recommendations requires strong senior management oversight and implementation planning that sets clear responsibilities and timeframes for addressing the required action.²⁰

2.7 Within Veterans’ Affairs, the primary responsibility for the implementation of audit recommendations rests with the relevant business area. The content of internal audits is discussed by business area senior managers and the relevant internal audit team at ‘findings and recommendations’ workshops.²¹ Key issues considered at the workshops are: the general findings; the rating to be applied to the findings²²; and indicative recommendations. The agreed implementation actions and timeframes to implement recommendations appearing in internal audit reports are formally signed-off at First Assistant Secretary level.^23,24 Audit reports contain the specific actions that the business area(s) have committed to undertake to implement the recommendations, as well as a date by which actions are anticipated to be completed. As such, the final reports effectively contain a summary implementation strategy and overall this process is consistent with better practice.²⁵

2.8 In contrast, the department did not consistently identify implementation timeframes for the external (ANAO) audit recommendations examined in this audit prior to finalisation of the relevant audit. Adopting an approach similar to the existing internal audit process would improve departmental planning and advisory processes for ANAO recommendations. In particular, information on estimated implementation timeframes for each proposed ANAO recommendation and any risks to implementation within the relevant timeframe could usefully be included.

2.9 For both internal and ANAO audit recommendations, information on the status of implementation, including an anticipated timeframe for completion, is collected as part of the quarterly update process undertaken by the Assurance Team in preparation for the sub-committee meetings. For ANAO recommendations, particularly where implementation involves more complex issues such as potential legislative changes, more detailed implementation strategies may not be developed until some months after the relevant ANAO report is finalised.

Are appropriate systems in place for monitoring the implementation of recommendations?

Audit recommendations database

2.10 While the implementation of audit recommendations is a management responsibility, an entity’s internal audit function is well placed to monitor progress.²⁶ Effective monitoring requires a system that accurately tracks progress and records the actions of the business area responsible for progressing action against timeframes.

2.11 Once internal audit reports are finalised, they are formally tabled at the audit sub-committee’s quarterly meetings.²⁷ ANAO audits involving Veterans’ Affairs are tabled at both audit committee and sub-committee meetings following their tabling in Parliament. These audits, and the relevant recommendations, are then entered into a customised database (the audit database) developed by the department’s contracted internal auditors. The database contains records for both ANAO and internal performance audits since 2000. For each audit report, the database contains:

• the individual recommendations;
• the rating (critical recommendation 1 (CR1), CR2, CR3 or business improvement recommendation) applying to each recommendation;
• the business area responsible for implementation;
• the original business area response or comment on the recommendation, including agreed implementing actions;
• the original estimated timeframe for implementation;
• for CR recommendations, any agreed extensions of time for implementation; and
• for CR recommendations, whether the recommendation has been implemented, and if so, the month and year that the recommendation was closed.

2.12 For CR recommendations, the successive quarterly updates from the business area against each recommendation are also included in the audit database. At least for the more recent records²⁸, the update is accompanied by a brief commentary by the Assurance Team, including whether progress towards implementation is progressing satisfactorily. The database is also capable of generating a range of reports.

Reporting processes

2.13 As previously noted, the sub-committee reviews the status of recommendations at its quarterly meetings and then makes proposals to the audit committee for closures and extensions of time. As the initial step in this process, the Assurance Team email a list of open audit recommendations to the relevant business areas for review and updating.²⁹ Table 2.1 outlines the key steps in the reporting process.

Table 2.1: Key steps in reporting implementation progress to the audit sub-committee and audit committee

Source: ANAO analysis of Veterans’ Affairs information.

2.14 The reporting arrangements have been refined over time to improve the quality of the quarterly reporting received from business areas. In particular, changes were made in mid-2014 to provide more detail in relation to how the risks associated with open recommendations were being managed by business areas while action on the recommendations was incomplete. A separate ‘extension request’ process was also developed for cases where the previously agreed date for completing implementation action on a recommendation would not be met.³⁰ Under the revised arrangements for extensions of time, the senior manager of the relevant business area is required to provide details of the reasons for the delay in action on the recommendation and provide a new target date. The process also requires the manager to confirm that they are satisfied that the risk of not implementing the recommendation by the previously agreed date is acceptable, and confirm that the action will be implemented by the revised date.

Quarterly Reporting

2.15 Departmental policy is that the quarterly business area updates, like the extension of time requests, are required to be approved by the senior manager of the responsible area. In the case of Divisions (which make up the majority of the department’s business areas), this is a First Assistant Secretary. The updates are generally provided to the Assurance Team by Divisional Coordinators. Variances in record keeping practices meant that in some cases evidence of the required approval was not retained by the business areas in either hard or electronic format and the department could not provide evidence of full compliance with departmental policy.

2.16 Examination of the quarterly business area updates by the ANAO shows that a minority of the updates did not always contain sufficient information, including the necessary information to enable the Assurance Team and internal auditor to determine whether implementation was complete or not. In such cases the Assurance Team sought clarification or further information from the business area via email or phone. Based on comparing the original updates with the revised updates, this follow-up process was successful in obtaining additional information in order to provide a reasonable level of assurance in reports provided to the sub-committee.

2.17 The reports provided to the sub-committee are accompanied by a covering brief that highlights, amongst other things, the number of open recommendations that are more than 12 months old and the number of open recommendations for each business area, all identified against the CR rating. This assists the sub-committee in readily identifying any issues of concern.

2.18 For the 31 recommendations proposed for closure at the May 2015 sub-committee meeting, the relevant business area had provided the Assurance Team with TRIM³¹ reference numbers to documentation providing evidence of implementation actions for 13 recommendations. For 10 of these 13 recommendations, the report provided to the sub-committee noted that the Assurance Team ‘sighted’ the relevant evidence. The ANAO was able to locate the relevant evidentiary documents in the nominated TRIM files for 12 of the 13 recommendations.³² Requiring the inclusion of TRIM reference numbers for all recommendations that are being considered for closure would: assist the Assurance Team to more readily verify the actions being taken to implement recommendations³³; and increase the level of assurance provided to the sub-committee and through it to the audit committee.

2.19 A key area in which the quarterly reports provided to the audit sub-committee could be improved is commentary on risk management of open recommendations. While this was a significant component of changes made to the reporting process from 2014, the most recent quarterly reports (May and August 2015) contain limited information on how the responsible business area was managing relevant risks to departmental operations pending implementation of the relevant recommendation. This was in spite of the Assurance Team specifically reminding business areas, before the August 2015 sub-committee meeting, of the need to address risk management issues in their quarterly updates.

Reporting on recommendations before their closure

2.20 The ANAO also reviewed the reporting for those recommendations which it assessed as only partially implemented when closed.³⁴ It is important that the reports provide an adequate basis for advising the audit sub-committee that the recommendations should be closed. Premature closure of recommendations reduces the ability of the sub-committee and the audit committee to adequately identify and monitor risks or identify the need for remedial action where a business area is not making adequate or timely progress in its implementation of a recommendation.

2.21 Two recommendations examined as part of the ANAO sample identified that the advice to the audit sub-committee (as recorded in the Veterans’ Affairs audit database) did not fully address the specific actions to implement the recommendations. For example, one internal audit recommended the development of seven performance measures.³⁵ The business area’s advice to the sub-committee stated that ‘improvements for reporting [Military Rehabilitation and Compensation Act] matters have been implemented’ and the recommendation was closed on this basis. However, during this audit, the department acknowledged that its systems failed to capture the necessary information to report on four of the seven measures to be implemented. To provide additional assurance to the sub-committee where a recommendation is being considered for potential closure, there would be benefit in the business area ensuring that its quarterly update report—which forms the basis of advice to the sub-committee—explicitly address whether it has undertaken all the actions necessary to fully implement the recommendation.³⁶ Where changes of policy or other developments mean that actions as originally contemplated cannot be carried out or may be delayed, any proposal for closure should note these circumstances so the sub-committee is fully informed, and can appropriately advise the audit committee and through it, the accountable authority.

The audit committee and sub-committee

2.26 A distinguishing feature of an audit committee within an entity’s governance framework is its potential for objectivity, as audit committees do not undertake management responsibilities. An effective agency system for implementing audit recommendations will be supported by an audit committee that monitors management’s implementation of audit recommendations, prioritising recommendations that are overdue or that pose significant risk or exposure to the department.

Considering implementation progress reports

2.27 The ANAO does not normally attend the sub-committee meetings. For the purposes of the current audit, the audit team attended the May 2015 meeting as observers. At that meeting, the implementation of audit recommendations report was considered in detail, with the sub-committee reviewing the status of each individual recommendation before adopting the Assurance Team’s proposals for the closure of specific recommendations, and the granting of extensions of time for other recommendations. At the subsequent (June 2015) meeting of the audit committee³⁷, progress in implementing audit recommendations was briefly touched upon by the sub-committee Chair when speaking to the sub-committee’s overall written report.

2.28 The ANAO’s review of audit committee and sub-committee minutes for 2013 to 2015 indicates that the committees have been conscious of the need for timely implementation of audit recommendations and placed some emphasis on seeking improvement. Notably, discussions at the sub-committee’s February 2014 meeting, and subsequently at the audit committee, resulted in changes to various procedures to: strengthen the involvement of senior business area managers in finalising internal audit reports; and improve quarterly business area reporting.

Seeking updates direct from business area management

2.29 As part of the process for maintaining strong relationships with entity senior management, audit committees may request management representatives to attend meetings, to facilitate further discussion on action to implement audit recommendations, or to explain why any recommendation has not been addressed appropriately or in a timely way.³⁸

2.30 The ANAO’s attention was directed by Veterans’ Affairs to one such occasion in which a business area senior manager attended an audit committee meeting in June 2012 to report on the status of a recommendation from a 2008–09 internal audit. The audit committee subsequently decided that where an audit recommendation is affected by significant developments, the responsible business area manager should attend the audit committee to discuss the surrounding issues, instead of just reporting through the regular committee reporting process.³⁹

2.31 In reviewing the audit committee and sub-committee meeting minutes from the last two years, the ANAO found only one other instance, at the October 2014 sub-committee meeting, where a business area manager attended a meeting to discuss progress in implementing a recommendation. Over the last three years there have been 16 internal audit recommendations that have been given three or more extensions of time, indicating significant issues with implementation of these recommendations. There would be benefit in developing guidelines, in consultation with the sub-committee, setting out when a management representative should be invited to a meeting of the sub-committee or audit committee to discuss implementation delays.

Introduction

3.1 Audit recommendations identify risks to the successful delivery of outcomes consistent with policy and legislative requirements, and highlight actions aimed at addressing those risks, and opportunities for improving entity administration. Entities are responsible for the implementation of audit recommendations to which they have agreed, and the timely implementation of recommendations allows entities to realise the full benefit of audit activity.

Audit approach

3.2 The ANAO assessed Veterans’ Affairs performance in implementing 12 recommendations from five ANAO performance audit reports tabled in Parliament between 1 July 2011 and 31 December 2014. These five reports represented all ANAO audit reports tabled in this period that had recommendations directed towards Veterans’ Affairs.⁴⁰ The ANAO also examined the implementation of a sample of 12 recommendations from 12 internal performance audit reports finalised during the same period. All ANAO and internal audit recommendations had been recorded as implemented (that is, closed) by the department’s Legal Services, Assurance and Deregulation Branch as at 30 June 2015.

3.3 The ANAO also assessed the timeliness of Veterans’ Affairs implementation of the 12 recommendations from the ANAO reports plus 236 recommendations from 60 internal performance audit reports finalised between 1 July 2011 and 31 December 2014. The 236 recommendations represent all internal audit recommendations made during this period that were also recorded as implemented as at 30 June 2015⁴¹, with the exception of 70 business improvement recommendations.⁴²

3.4 Table 3.1 summarises the number of audit recommendations assessed by the ANAO for progress in implementation and the timeliness of implementation.

Table 3.1: Recommendations assessed July 2011–December 2014

Source: ANAO.

3.5 The ANAO reviewed entity documentation, interviewed key staff, and extracted data from the department’s audit database to inform its assessment.

Did Veterans’ Affairs implement audit recommendations adequately?

3.6 The definitions used to assess the extent to which recommendations were implemented are provided in Table 3.2.

Table 3.2: ANAO’s categorisation of implementation

Source: ANAO.

3.7 Table 3.3 provides a summary of Veterans’ Affairs implementation of 12 ANAO audit recommendations and 12 internal audit recommendations. The audit recommendations have been provided in full at Appendix 3 including the ANAO’s assessment of them. In summary, 17 (71 per cent) of the sampled recommendations were adequately implemented. The remaining seven recommendations (29 per cent) were only partially implemented, despite having been recorded as ‘closed’ in the department’s audit database.

Table 3.3: Overview of the ANAO’s assessment

Source: ANAO analysis.

3.8 The department performed slightly better in implementing the recommendations from internal audits compared to ANAO recommendations. Eight (67 per cent) ANAO recommendations were adequately implemented, compared to nine (75 per cent) internal audit recommendations.

ANAO recommendations

3.9 Five of the twelve ANAO recommendations were ‘agreed with qualification’ by Veterans’ Affairs. For three of these recommendations, the department took actions that met the intent of the recommendation, and were assessed as being adequately implemented. For the other two recommendations that were ‘agreed with qualification’, while the department took a variety of actions to improve departmental performance (including through staff training, staff reminders and some limited internal quality assurance processes) these did not fully meet the intent of the recommendations. These actions were however broadly consistent with the department’s responses to the recommendations contained in the audit report.⁴³

3.10 In the case of the remaining two ANAO recommendations that were fully agreed by Veterans’ Affairs but were assessed by the ANAO as only being partially implemented, the department had either not fully completed implementation, or some of the intended actions had not proceeded due to uncertainty about whether they would be fully consistent with government policy on performance reporting.⁴⁴

Internal Audit Recommendations

3.11 In relation to the three internal audit recommendations that the ANAO assessed as partially implemented, the basis for reaching this assessment varied. In one case, there were delays in negotiations with other jurisdictions about obtaining health data and all relevant actions could not be implemented to fully meet the intent of the recommendation. In another instance, involving the development and application of a quality assurance process for departmental procurement activity, a draft quality management framework was put in place by early 2014, but application of the framework remained incomplete as at mid-2015.⁴⁵

How Veterans’ Affairs compared with other entities

3.12 In recent years the ANAO has conducted a program of performance audits that have assessed the implementation of audit recommendations by eight entities, using the same assessment criteria outlined in Table 3.2. While recognising that individual audit recommendations may vary in their scope and complexity, the ANAO’s published findings provide a basis for comparative assessment of entity performance.

3.13 Figure 3.1 presents the results from the four ANAO audit reports tabled in Parliament to date⁴⁶, alongside the results of the ANAO’s assessment of Veterans’ Affairs in the current audit. The figure shows the proportion of recommendations that were assessed by the ANAO as adequately, partially or not adequately implemented by the audited entities, including Veterans’ Affairs. In summary, Figure 3.1 indicates that:

• an average of 65 per cent of recommendations were assessed as adequately implemented across the eight entities, as compared to 71 per cent in Veterans’ Affairs;
• an average of 24 per cent of recommendations were assessed as partially implemented across the eight entities, as compared to 29 per cent in Veterans’ Affairs; and
• an average of 11 per cent of recommendations were assessed as not adequately implemented across the eight entities, whereas Veterans’ Affairs had no recommendations in this category.

Figure 3.1: Implementation of audit recommendations by Australian Government entities

Source: ANAO analysis.

Did Veterans’ Affairs implement audit recommendations in a timely manner?

3.14 Audit recommendations can vary in scope and complexity, and as a consequence the implementation task may require coordination across a range of business areas within an entity. The risks involved and the time taken to implement recommendations within entities can also vary. If implementation is not progressed promptly, and individual risks remain untreated, the full value of the audit may not be achieved. In this context it is important that the entity’s assurance function and the audit committee keep the accountable authority informed on progress with implementing recommendations that are complex, difficult or overdue.

3.15 Table 3.4 provides an overview of the timeliness of Veterans’ Affairs implementation of 248 recommendations from performance audit reports finalised between 1 July 2011 and 31 December 2014.⁴⁷ On average, ANAO recommendations took 311.3 days to implement, as compared to 216.6 days for internal audit recommendations.⁴⁸ This is superior to the implementation times of other Commonwealth entities examined by the ANAO in 2013–14 and 2014–15⁴⁹ although direct comparisons are difficult as: the relevant information (particularly implementation dates) is often recorded in different ways by entities; and the complexity of the recommendations will vary.

3.16 The average time taken for Veterans’ Affairs to implement internal audit recommendations has shown some improvement over time, dropping from an average of 279 days for recommendations contained in reports published in 2011–12 to 197.1 days for reports published in 2013–14.^50,51 Similarly, the average time taken for Veterans’ Affairs to implement ANAO recommendations has dropped from an average of 319.4 days for recommendations contained in ANAO reports tabled in Parliament in 2011–12 to 216.5 days for reports tabled in 2013–14.⁵²

Table 3.4: Timeliness of implementation of audit recommendations

Notes: Percentages are rounded to the nearest whole number. The percentages provided in the table do not equal 100 as 1 (or 8 per cent) of the ANAO recommendations and 14 (or 6 per cent) of the internal audit recommendations did not have clear estimated implementation dates.

Source: ANAO analysis.

3.17 Overall, the average time taken to implement recommendations has been considerably longer than originally estimated by the department.⁵³ For both ANAO and internal audits, around 75 per cent of the sampled recommendations were implemented after the original estimated time, although the majority of these completions were ‘overdue’ by no more than three months.⁵⁴ For overdue recommendations, the extra time taken to complete implementation as compared to the original estimate averaged 102.7 days for ANAO recommendations and 144.4 days for internal audit recommendations.⁵⁵

3.18 Figure 3.2 provides a more detailed breakdown of the time taken to implement recommendations. Notably, 28 percent of internal audit recommendations were implemented within three months of the report being finalised, a significant factor underpinning the considerably shorter average period to implement internal audit recommendations as compared to ANAO recommendations. For reports published from 1 July 2011 onwards, the longest time taken to implement an internal audit recommendation was 1247 days (3.4 years); for an ANAO recommendation it was 749 days (2.1 years).⁵⁶

Figure 3.2: Time taken to implement audit recommendations

Source: ANAO analysis.

3.19 Strategies to avoid under-estimating implementation timeframes (where the relevant actions involved multiple business areas or were reliant on IT systems changes) were discussed by the audit sub-committee and audit committee in the first half of 2014. For internal audits, it was decided that from July 2014: business areas should be represented at least at Assistant Secretary level at the findings and recommendations workshops⁵⁷; and that the agreed implementation actions and associated timeframes to implement the recommendations should be signed-off at First Assistant Secretary level.⁵⁸ The department’s contracted internal auditors advised the ANAO that the changes were expected to assist in a more accurate estimation of implementation timeframes.

Implementing priority recommendations

3.20 The department applies a useful priority rating scale to all recommendations, so as to ‘communicate [its] importance [to] the audit committee, management and staff of the Department.’ The department moved to a new three-point rating scale in March 2015. Before then, the scale consisted of three main ratings—with the highest priority being ‘CR1’ (critical recommendation 1) followed by CR2 and CR3—and a separate rating for ‘business improvement recommendations’.⁵⁹ As a matter of policy, Veterans’ Affairs gave a default CR2 rating to ANAO recommendations. Table 3.5 shows that, in relation to internal audit recommendations from reports finalised between 1 July 2011 and 31 December 2014, only three have been rated as CR1, with the remainder fairly evenly divided between CR2 and CR3. The average time to implement the three CR1 recommendations was 182 days.

Table 3.5: Priority rating scale and assessment of the sample recommendations

Note: The relevant departmental policy document also notes that ‘corrective actions vary considerably in complexity and the complexity of an agreed action influences the timeframe for its implementation. Notwithstanding the need to take prompt corrective action, some aspects of a permanent solution to a CR1/CR2 observation may require more time than the nominal three to six months associated with this level of severity.’

Source: Department of Veterans’ Affairs records.

3.21 Figure 3.3 shows the length of time taken to implement the CR2 recommendations. According to the department’s priority rating scale, moderate recommendations should preferably be implemented within six months. Some 58 per cent of all CR2 recommendations were implemented within the six month timeframe. The proportion implemented within six months falls to 33 per cent if only the ANAO recommendations are counted. The average time to close all CR2 recommendations is 255.3 days. Over time there has been improvement, from an average of 348.5 days for recommendations reported in 2011–12 reports to 206.9 days for recommendations reported in 2013–14 reports. The average figure for 2014–15 (to 31 December 2014) is lower again at 124.5 days, but this will increase once the remaining open recommendations are implemented.

Figure 3.3: Time taken to close CR2 recommendations

Source: ANAO analysis.

3.22 Figure 3.4 shows the length of time taken to implement CR3 recommendations. According to the department’s priority rating scale, CR3 recommendations should be implemented by the business area ‘when resources permit’. A relatively high proportion of recommendations (70 per cent) was implemented within six months, with over half the remaining recommendations implemented within twelve months. Over time there has been improvement, from an average of 214.3 days for CR3 recommendations reported in 2011–12 reports to 187.6 days for CR3 recommendations reported in 2013–14 reports. The figure for 2014–15 (to 31 December 2014) is lower again at 131.9 days, but this will increase once the remaining open recommendations are implemented.

Figure 3.4: Time taken to close CR3 recommendations

Source: ANAO analysis.

Appendices

Please refer to the attached PDF of the report for the Appendices:

• Appendix 1: Entity Response
• Appendix 2: Classification of audit recommendations
• Appendix 3: The ANAO’s assessment of implementation