Background

The Department of Veterans' Affairs (DVA) exists to serve members of Australia's veteran and defence force communities, war widows and widowers, widows and dependants, through programs of care, compensation, commemoration and defence support services. DVA's income support and compensation responsibilities are, primarily, supported by the following three Information Technology (IT) systems:

• Veteran Information Enquiry Window (VIEW)-an enquiry system that consolidates all the DVA client information into one application and allows staff to easily examine this information;
• Pension Information Processing System (PIPS)-which allows DVA staff to retrieve client financial and personal data; and
• Compensation Claims Processing System (CCPS)-used by departmental staff to assess war veterans' claims.

DVA administrative and management systems that support these operational systems include: DOLARS-a financial management information system; and PAHRIS-a human resource management system.

Audit Report No.44, 2000–01, Information Technology in the Department of Veterans' Affairs

The Australian National Audit Office (ANAO) Audit Report No.44¹ detailed the findings of a joint financial statement and performance audit of DVA's IT systems. The objective of the financial statement component of the audit was to determine if the IT systems supported the production of reliable financial statements. The objective of the performance component of the audit was to determine if the outputs of the IT systems met quality and service delivery targets. Fieldwork for the performance component of the audit focused on VIEW, PIPS and CCPS.

The ANAO concluded that, overall, DVA satisfactorily managed its IT systems. In particular, the IT systems supported the production of a reliable set of financial statements and the outputs for the (three) systems reviewed met quality and service delivery targets.

The ANAO made two recommendations (the second having five parts) to address problems that were identified in the course of the audit. The recommendations addressed the monitoring of IT changes; IT performance information; information systems model documentation; and the facilitation of the interpretation of performance information. DVA agreed with the recommendations.

DVA IT governance audit

The Strategic Review Services (SRS) section within DVA undertook an audit of DVA's IT governance in the second half of 2003. The SRS audit, among other things, addressed the following issues and recommendations in Audit Report No.44:

• DVA IT change management (Recommendation No.1);
• DVA IT performance information (Recommendation No.2 parts i, ii); and
• DVA IT systems model documentation (Recommendation No.2 part iii).

As relevant, the ANAO has used the findings of the DVA IT governance audit to support ANAO findings in this report. This approach is consistent with ANAO auditing standards that are based on codified auditing pronouncements issued by the Australian Accounting Research Foundation on behalf of the CPA Australia and the Institute of Chartered Accountants in Australia, known as the Auditing Standards (AUSs) and Auditing Guidance Statements (AGSs), specifically, AUS 806.

Audit objective, scope, approach and methodology

The objective of the follow-up audit was to determine the extent to which DVA has implemented the recommendations in Audit Report No.44. The scope of the audit was restricted to an examination of how DVA put these recommendations into operation and monitored progress.

The criteria for IT aspects of the audit were developed from the high-level Control Objectives for Information and related Technology (COBIT) framework associated with each recommendation/sub-recommendation in Audit Report No.44. Supplementary criteria were based on the experience obtained by the ANAO when undertaking performance information related audits and developing better practice guides on this subject.

Key Findings

IT change management

Audit Report No.44-Recommendation No.1

The ANAO recommended that DVA monitor the time taken to implement changes of its application software, from initiation of a change to its execution, to help ensure all changes which have a significant impact on the business are completed in a timely manner.

DVA has substantially implemented Recommendation No.1 that relates to change management. The implementation of changes to IT software is monitored on a project basis. However, DVA does not produce quantitative information on the timeliness of the implementation of software changes.

IT Performance Information

Audit Report No.44-Recommendation No.2, parts i, ii

The ANAO recommended that DVA:

2.i formulate IT Balanced Scorecard measures that directly link, and relate to, relevant IT strategy statements. These measures would then enable effective monitoring of progress in implementing the IT Strategic Plan; and

2.ii develop suitable measures for IT systems that support DVA's client service reporting requirements.

DVA has not implemented Recommendation No.2.i that relates to the IT Balanced Scorecard. DVA's IT Balanced Scorecard consists of operational indicators, and there is an insufficient number and range of indicators to directly link, and relate to, the IT Strategic Plan. However, the Department has commenced a work program that has the capacity to address the problems identified by the ANAO.

DVA has substantially implemented Recommendation No.2.ii that relates to client service performance information. Although client service performance information is not included in DVA's IT Balanced Scorecard, the information is collected and is readily accessible to DVA staff and management.

IT architecture: Systems model documentation

Audit Report No.44-Recommendation No.2, part iii

2.iii The ANAO recommended that DVA review and update the information systems model documentation to ensure it is complete and current to facilitate operations and any changes that may be required.

DVA has partially implemented Recommendation No.2.iii that relates to systems model documentation. DVA recognises the importance of, and is working towards, establishing a formally-documented information architecture. DVA is making progress on the improvement of its corporate data models and associated data management for a complex IT environment.

The interpretation of performance information

Audit Report No.44-Recommendation No.2, parts iv, v

The ANAO recommended that DVA:

2.iv clearly articulate in its Balanced Scorecard reports:

• inherent limitations on the accuracy of performance reports; and
• clear explanations and definitions of key terms used in the reports, including what is meant by ‘error' and ‘accuracy'; and

2.v formalise acceptable variations from agreed targets for all internal and external IT system reporting.

DVA has implemented Recommendations No.2.iv and 2.v that relate to the reporting of IT performance information. The IT Balanced Scorecard reports examined were of an appropriate quality. Accuracy limitations were evident and the reports defined key terms and identified problems. The reports included clear explanations of major service breakdowns and, although there were no explicit targets, there was an implicit target of zero significant IT breakdowns.

DVA monitoring of the implementation of the recommendations in Audit Report No.44

DVA's National Audit and Fraud Committee (NAFCOM) was prematurely advised that it was not necessary for it to continue to monitor the implementation of the recommendations in Audit Report No.44. As a consequence, the implementation of the recommendations in this report was not scrutinised. As well, as DVA did not have an implementation plan for the recommendations, it was difficult for DVA to evaluate the actions to operationalise the recommendations.

Recommendations and DVA Response

The ANAO made one recommendation in this report. This was that, to promote the improvement of IT management and accountability, DVA identify performance information for inclusion in its IT Balanced Scorecard, including information on client service, which is linked to DVA's IT Strategic Plan. DVA agreed with the recommendation.

The Secretary of DVA's full response to the draft audit report was:

The ANAO report had one recommendation and several improvement suggestions. I have responded to the recommendation below and accept the principles of the improvement suggestions included in the report, which we will be taking up in further work.

I agree with the overall conclusion in the report that DVA has substantially implemented the recommendations of the initial ANAO report and consider the recommendation in the report as a sensible enhancement of the recommendation relating to the IT component of the Balanced Scorecard in the original report.

Footnotes

1 ANAO, Audit Report No.44 2000–01, Information Technology in the Department of Veterans' Affairs, June 2001.