e-Business

The Australia Government is committed to improving the delivery of information and services to the Australian community, particularly through the use of information technology (IT) including the Internet. In 1997, the Government set a policy to make ‘all appropriate services available online via the Internet by 2001'.¹ Over the past several years, the Government and its lead information management agency have reiterated this commitment to e-Business; that is, conducting government service delivery online through the Internet.

The Government through the Department of Education, Employment and Workplace Relations (DEEWR) provides advice and administers programs relating to education, training and employment. Most of these programs are supported by IT applications, with several of the applications accessible to the department's clients and contractors through the Internet.

ANAO's previous audit on managing e-business

In March 2003, the ANAO tabled a performance audit of the former Department of Education, Science and Training's (DEST's) management of its Internet presence and its e-Business.2 The objective of the 2003 audit was to determine whether DEST:

•  had effective governance practices for its IT and e-Business;
•  had adequate systems in place to measure the efficiency and effectiveness of its IT and e-Business;
• implemented and maintained appropriate quality standards within its IT and e-Business systems; and
• implemented proper controls, including risk management, to achieve maximum benefit from its IT and e-Business.

The ANAO concluded at the time that DEST's management of its Internet presence and its

e-Business was sound. Where improvements could be made, such as in measuring the benefits of, and in assuring the quality of, its e-Business systems, DEST was addressing these aspects with the introduction of a new systems development methodology.

The ANAO made six recommendations concerning improvements to the department's IT governance, recordkeeping, performance monitoring and to its reviews of IT including Internet services. The department agreed with all six recommendations.

Audit objective, scope and methodology

The objective of this follow-up audit is to examine DEEWR's implementation of the six recommendations made in the ANAO's 2003 report. This audit has had regard to the issues underlying the recommendations, and new administrative issues affecting their implementation.

The Government established DEEWR in December 2007, bringing together the education and training functions from the former DEST, with the functions of the former Department of Employment and Workplace Relations (DEWR) and the child care functions from the Department of Families, Community Services and Indigenous Affairs (FaCSIA). The audit, which was originally designated in the former DEST, did not consider e-Business functions which transferred from, or to, the portfolio. To assist readers, the term DEEWR has been used throughout the audit to identify the department.

The audit examined DEEWR's actions and outcomes in addressing recommendations in the following areas:

• improvements to IT governance through clarifying roles and responsibilities of its IT committees;
• accuracy of records held in the Training and Youth Internet Management System (TYIMS) relating to those apprenticeships that would have been expected to be finalised but not recorded as such;
• implementation of a Memorandum of Understanding (MOU) between DEEWR and the Department of Immigration and Citizenship (DIAC), for the transfer of data affecting DEEWR's Provider Registration and International Students Management System (PRISMS). The audit also engaged DIAC on this issue to determine its perspective;
• identification of measures and targets to assess the success or otherwise of e-Business projects;
• conduct of post-implementation reviews on e-Business projects; and
• implementation of a single computer system to record, authorise and track application changes.

Audit conclusion

DEEWR has actioned the recommendations from the previous audit. Three have been implemented, with a further recommendation in the process of being implemented. Two recommendations are partially implemented. Set out in the Key findings are the original recommendations as agreed by DEEWR, and a summary of ANAO's assessment of progress against each.

DEEWR has continued to enhance its management of e-Business since the previous audit. In particular:

• IT governance has improved with IT committees operating within clearly defined roles and responsibilities;
• the accuracy of TYIMS records has improved, reducing the number of records where there is uncertainty about whether apprentices had completed their apprenticeships;
• DEEWR is in the process of rolling out a single computer system that will record, authorise and track application changes;
• DEEWR and DIAC have an agreement through an MOU that outlines the respective roles and responsibilities for the transfer of data on the visa and enrolment status of international students; and
• DEEWR has improved its guidance and tools for IT project managers.

The ANAO considers that there are opportunities to improve the governance of the MOU between DEEWR and DIAC for timely resolution of IT issues that have arisen in DIAC systems, affecting the accuracy and completeness of PRISMS, a database that supports education providers for international students.

The ANAO also considers that there is scope for improving the evaluation of e-Business projects. In particular, better governance, guidance and tools are required. Such evaluations assist in assessing the benefits achieved from investment in such projects, as well as supporting continuous improvement in project management.

The ANAO made two recommendations to improve the transfer of data relating to international students, and to improve the evaluation of projects.

Key findings

IT Governance (Chapter 2)

Original recommendation 1: The ANAO recommends that DEEWR clarify the respective roles and responsibilities of its Corporate Information Technology Committee and its Information and Business Technology Committee. In doing so, DEEWR should consider the development and use of committee charters that include the authority and role of the committee, the roles of the chair, members and secretariat, and a work program together with a regular review of the progress and achievements of the committee.

Finding

DEEWR has implemented this recommendation. The audit identified that:

• the respective and separate roles and responsibilities for the corporate IT committees were clarified in their charters;
• the charters identified the authority of each committee and the role of their members;
• both the corporate IT committees operated in line with their respective charters;
• there was an annual work program for the committees; and
• the two IT committees regularly reviewed the progress of work in their own annual programs.

Implementation of this recommendation has resulted in both committees operating with clear lines of authority and without unnecessary duplication of effort, leading to improved efficiency in their work.

IT Applications – TYIMS and PRISMS (Chapter 3)

TYIMS – Exception Reporting

Original recommendation 2: The ANAO recommends that DEEWR upgrade the Training and Youth Internet Management System (TYIMS) to produce an exception report that identifies apprenticeship records that would have been expected to be finalised but are not, and seek assistance from the appropriate New Apprenticeship Centres (NACs) to finalise the records.

Finding

DEEWR has implemented this recommendation, completing the actions outlined in their response to the recommendation. The department has made enhancements to TYIMS, and to its contracts with the Australian Apprenticeship Centres (AACs), the successors to the NACs. Although the number of doubtful incomplete cases decreased by two-thirds, there remains a number of doubtful incomplete apprenticeship records. The exception reports developed in response to the recommendation do not highlight incomplete records that have remained on the system since the implementation of those reports. Clarifying the status of these doubtful incomplete cases would enhance the certainty of DEEWR's performance reporting and the amount that DEEWR owes NACs for final payments.

PRISMS – MOU with DIAC

Original recommendation 3: The ANAO recommends that DEEWR initiate discussion with DIAC with a view to developing a Memorandum of Understanding to clearly define each agency's responsibility for the timeliness, accuracy and completeness of the data transfer between the agencies, any requirements for reconciliation of data between the agencies, and each agency's responsibility for timeliness of system changes. In doing so, the ANAO recommends that DEEWR seek to resolve the discrepancy in the Provider Registration and International Students Management System (PRISMS) error rates arising from incomplete transfer of data between DEEWR and DIAC.

Finding

DEEWR has implemented this recommendation. DEEWR has made substantial efforts to address the accuracy and completeness of international student records resulting from data transfers between DEEWR and DIAC. In particular, the departments implemented an agreement on data transfer, consistent with the original recommendation, documented in an MOU. In addition, DEEWR made changes to its international student database, PRISMS, to minimise errors and initiated comparisons of the data held by DEEWR and by DIAC.

The accuracy and completeness of data on PRISMS is compromised by data transfer issues between DEEWR and DIAC. The current MOU governance arrangements are not operating in line with the agreement. Given its competing IT funding priorities, DIAC has not undertaken all the IT changes necessary to address accuracy and completeness of data transfer to DEEWR, although it has made a number of system changes over the period since the last audit. Instead, DIAC currently places a high reliance on manual processes and DEEWR's technical resources to address its data transfer issues.

The interim solutions implemented by DIAC to support the accuracy and completeness of the data transferred to DEEWR have adverse impacts on the data integrity of PRISMS. To date, the efforts of officers in both departments have minimised the short-comings of the PRISMS data for end-users, namely, education providers to international students. The education providers' current level of satisfaction with PRISMS could change, if they do not consider that they are receiving value for money from their efforts to provide DEEWR with accurate PRISMS information.

Measuring and Assessing the Effectiveness of e-Business Projects (Chapter 4)

Project measures and targets in business cases for e-Business proposals

Original recommendation 4: The ANAO recommends that DEEWR ensure that any business case for e-Business proposals align with the guidelines provided in its new Systems Development Methodology that requires inclusion of measures and targets by which the success or otherwise of the project can be assessed.

Post implementation reviews – risk-based approach

Original recommendation 6: The ANAO recommends that DEEWR consider a two-tier approach to post implementation reviews based on the level of risk inherent in the change. After a periodic release, DEEWR could conduct a simple and brief self-assessment by the change team and business owners of the management, and effect, of the change (what we did right, what we did wrong, what can we improve). After major enhancements or changes it would be appropriate for a post-implementation review to include an assessment of the costs and benefits, and to include surveys of internal and external clients on the effect of the change.

Finding

DEEWR has partially implemented these recommendations. DEEWR made substantial progress in developing a project management methodology to guide its IT developments, consistent with established, widely used approaches to IT governance, project management and software development. As part of this methodology it has guidance and procedures for measuring IT and e-Business project effectiveness, as well as reviews of IT projects. However, DEEWR is inconsistent in its application of its guidance on measuring project effectiveness, while its adherence to its guidance on project evaluation was limited.

While DEEWR has completed the actions that it agreed to against the original Recommendations 4 and 6, it has not applied the guidance in the way that addressed the original intent of the recommendations.

Managing e-Business System Changes (Chapter 5)

Original recommendation 5: The ANAO recommends that DEEWR record all application change requests and faults on a single computer-based system. The system should also track application changes through the change process and include authorisations.

Finding

DEEWR is in the process of introducing a single computer-based system which, based on its design, is expected to address the requirements of this recommendation.

In the interim, DEEWR implemented processes to manage IT application changes, which address many, but not all, of the underlying issues that led to this recommendation.

DEEWR's response

DEEWR accepts the findings presented in this report. The department notes the original audit findings that the department's management of its Internet presence and its e-Business was sound, and we are pleased to note that actions agreed to further improve this management have been found to be largely completed.

We accept the need for continued efforts to improve management of these critical assets and welcome the recommendations provided in the report.

DEEWR agreed to the two follow-up recommendations.

Diac's response

The report identifies issues relating to data transfer between DIAC systems and PRISMS.

DIAC's student visa program supported growth in the international education industry by delivering a record 228 000 visas in 2006–07. The data transfer between DIAC systems and PRISMS plays an important part in managing the student visa program. While there are currently high levels of satisfaction with PRISMS from end users, DIAC agrees that ongoing work is required to support the data transfer.

DIAC agreed to the follow-up recommendation related to the transfer of data on international students between the two departments.

Footnotes

^{1 Commonwealth of Australia, Investing for Growth. The Howard Government's Plan for Australian Industry, 1997.}

^{2 ANAO Audit Report No.33 2002–03. Management of e-Business in the Department of Science, Education and Training.}