Background

As a purchaser of publicly funded employment services, the Department of Employment and Workplace Relations (DEWR) has contracted Job Network Members (JNMs) and has an arrangement with Centrelink to deliver employment and referral related services. DEWR specifies, purchases and monitors these services through a contract with JNMs and a Business Partnership Arrangement with Centrelink. While DEWR does not provide services directly to job seekers, it is ultimately accountable for the quality of services provided by JNMs and it expects Centrelink to meet the requirements of the Business Partnership Arrangement. Centrelink, as an agency of the Human Services portfolio, is also accountable to its Minister and Parliament.

JNMs specialise in finding jobs for unemployed people, particularly those that are long-term unemployed. JNMs provide two major services to job seekers—Job Search Support and Intensive Support. The combination of these two services together with Mutual Obligation arrangements form a continuum of service that increases in intensity if the job seeker remains unemployed. Centrelink is the main ‘gateway' for people accessing Job Network services. Its gateway services include information provision, registration, assessment, and referral of job seekers to JNMs.

Statements by successive government Ministers have affirmed the Government's commitment to providing job seekers with a high quality of service. The objectives of the Job Network programme, reflect the key role of quality service, and are to:

• deliver a better quality of assistance to unemployed people, leading to better and more sustainable employment outcomes;
• target assistance to job seekers who need it and who can best benefit from it;
• address the structural weaknesses and inefficiencies inherent in previous arrangements for labour market assistance, and put into effect the lessons learnt from international and Australian experience; and
• achieve better value for money (especially in a tight budgetary environment).¹

Total Job Network programme payments to JNMs and Centrelink were $924.01 million in 2003–04 and are expected to be $1475.78 million for 2004–05.²

The Job Network programme is now in its seventh year and third phase. The current phase commenced on 1 July 2003 with the Employment Services Contract (2003–2006), which is referred to in this report as the third Employment Services Contract, and a new Business Partnership Arrangement with Centrelink. The current phase implemented an Active Participation Model (APM) of service provision. The APM simplified access to services for job seekers and streamlined services provided by Centrelink and JNMs as well as related service providers. The new service requirements of the APM and the third Employment Services Contract required major changes to DEWR's employment business systems.

There are many factors that influence the outcomes of publicly funded employment services. These include: macroeconomic conditions such as the state of the labour market and economy generally, changes in job seeker populations, and the mix of service providers, as well as compliance effects³ and the quality of services provided to job seekers. While all these factors influence employment outcomes, and need to be taken into account in determining employment policy,⁴ DEWR's contribution is the delivery of services to the unemployed to assist them into work or to provide assistance to prepare them for employment through the Department's policies and programmes—such as the mix of service providers, compliance settings and service requirements. During the course of the audit, DEWR advised that the APM was ‘achieving record vacancies, placements and long term (13 week) job outcomes, and doing so at a significantly reduced unit cost.' DEWR highlighted that the Job Network was ‘delivering record job results for highly disadvantaged job seekers, the long term unemployed and diverse groups including people with disabilities, parents and Indigenous Australians.'

Audit objective and scope

The objective of this audit was to assess whether DEWR's oversight of the Job Network ensures that job seekers are provided with high quality services. In particular, the ANAO examined whether DEWR had:

• an appropriate strategic approach to, and focus on, service quality across the Job Network;
• appropriate specification of the services to be provided to eligible job seekers, and of the quality of service provision;
• provided job seekers with a high quality of service at key Job Network service points; and
• appropriately monitored and reported the quality of service delivery, and appropriately managed service performance.

As well, the ANAO examined whether the Job Network has appropriate mechanisms for identifying, assessing and implementing improvements to service delivery.

This audit did not examine the implementation of the third Employment Services Contract and its associated computer application, Employment Assistant 3000 (EA3000), which was the focus of other ANAO audit activity. The audit of the implementation of the third Employment Services Contract will include an analysis of the performance information provided by DEWR on Job Network performance and the use of the ‘Star Ratings' system.⁵

Key findings

DEWR's corporate approach to Job Network services and service specification (Chapter 2)

Corporate approach to service delivery

A clear objective of the Job Network is to ‘deliver a better quality of assistance to unemployed people, leading to better and more sustainable employment outcomes'. The ANAO found that DEWR's business and risk plans demonstrate a corporate focus on the achievement of the goal of maximising job placements through the actions of the Job Network.

DEWR also views quality servicing as an integral feature of the Job Network. However, at the time of the audit, DEWR's strategic documents did not explicitly link the goal of achieving employment outcomes with high quality service delivery. During the audit, DEWR addressed this issue by documenting its role and contribution towards the achievement of high quality service for job seekers. The corporate statement on Job Network service quality has the potential to assist DEWR staff, service providers and other stakeholders to understand better DEWR's overall approach to service delivery for job seekers.

Specification of services and service entitlement

DEWR has appropriately specified the major service components and job seeker service entitlements in the third Employment Services Contract with JNMs and the Business Partnership Arrangement with Centrelink. The ANAO identified areas where the Contract and Arrangement could be improved and kept up-to-date. This includes the specification of new flexible servicing arrangements in the Business Partnership Arrangement, and services to be provided to job seekers in their second period of Intensive Support customised assistance (ISca) in the Contract.

Specification of service standards

DEWR has specified the standards of service that job seekers should receive in a number of documents, most particularly the Employment Services Code of Practice (the Code) and Job Network Service Guarantee (the Service Guarantee) that form part of the third Employment Services Contract.

The Code and Service Guarantee require that JNMs deliver a guaranteed set of services in accordance with specified principles and processes in a manner that is sensitive to the job seeker's culture, circumstances and background. While preparation of the Code and the Service Guarantee is a positive step, the ANAO found that the documents:

• contain service commitments that are largely subjective and DEWR has not prepared objective indicators and corresponding measurable performance standards against which DEWR and JNM staff and job seekers could form an assessment of the level of service performance;
• do not specify the expected manner of job seekers' behaviour in their interaction with JNMs; and
• do not clearly specify the key role played by JNMs in compliance aspects of the welfare system.

Job seeker awareness of the Code, Service Guarantee and associated complaints mechanisms remains very low. This means that job seekers may not take appropriate action in response to poor service and reduces the effectiveness of the standards as the means by which poor service is identified and the basis on which corrective action should be taken.

DEWR assesses, using client satisfaction survey research, whether its service providers are meeting most of the service commitments in the Code and Service Guarantee. However, the performance of DEWR's service providers in meeting around 10 per cent of the service commitments in the Code and Service Guarantee is not assessed by this client survey research. This is depriving DEWR of valuable management information.

DEWR's oversight of the performance of Job Network members (Chapter 3)

Performance information–the Quality Key Performance Indicator

DEWR has introduced a Quality Key Performance Indicator (Quality KPI) for the third Employment Services Contract. This is defined as ‘DEWR satisfaction with the delivery of services in compliance with the Code and the Service Guarantee.' The Quality KPI is a binary, pass/fail ‘hurdle', to be assessed by exception, whereby all JNMs receive a pass, unless a clear reason has been identified for applying a fail. DEWR expected that failure to meet the Quality KPI would occur in rare circumstances, such as a serious breach or systemic non-compliance with the contract, and/or repeated failure to address such issues where they have been raised by DEWR. This exceptions–based approach enables sanctions to be applied to a JNM that fails to reach the quality hurdle.

The ANAO found that assessment of the Quality KPI requires subjective judgements by DEWR contract managers, because most of the service commitments in the Code and Service Guarantee are not clear, measurable statements of service requirements. The ANAO acknowledges that any approach to performance assessment is likely to have subjective elements. However, the subjective elements can be minimised by preparing clear and measurable service commitments that aid the objective and consistent assessment of performance at any point in time, as well as trends in performance.

The current approach to assessment of performance does not enable DEWR to measure progress against the objective of the Quality KPI, to ‘maximise the delivery of high quality, ethical, employment services.' Assessment by exception does not allow DEWR to assess changes in performance over time or track progress against the overall objective of the Job Network to provide an ‘an even better quality of assistance'. This is because it does not provide a systematic basis for DEWR to consistently and objectively assess the quality of services delivered by its service providers. Rather, it enables sanctions to be applied to a JNM that fails to meet the quality hurdle, as assessed by contract managers and upheld by DEWR's internal review process.

Approach to identifying and managing key risks to service quality

DEWR has a sound overall approach to managing its Job Network contracts, incorporating a structured framework for identifying and managing key risks to quality service delivery. DEWR's contract managers are supported by performance reports and performance reporting functionality that enables them to monitor various service risks from the desktop, as well as checklists to support assessment of other risks during monitoring visits. DEWR has a clear capacity—through these monitoring mechanisms—to assess the contractual compliance of JNMs, and to initiate corrective action where contractual breaches are identified.

The ANAO found that DEWR's implementation of this overall approach has been adversely affected by a combination of supporting systems issues and high administrative workload associated with the introduction of the third Employment Services Contract. This has had two impacts.

First, the process DEWR developed to manage risks with Job Network contracts has not functioned as intended—risk assessments were not timely and lacked substantiation; there was limited weighting of monitoring effort toward higher risks; and risk assessments were not accurately updated to reflect issues identified in monitoring visits. DEWR's action to address problems with the risk management was timely, involving the re-development of supporting systems.

Secondly, the commencement of DEWR's programme of JNM site monitoring visits for the third Employment Services Contract was delayed until the latter stages of 2003–04, meaning that approximately only 30 per cent of sites received a monitoring visit over the financial year. There was no target for the number of monitoring visits to be undertaken.

DEWR senior managers advised the ANAO that the delayed schedule of monitoring visits was part of a consciously planned approach to the introduction of the third Employment Services Contract, but were unable to provide evidence of this planned approach.

The ANAO found that the delayed commencement of monitoring visits was not conducive to effective monitoring and review, as errors identified during these monitoring visits could have been addressed earlier in the contract period. The impact of this was ameliorated, to some degree, by monitoring visits completed in the last quarter of 2003–04. Greater assurance as to the investment of monitoring effort would be gained from the timely completion of risk assessments and the introduction of targets for the number of site monitoring visits required in relation to assessments of risk.

The ANAO found (through examination of documentation for monitoring visits completed in the latter stages of 2003–04)⁶ that early monitoring visits did not pay sufficient attention to key aspects of client service identified by DEWR as corporate priorities. These include: complaints handling practices; staff skills; continuous improvement practices; and the standard of facilities at new sites.

DEWR's oversight of the performance of Centrelink (Chapter 4)

Under the DEWR–Centrelink Business Partnership Arrangement, DEWR relies on monitoring of agreed performance information, Centrelink's internal monitoring of service delivery, and consultation and reporting mechanisms, in order to gain assurance that services are being delivered as specified, and to the agreed standard.

hile DEWR and Centrelink had indicative internal data to assess limited aspects of Centrelink's service performance, the ANAO found that the establishment of agreed management information as envisaged under the Business Partnership Arrangement was slow. DEWR advised that:

the establishment of agreed management information as envisaged under the Business Partnership Arrangement was delayed by the need to develop new reporting frameworks which in many cases required data which had not been previously available as well as the need to resolve issues relating to data ownership (these have since been resolved as a result of the Machinery of Government changes).⁷

The ANAO found that more than a year into the three-year Business Partnership Arrangement, measures for the overwhelming majority of performance indicators, reciprocal requirements and business processes identified in the Arrangement had not been developed. Reasonable interim indicators for two of the five performance indicators for the high–level KPIs measures were agreed, but the remaining three performance indicators were inadequate.

The ANAO also found that Centrelink's internal monitoring was not effective in providing assurance to DEWR about its performance in delivering most of the services specified in the Business Partnership Arrangement on DEWR's behalf.

The ANAO found that DEWR and Centrelink have established consultation and reporting mechanisms, and (over the course of the audit) a risk–based approach to managing the key business risks associated with the delivery of the services under the Business Partnership Arrangement. These frameworks—when supported by complete management information, and other necessary assurances—should provide a sound basis for monitoring and managing the services specified in the Arrangement.

Management of complaints (Chapter 5)

Complaints from job seekers about Job Network services are handled by a number of different organisations, including the Ombudsman, DEWR, Centrelink and JNMs, each of which have their own internal processes for recording and responding to complaints.

DEWR maintains a database on which it records complaints made to the DEWR Customer Service Line. However, the ANAO found that DEWR does not collect data about complaints received by its service providers, JNMs and Centrelink. This weakens the accountability of JNMs and Centrelink to DEWR for the quality of their complaints handling practices, and also means that DEWR does not have comprehensive information about complaints from across the Job Network (including Centrelink) that it can use for continuous improvement purposes.

The ANAO found that there is considerable scope for DEWR to improve the quality of complaints information that is recorded in the various complaints recording systems. First, the ANAO identified problems with DEWR's recording of complaints through its complaints database, including widespread inconsistencies in entry of data and weaknesses in the system for classifying the causes of job seeker complaints that limit the usefulness of the management information on complaints that can be extracted. Secondly, the recording of complaints by JNMs in the ANAO's sample⁸ was poor, which weakens DEWR's capacity to assure itself through review of complaint records kept by JNMs, that they are appropriately handling job seeker complaints.

The ANAO also found that the DEWR monitoring visits reviewed by the ANAO⁹ failed to identify the substantial shortcomings with the recording of complaints by JNMs. Not all DEWR monitoring visits checked the complaints handling practices of JNMs, and those checks that were done were generally not sufficiently detailed to determine whether the site was complying with the complaint handling requirements of the third Employment Services Contract.

Continuous improvements to Job Network service delivery (Chapter 6)

Approach to continuous improvement

DEWR advised the ANAO that it considers that its role in managing the purchaser–provider arrangements for delivery of services to job seekers includes facilitating continuous improvement. DEWR also advised that ‘the JNM performance management framework that rewarded JNMs which most successfully achieved long term jobs for their job seekers is an engine for continuous improvement.'¹⁰ However, DEWR has not informed its staff or service providers of their respective roles in facilitating continuous improvement; nor articulated to its staff or service providers how it will ensure that its service providers appropriately pursue continuous improvements.

DEWR identified continuous improvement by JNMs as one of four components of service quality in the third Employment Services Contract Request for Tender, and made continuous improvement by JNMs a service commitment in the Employment Services Code of Practice. However, the ANAO found that DEWR's risk assessment and assurance processes provide it with little assurance that Job Network service providers are appropriately and consistently pursuing continuous improvement in service delivery.

An ANAO survey of Centrelink and JNM staff found that both Centrelink and JNMs have a strong client service focus and that their agencies perform well in direct interaction with job seekers. However, staff regard their agency's processes to continuously improve their servicing as underdeveloped when compared to other aspects of their servicing practices, and specifically
identified areas of internal management practice where performance should be improved if the overall level of service quality is to be enhanced.

Involving staff and stakeholders

A range of formal and informal mechanisms have been established by which information relating to Job Network service delivery improvement is shared within and between DEWR, Centrelink and JNMs. The ANAO found that these mechanisms should provide a sound basis for communications that contribute to continuous improvement in Job Network service provision.

Conduct and use of client research and feedback

DEWR conducts high–quality, systematic client research. In particular, the Job Seeker Omnibus Survey contains a wealth of valuable data for continuous improvement purposes in Job Network services. The survey data is used by DEWR for policy development and reporting purposes. High-level results are shared with Centrelink.

JNMs are aware of DEWR's survey research and JNM staff consider their agency's collection and use of feedback from job seekers as an area of major weakness in relation to continuous improvement. However, the ANAO found that DEWR's survey data that could address these shortcomings, is not provided to JNMs in a level of detail and format that would support their continuous improvement activities.

Job seeker experience at key service points (Chapter 7)

Accessing Job Network services

Centrelink plays a key role in facilitating the choices of job seekers by providing information to job seekers about Job Network services. It does this primarily through the information seminar it delivers for job seekers, but also by providing information about Job Network services in information displays at Customer Service Centres (CSCs) and at new claim interviews. Responsibility for the quality of information is specified in the DEWR–Centrelink Business Partnership Arrangement.

Job seekers require high quality, up-to-date, accurate and relevant information so that they can make an informed choice of JNM and comply with their obligations as income support recipients. The ANAO assessed the quality of the information provided to job seekers by Centrelink at four CSCs, including in information seminars and information displays. This indicative assessment highlighted problems in the quality of the information that is provided to job seekers to help them make an informed choice of JNM. The provision of information and information products at information seminars and in information display areas was variable, often poor, and did not meet minimum requirements specified in the Business Partnership Arrangement. Many information seminars were not conducted prior to the job seeker making a choice of their JNM, and some job seekers did not attend a seminar at all.

The ANAO has made suggestions to improve the consistency, quality, and accuracy of the information Centrelink provides to job seekers about Job Network services including mechanisms for directly monitoring the quality of information provided to job seekers through information seminars and information displays. This would help job seekers to make an informed choice of JNM, and comply with their obligations as income support recipients.

Intensive Support customised assistance (ISca)

ISca is a major phase of the Active Participation Model service continuum of Job Network services. It aims to provide intensive and personalised assistance tailored to the employment needs and available job opportunities for the most disadvantaged job seekers. DEWR research indicates that job seekers in ISca are generally more satisfied than job seekers in other parts of the service continuum.

The third Employment Services Contract stipulates that the JNM will contact and meet face-to-face with each job seeker once every fortnight during their first period of customised assistance. A total of 12 service contacts are required in the course of this six-month timeframe and these must be recorded in DEWR's information systems.

To determine if ISca services are intensive and personalised assistance, the ANAO assessed the services received by 23 job seekers at 12 JNMs in four States. This indicative assessment identified a number of problems, including:

• levels of contact between JNMs and job seekers rarely met contractual specifications, for a range of reasons. The level of contact and associated payment arrangements needs to be clarified;
• documentation of JNM assessment of job seekers' barriers to employment, a contractual requirement, was limited. Better collection of information for each job seeker could improve both their chances of achieving successful outcomes and DEWR's overall outcomes; and
• customisation of job seekers' Job Search Plans through the course of their time in ISca was very limited.

While the ANAO sample used to underpin these findings was indicative only, the nature and level of problems identified raises concerns about whether assistance is actually intensive and personalised.

Overall conclusion

The Job Network programme provides employment services to around 950 000 clients that register at Centrelink as job seekers per annum. To meet the objectives of the programme, Job Network providers need to have flexibility to tailor their services to the needs of individual job seekers. In order to manage a programme of this kind, DEWR requires a sound management framework, including effective performance management arrangements with its service providers, clear specification of services and service standards and reliable information on which to make management decisions on programme performance and development.

The ANAO concluded that DEWR, as the purchaser of Job Network programme services, required additional assurance that job seekers were being provided with key aspects of employment services as intended by the department.

A critical issue for DEWR in administering the delivery of Job Network programme services is striking an appropriate balance between the degree of purchaser oversight and the operational flexibility afforded to JNMs. While DEWR's overall approach has been not to interfere with internal decisions of JNMs in placing job seekers in jobs, it has put in place a ‘quality hurdle'. This is an assessment of JNMs against standards set out in DEWR's Employment Services Code of Practice and the Job Network Service Guarantee, as part of a commitment to continuous improvement in delivering a personalised and individualised service to job seekers and employers.

However, DEWR's ability to gain assurance that job seekers receive high quality services from JNMs is limited by the lack of objective and measurable performance indicators relating to DEWR's specified service standards. DEWR's capacity in this area has been further hampered by delayed commencement of the monitoring of Job Network contracts over 2003–04.

DEWR and Centrelink have agreed to a number of service standards for the services Centrelink delivers to job seekers on DEWR's behalf, namely: accurate and timely assessment of jobseekers; rapid referral of job seekers to Job Network services; and rapid reconnection of job seekers to Job Network services. Delays in the development of the DEWR–Centrelink 2003–06 Business Partnership Arrangement meant that at the time of this audit, DEWR had no management information to assess Centrelink's performance against the agreed standards.

While individual complaints from job seekers are being handled by DEWR, JNMs and Centrelink, substantial weaknesses in the collection and recording of complaints data means that this otherwise valuable source of information for the performance management of service providers and broader monitoring of complaint trends and continuous improvement purposes is compromised.

A key service to be delivered through the Job Network is ISca, which is intended to provide intensive and personalised assistance to the most disadvantaged job seekers, including the long-term unemployed. While job seekers in ISca are generally more satisfied than job seekers in other parts of the Job Network service continuum, the ANAO's assessment of the services received by an indicative sample of ISca job seekers raised concerns about whether the customised and individualised contacts and services required in the third Employment Services Contract were provided.

The Government's changes to administrative responsibilities introduced in October 2004, and DEWR's preparation for the next round of Employment Services Contracts, provide the opportunity for DEWR to draw on its experience and the matters raised in this audit to improve its assurance about the quality of services being provided to job seekers.

Recommendations

The ANAO made eight recommendations aimed at improving DEWR's assurance that job seekers are provided with high quality employment services as intended by the department. DEWR agreed, either in full or in part to all eight recommendations. Centrelink agreed to the two recommendations specifically directed to its area of responsibility.

Agencies' responses

DEWR

DEWR full response to the audit report is reproduced at Appendix 7. DEWR's summary response was:

The latest Job Network contract introduced a new Active Participation Model which is achieving record vacancies, placements and long term (13 week) job outcomes, and doing so at a significantly reduced unit cost. Job Network is delivering record job results for highly disadvantaged job seekers, the long term unemployed and diverse groups including people with disabilities, single parents and Indigenous Australians.
Job Network's achievements in the 12 months to end March 2005 are new annual records: around 963,400 New Vacancies were lodged on the national database—an increase of 26% on the previous year; a total of around 651,100 job placements were recorded—an increase of 44% on the previous 12 months; and over 176,400 long term (13 week) jobs were achieved for disadvantaged job seekers and those unemployed more than three months—more than double the previous year.

Moreover, quality servicing is inbuilt into Job Network contracts and its performance management regime. This includes, for example, the introduction, for the first time, of a contracted Job Network Service Guarantee for job seekers. Job Network providers must also conform with a Quality Performance Indicator or face contract penalties or termination of services. A national contract and performance management framework ensures these requirements are monitored and applied.

Centrelink

Centrelink advised the ANAO that it welcomed the audit report and agreed with the two particular recommendations specifically related to Centrelink's responsibilities. Centrelink also advised that while three other recommendations are directed at DEWR, each contain reference to the DEWR/Centrelink relationship and that it therefore considered it appropriate to make comment on these recommendations. No additional comments were provided for attachment to the report.

Footnotes

1 DEWR, 2002, Job Network Evaluation Stage 3, p.12.

2 Combined actual 2003–04 payments and estimated 2004–05 payments to JNMs and Centrelink.

3 A compliance effect arises when requirements to participate in a labour market programme cause some job seekers to increase their job search efforts, either because they become more motivated as a result of the process, or to avoid taking part in the programme. Productivity Commission, 2002, Independent Review of the Job Network, section 5.16.

4 DEWR advised the ANAO that ‘it is very difficult to determine quantitatively the effect of Departmental programmes (including Job Network) on the average duration of unemployment and the rate and incidence of long-term unemployment—most of the explanatory variables…are largely outside the Department's influence.' Email from DEWR to ANAO 5 July 2004.

5 Star Ratings are DEWR's system of setting a comparative order of merit among Job Network providers, reflecting its assessment of their performance. DEWR uses star ratings as an incentive to improve provider performance through competitive pressure.

 6 The ANAO reviewed contract management documentation relating to a sample of 226 Job Network sites across 15 organisations, including the documentation relating to the 65 sites in the sample that had received a monitoring visit. This documentation was compiled by DEWR in the process of undertaking a quality assurance review of contract management processes over April to June 2004.

7 DEWR advice 27 April 2005.

8 The ANAO reviewed the complaints registers for 14 JNMs visited during audit fieldwork. Notwithstanding contractual requirements, the ANAO found that none of the 14 JNMs visited adequately recorded all complaints in their registers.

9 The ANAO examined documentation relating to monitoring visits to 11 of the 14 sites visited by the ANAO during audit fieldwork. For three sites, DEWR monitoring visits had not been conducted at the time of the audit.

10 DEWR advice 27 April 2005.