Introduction

1.1 Australia’s Humanitarian Program comprises up to 18,750 places in 2019–20.² The Humanitarian Settlement Program (HSP or the program) provides support to the majority of the clients in the Humanitarian Program as shown below.³ Since the HSP commenced in October 2017 the largest numbers of clients have come from Iraq, Syria, Myanmar, the Democratic Republic of the Congo and Afghanistan.⁴

Figure 1.1: Onshore and offshore visas granted and HSP client numbers^a 2010–11 to 2018–19

Note a: This figures includes humanitarian entrant numbers in years prior to the commencement of the HSP in 2017. Actual HSP clients for any year may not equal visas granted in that year due to the continuation of clients from the previous year.

Note: The Department of Home Affairs (Home Affairs) manages the client intake, which has two key components:

1. offshore resettlement: offered to people overseas who are in the greatest need of resettlement; and
2. onshore protection: offered to people already in Australia who are found to meet Australia’s protection obligations.

Note: Most off-shore humanitarian entrants are automatically eligible to access supports from the HSP. Onshore, high-needs humanitarian entrants can access the HSP subject to an assessment.

Source: ANAO analysis of Home Affairs reports.

The Humanitarian Settlement Program

1.2 Until October 2017, humanitarian entrants to Australia were supported through the Humanitarian Settlement Services (HSS) and Complex Case Support (CCS) programs which operated from 2011 and 2008 respectively. A 2015 evaluation proposed merging the HSS and CCS programs into a single program to deliver administrative efficiencies.⁵ The Settlement Services Advisory Council also made recommendations to improve the English language, education and employment outcomes of the programs.

1.3 The HSP commenced in October 2017, replacing the HSS and CCS. The objective of the HSP is to:

Build skills and knowledge for social and economic well-being of eligible clients through a tailored, needs-based, case management approach.⁶

1.4 The Department of Social Services (DSS) had responsibility for the HSP until 30 June 2019. Under the Administrative Arrangements Order of 29 May 2019 settlement programs, including the HSP, moved to Home Affairs from 1 July 2019.⁷ In this report:

• if ‘DSS’ is used, the described activity was conducted prior to 1 July 2019; and
• if ‘Home Affairs’ is used, the described activity was conducted after 1 July 2019.

1.5 HSP services are intended to be needs-based and there are three service tiers. The highest need clients are in tier three: Specialised and Intensive Services (SIS).⁸ HSP services are provided to clients until they achieve the settlement outcomes, decide to stop receiving services, or until Home Affairs decides the services should cease. It is expected that most tier one and tier two clients will be provided HSP services for up to 18 months. The duration of delivery of SIS to tier three clients will vary according to the client’s needs.

Eligibility

1.6 Figure 1.2 shows the visa classes that are eligible for the HSP. The offshore Humanitarian Program has two components. Entrants who arrive in Australia under the refugee component (Visa Subclasses 200, 201, 203 and 204) are eligible for the HSP. Entrants who arrive under the Special Humanitarian Program component (Subclass 202) are also eligible for the HSP.

1.7 There are different rules for the Community Support Program (CSP) component of offshore visas. The CSP allows individuals, communities and businesses to support people in humanitarian need through the visa application and settlement processes. People granted a Subclass 202 visa under the CSP are not generally eligible for the HSP and receive settlement services from their Australian supporters coordinated through their Approved Proposing Organisations. However, subject to departmental approval, CSP entrants with complex needs are eligible for HSP SIS (tier three).

1.8 The onshore component of the Humanitarian Program is for non-citizens who are in Australia (other than illegal maritime arrivals), and meet the criteria for the Onshore Protection visa. Some Onshore Protection visa holders who have complex needs are eligible for SIS.

Figure 1.2: HSP eligibility by visa type (visas granted in 2018–19)^a

Note a: DSS HSP Fact Sheet, August 2018 [Internet] available from: https://immi.homeaffairs.gov.au/settlement-services-subsite/files/hsp-factsheet.pdf, [accessed 29 November 2019].

Source: ANAO and HSP documentation. Refugee visas are visa sub-classes 200, 201, 203 and 204.

Current HSP providers and budgeted expenditure

1.9 In July 2017 DSS signed contracts with five service providers for 11 regions across Australia to deliver settlement services to HSP clients (see Table 1.1).⁹

Table 1.1: HSP Service Providers

Note a: On 11 August 2017 Armidale was announced as a new settlement location for humanitarian entrants. SSI was given a one-off payment for establishing the services in Armidale.

Note b: Formerly MDA Ltd.

Source: Humanitarian Settlement Program Fact Sheet, Department of Home Affairs, [Internet], available at: https://immi.homeaffairs.gov.au/settlement-services-subsite/files/hsp-f… [accessed 23 November 2019].

1.10 The budget allocation for the HSP is outlined in Table 1.2.

Table 1.2: Humanitarian Settlement Program budget

Source: ANAO summary of DSS documents.

Rationale for undertaking the audit

1.11 The HSP provides initial settlement support to people who have recently arrived in Australia on refugee or humanitarian visas and to some eligible onshore people. The objective of the HSP is to build people’s skills and knowledge for social and economic well-being through a tailored, needs-based, case management approach. Over 17,000 clients are eligible for the HSP each year and the program is allocated over $120 million annually.¹⁰ ANAO has not previously audited the HSP or the prior versions of this program but audited settlement services grants in 2009.¹¹ This audit assessed whether DSS and Home Affairs have managed the HSP effectively following the commencement of the HSP in October 2017.

Audit approach

Audit objective, criteria and scope

1.12 The objective of this audit was to examine whether the HSP is being delivered effectively. To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria:

• Are the HSP contracts well designed and managed?
• Is the performance of the HSP effectively managed to achieve program outcomes?

Audit methodology

1.13 The audit methodology included:

• examining program documentation including contracts, program guidance, performance information, risk management documentation, program reviews and reporting information;
• examining HSP IT system documentation and performance;
• interviewing relevant departmental officers, including staff in regional offices;
• interviewing representatives of the five HSP providers; and
• interviewing representatives from the Settlement Council of Australia.

1.14 The audit was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $418,080. The team members for this audit were Katherine Lawrence-Haynes, Deanne Allan, Joel Smith, Hugh Balgarnie, Jocelyn Watts, Samuel Painting and David Brunoro.

2.1 In order to assess the design and management of the HSP contracts, the ANAO examined whether the HSP:

• contracts align with program outcomes and have appropriate provisions — because effective contract design should include a clear line of sight from relevant contract deliverables to program objectives, and contract provisions help protect Government interests;
• contracts are well managed — because robust contract governance, assurance activities, reporting and engagement with providers is needed to ensure a program is delivered effectively; and
• whether the HSP IT system supports contract management — because the IT system was purpose built, is used by both departmental staff and providers, and facilitates payments and data collection.

Do the contracts reflect the HSP outcomes and have appropriate provisions?

2.2 The Humanitarian Settlement Services (HSS) and Complex Case Support (CCS) programs transitioned to the Humanitarian Settlement Program (HSP) in October 2017. DSS intended to gain a number of benefits when moving to the new contract arrangements which included moving towards an outcomes focus. An assessment of whether these benefits have been realised is included in Chapter 3.

The Program Outcomes Framework and the services

2.3 The objective of the HSP is to ‘build skills and knowledge for social and economic well-being of eligible clients through a tailored needs-based case management approach’. DSS developed an HSP Outcomes Framework (see Table 2.1) which is reflected in the contracts (it is included as an appendix and it also aligns with the services in the contracts).

Table 2.1: The Program Outcomes Framework (excluding the innovation target outcomes)^a

Note a: The Program Outcomes Framework also included Innovation Target Outcomes. They had not been activated as at July 2019.

Source: ANAO Summary of DSS documents.

2.4 Consistent with the Program Outcomes Framework, the HSP deliverables in the contract are divided into ‘immediate deliverables’ (within three weeks of arrival to Australia) and ‘foundation services’ (delivered following the initial three weeks). A key feature of the foundation services is the delivery of an onshore orientation program to build clients’ knowledge about the Australian way of life. Services then focus on longer term needs including assistance to source long-term accommodation, access services and connect with the community and employment.

2.5 The HSP service providers employ case managers to assess the needs of HSP clients and services are provided in line with client needs, with reference to the client’s tier. Tier one clients generally need minimal assistance to settle into the community; tier two clients are likely to have experienced significant challenges and need more support; and tier three (Specialised and Intensive Services) clients face multiple and complex settlement barriers.

2.6 Program Outcomes Framework immediate deliverables are linked to specific provisions in the HSP contracts and to related payments in the payment schedule. An example is shown in Table 2.2.

Table 2.2: Links between the outcomes framework and contracts — example

Source: ANAO analysis of the HSP contract.

Common contract provisions

2.7 The Australian Government Contract Management Guide (July 2019) states that contracts should include 11 common contract provisions.¹² The HSP contracts contain the 11 common contract provisions which cover: access and records; insurance, liabilities, guarantees, indemnities and warranties; intellectual property rights; key personnel capabilities; and sub-contracting. There are also other standard provisions including ones relating to contract variation and termination, fraud, confidentiality and conflict of interest.

The performance regime and reporting

2.8 The HSP performance regime centres on an abatement regime and nine Key Performance Indicators which are discussed in the next Chapter.¹³ Service providers must submit an annual business plan and six-monthly reports for departmental approval (details in Table 2.4).

Payment provisions

2.9 The HSP Payment Schedule has two categories of payments:

• Monthly Operating Charge (MOC) — a fixed amount payable each month to cover direct and indirect costs to maintain the base infrastructure¹⁴; and
• Settlement Service Charge (SSC) — there are 71 SSCs (or payment points) which have prescribed prices for a completed service (for example, orientation services, supporting a client to manage health appointments, or reimbursement of a cost).

2.10 Many of the SSCs are client-specific (for example, completion of a case management plan) and others are for the client group (for example, provision of a mobile phone for a family). Some services are capped based on: a price cap linked to the number of clients in the group (for example, Basic Household Goods) or a cap on the number of times a service can be claimed before departmental approval is required (for example, health appointments).

2.11 The payment levels for most of the SSCs vary between providers, reflecting the outcome of the tendering process.¹⁵

Service-related payment points

2.12 A June 2016 DSS financial compliance review of the HSS recommended more payment points to increase transparency and minimise compliance risk. DSS agreed with this and the 35 payment points in the HSS contracts were increased to 71 SSCs in the HSP contracts.

2.13 DSS also stated to the ANAO that the additional payment points can better reflect the costs when a client moved between regions. Claim points were intended to align with outcomes and to provide data on the cost and timeliness of each component service.

2.14 As discussed in paragraphs 2.52–2.56, the large number of payment points and some caps on the number of times a service can be provided (without separate approval) were reviewed after service provider input. There is a recommendation in a draft departmental report reviewing the operation of the first 18 months of the HSP to increase the number of times some services can be claimed before the cap is reached.¹⁶ Home Affairs also plans to explore consolidation of payment points (refer to paragraph 3.46).

Pricing considerations

2.15 ANAO review of the tender and contract pricing considerations showed:

• DSS provided guidance to tenderers to help them establish their costings; and
• the contract pricing for some regions was below the DSS price assumption.

Guidance on tender pricing

2.16 DSS provided guidance to tenderers on pricing, including clarification of what the MOC was to cover in May 2017. The MOC was principally to capture costs that would enable the tenderer to maintain a minimum capacity given uncertain client numbers.¹⁷Figure 2.1 shows the variability in the arrivals for the HSP and prior programs. Despite the MOC, this variability was raised as an issue by one service provider (refer to paragraph 2.61).

Figure 2.1: The pattern of arrivals of humanitarian entrants by month

Source: ANAO analysis of DSS data.

2.17 DSS also provided guidance to tenderers on the other pricing components. A May 2017 Tenderer’s Pricing Response Schedule had instructions for tenderers to complete an embedded Pricing Schedule for all program costs.

2.18 Clarifications on pricing, including administration costs, were provided in three question and answer documents for tenderers issued in March and April 2017. For instance, one answer stated that the $150 allowance for a mobile phone ‘could be for the phone, SIM, credit, and any other associated costs’.

Overall contract pricing

2.19 To review tender pricing, DSS developed price assumptions for each contract region for all payment components. DSS ranked tenders according to seven evaluation criteria (capability, organisation capacity, service delivery capacity, Indigenous Participation Plan, pricing, financial viability / capacity and risk).

2.20 DSS compared prices in each tender against its price assumptions. The preferred tenderers then participated in contract negotiations in June 2017. DSS sought revised tender prices where tender prices were above its price assumptions. Over the five year life of the contract, there was a 24.6 per cent (average) reduction between the original tender pricing and the final contract pricing across all contract regions.

2.21 Prior to the contract negotiations, DSS noted that the pricing for one preferred tenderer (Service Provider K) was low and testing of its service delivery assumptions was required.¹⁸ Service Provider K provided an overview of its model for case management and orientation costs and pricing was adjusted as a result.

2.22 Although Service Provider K’s overall tender prices were below the DSS final price assumptions for each region it tendered for, prices in seven priority areas were higher than DSS expected. The outcome of the price negotiation was that the final contract price for each of Service Provider K’s regions was lower than its original tender price. Three other service providers had a region where the final contract price was below the DSS price assumption.

Are the contracts well managed?

2.23 To examine how well the HSP contracts were managed five areas were examined:

• program governance;
• contract management, including the abatement regime;
• assurance and compliance activities;
• service provider reporting; and
• engagement with service providers.¹⁹

Program governance

2.24 DSS’ program governance of the HSP has been appropriate. DSS had a range of HSP oversight and advisory committees for management of the HSP, including to manage the transition to the HSP.

2.25 After the HSP Transformation Board (comprising senior executive staff) ceased in June 2018, DSS planned that a HSP Board would commence in May 2019. With the transfer of the program to Home Affairs, this did not occur. Home Affairs stated that it is reviewing the governance structures for the HSP with a view of establishing a Settlement Board to look at settlement in a wider context than just the HSP.

2.26 There are two advisory groups for the HSP. The Senior Officials Settlement Outcomes Group, comprising senior settlement officials in all levels of government, meets bi-annually to consider settlement data and programs. It is now chaired by Home Affairs. There is also a Settlement Services Advisory Council appointed by the Minister. It will cease in late 2019, with a new advisory council to be agreed by the Prime Minister.

2.27 In DSS, HSP operations were delivered by the DSS Community Grants Hub regional network. Policy and other coordination activities were managed by the national office. In Home Affairs, the HSP contracts as well as associated policy and coordination activities are managed by the National Office.

HSP contract management

Contract management plan

2.28 DSS’ Contract Management Guide 2017 lists contract start-up activities and states that the development of a contract management plan would ‘usually be required’. The July 2019 Australian Government Contract Management Guide (released post commencement of the HSP) requires contract management plans for programs like the HSP.²⁰

2.29 DSS did not use a contract management plan to manage the HSP contracts.²¹ As evidence of contract management, DSS provided a PDM ‘Manage Function’ diagram for the Community Grants Hub (responsible for the HSP). This has reference to management functions including planning and assurance. It notes roles of the Service Assurance Centre for Excellence, including HSP service cap increases.

2.30 The detail on how the HSP was managed in practice is not evident from this diagram, including how claims for payment were assessed. Home Affairs should adopt the Australian Government Contract Management Guide in managing the HSP.

The abatement regime

2.33 The HSP contracts contain an abatement regime where service failures incur ‘failure points’ which can lead to a reduction in the Monthly Operating Charge.²² The contracts state that the abatement regime would be inactive for the first 12 months of the HSP, from October 2017 to October 2018.

2.34 The abatement regime has not been activated. Home Affairs stated that it will review the abatement regime during the KPI transition period up to the end of 2019 (see Chapter 3, paragraph 3.29).

HSP expenditure

2.35 ANAO analysed data in the HSP system on expenditure since the program commenced in October 2017 to 30 June 2019.

2.36 The ANAO analysis shows that expenditure under all contracts in 2017–18 was less than anticipated at the start of the contracts (due to the fact that the contracts only ran for eight months in that year and the budget was for a full year). In 2018–19 overall HSP expenditure was 1.8 per cent below the cost estimated for that year at program commencement.

2.37 Over the life of the HSP the largest expenditure line item is the Basic Household Goods packages (further discussed at paragraph 3.55). Development of case management plans, assisting with health appointments and accommodation costs are the next highest categories.

2.38 The analysis also looked at average payments per client per tier (excluding payments made to a family group such as the Basic Household Goods package). The greatest overall expenditure is for tier two clients (93.5 per cent of clients) but the highest average expenditure per person is for tier three clients who have the highest need levels. The analysis showed that the average spend per client was around $1,300 for tier one, $2,000 for tier two and $3,000 for tier three.

Assurance and compliance activities

2.39 Compliance and assurance activities for the HSP have been undertaken in three phases²³:

• transition-in monitoring (October 2017 — June 2018);
• interim monitoring (from July 2018); and
• a new Assurance and Compliance Strategy (planned to commence at the start of 2020).

Transition-in monitoring

2.40 The transition-in monitoring consisted of case management plan reviews, client interviews, and observations, including client arrivals to verify the required on-arrival processes. A total of 88 transition-in monitoring activities were undertaken, for the 7,579 clients who transitioned in to the HSP and arrived in this period. This is on the low side for a sample size.²⁴ Client interviews have not been conducted since July 2018.

Interim monitoring

2.41 The interim monitoring involved a review of 195 client case management plans and invoices (1.7 per cent of the 11,317 HSP clients who arrived in this period). DSS based the sample size on the number of HSP clients and the assessed risk level in each region (not statistical significance). A January 2018 internal report recommended statistically valid sampling with around 1000 reviews per year for a confidence level of 90 per cent (around five times the number of reviews that were conducted).

2.42 Table 2.3 shows the average result data in the May 2019 Monitoring Summary Report.

Table 2.3: Summary of HSP compliance results, July 2018 to April 2019

Source: ANAO summary of DSS data.

2.43 All of the elements in Table 2.3 are mandatory requirements in the HSP contracts. While nearly all providers were creating case management plans for HSP clients, some were not developed in a timely manner and did not contain the required elements.²⁵ The DSS Monitoring Summary Report noted that the findings may be affected by factors including an inconsistency between assessors as to what constituted sufficient evidence and the low numbers of desktop reviews.²⁶

2.44 The report recommended enhanced evidence to support claims and staff training towards more consistent desktop reviews. DSS had provided desktop review training to over 20 delivery network staff in March 2019 informed by findings from recent reviews, including on consistency.

A new Assurance and Compliance Strategy

2.45 DSS commissioned a review of the quality assurance process to inform development of a potential new HSP Assurance and Compliance Strategy in May 2019.

2.46 A July 2019 draft of the Strategy included:

• a client interview for verification and feedback on the provision of a range of immediate settlement services three months after the client’s commencement in the HSP;
• a follow-up client interview to seek client verification and feedback on the provision of a range of settlement services six to seven months after their commencement in the HSP; and
• a desktop review based on case management plans and other documentary evidence uploaded by service providers to the HSP IT system to support HSP claims.

2.47 Home Affairs stated that it plans to finalise and implement the new strategy at the start of 2020 and estimates that a sample size of 787 clients per annum will provide a statistically valid sample for the future performance monitoring processes.

Reporting by service providers

2.48 HSP service providers have submitted the majority of the reports required under the contracts, except one provider has not reported on Indigenous participation and there is no evidence of reporting on compliance with the Workplace Gender Equality Act 2012 (see Table 2.4). Home Affairs could consider offering some guidance to staff on assessing the business plans and six-monthly reports.

Table 2.4: HSP reporting requirements

Source: ANAO analysis of DSS records.

2.49 The contracts require that the six-monthly reports and business plans are assessed and approved by the department before related payments are made to service providers. DSS produced an overall assessment of providers’ six-monthly reports for the period ending 31 December 2018 (see paragraph 3.47). This stated that the reports were found to be ‘comprehensive’.

2.50 Home Affairs has received the six-monthly reports for 1 January to 30 June 2019 and stated that it is summarising these to identify trends and strategies to improve service delivery.²⁷

Engagement with service providers

2.51 DSS established appropriate arrangements for engagement with service providers (see Table 2.5). Service providers considered that DSS’ relationship management was a key strength of the HSP. Home Affairs stated that the DSS engagement arrangements will largely continue under its administration of the HSP.

Table 2.5: Engagement with HSP service providers

Source: ANAO analysis of DSS meeting records and Home Affairs advice.

Issues raised by HSP service providers

2.52 Key issues identified by service providers were communicated to DSS in a 2018 ‘Community of Practice’ issues paper. The issues paper made 16 recommendations. IT system issues were noted. Service providers said that the number of Settlement Service Charge points had created an administrative burden that they did not anticipate.

2.53 The issues paper stated that service caps and the process for requesting increases was overshadowing the intended move towards outcomes instead of outputs.²⁸ It also stated:

The tension point between changing from outputs to outcomes requires some inbuilt flexibility within the program to enable Service Providers to make some judgement calls that reflect individualisation of services to clients rather than strictly adhering to service line items.

2.54 The Settlement Council of Australia (a peak body for settlement service providers with members including the HSP service providers) surveyed its members and produced a HSP Online Survey Report in October 2018. The report was submitted to DSS and noted an over-reliance on outputs (attached to the payment model) detracts from outcomes.

2.55 The Community of Practice issues paper stated that clients in crisis (including those experiencing reportable critical incidents such as family violence) can require extensive case management and administration, ‘a large portion of which is non-claimable.’ In February 2019 DSS and service providers noted that this issue mainly affected tier three clients.

2.56 Many of the issues raised in the Community of Practice paper were noted in DSS’ reviews of the program and in planned responses to these (see paragraph 3.46). DSS noted that an external evaluation of the program design and outcomes is planned in 2020–21. Home Affairs is planning to consider a number of potential changes to HSP management, further discussed at paragraph 3.45.

HSP service provider financial viability

2.57 In July 2018, Service Provider K raised concerns to DSS about additional (unfunded) costs across its contract regions, noting a higher than expected administrative burden. DSS and Home Affairs have considered the Service Provider K viability issues in detail.

2.58 A January 2019 DSS briefing to the Minister sought approval to increase Service Provider K’s payment points under the contract. The briefing noted that:

During HSP contract negotiations, despite attempts by the Department to encourage Service Provider K to reconsider and appropriately cost its price points, Service Provider K assured the Department that it could deliver services for the tendered prices. Unfortunately, this is now resulting in viability issues for a number of Service Provider K’s services and their sub-contractors.

2.59 The briefing did not state that Service Provider K’s contract pricing was lower than the DSS price assumptions, but said that its average cost per client was below the national average.

2.60 The Minister did not approve the proposed increase in payments to Service Provider K, stating that more needed to be understood about the issues before releasing additional funds to an organisation that tendered a contract price.

2.61 A February 2019 briefing to the Minister proposed five options on additional funding (over three months) to Service Provider K and its subcontractors. On 6 March 2019 the Minister approved one-off funding over three months from March to May to Service Provider K’s sub-contractors. The briefing included advice from a Service Provider K subcontractor on the impact of high rents and uncertain arrivals in a particular region. It noted that DSS would respond to this by ensuring a more sustainable and consistent flow of clients to that region. Home Affairs stated that it anticipates a smoother rate of referrals to that region throughout 2019–20.

2.62 An April 2019 consultant report to DSS on subcontracting focussed on Service Provider K and concluded that the majority of subcontractors who provided data were running at a loss for the HSP.

2.63 Home Affairs’ September 2019 HSP risk assessment noted the March 2019 payment to Provider K’s subcontractors (for a three month period) and Home Affairs plan to manage the viability risk by the potential consolidation of payment points and the HSP recalibration project (see paragraph 3.46). Home Affairs stated that it continues to work closely with the Service Provider K on contract deliverables and financial viability.

Does the HSP IT system support contract management and the delivery of the program?

Key features of the HSP IT system

2.64 The HSP IT system (HSP system) was developed to facilitate client referrals, travel, and service management (including claims submitted by providers and tracking of client outcomes). As at 8 November 2019, the HSP system had processed and paid over 769,489 claims across 36,950 clients, and has over 800 users across Home Affairs and the five HSP provider organisations.²⁹

2.65 DSS relied on Home Affairs’ Humanitarian Entrants Management System (HEMS) to manage the HSS programs since 2013. A June 2015 evaluation of HSS and CCS noted that HEMS had ‘limited case management functionality’ leading to ‘duplication and lack of visibility of client information and analytical capability.’ It noted that an improved HSP system could lead to:

• decreased burden on providers by having one system to replace both the Home Affairs HEMS system and providers’ own case management systems (this would also reduce the need for provider reporting, as data would be more accessible);
• efficiencies in departmental processes by allowing easier reporting, better quality assurance activities, and easier invoicing; and
• the collection of more robust client information.

2.66 The evaluation’s recommendation to review the IT system was agreed by DSS. DSS also noted two additional reasons for a system redesign, including: issues with ongoing operational stability as the existing systems were not scalable, flexible, or sustainable; and, a high risk of unrecoverable failure and limited ongoing technical support or capacity for enhancements.

2.67 While the new HSP system has successfully processed payments across a large number of clients, the other benefits of developing a new HSP system have not been realised.

Developing the HSP IT system

2.68 DSS finalised a Project Management Plan to support the implementation of the new IT system in February 2017, before the commencement of the HSP program in October 2017.

2.69 The HSP system functionality was to be delivered in two phases, as summarised in Table 2.6, at a total cost of $13.2 million. The system was built in-house by the Information Technology and Communications Group (ITCG) on one of DSS’ central IT platforms (Siebel).

Table 2.6: HSP system phased functionality and cost estimates

Source: DSS HSP Project Management Plan, February 2017.

Implementation of the HSP system

Governance

2.70 DSS established or used existing governance boards to oversee and manage the implementation of the HSP system. This included the HSP Transformation Board which managed the transition from the HSS and CCS to the HSP, and the HSP IT Project Board which supported the delivery of IT system enhancements and fixes after the HSP Transformation Board was disbanded. The boards met regularly in line with established terms of reference and maintained records. The boards considered achievements, issues, decisions, priorities, and risks.

2.71 DSS established guidance for HSP system users, including regularly updated system manuals and task cards, monthly communiques, and a GovDex site for users, which housed the guidance.

Adjustment of the Minimum Viable Product

2.72 In March 2017, DSS received legal advice signing-off on the release of the request for tender. The legal advice noted that delays to the delivery of the HSP system risked the department’s ability to meet its contractual obligations. In the same month the HSP Transformation Board agreed:

• that the system would go live on 30 October 2017, in line with contract commencement; and
• to a Minimum Viable Product (MVP) — the minimum system functionality needed to support the HSP.

2.73 Delivery of the MVP by 30 October 2017 was again confirmed at the HSP Transformation Board meeting on 26 July 2017.

2.74 On 6 September 2017 the HSP Transformation Board was advised by the program manager that the MVP could not be delivered by the end of October and a ‘revised and reduced scope’ MVP had been developed in order to deliver a system on time. The HSP Transformation Board was informed that the removed functionality would now be delivered in phase 2. The delivery risk was upgraded from amber to red and the reduced scope HSP system was delivered on 30 October 2017.

2.75 The delayed functionality and a description of its purpose, the proposed work around and the current implementation status is outlined in the Table 2.7.

Table 2.7: Delayed HSP system functionality and current status

Note a: The HSP Case Management Guidelines define a critical incident, discussed further in Chapter 3.

Note b: Implementing the full critical incident reporting functionality was also recommended in a 2018 review and was estimated to cost $250,000.

Source: ANAO analysis.

Key HSP system issues

2.76 There have been ongoing issues impacting the effectiveness of the HSP system. These have included: an increased administrative burden on providers and limited user acceptance testing; issues with the data exchange with external systems; and the system’s ability to monitor key performance indicators. These issues relate to the proposed benefits of the system.

Administrative burden

2.77 The HSP system was proposed partly to reduce the administrative burden on providers, to replace providers’ own systems, and to reduce duplication of effort. These benefits have not been fully realised.

2.78 Providers reported various issues with the HSP system in mid-2018.³⁰ Two issues related to the functionality that was removed when DSS agreed to a revised MVP, including:

• manually reporting DEX scores; and
• manually reporting critical incidents.

2.79 Providers continued to use their own case management solutions alongside the HSP system due to the limited HSP system functionality. They advised DSS that this was leading to duplication.

2.80 DSS acknowledged that the functionality of the HSP system was resulting in a ‘considerable additional administrative burden for providers’. In January 2019 the Minister for Families and Social Services approved a one-off administrative payment to service providers. The amount was to be based on an additional four hours of administrative work per client, and providers were advised that DSS ‘expected’ providers to pass a proportion of the funding to sub-contractors. The payment totalled $1.95 million for the period 30 October 2017 (HSP commencement) to April 2019.

2.81 User acceptance testing (UAT) is used to ensure an IT solution is working well. It allows users to access the solution and provide feedback before the system is finalised. Providers, the majority of HSP system users, participated in UAT on 9–10 October 2017, however there was insufficient time available to address the concerns raised before the system commenced operation on 31 October 2017.

2.82 A second round of UAT with the providers occurred in November 2018, over 12 months after the HSP commenced. Providers completed acceptance agreements for the December 2018, January 2019, and February 2019 updates to the HSP system. The enhancements and fixes in these releases were largely to resolve the administrative burden issues before the end of the ‘one off administrative payment’ period. DSS surveyed users in the five provider organisations after these releases and found a reduction in administrative burden for most survey participants across seven different system functions.³¹

Data exchange between HEMS and the HSP system

2.83 Replacing HEMS and improving the stability of the system supporting settlements was identified as a reason to develop the HSP system. This benefit has not been fully realised. Establishing a data exchange was also identified as a risk and was monitored throughout implementation by the HSP Transformation Board. The migration of historical HEMS data into the HSP system and the ongoing exchange of live data was implemented at commencement of the new IT system on 30 October 2017.

2.84 DSS has experienced issues with the data exchange which impacted the availability of information, including:

• critical health information not transferring to the HSP system;
• visa classification types listed in the HEMS system changing when imported to the HSP system;
• clients not being created in HSP system automatically from HEMS data; and
• personal information such as languages spoken, dates of birth, or arrival dates not transferring to the HSP system.

2.85 Home Affairs data is sent through to DSS via spreadsheets which were then loaded in to the HSP system. DSS had established protocols for actioning errors in processing the data. Providers are continuing to liaise with Home Affairs to investigate missing or incorrect data in the system.

2.86 DSS and Home Affairs have resolved a number of data exchange issues, but have also estimated it would cost $2.7 million to implement a web services data exchange to resolve multiple data issues. The planned benefit of the new system to reduce risk to operational stability was not fully realised.

Monitoring of key performance indicators

2.87 The collection of more robust client information and easier reporting were identified as a benefit of developing the HSP system. DSS uses a software program called Qlik Sense to access and visualise data from DSS’ central Siebel system, which is where HSP is housed. DSS stated to the ANAO that it had the potential to use the Qlik Sense software to access and visualise data from the HSP since implementation.³² Home Affairs began developing reports in Qlik Sense only in July 2019 (refer to Chapter 3 on key performance indicators). These were previously delayed due to system issues.

HSP system enhancements and fixes

2.88 DSS spent $141,382 on planning and scoping the HSP system. It estimated that the development and implementation would cost $13.2 million and be delivered by June 2018. DSS expended a total of $16.2 million on the HSP as at 30 June 2019 with some key functionality still missing. At September 2019, there have been 123 enhancements and fixes to the HSP system.

2.89 The system enhancements and fixes were identified via feedback from DSS staff, other user groups, including providers, and a 2018 review of the HSP. Early enhancements and fixes in 2018 related to issues such as user interface design. Later fixes were more substantive such as for tier three referrals.

2.90 As of September 2019, DSS had identified 105 further enhancements and fixes, including eight critical and 23 high priority. Release dates have been estimated for one critical and two medium priority.

2.91 The settlement services function was transferred to Home Affairs in July 2019. Home Affairs has signed a memorandum of understanding with DSS which includes responsibilities for managing the HSP System. Home Affairs and DSS are also establishing priority system enhancements and a release schedule.

3.1 In order to assess the overall program performance of the HSP, the ANAO examined:

• program risk management processes — because effective management of risks supports program planning and delivery;
• performance measurement and reporting — because complete and accurate performance information is needed to determine whether objectives are being met; and
• whether continuous improvements are made to the program — because the HSP was intended to improve settlement services compared to previous similar programs and the program has been reviewed a number of times since commencement.³³

Is there an appropriate HSP risk management process?

Compliance with the Commonwealth Risk Management Policy

3.2 The Commonwealth Risk Management Policy has nine elements and requires entities to establish and maintain risk management controls.³⁴ While the elements take an entity-wide view, ANAO assessed DSS’ risk management for the HSP and found it aligned with the policy.³⁵ Home Affairs’ overall approach to risk management is also consistent with the policy and it is updating the risk documentation for the HSP in line with its Risk Framework. The following sections cover key elements of HSP risk management.

The initial approach to program risk management

3.3 Reports were made to the HSP Transformation Board on the IT system and program transition risks before and after the program commenced in October 2017.³⁶

3.4 The risk management section of the HSP contracts says that service providers must advise the department about any contract performance issues, report critical incidents and outline risks and mitigation strategies in their annual business plans. Program risks are also communicated to the department in the service providers’ six-monthly reports.

Risk management after the HSP contracts commenced

3.5 The DSS Contract Management Guide required a finalised risk management plan at commencement of a contract. DSS stated that this risk assessment was commenced in February 2017. The contracts commenced in October 2017 but the risk assessment was not finalised until April 2018. Table 3.1 shows that at four points, risk-related documents were prepared (or drafted) and risk assessments were conducted (equivalent to a risk management plan).

Table 3.1: Risk management documents and assessments

Source: ANAO analysis of DSS and Home Affairs documents.

DSS April 2018 HSP risk assessment

3.6 The April 2018 risk assessment listed nine risks, sources for each risk, current controls, a control effectiveness rating, and risk ratings before and after planned treatments. The risk assessment included the service delivery risks identified in the DSS Contract Management Guide (except for fraud).³⁷

3.7 Sources of risk for ‘service provider failure to deliver services’ included service provider or sub-contractor insolvency and sub-contractors having insufficient resources.³⁸

3.8 The initial risk rating for the HSP IT system was ‘high’ but the target was ‘medium’ given planned treatments (note Table 2.7 on the HSP IT system).

DSS February 2019 HSP risk assessment

3.9 DSS updated the risk assessment in February 2019 to take into account September 2018 risk management recommendations in a consultant report which also identified fraud as an additional risk.³⁹ Treatments were also updated.

3.10 A March 2019 HSP Risk Management Plan stated that high and extreme risks were generally considered unacceptable but could be accepted in some circumstances.⁴⁰ In the February 2019 risk assessment two risks remained high post treatment: client settlement outcomes not being achieved (Risk 1) and compromised well-being of client (Risk 4).

3.11 Given the Service Provider K financial viability issues, a new source of risk was added: ‘service provider or subcontractor withdrawal’. A February 2019 minute to the Group Manager for the HSP noted the high post treatment rating for Risk 1. It said that the proposed treatments (especially for Service Provider K) were temporary and longer-term solutions were required.⁴¹ It was noted that Risk 4 was rated high as the consequence will always be severe.

3.12 In line with the HSP Risk Management Plan, the medium risks were reported to the Branch Manager Settlement Support and the high risks were reported to the Chief Risk Officer.

Home Affairs September 2019 HSP risk assessment

3.13 Home Affairs prepared a HSP Strategic Risk Assessment in September 2019. The risk categories reflected those in the DSS February 2019 assessment. For the six risks, there was an initial risk rating, treatments (with owners) and a target risk rating post treatment (see Table 3.2).

3.14 The post treatment risk was medium for two risks and low for four risks (see Table 3.2). The Home Affairs Risk Management Policy states that Home Affairs has a low appetite for risk to the delivery of refugee and humanitarian programs but this predates the transfer of the HSP to Home Affairs. HSP risk is to be reviewed in line with the Department’s Risk Framework.

Table 3.2: Risk ratings September 2019 risk assessment^a

Note a: Colour coding is based on Home Affairs colours for the risk level — there are no severe (red) risk ratings.

Note b: HSP IT system, systems, processes and policy etc. The post treatment risk rating for HSP enablers (principally related to the HSP IT system) fell from medium in February 2019 to low. A Memorandum of Understanding between DSS and Home Affairs on the HSP IT system (under negotiation) was noted as a treatment.

Source: Home Affairs HSP Risk Assessment September 2019.

3.15 Home Affairs has stated that it will establish a Settlement Board (as an alternative to the HSP Board that DSS planned) to look at overall settlement issues, including the HSP. Home Affairs stated that, in the interim, the September 2019 risk assessment was reported to the Assistant Secretary and First Assistant Secretary. It was also reported to the Audit Committee in October 2019.

Client settlement outcomes risk

3.16 The post treatment risk for client settlement outcomes not being achieved (Risk 1) was high in the February 2019 risk assessment. In the September 2019 risk assessment, the initial high risk rating for this risk dropped to medium as a result of the planned treatments.

3.17 These treatments for Risk 1 included the March 2019 one-off payment to Service Provider K (which was for three months). As noted in paragraph 2.63, treatments still to be potentially implemented for this risk (which are relevant to Service Provider K) include consolidation of payment points and additional payments for tier three clients.

Client well-being risk

3.18 In Home Affairs September 2019 risk assessment, the post treatment risk rating for ‘compromised well-being of client’ was medium, although it had been high in the February 2019 risk assessment. This change was a result of a clarification of the definition of critical incidents, which are used to monitor this risk. In December 2018 the critical incident definition was updated to specify that reporting of critical incidents should only occur when they are ‘as a result of the HSP service’.

3.19 The half-yearly HSP Incident reportsstate that, in addition to reporting critical incidents, providers are encouraged to report non-critical incidents. No critical incidents (in line with the updated definition) were reported. The half-yearly reports note that there were 67 non-critical incidents between July and December 2018 and 72 between January and June 2019.⁴²

Is program performance appropriately managed and reported?

3.20 This section examines the overall program performance management and reporting, as opposed to the management of the specific contract issues covered in Chapter 2. The aggregated picture for the program was examined by reviewing:

• client settlement outcomes;
• key performance indicators; and
• internal and external reporting on HSP performance.

3.21 Longitudinal scores on client attainment in areas including health and employment were recorded but not reviewed by the department, although this data could reveal program performance and potential areas for improvement. Data is only being collected on three of the nine KPIs in the contracts. Data on all contract KPIs could be aggregated to assess the overall program status but this is only partially possible given the use of only three KPIs and potential data issues. In line with the limited performance data, there has been limited internal reporting on program performance.

Client settlement outcomes

3.22 To track progress of client outcomes the HSP Case Management Guidelines include provisions on Data Exchange Reporting (DEX scores). DEX scores were designed to cover client outcomes across 13 outcome domains, such as health, housing, education and employment.

3.23 Under the contracts, service providers must assess and record DEX scores for each client at the start of the program, formal review points (varies according to client tier), and when exiting from the HSP.

3.24 Client outcome data DEX scores are not being tracked, as:

• DSS scoped out the ability to record DEX scores from the HSP IT system Minimum Viable Product (October 2017) — DSS created spreadsheets for manually recording DEX scores;
• DSS chose not to extract the data from the individual client spreadsheets and has not summarised or reported on the DEX scores for any HSP clients — Home Affairs stated that it will add a review of recording of DEX scores to the desk top reviews; and
• a February 2019 IT system upgrade allowed service providers to report DEX scores in the HSP system. However, this data cannot be extracted from the HSP IT system (for instance, to provide data on the areas covered by the six inactive KPI — see Table 3.3).

Key performance indicators

3.25 The HSP contract lists nine key performance indicators (KPIs) which were due to commence with signing of the contracts. If a service provider fails to meet a KPI consistently over four quarters, the department may terminate, suspend or reduce the scope of the contract. Failure to meet KPIs is not linked to the abatement regime (discussed in paragraphs 2.33–2.34).

3.26 HSP IT system issues at the commencement of the HSP hampered KPI data collection. A planned alternative manual system for reporting KPIs and holding service providers accountable for the results was not implemented. Instead, in November 2017, DSS instituted a ‘KPI relief period’ during which service providers were exempt from meeting the KPIs. DSS advised the providers in May 2018 that the KPI relief period was being extended and that “Service Providers must continue to meet or exceed KPIs,” however they noted that formal monitoring had not commenced. The relief period was extended multiple times (Figure 3.1).

Figure 3.1: Approval and extensions of the KPI relief period, timeline

Source: ANAO analysis of DSS documents.

3.27 DSS commenced a review of the KPIs in mid-2019, which is being continued by Home Affairs. Initial findings of the review found that six of the nine KPIs do not effectively measure the intended outcomes and were therefore considered not fit for purpose by DSS (see Table 3.3).

Table 3.3: Current status of the HSP contract key performance indicators

Source: ANAO summary of DSS data.

3.28 Reporting was not implemented for KPIs 3 — 8 as shown in Table 3.3 above and DSS analysis was that:

• the service was not required (KPIs 3, 4 and 5) — service providers advised DSS that clients engaged in education themselves and humanitarian entrants are now exempt from entering jobactive for 12 months;.
• the data could not be accessed or monitored (KPI 6) — client needs and case management plans cannot be tracked in the HSP system;
• two KPIs were not a contractual requirement or within service provider control (KPIs 7 and 8) — the service providers are not accountable for clients being employed although they are required to support clients to attend employment services.

3.29 Home Affairs implemented internal reporting against KPIs 1, 2 and 9 from 1 July 2019 to 1 January 2020 as part of a ‘transitional period’ to ‘test and refine the [KPI] reports’.⁴³

3.30 If Home Affairs commences formal KPI monitoring from 1 January 2020 as intended, enforcement of any KPI failures through provider contracts would be possible from 1 January 2021.

Quality of the key performance indicators

3.31 Two of these KPIs relate to the orientation program which delivers ‘ten core settlement topics’.⁴⁴ Clients develop comprehension at three progressive levels: awareness, knowledge and application. The orientation program is delivered to each HSP client aged 15 years and over if the case manager determines that the client needs this assistance.

3.32 Home Affairs stated that the targets of 70 per cent and 60 per cent account for clients who may not need this service, despite the department’s stated expectation that all eligible clients should participate in HSP orientation. However, if a client genuinely does not need the orientation service and it is therefore not delivered, this is counted as ‘not achieved’ by the HSP IT system. This could introduce a negative bias to the results.

3.33 Home Affairs stated that service providers must assess whether a client has achieved the outcome level. The Orientation Facilitators’ Guide says that a client meets the achievement standard for orientation if they demonstrate a majority of: attendance and participation; self-assessment of results; and an understanding of the topic and ability to apply it independently. Some providers stated to the ANAO that they have developed internal guidance to assist their staff to assess comprehension levels.⁴⁵ Home Affairs stated to the ANAO that there is an opportunity to provide additional guidance to service providers on how to assess achievement of orientation levels.

3.34 Home Affairs also stated that there is quality assurance for orientation claims via the desk top reviews, but it intends to expand assurance activities to reach a higher volume of clients and include client interviews from the start of 2020 (refer to Chapter 2, paragraphs 2.45–2.47 on the new assurance and compliance strategy).

3.35 The KPIs in each contract only provide a partial picture of performance. The two orientation KPIs (KPIs 1 and 2 in Table 3.3) are relevant to the overall objective of the HSP but they do not provide the information necessary to assess whether or not clients have achieved the intermediary outcomes listed in the HSP Outcomes Framework.

3.36 Although the third transitional KPI (KPI 9 in Table 3.3) covers administration from the service providers’ side, there are no KPIs that consider the effectiveness of the delivery of the program by the department (for example, related to the accuracy of data in the HSP system and whether payments are made on time).

3.37 Potential improvements to KPIs are outlined at the end of this Chapter.

Reporting on performance

Reporting on key performance indicators

3.38 A dashboard was provided to the Home Affairs executive in October 2019. It included performance data for KPIs 1, 2, and 9, for each provider for the July to September 2019 quarter. Home Affairs stated to the ANAO that the dashboard will be reported to the executive monthly in 2019 and quarterly from January 2020. KPI performance varied across the service providers. In summary, the report found:

• KPI 1, orientation outcomes at the ‘knowledge’ level⁴⁶: met by only one of the five providers for all three months in the quarter.
• KPI 2, orientation outcomes at the ‘application’ level: met by only one of the five providers for all three months in the quarter.
• KPI 9, data entered in to the HSP system within 14 days: met by only two of the five providers for all three months in the quarter.

3.39 One provider (Service Provider Z) only met one of the three KPI’s, for one month in the quarter. Home Affairs met with Service Provider Z in October 2019 and Service Provider Z noted it was implementing strategies to improve performance including a new case management system and a quality management system.

3.40 Home Affairs has organised meetings to discuss performance with each provider in October and November 2019 based on this performance information.

Reporting on service provision

3.41 Six HSP services have to be delivered within specified time periods. For each of these services, the July 2019 dashboard shows the percentage of service instances in July that met the timeframe requirement.⁴⁷ These outcomes varied across providers, with Service Provider Z having much lower percentages than other providers. The overall results for all providers were:

• health appointment (80 per cent of clients);
• client registered for health follow up (76 per cent of clients);
• client registered with English language opportunities (100 per cent of clients);
• school enrolment (69 per cent of clients);
• case management plan developed (79 per cent of clients); and
• essential registrations (for example, Medicare) (66 per cent of clients).

Other internal and external reporting

3.42 Overall, there has been limited internal and external public reporting on HSP outcomes, most of which has occurred from 2019. Reporting has consisted of:

• a May 2019 internal report to the HSP Working Group on transfers between regions;
• the July to September 2019 dashboard reported to the Home Affairs Executive in October 2019, with monthly reporting planned for the rest of 2019 and quarterly reporting in 2020 (see paragraph 3.38);
• a report to the Minister on HSP data and outcomes in July 2019⁴⁸;
• external reports to the Provider Advisory Group in December 2018 and July 2019 (numbers of services provided and numbers of clients reaching orientation outcomes);
• based on reports from the five service providers, DSS prepared the first six-monthly report on the HSP for the period up to 31 December 2018 (see 2.47);
• KPI data from July to December 2018 and for the July 2019 KPI report was given to service providers after June 2019; and
• Home Affairs provides monthly de-identified HSP arrivals and referrals data for planning purposes to the Senior Officials Settlement Outcome Group (inter-governmental forum).

3.43 There is very limited public reporting on the HSP via the DSS Annual Reports. The 2018–19 report notes that the program moved to Home Affairs in July 2019 and each annual report shows the number of individuals assisted under the HSP for the last three years.⁴⁹

Are continuous improvements made to the delivery of the HSP?

3.44 In order to assess whether continuous improvements were being made to the HSP the ANAO considered:

• if the recommendations made before transitioning to the HSP were implemented;
• the status of HSP reviews and improvement projects; and
• the availability of performance information.

Improvements to be made via transitioning to the HSP

3.45 DSS intended to achieve a number of benefits from moving to the HSP (see Table 3.4). The contract design largely reflects these but the benefits have only been partially delivered.

Table 3.4: Achievement of the intended benefits of moving to the HSP

Source: ANAO analysis.

Reviews and reports on the Humanitarian Settlement Program

3.46 There have been a number of reports and reviews on the HSP and some changes were made in response. In August 2019 Home Affairs provided the ANAO with a high-level schedule of planned improvements to the HSP drawing on reviews to date. This work has seven elements:

• progressively implement recommendations from the 18 month review from October 2019;
• Stage one recalibration: consolidation of payment points and options to improve employment outcomes (by early 2020);
• Stage two recalibration which involves a new outcome based and tier three funding model (by 1 July 2020);
• commence the HSP Assurance and Compliance Strategy (1 January 2020) — linked to this there is a September 2019 version of the draft strategy and initial and follow-up client interview questionnaires have been developed;
• transition in KPIs 1, 2 and 9 (by 1 January 2020) and consider potential further KPIs;
• develop options to manage service provider financial viability (these are noted in Chapter 2, paragraph 2.63 and include additional payments for tier three clients); and
• review HSP sub-contracting and implement recommendations (by 30 December 2019).

The Humanitarian Settlement Program Six-Monthly Reports

3.47 Based on reports from the five service providers, DSS prepared the first six-monthly report on the HSP for the period up to 31 December 2018. Issues to be actioned included:

• considering the Provider K subcontractor risks (later action is detailed in paragraph 2.61)
• engaging with providers on the abatement regime before it was to go live in July 2019 (the abatement regime was not active as at September 2019); and
• liaising with other agencies to resolve issues (for example, the National Disability Insurance Agency and Centrelink as forms were not client-friendly and there was difficulty linking clients to the National Disability Insurance Scheme).⁵¹

3.48 As noted in Table 2.4, providers have now submitted their six-monthly reports for January to June 2019. Home Affairs has not yet prepared a consolidated report on these.

Business Improvement Future State Report

3.49 To support the Government’s streamlining and deregulation agenda, DSS commissioned Ernst and Young (EY) to develop an HSP — Business Improvement Future State Report. This included a risk management plan (see paragraph 3.9), a high-level assurance and compliance framework, and 69 recommendations.

3.50 EY proposed that assurance should be based on desk-top reviews and recommended eliminating client visits, which ceased in July 2018. Client interviews are planned under the new draft Assurance and Compliance Strategy due to commence at the start of 2020.

3.51 When it received the report in September 2018, DSS decided that 29 recommendations would be implemented over 18 months, 20 had been recently implemented, 12 related to the HSP IT system and would be dealt with separately given funding considerations and an alternative approach would be taken for eight recommendations.

3.52 In February 2019 DSS consolidated the EY recommendations to focus on six high priorities for 2019. Home Affairs advised that the six priorities did not proceed as planned, but that the priorities are planned to be actioned by:

• developing a HSP monitoring strategy (now in the draft Assurance and Compliance Strategy);
• centralising client referral functions into the National Office which occurred in November 2019;
• formalising HSP governance forums and structures — Home Affairs stated that it is currently reviewing HSP governance structures;
• updating standard operating procedures for internal governance — now to be undertaken by Home Affairs on an ongoing basis;
• updating standard operating procedures for service provider risk assessments — now to be undertaken by Home Affairs on an ongoing basis; and
• providing training to staff and service providers — will be incorporated into Home Affairs’ training strategies over 2019–20.

DSS report on the first 18 months of the program

3.53 As at September 2019 Home Affairs was finalising an internal report on the first 18 months of the HSP from 30 October 2017 to 30 April 2019. The draft report states that the HSP was ‘mainly working well’ but there is an opportunity for improvements. The draft has 13 recommendations (Table 3.5 below), to be progressively implemented from October 2019.

Table 3.5: Recommendations in the September draft of the 18 month report on the HSP

Source: September draft of the First 18 months of the HSP Report

Review of the Basic Household Goods package

3.54 Under the HSP, clients can be given a Basic Household Goods (BHG) package (furniture and household goods) to help establish them in their on-going accommodation.⁵² From 31 October 2017 to 31 May 2019, expenditure on BHG ($31.7 million) accounted for 26 per cent of total HSP expenditure (the highest category of expenditure).

3.55 DSS conducted a review of BHG pricing and invoicing one year after the HSP commenced. It examined 809 BHG claims and found that in two contract regions (one each for Service Providers K and R) there were much higher rates of maximum claims (95 per cent and 60 per cent respectively). The review then examined six invoices from November 2017 to February 2018 for each contract region (and 12 for the regions with the high maximum claims).

3.56 The review found that invoicing varied across the contract regions; some invoices were not clear; some invoices lacked itemisation; some invoice totals were higher than what had been claimed; and some clients were residing together but were each claiming a BHG package.

3.57 In the regions with the high maximum claim levels, a single BHG package supplier was used (Package Supplier P). It invoiced a single price for the whole BHG package of goods. DSS requested an itemised breakdown of the BHG prices in the two regions with the high levels of maximum claims.

3.58 Service Provider R responded with the itemised invoices. Service Provider K advised DSS that it could not provide the itemised amounts. From the information Service Provider R submitted, DSS concluded that the price Package Supplier P charged for some items (for example, toasters) appeared to be above what would be considered to be reasonable market prices.

3.59 The review proposed obtaining the breakdown of individual BHG prices from Service Provider K, investigation of discrepancies across three providers and that service providers be reminded of the appropriate use of tax-payer funds in purchasing BHG packages. A broader review of BHG was also proposed.

3.60 Recommendations from the 2018 review are yet to be fully actioned. Home Affairs stated to the ANAO in July 2019 that it was considering the recommendations and had already implemented some changes such as enhanced guidance to contract managers.⁵³

Performance information

Employment

3.61 Home Affairs’ high-level schedule of future potential improvements states that it is planning to consider options to improve employment outcomes. A 2017 briefing to the Assistant Minister for Social Services and Multicultural Affairs noted that barriers to obtaining work included limited English, physical or mental health status and difficulties in recognising overseas qualifications.

3.62 The Australian Bureau of Statistics (ABS) Australian Census and Migrants Integrated Dataset 2016 has data on employment outcomes: 48.1 per cent of humanitarian entrants were in the labour force, of which 79.7 per cent were employed and 20.3 per cent were unemployed.⁵⁴

3.63 Home Affairs has noted that it will use external reports such as a 2018 report on the settlement experiences of refugees who settled in Queensland after coming from Syria, Iraq and Afghanistan.⁵⁵ Key findings were:

The policies and procedures that have been established to support humanitarian immigrant and refugee settlement in the first years of arrival — and the organisations that have successfully tendered to provide these services — are very successful…The evidence strongly attests that for refugees and humanitarian immigrants getting access to the Australian labour market is perhaps the greatest settlement challenge that they face.

3.64 The departments have not used the seven DEX score categories for employment outcomes to assess employment outcomes (these categories range from ‘significant disengagement in the labour market’ to ‘appropriate long term engagement in employment).’ However, KPI data extracted in October 2019 and backdated to commencement of the program shows 74% of clients aged 15 years and over have met the ‘application’ outcome level for the employment orientation. The ‘application’ outcome level indicates that these clients know there are a number of employment services to assist in finding work, and they can develop a job resume and complete a job application.

Health and well-being

3.65 The health status of HSP humanitarian migrants was covered in a longitudinal survey by DSS’ Australian Institute of Family Studies’ called Building a New Life in Australia.⁵⁶ In the period when participants would have been in Australia for up to nine months and likely receiving settlement services (under the prior programs to the HSP), 12 per cent of males and 18 per cent of females indicated poor or very poor health over the previous four weeks.⁵⁷ Thirty five per cent of males and 46 per cent of females had moderate or high levels of psychological distress.

3.66 The departments have not tracked the DEX score outcomes which support employment and quality of life, including mental and physical health. However, KPI data extracted in October 2019 and backdated to commencement of the program shows 85% of clients aged 15 years and over met the ‘application’ outcome level for the health orientation. The ‘application’ level outcome indicated clients can independently locate and make use of appropriate health services.

Future reviews of performance information

3.67 Given the transfer of the program to Home Affairs on 1 July 2019, there is an opportunity for Home Affairs to review the performance information it obtains to be better able to determine if the objective of the HSP is being met.

3.68 To date the departments have not used client outcome data (DEX scores) to assess HSP performance or to inform KPI design or program improvements.⁵⁸ As noted at paragraphs 3.22–3.23, client outcome DEX scores are recorded in 13 outcome areas at points in the client’s journey. Home Affairs plans to consider additional KPIs for service providers in 2020. This review should be expanded to include a review of all program performance information.

3.69 The departments have also not collected client satisfaction data for the HSP at client exit interviews.⁵⁹⁶⁰ The draft Assurance and Compliance Strategy, planned to commence at the start of 2020, includes sampling through client interviews. The draft makes reference to client satisfaction with rental accommodation, basic household goods and English classes and has a ten point quality rating scale for the HSP services overall.

3.70 The departments have not established a data management plan, which could have helped plan data collection, analysis and application. Such a plan could make reference to DEX scores, KPIs, client satisfaction data or other settlement outcomes data in order to be better able to determine if the program is meeting its objective.

Appendix 1 Entity responses