Introduction

1.1 Primary Health Networks (PHNs) were established by the Department of Health and Aged Care (Health) on 1 July 2015 as a delivery model for primary health care.² There are 31 PHNs across all states and territories in Australia. Each PHN is responsible for the ongoing assessment of health needs in the PHN region, supporting health services and stakeholders, and commissioning and integrating health services at the local level ³, to ensure that people can receive ‘the right care, in the right place, at the right time’.⁴ PHNs have two key objectives:

• improving the efficiency and effectiveness of health services for people, particularly those at risk of poor health outcomes; and
• improving the coordination of health services and increasing access and quality support for people.⁵

1.2 PHNs work across seven priority health areas comprising mental health, Aboriginal and Torres Strait Islander health, population health, the health workforce, digital health, aged care, and alcohol and other drug services. PHNs contributed to seven programs in Health’s 2022–23 Corporate Plan.⁶

1.3 Health awards and manages grant funding agreements with the PHNs and is responsible for:

• the provision of guidance materials;
• facilitating regular communications and information sharing with PHNs;
• provision of data to support PHN needs assessments; and
• monitoring the compliance and performance of PHNs and supporting PHNs to improve.⁷

Funding of Primary Health Networks

1.4 PHNs are non-government organisations funded through Australian Government grants. PHNs are registered with the Australian Charities and Not-for-profits Commission.

1.5 Health provides four streams of funding to PHNs: operational funding (for the administration, governance and core functions of PHNs); flexible funding (for PHN priority areas); program funding (for programs previously managed by Medicare Locals (see paragraph 1.8) which remained in scope for PHNs after Medicare Locals were abolished); and innovation and incentive funding (for new models of primary health care delivery, that may be introduced across PHNs if successful). The amount of funding provided to each PHN is dependent on the population size of the PHN region, rurality and socio-economic characteristics, among other factors.

1.6 Between 2015–16 and 2023–24, $11.576 billion in grant funding commitments were made to PHNs (Figure 1.1 and Figure 1.2).⁸ Of the total, $2.814 billion was committed to core funding agreements to fund PHN governance, operations, commissioning and coordinating activities, and $6.407 billion was committed to three of the seven PHN priority areas ($4.9 billion to mental health, $848 million for alcohol and other drugs and $682 million for Aboriginal and Torres Strait Islander health). There was also $2.104 billion committed to PHNs for various pilot and other specific programs. Separate expenditure on the population health, health workforce and digital health priority areas cannot be separately quantified due to the structure of Health’s financial information. Appendix 3 shows grant commitments for each PHN.

Figure 1.1: Grant commitments to PHNs by state and territory, 2015–16 to 2023–24 (as at August 2023)

Source: ANAO analysis of Health’s master financial spreadsheet, as at August 2023.

Figure 1.2: Grant commitments to PHNs by core funding and priority area, 2015–16 to 2023–24 (as at August 2023)

Note: Pilots and other programs comprise: a range of targeted programs in addition to funding for the Community Health and Hospitals Program; after hours support; the Commonwealth Psychosocial Support Program; continuity of support; mental health – bushfire support; National Mental Health and Suicide Prevention Agreement and Bilateral PHN Program; National Psychosocial Support measure; Partners in Recovery, and urgent care clinics.

Source: ANAO analysis of Health’s master financial spreadsheet, as at August 2023.

1.7 Commonwealth grants provided to PHNs are subject to the requirements of the Commonwealth Grants Rules and Guidelines, issued by the Minister for Finance under section 105C of the Public Governance, Performance and Accountability Act 2013. ⁹

Establishment of Primary Health Networks

1.8 Between 2011 and 2015, ‘Medicare Locals’ fulfilled a similar function to PHNs. A total of 61 Medicare Locals were created in three tranches in July 2011, January 2012 and July 2012 in response to the 2011 National Health Reform Agreement.¹⁰ Medicare Locals were created to identify the gaps in primary health care and improve service delivery at a local level.

1.9 In 2013 the Minister for Health commissioned Professor John Horvath AO to conduct a review into Medicare Locals, including all aspects of their structure, operation and functions. The Minister for Health stated that the purpose of the review was ‘reducing waste and spending on administration and bureaucracy, so that greater investment can be made in services that directly benefit patients and support health professionals who deliver those services to patients’.¹¹ The review’s March 2014 report found that:

• the name ‘Medicare Local’ was confusing;
• general practitioners were not appropriately involved in Medicare Locals;
• a number of Medicare Locals were not fulfilling their intended role of integrating care for patients, who continued to experience ‘fragmented’ health care;
• there was variability between Medicare Locals in expenditure on administration, levels of funds allocated to frontline services, and accounting practices, and inconsistencies between planned and actual budgets;
• there was a lack of clarity in what Medicare Locals were trying to achieve, variability in scope and delivery and inconsistent outcomes;
• reporting requirements were burdensome for Medicare Locals; and
• performance measures were input and process driven rather than outcome focused.

1.10 The Medicare Local review made 10 recommendations, including that a network of ‘Primary Health Organisations’ (PHOs) be established through ‘contestable’ processes. The tenth recommendation was that ‘PHO performance indicators should reflect outcomes that are aligned with national priorities and contribute to a broader primary health care data strategy’.

1.11 In April 2014 the Minister for Health accepted the review’s recommendations. Health provided the Minister with two options for the establishment of PHOs: a new policy proposal outlining options for establishing PHOs by July 2016 (Approach 1), or providing funding in the May 2014 Budget to ‘fast track’ PHO establishment by 1 July 2015 (Approach 2). Health explained that Approach 1 had the advantage of consideration by Cabinet and the sector through consultation, a policy development process and the commissioning of design expertise, facilitating a ‘robust and viable PHO model’. Approach 2 had the advantage of reducing uncertainty for the sector. The Minister for Health chose Approach 2. The 2014–15 Budget indicated that PHNs would replace Medicare Locals on 1 July 2015 and cease Commonwealth funding to Medicare Locals.

1.12 Twenty nine providers received Australian Government grants on 1 June 2015 to establish 31 PHNs.¹² PHNs were established to be closely aligned with state and territory Local Hospital Networks.¹³ PHNs represent populations ranging from approximately 64,000 people in Western Queensland to around 1.7 million people in North Western Melbourne.

Figure 1.3: Primary Health Network boundaries, 2022

Source: Health.

Rationale for undertaking the audit

1.13 Since the establishment of PHNs in 2015–16, Health has committed $11.6 billion to PHNs in grants funding. The commissioned services and activities of PHNs contributed to seven programs across three outcomes in Health’s 2022–23 Corporate Plan. The audit was identified by the Joint Committee of Public Accounts and Audit as an audit priority of the Parliament in 2021–22 and 2022–23.

1.14 Auditor General Report No. 9 of 2019–20 National Ice Action Strategy Rollout examined the expansion of alcohol and other drug treatment services through PHNs. Health agreed with the ANAO’s recommendation to ‘finalise the PHN Quality and Assurance Framework, with appropriate actions to assess whether PHNs are operating appropriately across the commissioning cycle’.

1.15 The audit aims to provide assurance to the Parliament that Health is appropriately monitoring compliance and performance of individual PHNs, as well as the overall performance of the PHN delivery model.

Audit approach

Audit objective, criteria and scope

1.16 The purpose of the audit was to assess the effectiveness of the Department of Health and Aged Care’s performance management of Primary Health Networks.

1.17 To form a conclusion against the objective, the ANAO examined:

• Has Health established fit-for-purpose compliance and assurance arrangements for the PHNs and the PHN delivery model?
• Has Health established fit-for-purpose program performance measurement and reporting arrangements for PHNs and the PHN delivery model?
• Has Health effectively evaluated the PHN delivery model to demonstrate that it is meeting its objectives?

Audit methodology

1.18 The methodology involved:

• examining Health records, including performance data and reporting;
• high-level examination of relevant record management systems;
• meetings with PHNs;
• meetings with Health personnel; and
• submissions to the audit from external stakeholders (31 submissions were received from individuals, primary health networks, industry groups, professional associations and health services providers).

1.19 Australian Government entities largely give the ANAO electronic access to records by consent, in a form useful for audit purposes. In April 2022 the Department of Health and Aged Care advised the ANAO that it would not voluntarily provide certain information requested by the ANAO due to concerns about its obligations under the Privacy Act 1988, secrecy provisions in Health portfolio legislation, confidentiality provisions in contracts and the Public Interest Disclosure Act 2013. For the purposes of this audit, the Auditor-General therefore exercised powers under section 33 of the Auditor-General Act 1997 to enable authorised ANAO officers to attend premises, and examine and take copies of documents. Health facilitated authorised officers attending the Health’s premises to examine and copy documents, however the requirement was extended by Health to all documents, including those that did not relate to Health’s obligations under legislation. Health advised that this type of information largely was not segregated in Health’s record-keeping systems and Health could not be certain, in providing documents through electronic means, that documents containing this type of information were excluded. To provide comfort to the Secretary regarding Health’s obligations under portfolio legislation, on 9 August 2023 the Auditor-General issued the Secretary of Health with a notice to provide information and produce documents pursuant to section 32 of the Auditor-General Act 1997. Under this notice, Health agreed to provide the information and documents requested through electronic means.

1.20 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $552,500.

1.21 The team members for this audit were Michael Commens, Lily Engelbrethsen, Katiloka Ata, Andrew Yam, Dale Todd, Alicia Vaughan, Elizabeth Robinson and Christine Chalmers.

2.1 PHNs receive Australian Government grant funding for their administration and operations under the core schedule to grant agreements. Funding for the priority health areas of mental health, Aboriginal and Torres Strait Islander health, population health, health workforce, digital health, aged care, and alcohol and other drug services is provided through different grant agreement schedules. Grant agreements may also cover specific projects.

2.2 The Commonwealth Grants Rules and Guidelines (CGRGs) state that officials achieve value with relevant money in grants administration by ongoing monitoring and management and adopting an active risk identification and engagement approach.¹⁴ This requires fit-for-purpose planning of assurance processes, assurance over individual grantees against grant agreements, and assurance over the collective performance of the grants program.

Is there a fit-for-purpose compliance and assurance framework?

Governance arrangements

2.3 The PHN delivery model is managed by the Primary Health Networks Branch (PHN Branch) in the Primary Care Division of Health. Since 2017–18 the PHN Branch has had approximately 40 full time equivalent (FTE) employees (Figure 2.1).

Figure 2.1: Primary Health Networks Branch full-time equivalent (FTE), 2017–18 to 2023–24

Source: ANAO analysis of Health data.

2.4 In September 2023 the PHN Program Board (see paragraph 2.6) considered the growth of the PHN delivery model since its establishment, noting an annual funding increase from $0.8 billion per annum in 2015–16 to just under $2 billion in 2022–23. The Program Board further noted that PHNs originally involved four policy areas in Health and 14 funding streams, and in 2022–23 involved 43 policy areas across 10 Health divisions and 47 funding streams.

2.5 Between February 2018 and August 2023 Health commissioned external consultants to undertake assurance, performance management, and risk management design and delivery activities for the PHN delivery model. Health advised the ANAO that there were $4.3 million in outsourced advisory services for work specific to the PHN Program between February 2018 and August 2023. This figure does not include funding that was included as part of broader Departmental contracts.

2.6 Line management and oversight arrangements for the PHN delivery model include the following.

• Executive Committee¹⁵ — The role of Health’s Executive Committee is to provide strategic direction and leadership to ensure the achievement of outcomes documented in Health’s Corporate Plan and Portfolio Budget Statements. The Executive Committee met 145 times between 27 November 2018 and 5 December 2023, and discussed PHNs 10 times over this period. The most substantive examples are: in April 2019 the Executive Committee noted the status of recommendations from a July 2018 internal audit (see Appendix 4); in July 2019 the Executive Committee discussed the outsourcing of grant service delivery through PHNs; in November 2020 the Executive Committee discussed PHNs in the context of telehealth¹⁶; and in October and November 2022 the Executive Committee discussed a September 2022 presentation that had been made to the Program Assurance Committee (discussed at Table 2.1).
• Program Assurance Committee (PAC) — The PAC is a Health advisory body in place since 2018 that is intended to provide a strategic view of program design and delivery, guidance to assist business areas to improve program design and delivery, and assurance to the Secretary and Executive Committee on the effectiveness of program management. The PAC terms of reference state that it will meet approximately seven times each year. Membership of the PAC comprises Health senior executive officials, primarily at the Deputy Secretary and First Assistant Secretary level. The PAC discussed PHNs and programs involving PHNs five times between September 2018 and December 2023, including a September 2022 presentation discussed further at Table 2.1.
• PHN Program Board — Health established a PHN Program Board in July 2018 to ensure a ‘strategic and integrated’ approach to PHN delivery model management. The PHN Program Board was established to monitor PHN delivery model performance, promote best practice program management, oversee the delivery of priority projects and manage program risks. It is chaired by the Deputy Secretary Primary and Community Care Group and its membership comprises Health senior executive officials who have policy responsibility for PHN grant agreement schedules and program activities. The PHN Program Board is required to meet between two and four times each year. The Program Board met 15 times between 6 July 2018 and 23 March 2023 (three times annually, on average).
• PHN Operational Working Group — The first meeting of the PHN Operational Working Group took place in March 2018. Health established terms of reference in August 2019, which were last updated in December 2021. The terms of reference stated that the PHN Operational Working Group was the primary mechanism to progress the work program of the PHN Program Board. The PHN Operational Working Group was comprised of Health directors with responsibility for PHN grant agreement schedules. The terms of reference state that the group will meet every three weeks, or about 17 times each year. From 1 March 2018 to September 2022, the group met 41 times (once in 2018, five times in 2019, 17 times in 2020, 15 times in 2021 and three times in 2022), prior to being disbanded in September 2022.
• PHN Branch — The PHN Branch within the Primary Care Division has primary oversight of PHNs and the PHN delivery model, manages PHN grants, communicates with PHNs, undertakes reporting and undertakes assurance.
• Other groups and divisions within Health — Other policy areas within Health are involved in aspects of the management of PHNs. These include the Mental Health and Suicide Prevention Division, the Ageing and Aged Care Group, the Population Health Division and the First Nations Health Division. Responsibilities include policy decisions for relevant grant agreements, reviews of PHN deliverables after these are processed by the PHN Branch; and reporting on specific performance measures and engaging with PHNs on these measures.

2.7 Health advised the ANAO that the PHN Operational Working Group was replaced by an ‘informal’ committee of assistant secretaries that ‘meets as required to share information and collaborate on strategic priorities for the PHN Program’. Health developed a ‘PHN Service Offer’ in June – September 2023, which describes the roles and responsibilities relating to the PHN delivery model (including those set out in paragraph 2.6, except for the Executive Committee and PHN Operational Working Group, and including reference to a ‘PHN Program Assistant Secretary Committee’). The PHN Program Assistant Secretary Committee is described as Health’s ‘senior governance forum with oversight of program delivery, performance and assurance’.

Grant agreements with individual Primary Health Networks

2.8 In addition to the core grant agreements and priority activity schedules described in paragraph 2.1, grants are awarded to PHNs for specific projects, trials, commissioning of third-party service providers and other activities. Health maintains records of funding commitments and payments to PHNs, including a history of all funding allocations since 2015. Although Health maintains a consolidated record of funding commitments, it does not maintain consolidated information on grant award numbers (as would be listed on GrantConnect), and the types of grants awarded. When the ANAO requested this consolidated grant information from Health, Health was unable to provide it.

2.9 The CGRGs state that reliable and timely information on grants awarded is a precondition for public and parliamentary confidence in the quality and integrity of grants administration. The CGRGs require Commonwealth entities to publish individual grant awards within 21 days of the grant taking effect.¹⁷ Auditor-General report No. 31 of 2022–23 Administration of the Community Health and Hospitals Program identified non-compliance with mandatory GrantConnect reporting requirements and included a recommendation for Health to establish a quality assurance process to confirm and where necessary correct the accuracy of reporting on GrantConnect.

2.10 Health’s reporting on GrantConnect of PHN grants is inconsistent with Health’s records of funding commitments and payments to PHNs. As at May 2023 GrantConnect included 841 PHN grants valued at $2.4 billion. This compares with $11.6 billion in funding commitments and $10.3 billion in payments made to PHNs by Health as at August 2023. As information about each grant awarded to each PHN has not been reported accurately by Health on GrantConnect and is not maintained by Health in another consolidated form, the ANAO was unable to determine the total number of grants awarded, the number of grants awarded to each PHN and the types of selection processes followed in determining grants awarded to PHNs. The accurate reporting of grant awards on GrantConnect is a mandatory requirement of the CGRGs, which are legal requirements under finance law. Health advised the ANAO that it is aware of this issue and intends to correct GrantConnect reporting by May 2024.

2.11 The ANAO reviewed core grant agreements in detail for a targeted sample of six PHNs.¹⁸ Core grant agreements for the sampled PHNs appropriately established governance, acquittal and performance reporting requirements.

• Governance requirements — grant agreements for each of the six sampled PHNs included requirements for:
  • maintaining appropriate governance structures, including a skills-based board, General Practitioner (GP)-led Clinical Councils and Community Advisory Committees;
  • maintaining governance arrangements to appropriately manage conflicts and related party involvement;
  • participating in and allowing access to premises, materials and documents, for the purpose of any audit as directed by Health;
  • publishing key documents including needs assessments¹⁹ and activity work plans²⁰ on PHN websites; and
  • supporting the monitoring and evaluation of the PHN through the provision of documents and surveys when requested by Health.
• Acquittal and performance reporting requirements — grant agreements for each of the six sampled PHNs included requirements for:
  • milestone reports²¹ to be submitted before payments are made, and which include annual needs assessments and annual activity work plans;
  • annual performance reporting against the 2018 PHN Program Performance and Quality Framework (discussed further at paragraphs 3.19 to 3.21); and
  • annual audited financial statements for core operations and activities, priority area activities, and pilots and other ad hoc projects (such as grants for mental health support for bushfire affected communities).

Assurance strategy, guidance and tools

2.12 Health first developed a documented strategy for assuring the performance of PHNs in 2021, five years after the PHN delivery model was implemented. The PHN Assurance Framework was developed in response to a recommendation from a 2018 internal audit which found that there were gaps in quality assurance processes. In 2021 Health engaged PwC to develop the assurance framework, which was to be operationalised from 2020–21 to 2023–24.

2.13 The PHN Assurance Framework states that it was developed to support the performance and operation of PHNs and provide greater assurance to the Australian Government over Health’s activities to ensure PHNs are operating appropriately, in accordance with their legal and financial obligations, and meeting objectives. The Assurance Framework states that it is designed to assure the performance of individual PHNs and the PHN delivery model as a whole, and that it takes a risk-based approach to conducting assurance over both. Assurance activities listed in the PHN Assurance Framework comprise: self-assessment checklists; desktop reviews; enquiries and walkthroughs; process reviews (including observation); deep dives and detailed reviews; risk based and random sampling; and independent assurance engagement.

2.14 The implementation of the PHN Assurance Framework is supported by two key documents developed in 2022 and 2023: an Assurance Strategy and a Risk Management Plan.

• PHN Assurance Strategy — This applies to 2022 to 2024. The PHN Assurance Strategy states that it will assist Health to identify key risks to the PHN delivery model, identify gaps in assurance and mitigation strategies, and target assurance activities to the areas of greatest risk. There is a timeline for the implementation of assurance activities.
• PHN Risk Management Plan — This was established in March 2023. It sets out the processes to identify, analyse, treat, monitor and manage risks and issues. Health maintains other guidance and templates to support risk owners and other officials to manage PHN risks, including a grant recipient risk tool for monitoring individual PHN risks.

2.15 Prior to and since the development of the 2021 PHN Assurance Framework and to support its implementation, Health has maintained guidance materials and tools for use by PHNs or Health officials tasked with assurance work. Forms of guidance materials have existed since 2016. Further development on materials was undertaken from 2019. Examples include the following.

• In 2019 Health commissioned KPMG to develop process maps for PHN grants management, reporting, assurance and compliance processes.
• In 2021 Health established guidance for the assessment of PHNs’ 12-month performance reports (discussed further at paragraphs 3.19 to 3.21) including an Assessment Process Guide; Compliance Checklist; Individual Performance Assessment template, PHN Program Electronic Reporting System ‘Indicator Guide’; and a ‘Performance Rubric’.²²
• In 2021 and 2022 Health revised guidance for reviewing PHNs’ needs assessments and activity work plans, including a revised assessment template for the activity work plan.

Are assurance arrangements for individual Primary Health Networks fit-for-purpose?

Assurance planning for individual Primary Health Networks

2.16 The PHN Risk Management Plan states that risks are identified and managed through day-to-day activities and policies, including annual individual PHN risk assessments (discussed at paragraph 2.18 and Table 2.1).

2.17 Separately to the annual individual PHN risk assessments, Health engaged PwC to undertake ‘baseline maturity assessments’ of the PHNs, with the final report provided to Health in April 2022.²³ The objective was to evaluate each PHN’s effectiveness, efficiency and economy of operations; identify risks and issues that would not be picked up during business as usual assurance activities; and provide Health with confidence that individual PHNs are operating appropriately, in accordance with their legal and financial obligations. Overall baseline maturity risk assessments (which ranged from ‘very low’ to ‘very high’) were derived from a quantitative financial risk indicator²⁴ and qualitative risk indicators. ²⁵ The 31 PHNs were assessed as having an overall baseline maturity risk of ‘very low’ (2), ‘low’ (12), ‘medium’ (7) and ‘high’ (10). None were assessed as having a ‘very high’ overall risk. The baseline maturity assessments included recommendations relating to sharing best practice and implementing recommendations from previous PHN delivery model reviews, as well as the revision of both the qualitative and quantitative risk indicators. The baseline maturity assessment included some commentary on areas of improvement for individual PHNs but did not include recommendations for individual PHNs. The PHN Program Board agreed that from July 2024 Health will commence a second round of PHN maturity assessments, following a review of the baseline maturity assessment methodology (which is scheduled to commence in March 2024).

2.18 Annual individual PHN risk assessments are documented using a template that contains some of the same risk indicators captured in the baseline maturity assessments, including PHN issues management, governance and performance management. However, the results of annual PHN risk assessments are inconsistent with the baseline maturity assessment. For example, two PHNs in the ANAO’s sample of six PHNs were assessed as an overall low risk in the baseline maturity assessment but were assessed as high risk in the annual PHN risk assessment (last updated for one of the PHNs in September 2022).²⁶ Health advised the ANAO that the baseline maturity risk assessments ‘utilised different criteria’ and had ‘lower tolerance levels’ than the annual PHN risk assessments.

2.19 Health first established a PHN Assurance Schedule for activities targeting the PHN delivery model and individual PHNs in 2022–23. The PHN Assurance Schedule outlines the assurance activities for the implementation of the PHN Assurance Framework. The one-page document sets out the timeframes (by month, over one year) for the achievement of milestones and assurance activities. ‘Next financial year’ activities are also summarised. The PHN Assurance Schedule categorises activities into three lines of assurance: first line (operational management), second line (oversight including risk management and compliance) and third line (internal audit).

2.20 The PHN Assurance Framework states that it takes a risk-based approach to conducting assurance. While the baseline maturity assessments have been used to inform the choice of PHNs to audit (a third line activity, see Table 2.1), there is no evidence that the annual PHN risk assessments or the baseline maturity assessments were used to inform the planning of Health’s first and second line assurance activities.

2.21 Grant agreements with PHNs require PHNs to publish annual needs assessments and activity work plans on PHN websites. Neither the Assurance Framework nor the Assurance Schedule includes compliance checks for these requirements. As at January 2024, out of 31 PHNs, the Northern Territory PHN website does not comply with relevant grant agreement requirements.

2.23 Grant agreements with PHNs state that PHNs are required to participate in and support evaluation including by providing financial and non-financial information and data to evaluators. A 2020 KPMG operational expenditure review (see Appendix 4), which was conducted in response to a recommendation of a 2018 evaluation, states that 26 per cent of PHNs did not respond to a survey seeking to quantify operational expenditure, and that other PHNs did not provide appropriate responses. The aim of the operational expenditure review was to identify the actual or ‘true’ operational expenditure of PHNs compared to what was reported, and identify unique and common cost drivers. As a consequence of PHN non-response, the review was unable to generalise findings across regional, remote and metropolitan regions, and comparative analysis included in the report was incomplete. Health advised the ANAO that it did not take any action in response to non-compliance with requirements to participate in the survey, including identifying which PHNs were non-compliant.

Assurance activities over individual Primary Health Networks

2.26 The ANAO examined whether the individual PHN assurance activities planned for 2022–23 were undertaken in accordance with the line one, two and three activities listed in the PHN Assurance Schedule (see Table 2.1 and paragraph 2.19 for Health’s definition of the three lines).

Table 2.1: Assessment of Health’s implementation of individual PHN assurance activities against the PHN Assurance Schedule

Key: ◆ Largely or fully implemented as planned ▲ Partly implemented as planned ■ Not implemented

Note a: National Archives of Australia, Information Management Standard for Australian Government, available from https://www.naa.gov.au/information-management/standards/information-management-standard-australian-government [accessed 8 November 2023].

Note b: Department of Finance, Commonwealth Grants Rules and Guidelines 2017, Finance, paragraph 12.6, p. 33.

Note c: The activity summary is a template report describing the various activities planned by PHNs and the alignment with assessed needs and government priorities. The summary includes planned expenditure and timelines for completion of each activity.

Note d: The budget summary has, for each priority area, the proposed expenditure and the anticipated variation between income and expenditure for the relevant activity.

Note e: The Performance Rubric contains three assessment standards: ‘On Track’ (‘PHN has met performance criteria for all indicators’), ‘Progressing’ (‘PHN has met performance criteria for half of the indicators and is working towards meeting the other indicators’) and ‘Initial’ (‘PHN has met performance criteria for one indicator’). Health assesses activities using a traffic light system where green is on track, amber is progressing, and red is not progressing.

Note f: The 2022–23 annual performance assessments were to be completed in December 2023.

Note g: Organisational capability indicators reported against in 2019–20 and 2020–21 were not required to be reported against in 2021–22.

Note h: North Queensland, Australian Capital Territory, South Eastern Melbourne, Adelaide, Northern Sydney and Northern Territory PHNs were subject to audits in 2022–23.

Source: ANAO analysis of Health documentation.

Are delivery model assurance arrangements fit-for-purpose?

Delivery model risk assessment

2.27 PHNs were established by Health on 1 July 2015 as a delivery model for primary health care. The PHN Risk Management Plan states that delivery model risks are identified and managed through an overarching delivery model risk register; a quarterly delivery model risk register review by PHN governance groups; and a quarterly review of risk by all program and policy areas. The ANAO did not review program and policy areas’ risk assessments as the focus of this performance audit was the management of the PHN delivery model, which is managed within the PHN Branch.

Delivery model risk register

2.28 Health maintains an overarching risk register for the PHN delivery model consistent with the Risk Management Plan. The ANAO reviewed risk registers, updated approximately every six months, covering the period October 2021 to September 2023. The September 2023 risk register documents eight risks (seven medium risks and one high risk relating to Health’s ability to manage the expansion of PHNs’ roles).

2.29 In the March 2023 and September 2023 risk registers, two moderate financial and fraud risks were accepted outside Health’s tolerance level, which is set at low for financial and fraud risks. Key controls for the fraud risk, such as audited financial statements and PHN internal controls, were described as fully effective. This is not consistent with analysis undertaken by the ANAO on financial reporting, or with financial governance and conflict of interest management findings and subsequent recommendations from PHN audits undertaken by Health (Appendix 5). The ANAO compared financial reporting in PHNs’ 12-month performance reports to audited financial statements required under grant agreements. For all six target sample PHNs, there were instances of 12-month performance reporting being inconsistent with the audited financial statements in 2019–20, 2020–21 and 2021–22. Health advised that it undertakes analysis to better understand inconsistencies in financial reporting and that it has established a dedicated finance team to work directly with PHNs to improve the accuracy of financial reporting.

2.30 Since 2019–20 Health has maintained a ‘hot issues’ register, documenting a range of issues including the development of compliant grant opportunity guidelines and the accuracy of PHN financial reporting. Hot issues registers also include issues relating to individual PHNs’ leadership and governance.

Quarterly review of delivery model risk register by governance groups

2.31 The Program Board discussed ‘hot issues’ or the PHN risk register in eight of 15 meetings held between July 2018 and March 2023. Since March 2023, when the Risk Management Plan was implemented, the Program Board met in March and in September 2023. The risk register was reviewed in the March and September 2023 meetings.

2.32 The ANAO reviewed 41 PHN Operational Working Group meeting records covering the period 1 March 2018 to its final meeting on 21 September 2022. The group discussed ‘hot issues’ at 36 of 41 meetings, including all meetings held after 13 November 2019, and was scheduled to meet on 16 December 2021 to review the PHN risk register, however the meeting was cancelled.

Assurance planning and activities over the delivery model

Assurance planning

2.33 Updates to the delivery model risk register did not feed into the 2021 Assurance Framework and the 2022 Assurance Strategy, which have not been reviewed or amended since their establishment.

2.34 There are some inconsistencies between the PHN Assurance Framework and the Assurance Schedule regarding delivery model assurance activities. The PHN Assurance Framework sets out meetings of the Program Board, ‘deep dives’ and process reviews as delivery model assurance activities. The Assurance Schedule also sets out Program Board meetings. Thematic assurance activities or ‘deep dives’ as described in the PHN Assurance Framework were not included in the 2022–23 Assurance Schedule. The Assurance Schedule indicated that a revised PHN Program Performance and Quality Framework, as well as an annual review of the Assurance Framework, would be undertaken in July 2023. Health advised that completion of this work has been delayed until June 2024.

Assurance activities

2.35 The ANAO examined whether the delivery model assurance activities planned for 2022–23 were undertaken in accordance with the PHN Assurance Framework and PHN Assurance Schedule (Table 2.2).

Table 2.2: Assessment of Health’s implementation of PHN delivery model assurance activities

Key: ◆ Largely or fully implemented as planned ▲ Partly implemented as planned ■ Not implemented

Source: ANAO analysis of Health documentation.

3.1 Internal and external performance measurement and reporting facilitates good management, governance and decision-making. External reporting plays an important role in maintaining public trust.

• The Commonwealth Performance Framework sets out requirements for Australian Government entities for measuring, assessing and publicly reporting on activities.²⁷
• The Commonwealth Grants Rules and Guidelines (CGRGs) require that ‘grants administration should have a performance framework that is linked to an entity’s strategic direction and key performance indicators.’²⁸
• The Commonwealth Evaluation Policy and Toolkit (which was established on 1 December 2021) emphasises the importance of a program logic model in developing performance information.²⁹
• The Archives Act 1983 makes Commonwealth agencies responsible for following record keeping requirements set out in the Information Management Standards. Principle Seven of the Information Management Standards states that business information is to be saved in systems where it can be appropriately managed, protecting the integrity of the information and supporting trusted and reliable use.³⁰

Is there a fit-for-purpose program logic?

3.2 The Australian Institute of Family Studies states that the purpose of a program logic is to:

… show how a program works and to draw out the relationships between resources, activities and outcomes … The logic model is both a tool for program planning — helping you see if your program will lead to your desired outcomes — and also for evaluation — making it easier to see what evaluation question you should be asking at what stage of the program.³¹

3.3 The Commonwealth Evaluation Toolkit includes a program logic template and guide. The program logic template suggests considering program objectives, inputs, activities, outputs, and causally linked short-term, medium-term and long-term outcomes.³²

3.4 Health first established guidance (the Performance Measurement and Reporting Framework) on developing a program logic model in 2021. The guidance includes contradictory information as it refers to two different sets of program logic elements, one of which is consistent with the Commonwealth Evaluation Policy and Toolkit. In March 2023 Health finalised a set of evaluation fact sheets, one of which contains guidance on developing a program logic and a template that is consistent with the Commonwealth Evaluation Policy and Toolkit.

3.5 Health has established a program logic for the PHN delivery model.

• The first model was developed in 2015 and was consistent with the Commonwealth Evaluation Policy and Toolkit.
• The 2015 program logic was replaced in 2018. The 2018 program logic model focused on the core function of PHNs (primary care) and included a separate program logic for each of the seven priority areas (see paragraph 1.2 and Appendix 6).
• Health established a new program logic in 2023 that consolidated the work PHNs do across core functions and all priority areas (see Appendix 6).

3.6 The 2018 program logic contains inputs, activities, outputs, and medium- and long-term outcomes that are consistent with the stated objectives (see paragraph 1.1) of the PHN delivery model. Longer-term outcomes are that ‘PHNs support local primary health care services to be efficient and effective, meeting the needs of patients at risk of poor health outcomes’ and ‘Patients in local region receive the right care in the right place at the right time’.³³ There are five ‘intermediate’ outcomes.³⁴

3.7 Although the 2018 core program logic contains most of the expected elements of a program logic, some of its characteristics reduce its effectiveness as a tool for program planning and evaluation. Outcomes are vague. For example, terms such as ‘quality care’ could be more defined. Three of the five ‘outcomes’ more closely resemble outputs.³⁵ Anticipated outcomes are not supported by more specific timeframes for their achievement, other than being described as ‘intermediate’ and ‘longer-term’. Outcomes in the seven 2018 priority area program logics are more specific and describe changes in knowledge, skills and behaviour.³⁶ On the basis of the program logics alone, the logical link between outputs and outcomes is not always clear.³⁷ However, the 2018 program logic was supplemented by a ‘theory of change’, which provided context for the causal relationships between the elements.³⁸

3.8 The 2023 program logic includes ‘objectives’ at the start of the logic chain, which the guidance defines as the ‘intended result of the work’. The 2023 program logic introduces five new ‘objectives’ for the PHN delivery model: ‘improved quality of care and population outcomes’, ‘enhanced patient experience of care’, ‘sustainable cost’, ‘improved provider experience’ and ‘advancing health equity’. Outputs and impacts are included but these are expressed as performance measures³⁹, and as such are more narrowly formulated than in the 2018 program logic, and the logical connection with the five new objectives is unclear. The 2023 program logic is not supported by a ‘theory of change’.

3.9 Neither the 2015, 2018 nor 2023 PHN program logic models considered potential unintended outcomes or adverse consequences from the PHN delivery model.

Are there fit-for-purpose performance measures?

3.11 When transitioning from Medicare Locals to PHNs in 2014, Health had advised the Minister for Health that PHNs would have a small number of key performance indicators that they would be expected to achieve, with ‘a move away from counting inputs and outputs to an environment of achieving outcomes’. In addition to having an outcomes focus, the performance indicator approach was meant to minimise administrative burden on PHNs such that ‘funding to frontline services will be maximised’.

3.12 Performance measures for Primary Health Networks are found in grant agreements, and in a separate performance framework.

Performance measures in grant agreements

3.13 According to the CGRGs, officials administering grants should establish appropriate performance measures on which to evaluate grants. The CGRGs note that:

Performance reporting requirements and other information sought from grantees are key inputs used by officials in evaluating whether outcomes have been achieved and whether a particular grant activity achieved value with relevant money. In developing the performance reporting and information requirements for particular grant opportunities and grantees, officials should balance the amount of information sought and the associated costs to grantees of collecting and collating such information, against the obligation to perform due diligence in relation to grant evaluation processes. Officials, in close consultation with government and non-government stakeholders, should design performance information to show the extent to which grant activities contribute to government outcomes, as well as producing outputs.⁴⁰

3.14 Grant agreements with PHNs include a total of 52 performance measures (see Appendix 7) associated with: activities in the core schedule to the grant agreement⁴¹; activities related to the aged care⁴², alcohol and other drugs⁴³, and mental health priority areas⁴⁴; and Aboriginal and Torres Strait Islander health activities.⁴⁵

3.15 The 29 grant agreement performance measures in the core, aged care, alcohol and other drugs and Indigenous health grant agreements are primarily (82 per cent) measuring compliance with grant agreement requirements, such as reporting requirements). As grant agreement management measures, they provide little information upon which to base future decisions for designing, continuing or concluding grant opportunities⁴⁶, and they provide little information about whether outputs and outcomes have been achieved.

3.16 The 23 measures included in the mental health grant agreements evenly comprise measures for grant agreement management, and measures that track performance against grant activity outputs and outcomes. Mental health grant agreement performance measures are supported by data sources, and they represent a mix of input, output and outcome measures. Of the 23 mental health performance measures, six have clear targets. The mental health grant agreement is also the only one to include measures of efficiency related to the cost and timeliness of services.

Performance measures in a performance framework

Performance measure framework

3.17 In a 2016 PHN Performance Framework, Health established a set of measures for PHNs that were intended to assist with performance monitoring of individual PHNs as well as evaluation of the PHN delivery model as a whole. The measures comprised the following.

• ‘Organisational’ measures — 28 measures to monitor the establishment of PHNs and the development of their key organisational capabilities. These related to governance, financial management, stakeholder management, supporting general practice, delivery of contracted services and direct services.
• ‘National headline’ measures — Four ‘measures that reflect the Australian Government priorities’. These outcome measures comprised: potentially preventable hospitalisations; childhood immunisation rates; cancer screening rates (cervical, breast, bowel); and mental health treatment rates (including for children and adolescents).
• ‘Local’ measures — Approximately five to 15 for each PHN, to measure the performance of individual PHNs at addressing local health priorities. Local indicators were developed by PHNs, included in activity work plans (see paragraph 2.8), and approved by Health. The 2016 PHN Performance Framework contained guidance for PHNs on developing local indicators and there was variation across PHNs in the number, topic and quality of local indicators.

3.18 A July 2018 EY evaluation of the PHN delivery model (see Appendix 4) identified an opportunity to improve performance management. The evaluation report noted that ‘the performance indicators that had been developed at a national, local (of which there were too many) and organisational level are not as useful as they could be’. Concerns were raised about the failure of the 2016 Performance Framework to include new programs, ‘inconsistent and fragmented reporting’, PHN reporting ‘burden’, and a lack of clarity over ‘whether data collection for performance monitoring was intended to measure compliance, accountability and/or quality improvement’.

3.19 A 2018 PHN Performance and Quality Framework (PHN Performance Framework) was intended to ‘move the Program towards outcome-based reporting’ and to measure the PHN delivery model’s progress towards achieving its objectives. The 2018 PHN Performance Framework is publicly available on Health’s website.⁴⁷ It included nationally consistent performance measures for the PHN delivery model for the first time since the establishment of PHNs in 2015–16. The measures provide the basis for 12-month PHN performance reports, discussed in the context of line two assurance activities at Table 2.1, and when aggregated underpin annual reports on the overarching PHN delivery model. The measures comprised the following.

• ‘Performance’ measures — 39 measures, comprising 13 related to the core functions of the PHN delivery model (labelled ‘P’ in Appendix 8) and 26 related to the priority areas (See paragraph 1.2).
• ‘Organisational’ measures — 15 measures based on those established in the 2016 PHN Performance Framework.
• ‘Aspirational’ measures — 13 measures that were to be developed after the implementation of the PHN Performance Framework. These measures were meant to relate to cultural appropriateness of health services, digital health services, aged care, and long-term impacts of improvements to services. As at December 2023 the ‘aspirational indicators’ had not been developed.

3.20 These measures were selected as they reflected areas where PHNs could be expected to influence changes.⁴⁸

3.21 Performance measure data sources comprise: data reported by PHNs to Health; and secondary data from the Australian Institute of Health and Welfare (AIHW), Australian Bureau of Statistics (ABS), Australian Digital Health Agency, and Services Australia (previously Department of Human Services).

Quality of performance measures

3.22 Section 16F of the Public Governance, Performance and Accountability Rule 2014 (PGPA Rule) contains requirements for an entity’s performance measures to be published in its annual report. The Department of Finance’s Resource Management Guide (RMG) 131 Developing good performance information outlines principles to assist entities with meeting the PGPA Rule requirements. Performance measures should be relevant; reliable and verifiable; unbiased; contain a mix of qualitative and quantitative measures; contain a mix of output, efficiency, and effectiveness measures; and provide a basis for assessment over time. Health’s 2021 Performance Measurement and Reporting Framework (see paragraph 3.4) provides guidance on developing performance measures that is aligned with best practice principles in RMG 131.

3.23 Of the 39 performance measures in the 2018 PHN Performance Framework, 18⁴⁹ provide assurance over the execution of grant agreement requirements and would be better described as grant agreement management measures.⁵⁰ The PGPA Rule and the RMG 131 do not disallow the use of such measures in external performance reporting, however, grant agreement management measures provide limited information about the achievement of delivery model outcomes (which was the stated purpose of the 2018 PHN Performance and Quality Framework).

3.24 Public submissions to this audit raised concerns with the quality of PHN performance measures, the alignment of measures to PHN activities and objectives, and the administrative burden associated with collecting and reporting against the measures. The ANAO undertook further assessment of the 20 performance measures that were not grant agreement management measures against the principles outlined in the RMG 131 (Table 3.1). The measures partly align with RMG 131 principles, with the main areas of weakness being their relevance to the PHN delivery model objectives, and the risk of inaccuracy and bias due to unverified data provided by PHNs for some measures.

3.27 RMG 131 notes that performance measures should provide an unbiased basis for assessment of performance. There is a strong reliance on self-reporting from PHNs, especially for the 18 grant agreement management measures. Health does not verify the information provided by PHNs, creating the risk of positive bias in the compliance and performance results (see paragraph 3.56).

3.30 As noted in Table 2.2, Health planned to revise the 2018 PHN Performance Framework and develop new measures with an expected implementation from 1 July 2023. Health did not develop indicators or commence implementation as scheduled. The new performance framework is to be implemented from 28 June 2024.

Has Health effectively reported on delivery model performance?

3.31 Health publicly reports on PHN delivery model performance through PHN annual reports and its corporate plan.

Primary Health Network annual reports

3.32 There was no requirement in the 2016 PHN Performance Framework for annual reports. The 2018 PHN Performance Framework indicated that Health would produce an annual ‘PHN Program Performance and Quality Report’.⁵¹ The reports are meant to provide ‘an overall assessment of the PHN delivery model’s performance in meetings its objectives’.⁵²

3.33 As at December 2023, Health had published three annual performance reports for 2018–19⁵³, 2019–20⁵⁴ and 2020–21⁵⁵, and, although required under the 2018 PHN Performance Framework, as at December 2023 there were no annual performance reports for 2021–22 or 2022–23. Annual reporting from 2018–19, 2019–20 and 2020–21 is summarised in Appendix 8.

Timeliness of annual reporting

3.34 Health published the 2018–19 report on 1 January 2020. Publication was aligned with the release of secondary data for the core performance measure P12 (‘Rate of potentially preventable hospitalisations’, see Appendix 8), which Health obtains from the AIHW. Health advised the ANAO that ‘Due to delays in receiving hospital data from states and territories, and having this data cleared by the [AIHW], there is currently a two-year lag from the end of financial year to report publication.’ This means that Health is allowing the availability of data for one measure to drive the timing of reporting on all measures. In addition to the lag associated with the AIHW data, there were further delays in annual reporting in subsequent years. The 2019–20 Annual Report was published on 23 February 2023 and the 2020–21 Annual Report was published on 1 June 2023. The delays in annual reporting are not consistent with public transparency over PHN performance.

Completeness of annual reporting

Reporting of established performance measures

3.37 The 2018 PHN Performance Framework states that the annual report will include the assessment of all 54 delivery model measures (see paragraph 3.19), against an outcome theme, ‘to provide an overall assessment of the PHN program’s performance in meeting its objectives’.⁵⁶ Annual reporting does not include and is not meant to include grant agreement performance measures.

• ‘Organisational’ measures — The three annual reports include reporting of all 15 ‘organisational’ measures. From 1 July 2021 Health no longer required PHNs to report against 13 out of 15 ‘organisational’ measures in each PHN’s 12-month performance report. Health advised that this decision was based on the outcomes of the 2021 baseline maturity assessments conducted by PricewaterhouseCoopers (PwC) (see paragraph 2.17) The two measures that PHNs were still required to report against involved activity variance reporting and the submission of key documents such as financial statements.
• ‘Performance’ measures — Of the 39 core and priority area performance measures, 36 were reported in 2018–19⁵⁷, and 35 were reported in 2019–20⁵⁸ and 2020–21.⁵⁹

3.38 All measures not reported on were based on secondary data. Excluded measures varied from year to year, however two measures were not reported in any of the three published reports. Data for one of these measures (P11 ‘Rate of discharge summaries uploaded to My Health Record’) is not available at all. Health did not provide an explanation in any of the three annual reports for why the second measure (P13 ‘Number of health professionals available’) was not reported. For the other unreported measures, the 2018–19 Annual Report stated that ‘information was not collected for 2018–19’ for one measure. The 2019–20 and 2020–21 annual reports provided no information as to why measures were not reported; however, Health advised the ANAO that the data for these indicators was unavailable for the relevant reporting period.

3.39 Health states in the 2018 PHN Performance Framework that it intends to disaggregate three indicators by Aboriginal and Torres Strait Islander status.⁶⁰ This disaggregation did not appear in any of the three annual reports. In 2018–19 Health reported that ‘data was unable to be sourced for this disaggregation’ for two indicators and did not comment on the disaggregation for the third indicator.

3.41 For measures that rely on PHN-supplied data, there were also issues with the completeness of data.

• The 2018–19, 2019–20 and 2020–21 annual reports note that reporting requirements were not met for nine measures over the three years, stating for example that ‘Five PHNs did not meet reporting requirements’.⁶¹ However, ANAO analysis of PHN reporting showed 14 PHNs did not provide evidence for at least one of 17 performance measures over the three years.
• In addition to non-reporting against whole measures, PHNs did not provide all required information for the performance indicator DH2 (‘Rate of health care providers using specific digital health systems’) in 2018–19, 2019–20 or 2020–21. Health advised that this was due to errors in the reporting template in the Primary Health Networks Program Electronic Reporting System (see paragraph 3.55).

Efficiency and financial reporting

3.42 PHN delivery model objectives include to improve ‘the efficiency … of health services for people, particularly those at risk of poor health outcomes.’ There is no measurement of, or reporting about, the efficiency of services delivered by individual PHNs or through the PHN delivery model as a whole.

3.43 The 2018–19 Annual Report included the distribution of allocated funding by priority area. This information was not included in 2019–20 and 2020–21 annual reports. ANAO reviewed annual expenditure reporting for all six PHNs included in its targeted sample and found significant underspends (see paragraph 2.29). Annual reports do not discuss this or other aspects of PHNs’ use of grant funding.

Reporting of individual PHN performance

3.45 Reporting on the performance of individual PHNs provides assurance to stakeholders that each PHN is operating effectively. Health states in the 2018 PHN Performance Framework that it will not discuss individual PHN performance in the annual report but that individual examples of best practice in commissioning or achievement may be included with the agreement of the PHN. The rationale for not disclosing individual PHNs’ performance is not stated in the 2018 PHN Performance Framework or elsewhere.

3.46 In accordance with the 2018 PHN Performance Framework, annual performance reports do not include individual PHN performance results or comparative analysis of PHNs. There is some analysis of PHN types in 2020–21 (for example, by state or metropolitan versus regional). Annual reports provide some de-identified information on the number of PHNs meeting or not meeting a performance measure, however underperforming PHNs are not identified. Individual examples of best practice also have not been reported. ANAO analysis showed significant variation in performance between PHNs and that some PHNs’ performance had declined. This is not conveyed in annual reporting.

Clarity and usefulness of annual reporting

3.49 The 2018–19, 2019–20 and 2020–21 annual reports differed in terms of what information was conveyed about each 2018 PHN Performance Framework measure (see Appendix 8). Reporting was of four types, which varied in usefulness to form a view about overall delivery model effectiveness.

• Type 1 — Reporting provided information about the number of PHNs that ‘met’ the measure plus some further information about the output.
• Type 2 — Reporting provided information about the output (this form was more commonly used for indicators based on secondary data).
• Type 3 — Reporting provided the number of PHNs that met the requirement, with no further information about the output.
• Type 4 — Reporting provided the number of PHNs that provided evidence of meeting the requirement, with no additional information about whether they met the requirement or about the output.

3.50 The way in which each performance measure was reported depended in part on the nature of the measure. Figure 3.1 shows the way each measure was reported in each of the three annual reports. Although the performance measures remained largely the same across the three annual reports, Figure 3.1 shows that the nature of reporting changed.

• The total number of reported organisational measures remained the same between 2018–19 and 2019–20 and 2020–21, while the total number of reported grant agreement management measures increased by one between 2018–19 and 2019–20 and 2020–21, and the total number of remaining measures decreased by two.
• Reported grant agreement management measures in 2019–20 and 2020–21 contained less useful information when compared to 2018–19.
• For the remaining performance measures, there was movement away from reporting information about the output (type 1 and type 2) towards reporting whether the PHN had ‘met’ the measure (type 3 and type 4) in 2019–20 and 2020–21.

Figure 3.1: Types of annual reporting by type of performance measure, 2018–19 to 2020–21

Note: Analysis is based on the number of reported measures in each year for the three types of measures. 2018–19: organisational (15); compliance (17); performance (19). 2019–20 organisational (15); compliance (18); performance (17). 2020–21: organisational (15); compliance (18); performance (17).

Source: ANAO analysis of PHN annual reports.

3.51 The 2019–20 and 2020–21 annual reports also reported some measures in a more ‘narrative’ style that did not identify the measure number. Health advised the ANAO that this was to ‘improve readability’ of the reports, however this approach impedes year on year comparisons of performance data.

Corporate Plan

3.54 Commonwealth entities must report in their annual report on actual results achieved against the performance measures published in their corporate plans and portfolio budget statements. This performance information must be presented as an annual performance statement in accordance with the Public Governance, Performance and Accountability Act 2013 and PGPA Rule. The PHN program contributes to seven programs included in Health’s 2022–23 Corporate Plan (Appendix 9). Auditor-General Report No. 16 Audits of the Annual Performance Statements of Australian Government Entities — 2022–23 includes analysis of Health’s performance statements and associated performance measures.

Has Health established fit-for-purpose IT systems for managing performance reporting?

3.55 PPERS was developed in 2019 to support the PHN funding process and monitoring of PHN delivery. The PPERS User Manual includes guidance for PHNs on uploading and inputting performance information relevant to milestone reports, including needs assessments, activity work plans and 12-month performance reports. Health has also developed guidance for Health officials administering PPERS.

3.56 Health uses data extracts from PHN 12-month performance reporting in PPERS as input into its annual performance assessment of PHNs (discussed at Table 2.1), and in PHN annual reports (discussed at paragraphs 3.33 to 3.38). PPERS characteristics limit its usefulness for PHN performance data entry, verification and reporting.

• Data entry — Performance information is captured primarily in free text fields, which limits the analytical usefulness of the data and decreases the ability of Health to verify PHN-supplied data. The ANAO identified data quality issues relating to inconsistency in the reporting of individual PHN names, lack of version control and inconsistent naming of schedules/funding streams.
• Data verification — There is guidance to PHNs on what to include in free-text fields, however the lack of defined system parameters for data means that verification at the data entry stage is not possible for some measures. The range of data captured does not facilitate data verification. For example, for the Indigenous Health indicator IH2 (‘Types of organisations delivering [Integrated Team Care] services’), PHNs are required to provide the number and type of organisations delivering services, but not the names of the organisations, which could be useful to a verification process. Using business rules provided by Health, the ANAO could not verify the data in PPERS.
• Reporting — PPERS does not have the capability to generate reports. Performance information is exported from PPERS for further analysis in other systems. While Health has established processes for extracting data from PPERS for further analysis, this is not a simple or timely process.

3.57 The ANAO received 31 submissions to the audit from individuals, PHNs, industry groups, professional associations and health service providers. PPERS and associated data issues were raised as an issue in 26 per cent of submissions. Since late 2019 Health has monitored issues with PPERS for PHN and Health users of the system. These include: problems with access and permissions; and uploading and revising needs assessments, activity work plans and financial reporting.

3.58 In May 2023 the division responsible for PPERS developed a business case seeking capital funding of just under $1 million for improvements to PPERS. Health approved funding to fix ‘fundamental bugs’ in the PPERS system to ‘enable processing to continue’ in May 2023. Health advised that no further enhancements were funded.

3.61 Beta testing for the Primary Health Operational and Commissioning Unified System (PHOCUS) was completed in September 2023. PHOCUS was developed by the WA Primary Health Alliance, the provider for the three Western Australian PHNs. Health advised the ANAO that PHOCUS is intended to replace the performance indicator reporting component of PPERS, but is not intended to replace other PPERS functions, such as financial reporting and milestone reporting.

4.1 The Commonwealth Evaluation Policy (Evaluation Policy) states that Australian Government entities are expected to deliver support and services for Australians by setting clear objectives and consistently measuring progress towards achieving those objectives.⁶² The PHN delivery model aims to: (1) improve the efficiency of health services for people, particularly those at risk of poor health outcomes; (2) improve the effectiveness of health services for people, particularly those at risk of poor health outcomes; (3) improve the coordination of health services; (4) increase access for people; and (5) increase quality support for people.⁶³

4.2 Evaluation is supported by an evaluative culture and good governance, including a plan to conduct evaluation activities.⁶⁴ The ANAO examined whether Health has a fit-for-purpose evaluation plan for the PHN delivery model, and whether it has conducted evaluation of the PHN delivery model in accordance with its plan. The ANAO also examined the existing evidence to determine whether the PHN delivery model can be said to be achieving its objectives.

Has Health appropriately evaluated the Primary Health Network delivery model?

Evaluation planning

4.3 The Commonwealth Evaluation Policy⁶⁵ and Department of Finance (Finance) guidance on conducting evaluation⁶⁶ state that plans to conduct fit-for-purpose evaluation activities should be established before beginning any program or activity; and should include timeframes, resources, baseline data and performance information. As part of a tool kit of evaluation resources, Finance produces a template for an evaluation plan.

4.4 Health introduced a departmental Evaluation Strategy in 2017 to strengthen policy and program evaluation practice and increase the use of evaluation evidence for decision-making, planning and reporting. The departmental Evaluation Strategy applied to the period 2016 to 2019.⁶⁷ It noted that ‘all policies and programs should be evaluated on a systematic basis to ensure they are appropriate, effective and efficient’. In its guidance for risk-based evaluation planning decisions, a ‘tier one’ program is described as one that is of strategic significance; involving a high level of funding, risk and public interest; or is new. The Evaluation Strategy indicates that a program with these characteristics indicatively uses external evaluation, multiple evaluation points, high quantitative data requirements, comprehensive qualitative evaluation and multiple evaluation reports.

4.5 The 2016–2019 Evaluation Strategy included a model for evaluating policies or programs in four lifecycle phases comprising development, planning, implementation, and transition. The Evaluation Strategy noted that in the planning phase it is essential to commence planning for evaluation, and that in the implementation phase (defined as the period in which a policy or program is operational), ongoing monitoring and evaluation is needed to determine whether the program is working as expected and is cost-effective. Health’s 2016–2019 Evaluation Strategy indicated that divisions are responsible for funding and conducting evaluations. Health internal guidance on grants administration states that Health’s Policy Division is responsible for the evaluation of grant programs.

4.6 In accordance with the principles later expressed in the 2016–2019 Evaluation Strategy, in 2015 Health had contracted EY to develop a PHN Evaluation Framework and to evaluate the PHN delivery model. The PHN Evaluation Framework was fit-for-purpose and included:

• a statement of the PHN delivery model’s objectives⁶⁸;
• a timeline for the evaluation to be completed between July 2015 and December 2017;
• plans for obtaining ethics approval;
• a detailed data collection and methodological approach that specifies the use of existing data wherever possible;
• evaluation questions aligned to the objectives of the delivery model, including to what extent PHNs are ‘fit-for-purpose’ and whether PHNs have increased the efficiency and effectiveness of medical services for patients, particularly those at risk of poor health outcomes;
• a mapping of evaluation questions to data sources;
• a consideration of baseline data ⁶⁹; and
• a risk analysis.

4.7 The PHN Evaluation Framework incorporated evaluation activities up to December 2017. Health did not develop a PHN evaluation plan that applied after this date and at December 2023 does not have a plan for future evaluation activities. A program logic model was updated in 2023.

Evaluation activities

4.8 Health published the report from the 2016 to 2017 EY evaluation in July 2018. The evaluation report, which is publicly available on Health’s website⁷⁰, described its aims as: ‘(1) assessing how the PHN Program was implemented in local contexts; (2) understanding the extent to which the PHN Program had an impact and achieved its intended objectives; and (3) informing the ongoing implementation of the PHN Program.’ The ANAO did not assess the 2018 evaluation to determine if it was conducted in accordance with the PHN Evaluation Framework or its overall quality. The evaluation’s conclusions, main findings and recommendations are described in Table 4.1 and Appendix 4. There are no other evaluation reports of the PHN delivery model on Health’s website, and no delivery model evaluation was conducted after July 2018.

4.9 Although there was no evaluation of the delivery model after 2018, between October 2015 and March 2023, Health evaluated 23 pilots, trial measures and other grant-funded activities delivered by some or all PHNs (Appendix 10). Health has not analysed these to determine if there are any recurring themes or consistent findings relevant to the achievement of PHN delivery model objectives.

Has the delivery model met its objectives?

Evaluation findings

4.12 The main overall findings of the 2018 evaluation are shown in Table 4.1. Against the objectives of increasing the efficiency and effectiveness of health services for patients, particularly those at risk of poor health outcomes, the evaluation concluded that ‘there were indications of progress’. Against the objectives of improvements to the coordination of care to ensure patients receive the right care in the right place at the right time, the evaluation concluded that there had been ’some progress in building the strong foundations required to improve regional coordination of care …’.

Table 4.1: Summary of 2018 PHN delivery model evaluation findings

Source: ANAO summary of 2018 EY evaluation of the PHN delivery model.

4.13 The evaluation provided 24 ‘opportunities for future development’ of the PHN funding model, across the themes of: governance; external collaboration and stakeholder engagement; commissioning; performance management; program guidance; departmental operations; PHN operations; funding model; national support function; Aboriginal and Torres Strait Islander health; and the use of data (see Appendix 4). The implementation of recommendations is tracked by Health. All but four recommendations were listed as complete as at July 2023. The unimplemented recommendations include to improve alignment between monitoring and evaluation processes.

Improved efficiency of health services

4.14 Health did not establish measures of efficiency in its 2018–19 performance framework. Reporting in the 2018–19, 2019–20 and 2020–21 annual reports did not demonstrate the efficiency of health services delivered through the PHN delivery model, either for users of those services or for the Australian taxpayer. Health’s PHN annual reporting does not include financial information such as the amount of funds committed to and expended by PHNs, and it does not include analysis of the financial performance of PHNs or the delivery model.

Improved effectiveness of health services

4.15 As discussed in Table 3.1, a lack of alignment between Health’s performance indicators and the PHN delivery model’s objectives impedes Health’s ability to demonstrate whether PHNs are meeting the key objective of improving the effectiveness of health services for people, particularly those at risk of poor outcomes.

4.16 Although the 2018–19 PHN Annual Report described the performance results for 10 measures as ‘baseline’ results, this baseline was not used for trend reporting. For two measures in 2019–20 and eight measures in 2020–21, annual reports included a brief comparison with the previous year’s result.

4.17 The ANAO examined trends for 13 of 14 quantitative measures⁷¹ based on PHN-reported data to identify any trends in compliance with grant agreement requirements and performance over the three years for which PHN performance information is publicly available (see Table 4.2 and Appendix 11 for a full description of the measures). Between 2018–19 and 2020–21, there was an upward trend in average PHN-reported performance for nine indicators⁷² (see Table 4.2). A lack of baseline data makes it difficult to determine if effectiveness has increased since the PHN delivery model was implemented.

Table 4.2: 13 performance measures, 2018–19 to 2020–21

Indicator	2018–19	2019–20	2020–21	Trend
AOD1	98%	100%	100%
DH1	100%	100%	100%
DH2	75%	88%	92%
IH1	962136	787617	991115
IH2	2	2	2
IH4	61%	77%	88%
MH1	.15	.15	.20
MH2	.36	.41	.49
MH3	.11	.10	.15
MH5	53%	64%	66%
MH6	29%	41%	43%
P3	79%	80%	82%
P13	1578	1625	1700

Source: ANAO analysis of Health data.

Appendix 1 Entity response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether all have been appropriately implemented.

• In February 2023 Health published the Primary Health Network (PHN) 2019–20 Annual Report (paragraph 3.34).
• In March 2023 Health developed a risk management plan for the PHN delivery model (paragraph 2.14).
• In May 2023 a business case to maintain and enhance the PHN Program Electronic Reporting System was submitted, and maintenance was funded (paragraph 3.58).
• In June 2023 Health published the PHN 2020–21 Annual Report (paragraph 3.34).
• In September 2023 Health established a ‘PHN Service Offer’, which described roles and responsibilities across Health for the administration of PHNs (paragraph 2.7).

Appendix 4 Delivery model reviews and internal audits, July 2018 to December 2023

Appendix 5 2022–23 Primary Health Network audits

Table A.3: Status of audit recommendations from individual Primary Health Network audits commenced or completed in 2022–23, at November 2023

Key: ◆ Recommendations fully implemented ▲ Recommendations on track for completion by due date ■ Recommendations behind schedule

Note a: Status is based on the Department of Health and Aged Care’s tracking of recommendations as at 7 November 2023, which the ANAO has not independently verified.

Source: ANAO analysis of draft and final PHN audit reports and the Department of Health and Aged Care’s recommendations tracker.

Appendix 6 Program logics, 2018 and 2023

Figure A.1: Primary Health Network program logic, 2018

Source: Department of Health and Aged Care.

Figure A.2: Primary Health Network program logic, 2023

Source: Department of Health and Aged Care.

Appendix 7 Primary Health Network grant agreement performance measures

Table A.4: Primary Health Network grant agreement performance indicators

Note a: This indicator (and variations of it, for example ‘Activities have been undertaken in accordance with the approved service model as agreed by the Department as appropriate’) is included in all grant agreement schedules, and for all activities. This indicator has been recorded once for each grant agreement in the table to avoid repetition. It appears nine times in the core schedule, including for all trial, pilot and emergency response activities that were included in the core grant agreements for some PHNs.

Note b: Indicator is also used as a delivery model indicator in annual performance reporting (see next section).

Source: ANAO analysis of PHN grant agreements.

Appendix 8 Primary Health Network delivery model performance measures

1. Table A.5 shows performance framework measures as well as the way the results against these measures was reported in three available annual reports. The ANAO notes that there are inconsistencies in reported results across years. The Department of Health and Aged Care advised the ANAO that data for some 2019–20 indicators were revised in the 2020–21 Annual Report ‘to improve the accuracy of the reporting and analysis’.

Appendix 9 Corporate plan measures contributed to by Primary Health Networks

Table A.6: Corporate plan measures contributed to by Primary Health Networks

Source: Department of Health and Aged Care Annual Report 2022–23.

Appendix 10 Evaluations of Primary Health Network activities, pilots and trials

Table A.7: Evaluations of Primary Health Network activities, pilots and trials

Source: ANAO analysis of evaluation reports involving PHNs.

Appendix 11 Reported data for quantitative Performance Framework measures

1. This appendix provides individual Primary Health Network (PHN) and average reported results for 14 out of the 39 measures in the PHN Performance Framework that are quantitative measures and based on PHN-reported data. The 14 quantitative measures include grant agreement management measures. The following tables summarise performance results for each individual PHN against the 14 quantitative measures based on PHN 12-montly reports to the Department of Health and Aged Care (Health). Averages were calculated by the ANAO. Data that was provided for measure DH2 (‘Rate of health care providers using specific digital health systems’) could not be shown due to inconsistencies across PHNs in the type of data supplied.

2. Health advised the ANAO that comparative analysis between PHNs is not meaningful ‘due to differences in contextual factors, such as demographics, per capita of target population, funding provided, and cost of [service delivery] in different environments (e.g. metro, regional, remote etc)’.⁷³

Table A.8: Core (primary health) measures

Note a: There are three PHN regions in Western Australia but they are managed by one PHN organisation (Western Health Alliance). Health did not disaggregate data for each PHN region in Western Australia for P3 and P13 in 2020–21. It reported a combined 91 per cent in 2020–21 for P3, and 11,342 in 2020–21 for P13.

Source: ANAO analysis of PHN reporting.

Table A.9: Alcohol and other drugs measures

Source: ANAO analysis of PHN reporting.

Table A.10: Digital health measures

Note a: PHNs are required to report against indicator DH2 and provide six different figures (the numbers of general practices, pharmacies and allied health service providers using smart forms, e-referrals and telehealth systems). Reporting was not presented as a rate as required and was too inconsistent between PHNs to allow for presentation in this table.

Source: ANAO analysis of PHN reporting.