Introduction

The National Broadband Network and NBN Co Ltd

1.1 The National Broadband Network (NBN) is a major upgrade to Australia’s fixed line telephone and internet infrastructure to provide Australian premises with access to high speed broadband and telephony services.

1.2 The NBN is managed by NBN Co Ltd (NBN Co), a Commonwealth company under the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and, more specifically, a government business enterprise.⁴ To date, the Australian Government has provided NBN Co with contributed equity of $29.5 billion and has entered into a loan agreement for an additional $19.5 billion. NBN Co is accountable to the Australian Government through its shareholder Ministers and provides wholesale NBN telecommunications infrastructure and services to its customers (retail phone and internet service providers (RSPs)) that then sell access to the NBN to end users (the Australian households and businesses using the NBN).

Overview of the migration process

1.3 Maintaining service continuity throughout the migration process from a legacy telecommunication service to the NBN⁵ involves multiple parties delivering interdependent activities. A coordinated approach is required to manage:

• communications regarding the migration process to an area that it is ready to be serviced via the NBN;
• individual connections for premises to the NBN, enabling an active service; and
• resolution of incidents, faults and complaints associated with the connections and activations process.⁶

1.4 NBN Co separates the process of migrating on to the NBN into four broad categories — Aware, Connect, Use, and Fix — that follow the linear chronological experience of migration from the perspective of the end user. NBN Co summarises its primary role within each of the migration phases as the following:

• Aware: end user awareness or understanding of information needed to connect to the NBN and who to contact;
• Connect: the end user experience during the process of being connected to an NBN service;
• Use: the end user experience with regard to expectations on speed and reliability of the NBN connection; and
• Fix: the end user experience where an issue has been reported about a service and parties are not satisfied with the resolution.

1.5 Defined in this way, the migration process does not change irrespective of the technology type being installed (see Appendix 2 for an explanation of the various technology types).⁷ The technology type does however influence the activities required to connect the service to a premises.⁸

Key stakeholders, legislation and policy governing the migration process

1.6 The key stakeholders involved in NBN migration activities and their associated roles are summarised at Table 1.1.

Table 1.1: NBN migration key stakeholders

Source: ANAO summary.

1.7 A detailed explanation of the roles and responsibilities of each stakeholder in the migration process is at Appendix 3.

1.8 Numerous Australian Government and industry-led policies establish how NBN Co and its stakeholders should fulfil their respective roles in the NBN migration process. These are summarised in Table 1.2.

Table 1.2: Legislative and policy context for migrating a premises on to the NBN

Source: ANAO summary.

NBN Co’s role in the migration process

1.9 According to the legislative and policy framework, NBN Co is responsible for specific aspects of the migration process and for the resolution of certain types of disruption to telecommunications service continuity that end users may experience. NBN Co does not have complete control of the end-to-end process for establishing an active NBN service at a premises — elements of the process must also be managed by end users themselves and RSPs, as illustrated in Figure 1.1.

Figure 1.1: Management of access to the NBN, and migration process summary

 

 

Note: This figure is a simplification of the actual arrangements for diagrammatic purposes only.

Source: ANAO adaptation of NBN Co documentation.

1.10 NBN Co’s central role is in facilitating the successful migration of phone and internet telecommunications services at a premises from legacy infrastructure to the NBN. NBN Co must do this in in a way that minimises disruption to the supply of telecommunications services to the end user.¹⁰ NBN Co is also responsible for undertaking public and industry awareness and education activities to support migration on to the NBN in order to maintain continuous access to a telecommunications service.

1.11 Once NBN Co declares that a location is ready for service, both household and business end users typically have 18 months to contact an RSP to indicate they are ready to connect their telecommunications services to the NBN, or to choose an alternate telecommunications infrastructure. The end of this 18 month window is known as the ‘disconnection date’. If the end user chooses to use the NBN, the end user’s RSP will contact NBN Co to request a connection and NBN Co then completes any necessary activities required at the premises to connect and activate a service.

1.12 Once a premises is connected with an active service on the network, NBN Co starts the process of wholesale billing the relevant RSP. An end user may require equipment to be installed (such as a modem or in-house wiring), however these activities are the responsibility of the RSP and the end user. Once a successful NBN service has been established, or the disconnection date has passed, NBN Co engages the owner of the legacy telecommunications infrastructure to arrange for its disconnection.

1.13 Within NBN Co’s span of control, a successful migration from legacy telecommunications infrastructure to the NBN is one where:

• the end user is adequately informed to begin the connection process once the NBN is available at their location;
• NBN Co works with its industry stakeholders to complete a connection within accepted timeframes;¹¹ and
• the connection remains active without any loss to continuous service¹² at the premises for 30 days following a service being activated.¹³

Status of the upgrade and migration process

1.14 As at April 2019, NBN Co reporting suggests that 10,023,604 premises were classified as being ready for service. Of the premises that NBN Co has listed as being ready for service, NBN Co reporting indicates that 88.1 per cent¹⁴ have initiated the process to connect to the NBN. The remaining premises comprise end users who have:

• not yet contacted an RSP to initiate a process of migrating to the NBN;
• contacted an RSP to commence the migration process, but the RSP is yet to contact the NBN Co; or
• elected not to migrate to the NBN at the current point in time.

Audit approach

Audit objective, criteria and scope

1.15 Under the Auditor-General Act 1997, as a Government Business Enterprise, performance audits of NBN Co can only be undertaken at the request of the Joint Committee of Public Accounts and Audit (JCPAA). A request from the JCPAA was received in October 2018 following an approach from the Auditor-General for such a request. The objective of this audit was to assess the effectiveness of NBN Co’s service continuity operations relating to the migration of telecommunication services on to the NBN.

1.16 To form a conclusion against the audit objective, the following high-level criteria were adopted:

• Has NBN Co implemented effective strategies to inform and educate telecommunication end-users about the connection process?
• Does NBN Co have effective processes to manage the risks to service continuity between an end user lodging a request to migrate to the NBN, and the end user having a fully operational telecommunication service?
• Does NBN Co effectively manage end user connection-related complaints and drive improvement?

1.17 The audit scope focused on fixed-line technologies that will support voice and broadband internet telecommunications services for around 93 per cent of Australian premises.

1.18 The audit examined NBN Co’s operations from 16 March 2018 until 28 March 2019 in order to assess its effectiveness against contemporary industry settings. This timeframe is henceforth referred to as ‘the audit timeframe’.

Audit methodology

1.19 The audit methodology included:

• examination of relevant NBN Co records;
• analysis of available and relevant data;
• discussions with relevant NBN Co employees; and
• discussions with NBN Co stakeholders, including NBN delivery partners, retail service providers, the TIO, the ACMA, the ACCC and the DoCA.

1.20 The audit was conducted in accordance with ANAO auditing standards at a cost to the ANAO of $731,492.

1.21 The team members for this audit were Glen Ewers, Renee Hall, Shane Armstrong, Song Khor and Paul Bryant.

2.1 In order to come to a conclusion against this criteria, the ANAO investigated:

• the effectiveness of communication strategies—because NBN Co’s governance structures for communications, the methods and channels utilised, and the alignment of communication practices with the requirements for public information contained in the Public Information on Migration Deed are relevant to evaluating the extent to which end users are informed and educated about the process of connecting to the NBN; and
• processes to monitor the effectiveness of communications—because NBN Co’s measures used to review the effectiveness of communications, and ongoing processes for reporting to key stakeholders, are important in ensuring that communication strategies effectively target the information needs of a diverse group of end users.

Has NBN Co developed effective communications strategies for the connection process?

Communications strategy and governance

2.2 Under the Information on Migration Campaign Deed (Public Information on Migration Deed, or PIM), the Australian Government required NBN Co to undertake a public information campaign in relation to the NBN which would satisfy a set of key principles (the PIM principles). The objective of the campaign is to:

…ensure to the maximum extent practicable that every end-user who is on a service to be migrated to the NBN receives advance notice of the planned migration, is familiar with the action required to migrate and, unless they make a decision that they do not wish to take an NBN service, initiates the necessary migration process prior to the disconnection of the relevant copper or hybrid fibre coaxial (HFC) service to their premises. The campaign will focus on the needs of the end-user to support an early migration and positive experience.

2.3 The PIM principles can be broadly separated into two categories that focus on end user awareness, and NBN Co engagement with the telecommunications industry (as summarised in Table 2.1).¹⁵

Table 2.1: Summary of PIM principles

Source: ANAO summary.

2.4 The NBN Co Corporate Plan 2019-22 outlines NBN Co’s objectives in relation to end user communications as part of the ‘Aware’ stage of the migration process, which is to:

…help Australians better understand what they need to do before signing up to retail services, clarify the role of NBN Co and RSPs, and support end users to get the best out of their broadband service, including education in relation to factors such as speed.¹⁶

2.5 The NBN Co Marketing, Channels and Partnerships (Marketing) business unit has primary accountability for NBN Co’s communications objectives. The Marketing business unit is led by an Executive General Manager reporting to the Chief Customer Officer Residential¹⁷ who reports to the Chief Executive Officer (refer to Appendix 4 for a summary NBN Co organisational chart). Clear responsibilities and performance measures have been established within the Marketing business unit to achieve its communications objectives.

2.6 A communications and marketing strategy has been articulated outlining the key processes and channels NBN Co utilises to undertake communications activities. This strategy is informed by data collected on measures of end user awareness of the NBN, the reach of NBN Co’s communications, as well as NBN connection uptake levels to determine the locations of end users, and those that are yet to make the decision to migrate on to the NBN.

2.7 Paragraph 2.22 outlines NBN Co’s process for the ongoing review of the effectiveness of communications strategies.

Communication methods

2.8 NBN Co communicates with end users directly and through RSPs.

Direct communications with end users

2.9 NBN Co has established direct mail (letters and postcards) and the NBN Co website (including the rollout map¹⁸) as the foundation communication channels for end users seeking information about migrating on to the NBN, with the objective that end users could review the information at a time suitable to them. This base level information is supplemented by broad-scale media activity such as television, radio, social media, and community engagement. NBN Co also communicates with end users via delivery partners and ‘experiential’¹⁹ communications. NBN Co’s multi-channel media approach is designed to ensure that coverage and education processes are nationwide.

2.10 NBN Co communications are pitched generically and apply to as many migration scenarios as possible. NBN Co communications frequently refer end users to their RSP as the entity with which the end user has a direct contractual relationship for service provision, and who are therefore best placed to discuss the end user’s specific circumstances.

2.11 NBN Co uses an external media and advertising agency²⁰ to inform its communications aimed at specific demographics (such as regional areas, culturally diverse communities, and high density apartments). The agency uses demographic analysis and other proprietary tools to determine which geographic areas receive targeted communications, and what channels are most suitable. NBN Co’s direct mail, website, television and radio communications also provide options for readers from a non-English speaking background or who have a disability or impairment to access relevant information. NBN Co reporting indicates that its communications are increasing awareness of the connection process among culturally and linguistically diverse populations. Details on the processes used to measure this awareness are provided in paragraphs 2.22 to 2.30.

2.12 NBN Co forecasts the timing of premises becoming ready for service using a database of information that also determines when to commence sending a series of direct mail communications (or emails for those who register online) about the NBN.²¹ NBN Co sends mail directly to premises from 20 months prior to its disconnection date, through to the time that a connection is activated,²² or two months after the disconnection date has lapsed.²³ Direct mail is sent at specific times in the lead up to the disconnection date for a premises, and varies in messaging depending on how far away this disconnection date is. Up to six individual pieces of direct mail correspondence may be sent over the 20 month period depending on when an end user chooses to initiate migration on to the NBN. Detail on the direct mailing sequence is shown in Appendix 5.

2.13 Since October 2018, letters and postcards distributed by NBN Co have either focussed information on the upcoming disconnection of legacy infrastructure, or on the benefit of connecting to the NBN. Irrespective of which headline messaging is used, all mail reviewed as part of the audit consistently conveyed the need for end users to migrate from legacy infrastructure to avoid being left without telecommunications services, and to contact an RSP to obtain further information.

Communications with end users through RSPs

2.14 The Telecommunications (Consumer Information) Industry Standard 2018 requires that RSPs meet minimum requirements to consistently communicate with end users about the NBN. NBN Co assists RSPs to meet these requirements by providing them with:

• samples of NBN Co communications;
• templates for RSPs and other industry participants to facilitate clear and consistent messaging when communicating with end users;
• data that NBN Co uses to inform when its communications will commence; and
• analysis that monitors how end users are responding to rollout uptake and sentiment.

2.15 NBN Co provides RSPs with visibility of roll-out and marketing activity in target areas on a weekly basis. NBN Co conducts quarterly briefings with RSPs and the Department of Communications and the Arts (DoCA) to discuss NBN Co’s marketing and the impact of this marketing on end user sentiment and behaviour.

2.16 NBN Co has been transparent in its engagement to assist RSPs to meet communication requirements under the Telecommunications Industry Standard.

Alignment of NBN Co communications with PIM requirements

2.17 The PIM outlines a set of principles that NBN Co must communicate to end users regarding the migration process via its public information campaign. Table 2.2 provides a summary of these principles, lists the various NBN Co marketing channels and how each is aligned to address the principles, and shows the ANAO assessment NBN Co’s overall coverage of each PIM principle.

2.18 NBN Co’s communications practices have satisfied the PIM principles. The following opportunities for improvement are noted:

• While NBN Co aligns its direct mail distribution and rollout map information with expected rollout dates for a particular location, it does not provide an explanation of any delays in an area deemed ready for service.²⁴ Analysis of complaints lodged with NBN Co indicated that the second largest category of complaints was ‘Rollout Maps/Timeframe’²⁵ at six per cent. While some complaints may be attributed to end users being unsatisfied with a rollout date when it has not changed, they may also be associated with unreasonable delays. Management of complaints is further discussed in Chapter 4.
• Opportunities exist for NBN Co to further improve the articulation of key roles and responsibilities in the migration process to end users. NBN Co’s direct mail²⁶ and webpage provide high level advice on its role as distinct from that of RSPs, however key industry responsibilities for NBN migration are commonly misunderstood. Direct mail does not explain why NBN Co was sending communications, as opposed to an RSP.

Does NBN Co have a process to review the effectiveness of communications strategies on an ongoing basis?

Policy requirements

2.19 The PIM principles include requirements to establish a process of ongoing monitoring and review of NBN Co communications strategies and activities, as illustrated in Table 2.1. These principles are also reflected in the Migration Assurance Framework, which states that:

NBN Co is responsible for working with community and peak industry body groups to seek input as part of a continuous improvement process focused on maximising service continuity and facilitating a seamless migration experience for customers’.²⁷

2.20 The Migration Assurance Framework further refers to the following:

…as part of a continuous improvement process, the Public Information on Migration Governance Framework allows community, industry and government groups the opportunity to provide feedback on Public Information on Migration activities.²⁸

2.21 The PIM also includes a requirement that:

In developing the [public interest] campaign strategy NBN Co will agree measurable targets with the Commonwealth and Telstra. A baseline awareness level will be identified at the start of the migration process and changes monitored.

Monitoring, review and evaluation of NBN Co communication strategies

2.22 NBN Co’s primary methods for monitoring, reviewing and evaluating the effectiveness of its communication processes are through post-communication take up analysis, customer experience survey activity, and qualitative message testing.

2.23 NBN Co undertakes analysis of the effectiveness of direct mail in supporting end user decisions to migrate to the NBN. This involves reviewing actual uptake against rollout forecasts within 30 days of direct mail being sent to a particular area. NBN Co conducts this analysis on a monthly basis.

2.24 NBN Co conducts customer experience surveys of residential and business end users on a quarterly basis. An external provider is utilised to conduct 20 minute telephone surveys each quarter of: 1,800 residential end users in ready for service areas that are connected, not connected and outside of rollout; 500 business end users that are connected and have not connected; and 400 fixed wireless²⁹ end users that are connected and have not connected. As part of these surveys, NBN Co tests the following:

• understanding: customers knowing the need to switch in order to retain service;
• readiness: customer readiness to connect³⁰;
• intent: customer urgency to connect; and
• sentiment: customer feelings towards the NBN.

2.25 Surveys results, key learnings and proposed actions are shared with RSPs through quarterly briefings. NBN Co survey results during the audit period are illustrated in Figure 2.1.

Figure 2.1: Percentage of NBN Co survey results on end user awareness (a) business and (b) residential

(a) business

(b) residential

Source: ANAO analysis of NBN Co data.

2.26 Figure 2.1 indicates that for both business and residential end users surveyed, a high proportion understood the need to switch in order to retain their telecommunications service. This indicates that NBN Co has satisfied its PIM requirements to educate and inform end users to know the need to switch. The survey results also indicate that end user readiness and urgency to connect remains low. While the readiness and urgency to connect are not requirements under the PIM, NBN Co works with RSPs to monitor and nuance communications in response to negative or inaccurate end user sentiments, with the intention of increasing end user sentiment towards the NBN. NBN Co analyses the possible causes of end user survey results to inform modifications to marketing activity.

2.27 NBN Co also uses market research to measure and calibrate its communications to maximise its appeal and accessibility to end users.

2.28 In response to results from both the customer experience surveys and qualitative message testing, NBN Co identifies and executes strategies, which are aimed at addressing deficiencies in end user understanding, and modifies existing communication processes to better satisfy end user knowledge requirements.

Measuring and reporting on the performance of NBN Co communication strategies

2.29 NBN Co has a series of measurable targets in its annual operating plans to monitor the impact of its communications. Specifically:

• market metrics, with targets for residential and business end users in relation to the understanding, readiness, intent and sentiment data generated from the customer experience surveys, the results of which as shown in Figure 2.1; and
• operational metrics, with targets of less than one per cent for the number of direct mailings that are sent to the incorrect address (‘erroneous mailings’) and less than five per cent for the number of mailings which are not sent out in a timely manner (‘delayed mailings’). NBN Co’s most recent reporting on erroneous delivery of direct mail reflects rates of less than one per cent. NBN Co did not meet its delayed mailing KPI over the same timeframe, and attributed this to a cease in mailing over the Christmas period that resulted in a significant increase in mailing activity in January.

2.30 While NBN Co could not provide documentation to substantiate that it formally agreed these targets with the Australian Government and Telstra, as required by the PIM, performance against these targets is reported quarterly to RSPs (including Telstra) and the DoCA allowing for any issues with the targets to be raised.

3.1 To form a conclusion against this criteria, the ANAO investigated:

• NBN Co processes to prevent risks to service continuity when connecting and activating services to the NBN—because the establishment of clear roles and responsibilities and effective processes and controls for managing NBN Co activities during the connect phase are important elements in maintaining service continuity;
• NBN Co operational performance against targets for resolving faults that occur within 30 days of a new connection—because the timely resolution of any faults arising during the migration process is critical in minimising interruptions to service continuity; and
• how service continuity is integrated into NBN Co’s risk management framework—because the significant volume and scale of NBN Co operations requires that service continuity risks are effectively identified, managed and monitored at an enterprise level.

When connecting and activating services to the NBN, has NBN Co effectively prevented risks to service continuity over which it has influence or control?

Connect phase roles and responsibilities

3.2 Once an end user is aware that their premises is ready for service and contacts a retail service provider (RSP) to access the NBN, NBN Co is responsible for working with its industry stakeholders to complete a connection within accepted timeframes.³¹ For NBN Co, the ‘Connect’ phase of migration is the process of installing NBN Co equipment to connect an end user’s premises to the NBN infrastructure and activating the service. The newly connected and activated service is then linked to the end user’s RSP.

3.3 Within NBN Co’s span of control, a successful migration has been defined for the purposes of this audit to be one where the service remains active without any loss to continuous supply of telecommunications³² at the premises for 30 days after it has been activated. Disruption or loss of service during this timeframe may include:

• when a connection to the NBN is attempted and is unsuccessful;
• during a network outage; or
• when a service fault occurs.

3.4 Once a premises has an active NBN service, the ‘Use’ phase of migration commences when NBN Co has notified the RSP of an operational NBN service and the service is available to the end user. Faults that NBN Co manage during the ‘Use’ phase are discussed in paragraphs 3.17 to 3.47.

Key industry standard requirements

3.5 The Telecommunications (NBN Service Continuity) Industry Standard 2018 (Service Continuity Industry Standard) sets out a number of responsibilities for carriers, such as NBN Co, and service providers to prevent an end user experiencing a disruption of their telecommunications service during the ‘Connect’ phase. Under section 8 of the Service Continuity Industry Standard, NBN Co ‘must take all reasonable steps to manage the migration at a premises in a way that minimises disruption to the continuous supply of carriage services to the consumer’.

3.6 Under subsections 8(2)(a) and (b) of the Service Continuity Industry Standard, NBN Co must also:

• not advise an RSP or owner of the legacy telecommunications infrastructure that the migration at the end users premises is complete until it has taken all reasonable steps to ensure that successful migration at the premises has occurred.
• where the migration at the end users premises has been unsuccessful, take all reasonable steps to notify, within one working day, the end users’ RSP that migration to the NBN service has not been successful.

NBN Co processes and controls for managing service continuity

General processes and controls

3.7 Once serviceability and specifications of an order at a particular premises is known, the NBN Co Customer Connections Business Unit coordinates with its delivery partners to install required equipment,³³ and establish a connection at the location.

3.8 A series of service classes are used by NBN Co to categorise individual premises based on the serviceability and the equipment requirements for connection. NBN Co defines 34 service classes, of which 28 are related to the fixed line technologies within the scope of this audit (see Appendix 2). For a newly connected premises, NBN Co is also responsible for activation of the service, and when NBN Co confirms an active service for a location it is linked to an RSP for billing purposes.³⁴

3.9 NBN Co has established clear, detailed procedures and operational structures to facilitate the process of connecting end users to the NBN. Specifically to:

• validate connections orders that it receives from the RSP;
• complete NBN connections, including the resolution of any issues by the technician required before a connection can be activated; and
• run tests while on site, with controls in place to ensure that work tickets are not closed without passing relevant checks.

Notifications to RSPs of an unsuccessful NBN connection

3.10 The Service Continuity Industry Standard took effect from 21 September 2018. Subsection 8(2)(b) of the Service Continuity Industry Standard (discussed in paragraph 3.6) establishes a requirement that where an NBN migration at a premises has been unsuccessful, NBN Co must notify the end user’s RSP within one working day. This notification process aims to manage the risk of service disruption at an end user’s premises by enabling NBN Co and the RSP to coordinate with the end user in a timely way, as well as determine the most appropriate action when an unsuccessful migration appointment occurs.³⁵

3.11 From January to April 2019, NBN Co developed a formal framework for managing compliance with the Service Continuity Industry Standard. Procedures were also updated to reflect implementation of a new notification for when an unsuccessful migration attempt warrants NBN Co action. A dedicated compliance monitoring team was established in May 2019 with responsibility for conducting ongoing testing of compliance with the Service Continuity Industry Standard. However, this team only measures compliance in relation to RSP notifications where NBN Co is directly responsible for any subsequent action following an unsuccessful appointment. This means that the following notification types are not assessed by NBN Co as part of its compliance review process:

• notifications that inform an RSP that RSP action is required after an unsuccessful appointment; and
• notifications that clarify responsibilities following an outstanding appointment notification.³⁶

3.12 While NBN Co’s procedures reflect the need for these notifications to be issued, NBN Co should revise the testing methodology used by the Service Continuity Industry Standard assurance team to ensure that there is appropriate monitoring of compliance with timeliness requirements for these notifications.

Controls to manage network events and outages during the connection process

3.13 Scheduled or unscheduled network outages may cause an interruption to an NBN service, and this may occur during the time that an end user is being migrated from legacy infrastructure on to the NBN. Under the Wholesale Broadband Agreement (WBA)³⁷, NBN Co is required to communicate scheduled outages for maintenance and assurance activity to RSPs in advance, and this is undertaken through outage activity notifications. During the audit timeframe, the volume of service faults recorded that were a result of an outage within 30 days of an activated service were low (approximately 1.7 per cent of all service faults).

3.14 NBN Co has stated that it plans to further increase its capacity to understand the impact of network events on particular premises during migration via post connection reports and through conducting live health checks³⁸ on a service.

General governance, reporting and communication processes with RSPs

3.15 Under the WBA, NBN Co is required to conduct governance meetings and performance reporting³⁹ with RSPs on a monthly basis. NBN Co engagement with 20 RSPs reviewed during the audit timeframe indicated inconsistency in the extent to which records were maintained of governance meetings and outcomes.⁴⁰ NBN Co stated that it offers the same governance approach to all RSPs, however the structure and timing of meetings and associated processes is then mutually determined with each individual RSP. Examination of RSP reporting indicates that NBN Co provides sufficient information to support monitoring and discussion of any performance issues and proposed improvement measures as they are necessary. Meeting records that were available indicate that representatives from the NBN Co connections and assurance areas attend all meetings. Specific issues relating to the management of performance for particular technologies, connection and activation timeliness, appointment scheduling and delivery partner capacity were regularly discussed, including:

• faults closed per 100 active premises;
• ‘no fault found’ volumes and trends;
• standard connections timeliness, causes of lower performance, and opportunities to raise performance levels; and
• priority assistance connections within agreed timeframes.

3.16 NBN Co has governance arrangements in place with RSPs in accordance with the WBA, and had more frequent and detailed performance discussions with those RSPs that have the most NBN market share. NBN Co reporting suggests that its performance discussions with RSPs include statistics on fault volumes and action taken to address them, and for some RSPs, analytics of what is contributing to missed or achieved service level agreements.

Does NBN Co resolve service faults associated with a newly connected NBN service in an effective and timely manner?

Service fault volumes and trends

3.17 The ‘Use’ phase begins when an NBN service has been connected and activated by NBN Co. When an end user experiences a loss of service during this phase, the circumstances are raised by the RSP and logged with NBN Co as a type of incident,⁴¹ otherwise known as a service fault. A total of 1.22 million service faults were recorded by NBN Co in the audit timeframe.⁴²

3.18 Service faults that occur within 30 days of a new connection to the NBN were the focus of analysis in this chapter,⁴³ as they can indicate how well the connection has been installed and whether the ‘Connect’ phase of the migration was successful.⁴⁴ Of the 1.22 million service faults in the audit timeframe, 127,464 (approximately 10 per cent)⁴⁵ occurred within 30 days of a newly connected service.

3.19 Service fault types can be divided into:

• disruption faults—those faults where there is no functionality in the NBN service itself for a period of time, specifically: new service never worked,⁴⁶ no data, intermittent service/dropouts, and no sync faults;⁴⁷ and
• non-disruption faults—those faults where the NBN service is functional, but the quality of the service is impacted, specifically: faults related to speed, speed degradation and performance latency.

3.20 The average weekly volume of new connections over the audit timeframe (16 March 2018 to 28 March 2019) was 40,353. Figure 3.1 illustrates the volume of faults occurring within 30 days of a newly connected service over the same period, divided into disruption and non-disruption categories.

Figure 3.1: Volume of service faults within 30 days of newly connected services

 

 

Notes: The records for service faults (recorded as customer incidents in NBN Co data) included those for priority assistance services. Priority assistance services are explained in Table 3.3.



The disruption faults are for all fixed line technology types.



Trend line is for visual representation of general trend only.

Source: ANAO analysis of NBN Co data.

3.21 Figure 3.1 reflects the following:

• There is a general downward trend in the volume of disruption faults within 30 days of newly connected services. This is a positive result given there was a general increase in the volume of weekly new connection activity over the same period (from 37,443 new connections in the week ending 22 March 2018 to 44,605 new connections in the week ending 28 March 2019 (refer to Figure 4.3).
• Disruption faults form the vast majority of service faults that occur within 30 days of a new connection.

Fault management process

3.22 NBN Co has established the WBA operations manual, which describes the procedures that apply to the operational aspects of the supply of products by NBN Co to each RSP under the WBA. Responsibilities and interactions between NBN Co and RSPs for resolving faults⁴⁸ that occur on an NBN service once it has been connected are clearly outlined in this document.⁴⁹

3.23 A service portal and a ‘Business-to-Business’ interface have been established, and are the two key avenues for NBN Co to share information and facilitate assurance activities over the connection process with RSPs.⁵⁰ In the first instance, the RSP must perform test and diagnostic tasks to identify whether a fault is a service fault. If an RSP determines after diagnostic testing that the issue is likely to be a service fault, it raises a ‘trouble ticket’⁵¹ with NBN Co for resolution. In response, NBN Co will diagnose, report, investigate and repair the fault through rectification or remediation, (see Figure 3.2).

Figure 3.2: End user service fault management process

 

 

Source: ANAO analysis of NBN Co documentation.

Fault diagnosis

3.24 Table 3.1 shows NBN Co diagnosis of disruption fault types within 30 days of a newly connected service over the audit timeframe.

Table 3.1: Disruption fault types within 30 days of a newly connected service

Note a: ‘No sync’ refers to circumstances where there is a problem in the premises which impact on connectivity with the NBN network (for example, a fault with the modem, or the wiring within the premises itself).

Notes: There were 378 service faults in the NBN Co dataset with a status of ‘cancelled’ that contained no categorisation for fault type.

Table 3.1 represents service disruption faults only, which is a proportion of the 127,464 total service faults within 30 days of a new connection shown in Figure 3.1.

Percentage totals may not add to 100 per cent due to rounding.

Source: ANAO analysis of NBN Co data.

3.25 Disruption fault types ‘no data’, ‘intermittent service/dropouts’ and ‘no sync’ relate to situations where the NBN Co connection process has occurred, however functionality of the service was then lost for a period of time. When an end user reports that their service has had no functionality since the connection and activation process occurred, it is classified as a ‘new service never worked’ fault type, and potentially reflects that the NBN connection process has not been effectively undertaken.⁵²

3.26 Within the audit timeframe, NBN Co internal reporting for all technology types⁵³ stated that approximately three per cent of new connections did not work in the first 30 days, against a KPI of 2.7 per cent as illustrated in Figure 3.3.

Figure 3.3: Trend in ‘new service never worked’ faults within 30 days of a new connection

 

 

Source: ANAO analysis of NBN Co data.

3.27 Figure 3.3 shows that, while the rate of new service never worked faults as a proportion of new connection activity is above NBN Co’s long term KPIs, this rate shows improvement in the five months to March 2019.

3.28 A total of 43,187 new service never worked fault types were recorded by NBN Co within the audit period. Figure 3.4 illustrates this fault type by technology type (service category).

Figure 3.4: Volumes by technology type of (a) NBN connections activity; and (b) new service never worked faults

(a) Volumes of new connections by technology type

(b) Volumes of new service never worked faults

Source: ANAO analysis of NBN Co data.

3.29 Figure 3.4(a) shows that the largest volume of new connections are in the ‘Fibre to the Basement’ and ‘Fibre to the Node’ (FTTPB/N) technology types, and Figure 3.4(b) shows that these generate the majority of new service never worked faults. Figure 3.4(b) shows that, following an initial increase, the volume of new service never worked faults for the FTTB/N technology types decreased over the audit period, although it is unclear whether this is driven by a more general decrease in the volume of new connections in these technology types. Figure 3.4 does show that, despite a general increase in the volume of connection activity for Hybrid Fibre Coaxial (HFC) connections, the volume of new service never worked faults remained low.

3.30 Although new service never worked faults are initially received by NBN Co’s assurance area, once the fault type is confirmed, it is managed by the connections area to complete the connection again. The metrics on this scenario are reported internally as ‘right second time’. NBN Co operational reporting in May 2019 indicates that it had achieved a financial year to date outcome of 98 per cent against a target of 97⁵⁴ per cent for its ‘right second time’ connection metric.

Fault resolution

3.31 As illustrated in Figure 3.2, outcomes of NBN Co’s fault resolution process can result in it being classified as being ‘no fault found’, a requirement for a technician work order (otherwise known as a ‘truck roll’ to the premises), or a remediation. For all service faults within 30 days of a new connection during the audit timeframe, 30.5 per cent resulted in ‘no fault found’, as shown in Table 3.2.

Table 3.2: Outcomes of all service faults within 30 days of a new connection

Note a: No fault found can be as a result of line testing, where NBN Co determines the service is working to specification, or after a technician has been to a premises and confirmed a fully operational service. In these instances the reported fault may be due to the end user.

Note b: These faults outcomes indicate action by the RSP or are related to RSP coordination with the end user. These resolution categories are labelled as ‘AS’ indicating access seeker, or ‘3rd party’. Examples of these resolution categories are: ‘AS – No Response Received’; ‘AS – End User Unavailable’; ‘3rd Party – Remote Configure’ and ‘3rd Party – Fault Fix’.

Note c: These service faults have been determined by NBN Co to be related to a planned outage or resolved as a network fault.

Note d: Examples of other service fault resolution categories in NBN Co data include those labelled with ‘restoration’, ‘blank’, ‘potential fault cleared’ or ‘other fix’.

Note: Percentage totals may not add to 100 per cent due to rounding.

Source: ANAO analysis of NBN Co data.

3.32 Table 3.2 shows that although the majority of faults received relate to service disruption, a large combined proportion (37.7 per cent) are related to RSPs or associated issues with end users, and are therefore not within NBN Co’s direct influence. Once NBN Co has resolved a fault, the outcome is communicated to RSPs to manage with the end user.

Timeliness to resolve

3.33 Timeliness for resolving faults is agreed in the WBA and determined by:

• premises location (specifically remoteness);
• technology type;⁵⁵ and
• the category of end user connection.

3.34 The agreed timeliness for service fault rectification in the WBA is based on these factors, and range from one to three days.

3.35 The two categories of end user connections defined in the WBA are ‘Standard’ and ‘Priority Assistance’ services, as explained in Table 3.3.

Table 3.3: End user connections categories

Note a: Service transfers occur at premises already connected to the network where an end user changes their preferred RSP. Service transfers are not migrating an end user off a legacy service to connect to the NBN.

Note b: Accelerated connections are for premises that are already connected to the network but inactive for the end user and the RSP has proposed to provide a standard telephone service. If the premises has previously been migrated to the network, RSPs should not order accelerated connections unless the order meets the required conditions.

Source: NBN Co documentation.

Resolving standard service faults

3.36 The WBA establishes that the KPI for faults on standard connections for fixed line technologies is 90 per cent or more will be resolved within agreed service level timeframes (i.e. between one and three days). Operational metrics on faults information are reported weekly to the NBN Co executive.⁵⁶ For the proportion of all service faults that were ‘resolved within WBA SLA due date’, NBN Co reported an average result of 89 per cent for the audit timeframe.

3.37 The method NBN Co uses to report on fault resolution timeliness excludes time associated with matters outside of its control (for example, where a technician attends an end user premises, yet the end user misses the appointment). The end user experience of timeliness to resolve a fault may therefore be longer than what is calculated and reported by NBN Co.

3.38 In relation to the end user’s experience, the average elapsed work days taken to resolve faults relating to service disruption within 30 days of a newly connected standard service⁵⁷ is shown in Table 3.4.

Table 3.4: Average elapsed work days for resolving service disruption faults for a standard connection from an end user perspective

Note: HFC technology is not available outside urban geographies.

Source: ANAO analysis of NBN Co data.

3.39 Table 3.4 indicates that there is no substantial variation in the average elapsed work days taken to resolve service disruption faults based on technology type or the geographic location of a premises. The average time experienced by the end user for having a service disruption fault resolved (shown in Table 3.4) is 3.1 working days, and 80 per cent of these faults were resolved in four days or less. A breakdown of fault resolution volumes over time by geography and technology is at Appendix 6.

Resolving priority assistance service faults

3.40 The WBA KPI for faults on priority assistance services for fixed line technologies is that 100 per cent will be resolved within agreed service level timeframes, which range from 24 to 48 hours.⁵⁸

3.41 NBN Co internal reporting covering the audit timeframe indicated that the reported average result for the resolution of priority assistance service faults within agreed service level timeframes was 95 per cent against a target of 100 per cent.

3.42 For the audit period, only 1.8 per cent of priority assistance connections resulted in a service fault.⁵⁹ On average, this small number of faults were resolved within the next business day.

3.43 NBN Co introduced an initiative in November 2018 for priority assistance connections to improve its performance.⁶⁰ The initiative involves assurance measures such as line testing the service each day post-activation for 30 days. The introduction of the initiative has improved NBN Co’s performance towards its 100 per cent KPI for resolving faults relating to priority assistance services.⁶¹ Individual case management and high level handling of priority assistance connections orders have also contributed to the improvement of NBN Co’s performance against this measure.⁶²

Continuous improvement

3.44 NBN Co is implementing an assurance improvement and maturity program that has included the introduction of a dedicated area for resolving problematic incidents, and has a current focus on reducing the number of ‘truck rolls’⁶³. The components and progress of this program is summarised in Table 3.5.

Table 3.5: Assurance improvement program components

Note a: Pattern management is an advanced operations area that analyses the network and faults for patterns where similar faults may be occurring. By conducting root cause analysis NBN Co intend to facilitate the reduction of unnecessary truck rolls. NBN Co advised that pattern management is intended to become a formal program of work for NBN Co in 2020.

Source: NBN Co documentation.

3.45 The case management function was created in December 2018 to address problematic or repeat incidents, including ‘aged incidents’ which are classified as faults that have not been resolved after seven days. NBN Co operational reporting on aged incidents does not distinguish if the incident relates to service faults, network faults or were particularly related to speed or disruption to service. NBN Co reported that for the 12 weeks from 14 February to 25 April 2019, it was managing approximately 1,100 aged incidents per week.

3.46 Reducing the number of truck rolls required for resolving faults is important for efficiency, expenditure and impact on the end user. NBN Co has a current focus on reducing the number of truck rolls, particularly those that result in ‘no fault found’. It is noted however that NBN Co executive reporting on 6 June 2019 indicated that reducing truck rolls relating to faults within 10 days of activation is a priority for NBN Co to improve.

3.47 Metrics for repeat incidents within 30 days of a new connection and repeat incidents within 30 days of a new connection that also required a truck roll are included in operational monitoring. From February 2019 to April 2019, NBN Co operational reporting of 12 week averages indicates that repeat incidents in 30 days (two or more faults) occurs for 18 per cent of locations reporting a fault in the first instance; a KPI has not been set for this metric. For the same quarterly time period, NBN Co also reports that repeat incidents that also required a truck roll was six per cent of locations reporting a fault in 30 days, which is in line with NBN Co’s current target for this metric.

Does NBN Co’s risk management framework support NBN Co operations for management of service continuity?

3.48 The Public Governance, Performance and Accountability Act 2013 (PGPA Act) is a principles-based Act⁶⁴ supported by the Commonwealth Risk Management Policy (CRMP), which aims to strengthen the risk management practices of Australian Government entities. Entities subject to the CRMP must ensure that systematic management of risk is embedded in key business processes.

3.49 It is not mandatory for Commonwealth companies such as NBN Co to comply with the CRMP. However ‘all GBEs are encouraged to consider applying the CRMP in relation to risk management frameworks and systems’⁶⁵.

3.50 In accordance with the CRMP, NBN Co reviews and maintains:

• a current risk management framework and policy which has been approved by the NBN Co Board;
• risk appetite and tolerance statements; and
• risk registers identifying operational controls, procedures and residual risk.

3.51 The Group Risk team is responsible for maintaining this framework across NBN Co, and reports to the Audit and Risk Committee every six months on the risk profile and status of risk management activities.

3.52 In August 2018, NBN Co began transitioning business function risks from spreadsheet-based risk register to a centralised enterprise level risk management system as part of a risk management and capability improvement plan.⁶⁶ In June 2019, 226 of the 493 risks included in NBN Co’s central risk register were ‘approved’, indicating that the risk was discussed and agreed by the risk manager, risk owner and risk coordinator in a business unit. However, the 226 ‘approved’ risks in the risk register included:

• 38 with no controls identified;
• 118 with no associated treatment; and
• 18 with no controls, or treatments.⁶⁷

3.53 The ANAO identified 75 risks in NBN Co’s central risk register which could broadly be considered as relating to the maintenance of service continuity throughout the migration process (as described in paragraph 1.3).⁶⁸ These are summarised in Table 3.6.

Table 3.6: Central risk register and disruption to service continuity

Note: The status of other risks in the register were: ‘awaiting review’, ‘in progress’ or’ blank’.

Source: ANAO analysis of NBN Co documentation.

3.54 The identified risks did not cover all aspects of disruption to service continuity that have been discussed in this chapter, for example there is no reference to:

• a delayed, or missing notification by NBN Co to an RSP of an unsuccessful attempt to connect an end user’s premises to the NBN;
• service fault volumes, and their impact on end user experience; or
• how service fault resolution timeliness impacts on end user experience.

3.55 NBN Co’s central risk management system does not provide historical reports and therefore did not enable review of historical risk registration or status changes in its system. ⁶⁹

3.56 Operational plans and risk registers for two key business units relating to service continuity—Customer Connections and Customer Service and Assurance (see NBN Co organisational chart in Appendix 4)—were reviewed for July 2018 and February 2019 respectively. This review identified that:

• the majority of the risks in the central register relating to the Customer Service and Assurance business area had no controls identified;⁷⁰ and
• risks are not systematically linked to operational risk and plans.⁷¹

3.57 As outlined in this chapter, NBN Co has embedded a number of key risk management practices into operational practices associated with service continuity, in particular through:

• the implementation of contractual arrangements with delivery partners for service standards;⁷²
• engagement with RSPs in accordance with the WBA; and
• NBN Co initiatives in response to monitoring and reporting of its business functions activities.

There is a need, however, for these practices to be formally reflected and monitored in NBN Co’s enterprise risk management framework.

4.1 In order to come to a conclusion against this criteria, the ANAO investigated:

• NBN Co complaints handling processes—as the establishment of clear complaints management roles and responsibilities, the provision of public guidance on the lodgement of complaints, and the maintenance of appropriate frameworks for the receipt and handling of complaints are critical elements in the effective management of end user connection-related complaints by NBN Co;
• the timeliness of connection-related complaints resolution—as the categorisation of complaints, and their resolution within NBN Co’s target timeframes, are important in ensuring that complaints management processes are supporting effective performance; and
• analysis and reporting of complaints data—as NBN Co processes to analyse the causes of complaints, to monitor performance in the resolution of complaints, and to use this information to drive continuous improvement are critical to ensuring the ongoing effectiveness of complaints management processes.

4.2 The audit assessed NBN Co alignment with the Telecommunications (Consumer Complaints Handling) Industry Standard 2018 (Complaints Industry Standard), given its responsibilities to provide reasonable assistance to RSPs, and to receive complaints as part of its public-facing role. However, it is recognised that NBN Co is not required to handle complaints in accordance with all parts of the Complaints Industry Standard.

Has NBN Co established an effective complaints management process?

Complaints management roles and responsibilities

4.3 The Migration Assurance Framework states:

[NBN Co] is a wholesale infrastructure provider that sells its products to retail service providers, who then add additional functionality and provide retail services to end users.

If a customer has a concern regarding the installation or operation of their NBN services, they should contact their retail service provider first, as they have the main relationship with the customer as well as with [NBN Co] and are best placed to provide assistance or resolve a query or complaint.⁷³

4.4 This statement is reflective of the Complaints Industry Standard, which also places responsibility for NBN complaints management predominantly with RSPs. However, Part 6 of the Complaints Industry Standard requires NBN Co to assist RSPs in the resolution of complaints through the provision of ‘reasonable assistance’ when the complaint relates to matters within NBN Co’s span of responsibility. While it is the intention of both the Migration Assurance Framework and the Industry Standard that RSPs act as the first point of contact for complaints, it is nonetheless appropriate that NBN Co establish policies and procedures for managing complaints given its responsibilities in the provision of reasonable assistance to RSPs, and that it receives complaints as part of its public-facing role.

4.5 Processes within NBN Co for the provision of information to end users on the lodgement of complaints, and for the receipt and management of complaints, are the responsibility of the Enquiries and Complaints team within the Customer Service and Assurance business unit (refer to Appendix 4 for a high level NBN Co organisational chart).

4.6 NBN Co’s complaints management and handling processes⁷⁴ are articulated in three key documents — the Complaint Management Policy, the Case Management Handbook, and the Customer Resolutions Team Agent Handbook. The Complaint Management Policy is a public-facing, end user focused document, and the Case Management Handbook and Customer Resolutions Team Agent Handbook are internal guidance documents for NBN Co complaint handling and case management staff.

Public guidance for the lodgement of complaints by end users

NBN Co Complaint Management Policy

4.7 As noted above, the intent of both the Migration Assurance Framework and the Complaints Industry Standard is to make RSPs the focal point for end user complaints. While NBN Co is therefore not bound by Parts 2 to 5 of the Complaints Industry Standard, the standard lists a useful set of information elements that should be provided to the general public about the complaints process. The Complaint Management Policy is NBN Co’s primary source of information to the public about making a complaint. The policy includes contact information to make a complaint via telephone, email, NBN Co’s website and fax. It also states that ‘[As NBN Co] is a wholesale only provider, complainants are encouraged to seek to resolve issues directly with their access seeker or telecommunications retail service provider’.⁷⁵

4.8 The current version of the Complaint Management Policy came into effect in July 2015, and was initially drafted in 2012 in consultation with the Telecommunications Industry Ombudsman (TIO) and the Australian Communications and Media Authority (ACMA). The requirements of the Telecommunications Consumer Protections Code (TCP Code) were considered as part of the policy development process, having regard to the fact that the TCP Code applied to RSPs at that time.

4.9 In November 2017, NBN Co completed a review and drafted a revised Complaint Management Policy for noting by the Board. The ANAO was advised that after the review had been completed and the new Complaint Management Policy had been drafted it ‘was not submitted for final approval’, and ‘the appropriate level of approval…was not obtained’ for the new version of the policy to be signed off by the appropriate officer, noted by the Board, and published. As a result, the 2017 draft version of the Complaint Management Policy was not put into effect and the July 2015 version remains the current version available to the public on the NBN Co website.

4.10 In September 2018, the TCP Code was replaced by the Complaints Industry Standard in response to problems with complaint handing by RSPs identified in a 2018 report by the ACMA.⁷⁶ NBN Co’s complaints management policy is therefore inconsistent with current telecommunications industry regulations. While NBN Co has provided updated guidance to RSPs on how reasonable assistance requests should be submitted, Table 4.1 lists the Complaints Industry Standard elements that are not addressed by the current (July 2015) NBN Co Complaints Management Policy.

Table 4.1: Complaints Industry Standard elements not addressed by NBN Co Complaints Management Policy⁷⁷

Source: ANAO Analysis.

4.11 NBN Co reporting to the Board indicates that the next review of the Complaints Management Policy is planned for November 2019.

General information to support end users in making a complaint

4.12 The Complaints Industry Standard states that:

A carriage service provider must ensure that information about how a consumer can contact the carriage service provider to make a complaint or enquiry can be accessed via a link on the homepage for its website.⁸²

4.13 As previously stated, while NBN Co is not compelled to comply with this requirement, it is important that clear information is made appropriately available to consumers on the lodgement of complaints.

4.14 The Complaint Management Policy document is not readily available to the public. There is no link to the Complaint Management Policy document from the NBN Co homepage, and the utilisation of the NBN Co website’s search function using the keywords ‘complaints’ and ‘complaints management policy’ respectively does not have any result. The policy document is located within a sub-folder of the NBN Co website,⁸³ however this can only be found via a specific keyword search in an external search engine.

4.15 The NBN Co website has a ‘Contact Us’ page, and the first result when utilising the website’s search function with the keywords / phrases ‘complaints’ and ‘how to complain’ is a link to the Contact Us page.⁸⁴

4.16 The Contact Us page has a list of common topics of enquiry, many of which may be standard topics of complaint (e.g. ‘Your address is missing or incorrect’). The Contact Us page also has a clear listing of matters / issues which an end user should contact their RSP for (e.g. ‘Concerns about speed’) versus those which they should contact NBN Co for (e.g. ‘Damages to property or [NBN Co] equipment’). The Contact Us page is not clear, however, on how and where to lodge a complaint. Two references to ‘complaints’ on the Contact Us webpage are not about the general process for making a complaint, but are instead specifically about NBN Co’s Privacy Policy and how an end user can complain if they believe the Australian Privacy Principles have been breached. The webpage links to a general inquiries web form titled ‘Didn’t find your answer? Complete the form below’, which is a way an end user can make a complaint online, though the end user must select from a drop down menu to determine that the form can be used to make a complaint. Further, the Contact Us webpage does not include the complaint-related contact details contained in the Complaint Management Policy, previously identified in paragraph 4.7.

4.17 NBN Co’s direct mail to end users is another channel which could be used to clarify how to lodge a complaint. NBN Co use only one letter in its direct mail series to end users to define its role and that of the RSP (‘Letter sent 16 months before disconnection date’ — refer Appendix 5). Under the list of responsibilities for the RSP identified in the letter, NBN Co include the statement ‘resolves any issues with your connection’, which is intended to direct end users to the RSP as their first point of contact for complaints.

Framework for the receipt and management of complaints by NBN Co

Complaint management guidance and procedures

4.22 The NBN Co Complaints and Enquiries team is responsible for the receipt and management of all customer complaints. The primary complaint handling guidance document supporting the team is the Case Management Handbook. This document:

…outlines the fundamental steps taken to manage [NBN Co] complaints. It describes the overarching requirements of the case management process and the stages of a complaint as it progresses through [NBN Co’s] complaint management system.⁸⁵

4.23 An additional document, the Customer Resolutions Team — Agent Handbook, provides guidance for case management staff where a complaint cannot be resolved at the first point of contact. Staff are also provided with:

• a complaints matrix to enable the appropriate triage of complaints⁸⁶;
• e-mail templates and scripts for phone conversations;
• procedures and associated guidance documentation; and
• quick reference guides.

4.24 These documents provide references to further related documents and templates, and information on document control.

4.25 NBN Co’s complaint handling process includes activities for recording and allocating complaints for resolution by subject matter expert teams, as illustrated in Figure 4.1.

Figure 4.1: NBN Co’s complaint handling process

 

 

Source: ANAO analysis.

4.26 A Customer Enquiries Quality Framework and a Case Management Compliance to Process procedure have been established, and case management teams are subject to regular compliance reviews to ensure they are managing complaints to the appropriate internal standards.⁸⁷ The results from these reviews are reported to case management teams on a quarterly basis. NBN Co stated that these reports are used to identify performance development opportunities, but was unable to substantiate how review results were reported at the business unit management level or how they were used to drive continuous improvement.

Complaint management system

4.27 Complaints made to NBN Co are logged and monitored in Mosaic—NBN Co’s complaint management system (CMS). NBN Co staff categorise the complaint types as ‘Formal’, ‘Complex’ or ‘Urgent’ using the Complaint Management Policy and Complaints Matrix (refer Table 4.2 for detailed explanation of these categories). Complaints are then further categorised according to:

• Phase: where in the ‘customer journey’ the complaint event occurred, such as Information, Planning, Build, Connection, Post-Connection, General, or Uncategorised,
• Category: the technology type, such as Fibre to the Node (FTTN), or Hybrid Fibre Coaxial (HFC); and
• Subcategory descriptor: to best describe the complaint subject, such as Equipment Not Working or Connection Quality/Speed.

Staff also log the subject of the complaint and a description using free text fields.

4.28 Accuracy and consistency in the logging and categorisation of complaint data in the CMS is important in ensuring the validity of NBN Co’s internal reporting on complaint data, which in turn supports root cause analysis and continuous improvement (refer to paragraph 4.47 to 4.50 for detailed analysis of continuous improvement processes).

4.29 Review of complaints across the audit period identified that structures are largely in place to ensure that data has been accurately captured by staff in a manner which is in line with NBN Co policy. However, the following ambiguities in structures to ensure the accurate classification of complaints were noted:

• Data does not clearly isolate connection-related complaints by Phase. Examples of connection-related complaints could be found in the Connection, Post-Connection, General, or Uncategorised phases.
• Data shows multiple, overlapping entries for technology types in the Category fields, which can create errors when analysing complaints according to technology. For example, a complaint about a service on Fibre to the Node could be classified in Fibre — Brownfields FTTN — Brownfields, FTTN — Greenfields, or FTTN.⁸⁸
• Utilisation of free text fields for complaint record data entry:
  • While there is a need for free text to capture the unique circumstances of each complaint, and free text fields are a common feature in a CMS, these fields are used by NBN Co inconsistently — some records include detailed information, others very little and on very few occasions the field is left blank.
  • The ability to continually edit a free text field provides flexibility allowing a staff member to work on a complaint they may be unfamiliar with (something NBN Co identifies as important), however this flexibility also makes it possible for relevant data to be deleted without a corresponding audit trail.

4.30 The potential impact of these ambiguities is that NBN Co analysis and reporting on complaints data may not accurately capture all complaints associated with a particular connection phase, or technology type; and individual records may not contain sufficient detail to understand the nature of the complaint, and therefore develop appropriate responses to address systemic issues.

4.31 NBN Co should revise its complaints management guidance to address these issues, and ensure that the regular compliance review process includes appropriate coverage of these items.

Are connection-related complaints resolved in a timely manner?

General complaint types and volumes

4.32 Both the Complaint Management Policy and Case Management Handbook define complaint types and resolution timeframe Key Performance Indicators (KPIs) as shown in Table 4.2.

Table 4.2: Complaint types and resolution timeframe KPIs

Source: NBN Co, Complaint Management Policy and Case Management Handbook.

4.33 The Operating Plan for the Customer Service and Assurance Business Unit, which is responsible for complaints management, has an 2019 target that 93 per cent of combined formal and complex complaints will be resolved within their respective KPIs (i.e. five or 20 working days) for these complaint types.

4.34 Within the audit timeframe, NBN Co logged 173,601 end user complaints. Figure 4.2 shows this volume of complaints according to complaint type.

Figure 4.2: Total complaint volumes during the audit time frame

Source: ANAO analysis of NBN Co data.

Connection-related complaints — types and volumes

4.35 Figure 4.3 shows the trend in the weekly volume of new NBN connections during the audit timeframe.

Figure 4.3: Weekly volume of new NBN connections

Note: Service class 4 (wireless) and 9 (satellite) were not represented in the NBN Co data.

Source: ANAO analysis of NBN Co data.

4.36 Of the 173,601 total complaints recorded by NBN Co within the audit timeframe, 34,196 (approximately 20 per cent) were attributed by NBN Co as being related to new connections.⁸⁹ Figure 4.4 shows the trend in connection-related complaints during the audit timeframe.

Figure 4.4: Connection-related complaint volumes during the audit timeframe

Source: ANAO analysis of NBN Co data.

4.37 Figure 4.4 reflects:

• a decreasing trend in the volume of Formal connection-related complaints, from a peak in the January 2019 (this peak correlates with a rise in the volume of weekly connections during the same period—refer Figure 4.3); and
• an increasing trend in the volume of Complex connection-related complaints toward the end of 2019—it is unclear what is specifically driving this, however NBN Co has stated that it expects to see an increase in complaint volume and complexity as the rollout, build and connect phases continue to grow, reflected in Figure 4.3.

Connection-related complaints — resolution within KPIs

4.38 For connection-related complaints, the proportion that were resolved within their respective KPI timelines were analysed, with the result shown in Figure 4.5.

Figure 4.5: Percentage of complaints resolved within respective KPIs during the audit timeframe

Note: Urgent complaints were not included because of the fluctuations caused by being in such small numbers over the timeframe compared with the other two complaint types. See paragraph 4.40 below for urgent complaint volumes and percentage resolved within NBN Co’s KPI.

Source: ANAO analysis of NBN Co data.

4.39 During the audit timeframe, the majority (87.7 per cent as an average of total connection-related complaints) of Formal and Complex complaints were resolved within their respective KPI timeframes. This is below the 93 per cent target set by NBN Co for the combined resolution of these complaint types in FY2019.

4.40 Urgent complaints also lag behind the target set by NBN Co that 93 per cent of complaints be resolved within timeframes, with only 45.5 per cent of Urgent complaints resolved within the two day target. While Urgent complaints are a very small proportion of the total connection related complaints received by NBN Co during the audit period (11 complaints, or 0.000063 per cent), these complaints potentially involve priority assistance end users.

Connection-related complaints — resolution outside of KPIs

4.41 For those connection-related complaints which were not resolved within their respective KPIs, analysis was undertaken to determine the timeliness in which these complaints were ultimately closed.

4.42 Figure 4.6, Figure 4.7 and Figure 4.8 show trends in the number of days that elapse before complaints that were opened and are not resolved within the respective category KPIs before they are ultimately closed during the audit timeframe.

Figure 4.6: Percentage of Formal connection-related complaints closed past five working day KPI

Note: For reference, the 22.1 per cent of complaints that were closed beyond 10 working days equates to 493 total individual complaints.

Source: ANAO analysis of NBN Co data.

Figure 4.7: Percentage of Complex connection-related complaints closed past 20 working day KPI

 

 

Note: For reference, the 4.6 per cent of complaints that were closed beyond 40 working days equates to 94 total individual complaints.

Source: ANAO analysis of NBN Co data.

Figure 4.8: Percentage of Urgent connection-related complaints closed past two working day KPI

Note: For reference, the 67.8 per cent of complaints that were closed beyond four working days equates to four total individual complaints.

Source: ANAO analysis of NBN Co data.

4.43 Figure 4.6, Figure 4.7, and Figure 4.8 reflect that, of those complaints that were not resolved within their category KPIs:

• the majority of Formal complaints (51.51 per cent) took up to four working days to resolve post the expiry of the associated KPI (five working days);
• the majority of Complex complaints (50.54 per cent) took up to nine working days to resolve post the expiry of the associated KPI (20 working days); and
• the majority of Urgent complaints (50 per cent) took up to five working days to resolve post the expiry of the associated KPI (two working days).⁹⁰

4.44 This data reflects that complaints that are unresolved within the respective KPIs for all three categories are typically not being resolved in a sufficiently timely manner once the KPI has elapsed. The cause of this issue is unclear, as discussed in paragraphs 4.48 and 4.49. A recommendation to address this issue has been raised at paragraph 4.51.

Does NBN Co undertake ongoing connection-related complaints analysis and reporting to provide transparency and inform continuous improvement?

Complaint policy and internal reporting

4.45 NBN Co’s Complaint Management Policy states that:

In order to seek to identify and reduce the recurrence of complaints arising from systemic issues, [NBN Co]:

• monitors complaints to seek to identify emerging issues requiring specific attention and address those as soon as practicable, and
• analyses and classifies complaint data at least every 3 months to seek to identify recurring problems and issues and action opportunities for improvement.

Processes also seek to ensure rapid and effective management of, and notification to appropriately senior management of, significant complaints or issues.⁹¹

4.46 NBN Co uses complaint data in a variety of internal reports for review by relevant business areas and at the executive level. Operational levels of the organisation produce between 13 and 17 complaint-related metrics, of which the NBN Co executive is provided with two — Formal and Complex complaint turnaround times. Reporting to the Executive is focused on complaint volumes, aged complaints,⁹² and resolution turnaround times. It does not contain information or analysis on the potential causative factors associated with volumes and trends in respective complaint categories and types. For the audit timeframe, there was some reporting on complaint categorisation during the months of March to August 2018, but this was not consistently maintained.

Continuous improvement

4.47 In January 2019, NBN Co introduced an initiative to reduce complaints volumes in response to ‘trends identified during the governance of [NBN Co’s] enquiries and complaints performance’. In particular, the most significant trend was the higher incoming contact volumes than forecast, which had corresponding impacts to the average speed of answer and aged complaint volumes.’ While NBN Co’s strategy documents outline an intention to transform the end-to-end process for receiving, resolving and managing complaints, it could not substantiate that the initiative has commenced.

4.48 Root cause analysis of individual cases within a subcategory of complaints for continuous improvement is not easily supported by the process in which complaints data is entered into NBN Co’s CMS. While NBN Co captures and reports complaint volumes and closure rates, NBN Co’s CMS does not record the specific actions taken to support changing the status of a complaint record to resolved or closed, aside from inconsistent use of a free text notes field. NBN Co record the complaint resolution date and status in the CMS, however for those complaints that require action by NBN Co or its contractors, the associated action taken to resolve an end user’s complaint cannot be substantiated, even if this was a judgement made by a case manager that no further action was required.

4.49 NBN Co has no established structure to regularly consider systemic causes of complaints with RSPs. Additionally, NBN Co does not analyse the relationship between complaint and incident records. For the audit timeframe, there were 80,788 premises that had logged both a complaint and a service fault. The majority of these complaints (73.3 per cent) were logged after an associated fault was raised with NBN Co via the RSP.

4.50 In June 2019, NBN Co commenced the following initiatives to address the issues identified in paragraph 4.49:

• complaints trends and analysis with a major RSP broken down by each phase in the migration process;
• planned processes to deal with issues arising out of complaints analysis; and
• internally report RSP escalations.

Appendix 1 NBN Co Limited response

 

 

 

 

 

 

Appendix 2 Technology types and service classes

NBN technology type descriptions

Source: NBN documentation.

Service class descriptions

Source: NBN Co documentation.

Appendix 3 Key responsibilities for the process of migrating on to the NBN

Note a: While there are other industry stakeholders that have responsibilities related to the NBN migration process, for the purposes of this audit the key responsibilities and relationships are among NBN Co, RSPs and end-users. See the legal and policy documents listed at Table 1.2 for more detail on NBN Co responsibilities.

Note b: Timeframes to complete many of these responsibilities are agreed between NBN Co and RSPs via the Wholesale Broadband Agreement, discussed in Chapter 3.

Note c: For the purposes of this table, ‘NBN Co’ refers to both its permanent and contracted employees, and so therefore covers the role of delivery partners.

Source: ANAO analysis.

Appendix 4 NBN Co high level organisation structure (as at 1 August 2019)

 

 

Note: Business units included are only those that have been referred to in the report.

Appendix 5 Direct mail content and timing leading up to disconnection date (DD)

Source: ANAO analysis.

Appendix 6 Fault resolution timeliness volumes over time by technology type and geography

Remote

Minor rural

Major rural

Urban

Notes: The ‘20+’ figure on each x axis has been used to allow for earlier fault volumes to be visible on each figure. By geographic region, the number of faults taking longer than 20 days to resolve were: urban = 213; minor rural = 22; major rural = 34; remote = 3.

According to NBN Co data within the audit timeframe, HFC service faults were only recorded at urban premises.

Source: ANAO analysis of NBN Co data.