Screening of domestic aircraft passengers

1.1 Everyone entering the secure area of an Australian airport terminal, and their carry-on baggage is screened to ensure the safety and security of all travellers. Screening utilises a combination of specialised equipment—walk through metal detectors, x-ray observation, and explosive trace detectors—as well as the judgement of screening personnel to prevent prohibited items, such as weapons and explosives from being carried onto aircraft.⁸ Across Australia, there are 62 security controlled airports⁹, where screening is undertaken to mitigate against: the unlawful seizure of an aircraft; intrusion on board an aircraft of a weapon or other material capable of threatening the integrity of the airframe; and the use of the aircraft as a weapon.

1.2 The Office of Transport Security (OTS), within the Department of Home Affairs (the Department)¹⁰ is responsible for ensuring that passenger screening is effective. Powers to conduct screening are provided under the Aviation Transport Security Act 2004 (the Act), and the Aviation Transport Security Regulations 2005 (the Regulations).

1.3 As well as being effective in order to meet the legislated and regulatory requirements, screening needs to be conducted efficiently to facilitate air travel. Since October 2012, the number of domestic aircraft passengers has trended upwards with 62.13 million domestic passengers carried on 639 400 regular public transport and charter aircraft trips during the year ending October 2017.¹¹

Implementation of increased security measures

1.4 On 29 July 2017, counter-terrorism raids were conducted in Sydney in response to a plan to attack an aircraft using an improvised device. Further to the raids, on 30 July 2017, the Government announced that increased security measures had been put in place at Sydney, Melbourne, Brisbane, Darwin, Perth, Adelaide, Canberra, Cairns, Gold Coast, and Hobart airports to:

complement security arrangements already in place and applied as an extra precaution, in coordination with counter terrorism raids in Sydney conducted on 29 July 2017.¹²

1.5 Subsequently, the Inspector of Transport Security was tasked with conducting a review of aviation security at regulated airports. This review of aviation security was instigated on 8 August 2017 to ensure that security measures remain appropriate, and are adapting to new and emerging issues. The report was delivered on 24 November 2017.

Monitoring screening authorities compliance

1.6 In most instances, an airport or airline will act as the authorised screening authority, and may sub-contract day-to-day screening operations to a specialist screening provider. Airport and aircraft operators that undertake screening must be authorised by the Department and are responsible for:

• meeting the minimum legislated security requirements outlined in the Act and the Regulations;
• delivering security on a day-to-day basis; and
• operating in accordance with the relevant Aviation Screening Notice.¹³

1.7 The Department is responsible for establishing an effective compliance monitoring program to: provide assurance that industry participants comply with regulatory requirements; and accurately assess the effectiveness of passenger screening. The National Compliance Plan (NCP) developed by the Department documents the number and type of compliance monitoring activities the Department plans to conduct on an annual basis. The plan outlines the allocation of Departmental resources to ‘core’¹⁴ and ‘non-core’¹⁵ compliance monitoring activities.¹⁶ Table 1.1 outlines the types of ‘core’ and ‘non-core’ compliance monitoring activities conducted by the Department.

Table 1.1: Types of compliance monitoring activities

Note a: The term ‘audit’ is defined by the Department as per Annex 17 to the International Civil Aviation Organisation (ICAO).

Note b: A Transport Security Program sets out the physical places within an airport that are subject to regulation, and how the airport operator will manage security for its operations, including for passenger screening. Adherence to the Transport Security Program is monitored through compliance activities.

Note c: This can include additional targeted activities on industry participants (for example, where a security failure has been identified).

Source: ANAO analysis of Departmental documentation.

1.8 Since the previous audit was tabled, the Department has adjusted the allocation of resources to ‘core’ compliance activities to increase the number of inspections and system tests and reduce the number of audits conducted (see Table 1.2). The Department has also revised the systems test regime. In January 2017, the Department released a system test strategy (co-designed with industry) which introduced a tiered approach to systems testing, re-introduced the Glock 17P test piece, and a range of new test pieces designed to test the ability of the system to detect contemporary threats. Subsequently, the Department adjusted the NCP to increase the usage rate of the new test pieces in January 2018.

Table 1.2: Allocation of resources to core compliance activities (2015–16 to 2017–18)

Note a: As at January 2018.

Source: ANAO analysis of Departmental documentation.

The ANAO’s previous audit

1.9 In August 2016, the Australian National Audit Office (ANAO) tabled Audit Report No.5 2016–17, Passenger Security Screening at Domestic Airports in the Parliament. In that audit, the ANAO found that the Department (then, the Department of Infrastructure and Regional Development) was unable to provide assurance that passenger screening was effective, or to what extent screening authorities had complied with the Regulations, due to poor data and inadequate records. Further, the ANAO found that the Department did not have meaningful passenger screening performance targets or enforcement strategies and did not direct resources to areas with a higher risk of non-compliance.¹⁷

1.10 The ANAO made five recommendations (see Box 1) aimed at improving the Department’s regulatory performance.¹⁸ The Department agreed to implement all five recommendations.

1.11 In its response, the Department advised¹⁹ the ANAO that:

The Department notes that following significant review work in 2015 it is investing in the broad reform of its transport security regulatory operations[…] This reform program comprises three elements: broad reform of Departmental transport security regulatory operations to ensure the Department is well positioned to respond to changing threats and risks; improving the Department’s collection and analysis of data pertaining to passenger screening, including incorporating non-compliance risk into its planning; and establishing a working group to develop a framework to measure the effectiveness and extent that screening authorities are complying with passenger screening regulations.

Progress in implementing the ANAO’s recommendations

1.12 Implementation of the ANAO’s recommendations has been monitored by the Department’s Audit Committee. The operational area responsible for implementing each recommendation established the target date to implement each of the five ANAO recommendations. The Department’s advice to its Audit Committee on the status of each recommendation is outlined in Table 1.3.

Table 1.3: Target date for implementation and the status of each recommendation

Note a: Calculated from 12 July 2016 the date of the Department’s response to the previous ANAO audit.

Source: ANAO analysis of Departmental documentation.

Recent Departmental and external reviews

1.13 Subsequent to the ANAO’s previous audit, there have been a further three external reviews into Australia’s aviation security regime (see Table 1.4 on the following page).

Table 1.4: Aviation Security Reviews

Source: ANAO Analysis of Departmental documentation.

Audit approach

1.14 The objective of this audit was to examine the extent to which the Department of Infrastructure and Regional Development, now the Department of Home Affairs (the Department) has implemented the five recommendations from ANAO Report No.5 2016–17, Passenger Security Screening at Domestic Airports.

1.15 To form a conclusion against the audit objective the ANAO adopted the following high level criteria:

• To what extent has the Department implemented an effective compliance monitoring program?
• To what extent has the Department implemented an appropriate learning and development framework?
• To what extent has the Department implemented effective performance monitoring and reporting arrangements?

1.16 The audit team has: reviewed, examined and analysed relevant documentation held by the Department; interviewed relevant Departmental staff and internal stakeholders; sought input and commentary from authorised screening authorities in relation to the Department’s compliance planning, monitoring, and enforcement activities, focusing on the changes made since the previous audit was tabled; observed screening procedures at Gold Coast, Brisbane, Melbourne and Avalon airports; and examined the results from systems testing undertaken by the Department.

1.17 The audit scope has not included the examination of air cargo, checked baggage or the screening of international passengers.

1.18 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $297 350.

1.19 Team members for this audit were Joyce Knight, Michael Commens and Sally Ramsey.

Has the Department established a data analysis function that can identify trends in non-compliance based on accurate, reliable compliance activity data?

2.1 ANAO Audit Report No.5 2016–17 found that the Department had limited information about the compliance history of individual industry participants and lacked reliable data and analysis to identify systemic issues, or compare performance across industry participants.²⁰ To improve the Department’s ability to target compliance activities at areas where there was a higher risk of non-compliance, the previous audit recommended (ANAO Recommendation No. 2a) that the Department establish an analysis function capable of identifying trends in non-compliance based on accurate, reliable compliance activity data.²¹

2.2 The data analysis function that was subsequently established by the Department has produced a single analysis of compliance activity data based on data extracted from the Regulatory Management System (RMS)²² for the period July 2015 to December 2016. The analysis was delivered in July 2017 and provided a breakdown of compliance activities conducted for each category of airport²³ and aircraft operator²⁴, the number of findings by type²⁵ for each state, and the number of findings per security mitigation category.²⁶ The analysis did not identify trends in non-compliance or identify industry participants that required a higher level of support to comply with the regulated requirements (such as those providers discussed in paragraph 2.9).

2.3 The ANAO analysed compliance activity data extracted from the RMS between July 2016 and October 2017 and found that while the quality and consistency of the data is improving, inconsistent and incomplete data is still being entered into the system. Further, the Department’s ability to retrieve data recording the outcome of compliance activities from the RMS is limited, as the data is stored in free text fields which do not facilitate easy retrieval for reporting and analysis.

2.4 As part of its 2017–20 strategic plan, the Office of Transport Security identified a number of priorities including Strategic Priority No.1: Improve our data and information collection, use and management.²⁷ In line with this strategic priority, the Department has acknowledged the limitations on its ability to obtain meaningful, comprehensive reporting and analysis from its systems and approval was given to develop a data product suite to meet business needs on 10 November 2017. The business case outlining the rationale for the development of a data product suite identified a number of limitations with the current systems including:

• the number of disparate business systems used to store data;
• the quality and continuity of data sets;
• limited data extraction and reporting capabilities; and
• different data structures and definitions used across the various business systems in use.

When approving the business case the Department noted that it will adopt the data and information framework applied in the Department of Home Affairs to develop the data product suite. However, as at March 2018, a plan outlining how the Department intends to develop the data product suite and address the limitations of the systems to provide meaningful analysis of compliance activity and outcomes as well as facilitate comprehensive reporting has not yet been developed.

Are the results from previous compliance activities being incorporated into the compliance monitoring program?

2.5 ANAO Audit Report No.5 2016–17 found that without effective data collection, storage and retrieval to enable analysis of individual industry participant performance the Department is unable to assess the risk of non-compliance or identify systemic non-compliance and apply resources where the risk is higher.²⁸ It also observed that there was no feedback loop in the compliance cycle to indicate whether compliance is improving or deteriorating and where additional monitoring may be required. Accordingly, the ANAO recommended that the results of previous compliance activities be incorporated into future compliance plans (ANAO Recommendation No. 2b) to focus compliance resources on areas where there is a higher risk of non-compliance.

2.6 The July 2017 analysis (as discussed in paragraph 2.2) was not used by the Department to develop the National Compliance Plan for 2017–18. Instead, separate source data was extracted from the RMS and the Microsoft Excel regional office work plans to track progress against the targets established in the 2016–17 National Compliance Plan. The data included a summary of the compliance activities the Department had conducted, the number of findings per security mitigation category and the results of systems tests undertaken across Category 1 to 3 airports.

2.7 As a result, the data used to inform the development of the 2017–18 National Compliance Plan was not informed by an analysis of trends in non-compliance or identify industry participants exposed to a higher risk of non-compliance.²⁹

Does the Department direct compliance resources to areas with a higher risk of non–compliance or that require higher levels of support and/or monitoring?

Allocating compliance resources and activities

2.8 The number and type of compliance activities planned for each category of airport and airline operator is based on the risk and compliance prioritisation rating applied. As shown in Figure 2.1 (below), more resources are applied to categories of airports and aircraft operators with higher risk and compliance prioritisation ratings. The Department determines the risk and compliance prioritisation³⁰ratings by:

• calculating a risk priority rating for each category of airport and airline operator using risk scenarios and an attack path methodology to assess the likelihood and consequence of a risk event occurring; and
• calculating an adjustment score using ten adjustment score questions for each category of airport and airline operator which is then applied to the risk rating to determine a compliance prioritisation rating.³¹

Figure 2.1: Proportion of compliance activities planned per risk rating

Source: ANAO analysis of Departmental documentation.

2.9 The process used by the Department to determine the risk priority rating is effective at identifying categories of airports exposed to a higher security risk—for instance, where an attempt to unlawfully interfere with an aircraft is more likely to occur. However, the Department has not yet established an effective approach to identify individual industry participants that present a higher risk of non-compliance and does not direct resources accordingly within categories. For example, the Department does not direct resources to industry participants within a category that present a higher risk of non-compliance due to:

• a poor record of resolving non-compliance notices;
• frequent non-compliance notices³², or observations³³ of potential security vulnerabilities have been issued in relation to the same security mitigation category over an extended period of time;
• the industry participant is operating screening equipment that is subject to an extension to the grandfathering arrangements; or
• the industry participant routinely needs additional support to comply with regulated requirements.

Note a: The ‘high’ risk airports were a Category 1 and 2 airport.

2.10 The process currently used by the Department to determine the compliance rating is a qualitative one which is applied to each category of airport and aircraft operator and not individual industry participants within the categories. Regional offices are to consider the compliance attitude, history, size and complexity of individual industry participants as part of planning the compliance activities. However, there is no evidence that the data compiled by the regional offices is analysed to determine if compliance is improving or deteriorating or if it forms part of a feedback loop to inform the national compliance planning process.

2.11 The national compliance planning process could be strengthened considerably by including accurate quantitative data from the analysis of previous compliance activities, to provide more granular information about the compliance history of individual industry participants. By doing so, the Department would be better placed to implement a nuanced approach to its compliance monitoring activities within each category, and effectively target compliance resources. The difficulties the Department has had in obtaining accurate data from its systems which would support it to undertake this type of analysis were examined at paragraphs 2.2 and 2.3.

Are non-compliances accurately identified, monitored and managed in accordance with a clear enforcement framework?

2.15 ANAO Audit Report No. 5 2016–17 found that the Department did not have an enforcement policy and did not utilise the full suite of enforcement actions available to it. Paragraph 3.18³⁵ of that report suggested the Department develop a clear enforcement strategy, and communicate its strategy to relevant stakeholders.

2.16 The Department’s strategic plan for 2017–20 broadly outlines its compliance approach (see Figure 2.2) in terms of a regulatory pyramid and the industry version of the NCP released in March 2017 informed relevant stakeholders about the Department’s regulatory operations (at a high level).

Figure 2.2: The Office of Transport Security’s Regulatory Pyramid

Source: Departmental documentation.

2.17 The Department’s strategic plan advises that it intends to build its capacity to undertake enforcement action over the course of the next three years.³⁶ In addition, the Department has developed a compliance and enforcement framework, and implementation plan. The objectives of the framework include ‘an enforcement capability to address seriously disengaged or non-compliant industry participants and/or breaches’. The implementation plan—which has not yet been approved—outlines how enforcement activities will be managed and the delivery model that will be used. It includes the establishment of an enforcement team and proposes to deliver training to improve the enforcement and investigative skills of the Transport Security Inspectors (TSIs), and identifies the need to develop clear guidance outlining the application of the various enforcement options available and the escalation process to assist manage non-compliance as identified in the previous audit.³⁷

2.18 The previous audit found³⁸ that the majority of enforcement action undertaken by the Department involved issuing observations³⁹ and non-compliance notices.⁴⁰ Where a non-compliance notice or observation requires that a corrective action plan be submitted by the industry participant, a timeframe for resolution and the actions that the industry participant will complete in order to resolve the non-compliance, is required.⁴¹

2.19 Non-compliances and observations are identified through the compliance monitoring activities undertaken. Where non-compliances are identified they are monitored and managed at the regional office level via weekly regional office committee (ROC) meetings where proposed, draft, and issued observations and non-compliance notices are addressed. Regional directors⁴² also have the power to compel industry participants to provide evidence such as CCTV footage, daily testing logs for screening equipment, training records and other relevant information where an alleged breach of the legislation or regulations is being examined.⁴³

2.20 Since the previous audit was tabled, the Department has sought to improve consistency when issuing an observation or non-compliance notice by providing administrative decision making and procedural fairness training to all TSIs.⁴⁴ The Department has also started to make greater use of additional enforcement options such as infringement notices.⁴⁵ However, as identified in the implementation plan there is a need for clear guidance to be made available to TSIs which outlines the escalation process.

2.21 Interviews with regional TSIs indicated that the process of issuing and managing observation or non-compliance notices was well understood. Notwithstanding this, the Department still requires a clear escalation process for serious or systemic non-compliance, and guidance detailing how various sanctions are to be applied is not currently available.

Has the Department completed a training needs analysis?

3.1 ANAO Audit Report No.5 2016–17 identified significant issues with the reliability (namely completeness and accuracy) of data captured in the Regulatory Management System (RMS) which arose because users were not trained in the use of the system. The ANAO recommended (ANAO Recommendation No.4) that the Department conduct a training needs analysis for users of the RMS system, deliver appropriate training and monitor its effectiveness.

3.2 The Department accepted the recommendation and responded that the recommendation would be implemented through the development of a broader learning and development framework⁴⁶, specifically that:

… a needs analysis will be undertaken to ensure staff are trained and supported to undertake all of their regulatory responsibilities. This will include an assessment of training requirements for the regulatory management system. Once the skills, knowledge and capabilities of staff have been assessed against their roles, a plan to address gaps will be developed and implemented. The learning and development framework will include a process for reviewing and maintaining skills.

3.3 The Department originally advised that a training needs analysis would be complete by December 2016, however it wasn’t until November 2016 that the Department entered into a contract to develop and deliver a training needs analysis. The analysis developed by the contractor was accepted by the Department on 28 February 2017 (more than twelve months later than planned). It identified skill and knowledge gaps that could be addressed through learning and development, and found that:

• a formal approach to learning and development is required;
• a nationally consistent qualification that incorporates role specific and generalist training modules would provide staff with formal recognition of their skills as well as enhancing career mobility⁴⁷;
• ownership of the learning and development framework should be established and implementation should be managed by a learning and development specialist;
• the Department should utilise a 70:20:10 (70 per cent on the job, 20 per cent coaching/mentoring, and 10 per cent formal training) approach to learning;
• the Department should identify a partner organisation to develop and implement the learning and development framework and deliver associated courses and tailored training modules; and
• measures of success, and a timeframe for review and evaluation of the framework should be included to assess if the framework is meeting ongoing needs.

Has the Department developed a learning and development framework which is continuously monitored, evaluated, and revised where necessary?

3.4 In June 2017, the Department commenced translating the recommendations from the training needs analysis into a learning and development framework. The framework was approved in November 2017, and breaks down the training and development into three phases:

• induction—intended to leverage existing training, on the job coaching, and demonstrations that address role requirements, business processes and IT systems;
• core skills and capabilities—delivery of general training including writing skills, stakeholder engagement, critical thinking, judgement and evidence based decision making; and
• specialist skills and professional capabilities—delivery of role specific training relating to key job functions within the regulatory assessment, compliance monitoring and administrative support roles.

3.5 The Department’s framework broadly aligns with six of the seven principles for better practice in learning and development identified by the Australian Public Service Commission (see Figure 3.1 below).⁴⁸ The exception being a monitoring and evaluation component.

Figure 3.1: A framework for managing learning and development in the APS

Source: Building capability: A framework for managing learning and development in the APS.

3.6 During the course of the audit, the Department advised the ANAO that work to develop a monitoring and evaluation component had commenced. As at March 2018, the Department has not yet developed a plan for monitoring and evaluating the implementation of the framework to determine if it is meeting objectives and continues to be appropriate in terms of quality, time, costs and alignment with strategic priorities.

3.7 The service provider engaged to design and deliver the training modules identified in the learning and development framework is contractually required to obtain feedback and evaluate whether training is meeting user needs (with oversight from the Department). This information, while useful, will not enable the Department to determine if the implementation strategy outlined in the learning and development framework continues to meet the needs of the Department as intended by the recommendation made in the ANAO’s previous audit.

3.8 Implementation of a monitoring and evaluation strategy early in the delivery of the new framework and using the training needs gap analysis as baseline data, will position the Department to better inform itself about the extent to which the objectives of the learning and development framework are being met.

Is the Department delivering appropriate training?

3.11 In December 2017, the Department entered into a contract for the delivery of the training modules outlined in the learning and development framework. Training commenced in late February 2018 with the Department advising the ANAO that specific training needs (including training on the Regulatory Management System, as originally recommended by the ANAO (Recommendation No.4) of the previous audit will be incorporated into tailored, specialist modules which are scheduled to be delivered in June 2018.

3.12 Prior to the formal commencement of the training identified in the learning and development framework, staff had access to ‘principles of decision making’ and ‘administrative law and legislation’ training courses and ‘non-compliance notices’ training workshops. This training was delivered between June and September 2016. Specific material for users of the Regulatory Management System was delivered through the development and release of in-house quick reference guides.

Has the Department consulted with stakeholders to develop performance measures that are practical, achievable and measurable?

4.1 The previous ANAO audit report found that the Department did not have appropriate performance measures in place to determine whether passenger screening is effective.⁴⁹ Subsequently, the ANAO recommended (ANAO Recommendation No.3) that the Department, in consultation with stakeholders, develop performance measures for passenger screening that are practical, achievable and measurable.⁵⁰ The Department agreed to implement the recommendation, and advised the ANAO that:

This began in June 2016 with the systems test working group, which involves industry stakeholders, and has been established to review and update the systems test regime. As part of its review of the regime, the working group will develop performance measures for some aspects of passenger screening. This work will be broadened once the systems testing regime has been updated.

4.2 In January 2017, the systems test working group delivered the strategy to guide the future development of the systems test program. While it has a role in improving the ability of the Department to provide accurate systems test results, and provide necessary performance information it was not responsible for developing the performance measures.

4.3 To develop the performance measures, the Department established a separate working group, the Screening Innovation Working Group (the working group) and in August 2017, members were asked to provide feedback on a draft set of proposed performance measures. The proposed measures were approved by the Department in October 2017 and subsequently tabled at the Aviation Security Advisory Forum (the forum) and the Regional Industry Consultation Meeting (RICM).

Development of the performance measures

4.4 Numerous reviews⁵¹ including ANAO Audit Report No.26, 2002–03, Aviation Security in Australia have recommended that the Department develop and implement performance measures that are practical, achievable and measurable.⁵² Subsequently, in 2009⁵³ the Department released the Review of Aviation Security Screening Report⁵⁴, which included 27 recommendations to improve screening. One of the recommendations was to develop a new performance measurement framework in partnership with industry. Suggested key performance indicators and performance targets were also outlined.

4.5 In response to the 2009 aviation security screening report, in May 2012 the Department engaged a consultant to develop key performance measures to assess the effectiveness of security screening checkpoints. This activity was a proof of concept exercise and was completed in June 2013, however the subsequent phase of the project⁵⁵ did not progress. ⁵⁶

4.6 In October 2016, the Department recommended that the development of a performance measurement framework be restarted. In August 2017, a series of draft measures informed by the measures recommended in the 2009 Review were presented to the working group for consideration. These draft performance measures were subsequently approved by the Department and circulated to industry participants in October 2017 (as discussed at paragraph 4.3).

The performance measures

4.7 The performance measures developed by the Department are designed to assess the effectiveness of the system and screeners at detecting weapons, and the use of technology designed to detect threats of most concern. Successful implementation of the performance measures relies on screening authorities providing accurate data to the Department in relation to the performance of screening providers and the results of systems tests conducted by the Department through its compliance monitoring program. The combination of data provided from external and internal sources should enable the Department to accurately assess the effectiveness of passenger screening.

4.8 Interviews with screening authorities along with discussions documented in the forum and RICM meeting minutes confirm that the measures selected, the reporting frequency and timeframes proposed are both practical and achievable from an industry perspective. The ANAO was also advised by the industry participants interviewed during the fieldwork phase of this audit that the performance data required is already captured by most industry participants.

Implementation of the performance measures

4.9 The Department advised the ANAO in November 2017 that implementation of the performance measures is scheduled to commence by 1 July 2018 as part of the Enhanced Mandatory Reporting Project. The aim of this project is to provide improved evidence that the Government will be able to use to assess the efficacy of Australia’s transport security regime, to target appropriate and proportionate security measures and address emerging issues by making it mandatory for industry participants to provide performance and security incident data. As at December 2017, initial consultations have been held with selected industry participants to discuss information requirements, however the scope, implementation strategy, deliverables and timeframes for the project have not been finalised.

4.10 The Department plans to initially receive data from a select group of industry participants by conducting a pilot program with a smaller subset of industry participants during the preparation stage of the project. This stage of the project is scheduled to commence in March 2018 with wider implementation to commence in July 2018 and is due to be completed by February 2019 – almost 10 years after the 2009 aviation screening review was released.

4.11 Without performance measures it will be difficult to: assess the effectiveness of the compliance monitoring program; determine which screening authorities are performing passenger screening effectively and require less monitoring; as well as those which are performing poorly, determine the cause, and implement corrective action. It will only be through the implementation of the measures that the practicality, measurability, and achievability of the measures can be assessed.

Does the Department provide performance reporting that meets the business requirements of its various stakeholders?

4.13 In November 2017, the Department reported that it was on track to complete the implementation of ANAO Recommendation No.5 by 1 July 2018, 24 months after the previous audit was tabled. The Department originally advised that:

since the commissioning of the regulatory management system, the Department has continued to invest in the development of its reporting capability and remediate data holdings. It is anticipated that this work will begin to deliver the foundations of a reporting capability by the end of 2016. The Department’s intention remains to provide useful reports on passenger screening compliance trends and patterns to stakeholders once this capability is available.

4.14 Since the previous audit was tabled, the Department has improved and expanded the range of reporting that it provides to its stakeholders, however the improvements made and the products delivered are produced using resource intensive manual processes to extract data from the Regulatory Management System (RMS) and the Microsoft Excel based regional office work plans.

4.15 The previous audit report noted that the RMS was released in July 2014, and identified issues with training and data quality, specifically noting the lack of a reporting function. To address the issue of the lack of a reporting function, in October 2016, the Department endorsed a list of 15 operational reports with a set of six identified as high-priority. However, in March 2017, the Department decided not to proceed, due to the estimated cost of $160 000 to produce the six high priority reports. Subsequently, the Department decided to build the reporting capability in-house.

4.16 The in-house reporting function currently requires data to be manually extracted from the RMS. Requests for reports are submitted by the user to a dedicated internal mailbox and must be endorsed before the report will be produced and distributed. As at December 2017, the issues in relation to the lack of a reporting function continue to persist and the RMS is still unable to produce accurate standard reports such as industry participant contact details, and compliance histories of individual industry participants.

Compliance activity reporting

4.17 The various types of reporting provided by the Department to its stakeholders are outlined below at Table 4.1.

Table 4.1: Reports provided to stakeholders

Note a: Ministerial submission provided include advice and updates in relation to the systems tests conducted as part of the suite of enhanced aviation security measures and the results.

Note b: In 2017–18, the performance targets in the Portfolio Budget Statements were amended to remove the target of completing 95 per cent of ‘high risk’ cases to completing 100 per cent of compliance activities within the National Compliance Plan.

Note c: The Department also provided an update to industry participants outlining the results of systems tests conducted to determine levels of achievement against the targets required as outlined in the enhanced aviation security measures.

Source: ANAO Analysis of Departmental documentation.

4.18 The Department has improved its ability to provide reports which meet the business needs of its various stakeholders. The reports outline progress achieved against the National Compliance Plan and the outcomes of compliance monitoring activities undertaken focusing on systems test results. However they remain resource intensive to produce; are dependent on the accuracy of data stored in Microsoft Excel based regional office work plans and the RMS. Further, the data that relates to the outcome of the compliance monitoring activities is not recorded in a format which facilitates easy retrieval for analysis and reporting.

Appendix 1 Entity response

a. The ANAO notes that the Department has disagreed with our assessment that it is not yet well placed to provide assurance that passenger screening is effective and that screening authorities comply with the Regulations. As discussed at (paragraphs 7, 2.2, 2.3 and 2.4) there continue to be constraints around the Department’s ability to extract and analyse accurate and reliable compliance activity data. Accurate and reliable compliance activity data, and the ability to effectively analyse it, is necessary to provide assurance that passenger screening is effective and screening authorities comply with the Regulations.

b. The report does not recommend that additional compliance activities should be imposed on airports granted grandfathering extensions for screening equipment. The report (see paragraph 2.9, 2.11 and Case Study No.1) highlight that the Department does not effectively utilise the available information (including the existence of grandfathering arrangements) when determining risk and compliance ratings. As such, compliance resources are not targeted towards individual airports and airline operators where the risk of non-compliance is higher or additional support is needed to comply with regulatory requirements.