Introduction

1.1 Parliamentarians are provided with a range of supports and allowances to assist them to carry out their duties including the following.

• Office expenses — for example printing and communication, office stationery and supplies, and property operating expenses (such as cleaning, security and fit out for electoral offices).²
• Travel expenses — for both parliamentarians and staff. This includes for example commercial transport (such as commercial aircraft, trains, taxis), travel allowance to cover accommodation, meals and incidentals for overnight stays and other allowances such as a budget for travel in large electorates.³
• Staff — ‘personal employees’ (such as Chief of Staff, advisor, media advisor or office manager) and ‘electoral employees’ (based in parliamentarians’ electoral offices).⁴

1.2 The use of office and travel expenses by parliamentarians is covered by:

• the Parliamentary Business Resources Act 2017 — provides for parliamentarian remuneration, expenses, allowances, and other public resources, it also imposes compliance and enforcement arrangements;
• Parliamentary Business Resources Regulations 2017 — provides details on travel expenses, travel allowances, office accommodation and equipment, insurance, and legal assistance available; and
• determinations made under the Parliamentary Business Resources Act 2017 that provide further clarifications on the arrangements.

1.3 Parliamentarians hire staff under the Member of Parliament (Staff) Act 1984 (MOPS Act) to help them fulfill their duties. The Commonwealth Members of Parliament Staff Enterprise Agreement sets the staff employment terms and conditions. Domestic travel expenses for parliamentary staff are generally not included in the Parliamentary Business Resources Act 2017. A determination of the MOPS Act and the Enterprise Agreement guide the eligibility and rules for parliamentary staff domestic travel, for example staff may only travel if directed by their employing parliamentarians and may only travel by the most efficient direct route available.

1.4 The Department of Finance (Finance) and the Independent Parliamentary Expenses Authority (IPEA) support the above arrangements as outlined in Table 1.1.

Table 1.1: Parliamentary expenses by administering agency

Source: Department of Finance, About MaPS [Internet], Finance, available from https://maps.finance.gov.au/about-maps-and-other-services/about-maps [accessed 29 August 2023].

Independent Parliamentary Expenses Authority (IPEA), Who we are [Internet], IPEA available from https://www.ipea.gov.au/about-ipea/who-we-are [accessed 29 August 2023].

Parliamentary Expenses Management System (PEMS)

1.5 PEMS is the IT system used by parliamentarians and their staff to claim office and travel expenses and perform human resources tasks. Administering agencies also use PEMS to process and pay claims and administer payroll services for parliamentary staff.

1.6 PEMS refers to the combination of an online portal, with a front-end interface, and a back-end SAP enterprise resource planning (ERP) solution.⁵ Parliamentarians and their staff interact with PEMS via the front-end interface for claims, employee self-services and reporting. IPEA and Finance access both the front-end and the back-end ERP to process and pay claims and administer payroll services for parliamentary staff.

1.7 PEMS was developed in response to the 2016 Independent Parliamentary Entitlements System Review that recommended Finance develop a business case for government consideration for a fit-for-purpose work expenses system.⁶

1.8 The Australian Government agreed to the second pass business case (business case) for the PEMS project as part of the 2017–18 Mid-Year Economic and Fiscal Outlook (MYEFO) process, including agreement for Finance to deliver the project in-house by June 2020 and provided funding of $38.1 million.

1.9 The project commenced in January 2018 with the aim to deliver a ‘fit-for-purpose online, integrated and agile ICT system to provide self-service capabilities, reduce manual processing and facilitate improved reporting to assist parliamentarians to better manage and monitor expenses.’

1.10 The project was delivered through eight milestones, with changes to scope and timeframes throughout the project (see paragraph 2.24). Milestones 1–5 delivered a front-end portal with online forms connected to the previous IT system (Entitlements Management System). Milestone 6 and Milestone 7 replaced this with the new ERP system for financial management (Milestone 6), payroll services (Milestone 7 phase 1) and office and travel expenses (Milestone 7 phase 2). Milestone 8 covers functionality to support public expenditure reporting. Finance advised the ANAO on 18 December 2023, that the expenditure reporting functionality was released in PEMS in November 2023, with IPEA publishing its first report in December 2023.

Table 1.2: PEMS project milestones

Source: ANAO analysis of PEMS project documentation.

1.11 There is a backlog of items from the project intended to be delivered over the next two to three years (see paragraph 3.14 to 3.16).⁷

1.12 Finance built the system, customising a SAP solution, with input from IPEA. The roles and responsibilities are outlined in Table 1.3.

Table 1.3: Finance and IPEA roles and responsibilities in the PEMS project

Source: Department of Finance PEMS project documentation.

1.13 In 2022–23 PEMS processed 35,833 office claims and 115,388 travel expense claims.

1.14 In 2022–23 PEMS paid out approximately $153 million in office expenses, $61 million in travel expenses and processed $270 million in payroll for parliamentary staff payroll.⁹

Rationale for undertaking the audit

1.15 The Auditor-General agreed to undertake an audit into PEMS on 4 April 2023. The audit was conducted to provide assurance to Parliament on how effective Finance was in successfully delivering the PEMS project. A previous ANAO audit of IPEA, tabled in March 2021, identified delays and budgetary overruns in the implementation of the project.

Audit approach

Audit objective, criteria and scope

1.16 The audit objective was to assess the effectiveness of Finance’s administration of the PEMS project.

1.17 The high-level criteria were:

• Did Finance effectively manage the PEMS project to achieve the agreed deliverables?
• Does the system meet the agreed deliverables, and achieve outcomes including alignment with relevant legislative requirements?

1.18 The audit scope included assessing Finance’s project management and implementation — project planning, governance, delivery, finances, and reporting — and whether the project delivered the required IT capability.

Audit methodology

1.19 The audit involved examining:

• governance arrangements throughout the project lifecycle;
• project documentation such as project plans, schedules, workforce planning and project reviews;
• evidence that project implementation plans were followed and evidence of change management processes and controls;
• the project’s financial management and reporting;
• the risk management approach and evidence of active management of project risks;
• internal and external reporting on the project including to project stakeholders;
• a survey of parliamentarians and their staff to assess the user experience;
• PEMS (through IT demonstrations) to test processes; and
• meetings with key staff in Finance, IPEA and Digital Transformation Agency.

1.20 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $589,626.

1.21 The team members for this audit were Anne Rainger, Ben Thomson, Renina Boyd, Dr Vivian Turner, Thea Ingold, Kelvin Le, Caitlin Williams, Aaron Ramshaw and Michelle Page.

2.1 When managing complex projects, it is important to have fit-for-purpose project planning documentation that clearly identify project scope, requirements, timeframes, and budget. Good governance and risk management enables monitoring of project requirements, sound decision making and ensures project outcomes are achieved. Effective project reporting supports transparency and accountability of project progress.

Was there effective project governance oversight in place?

2.2 Finance’s project management guidance states that:

• governance should provide an identifiable, single point of accountability for project delivery;
• governance structures should reflect the size and complexity of the project;
• projects require two levels of governance — operational and strategic — that for complex projects should include a project control group (operational) and a project steering committee (strategic); and
• projects may establish working groups where appropriate to support the project.

PEMS project governance

2.3 The PEMS project governance forums comprised a project board between January 2018 and August 2022¹⁰ and steering committee throughout the project. A design authority also met from June 2020 to November 2021 to provide guidance, advice, and assurance on scope and business requirements for Milestone 7. As set out in the PEMS project management plan, the governance forums were intended to report to the Project’s Senior Responsible Office (Deputy Secretary, Business Enabling Services)¹¹ who reported into Finance’s enterprise governance structures as outlined in Figure 2.1.

Figure 2.1: PEMS planned project governance roles and responsibilities

Note: While the project governance structures were intended to report to Finance’s Executive Board it was not involved in project governance outside of funding decisions (see paragraph 2.17) and a project status update in September 2023.

Source: ANAO analysis of the PEMS Project Governance Roles and Responsibilities figure from the PEMS project management plan.

2.4 The terms of reference for the project board and steering committee contained information on key areas such as objectives, escalation and delegation of issues, frequency and recording of meetings, and quorum. The terms of reference were not dated nor were changes clearly identified through version control. The remit of the project board and steering committee as per their terms of reference is outlined in Table 2.1.

Table 2.1: Project board and steering committee terms of reference

Note a: Between January 2018 and August 2018 project board meetings were monthly and steering committee meetings were every second month. The meeting frequency was changed to fortnightly and monthly respectively after recommendations from the second Gateway Review (July 2018).

Note b: The Australian Tax Office representative was on the steering committee until June 2021 when the representative changed job roles and was not replaced.

Note c: From September 2022 the Special Minister of State’s Chief of Staff became an observer on the steering committee. From March 2023 an independent external advisor (who was a former Gateway Reviewer of PEMS) also joined the steering committee.

Source: Project board and steering committee terms of reference.

2.5 In line with Finance’s internal project management guidance, the project board was set up to monitor progress of the project on an operational level. The steering committee was set up to provide oversight and strategic advice to the project board and Senior Responsible Officer as the decision maker.¹²

2.6 The project board and steering committee met as per the terms of reference. Meetings were supported by meeting papers, action items were tracked and meeting minutes recorded. The ANAO reviewed project board papers and minutes from January 2019 to August 2022 and steering committee papers and minutes for the period January 2019 to April 2023. Project updates, schedule, risks, issues, budget and change management were consistently discussed.

2.7 In March 2023 the steering committee agreed to revised governance arrangements to support ‘business as usual’ management of PEMS and the project moving from ICT division (as an IT-led project) to MaPS division (business-led project). The new arrangements included continuing the steering committee¹³, a PEMS management board (previously the project board), and PEMS working group (with a key focus being the prioritisation of the backlog as outlined in paragraph 3.14 to 3.16). The steering committee and management board terms of reference were comparable to their predecessors in terms of roles and delineation between the forums. The steering committee terms of reference were amended to include the committee actively incorporating the experience of users in decision making.

2.8 Other project governance roles are outlined below.

• Senior Responsible Officer/Project Sponsor — chair of the steering committee, oversight and approval, ultimate responsibility for achieving the objectives and benefits. This was the Deputy Secretary, Business Enabling Services.
• Project Executive — chair of the project board, oversight and approval, authorise project status and transitions. This was the First Assistant Secretary of the Information, Communication and Technology (ICT) Division who led the project between 1 January 2018 and March 2023. This was the First Assistant Secretary of MaPS from March 2023.
• Senior Supplier — responsible for producing agreed deliverables, technical integrity and quality of project and represents those designing, developing and procuring the project products. This was the First Assistant Secretary of SDO between 1 January 2018 and February 2021 and from 1 July 2022. The First Assistant Secretary ICT Division (also the project board chair and Project Executive) was the Senior Supplier between February 2021 and 1 July 2022 (covering the period when responsibility for the technical build moved to the ICT Division).

Were there fit-for-purpose project implementation plans in place which were followed by Finance?

Project planning

2.9 The initial project documentation for the PEMS project included a project brief, project management plan and an initial risk and issues register.

2.10 Finance’s guidance outlines that initial project documentation should be provided to the Project Sponsor and governance bodies for review, comment and final approval. The approval process for the initial project documentation did not follow the internal guidance as outlined in Table 2.2. Finance was also unable to provide final signed copies of initial project documentation.

Table 2.2: Initial PEMS project documentation and approval

Note a: A Gateway Review (see paragraph 2.53) in March 2018 included a critical recommendation that a project management plan for PEMS is completed. A subsequent Gateway Review in July 2018 outlined that the action was partially implemented and no date of approval for the project plan was provided.

Note b: The risk register was an operational/living document, with updates discussed at risk workshops and provided to governance bodies with meeting papers.

Source: Initial PEMS project documentation and governance papers.

2.11 The project brief and project management plan included appropriate information specific to the PEMS project, except for sufficient details on scope (see paragraph 2.14) and budget. The 13 capabilities detailed in the project brief did not specify clear or sufficiently detailed deliverables, and budget was provided as a total project cost with no breakdown of how it was to be spent.

2.12 The project management plan was updated in response to the significant changes to scope and timeline — creation of Milestone 7 (plan updated July 2019), delay of Milestone 7 from April 2020 to July 2021 (plan updated January 2021) and splitting Milestone 7 into phase 1 and 2 (plan updated February 2021). A project management plan was not in place to support the delivery of Milestone 8.

2.13 The revised project management plan was approved by the project board each time the project was re-baselined. The approved project plan was provided to the steering committee for visibility, not endorsement as per Finance’s internal project management guidance. The updated plans were also not finalised following the review by governance forums.¹⁴

Project scope

2.14 During the project Finance did not have a clear scope of the PEMS project agreed with MaPS and IPEA as business owners. Scope was inconsistently represented throughout project documentation, presented as 13 core capabilities (see paragraph 3.2) and as high-level milestone deliverables.

2.15 Unclear and inconsistent scope definition created risks and issues throughout the project as evidenced by:

• Gateway Review recommendations — outlined the critical need to finalise business requirements, including governance sign-off;
• discussion surrounding scope at project board and steering committee meetings;
• risks and issues included in the risk register (i.e. scope is referenced in 14 risks and six issues from a sample risk register dated 12 April 2022);
• scope being reported in the project status report as amber or trending down for eight months between September 2019 and May 2020;
• recommendations from a review of the PEMS build (as at 20 November 2020) conducted by SAP that found there needed to be alignment between business, SDO and ICT Division to ensure functional scope was delivered; and
• scope items being moved to the backlog of outstanding deliverables (see paragraph 3.14).

Budget

2.16 Through the 2017–18 Mid-Year Economic and Fiscal Outlook (MYEFO) process the Australian Government approved $38.1 million (to 2020–21) for the development of PEMS.¹⁵ The initial funding included project funding, comprising a mix of capital and operating funding, and ongoing funding to support the operating and maintenance costs of PEMS. Throughout the project, Finance made internal reallocations of funding towards the project and IPEA also made contributions. The PEMS funding and expenditure breakdown is outlined in Table 2.3.

Table 2.3: PEMS funding and expenditure 2017–18 to 2022–23

Note a: The initial MYEFO decision provided for $38.1 million, including $30.2 million in capital funding for PEMS.

Note b: Finance provided a total of $23.4 million and IPEA provided $5.7 million, including $5.0 million in June 2020 and $0.7 million in 2021–22 from its administered funding (see paragraph 2.18).

Note c: The initial MYEFO decision included $2.7 million for operating and maintenance costs for 2019–20, $4.0 million for 2020–21 and then $4.1 million per financial year ongoing funding. This figure includes the initial $6.7 million to 2020–21 from the initial MYEFO announcement and $4.1 million per annum for 2021–22 and 2022–23.

Source: PEMS budget breakdown provided by Finance.

2.17 Finance’s main internal funding reallocations were provided in June 2019, December 2020, and March 2021. These budget injections were approved by Finance’s Executive Board. Specific papers were provided to the Executive Board outlining the need for the additional funding for the March 2021 decision. The June 2019 and December 2020 decisions were part of Finance’s broader process for requesting internal budget adjustments. The PEMS project was included in the overall request for bids.

2.18 IPEA contributed $5.7 million to the project from its administered appropriation related to Outcome 1, consisting of $5.0 million in June 2020 and a second transfer of $0.7 million in 2021–22.¹⁶ This was used by Finance for departmental expenditure on the PEMS project.¹⁷ There was no documented evidence of Finance’s decision making and how Finance considered the relevant frameworks to apply to the transactions, noting that money initially appropriated for administered purposes was used for departmental purposes. This increases potential risks to the controls put in place by the Parliament, in particular to ensure that appropriations are used for the intended purpose and with the Parliament’s permission.¹⁸

2.19 Finance advised the ANAO on 4 October 2023 that total expenditure to 30 June 2023 was $74.3 million, including $60.7 million from the PEMS project budget. This represents a 96 per cent increase from the original PEMS project budget. In Finance’s financial statements as at 30 June 2023, the total expenditure is reflected as $45.9 million of software assets. The remaining $28.5 million spent has been recognised as operating expense through 2017–18 to 2022–23, including $3.9 million of impairment.

2.20 In addition, Finance was provided $4.0 million in the 2023–24 Federal Budget for further development of PEMS during 2023–24 (see paragraphs 3.14 to 3.16).

Workforce

2.21 Finance advised the ANAO that the majority of expenditure for the PEMS project was on contractors, consultants, outsourcing and employees.¹⁹ For Finance to build PEMS in-house the business case stated that a fully contracted workforce would be required. Finance advised that the project workforce planning was covered under an initial procurement plan. This document is the approval to approach the market for resources for PEMS and another project. It was not a discrete document that maps the resources or capability required over the project and it did not specify what resources were specifically for PEMS. The initial PEMS project management plan listed the areas resources would be required (for example, organisational change management and technical architecture and configuration) and included a high-level structure of the SDO delivery teams. A resourcing plan was implemented for the updated project management plan for Milestone 7. The plan covered July 2019 to June 2020, and outlined the roles required, expected effort (represented as estimated hours/days of work required and full-time equivalent resourcing) according to build component (for example, payroll services, travel, office and budget).

2.22 During the project additional resources were required. Finance advised the ANAO that the decision to procure additional project resources was managed through changes to the integrated project plan and schedule. Updates were made to the integrated schedule throughout the project. However, Finance was unable to demonstrate how changes to resource requirements were operationalised. For example, when the schedule was delayed there was no evidence that the resource plan was updated in response. Conversely, when there was budget overspend there was no separate documentation or process that captured a new resourcing profile to reduce expenditure.

Project schedule and timing

2.23 The business case, project brief and project management plan included the proposed timeline for project delivery, outlined through a series of milestones. Each milestone had a brief description of the expected functionality to be delivered and the proposed delivery date. The milestone descriptions were presented differently to the way project scope was detailed, which was through the 13 core capabilities (see Table 3.1).

2.24 The project experienced delays in particular for the delivery of the Enterprise Resource Planning (ERP) solution for human resources, office and travel expenses and reporting. In addition, milestones were re-named, functionality was moved between milestones and scope not delivered as further discussed in paragraphs 3.8 to 3.16. The changes to the milestones are set out in Appendix 3. Key points in the delivery timeline are outlined below.

• The original Milestones 1 to 5 were largely delivered on time with minor changes to timeframes and milestones re-named.²⁰
• Milestone 6 was originally the delivery of the ERP solution, including implementing financial backend related functionality, human resources and integrated expenses management for office and travel claims. The original due date for Milestone 6 was 1 July 2019. In February 2019 the steering committee agreed to deliver the financial backend functionality on 1 July 2019 and deliver the remaining functionality for the project in Milestone 7 with a new delivery date of April 2020.
• In September 2019 the steering committee agreed to delay the delivery of Milestone 7 to 1 July 2020 and in May 2020 it agreed to further delay the delivery to 1 July 2021.
• In December 2020 the steering committee agreed to split Milestone 7 into two phases — phase 1 for human resources (due 1 July 2021) and phase 2 for expense management of office and travel claims and expenditure reporting (with delivery date to be set later).
• In July 2021 the steering committee agreed to set the delivery date of Milestone 7 phase 2 as April 2022. In December 2021 the steering committee agreed to move this to July 2022.
• Further delays occurred for the expenditure reporting component of PEMS, discussed further in paragraph 3.20.

2.25 Decisions to delay milestone releases were approved by governance forums (both project board and steering committee), with the reasons for the main delays outlined in Table 2.4.

Table 2.4: Reasons provided to project governance on key delays in the project schedule

Note a: The steering committee decided in July 2021 to set phase 2 as April 2022.

Note b: The build component of the project was put on hold due to a fraud case which resulted in termination of IT resources on the project and the need to hire replacement IT resources.

Source: ANAO analysis of PEMS project and governance documentation.

2.26 The project delays increased the project length from the proposed two and a half years to more than five years duration. This three-year delay has impacted the project resources required.

User requirements

2.27 Finance did not have a consistent approach to gathering business requirements, in particular for the full ERP solution for human resources and office and travel expenses. Various approaches were used to gather and confirm requirements throughout the project.

• High-level requirements developed by Finance in 2017 that informed the IT solution (as an ‘commercial off-the-shelf’ product) and the intended core capabilities of PEMS.
• Business process models — documents developed by the SDO between August 2018 and November 2020 that articulate system business rules.
• User stories — as a parallel process to the business process models. Between August and October 2018 Finance worked with IPEA to develop user stories to cover a high-level statement of what the system should do.
• Knowledge transfer workshops — demonstration workshops were held in March 2020 to show and explain the solution to business areas to assist with user acceptance testing. Subsequent reporting to the project board showed these sessions highlighted design gaps that required further IT build work.
• Solution playback sessions — these sessions were held between May 2020 and June 2020 to identify system operability issues and gaps between what was required functionality and what functionality was available in the system at the time.
• List of ‘essential items’ — for Milestone 7 phase 2 (July 2022) a list of ‘essential items²¹’ was developed (informed by the solution playback sessions and the work of the design authority). This would form the minimum viable product (MVP) for delivery of phase 2.

2.28 Finance did not articulate the connection between the above approaches and whether these approaches captured a complete view of user requirements to inform the PEMS build.

2.29 Throughout the project there were delays in agreeing business requirements. The project status report (see paragraph 2.48) consistently reported requirements definition (under ‘design’) as red and amber throughout the project. On 26 November 2019 the steering committee agreed to ‘baseline²²’ the requirements in the business process model documents without final approval from IPEA as business owners of the travel expenses component. The steering committee minutes noted that IPEA did not agree to this approach. There is no evidence that the travel expenses requirements in the baselined documents were subsequently formally agreed by Finance and IPEA.

2.30 The delays in agreeing business requirements resulted in Finance building and customising the system, as early as February 2019, without requirements being agreed by business owners. Finance called this ‘building at risk’ and the approach was noted in discussions at governance forums and in risk registers. The risk of this approach was realised when the system build had to be re-worked when it was identified that the solution would not meet business requirements.

2.31 Unclear requirements and poorly defined scope at the start of the project also led to incorrect assumptions on the amount of customisation required from a ‘commercial off-the-shelf’ IT system. The initial project plan had an assumption that no more than 10 per cent of the system would need to be configured. Finance advised that it did not track the percentage of configuration required, and that office and travel expense management was a largely bespoke system and Milestone 8 is an entirely bespoke system. This meant that more IT build work was required than planned, leading to schedule delays and continued increases in project cost.

Project change management

2.34 The project management plan outlined the approach to managing changes to the ‘approved baseline for the project’.²³ This included a change request template to capture the details and impact of the change and a change register to track requests over the project.

2.35 There is no evidence that the governance bodies used this process to effectively manage change in the project. Separate scope decisions were being made by business areas without undergoing the change request process. For example, MaPS approved changes in 2019 to the scope of human resources capability to remove on-boarding, personnel administration, time administration and organisational planning. Further, in addition to the change request process, system requirements that were not delivered were placed on a backlog (see paragraphs 3.14 to 3.16). Finance’s processes were unclear on what items should have gone through the change request process and what decisions could be made outside the process.²⁴

Risk management

2.36 The PEMS project was initially rated as a medium risk project as outlined in the business case. During the project, significant changes to schedule, budget and scope occurred. There is no evidence that Finance reevaluated the overall project risk rating during the project.

2.37 The PEMS project had risk documentation that aligned to Finance’s enterprise risk management guidance from 2017. Project risk documentation and risk management activities for the project included:

• risk register;
• risk workshops;
• PEMS project risk and issue management plan; and
• risk reporting.

2.38 The risk register was established at project onset. The risk register was updated for individual risks regularly, and included risk description, source/cause, date raised, current controls, proposed treatments and risk owners. Project issues were also included in the risk register.²⁵

2.39 Finance advised that risk workshops were held approximately monthly, were not formal meetings, minutes were not kept and that the workshops were responsible for maintaining and updating the risk register. Finance also advised meetings included Project Executive, Project Manager/Director and representatives from IPEA, MaPS and the SDO.

2.40 The PEMS project risk and issue management plan was established 18 months into the project, in July 2019. It provides high-level information on how risks and issues are to be managed throughout the project.

2.41 Risks and issues were reported at every project board and steering committee meeting and were included in the project status reports provided to governance forums (see paragraph 2.48).

2.42 Although risk documentation and risk management procedures were in place, it did not necessarily prevent risks being realised. Table 2.5 provides an example of a key risk, including risk description, source/cause of the risk, current controls and proposed treatments.

Table 2.5: How budget risks were managed through the risk register

Note: The above table provides a snapshot of how budget risks were addressed throughout the project. Risk #42 was the first risk to address budget issues in October 2019. Additional risks were raised in May 2020 and June 2022 to address separate budget issues. This included potentially ceasing the project unless additional funding is sourced, and the cost of enhancements was not affordable without further funding. A project issue was raised in response to the realisation of several risks in October 2022, with the cost to deliver the remainder of the project exceeding the available funds. This issue has not been closed.

Source: Risk registers provided by Finance.

2.43 The ANAO assessed the effectiveness of the treatment in managing the budget risk. The risk controls in the risk registers are focused on reducing the risk through monitoring or remediation once the risk is realised, i.e. seeking additional funding and re-directing existing Finance employees to the project instead of using contractors. However, the project had budgetary risk throughout the project and both the monitoring and reactionary remedial actions did not effectively manage the risk of budget overspend.

Benefits realisation

2.44 The benefits of PEMS to parliamentarians, administering agencies and government were outlined in the business case. These included: client centric service delivery; improved administration efficiency and effectiveness; increased transparency of parliamentarian expenses; providing agility in enabling ICT to cater for future changes; and value for money by minimising ICT capital investment. Benefits management was included in the project brief, the initial project management plan and the updated plans.

2.45 At times during the project Finance attempted to develop processes to track benefits, for example a benefits realisation plan was developed in 2019 and updated in 2021. Given the delays to the project, reduced scope and user views of the system, coupled with Finance not having a current process in place to track benefits, there is no evidence that the expected benefits of PEMS have been realised. Finance advised it intends to undertake work on benefits realisation as part of the two to three year strategy for PEMS improvements, with the approved strategy due early 2024.

Was there effective reporting to relevant stakeholders?

Project reporting

Project status reports

2.48 Project status reports were provided to the project board from September 2018 until August 2022 and to the steering committee until March 2023. The reports contained an overview of the project status and a range of subcategories aligned with project components, such as schedule, resources and testing. The report used a ‘traffic light’ system to rate the overall project status and against the separate subcategories, with accompanying points to provide context to the ratings. Figure 2.2 shows the overall status results reported to the project board and steering committee.

Key: ■ = project status reported as red (one or more of the subcategories were rated red).

▲= project status reported as amber (more than one subcategory was rated amber but no red in the subcategories).

◆ = project status reported as green (all subcategories rated as green or only one amber subcategory, with no red rated subcategory).

Note A steering committee meeting was not held in January 2023.

Source: ANAO analysis of project status reports.

2.49 The project status reports provided sufficient detail on project implementation. However, throughout the reporting of the project ‘risks’, ‘issues’, ‘governance’ and ‘project management’ continued to be rated as green. The description of a green rating in the report was ‘identified issues being managed and updated regularly’. The updates in the report under these subcategories were more around routine matters rather than providing a clear view of project issues. For example, under ‘governance’ the update was the next meeting date and under ‘risk’ the update was how many risks were in the risk register (regardless of their severity). This approach to reporting masked the overall project health.

Other operational reporting

2.50 Other operational reports were produced to support the project, for example:

• PEMS weekly status report — these reports were originally a two–four page summary by respective areas of the project (for example SDO, ICT Division and data migration). The format changed from May 2020 to support increased reporting in response to delaying Milestone 7. Key components of the updated report were a traffic light assessment across project components (for example, schedule, testing), key activities in the previous and upcoming week, escalation of issues and key milestones. Finance advised these were provided to the Project Executive and business areas (MaPS and IPEA) involved in the project teams.
• Adoption tracker — the project board received statistics on the usage of PEMS (smart form solution) from October 2019 to September 2021. The statistics included the number of users and transactions performed. Meeting minutes from the project board did not demonstrate discussion of the contents of the tracker.
• Routine updates were also provided to the Major Projects Committee between July 2021 and September 2023.²⁶ This committee did not have governance responsibility for PEMS and the project was presented for information only.

Ministerial briefings

2.51 Finance provided briefings to the Minister for Finance and the Special Minister of State as outlined in Table 2.6.

Table 2.6: Summary of briefs on PEMS provided to Ministers during the project

Note: PEMS was also covered in the incoming government brief for the Special Minister of State.

There was a gap in briefings between July 2018 and October 2020. Finance advised that this was in part due to the 2019 election and ministerial and machinery of government changes and the Australian Government and Finance responding to natural disasters and the COVID-19 pandemic.

Source: ANAO review of ministerial briefs provided by Finance.

Secretary briefings

2.52 Three written briefs were provided to the Secretary of Finance (September 2018, November 2020 and June 2021) on the PEMS rollout and project issues. Operational updates on project activities were also provided.²⁷ The Executive Board of Finance received updates on PEMS in February 2021 and when it agreed to provide additional funding to the project in June 2019, December 2020 and March 2021 (see paragraph 2.17). The Secretary is chair of the Executive Board.

Gateway Reviews

2.53 Finance has responsibility for the Gateway Review policy and co-ordination of Gateway Reviews across non-corporate Commonwealth entities. Gateway Reviews aim to facilitate effective planning, management and delivery of major IT projects being delivered in entities.²⁸ Gateway Review reports are intended to be given to the Senior Responsible Officer to provide an assessment of issues that may jeopardise delivery and benefits. Five of the seven PEMS Gateway Reviews were addressed to the Project Executive as Senior Responsible Officer and not the actual Senior Responsible Officer (Deputy Secretary).

2.54 The business case approved by government for PEMS included that the project would be subject to Gateway Reviews. As at October 2023, the project has undergone seven Gateway Reviews as outlined in Table 2.7.

Table 2.7: Summary of Gateway Reviews and outcomes

Note a: The Gateway Reviews process includes multiple reviews throughout the project at key points (referred to as ‘gates’ for project or ‘review stages’ for programs which have interrelated projects). Each review has a specific focus and the number of reviews is determined by the complexity and timeframes of the project or program. Depending on structure of the program or project a blended approach between gates and review stages can occur. More information is available in Guidance on Assurance Reviews Process, Resource Management Guide 106 [Internet], Department of Finance, available from https://www.finance.gov.au/publications/resource-management-guides/guid…- [accessed 4 November 2023].

Note b: The PEMS project had a combined review for gate 0–3 in March 2018 given the first deliverable was due in July 2018. There were multiple reviews for stage 4 with each looking at different milestone delivery. A blended approach was adopted from February 2021 to combine the review points of a project and program.

Note c: Red was defined as: ‘Successful delivery of the program/project appears to be unachievable. There are major issues on program/project definition, schedule, budget, quality or benefits delivery. The program/project may need to be re-baselined and/or overall viability re-assessed.’

Note d: Amber was defined as: ‘Successful delivery of the program to time, cost, quality standards and benefits realisation appears feasible but significant issues already exist requiring management attention. These need to be addressed promptly.’

Note e: Green/Amber was defined as: ‘Successful delivery of the program to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery.’

Note f: Green was defined as: ‘Successful delivery of the program/project to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly.’

Source: Summarised by ANAO from Gateway Review reports and meeting minutes from the project board and steering committee.

2.55 ANAO analysis of gateway review ratings against project status report ratings for the overall project status (see paragraph 2.48) shows discrepancies as outlined in Figure 2.3.

Figure 2.3: Gateway Reviews and project status reporting timeline

Note: Project status reports started in September 2018.

The May 2019 red rating in the project status report was due to delays in the schedule for data migration and preparation for the technical cutover (series of activities to prepare for the new system) for Milestone 6. The delay was resolved by the next report.

Source: ANAO analysis of Gateway Review reports and PEMS project status reports.

2.56 Factors impacting the overall rating in each review are outlined below.

• March 2018 — rated as amber with the report noting that Finance was using a ‘commercial off-the-shelf’ solution and implemented by the SDO that has used the technologies with other agencies, which lowers the technical implementation risk. The review however noted the project was under schedule pressure, it was unclear whether a ‘commercial off-the-shelf’ solution would meet the expectations of parliamentarians and their staff and that extensive communication and organisational change management was required for successful implementation.
• July 2018 — rated as amber noting the schedule continued to be under pressure and it was still unclear if the scope of the work would meet expectations of all stakeholders or if the budget would be able to meet emerging requirements from business areas. The review noted that if the recommendation to capture and prioritise user needs and requirements is not implemented promptly the project may become unachievable.²⁹
• May 2019 — rated as green/amber noting progress was being made, including implementation readiness for financial related functions (Milestone 6); the new delivery date of April 2020 is achievable and that it was anticipated that Finance’s Executive Board would fund increased scope.
• February 2021 — rated as amber noting issues outside the control of the project during 2020 (for example, IT build paused due to loss of technical resources and need to re-hire) that necessitated replanning and re-baselining. The report also noted issues involving clearer responsibilities and accountability within the project, business readiness and operational support. The report did note the considerable work in progress as a mitigating factor.
• May 2021 — rated as amber due to issues with the readiness for the next phase (Milestone 7 phase 2) and with stakeholders/end-user engagement. The review stated these issues were mitigated with the planned successful delivery of Milestone 7 phase 1 and green ratings for governance and planning and risk management. Although Milestone 7 phase 1 was delivered on time, it was with reduced functionality (see paragraphs 3.10 to 3.16).
• April 2022 — rated as green/amber with delivery of Milestone 7 phase 2 (due 1 July 2022) a ‘high probability’. The report noted issues in arrangements for post ‘go-live’, deferral of expenditure reporting and testing and defect remediation.
• December 2022 — rated as amber noting PEMS had ‘been delivered and is operating’ and the schedule for expenditure reporting (Milestone 8) ‘appears reasonable’. However, the report notes considerable stakeholder concerns, the budget is exhausted and work is required to explore usability issues.

2.57 The definition of an overall red rating was met during the project, for example there were ongoing issues with the scope of the project, the schedule (and therefore benefits realisation) was re-baselined several times, and the budget was supplemented by additional funding. However, Gateway Reviews are a point in time assessment and the PEMS reviews occurred after the project was re-baselined. The reviews therefore looked at the overall likelihood of success for the project based on the new parameters. This approach potentially reduced the impact of the Gateway Reviews in being able to appropriately escalate issues for action by the Senior Responsible Officer.

2.58 The December 2022 Gateway Review listed two further reviews that needed to be completed. The first in May 2023 to confirm the readiness of Milestone 8 and the second in December 2023 to conclude the project. The review covering Milestone 8 commenced in September 2023.³⁰

2.59 The ANAO examined the recommendations made in the Gateway Review reports and their level of urgency as set by the Gateway Review teams, shown in Figure 2.4.

Figure 2.4: Number of recommendations raised in review reports for March 2018 to December 2022 (broadly categorised by ANAO) and their assigned urgency

Note: The Gateway Review recommendation categories are:

Note a: Critical (do now): ‘To increase the likelihood of a successful outcome it is of the greatest importance that the program should take action immediately.’

Note b: Essential (do by): To increase the likelihood of a successful outcome the program should take action in the near future. Whenever possible essential recommendations should be linked to program milestones (for example, before contract signature and/or a specified timeframe i.e. within the next three months).

Note c: Recommended: The program should benefit from the uptake of this recommendation. If possible, recommendations should be linked to program milestones (for example, before contract signature and/or a specified timeframe i.e. within the next three months).

Source: ANAO analysis of Gateway Review reports

2.60 When progress of the recommendations was reviewed in subsequent Gateway Reviews, recommendations in the planning and governance categories were more likely to be fully met, as shown in Figure 2.5. Recommendations in the stakeholders and end users category were predominantly partially met, with the implementation of some recommendations remaining unclear to the gateway reviewers.

Figure 2.5: Outcomes of the Gateway Review recommendations for March 2018 to April 2022 as detailed in the subsequent Gateway Review

Note: This figure does not include if recommendations from the December 2022 Gateway Review have been completed as the subsequent Gateway Review has been delayed and not conducted as at October 2023.

The N/A relates to a project to support the COMCAR Automated Resource System (CARS project). This project was included in the PEMS second pass business case but managed as a separate project. The July 2018 Gateway Review was a combined PEMS and CARS review with some recommendations for the CARS project. These are represented as a N/A in the figure as the PEMS project did not need to implement these.

Source: ANAO analysis of Gateway Review reports.

2.61 Gateway Reviews have an enhanced notification process which requires escalation of issues to an agency’s accountable authority and/or Minister. This process is triggered by red ratings or sequential amber or amber/red ratings.³¹ The project twice received two consecutive amber ratings that were then followed by a green/amber rating, triggering the first level of enhanced notification. Finance complied with the enhanced notification process after both sets of consecutive amber ratings.³²

External reporting

External performance reporting

2.62 The Public Governance, Performance and Accountability Act 2013 (PGPA Act) establishes a framework for monitoring and evaluating performance that requires entities to have meaningful performance information which relies upon a clear understanding of the entity’s purpose and expressing it in a way that is related, measurable and complete.

2.63 PEMS was included in the performance measures in Finance’s corporate plan and annual reports from 2017–18 to 2023–24. The 2022–23 measure is depicted in Table 2.8.³³

Table 2.8: Finance performance measure for PEMS 2022–23

Note a: The corporate plan defines a tick as: ‘that an assessment approach will be used to measure performance for the given reporting period/s.’ The methodology detailed against the performance measures is ‘Measured through provision of additional functionality and increased number of claims processed through PEMS’.

Source: Finance’s Corporate Plan 2022–23.

2.64 The ANAO assessed the 2022–23 measure and target against section 16EA of the Public Governance Performance and Accountability Rule 2014 (PGPA Rule) as set out in in Table 2.9.

Table 2.9: Assessment of performance measure 2022–23

Key: ■ Does not meet requirements.

▲ Partially meets requirements.

◆ Fully and/or mostly meets requirements.

Note a: Related refers to the requirement of subsection 16EA(a) of the PGPA Rule 2014, as amended. In applying the ‘related’ criterion, the ANAO assessed whether the entity’s performance measures:

related directly to one or more of the entity’s purposes or key activities; and

provided a clear link between purposes, key activities and performance measures.

Note b: In applying the ‘measurable’ criterion, the ANAO assessed whether the entity’s performance measures were:

reliable and verifiable — supported by clearly identified data sources and methodologies;

free from bias — provides an unbiased basis for the measurement and assessment of the entity’s performance; and

assessable over time — able to provide a basis for an assessment of performance over time.

Source: ANAO analysis based on the PGPA Act and PGPA Rule.

2.65 The measure, ‘improve administration of parliamentary work expenses — the timeliness, efficiency, clarity and transparency of the administration of parliamentary work expenses is improved’ and the associated target of ‘increased usage of PEMS’ target was determined to be related to a key activity, however, is not measurable as it is not reliable, verifiable, free from bias or assessable over time. This is based on the following factors.

• The measure relates to Finance’s key activity to ‘deliver ministerial and parliamentary services’ under its purpose to provide services to and for government.
• The measure and description are not reliable or verifiable with terms such as ‘timeliness’, ‘clarity’ and ‘transparency’ used without any definitions. It is not clear how these would be measured or verified nor how the target or underpinning methodologies correlate to these concepts.
• In addition, the data source for this measure is a spreadsheet which requires a high level of manual intervention, impairing the reliability and validity through increased potential for errors.
• The measure and target were not free from bias as the methodology for the target of ‘increased use’ was based on number of registered users and number of certified claims paid. This does not provide a complete view of PEMS performance and how it contributes to improved administration.
• The measure and target are not assessable over time as the target and methodology does not allow for baseline or benchmarking for improved performance. The single target does not give a view of the effectiveness or efficiency of PEMS, rather it is a target based on registrations for a system that is mandatory (aside from paper-based forms) and for which users register for once.

2.66 For the 2022–23 results, the target was revised to ‘sustained use — 99 per cent of offices submitting and certifying claims using PEMS’. Despite the change to the target mid-way through the reporting cycle, the results continue to be based on user registrations and number of certified claims being paid. The discrepancies between the Corporate Plan and Annual Performance Statements are outlined in Table 2.10.

Table 2.10: Comparison between 2022–23 Corporate Plan and 2022–23 results

Note a: In its 2022–23 Corporate Plan, Finance stated the target was ‘increased usage’. In its 2022–23 Annual Performance Statement it reported against a target of ‘sustained usage’.

Source: Finance’s 2022–23 Corporate Plan and Finance’s 2022–23 Annual Performance Statements.

2.67 Finance reported in its Annual Performance Statements that it fully achieved the target of ‘sustained usage’ in 2022–23 — at least 90 per cent of parliamentarians have registered to use PEMS, and of the 138,730 office claims/transactions lodged in PEMS 137,940 (99.4 per cent) have been paid.

2.68 The PEMS measure in Finance’s 2023–24 Corporate Plan has been updated to ‘Improved administration of parliamentary work expenses — sustained usage of PEMS by parliamentarians and their staff and successful processing of claims in PEMS’, removing reference to timeliness, efficiency, clarity and transparency from the measure description. There are three corresponding targets against the measure: sustained usage of PEMS by parliamentarians and their staff; successful processing of claims in PEMS; and availability of public reporting on parliamentarian work expenses. The new measure description and targets represent an improvement, although they could still be strengthened to better address the intent of the performance measure to improve administration of parliamentary work expenses.

Reporting to the Digital Transformation Agency

2.70 As part of its role to provide advice to the Australian Government on digital investments, the Digital Transformation Agency (DTA) collects information from agencies on the status of their digital and IT projects. Reporting provided to DTA by Finance detailed the PEMS project’s progress, budget, benefits, risks, and a rating of the delivery confidence. Finance’s self-reported ratings for delivery confidence were medium-high for the first year of the project in 2018 followed by a reduction to medium and eventually low by the October 2020 report. Ratings gradually rose again to high by June 2022. The September 2022 rating was medium, followed by two high ratings in January and May 2023. The July 2023 rating reduced to medium-high due to a delay in the schedule caused by further clarification on business and testing requirements.³⁴ The DTA used these reports, along with its membership on the steering committee, to monitor the project’s progress and brief its executive and the Minister of Finance as minister responsible for digital transformation.

3.1 PEMS was to be the key enabling IT system to support better administration, accountability, and transparency of work expenses for Parliamentarians and their staff. As parliamentary expenses and workplaces are covered by a legislative framework, it is critical that PEMS not only aligns with the high-level legislative requirements but also meets the needs of users to facilitate user adoption and makes the system usable and simple to support parliamentarians and their staff to operate within the rules.

PEMS capabilities

3.2 The ANAO assessed the current functionality as of December 2023 against the original PEMS capabilities as outlined in the business case and project brief. The assessment considered the high-level definition of the 13 PEMS capabilities³⁵ and whether the functionality described has been released (see Table 3.1).

Table 3.1: Scope breakdown according to capability and delivery

Key: Delivered = Capability delivered.

Partial = Scope was either reduced from original business case or full functionality was not (or is yet to be) delivered.

Note a: The description provided in quotation was provided only in the approved project brief and was not included in the business case.

Note: Capability C8, C12 and C13 were delivered when Milestone 8 was complete, with the Independent Parliamentary Expenses Authority (IPEA) publishing its first report in December 2023.

Source: ANAO analysis against PEMS business case and project brief.

Does the system allow for correct processing of expenses and payroll services?

3.3 The Parliamentary Business Resources Act 2017 states that the Commonwealth must pay for expenses as prescribed in the regulations relating to the conduct of a member’s parliamentary business. Section 66 of the Parliamentary Business Resources Regulations 2017 sets out what office expenses are prescribed and include items such as office supplies, printing and communications, and ICT and telecommunications. Claims must be within a set budget.³⁶

3.4 The Parliamentary Business Resources Act 2017 also provides examples of expenses that can be paid for travel, such as cost of fares and travel allowance. This is supported by the Independent Parliamentary Expenses Authority Act 2017 that requires the Independent Parliamentary Expenses Authority (IPEA) to process claims (travel expenses) and incur expenses (travel allowance) for travel resources for parliamentarians and their staff. Parliamentarians and their staff can claim for travel.³⁷

3.5 The principles outlined in the legislation are supported by regulations, determinations, supporting documentation and procedures.

Business process and system capability

Claiming office and travel expenses

3.6 Office and travel claims are either:

• created and submitted directly into PEMS by parliamentarians or their staff;
• submitted by parliamentarians or their staff via a form which is then manually entered into PEMS by Finance or IPEA staff; or
• paid directly to a supplier under separate contractual arrangements between Finance or IPEA and a preferred supplier.³⁸

3.7 Office and travel claims go through a two-step process of certification by the parliamentarian and verification by Finance or IPEA to process payment. If the parliamentarian submits the claim in PEMS, the parliamentarian certifies the claim in the system. If they submit a manual form, Finance or IPEA creates a new claim and completes the certification in the system based on certification contained in the manual form, with a second officer in Finance or IPEA verifying the claim. Table 3.2 and Table 3.3 summarise the high-level process for office and travel expenses.

Table 3.2: Office expenses claiming process

Note a: If it is a parliamentary staff member arranging the office expense they can create the claim but then submit it to the parliamentarian in PEMS for the parliamentarian to certify.

Note b: Finance has advised that as of 9 May 2023 there are 20,100 vendors set up in PEMS.

Source: ANAO summary of IT functionality observed during IT demonstrations.

Table 3.3: Travel expenses claiming process

Note a: For travel allowance claims evidence may not be available at time of entering claim (for example, if travel allowance is claimed prior to travel and the parliamentary staff has not paid for accommodation yet). In these cases, the staff select, ‘available on request’. IPEA then does assurance checks on a sample of these claims based on their assurance processes.

Source: ANAO analysis of IT functionality observed during IT demonstrations.

3.8 Overall, PEMS supports the key steps in claiming office and travel expenses, enabling the payment of office and travel claims submitted by parliamentarians and their staff. There are a range of controls in place to provide for accurate processing of office and travel expenses, noting the parliamentarian has ultimate accountability that the claim meets the parliamentary expenses framework when submitting the claim.

3.9 There is a backlog of undelivered functionality and system incidents/defects for expense management (office and travel) discussed further in paragraph 3.14 to 3.16. This means that the current functionality needs to be supported by manual workarounds by Finance and IPEA.

Payroll services

3.10 Parliamentarians employ staff within their offices under the Members of Parliament (Staff) Act 1984. The conditions of employment are primarily determined by the Commonwealth Members of Parliament Staff Enterprise Agreement. The business case for PEMS acknowledged the complexity of these employment arrangements as a cost driver for the PEMS solutions, for example concurrent employment across parliamentary offices is a practice that is seen in this operating environment that is not common in other organisations.

3.11 Finance supports parliamentarians in administering employment arrangements through a range of payroll and human resource services which PEMS was intended to support. This includes payroll services (including administering leaving and allowances), human resource policy and work, health and safety.

3.12 The payroll component of PEMS uses a SAP-solution that has not been customised to support the unique employment arrangements of parliamentary staff. This has resulted in manual workarounds for key components of payroll services, with examples outlined in Table 3.4.

Table 3.4: PEMS functionality to support payroll services

Key: ○ No functionality delivered ◑ Partial delivery ● Functionality delivered through PEMS.

Note: For example, MOP(s) Act employees can access additional leave if they are required to travel from a rural or remote location to Caberra on a Sunday or Public Holiday.

Source: ANAO summary of IT demonstrations and walk-throughs.

3.13 Overall, the system capability delivered for payroll services meets basic requirements, i.e. system capability for pay to be processed and key leave types supported through self-service. However, payroll services are at risk of incorrect processing given the extensive reliance on manual workarounds, for example on-boarding/off-boarding process and manual calculation of more complex leave types such as long service leave. A key driver of the PEMS project was to replace previous systems that had high levels of manual interventions.

Backlog

3.14 During the project, a backlog of in-scope items that were not delivered prior to scheduled release or lower severity incidents/defects from earlier releases was created. There was discussion at governance forums throughout the project, including on the process for managing and prioritising the backlog. For example, in May 2021 the steering committee noted there were 471 items on the backlog. In response, a specific working group — the design authority — was established to prioritise the backlog and manage how it related to the essential items list for Milestone 7 phase 2.

3.15 From April 2023 the new PEMS Working Group (see paragraph 2.7) commenced a prioritisation process of the current backlog. As at August 2023, there were 348 open items in the following areas³⁹:

• travel expenses (domestic and international) — 172 items;
• office expenses — 56 items;
• reporting — 40 items;
• human resources — 48 items; and
• other — 32 items.

3.16 The PEMS Working Group assigned backlog items a priority rating from one to four, based on a group of drivers such as client impact, legislative impact and resourcing impact. Human resources, travel and reporting were assessed by Finance as the top priorities. Finance advised the ANAO on 18 December 2023, that ‘members of the 25 October Management Board agreed to prioritise the top 5 PEMS issues from each business group and put these forward to the SDO for remediation’ and that the steering committee noted these items in December 2023. The $4 million allocated in the 2023–24 Federal Budget for further development of PEMS will address these backlog items. Finance further advised that the remaining items on the backlog have been prioritised by business owners and will be developed into a release schedule for the next two to three years.

Internal assurance process

3.17 On 14 August 2023 Finance advised that in July 2023 it had approached the market to engage a system auditor for legal, assurance, fraud and risk advice on PEMS in July 2023. This is in response to a recommendation from the December 2022 Gateway Review that there was no evidence of an assurance plan or assurance activities that demonstrated PEMS conformed with legislative requirements. Finance further advised the ANAO on 18 December 2023, that the final report for this work is due mid-February 2024.

Testing of IT general controls

3.18 The ANAO undertook IT general controls⁴⁰ assurance testing of PEMS in August 2023. No issues were found in terms of privileged user access or segregation of duties between processes, and system maintenance is managed through enterprise change control.

Is the system designed to facilitate transparent reporting on expenses?

3.19 Under its legislation IPEA is required to prepare and publish regular reports on office and travel expenses, which it does on its website and data.gov.au.⁴¹ A core requirement of PEMS was functionality to facilitate this expenditure reporting, including moving to monthly public reporting. Since 1 July 2022 IPEA was unable to publish expenditure figures, until December 2023 where it published its first reports using PEMS expenditure reporting functionality for the period July–September 2022.⁴² IPEA has advised it uses the data from PEMS to conduct its audit functions off system.

3.20 The expenditure reporting functionality was initially due to be delivered in January 2020, with delays experienced throughout the project as described below.

• In February 2019, the steering committee agreed that expenditure reporting would become part of the new Milestone 7 with an expected delivery date of April 2020.
• In May 2020, the steering committee agreed to push the delivery date of Milestone 7 to July 2021 and in December 2021 the steering committee agreed that expenditure reporting would be part of Milestone 7 phase 2.
• In April 2022, IPEA agreed that public expenditure reporting would be delivered after the July 2022 release⁴³, with other operational reports for parliamentarians and IPEA still being delivered as part of the July 2022 release.
• In August 2022, the steering committee confirmed the release as 31 October 2022.
• In November 2022, the steering committee agreed to a revised timeframe — build to be completed by 30 June 2023 and in use by IPEA in September 2023. A formal Milestone 8 was also created in recognition of the work required.

3.21 Given this timing, the analysis of the expenditure reporting is based on ANAO’s observation of the design and build at the time of fieldwork (April to August 2023).

Reporting requirement and system capability

Expenditure reporting

3.22 To support the publishing of office and travel expenses data IPEA had the following high-level requirements for the expenditure reporting component of PEMS: a workflow to enable users to view, query and certify reports; the ability to extract the expenditure report in a format that can be published; and to be able to move to monthly reporting. These high-level requirements were supplemented by several lower-level requirements captured in supporting documentation that was agreed between IPEA and SDO and ICTD. There were also approved change requests to support changes to these requirements as needed.

3.23 If implemented according to design, there are capabilities in PEMS to support public reporting. Features of the process included:

• the report generating the data from the other office and travel expense modules into the reporting module;
• a workflow process that includes: IPEA creating the reporting period; parliamentarians certifying the report to be published and then IPEA extracting the approved data and publishing;
• system capability for IPEA to adjust data deemed ‘not for publication’ which removes the data from the report⁴⁴;
• supporting manual options, for example a Parliamentarian signing a printout or PDF softcopy and uploading as evidence of certification;
• office staff being delegated to review reports prior to parliamentarian certifying report; and
• data controls.⁴⁵

Was the system designed and implemented in a user centric manner?

User considerations through project planning

3.24 PEMS end users were included in initial project planning documentation as outlined in Table 3.5.

Table 3.5: Summary of how end users were represented in planning documentation

Source: PEMS project planning documentation.

Implementation

User reference group

3.25 Finance advised the ANAO in May 2023 that interactions with parliamentarians and their staff was primarily conducted through the PEMS user reference group. The user reference group was also listed as a mitigation to users not utilising PEMS in the initial project brief.

3.26 The user reference group was initially established in July 2018, with the Finance Minister issuing a circular to invite all current parliamentarians and their staff to participate in the group. The purpose of the group as outlined in its terms of reference were to, ‘participate in the design of the new system, making sure it is easy to use and meet their requirements’. The terms of reference indicate that there was originally 29 members (although this number and membership changed throughout the project).

3.27 Following its formation meetings were held on:

• 29 November 2019 — re-endorsing the terms of reference and updates on the project, preview of PEMS potential front-end interface, future consultations and planned user acceptance testing;
• 28 February 2020 — project update, PEMS previews (screen shots of potential design for human resources for example leave, payslips, work schedule);
• 30 July 2020 — user reference group told of new ‘go-live’ date of July 2021 for human resource management and expense management and an update on communication and education;
• 24 September 2020 — new format of an online drop-in session; and
• 29 April 2021 — first meeting with new membership, which included discussion on project status and timing and training plan, in particular for the July 2021 release of Milestone 7 phase 1.

3.28 The meetings were primarily informational updates to the reference group members on project status, given the delays in functionality throughout the project.

User acceptance testing

3.29 The user reference group’s terms of reference outlined that group members will have the opportunity to view and comment on system design, training and support materials and participate in system demonstrations aligned to the delivery of milestones. Finance advised the ANAO that parliamentarians and office staff were involved in one round of user acceptance testing between 16 and 29 January 2020 for leave requests/approval, detailed views of electoral staff allocation and payslips. Another round was scheduled for December 2021 for further testing of human resources and office expenses, however Finance advised no user reference group members agreed to participate.

3.30 IPEA and MaPS staff undertook user acceptance testing as business owners and users of the system instead, with parliamentarians or their staff not involved in testing. The test environment delivered for Milestone 7 phase 2 was missing a key feature of system integration capability which impeded the ability to perform full end-to-end user acceptance testing.

Communications and training planning

3.31 The PEMS project brief included organisational change management, communication management and stakeholder management as in-scope for the project. The initial project management plan identified that a change management strategy would be established as well as stakeholder and communication activities. Updated versions of the project management plan for Milestone 7 included a separate work stream for organisational change management.

3.32 Organisational change management planning from the beginning of the project until Milestone 7 was limited. It included a PEMS ‘launch strategy’ document, change impact assessment and one change project status report. There is no evidence that any practical organisational change management activities occurred during this time. Planning for Milestone 7 phase 1 included a communication plan, a training plan and an internal training plan for MaPS staff.

3.33 Detailed planning occurred for Milestone 7 phase 2. For the Milestone 7 phase 2 release, Finance developed communication and training planning documentation including:

• communication plan — covering communication objectives, audience, key messages, communication matrix, evaluation and risks;
• training plan — strategy for overall education and training which includes approach, evaluation, goals, risks and stakeholders approved by project board and noted by the steering committee in March 2022; and
• communication and training planner — operational document showing scheduled training and communications events around PEMS go-live.

3.34 Finance and IPEA have created educational and training products in different forms.

• Factsheets and demonstration videos — IPEA and Finance have produced several factsheets on the basic topics of PEMS including the creation of claims (travel and office), accessing reports, changes and reminders.
• Training sessions — Finance provided external training sessions to members of parliament and their staff surrounding the release of Milestone 7 phase 2, July 2022. There were twice daily sessions conducted on the basic functions of PEMS during July and August 2022. Finance did not provide evidence of how many individuals attended these training sessions. Finance positioned a team at Parliament House during sitting week for assistance on site. Individual training sessions were also provided if requested, with Finance conducting 62 appointments with 73 staff members.
• Websites — Finance and IPEA developed websites that has help-guides, factsheets and information about the system.

Digital Services Standards

3.35 The Digital Transformation Agency (DTA) has a set of best-practice principles for designing and delivering services that are user-friendly, inclusive, adaptive and measurable. The standards include 13 criteria and the DTA website notes that applying the criteria may be an ongoing process.⁴⁶ Finance confirmed PEMS was subject to the Digital Services Standards and compliance with the standards was part of the original high-level requirements that informed the business case. The commitment to the standards was also reflected in the project board and steering committee terms of reference.⁴⁷

3.36 Between January 2018 and January 2019, Finance reported to the DTA (as part of regular reporting, see paragraph 2.70) that user interfaces from 2018 would comply with the standards. The July 2023 report stated that the digital transformation benefits of the system would be measured against compliance with the standards. However, there was no discrete assessment of PEMS against the Digital Services Standards throughout the project.

PEMS adoption rates

3.37 As described in paragraph 3.6 and 3.7 office and travel claims can either be directly entered into PEMS by parliamentarians and their staff or submitted using paper forms. Based on data provided by Finance for 1 July 2022 to 31 August 2023, Figure 3.1 shows the percentage of claims directly entered by the end user since the release of the current solution in July 2022.

Figure 3.1: Proportion of claims lodged using PEMS direct entry — 1 July 2022 to 31 August 2023

Note: Data does not include if Finance or IPEA has entered the claim into PEMS on behalf of the parliamentarian or their staff or when Finance or IPEA pays the supplier directly.

Source: ANAO analysis of Finance PEMS data.

End user views on PEMS

ANAO survey of users

3.38 In September 2023, the ANAO sent a survey to 2117 parliamentarians and their staff on their experience using PEMS. 472 parliamentarians and staff responded. Most respondents primarily use PEMS for travel.⁴⁸ Key results from the survey are presented in Figure 3.2 to Figure 3.4.

Figure 3.2: End user overall satisfaction with PEMS

Note: One per cent of survey respondents replied ‘don’t know’ to overall satisfaction with PEMS.

Source: ANAO survey of parliamentarians and staff.

Figure 3.3: Satisfaction of users with key components of PEMS

Note: The results in Figure 3.3 do not include where the respondent answered ‘don’t know’, as the rate was minimal.

Source: ANAO survey of parliamentarians and staff.

Figure 3.4: End user views on PEMS usability

Source: ANAO survey of parliamentarians and staff.

IPEA customer satisfaction survey

3.39 IPEA conducts annual customer satisfaction surveys with parliamentarians and their staff covering their key service areas. The 2022 survey was conducted between 8 November and 12 December 2022, five months after the release of office and travel expense management. While the survey covers only the travel expenses component of PEMS, the results provide insight into user views on PEMS functionality as outlined in Table 3.6.

Table 3.6: Summary of PEMS satisfaction scores for travel expenses in IPEA customer satisfaction survey

Note: 132 users completed the 2022 survey. The research reports notes contextual factors that could have impacted the results including change of government and new users, the introduction of PEMS and the delay in expenditure reporting functionality.

98 users completed the survey in 2021. For the 2021 survey parliamentarians and their staff were using online forms for some travel claims with the remainder of claim types supported by manual forms and processing conducted by IPEA in the Entitlements Management System.

Source: IPEA, Customer Satisfaction Survey 2022 Research Report [Internet], IPEA available from https://www.ipea.gov.au/sites/default/files/2023-05/IPEA_Customer%20Satisfaction%20Survey%202022.pdf [accessed 3 September 2023].

Helpdesk calls

3.40 The PEMS helpdesk, run by Finance, received 3586 call in 2022–23, with 96 per cent categorised by Finance as ‘Severity 4 (Low)’.⁴⁹ In the previous year, 2021–22 (prior to Milestone 7 phase 1 release for office and travel expenses), there were 1647 helpdesk calls, 97 per cent of which were also categorised as ‘Severity 4 (Low).’ In December 2022 Finance analysed the help desk calls it was receiving and identified the most common queries as: the claim approval process for parliamentarians and different workflows for different claim types; issues with the workflow for when claims are returned from MaPS to the parliamentarian’s office for updating; and users being able to find claims in different statuses.

Recent efforts to engage end users

3.41 Since October 2022, Finance has engaged with parliamentarians and their staff, for example:

• re-establishing the user reference group — terms of reference were approved by the user reference group in May 2023 at the first meeting.⁵⁰ This meeting also covered updates to the project and upcoming user research work. Meetings were held in June 2023 and in September 2023 to demonstrate expenditure reporting functionality. The user reference group were also involved in user acceptance testing for Milestone 8 in October 2023;
• conducting user research (workshops and individual feedback session) to better understand PEMS usability issues. Key themes included opportunities to improve process flow, error messages, approach to requesting different access types, mobile experience, understanding where claims are up to and level of information in PEMS (with user having to maintain ‘shadow spreadsheets’). Finance advised it will add these issues to the backlog and consider as part of the prioritisation process; and
• developing a revised approach to training, awareness and communications with the plan to use a mixture of methods such as face-to-face sessions and ‘train the trainer’ model, updates to web content and development of a new communications plan.

Appendix 1 Entity responses

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• Updates to governance arrangements (see paragraph 2.7).
• A set of agreed user requirements and improved change management process for Milestone 8 expenditure reporting (see paragraph 3.22).
• Finance conducted an informal assessment of PEMS in October 2023 on the aspects of the project that is considers have met the standards (see paragraph 3.36).
• Efforts to engage parliamentarians and their staff on PEMS (see paragraph 3.41).

Appendix 3 Project delays by milestone

Note a: Milestones 3, 4 and 5 from the original project documentation were internal milestones that went towards the delivery of ERP operational. Their deliverables and scope were moved to milestone 6 (ERP Operational) and the mini releases for Milestone 2 were called Milestone 2–5 in future documentation.

Note b: The steering committee agreed in April 2018 to move two of the forms to the first release of Milestone 2 (August 2018).

Note c: Previous systems (Entitlements Management System and Chris 21) interfaces were updated to allow for continued use.

Note d: Scope and user requirements for these milestones were not fully delivered during the project. See paragraphs 3.8 to 3.16.

Note e: In February 2019, the steering committee agreed to split the scope of Milestone 6 so that the financial related functions were implemented in July 2019 and the remaining deliverables across expense management, including single view profile, and reporting were moved into Milestone 7. A new Milestone 8 was then created in November 2022 for expenditure reporting.

Source: ANAO analysis of scope and deliverables timeline.