Introduction

1. The provision of legal assistance to disadvantaged persons is recognised as a key element of equitable and accessible justice systems. In general, government-funded legal assistance arrangements aim to provide people with better and early access to information and services that can help them prevent and resolve disputes, and receive appropriate advice and assistance, no matter how they enter the justice system. This support may occur through legal assistance services available for all eligible members of the community or through services specifically directed at particular matters or groups. The Australian Government funds legal assistance services including state and territory run legal aid commissions; community legal centres; family violence prevention legal services and Aboriginal and Torres Strait Islander legal services.

2. In the Australian context, Aboriginal and Torres Strait Islander (Indigenous) people are generally over-represented in the criminal justice system, and adverse contact with the justice system is recognised as contributing more broadly to disadvantage. The Australian Government has funded Indigenous-specific legal assistance services since 1971. Providing Indigenous people with access to justice that is effective, inclusive, responsive, equitable and efficient remains a priority within the Australian Government’s approach to improving the ability of the justice system to meet the needs of Indigenous people.¹

3. Relative to other Australians, Indigenous people are considered to face particular barriers when accessing law and justice services including: financial capacity; language barriers; and mainstream services being less culturally sensitive or not delivering services to more remote parts of Australia. Other issues such as anxiety, lack of familiarity, fear of detention, and reluctance to use available services are seen as further contributors to Indigenous people not fully accessing mainstream legal services. In this setting, a cycle of disadvantage can arise as barriers to justice lead to poorer outcomes and high imprisonment rates, which in turn negatively affects wellbeing, opportunity and community safety—potentially resulting in further engagement with the justice system.

4. State and territory governments are responsible for the operation, and funding, of the law and justice sector in their jurisdictions. Within this arrangement, the Australian Government administers the Indigenous Legal Assistance Programme (ILAP). Under the program, legal assistance services are provided by Aboriginal and Torres Strait Islander legal services (referred to in this report as ILAP service providers) specifically for Indigenous people—mostly with respect to criminal law but also servicing family and civil law. ILAP service providers are Indigenous-controlled, community based, not-for-profit organisations responsible for the delivery of legal assistance services. Since 1971 the numbers of providers and their funding arrangements have varied. Under the funding arrangements in place since 2011, services have been delivered by eight providers which, with some exceptions, are located in each state and territory.² Each provider maintains local service outlets in urban, regional and remote areas and as at December 2014 there were 80 outlets nationally.

5. The Attorney-General’s Department (AGD) has been responsible for administering ILAP since 2004. Reflecting the overall policy intent to facilitate access to justice services, the objective of ILAP is to:

deliver culturally sensitive, responsive, respectful, accessible, equitable and effective legal assistance and related services to Indigenous Australians so that they can fully exercise their legal rights as Australian citizens…³

6. Ongoing concerns about the high rates of Indigenous imprisonment and deaths in custody have resulted in legal assistance services being mainly focussed on criminal law matters and on supporting Indigenous people at risk of being detained or imprisoned.⁴ Although ILAP service providers are funded by the Australian Government, the matters they deal with generally arise from the laws of the states and territories, and providers therefore work through the state and territory justice systems.

7. How an Indigenous person accesses legal services provided through ILAP will depend on an individual’s legal needs and circumstances. An individual’s legal need may involve a civil, criminal and/or family law matter and require legal assistance ranging from legal information and initial advice, one-off representation at a court or tribunal, known as duty lawyer assistance, or ongoing legal assistance and representation in the form of casework services. The approach to service delivery under ILAP is that when an Indigenous person has a legal problem they can seek to access legal services directly at an ILAP service provider office or outlet. In circumstances where an Indigenous person has been arrested or is being detained, providers maintain after hours telephone numbers that Indigenous people can contact. Access to duty lawyer services is also provided at some courts, including some remote area courts, where duty lawyers are available to assist Indigenous people that have to appear before court on a particular day or have been detained for criminal offences.

8. In 2013–14, AGD distributed $74.9 million amongst the eight ILAP service providers—each of which has a grant funding agreement with AGD until July 2015. Funding levels for each provider are determined by a funding allocation model, which considers a range of demographic and social risk factors in estimating the likely level of need for legal assistance services in the areas covered by each provider. During 2013–14, AGD reported that service providers gave 90 103 advices, undertook 29 436 duty lawyer services and conducted 86 949 cases. Australian Government funding arrangements for Aboriginal and Torres Strait Islander legal assistance providers along with legal aid commissions, community legal centres and family violence prevention legal services expire on 30 June 2015. From July 2015, a new national strategic framework for all Australian Government funded legal assistance services is expected to be in place and will provide high level policy direction and set out shared national objectives and outcomes. Once agreed by the Commonwealth, state and territory ministers, the framework is expected to inform future funding agreements with service providers, including providers of Indigenous legal assistance.

Audit objectives and criteria

9. The objective of the audit was to assess the effectiveness of the Attorney-General’s Department’s administration of the Indigenous Legal Assistance Programme. To conclude on this objective the ANAO adopted high level criteria relating to the effectiveness of program management arrangements, AGD’s management of funding agreements, performance monitoring and reporting arrangements.

Overall conclusion

10. Access to appropriate legal assistance services is an important element of a fair and equitable legal system. Since 2011, funding through the Indigenous Legal Assistance Programme (ILAP) has enabled service providers to deliver a cumulative total of 604 519 legal assistance services, or an average of 201 506 per year. Over the same period, ILAP funding has enabled these service providers to operate an average of 84 outlets each year to provide services. Research undertaken by a range of government and non-government bodies indicates generally that the level of unmet demand for Indigenous legal assistance services is higher than the supply of those same services. Additionally, the Productivity Commission has reported in the 2014 Overcoming Indigenous Disadvantage report that between 2000 and 2013, Indigenous imprisonment rates have continued to worsen with the imprisonment rate for Indigenous adults increasing by 57.4 per cent.⁵ There have also been significant increases in juvenile detention rates since 2001. While these trends in themselves are not necessarily reflective of levels of access and the quality of services provided through ILAP, they do indicate that demand for Indigenous legal assistance services is high and is likely to remain so.

11. Facilitating appropriate access to justice is complex and involves a number of different institutions. For the most part, Australians interact with the justice system at the state and territory level, and accessibility is largely determined by how that justice system functions. As a result, while Australian Government support through ILAP is able to address some barriers to access, it is not able to address all factors relevant to improving access to justice for Indigenous people. Further, the demand for services arises largely from the operation of state and territory laws. In this respect, demand for Indigenous legal assistance services is not in the control of the Australian Government and can be affected significantly by changes made to state and territory laws.

12. In this context, and considering the small size of the program, the overall management approach taken by the Attorney-General’s Department (AGD) is reasonable. The approach recognises that there are institutional and jurisdictional differences in justice systems across Australia, and acknowledges that ILAP service providers are better placed than AGD to identify appropriate service approaches in the areas they service. Accordingly, AGD allows service providers flexibility in their planning and delivery approach. In circumstances where national programs allow for local flexibility, program management arrangements need to be designed to achieve appropriate levels of consistency. Performance information arrangements also need to be well developed to enable appropriate comparative analysis and assessment of program performance overall.

13. AGD has put in place a range of approaches to promote consistent national management of ILAP. These include a formula based funding allocation model that incorporates information specific to each jurisdiction, but assesses each jurisdiction against the same criteria and weightings to determine the funding levels to be allocated to each service provider. Expectations in relation to service delivery are promoted by the Service Delivery Directions, made available to service providers to inform the annual development of service plans, and the development of the Indigenous Quality Practice Portal designed to support the collection and analysis of relevant performance information and monitor delivery against Service Standards. Grant funding is managed through standard funding agreements that are in place with each service provider.

14. AGD’s management of ILAP has matured since it assumed responsibility for the program in 2004, and while the current management framework is reasonable overall, improvements can be made in the following areas. Firstly, to give better effect to the intent of the program to prioritise assistance to communities with the highest need, the funding allocation model, which is currently being revised by AGD, could be enhanced by the inclusion of additional social and economic indicators of disadvantage to better target available resources. Secondly, there would be benefit in developing greater consistency in relation to performance expectations. Currently, each ILAP service provider proposes targets for the number of services they expect to provide in their annual service plans, which are endorsed by AGD as part of the annual planning process. However, the definition of a service can vary between jurisdiction and AGD has not developed a benchmark level of service against which an assessment of proposed targets can be made. Further, service plans are not integrated into funding agreements, and as a result, there is no clear link between the funding amounts provided in the agreements and the expected level of performance.

15. Generally, ILAP funding agreements are compliance focussed and show only a limited performance orientation. While the reporting requirements established in the funding agreements give AGD sufficient visibility to identify issues of compliance, AGD has not always been timely in verifying or addressing matters relating to service provider compliance. Further, the focus of AGD’s program measurement and reporting is mainly on the levels of funding expended and the number of legal services delivered. This information is relevant in view of the demand-driven nature of ILAP. However, in the absence of targets or baseline information in relation to access, AGD is unable to assess whether access to justice has improved as a result of ILAP funding. Improvements to the collection, reporting and use of currently available performance information—directed towards measuring and reporting on the quality of services delivered and supported by appropriate assurance and verification mechanisms—would better enable AGD to strengthen its focus on whether the program is performing in line with expectations, and that barriers to access are being appropriately addressed.

16. The ANAO has made one recommendation aimed at improving AGD’s approach to setting and assessing performance targets.

Key findings by chapter

Program Management Arrangements (Chapter 2)

17. Program objectives for ILAP have been clearly stated in the program guidelines and focus on the provision of legal assistance services that are culturally sensitive, responsive, respectful, accessible, equitable and effective. The program’s funding strategy of using Indigenous organisations to provide these services using local outlets, is reasonably aligned with the program’s objectives, and the service usage patterns are generally consistent with the overall distribution of the Indigenous population.

18. Allocation of funding between jurisdictions is based on the Funding Allocation Model (FAM). The FAM uses statistical information from census and other sources, and funding is allocated according to weighted factors which address the cost of service delivery, number of and demand for services, and risk of detention or imprisonment. Although a stated priority of ILAP is to deliver services to those communities with the highest need, communities have not been defined beyond the jurisdictional boundaries of service providers, and funding allocations are determined at a broader level—derived from assessing a combination of population, risk and cost factors. To date, AGD has not undertaken any significant work to determine which areas of the Indigenous community are considered to be ‘high need’, or which communities are experiencing access barriers, and the FAM currently does not include factors or indicators relating to relative disadvantage between areas.

19. Service providers are given the flexibility to prioritise where they deliver services, within the available funding. ANAO analysis of case work matters indicates that most matters are undertaken in regional areas (41.5 per cent) followed by metropolitan areas (29.6 per cent) and remote areas (28.9 per cent). Proportionately, however, the share of services relative to population is lower for urban areas with 71.1 per cent of matters delivered in metropolitan and regional areas, where 78.6 per cent of Indigenous people are located. Conversely, some 28.9 per cent of services are delivered to remote areas where 21.3 per cent of the Indigenous population is located. At a broad level, this data indicates a slight over-servicing of remote populations. However, it is likely that remote services are concentrated in areas that are relatively more accessible, which masks a more limited delivery of services in other more remote areas.

Managing Service Delivery (Chapter 3)

20. To support the management of the program, AGD has developed a grants administration framework that comprises program guidelines, funding agreements, supporting guidance and a set of reporting requirements. These arrangements support AGD in monitoring compliance with the terms of the funding agreements, but overall do not support a strong focus on performance against the objectives of the program. Key performance requirements are described at a broad level but these are not set out in the actual agreements and are managed separately through the annual service plan submitted by service providers. While in place, the comprehensiveness and quality of these plans varied across different service providers. Additionally, there are some variations in the definition of a ‘service’ across some providers which makes it difficult for AGD to assess the efficiency of service delivery consistently. The approach to annual target setting is largely based on assessing levels achieved in prior years. AGD would be better positioned when endorsing annual targets if it developed benchmark levels of performance against which proposed service targets could be assessed.

21. Monitoring arrangements rest on the provision by service providers of a range of reports covering service plans, service quality and financial performance. In an effort to simplify reporting requirements, AGD developed the Indigenous Justice Quality Practice Portal, an online portal through which service providers submit their reports. This is a positive initiative, however, while the system provides AGD with a high level view of compliance it gives generally limited visibility over the details of actual performance against service standards. Overall, AGD’s arrangements for managing the funding agreements are reasonable in their design, but would be strengthened by introducing a greater focus on expected results in the funding agreements, and periodic assurance by AGD of relevant performance data submitted by providers to inform assessments of improvements in access, the quality of services provided and the performance of service provider organisations.

Program Performance Monitoring and Reporting (Chapter 4)

22. Program performance information collected by AGD is currently focused on the quantity and type of services delivered, and whether providers are complying with their funding agreements. These reporting arrangements are useful in respect to ascertaining levels of activity, location of services and types of legal assistance matters addressed. However, they are not sufficiently comprehensive to provide AGD with an appropriate level of visibility over other more qualitative aspects of program performance, including whether services are of high quality, appropriate, accessible and equitable.

23. While it is reasonable to expect that the provision of legal assistance services through ILAP would contribute to improved access to justice outcomes for Indigenous people, no baseline information on existing levels of access, or desired levels of access, has been collected by AGD. As a result the performance reporting framework is not able to generate robust and meaningful program performance information that gauges whether access is improving, and the contribution of ILAP service providers to this result.

Summary of entity response

24. AGD’s summary response to the proposed report is provided below, while the full response is provided at Appendix 1.

The department welcomes the performance audit of the Indigenous Legal Assistance Programme, and largely agrees with the findings and recommendations of the report.

The report presents an accurate view of the unique challenges faced by the Attorney-General’s Department in administering the Indigenous Legal Assistance Programme and ensuring services are delivered in culturally sensitive and accessible manner, so that Indigenous Australians can fully exercise their legal rights as Australian citizens.

The department is committed to working with Indigenous legal assistance providers, stakeholders and community to support the delivery of quality legal assistance services and achieving the outcomes of the programme.

The findings in this report will assist in the future effective delivery of the Indigenous Legal Assistance Programme.

Recommendations

Background

1.1 Adverse contact with the justice system is recognised as having a strong, perpetuating influence on Indigenous disadvantage. The Aboriginal and Torres Strait Islander Social Justice Commissioner noted in 2013:

While we have neglected targets for justice issues, there has been a danger that this has undermined our efforts to meet the other targets set in the Closing the Gap agenda. This is because we know that imprisonment has such a profoundly destructive impact, not only on individuals, but on the entire community. It affects areas such as health, housing, education and employment—all the building blocks of creating stable and productive lives.⁶

1.2 In 1991 the Royal Commission into Aboriginal Deaths in Custody (RCIADIC) found there were a disproportionate number of Aboriginal and Torres Strait Islander (Indigenous) people, compared with non-Indigenous people, in both police and prison custody.⁷ Despite governments around Australia making commitments in response to RCIADIC recommendations, the proportion of Indigenous people in prison has increased steadily since 1991.⁸

1.3 The Productivity Commission has reported in the 2014 Overcoming Indigenous Disadvantage report that between 2000 and 2013, Indigenous imprisonment rates continued to worsen with the imprisonment rate for Indigenous adults increasing by 57.4 per cent.⁹ In 2013 the Australian Bureau of Statistics (ABS) reported that Indigenous prisoners comprised just over a quarter (27 per cent) of the total prisoner population. The trend increase in Australia’s per capita Indigenous adult prisoner population for 2003–2013 is shown in Figure 1.1.

Figure 1.1: Rates of adult imprisonment—Australia

Source: ANAO analysis of ABS, 4517.0 Prisoners in Australia—2013, Indigenous Status of Prisoners, Canberra, 5 March 2014 update, available from: <http://www.abs.gov.au/AUSSTATS/abs@.nsf/ Lookup/4517.0Main+Features12013?OpenDocument> [accessed 12 March 2014].

1.4 High rates of Indigenous imprisonment are accompanied by disproportionately high levels of expenditure on Indigenous people in the justice system. In 2012–13, total direct Indigenous-specific expenditure on building safe and supportive communities by all Australian governments was $8 billion.¹⁰ This expenditure is incurred primarily by state and territory governments in the operation and administration of their justice systems, with most expenditure relating to policing and corrective services. By comparison, the Australian Government’s component of overall expenditure is smaller and focuses on contributing to the provision of legal assistance services.¹¹

Legal assistance arrangements in Australia

1.5 Effective legal assistance arrangements are acknowledged to provide benefits for the overall justice system. A 2009 study of the economic value of legal assistance observed that:

There is a direct relationship between the efficiency of the court and the provision of legal aid. Efficiency is achieved through the provision of information, advice, legal assistance, dispute resolution, and representation for matters that would otherwise be self-representing. Costs to the justice system are also avoided because cases are diverted from court rather than needing a hearing or decision by the court.¹²

1.6 The Australian Government funds various legal assistance services using different funding arrangements to assist Australians whose relative disadvantage would otherwise prevent access to justice to equitable treatment before the law.¹³ Services funded by the Australian Government are: legal aid commissions in each state and territory; community legal centres, family violence prevention legal services and Aboriginal and Torres Strait Islander legal services. A wide range of activities have been undertaken across these legal assistance services, including: information and referral; discrete task assistance; dispute resolution; legal representation; community legal education; and policy and law reform.

National Partnership Agreement on Legal Assistance Services

1.7 Broad direction for the provision of legal assistance services is provided by the National Partnership Agreement on Legal Assistance Services (NPA) which was signed by the Australian Government and state and territory governments in June 2010. The objective of the NPA is to provide ‘a national system of legal assistance that is integrated, efficient and cost-effective, and focused on providing services to disadvantaged Australians in accordance with access to justice principles of accessibility, appropriateness, equity, efficiency and effectiveness’.¹⁴

1.8 The NPA promotes a collaborative and ‘holistic’ approach to providing services for disadvantaged Australians.¹⁵ These services consider characteristics and needs of client groups and how they will access services. Familiarity and trust, flexibility, timeliness, confidentiality and consistency have been identified as key components of effective service delivery for individuals with complex legal needs. While the NPA provides a policy framework for all types of legal assistance services, it specifically only funds state and territories for the provision of legal aid commissions. Separate funding arrangements are in place for community legal centres, family violence prevention legal services and Aboriginal and Torres Strait Islander legal services. From July 2015, a new national framework for all Australian Government funded legal assistance services is expected to be in place.

Aboriginal and Torres Strait Islander legal services

1.9 Indigenous Australians experience a range of barriers to accessing the justice system. In many cases, legal assistance services provided by the private sector are outside the financial capacity of many Indigenous Australians, may not be delivered in a culturally sensitive way or may not be delivered in regional and remote areas. Issues such as anxiety, lack of familiarity, fear of detention, and reluctance to use mainstream legal assistance services are also considered to affect access to justice for Indigenous Australians.

1.10 The Australian Government has funded legal assistance services for Indigenous Australians since 1971. Service providers are Indigenous-controlled, community based, not-for-profit organisations. Currently, there are eight service providers that generally operate based on state jurisdictions, except in the Northern Territory where two service providers operate, and the Australian Capital Territory which is serviced by the New South Wales service provider. As at December 2014 there were 80 service provider outlets in operation nationally. The majority of service outlets (88 per cent in 2012–13) were located in regional and remote areas. In order to provide legal assistance in areas where outlets are not easily accessible, most providers have adopted an outreach service delivery model. This enables some services to be provided at circuit courts and ‘bush courts’.¹⁶

Indigenous Legal Assistance Programme

1.11 Service providers receive funding through the Indigenous Legal Assistance Programme¹⁷ (ILAP) which is administered by the Attorney-General’s Department (AGD). The objective of the program is to:

deliver culturally sensitive, responsive, respectful, accessible, equitable and effective legal assistance and related services to Indigenous Australians so that they can fully exercise their legal rights as Australian citizens …¹⁸

1.12 ILAP has been administered by AGD since 2004. Prior to this, the program had been administered by the Department of Aboriginal Affairs, Aboriginal and Torres Strait Islander Commission and Aboriginal and Torres Strait Islander Services respectively, using grant funding arrangements.

1.13 In 2004, AGD conducted a competitive tender process for the continued provision of legal assistance services. The intention of this tender process was to improve the quality and efficiency of service delivery by ILAP service providers.¹⁹ As a result of the tender, 25 organisations, each of which had an individual grant agreement with AGD, were replaced by nine state or zone-wide service provider organisations operating under contracts with AGD. Another tender process in 2007–08 resulted in the further rationalisation of providers, from nine to eight, nationally. In 2011 the Government reverted to funding service providers through grant funding agreements but maintained the overall number of providers at eight. These new funding agreements were initially scheduled to run from 1 July 2011 until 30 June 2014 but were extended in 2013 to 30 June 2015.

1.14 As providers of legal services, service providers must comply with various rules and regulations governing the legal profession and practice in each state and territory. However, one service provider is not a registered legal service provider, being responsible for the delivery of other Indigenous programs in addition to ILAP. Under the relevant state’s legislation the organisation is unable to deliver legal services to third parties, and therefore outsources its legal services.

1.15 In 2013–14, approximately 84 per cent of all legal assistance matters addressed through ILAP related to (mostly state and territory based) criminal law matters. Further, in the same year, over 90 per cent of casework and duty matters were in criminal law.²⁰ Although service providers have sought to broaden the focus of the law types they service, the primary focus of the program remains on those Indigenous people at risk of being detained in custody, reflecting concerns regarding the high Indigenous imprisonment rates.

Structure of ILAP

1.16 Between 2011 and 2013, ILAP consisted of four funding sub-programs:

• Indigenous Legal Assistance—the core activity of providing legal services to Indigenous people;
• Indigenous Test Cases (discontinued in 2012);
• Law Reform, Research and Policy Development; and
• Program Support and Development.

1.17 In late 2013, the Government announced a reduction of $43.1 million over four years across all Indigenous and non-Indigenous legal assistance programs, refocussing program activity away from policy reform and advocacy activities towards ‘front line’ legal services. Of the $43.1 million reduction in funding, $13.4 million of funding was removed from ILAP policy reform and advocacy activities over four years (2014–17). AGD advised the ANAO that while the Australian Government recognises service providers may choose to undertake such activities on their own account, these activities will not be funded by the Australian Government.

1.18 How an Indigenous person accesses legal services provided through ILAP will depend on an individual’s legal needs and circumstances. An individual’s legal need may involve a civil, criminal and/or family law matter and require legal assistance ranging from legal information and initial advice, one-off representation at a court or tribunal, known as duty lawyer assistance, or ongoing legal assistance and representation in the form of casework services. The approach to service delivery under ILAP is that when an Indigenous person has a legal problem they can seek to access legal services directly at a provider’s office or outlet. In circumstances where an Indigenous person has been arrested or is being detained, providers maintain an after hours telephone numbers that Indigenous people are able to contact. Access to duty lawyer services is provided at some courts, including some remote area courts, where duty lawyers are available to assist Indigenous people that have to appear before court on a particular day or have been detained for criminal offences.

1.19 In 2013–14, AGD distributed $74.9 million amongst the eight service providers—each of which has a grant funding agreement with AGD until July 2015. Funding levels for each provider are determined by a FAM which considers a range of demographic and social risk factors in estimating the likely level of need for legal assistance services in the areas covered by each ATSILS. During 2013–14, AGD reported that ILAP providers gave 90 103 advices, provided 29 436 duty lawyer services and conducted 86 949 cases.²¹ Australian Government funding arrangements for Aboriginal and Torres Strait Islander legal assistance providers along with legal aid commissions, community legal centres and family violence prevention legal services expire on 30 June 2015. From July 2015, a new national strategic framework for all Australian Government funded legal assistance services is expected to be in place and will provide high level policy direction and set out shared national objectives and outcomes. Once agreed by the Commonwealth, state and territory ministers, the framework is expected to inform future funding agreements with service providers, including providers of Indigenous legal assistance.

Audit objective

1.20 The objective of the audit was to assess the effectiveness of the Attorney-General’s Department’s administration of the Indigenous Legal Assistance Programme.

Scope

1.21 This audit examined the AGD’s management of ILAP under current (2011–2015) funding arrangements. In some areas, to provide context and establish trends, the audit considered AGD’s management over a longer period. The audit did not include examination of the governance or performance of individual service providers.

Criteria

1.22 To form a conclusion against this audit objective, the ANAO adopted the following high level criteria:

• program management arrangements support the achievement of program objectives;
• funding arrangements support the achievement of program objectives; and
• performance monitoring and reporting arrangements support program performance.

Audit approach

1.23 In undertaking the audit, the ANAO:

• examined program-related information, including: key program management documentation, such as program guidelines and standard operating procedures; transactional and compliance documentation; and performance monitoring and reporting documentation;
• analysed available performance data on service delivery;
• interviewed relevant AGD program officers and managers; and
• conducted site visits and interviews with managers and staff from ILAP service providers, their peak representative body, and visited legal offices from which services are delivered.

1.24 The audit was conducted in accordance with the ANAO Auditing Standards at a cost of $468,405.

Report structure

Table 1.1: Structure of report

Introduction

2.1 Sound program management arrangements underpin the effective delivery of programs. Such arrangements include developing and communicating clear objectives for a program, aligning funded activities and allocations to program objectives and managing risks to the achievement of program objectives. The Attorney-General’s Department (AGD) has been administering the Indigenous Legal Assistance Programme (ILAP)²² since 2004. Over this time, AGD has established supporting frameworks to implement and deliver the program. In this context the ANAO examined:

• ILAP’s program objectives;
• program funding allocations and distribution; and
• ILAP risk management.

ILAP program objectives

2.2 Clear program objectives are important to enable prioritisation of funded activities, development of consistent and targeted performance measurement arrangements and support clear reporting to Parliament and other stakeholders on the outcomes expected from funding. Reflecting the provision of legal assistance services as an ongoing, demand driven service, the objectives of ILAP are described in the ILAP program guidelines as being to: ‘…deliver culturally sensitive, responsive, respectful, accessible, equitable and effective legal assistance and related services to Indigenous Australians so that they can fully exercise their legal rights as Australian citizens.’²³ The guidelines further state that priority will be given to investing in Indigenous communities with the highest need.

2.3 The objective of ILAP is framed within the principles of the Australian Government’s strategic access to justice framework and also the National Partnership Agreement on Legal Assistance Services (NPA). The access to justice principles are: accessibility, appropriateness, equity, efficiency and effectiveness. Under the access to justice framework, Indigenous people are generally recognised as a disadvantaged group requiring specifically targeted or specialised services.²⁴ As a result, cultural appropriateness of services is a relevant consideration for accessibility, along with physical distribution of service outlets in relation to the target population.

2.4 The objective of the NPA is to achieve a ‘national system of legal assistance that is integrated, efficient and cost-effective, and focused on providing services for disadvantaged Australians in accordance with the access to justice principles of accessibility, appropriateness, equity, efficiency and effectiveness’.²⁵ While ILAP is not directly funded through the NPA, it is expected to contribute, along with three other types of legal assistance services,²⁶ to the NPA’s objective. The NPA does not include specific performance requirements or deliverables for ILAP, but at a broad level there is a close alignment between the objectives of ILAP and the NPA, and the effective operation of ILAP could be reasonably expected to contribute to the broader national objectives. The NPA expires on 1 July 2015 (having been extended by one year from the original expiry). As noted in paragraph 1.19, from July 2015, a new national strategic framework for all Australian Government funded legal assistance services is expected to be in place which will provide shared national objectives and outcomes. AGD advised that the objectives of future ILAP funding agreements will reflect the intention of the national strategic framework, as agreed between the Australian Government, states and territories.

2.5 ILAP’s objective clearly indicates its nature as an ongoing service and the nature of its target group. However, the key concepts of accessibility, equity and effectiveness have not been defined by AGD at an operational level in either the providers’ funding agreements or the program guidelines. The high level principles act as a framework within which service providers operate, but do not serve to guide the priorities of service providers or to prioritise the collection of data to assess and report on the performance of providers individually or ILAP as a program.

2.6 The communication to the Parliament of priorities and the results expected from public funding is an important consideration for Australian Government entities and occurs through the publication of Portfolio Budget Statements (PBS).²⁷ In the Attorney-General’s PBS, ILAP is part of the Indigenous Law and Justice (ILJ) program. Until September 2013, AGD administered several activities under the ILJ program, however, these other programs were subsequently transferred to the Department of Prime Minister and Cabinet (PM&C). Accordingly, in 2014–15 ILAP comprised 99 per cent of the funding for the ILJ program.²⁸

2.7 The ILJ program is one of eight programs intended to contribute to the broader outcome of a ‘just and secure society through the maintenance and improvement of Australia’s law and justice framework and its national security and emergency management system’.²⁹ AGD has adopted generic objectives for all eight programs with the intention that their objectives will contribute to:

• protecting and promoting the rule of law; and
• building a safe, secure and resilient Australia.³⁰

2.8 This generic approach provides limited insight into the specific results the Australian Government is seeking to achieve through the ILJ program, although the provision of services to support access to justice is identified as the main deliverable for the ILJ program. The key performance indicators (KPI) established for the ILJ to assess performance of the program are ‘improved access to justice for Indigenous people’ and ‘effective administration of the access to justice programmes for Indigenous people’.³¹

2.9 There are two aspects to the KPIs which could be improved. Firstly, no baseline information has been established to assess whether any improvements have occurred (discussed further in chapter 4). Secondly, the qualitative indicator does not fully align with the main activity funded through the program. While ILAP contributes to the provision of access to justice, its objectives reflect that it focuses on the ongoing provision of a demand-driven service to eligible clients, rather than on change-focussed interventions designed to improve access.

2.10 AGD advised that the indicators were not amended when other ILJ programs were transferred to PM&C in 2013 and that those other programs were designed to improve access. AGD further advised that, in the future, ILAP will be funded under the Justice Services Program, which will more closely reflect the objectives of ILAP.³²

Program funding allocations and distribution

2.11 As noted in paragraph 1.10, there are eight ILAP service providers across Australia generally covering each state jurisdiction, although there are two providers serving the Northern Territory and a single provider operating across New South Wales and the Australian Capital Territory (ACT). As at December 2014, there were 80 service provider outlets located in metropolitan, regional, remote and very remote areas, supplemented by outreach services. In 2012–13 88 per cent of outlets were located in regional or remote areas.

2.12 The funding agreements between AGD and each provider do not include details on the numbers of outlets or their locations. While AGD is kept informed of office locations and resource allocations, AGD does not play an ongoing role in assessing demand within or between jurisdictions, or advising where offices should be located. Service providers are considered to be best placed to determine the locations and numbers of outlets and although AGD endorses these as proposed by the provider, no adjustments to funding are made when a provider chooses to close or move an outlet. Due to difficulties in attracting staff and increasing operating costs, some service providers have been forced to close some outlets. The ANAO was advised that during 2011 the Western Australian provider closed three offices (reducing the number of offices from 18 to 15) while the Queensland service has since closed six offices (reducing the number of offices from 27 to 21).

2.13 The numbers of outlets maintained by providers as at September 2014 are shown in Table 2.1.

Table 2.1: Provider outlets by jurisdiction, as at September 2014

Source: Australian Bureau of Statistics, Estimates of Aboriginal and Torres Strait Islander Australians, June 2011; and ANAO analysis of provider information.

2.14 The trend in the number of outlets since 2008 is shown in Figure 2.1.

Figure 2.1: Number of provider outlets

Source: ANAO analysis

2.15 In late 2013, the Government announced a reduction of $43.1 million over four years across all Indigenous and non-Indigenous legal assistance programs. Of the $43.1 million reduction in funding, $13.4 million of funding was removed from ILAP policy reform and advocacy activities over four years. Even though these reductions were intended by the Australian Government to be confined to policy activities rather than front line services, during interviews with the ANAO, service providers felt that the planned $13.4 million funding reduction would significantly affect their ability to deliver services and force further office closures.

Assessing and servicing communities with greatest need

2.16 The ILAP program guidelines note that a priority for ILAP is to ‘invest in an efficient, effective and ethical manner in Australian Indigenous communities with the highest need.’³³ Although this is a stated priority, it has not been given practical effect by AGD, which has adopted only a broad definition of Indigenous community and equates the community to the (mainly) jurisdictional boundaries of service provider areas.

2.17 Legal assistance services, by their nature, are targeted at disadvantaged clients and Indigenous people have been identified generally as a disadvantaged group.³⁴ Studies undertaken on Indigenous offending have found that a relationship exists between socioeconomic disadvantage and contact with the justice system.³⁵ However, not all communities are the same and significant differences in conditions can be experienced, for example, between urban areas and very remote areas. Until June 2014, when the Review of the National Partnership Agreement (NPA) on Legal Assistance Services report was released, no analysis or assessment had been undertaken by AGD to identify specific Indigenous communities of highest need. Further, AGD advised the ANAO that the department does not undertake work to determine areas of highest need as it considers that service providers are best placed to determine this.

2.18 The ANAO analysed data relating to relative disadvantage collected by the Australian Bureau of Statistics (ABS) in the 2011 census to assess the alignment between disadvantaged areas and the locations of service provider outlets. Disadvantaged communities are classified by the ABS based on the relevant Local Government Area (LGA). The most disadvantaged LGAs with a high proportion of Indigenous residence, (Indigenous population greater than 50 per cent), are presented in Table 2.2. A full list of the most disadvantaged LGAs by state and Indigenous population is provided in Appendix 2.

Table 2.2: Number and location of most disadvantaged LGAs

Source: Australian Bureau of Statistics, Estimates of Aboriginal and Torres Strait Islander Australians, June 2011, Canberra.

Note: The Australian Capital Territory, Victoria and Tasmania do not have any disadvantaged LGAs with an Indigenous population of greater than 50 per cent.

2.19 There are significant variations across Australia in the distribution of disadvantaged Indigenous populations. As demonstrated in Figure 2.2, the most disadvantaged LGAs are located predominantly in remote and very remote areas of Australia, mostly in the north and west of the continent. There are relatively few service provider outlets located within these areas. The Northern Territory has the greatest number of Indigenous residents living in highly disadvantaged areas with a total of 44 419 people.³⁶ This comprises over 64 per cent of the Northern Territory’s total Indigenous population yet is serviced by only five outlets. Highly disadvantaged areas with high Indigenous populations in South Australia have no outlets, relying entirely on outreach services. In contrast, the ACT, Victoria and Tasmania have much smaller Indigenous populations and do not have any LGAs where more than 50 per cent of the population is Indigenous. However, when considered against the general distribution of disadvantaged areas (and not considering the levels of Indigenous population), in most cases ILAP outlets are located in areas of each state that are relatively more disadvantaged than other areas.³⁷

Figure 2.2: Disadvantaged Local Government Areas where over 50 per cent of the population is Indigenous

Source: ANAO.

Note: Map not to scale.

2.20 As noted in paragraph 2.17, under the Australian Government’s National Indigenous Law and Justice framework, Indigenous people are generally considered a disadvantaged population in relation to legal services. On this basis, it would be expected that service delivery patterns generally follow population patterns. The number of services delivered by ILAP providers in metropolitan, regional and remote areas, compared to the proportion of Indigenous people living in those areas, is shown in Table 2.3.

Table 2.3: Indigenous population compared to the proportion of legal services delivered

Source: ANAO analysis.

Note: Due to the rounding of figures, totals may not add up to 100 per cent.

2.21 As outlined in Table 2.3, the proportion of services delivered is generally aligned with the Indigenous population located in each of the geographic areas. However, while the number of Indigenous people living in remote areas is only 21.3 per cent, almost 27 per cent of services are delivered to those people, indicating high demand and also that few other legal assistance service options are likely to exist to service remote Indigenous communities. By contrast, the share of services relative to population is lower for urban areas with 71.1 per cent of matters delivered in metropolitan and regional areas, where 78.6 per cent of Indigenous people are located. Service providers and stakeholders further informed ANAO that some very remote communities do not have any access to legal services. As a consequence, while Indigenous people residing in remote communities are still required to attend court for sentencing, many do not have access to legal representation which may result in higher imprisonment rates and associated adverse outcomes.

Types of legal services

2.22 In response to the 1991 Royal Commission into Aboriginal Deaths in Custody (RCIADIC), ILAP has over time tended to focus primarily on providing criminal law services to Indigenous people. For many reasons, including long-standing disadvantage and ongoing discrimination, Indigenous people experience much higher rates of adverse contact with the justice system and are imprisoned at significantly higher rates than other Australians. In 2013, the ABS reported that Indigenous prisoners comprised just over one quarter (27 per cent) of Australia’s total prisoner population. In this context, 80 per cent of all legal services delivered by ILAP providers relate to criminal law. The comparison between the amount of criminal case work and that undertaken in the family and civil law areas is shown in Figure 2.3.

Figure 2.3: ILAP 2013–14 case work services by law type—metropolitan, regional and remote areas (percentage)

Source: ANAO, based on AGD data.

Note: Due to the rounding of figures, totals may not add up to 100 per cent.

Availability of family and civil legal services

2.23 Although the focus of ILAP is on the provision of criminal legal services, the objective of the program: to ‘deliver legal assistance and related services to Indigenous Australians so that they can fully exercise their rights as Australian citizens’ has a broader focus than criminal law alone. Of the eight ILAP service providers interviewed by ANAO, seven advised that there was a significant unmet need for family and civil legal services. This observation is also supported by the views expressed by related industry stakeholders interviewed by the ANAO.³⁸ Service providers further advised that unmet demand is particularly acute in remote or very remote areas, with high Indigenous populations. While providers are aware of this unmet demand, they advised that they are unable to increase the family and civil services delivered because of funding constraints associated with ILAP’s focus on criminal law.

2.24 Service providers particularly expressed concern about the increasing incidence of children being removed from their homes while parents were, in many cases, unaware of their legal rights or unable to obtain legal assistance. One industry stakeholder described the lack of family legal services available to help Indigenous families and children in one jurisdiction as ‘dire and acute’. In Queensland, service provider satellite outlets³⁹, servicing many of the most disadvantaged LGAs, are resourced to provide criminal legal services only, which results in those communities having no immediate access to any family or civil legal services.

2.25 In 2011, the James Cook University commenced a national research study of the civil and family law needs of Indigenous people called the the Indigenous Legal Needs Project (the project). The project aims to identify and analyse the legal needs of Indigenous communities in non-criminal legal areas. As of June 2014, the project had delivered two reports: the NT Report (November 2012) and the Victorian Report (November 2013). Both reports identified significant gaps in Indigenous access to justice with respect to civil and family law. The reports specify housing and tenancy as the major areas of unmet demand with other priority areas being: child protection; discrimination; neighbourhood issues; social security; victim’s compensation; wills; credit and debt; and consumer issues.⁴⁰ Similar observations have also been made by the Productivity Commission, which found that unmet need remains widespread, particularly in more remote locations⁴¹ and the Review of the National Partnership Agreement on Legal Assistance Services which found that the amount of legal assistance available to those in remote communities is not adequate to meet residents’ needs.⁴²

Program funding allocations

2.26 In most grant programs service providers request an amount of funding required to deliver specified outcomes, which is then subject to negotiation with the funding department. This differs in ILAP, where funding is determined by the department using a Funding Allocation Model, which takes into account conditions particular to each service area.

Table 2.4: Funding allocations to service providers, 2012–15

Source: ANAO analysis of AGD information. Figures are rounded.

Funding Allocation Model

2.27 In order to allocate funds consistently and take account of circumstances specific to the different service provider areas, AGD has developed a mechanism to allocate funding—the Funding Allocation Model (FAM). AGD commissioned the development of the FAM in 2006 with the completed mechanism being first implemented in 2008.⁴³ The FAM has been updated twice since then, once in 2009 and again in 2013 to incorporate up-to-date census data into the mechanism. As of December 2014, AGD was working toward the development of a new FAM to support proposed reforms to Indigenous legal assistance funding arrangements expected to be implemented from 1 July 2015.

Application of the FAM

2.28 The FAM applies a weighting to take into account a range of mainly demographic factors—particularly the age and gender distribution of Indigenous communities. These factors attract the highest weightings in the FAM. The FAM also includes ‘special purpose’ weightings for residents removed from their families as children, and for prison locations. The FAM also has the capacity to incorporate additional weighting factors as empirical data becomes available to accurately support them. The process involved in allocating funds using the FAM is shown in Figure 2.4.

Figure 2.4: Process used to determine funding allocations

Source: ANAO.

2.29 The use of the applied weightings results in the calculation of a per-capita amount specific to each jurisdiction. These amounts are then multiplied by the Indigenous population of a jurisdiction to arrive at the final funding allocation. Figure 2.5 demonstrates the resources allocated to each of the eight service providers compared to their respective Indigenous populations.

Figure 2.5: Funding allocation compared to Indigenous population

Source: ANAO, based on AGD updated funding allocation mechanism data.

Integrity of the FAM

2.30 To consider the integrity of the FAM, the ANAO conducted testing on the 2013 model. The FAM has been built using a Microsoft Excel workbook which is made up of 15 separate sheets. Analysis of the FAM found that while the spreadsheets did allocate funding in accordance with the intended methodology, some weaknesses were identified, as outlined in Table 2.5.

Table 2.5: Identified FAM weaknesses

Source: ANAO analysis of the FAM.

2.31 While none of the issues identified in the current FAM were found to have a substantial impact on its results, they do highlight common areas of vulnerability that should be addressed by AGD when revising the mechanism. AGD advised that it is revising the FAM as a part of reforms to legal assistance services proposed to commence from 1 July 2015. Issues under consideration include Indigenous population levels, socioeconomic status and geographical remoteness. The revised funding model is to inform the allocation of ILAP funding from 1 July 2015.

Availability of funding

2.32 Service providers informed the ANAO that the funding received under ILAP is currently inadequate, in their view, to allow service providers to meet demand, or to provide the level of services needed—particularly in remote and very remote areas. The level of funding currently received was considered to have not kept pace with rising delivery costs, nor to reflect policy and legislative changes that increase demand, or the cost of delivery over the funding period. Over recent years some state and territory governments have introduced new legislation aimed at improving behaviours relating to child welfare, anti-social behaviour, and housing, among others. Such legislation is often designed around reducing tolerance to, and taking a tougher stance on, crime and anti-social behaviour. The implementation of these policies often has an impact on Indigenous Australians, significantly increasing the number of people who require legal assistance.

ILAP risk management

2.33 Actively managing risks is an important part of program administration, enabling managers to identify and treat potential threats to program outcomes. Risk management involves the identification, analysis, treatment and allocations of risks, both in relation to the overall design of a program and, where relevant, administration of individual funding agreements. The ANAO examined AGD’s approach to managing risk at both these levels to understand the extent to which ILAP risks have been identified, and how they are incorporated into the ongoing program implementation and decision-making.

Program level risk management

2.34 To manage identified risks well, regular monitoring and treatment of risks should occur throughout the entire funding period. AGD developed the ILAP program risk assessment at the beginning of the funding term in 2011. However, no evidence was available to indicate that the risk assessment had been actively implemented or reviewed since that time, or how it has influenced decision making. From analysis of the risks listed in the ILAP risk assessment, the ANAO observed that over 50 per cent of the risks identified had been realised, yet AGD had not applied remedial action to address the consequences. Risks realised related primarily to:

• changes in state and territory policy directions resulting in increased demand for legal services;
• available funding not meeting demand;
• high demand for services impacting adversely on the quality of services delivered by ILAP service providers;
• performance data submitted by ILAP service providers being inconsistent or inaccurate; and
• change in government policy which could impact on resourcing available to administer the program.

2.35 AGD’s current approach to program level risk management is reactive and, for the most part, has seen AGD absorb risk consequences as they occur. While the ANAO recognises that some risks identified in the ILAP risk assessment are beyond the control of AGD⁴⁴, others were foreseeable and a more active approach could improve AGD’s ability to deal with and remedy risks as they are realised, reducing the impact of program risk on AGD, Indigenous clients and service providers.

Project level risk management

2.36 AGD undertakes risk assessments for each service provider and adjusts its reporting requirements and payment frequency according to the level of assessed risk. Low risk providers receive funding on a six-monthly basis, with medium risk providers paid quarterly and high risk providers paid monthly. This framework is the main tool through which AGD gives effect to its risk management approach to ILAP at the project level. Where risks are assessed as higher, the service provider’s reporting requirements increase, and the provider receives less autonomy with respect to its forward funding horizon. While such an approach could provide a basis for effective project level risk management, as outlined below, AGD has not established systems to support its consistent application, thus undermining its usefulness.

2.37 AGD uses a risk assessment spreadsheet to assess and allocate risk ratings. The risk assessment spreadsheet is partly aligned with the funding agreement reporting requirements and partly reflects other information available to AGD. Overall the spreadsheet is limited to guiding an assessment of whether a service provider has fulfilled its administrative and reporting obligations. The accuracy and quality of the information contained within the reports submitted is not verified by AGD. Consequently, the approach provides only a limited degree of assurance to AGD, reflecting a reporting compliance checklist, rather than providing the basis for managing risks.

ANAO review of risk assessments completed by AGD—January to June 2014

2.38 The ANAO examined each risk assessment undertaken by AGD for the eight service providers for the period January to June 2014. For these assessments, AGD had used different versions of the risk assessment tool, leading to inconsistent approaches to calculating risks for providers, and the instructions for calculating risks were not accurate or aligned to the data included in the spreadsheet. Generally, AGD’s risk assessments did not identify risks specific to each provider; did not include analysis, evaluation or rating of risks; or provide a framework to treat risks as they arise. Of the eight assessments examined, none of the providers had been assigned a rating using the spreadsheet. In two cases the risk ratings assigned by the risk ratings template differed to the formal risk rating that AGD assigned to each of the providers, as shown in Table 2.6.

Table 2.6: Comparison of risk ratings given to providers by AGD

Source: ANAO analysis of service provider risk assessment January to June 2014.

Note: * This risk rating was calculated by ANAO using AGD’s risk assessment sheets.

2.39 AGD informed the ANAO that when the department considers it necessary to monitor a particular provider more closely, other indicators can be used from time to time to force a risk rating higher, thus triggering more frequent reporting and payment cycles under the funding agreement.⁴⁵ Overall, the assignment of risk ratings largely occurs reactively and in response to problems as they arise, rather than as a result of a forward-looking and consistently applied project risk management framework. AGD acknowledged the limitations of the current risk assessment tool and advised that the current risk framework, including the spreadsheet, was being reviewed to ensure a more rigorous approach, with a greater focus on performance and not just compliance.

Conclusion

2.40 An important factor in the successful delivery of programs is establishing the objectives to be achieved, funding appropriate activities to achieve those objectives, and managing risks to the achievement of the objectives. ILAP objectives are clearly stated in the program guidelines and focus on the provision of legal assistance services that are accessible, culturally appropriate, equitable, efficient and effective. The program’s funding strategy of using Indigenous organisations to provide these services, using local outlets, is reasonably well aligned with the program’s objectives, and service usage is generally consistent with the distribution of the Indigenous population. Providing services in regional and remote areas to disadvantaged communities has inherent challenges and is costly, and the closure during 2014 of a number of regional outlets is likely to have affected access in those areas.

2.41 AGD is kept informed of office locations and resource allocations made by service providers but does not play an ongoing role in assessing demand within or between jurisdictions, or advising where offices should be located. Service providers are considered to be best placed to determine the locations and numbers of outlets and although AGD endorses these as proposed by the provider, no adjustments to funding are made when a provider chooses to close or move an outlet. The priority of ILAP is to deliver services to Indigenous communities with the highest need. However, to date, AGD has not undertaken any significant work to determine which areas of the community are considered to be ‘high need’ or which communities are experiencing access barriers. In this respect, funding allocations at a broad level are derived from assessing a combination of population, risk and cost factors and, while these generally reflect broader population distribution, the relationship to levels of disadvantage at the local government area was less consistent.

2.42 Under AGD’s management framework, low risk providers receive funding on a six-monthly basis, with medium risk providers paid quarterly and high risk providers paid monthly. This framework is the main tool through which AGD gives effect to its risk management approach to ILAP at the project level. While such an approach could provide a basis for effective project level risk management AGD has not established systems to support its consistent application, thus undermining its usefulness.

Introduction

3.1 Grant funding is a common means for governments to deliver services and pursue policy objectives. It is important that approved grants are administered in a way that promotes cost-effective and accountable achievement of the objectives. In this respect, the grant agreement (or funding agreement) provides a key framework for the administration of grants and the mechanism for identifying the outcomes expected to result from the grant, the terms and conditions to be complied with, reporting and requirements and the basis for payments. Effective grants administration also encompasses the implementation of an appropriate monitoring regime to ensure that grant recipients are meeting agreed conditions and that performance meets expectations. The ANAO examined the grant management arrangements put in place by AGD for ILAP.

Managing service delivery

3.2 AGD has developed and implemented arrangements to manage and monitor the delivery of Indigenous legal services by the eight legal service provider organisations. The primary tools enabling AGD to manage and monitor the service delivery are the ILAP Program Guidelines 2014 (ILAP guidelines) and a funding agreement with each of the service providers.

Program guidelines and funding agreements

3.3 Under the then Commonwealth Grants Guidelines (CGGs) ‘Agencies must develop grant guidelines for new grant programs, and make them publicly available (including on agency websites) where eligible persons and/or entities are able to apply for a grant under a program’.⁴⁶ The ILAP guidelines, originally released in 2011, were developed in consultation with the then Department of Finance and Deregulation and approved by the then Attorney-General. Prior to their approval, AGD sought further advice from the Australian Government Solicitor that the ILAP guidelines and funding agreements complied with the CGGs.

3.4 The guidelines provide stakeholders with general information on ILAP including: purpose, objectives, priorities, structure, eligibility, performance measures and compliance, the application process and complaints mechanisms. In July 2014, updated ILAP guidelines were released covering the 2014–15 extension of the funding agreement.⁴⁷ Under a typical grant program, guidelines would play an important role during funding rounds by communicating the program objectives and requirements to potential stakeholders and eligible organisations. However, as AGD currently administers the program using a direct sourcing arrangement—rather than conducting funding rounds, the ILAP guidelines do not play such an integral role in communicating the details of the program, and facilitating access to grants, as they would in other grants programs.

Funding Agreements

3.5 A funding agreement should clearly set out the services to be delivered and the expected level of quality of those services. The funding agreements currently in place for ILAP do not show a strong performance orientation or specifically identify objectives or expected targets. General statements are included in the agreements that providers are to deliver legal assistance services that are of high quality, culturally sensitive, equitable and accessible although these important features are not defined in the agreement, which affects the ability of the service provider and AGD to objectively assess whether services in fact meet those expectations. Further, the stated approach to performance assessment is simply to measure performance against the service providers’ compliance with the terms and conditions of the funding agreement rather than against the achievement of any specific objectives. AGD advised that as a part of the proposed future arrangements for legal assistance, all current ILAP documentation including funding agreements, program guidelines and policy documents, such as the Service Delivery Directions are being reviewed.

3.6 The types of legal assistance services to be delivered are specified in the funding agreements but there is no information on the expected numbers of services to be provided, the number of outlets or the location of service outlets. Standards for the delivery of these services are also not included in the agreements. These matters are covered to some extent in other documents such as the Service Plan that is prepared annually by each service provider. Some guidance is also given in supporting documents including the Service Delivery Directions and Service Standards. Overall, the funding agreements are largely compliance focussed and give limited information on expected performance. In the absence of such information, AGD is not well positioned to objectively assess the effectiveness of providers in relation to the program’s objective.

Operational guidance

3.7 AGD has developed a suite of operational guidance documents which include:

• Service Delivery Directions;
• Service Standards;
• Reporting Requirements; and
• Data Protocol.

Service Delivery Directions

3.8 The Service Delivery Directions (SDDs) outline the activities that providers are required to comply with, including:

• areas of practice—types of legal assistance and related services able to be provided to clients;
• eligibility—who is eligible to receive assistance, including how the means test should be applied and priority clients; and
• how providers should co-operate with other legal service providers.

3.9 Of the eight service providers interviewed by the ANAO, all advised that the SDDs are flexible enough to generally allow the delivery of services as necessary. However, minor services relating to wills, funerals, and personal tax or billing issues were not included in the SDDs, but were matters that clients often sought assistance with.

3.10 The SDDs provide the criteria used to assess a client’s eligibility for ILAP assistance which includes guidance to help service providers determine a person’s Indigenous status. Generally, the guidelines state that anyone who has descended from, identifies with, or is accepted as an Aboriginal or Torres Strait Islander is considered to be Indigenous. The SDDs also state that flexibility should be used when applying the guidelines. Service providers are also required to apply a means test to applicants seeking assistance with legal matters. The means test involves ascertaining whether a client satisfies one of the following: is under 18 years of age; main source of income comes from Centrelink benefits; or gross personal income is under $46 000 per annum. In cases where an applicant does not satisfy the above requirements, the SDDs provide the flexibility for eligible clients to contribute to the cost of providing the legal assistance service.

3.11 Eligibility is also assessed based on priority. The SDDs state that priority should be given to an eligible client who is, or is likely to be, detained in custody or faces a real risk of being significantly disadvantaged were assistance not provided. This reflects ILAP’s focus on providing criminal law services. The SDDs further state that assistance should not normally be provided in civil or family law matters unless the eligible client meets the exceptions set out in the SDDs.⁴⁸

3.12 Some providers advised that while they are conscious of their obligations to apply the eligibility criteria, for many cases, they do not have the time available to properly investigate a client’s eligibility. In remote areas, providers observed that the demand for services, and the workload experienced by lawyers is such that there is little opportunity available to apply the eligibility criteria to every client. Providers advised that while they did not always apply eligibility criteria as required, the majority of clients serviced would be eligible.

Service Standards

3.13 The SDDs are supplemented by the Service Standards. The Service Standards outline the expected level and quality of services that providers are to deliver under the funding agreement. The five service standards that are to be applied relate to the:

• delivery of legal services;
• accessibility and cultural sensitivity;
• co-operation and relationships with other legal service providers;
• organisational management; and
• assessing client satisfaction and managing complaints.

3.14 For each of the service activities conducted, providers must ensure that the activity is provided in:

• accordance with any relevant law regulating the legal profession and the provision of legal services in the jurisdiction in which the activity is delivered;
• accordance with best practice, where all service provider personnel conduct themselves consistent with professional standards;
• an efficient and effective manner; and
• an accessible and culturally sensitive manner.

3.15 Compliance with the Service Standards was initially to be assessed through Service Standards Audit Reports, requiring providers to complete and submit to AGD a self-assessment against each of the five service standards. However, the requirement to complete the Service Standards Audit Report was not enforced by AGD due to the subsequent introduction of the electronic Indigenous Justice Quality Practice Portal (the portal).⁴⁹

3.16 The portal has streamlined reporting against the Service Standards and although providers comply with their obligations to report using this arrangement, the system has not necessarily improved AGD’s ability to effectively monitor compliance with the Service Standards. The implementation of the portal created two primary problems. The first of these was that the portal had not been designed in a way that aligned closely with the ILAP Service Standards. Additionally, the portal only allows AGD officers to see whether providers have completed a Service Standard or not. The design of the portal does not allow AGD officers to view the details that service providers have entered against each of the Service Standards or the level to which the Service Standards have been achieved—significantly reducing AGD’s visibility over the compliance with the Service Standards and leading providers to potentially undertake reporting effort unnecessarily.

Reporting Requirements

3.17 The Reporting Requirements guidance document outlines reporting obligations under the current funding agreement, as shown in Table 3.1.

Table 3.1: ILAP provider reporting requirements

Source: ANAO analysis of ILAP guidance documents and funding agreements.

3.18 Business plans, including Organisational Constitutions, Fraud Control Plans, Disengagement Plans and Service Plans, must be provided at the commencement of the funding agreement. Each business plan must be consistent with the requirements set out in the funding agreement and program guidelines. AGD reviews each of the plans submitted, before the initial funding payments are made to the provider.

Service Plans

3.19 Service Plans, introduced in 2011 as a strategy to enhance service providers’ strategic planning, are required to be updated and submitted to AGD annually throughout the life of the funding agreement. These plans are to include strategies to meet the needs of Indigenous people at the community level, and address any disadvantages that may result from remoteness or language difficulties. In particular, Service Plans are intended to outline: how and where providers will deliver services; manage the service; cooperate with other legal service providers; and how success will be measured. Service targets outline the anticipated number and type of matters to be completed. Performance against targets is discussed during performance meetings which are conducted bi-annually. Currently, each ILAP service provider proposes targets for the number of services they expect to provide in their annual service plans, which are endorsed by AGD as part of the annual planning process. However, the definition of a service can vary between jurisdiction and AGD has not developed a benchmark level of service against which an assessment of proposed targets can be made. As a result service targets are largely based on prior year experience and the budget made available through the funding allocation model.

3.20 The ANAO found that each provider had a Service Plan in place, however, the level of detail provided in each of the Service Plans varied. While some Service Plans were thorough and addressed the requirements set by AGD, others were less comprehensive. The ANAO found:

• Service Plans were inconsistent in detail and quality of content;
• four of the eight Service Plans examined, did not provide dates or deadlines for when monitoring activities would be conducted or for when deliverables would be achieved;
• Service Plans generally did not align adequately with the Service Standards; and
• three of the eight Service Plans reflected a statement of intent rather than a plan for how their organisation would be managed.

Data Protocol

3.21 The Data Protocol guidance document outlines the obligations for reporting service data. Every quarter, providers are required to submit three data reports from each service outlet. The data reports contain information about service outlets, and clients including: Indigenous status; whether the means test applied; matter type (for example, advice, duty or case matters); law type; whether the client was referred or refused aid and the reason; and the outcome for the client. Data is collected and collated by providers using their own internal Client Management Systems (CMS), which must correspond with requirements for data set out in the Data Protocol, ensuring that the data is compatible with AGD’s data management system—the Indicator Reporting Information System (IRIS). Further information on AGD’s use of IRIS is provided in chapter 4.

3.22 Although guidance on data definitions is provided, providers observed that interpretations of the definitions vary between jurisdictions, which affects how each provider reports on the number of matters completed. AGD uses the collected data to compare and cost performance, inform risk ratings and determine payment schedules. However, AGD informed the ANAO that it does not undertake any verification of the use of the Data Protocol, nor does it validate the data received. AGD’s use of the data collected from providers is discussed further in chapter 4.

Service provider views on reporting

3.23 Service providers differed in their opinion on how burdensome reporting requirements were. While two of eight providers advised that they did not regard reporting requirements to be onerous, the remaining providers considered that reporting requirements were onerous. This was particularly prevalent among those providers who reported that they had insufficient resources to engage dedicated administrative staff to attend to reporting obligations.

3.24 More significantly, providers reported that the current method of reporting does not fairly represent the full work undertaken to complete each case. For example, a single legal assistance matter could be a highly complex case involving a high level of effort over a long period, or a simple case which can be completed in a matter of days. In both cases, this would be reported as a single matter without reflecting the differing resources required. Moreover, reporting does not account for the amount of ‘behind the scenes’ work undertaken by both the Client Service Officers (CSOs), interpreters (where engaged) and lawyers. One provider advised that no ‘effort index’ is applied to distinguish between the amount of resources required to complete different matters. Providers indicated that while they were prepared to provide data to account for their performance, they would prefer to collect and report data that would benefit the program by accurately reflecting the amount of work actually undertaken and that this be used to influence funding allocations.

Monitoring performance against funding agreements

3.25 Ongoing monitoring of performance against funding agreements is necessary to enable AGD to assure itself that providers deliver legal assistance and related services as intended and resources are appropriately applied. Financial monitoring enables assessment of whether relevant accountability procedures, associated with the funds, have been complied with. Performance monitoring also enables assessment of whether results are being achieved in line with expectations.

Financial monitoring

3.26 During the term of the funding agreement, providers are required to: submit accrual budgets; income and expenditure reports; and audited financial statements. Each of these must be submitted annually, with the exception of income and expenditure reports which are submitted bi-annually. The ANAO found that where service providers had not submitted reports on time, or had not submitted reports to the standard expected, AGD officers worked with them to enable reporting obligations to be met.

Service provider audits

3.27 Prior to the 2011 funding agreement being signed, AGD commissioned a limited assurance audit of the eight ILAP service providers. The report found that seven of the eight providers were financially sound but identified areas where financial practices could be improved, including: administration of assets registers; purchase orders; employee contracts; and delegations of authority. Subsequent to the audit findings, AGD included a clause in the 2011–15 funding agreement allowing AGD to appoint a funding controller if AGD considers that a service provider is under performing.

3.28 During 2014, AGD engaged an external firm to conduct further audits on all ILAP service providers. The audits were initiated as:

• AGD had not undertaken a financial assessment on the providers since 2009–10 (prior to the commencement of the 2011–15 funding term); and
• further investigation of compliance issues was required as AGD had been unable to consistently verify compliance using desk-top reviews or from information submitted by service providers.

The audits had not been finalised as at December 2014, however initial findings indicate that governance risks are present in the management of some service providers which will require attention from AGD.

Performance monitoring

3.29 To assess whether service providers are meeting their various performance and accountability requirements set out in the funding agreement and operational documents, AGD has developed a monitoring framework that anticipated the implementation of:

• performance meetings with service providers;
• field visits;
• stakeholder surveys; and
• a complaints management system.

Performance meetings with service providers

3.30 Bi-annual performance meetings are conducted via video or phone link between AGD and each provider. Performance meetings were established for the current funding agreement to assist AGD to monitor service quality and performance. Meeting minutes examined by ANAO show that AGD has facilitated regular performance meetings. Issues discussed during performance meetings include:

• risk assessments (and relative rating) undertaken against the service provider;
• audited financial statements and payments;
• performance data;
• compliance with SDDs and Service Standards;
• staffing and recruitment;
• collaborative projects and initiatives with other stakeholders;
• relevant government issues; and
• other matters relevant to each particular service provider.

3.31 Meeting minutes showed that although these topics were generally discussed during performance meetings, this is done at a very high level. The ANAO found that AGD did not use performance meetings as an opportunity to examine compliance with the SDDs and Service Standards, or to thoroughly investigate issues relating to performance or governance. For example, AGD was aware of performance issues or complaints about some providers relating to:

• recruitment;
• governance and organisational management problems; and
• policies on determining client eligibility.

These issues, although recognised by AGD, were generally not addressed during performance meetings. AGD advised that although these issues are not directly addressed in performance meetings, they have been addressed in other ways outside of performance meetings.

Field visits

3.32 During the preceding funding agreement, AGD conducted regular field visits which were generally received positively by service providers. However, under the 2011–15 funding agreement, AGD has not conducted regular field visits. AGD advised that during the development of the 2011–15 ILAP funding agreement, it had initially planned to meet with providers annually to supplement one of the performance meetings however field visits have been rarely undertaken.

Stakeholder surveys

3.33 ILAP Reporting Requirement guidelines indicate that AGD would conduct annual stakeholder surveys as part of assessing provider performance. As at December 2014 AGD had only undertaken one stakeholder survey activity which was conducted in 2013. This was undertaken by AGD staff using an online survey tool and telephone calls. The survey gathered feedback from a variety of stakeholders including: legal aid commissions; magistrate courts; court support services; relevant correctional services; and other non-government agencies. Responses from stakeholders were generally positive and highlighted that, in their view, ILAP service providers:

• play a critical role in the community and are active in local service provider networks, allowing for appropriate and timely referrals to be made;
• assist in breaking down barriers between the Indigenous community and the justice system; and
• are highly committed to, and advocate for, their clients.

3.34 Challenges identified during the stakeholder survey included:

• physical accessibility to legal representation—due to staff having limited time at remotely located courts, and the difficulty in accessing legal services from very remote locations;
• adequacy of legal representation—staff workload and working conditions, as well as level of experience and qualifications, impact on staff being able to deliver quality legal assistance services; and
• aligning services with the demands of Indigenous clients particularly within the context of family and civil law.

Management of complaints

3.35 Complaints about a service provider can be made to both the provider or AGD, although in the first instance complaints are to be submitted to the service provider. The SDDs require all providers to develop, maintain and apply a complaints policy that addresses how complaints are received, recorded, responded to, and investigated. The ANAO observed that the majority of providers had established complaints handling policies and therefore complied with the SDDs. However, one complaints policy had not been updated since 2005 and did not detail the information to be recorded about a complaint, the procedure required when complaints are not resolved within 40 days, or if there are any avenues for review. Another complaints policy did not specify the review procedures available to a complainant.

3.36 If a complainant is not satisfied with the response by the service provider, a complaint can be escalated to AGD. The ANAO examined 14 complaints received by AGD which were recorded using a spreadsheet and emails. However, the ANAO found that the spreadsheet was not well maintained and contained inaccuracies and incomplete information. Moreover, investigative and remedial action undertaken by AGD was not recorded on the spreadsheet. Overall, improvements could be made to the department’s approach to managing client complaints.

Managing provider compliance with funding agreements

3.37 During the term of the current funding agreement, AGD has identified through performance meetings and complaints, a number of ongoing problems with two service providers. One provider was considered by AGD to have problems relating to client discrimination and service delivery. The other provider was considered to be experiencing ongoing governance and management issues which affected staff and service delivery. However, until 2014, AGD had not undertaken any further or direct action to manage these issues of concern, other than advising service providers of their obligations under the funding agreement and the provision of governance training for board members and management in seven of the eight service providers.

Conclusion

3.38 Program guidelines, funding agreements, support guidance and a set of reporting requirements have all been developed by AGD to support its management of ILAP service providers. These arrangements provide AGD with a reasonable framework to manage the grants but the actual operation of the framework could be improved. Arrangements have a strong focus on monitoring compliance by service providers with the terms of the funding agreements, but overall do not support a strong focus on performance against the objectives of the program in relation to legal assistance services that are of high quality, culturally sensitive, equitable and accessible. Key performance requirements are described but these are not detailed in the funding agreements and are managed separately. A key document in this respect is the annual Service Plan which is submitted by service providers. While in place, the comprehensiveness and quality of these plans varied across different service providers.

3.39 Reporting requirements of the service providers include providing service plans, self-assessment reports on service quality and other reporting covering financial and non-financial performance. In an effort to simplify reporting requirements, AGD developed an online portal through which providers submit their reports. This is a positive initiative, however while this approach provides AGD with a high level view of compliance, it gives generally limited visibility over the details of actual performance against service standards. In addition, planned annual stakeholder surveys have not been carried out, other than on one occasion, leaving AGD with limited information on client and other stakeholder perceptions of the quality of services provided. Overall, AGD’s arrangements for managing ILAP service providers are reasonable in their design, however could be strengthened by introducing a greater focus on results in the funding agreements including greater verification by AGD of relevant performance data submitted by service providers. Over time, this would enable AGD to better monitor and assess improvements in access, and the quality of legal services provided to clients.

Introduction

4.1 Performance monitoring and reporting is an integral part of sound program management. ILAP service providers are directly responsible for the delivery of legal assistance services, however, as the administering entity, AGD remains responsible for overseeing the quality of services being delivered, as well as monitoring and reporting on the program’s performance against the objectives agreed by the Australian Government. ANAO examined the approach taken by AGD to monitoring and reporting program performance. This included examining ILAP’s:

• performance reporting framework;
• internal performance monitoring and reporting arrangements; and
• external performance reporting arrangements.

Performance reporting framework

4.2 To be able to effectively assess program performance, entities need to establish, and communicate clearly and consistently, the objective or expected outcomes of a program. This includes having an agreed understanding of the key operational definitions, a baseline against which program performance can be assessed and relevant targets and indicators against which program delivery can be assessed. Performance reporting frameworks should also be supported by effective systems and practices to systematically monitor performance, capture a variety of relevant data, and facilitate accurate and reliable reporting.⁵⁰

4.3 As noted in paragraph 2.6 ILAP has been funded under the Indigenous Law and Justice (ILJ) program and in 2014–15 comprised 99 per cent of that funding.⁵¹ The provision of services to support access to justice is explicitly identified as the main deliverable for the ILJ program. For the purpose of reporting to the Parliament, AGD has used two KPIs in the 2014–15 PBS to assess performance against the objectives of the ILJ program. These KPIs are:

• improved access to justice for Indigenous people; and
• effective administration of the access to justice programme for Indigenous people.⁵²

4.4 The KPI ‘[i]mproved access to justice for Indigenous people’ is intended to provide a measure of the impact of the ILJ program. However, without targets or baseline information on the level of access, or an operational definition of access, AGD is unable to assess whether access to justice is improving as a result of the ILJ program and the associated ILAP funding. The KPI ‘[e]ffective administration of the access to justice programmes for Indigenous people’, measures internal activities and support processes required to manage the ILJ program, although similarly in the absence of targets or benchmarks limited assessment can be made of performance in this respect.⁵³ Changes to program KPIs were not made when other parts of the ILJ program were transferred to PM&C (refer to paragraph 2.6). AGD has advised that ILAP will be administered under a different program (the Justice Services program) in future years and that this will improve the alignment between the program activities and related KPIs.

4.5 AGD has described specific, and more qualitative, objectives of ILAP in the program guidelines. As stated in the program’s guidelines, the objective of ILAP is to ‘… deliver culturally sensitive, appropriate, accessible, equitable, efficient and effective legal services to Indigenous Australians, so that they can fully exercise their legal rights as Australian citizens’. ⁵⁴ ILAP service providers are expected to achieve this by working collaboratively with other Indigenous and non-Indigenous legal and non-legal service providers, such as legal aid commissions and Indigenous community controlled organisations.

4.6 The legal assistance services provided under ILAP include: information and initial legal advice; duty lawyer assistance, which involves providing advice or representing a client at a court or tribunal; and ongoing legal assistance and representational services in the form of casework services. Other related services often provided include: community legal education, and early intervention and prevention initiatives, including delivering outreach, information sessions and resources on legal rights and responsibilities.⁵⁵

4.7 To assess performance against the combination of objectives described in the Portfolio Budget Statements (PBS) and the ILAP guidelines, important information to capture includes data relating to the quantity, types and locations of services provided under the program. AGD’s performance framework operates sufficiently to capture this information, and this is discussed in more detail in the following section. However, the performance framework does not systematically capture qualitative information in relation to whether services are in fact culturally sensitive, appropriate, accessible, equitable, efficient and effective. As noted in paragraph 2.5, these terms have not been given operational definitions in program guidelines or funding agreements and are primarily high level principles, which are not reported on by AGD.

Internal performance monitoring and reporting

4.8 Reporting by service providers to AGD focuses on the extent to which providers are meeting obligations in their funding agreements and is largely compliance oriented. Reporting requirements are set out in chapter 3 (Table 3.1) and include: service plans, annual reports, accrual budget, audited financial statements, income and expenditure reports and data reporting.

4.9 In addition to submitting reports for compliance purposes, service providers also report on an ongoing basis on the location and types of services being delivered and the quality of services provided. This reporting occurs through:

• quarterly data reports, providing an overview of service levels for advice, duty and casework services for criminal, family and civil law matters, for each service outlet; and
• six monthly reports against the Service Standards, self-assessing performance against the expected level and quality of services provided.

Reporting on service levels

4.10 ILAP service providers determine the targets for the number and types of services to be delivered annually in their service plans, which are intended to be key planning documents outlining how service providers intend to deliver and manage services.⁵⁶ The service targets are endorsed by AGD as part of the annual approval of service plans. As noted in paragraph 3.19, the service level targets are largely based on previous year results and AGD has not developed any baseline or benchmark for levels of activity against which the targets proposed by service providers could be assessed. The aggregate numbers of advice, duty and case matters reported against service level targets from 2011–12 to 2013–14 are shown in Figure 4.1. This shows that overall there has been a gradual, but not significant, increase in the number of services delivered by ILAP service providers from 2011–12 to 2013–14.

Figure 4.1: Number of advice, duty and case matters against service level targets, 2011–12 to 2013–14

Source: ANAO analysis of IRIS data.

4.11 AGD has reported to the ANAO that service level targets are used as a basis for understanding the number and types of services expected to be delivered within a financial year. If a service provider is not achieving the expected service level, AGD seeks further information from the provider, in order to assist with implementing remedial action.

4.12 Quantitative service delivery data on its own does not provide information about why changes in service levels have occurred. In 2013–14 providers overall exceeded their service level targets by three per cent, with the number of duty lawyer matters exceeding service level targets by 20 per cent. However, reporting this information alone does not inform AGD about whether providers have improved the accessibility of their services (for example, by hiring more staff, opening new outlets or improving customer focus), or whether demand for services has increased as a result of changes in policy, such as harsher stances on crime or legislated changes.

Figure 4.2: Number of advice, duty and case matters against service targets, 2013–14

Source: ANAO analysis of IRIS data.

Reporting on service quality

4.13 To monitor whether providers are delivering high quality legal assistance, in line with various rules and regulations governing the legal profession and practice in each state and territory, AGD requires reporting though the Indigenous Justice Quality Practice Portal (the portal). The portal is an online self-assessment system for providers to use to assess themselves against the Service Standards. The Service Standards outline the expected level and quality of services that ILAP service providers are to deliver under the funding agreement.⁵⁷ AGD considers providers to have met the Service Standards, and be fully compliant with the portal requirements, when they have completed the online self-assessment questionnaire analysed through the portal, and provided relevant supporting documentation.

4.14 Using the portal, AGD project officers are able to view when providers have completed the online self-assessment questionnaire and uploaded the relevant supporting documentation. However, project officers cannot view the answers to the questionnaire itself and the questionnaire does not inform AGD about whether quality legal services are being provided. In practice, AGD uses the portal as a central document collection and storage system. Supporting documentation uploaded by providers is used by AGD to assess compliance with the funding agreement and to give providers a risk rating. However, AGD does not periodically check that all of the information on the portal is up to date or complete.

4.15 Ensuring that providers have loaded documentation and completed the self-assessment questionnaire is time consuming, requiring a significant administrative effort from AGD. At the time of their last bi-annual performance meetings in June 2014, none of the service providers were in complete compliance with the information required on the portal. Furthermore, in June 2014, some providers advised the ANAO that they did not find the portal useful as an organisational management tool, despite being designed as such. Overall, while the portal is a useful initiative it does not fully deliver the benefits it was intended to. AGD needs to consider the way it assesses the quality of services delivered by providers and whether refinements can be made to the portal and the processes supporting it. Periodic verification by AGD of the service information reported by providers would strengthen AGD’s performance management arrangements.

Quantitative data collection

4.16 Service providers submit three data reports from each of their service outlets every quarter. Information reported to AGD includes the number of: advice matters; duty matters; and casework services for criminal, family and civil law matters. In 2013–14, AGD reported 90 103 advice matters, 86 949 case matters and 29 436 duty lawyer matters were achieved under the program across all providers.

4.17 AGD requires service providers to report client and service data in a pre-defined format that is compatible with the Indicator Reporting Information System (IRIS), the system used by AGD to monitor ILAP activity levels and report both internally to management and to external stakeholders.

4.18 The data currently used by AGD to monitor performance against service level targets is based on information primarily contained in progress reports and includes the number of criminal duty lawyer matters, and the number of criminal, civil and family advice and case matters. The progress reports do not count the number of family and civil duty lawyer matters, or matters for inquests (death in custody) and violence protection, although this information is recorded in IRIS. A comparison of the data used by AGD to monitor service providers against their service level targets and data recorded in IRIS is shown in Figure 4.3. Actual ILAP service levels in 2013–14 were slightly under reported by 11 416 matters. By reviewing reported service levels using less comprehensive data compared to that available in IRIS, AGD cannot accurately monitor whether service providers are meeting or exceeding expected service delivery.

Figure 4.3: Comparison of data used to monitor progress against service level targets, and total matters reported in IRIS, 2013–14

Source: ANAO Analysis of IRIS data.

4.19 Despite an ILAP data protocol being in place that requires service providers to report client and service data in a predefined format, generating accurate and reliable reports from IRIS is time consuming and requires a significant administrative effort from both providers and AGD. Issues relating to the IRIS system identified by AGD and providers include:

• inconsistent reporting of the number and types of matters delivered by service providers due to the use of different client management systems (CMSs) by service providers to collate and record data and different interpretations of data specifications in the data protocol;
• difficulties reconciling data recorded in service provider data systems with that recorded by AGD in IRIS; and
• inaccurate identification of data errors due to AGD and service providers running different versions of IRIS.

4.20 AGD advised the ANAO that it is aware of the inconsistencies in the data reporting and is currently exploring ways to improve the reporting process, including the implementation of a national legal assistance data standards manual.

Qualitative data collection

4.21 To supplement and provide additional perspective on performance, AGD recognised the need to include qualitative approaches in the design of its performance framework. These included plans for regular stakeholder surveys and field visits by AGD staff.

4.22 Under the 2011–15 funding agreement AGD indicated that it would conduct annual stakeholder surveys to obtain an independent measure of the level of stakeholder satisfaction with the services provided by ILAP. However, as discussed in paragraph 3.33, AGD has conducted only one stakeholder survey throughout the life of the funding agreement. Field visits were also initially planned to occur at least once a year to supplement one of the biannual performance meetings, and were intended to be used, in addition to the portal, to assess service provider compliance with Service Standards. During 2013–14 AGD visited five out of eight service providers. AGD advised that resource constraints have meant that field visits occur as required and that its uses field visits to assess compliance where necessary.

4.23 In place of regular field visits and stakeholder surveys, AGD has supplemented the reporting requirements outlined in the funding agreement with biannual performance meetings. While performance meetings provide AGD with increased visibility over the services being delivered, AGD’s overall approach to gathering qualitative data on services has limitations. Gathering qualitative data can be resource intensive and expensive for the entity but also the service provider organisation. Self-reporting by service provider organisations can be relatively cost-effective, however, if the data is not subsequently verified periodically, or used, it ultimately acts as a cost burden.

4.24 According to the data protocol, ILAP service providers are also required to collect a range of information for practice management purposes, but this is not required to be routinely provided to AGD. The information includes data on:

• accessibility and targeting of services—client feedback, referrals and refusals, and the number of interpreters and field officers involved in a matter;
• early resolution of legal problems—number of advice, minor legal assistance and advocacy services, and the time it takes to resolve a legal problem based on complexity of the problem; and
• collaboration and cooperation between legal assistance and other service providers to address legal and other problems—the number of matters referred or refused, including the reason for referral and who the client was referred to.

4.25 This information, if consistently reported and verified by AGD, would provide AGD with relevant data on the accessibility and effectiveness of legal assistance services provided through ILAP and serve to strengthen the overall performance framework for ILAP.

External reporting

4.26 The objective of external reporting is to provide both the Parliament and the public with information about how a program is performing against its objectives. Information in relation to ILAP is currently reported externally by service providers in annual reports published on their websites, and in AGD’s annual reports to the Parliament.

Publication of service provider annual reports

4.27 To fulfil their reporting requirements, service providers submit an annual report to AGD. Annual reports summarise the service provider’s achievements for the year and describe services provided and activities relevant to their work. Currently, seven out of eight providers make their annual reports available on their websites. Of those annual reports available externally, some variations were observed in the level of reporting by each provider in relation to financial statements (some were extracts and did not attach the auditor’s opinion), performance information, and governance information. While the annual reports made externally available vary in quality, the majority of annual reports provide useful information on how ILAP service providers are delivering legal assistance and related services to Indigenous Australians.

AGD annual reports

4.28 AGD reports to the Parliament and the community in its annual reports. The information reported against KPIs relevant to ILAP in AGD’s annual reports, from 2011–12 to 2013–14, is shown in Table 4.1.

Table 4.1: AGD performance against relevant KPIs reported in AGD’s annual reports, 2011–12 to 2013–14

Source: Attorney-General’s Department, 2011–12 Annual Report, 2012, Canberra, p. 120; Attorney-General’s Department, 2012–13 Annual Report, 2013, Canberra, p. 59; Attorney-General’s Department, 2013–14 Annual Report, 2014, Canberra, p. 42.

4.29 AGD has reported that access has been achieved through providing grant funding for service providers to deliver legal assistance and related services. However, as noted in paragraph 4.4, no baseline figure against which to measure access is published and no objective measure of improvements is included in the annual report. Moreover, reporting on the provision of grant funding on its own is not a measure of whether there has been any effect on access to legal assistance services as a result of program activities.

4.30 There are particular challenges for demand-driven services in determining whether access is appropriate, as to do so also requires knowledge of the population that need the service but are not accessing it. Additionally, changes in usage levels can stem from a range of external factors, for example the availability of interpreters in courts, which would need to be understood in order to develop a meaningful assessment of whether services are accessible and equitable. In view of the relatively small size of ILAP, AGD should consider whether other proxy measures, are better able to indicate whether the program is maintaining a reasonable level of services against estimated demand. As noted in paragraph 4.24, service providers are required to collect additional information relevant to their organisations, some of which could be directed toward better assessments by AGD of issues relating to access. Other improvements to reporting could include identifying and reporting on:

• whether the number of delivered services are in line with the number of expected services, supported by commentary on why expected service levels were, or were not achieved; and
• lawyers available to give advice and undertake duty matters.

4.31 By supporting these performance indicators through periodic verification AGD would be better placed to understand the issues affecting service delivery.

Accuracy of reporting

4.32 From 2012–13, AGD commenced providing additional information on the number and types of matters provided by service providers in its annual report. In 2013–14 AGD reported that the total number of services delivered by service providers was 206 488.⁵⁸ However, ANAO analysis of performance data captured in IRIS found that the total number of matters reported in AGD’s 2013–14 annual report was lower than the number of matters actually reported in IRIS. This difference is shown in Table 4.2.

Table 4.2: The number and types of matters reported in AGD’s 2013–14 annual report and IRIS

Source: Attorney-General’s Department, 2013–14 Annual Report, 2014, Canberra, p. 42 and IRIS.

4.33 The discrepancies in the number of matters reported are relatively minor, however, it does indicate that some further refinement of data controls and accuracy would benefit AGD and support more accurate reporting.

Conclusion

4.34 Performance information collected by AGD for ILAP is currently focused around the quantity and type of services delivered, and whether service providers are complying with their funding agreements. These reporting arrangements are useful in respect of ascertaining levels of activity, location of services and types of matters addressed by providers. However, they are not sufficiently comprehensive to provide AGD with an appropriate level of visibility over other more qualitative aspects of program performance indicating whether services are high quality, appropriate, accessible and equitable.

4.35 To monitor whether providers are delivering high quality legal assistance, AGD developed an online self-assessment system for providers to assess themselves against the program’s agreed Service Standards. This system is a useful initiative, however, it does not fully deliver the benefits that were expected. AGD needs to consider the way it assesses the quality of services delivered by providers and whether refinements can be made to the online system and the processes supporting it. Periodic verification by AGD of the service information reported by providers would strengthen AGD’s performance management arrangements.

4.36 While it is reasonable to expect that improving performance of ILAP service providers would contribute to improved outcomes for Indigenous Australians, no baseline information on existing levels of access, or desired benchmarks, has been prepared by AGD. As a result, the performance reporting framework is not able to support an objective assessment of whether the levels of access enabled by ILAP are appropriate and whether they are improving.

Recommendation No. 1

4.37 To support a stronger performance focus in the development of future funding arrangements, ANAO recommends AGD further develops its performance measurement and reporting framework by:

1. developing and incorporating baseline data against which service targets could be assessed and changes measured; and
2. strengthening systems and processes to capture, monitor and report relevant data, including conducting periodic data integrity checks to assess the accuracy and reliability of the data collected.

AGD Response:

4.38 Agreed. The department acknowledges the importance of improving the collection of data as well as ensuring services are fostering social, cultural and economic benefits and contributing to the maintenance of the rule of law.

4.39 The department is developing new performance and reporting measures as part of the future arrangements for the programme from 1 July 2015. This includes working with the National Data Standards Working Group to develop a national standard for legal assistance data, in order to facilitate a minimum consistent data set across all four legal assistance providers.

4.40 In addition to this, the programme objectives will change from output focused to outcome focused, with improved processes both internally for programme managers and externally for service providers.

4.41 The new arrangements to legal assistance from 1 July 2015, including the introduction of a national strategic framework, will set the future direction of legal assistance in Australia for all government-funded services, including the Indigenous Legal Assistance Programme and guide future legal assistance policy and service delivery.

Appendices

Please refer to the attached PDF for the Appendices:

• Appendix 1: Entity Response
• Appendix 2: Most Disadvantaged Communities—Indigenous Population over 50 per cent

Abbreviations

Glossary