Browse our range of reports and publications including performance and financial statement audit reports, assurance review reports, information reports and annual reports.
The Design of, and Award of Funding Under, the Living Safe Together Grants Programme
Please direct enquiries relating to reports through our contact page.
The objective of the audit was to assess the effectiveness of the design of, and award of funding under, the Living Safe Together grants programme.
Summary and recommendations
1. The Living Safe Together (LST) grants programme was developed as a component of the Australian Government’s Countering Violent Extremism (CVE) Strategy. The programme’s objective was to support community-based, non-government and local government organisations to develop new and innovative services to help individuals move away from violent extremism (either directly, or through their families and friends). The programme is administered by the Attorney-General’s Department (AGD).
2. Programme design work was undertaken between November and December 2014. Applications were called for in early January 2015 and closed on 2 March 2015. At the time the programme was announced, up to $1 million was available in 2014–15 for grants of between $10 000 and $50 000. The department received 97 applications seeking a total of approximately $4.9 million.
3. Programme funding was approved in two tranches:
- in late April 2015 AGD recommended, and the Attorney-General approved, funding of $1.6 million for 34 applications; and
- in late May 2015 AGD recommended, and in early June 2015 the Minister for Justice1 approved, $365 122 in grant funding for a further eight projects.
Audit objective
4. The objective of the audit was to assess the effectiveness of the design of, and award of funding under, the LST programme.
5. Consistent with a recent recommendation from the Joint Committee of Public Accounts and Audit (JCPAA), the scope of the audit included assessing whether AGD had applied the lessons it advised the JCPAA that it learned in response to the Australian National Audit Office’s (ANAO) audit of the Safer Streets programme.2
Conclusion
6. Programme design was undertaken in a timely manner and was reasonably effective. This included published guidelines that outlined a robust assessment process and included well-designed merit criteria. But the processes through which applications were selected and subsequently awarded funding were flawed in significant respects. Of note was that AGD departed in some important respects from the assessment approach set out in the programme guidelines. Only 21 of the 42 recommended and approved applications should have been successful (totalling funding of $1 million of the $1.9 million awarded) had AGD only recommended eligible applications that had been scored as satisfactorily meeting the published merit criteria.
7. At the time he was asked to approve funding, the Attorney-General was advised that, upon completion of their projects, it was expected that funding recipients would register on the Directory of CVE Intervention Services (CVE Directory). The CVE Directory was to be used to connect at-risk individuals with appropriate services.3 However, a key shortcoming in the programme guidelines was that AGD had not made clear enough to applicants that a key purpose of awarding grants was to have funding recipients register for the CVE Directory. Thirteen funding recipients have indicated to AGD that they will participate in the Directory, but two have advised they will not and the intentions of a further 26 recipients are not yet known.4
8. Whilst some progress has been made, there remains considerable scope for improvement in AGD’s administration of grant programmes. This is reflected by the continuing deficiencies in AGD’s approach to assessing the eligibility and merit of grant applications, as well as in the advice to decision-makers about those applications that should be funded and those that should be rejected.
Supporting findings
Design of the programme (Chapter 2)
9. The programme was developed and designed in response to calls from community stakeholders and an identified gap in Australia’s capability to deliver localised and tailored intervention services. The provision of these services by community and local government organisations was identified as important to the government’s broader CVE Strategy. Options other than a grants programme were not considered by AGD for addressing the capability gap.
10. Planning and design for the LST grants programme was conducted in a five week timeframe. Most important elements of grant programme design were attended to. But, in the time available before applications were sought, work was not undertaken in relation to developing neither the funding recipient reporting and acquittals processes, nor an evaluation plan for the programme. In addition, the programme risk assessment was too optimistic.
11. In most respects, the programme guidelines provided a reasonable basis for the implementation of the programme. Nonetheless, a key shortcoming was that the guidelines did not communicate to potential applicants the importance of the link between receiving grant funding and registering on the CVE Directory.
Application process and eligibility checking (Chapter 3)
12. The department took a number of effective steps to make the programme accessible and equitable. This included promoting the funding opportunity in a number of ways in recognition that potential applicants would not be aware of the programme and the impending call for applications. But there were also some aspects of the department’s approach to administering the programme that were not consistent with accessibility and equity principles.
13. Some of the eligibility requirements were less accessible to applicants than they could have been because they were dispersed across sections of the programme guidelines and within other, related, documentation. This approach does not assist applicants to submit quality applications, nor does it provide a sound basis for an effective eligibility checking process.
14. The approach taken to assessing the range of eligibility requirements that had been set out in programme documentation was poorly planned and ineffective. Only a small number of the eligibility requirements were explicitly addressed in the assessment template with the result that ineligible applications proceeded to the merit assessment stage and, in a number of instances, were recommended and approved for funding.
Merit assessment (Chapter 4)
15. The merit criteria were well designed and appropriate to the programme. They provided a solid foundation for the merit assessment process. As such, the data produced by assessing applications through this process should have allowed AGD to identify which had satisfactorily met the published merit criteria, and which had not. However, this was not the case due to a number of departures from the published programme guidelines. This involved steps outlined in the guidelines not being undertaken, as well as an additional, unpublished, assessment process employed.
16. Notably, the approach taken to assessing applications against the first merit criterion was inconsistent with the guidelines. The guidelines stated that only applications that satisfactorily addressed and met this criterion would proceed to a full assessment (and therefore were able to be recommended for funding approval). AGD allowed 46 of the 59 applications that did not satisfactorily meet that criterion to proceed to a full assessment. As a result, four of those 46 applications that had scored poorly against the key policy criterion were approved for funding in April 2015, and a further six were approved by the Minister for Justice in June 2015.
17. The result from the department’s assessment of applications identified that the majority of applications lacked sufficient merit in terms of the published programme guidelines. Applications were also not ranked in priority order. Whilst a numerical rating scale was used during the assessment process, the scores resulting from the assessment process were not used as the basis for identifying the applications that were recommended for funding.
Funding recommendations and decisions (Chapter 5)
18. The department provided written funding recommendations for the LST grants programme to the Attorney-General in April 2015 and the Minister for Justice in May 2015.
19. The briefing to the Attorney-General contained most of the information that is required to be provided to Ministers when approving grants. However, the extent of the accuracy and comprehensiveness of the application-specific information provided meant that AGD fell short of fully meeting these requirements. Basic project information was provided for the 34 applications that AGD recommended for funding, as well as advice that all 34 had satisfactorily addressed all the merit criteria. Based on applications’ final assessment scores, this advice was incorrect for 12 of these applications.
20. Also provided was a list of the organisations whose applications were not recommended. Notably absent from this list was advice as to whether or to what extent each individual application had been assessed and scored as meeting the merit criteria.
21. The process through which AGD identified that a further eight from the 62 unsuccessful applications would be recommended for funding was not sufficiently transparent, or equitable in respect to all unsuccessful applicants.
22. The written funding recommendation provided to the Minister for Justice in relation to these eight applications was deficient in that it provided no advice on how the eight applications had performed against the published criteria. Of particular note in this regard was that the available evidence indicates that applications were not reassessed against the merit criteria until well after they had been approved for funding by the Minister and grant funding paid in full to all recipients.
Recommendations
Recommendation No.1 Paragraph 5.39 |
The ANAO recommends that the Attorney-General’s Department take effective steps to provide greater accountability for, and effectively address, continuing deficiencies in its approach to:
Attorney-General’s Department response: Agreed. |
Summary of entity response
23. The Attorney-General’s Department provided formal comment on the proposed audit report. A summary of this response is provided below, with the full response provided at Appendix 1.
The Attorney-General’s Department considers the LST grants programme to have been successful in meeting its critical policy objective: to support community-based, non-government and local government organisations to develop new and innovative services to help individuals move away from violent extremism. The department is satisfied that all grant recipients recommended for funding met this policy objective, and that all except one are expected to have achieved their milestones and developed, by the conclusion of the project, the capacity to deliver an intervention service.
The department welcomes the ANAO’s conclusion that AGD has made progress in improving its grants programme administration, as well as its findings that programme design was undertaken in a timely manner, steps were taken to make the programme accessible, the merit criteria were well-designed and appropriate to the programme, and that applications were assessed through a sufficiently rigorous process.
The department agrees with the ANAO’s recommendation that it take effective steps to provide greater accountability for, and effectively address, continuing deficiencies in its approach to assessing the eligibility and merit of grant applications, and in advising decision-makers about those applications that should be funded and those that should be rejected.
Since the tabling of the Safer Streets audit report, the department has progressively implemented significant improvements to its grants administration practice, as outlined below. The department recognises the importance of appropriately administering grants programmes and is committed to a process of continuous improvement to maintain and enhance its grants administration performance and compliance.
It is now mandatory for all staff working in grants administration line areas to be trained in the use of the department’s grants administration ‘tool kit,’ which includes the Guide to Grant Administration, Gap Analysis Checklist, help cards and a suite of standard templates and documents available from the department’s internal data base.
The department acknowledges the ANAO’s audit report sets out a number of areas where the department can continue to improve in order to achieve best practice. This includes the need to improve the documentation of its deliberative decision-making processes; the need to establish clear and measurable thresholds for assessment of eligibility; and providing more comprehensive advice to decision-makers on the strengths and weaknesses of each application.
In response to the findings of the audit report, the department is now reviewing and updating its guidance material and training. The department is grateful for the ANAO’s separate advice about best practice briefing approaches employed by other agencies, and is working to incorporate those practices into its provision of advice to decision-makers.
1. Introduction
Background
1.1 In August 2014, the Prime Minister and Attorney-General jointly announced that ‘[t]o lessen the terrorist threat at home and abroad, the Government [would] invest more than $64 million in measures to counter violent extremism and radicalisation.’5 Of this, $13.4 million was allocated to a new four year Countering Violent Extremism (CVE) programme.
1.2 The CVE programme is aimed at combating the threat posed by home-grown terrorism and discouraging or deterring Australians from travelling offshore to participate in conflicts. It is overseen by the Attorney-General’s Department (AGD), with other Australian and state and territory government agencies playing key roles in the delivery of work under four critical streams of activity (see Figure 1.1).
Source: AGD records and ANAO analysis.
The Living Safe Together Grants Programme
1.3 The Living Safe Together (LST) grants programme was developed under the intervention and diversion stream of the CVE programme. Its objective was to support community-based, non-government and local government organisations to develop new and innovative services to help individuals move away from violent extremism (either directly, or through their families and friends).
1.4 At the time the programme was announced, up to $1 million was available in 2014–15 for grants of between $10 000 and $50 000.
1.5 Applications were called for in early January 2015 and closed on 2 March 2015. AGD received 97 applications seeking a total of approximately $4.9 million.
1.6 AGD was responsible for assessing the eligibility and merit of applications against the published guidelines. Funding was approved in two tranches:
- in late April 2015 AGD recommended, and the Attorney-General approved, funding of $1.6 million for 34 applications; and
- in late May 2015 AGD recommended, and in early June the Minister for Justice6 approved, $365 122 in grant funding for a further eight projects.
Audit objective, criteria and scope
1.7 The objective of the audit was to assess the effectiveness of the design of, and award of funding under, the LST programme.
1.8 To form a conclusion against the objective, the ANAO examined whether:
- the programme was well designed; and
- the award of funding was undertaken in a manner consistent with sound grants administration practice. In particular, whether:
- the application and eligibility checking processes promoted transparent and equitable access to the available funding;
- the merit assessment process identified and ranked in priority order the best applications in terms of the published criteria; and
- the Minister was appropriately briefed on the assessment results; was given a clear funding recommendation; and the decisions taken were transparent and consistent with the programme guidelines.
1.9 The scope of the audit included the design of the programme, the application processes and the assessment and approval of candidate projects. In addition, consistent with a recent recommendation from the Joint Committee of Public Accounts and Audit (JCPAA), the scope of the audit included assessing whether AGD has applied the lessons it advised the JCPAA that it learned in response to ANAO’s audit of the Safer Streets programme.7
1.10 The Safer Streets programme audit report was tabled on 4 June 2015, and was examined in JCPAA Report 452, tabled on 7 December 2015. On 20 March 2015, during the assessment of LST applications, AGD’s Grant Policy Unit advised the CVE Branch Head of the Safer Streets audit report (which had not yet been tabled) and that the ANAO report of the Safer Streets programme was ‘critical of many of the processes and/or procedures that were followed’ and identified the senior executives within AGD that could provide the CVE Branch with ‘further insights’.
1.11 The audit was conducted in accordance with ANAO auditing standards at a cost to the ANAO of $237 970.
2. Design of the programme
Areas examined
The ANAO examined whether the programme was well designed.
Conclusion
The programme was developed and designed in response to calls from community stakeholders and an identified gap in Australia’s capability to deliver localised and tailored intervention services.
Planning and design for the grants programme was conducted in a five week timeframe. Nevertheless, most important elements of grant programme design were attended to.
Of particular note was that the programme guidelines, which were developed in less than one month, provided a reasonable basis for the implementation of the programme. A key shortcoming was that the guidelines did not communicate to potential applicants the importance of the link between receiving grant funding and registering on the Directory of Countering Violent Extremism Intervention Services (CVE Directory). Thirteen funding recipients have indicated to AGD that they will participate in the Directory, but two have advised they will not and the intentions of a further 26 recipients is not yet known.
Areas for improvement
The ANAO has not made any recommendations in this chapter. But there would be benefits in more attention being paid to stronger linkages between the expressed objective for a grant programme, and the broader policy intent.
Why was a grants programme established?
The programme was developed and designed in response to calls from community stakeholders and an identified gap in Australia’s capability to deliver localised and tailored intervention services. The provision of these services by community and local government organisations was identified as important to the broader CVE programme. Options other than a grants programme were not considered by the Attorney-General’s Department (AGD) for addressing the capability gap.
2.1 The previous CVE programme ceased on 30 June 2014. It contained a competitive grants programme known as the Building Community Resilience programme. The aim of that programme was to support community projects across Australia that build resilience to violent extremism. The new policy proposal for the 2014–18 CVE programme, approved by the Government in August 2014, did not include the Living Safe Together (LST) Grants Programme.
2.2 In November 2014, AGD advised the Attorney-General that there was merit in re-establishing a competitive grants programme as part of the broader CVE Programme, but this time specifically designed to complement the work within the intervention and diversion stream of the CVE programme (one of four complementary streams, represented in Figure 1.1). Other options to a grants programme were not canvassed in that briefing. Departmental records did not identify that options other than a grants programme were considered by AGD for addressing the capability gap, and although AGD advised the ANAO that alternatives had been considered, no examples of these were provided.
2.3 Also as part of the intervention and diversion stream, and intended to have a close link with the LST grants, AGD developed the CVE Directory.8 This was intended to be a broad list of potential service providers for use by Commonwealth officials and law enforcement agencies when connecting individuals assessed as being on the path to radicalisation with appropriate services or programmes.9 AGD administered a concurrent—but separate—application process10 for the CVE Directory alongside the LST grants programme process.
Was the design of the programme timely?
Planning and design for the LST grants programme was conducted in a five week timeframe. Most important elements of grant programme design were attended to. But, in the time available before applications were sought, work was not undertaken in relation to neither the funding recipient reporting and acquittals processes, nor the development of an evaluation plan for the programme. In addition, the programme risk assessment was too optimistic.
2.4 The timeframe in which the LST grants programme was designed and delivered was proposed by the department. The four month timeframe11 AGD proposed was ambitious, but not impossible, given the small scale of the programme.
2.5 Programme design and development of the supporting documentation commenced on 17 November 2014, with an initial aim to open the funding round and call for applications the week commencing 22 December 2014 (illustrated by Figure 2.1). During this five week period AGD:
- sought and received policy approval for the grants programme;
- developed the programme guidelines (and had them approved) together with some supporting documentation12;
- conducted a risk assessment; and
- consulted with other government agencies13 and a key community stakeholder on the content of the guidelines.
Source: ANAO analysis of AGD records.
2.6 In the time available before applications were sought, work was not undertaken in relation to neither the funding recipient reporting and acquittals processes, nor the development of an evaluation plan for the programme.
2.7 A reporting and acquittals process was later developed, once the successful applicants had been decided. In relation to programme evaluation, AGD was unable to provide the ANAO with an evaluation plan. Rather it advised the ANAO that:
Evaluation of the intervention programmes is part of a broader body of work.
… Evaluation of the grants themselves is a much more discrete and time limited activity, which is being conducted through the quarterly questionnaire and follow-up, as well as confirmation at the end of the project that services are available to support intervention case management. This will be based on intervention with state based intervention coordinators.
It was originally thought that registration on the Directory would form part of the evaluation of the LST grants programme. However, with the re-scoping of the Directory, evaluation will now be based on an evaluation against the LST grant programme objective of assisting organisations to build their capacity to deliver services that will help deradicalise or divert individuals from radicalisation or violent extremism.
Risk assessment
2.8 AGD’s assessment of the LST grants programme and proposed guidelines resulted in a risk rating of low14 being agreed with the Departments of Finance (Finance) and the Prime Minister and Cabinet (PM&C).15 The LST guidelines were subsequently approved by the Attorney-General on this basis. Table 2.1 outlines a number of risks and respective mitigation strategies that formed part of the overall low rating.
Risk |
Controls and mitigation strategies |
Residual risk level |
Grants awarded to ineligible applicants. |
Programme guidelines explicitly state eligibility criteria. Assessment tool checks eligibility prior to assessment. |
Very low |
Grants awarded for projects or activities that are inconsistent with the objectives of the programme. |
Programme guidelines explicitly state programme objectives and assessment criteria—criterion 1 is weighted as the most important criterion. Internal panel to determine projects recommended for funding. |
Very low |
Applicants treated inequitably in the appraisal and awarding of grants. |
Peer review of all assessments. Overall recommendations reviewed by senior officers prior to recommendation to the decision maker. |
Very low |
Grants being awarded to applicants that may not be able to complete a project effectively. |
Referee checks. Assessment phase to include cross checks of each application recommended for funding. Funding agreement builds in reporting milestones to ensure progress is monitored (this could include teleconference progress reports). |
Low |
Grants funding is redirected by successful recipients toward extremist activity. |
Programme guidelines detail clear controls that will be applied during assessment of applications, e.g. security checks by intelligence and law enforcement agencies. Robust assessment process to ensure all organisations recommended for funding have security checks. All successful applicants will be required to have an ABN, business bank account and funding will be subject to progress and acquittal checks. |
Medium |
Source: ANAO analysis of AGD records.
2.9 As discussed further in Chapter 3, a number of the mitigation strategies identified by AGD were not implemented.
2.10 Notably absent from the list of mitigation strategies was the ability to make milestone-based payments to LST grants recipients. If adopted, the provision of funding by instalments following the completion of agreed milestones significantly increases the Commonwealth’s level of control over how grant funding is used. Following its review of the LST risk assessment in December 2014, PM&C drew to AGD’s attention that ‘[f]ull payment of funds upfront poses considerable risks of fraud and projects not being completed’. However, AGD decided to persist with its proposed approach of providing the successful applicants with upfront grant payments prior to 30 June 2015.16
Were appropriate programme guidelines developed?
In most respects, the programme guidelines provided a reasonable basis for the implementation of the programme. A key shortcoming was that the guidelines did not communicate to potential applicants the importance of the link between receiving grant funding and registering on the CVE Directory. Indications are that at least two, and as many as 26, of the funding recipients will not seek to be registered on the directory.
2.11 Central to the planning of a granting activity, and a key obligation under the Commonwealth Grants Rules and Guidelines, is the development of programme guidelines. The LST guidelines were succinct (five and a half pages, including the cover page). In the main, they provided a reasonable basis for the implementation of the programme. This included specifying:
- the purpose of the grants and a programme objective;
- the amount of funding that was available;
- eligibility requirements;
- the application process including the deadline for submitting applications;
- four merit criteria, and their associated weightings, along with an outline of the assessment process;
- funding terms and conditions; and
- departmental contact details for any inquiries or complaints.
Programme objective and link to the CVE Directory
2.12 The objective of the LST programme was to support community-based, non-government and local government organisations to develop new and innovative services to help individuals move away from violent extremism (either directly, or through their families and friends). At the time he was asked to approve funding for 34 applications, the Attorney-General was advised that, upon completion of their projects, it was expected that funding recipients would register for the CVE Directory that would be used by governments as a multi-list panel to connect in-need individuals with services.
2.13 Prior to submission of the LST guidelines to the Attorney-General on 15 December 2014 for approval, AGD sought input from Finance and PM&C on their content.17
2.14 Both Finance and PM&C recommended that the LST guidelines be amended in some key areas18 so as to provide greater clarity of the programme’s intent for potential applicants and more consistency between the guidelines’ sections. A key theme from the comments of both agencies related to making it clearer that the grant funding was intended to build or enhance organisations’ skills and capabilities, rather than for the delivery of services.
2.15 The recommendations made by Finance and PM&C were substantially incorporated by AGD into the guidelines. In doing so, a decision was taken to remove any reference within the LST guidelines to the CVE Directory, including the explanation of the relationship between it and the LST grants. This was done on the basis that keeping the two initiatives separate would create less confusion for potential applicants.
2.16 However, the section within the guidelines that referred to the CVE Directory—which was subsequently deleted—formed part of the LST programme objective. The removal of this component of the programme’s objective from the guidelines weakened the communication of the programme’s intent to potential applicants.
2.17 Notwithstanding the decision to remove the details about the directory from the LST guidelines, included in the LST ‘Frequently Asked Questions’ (FAQs) webpage was an explanation of the link between the two initiatives. Whilst FAQs are a useful resource for providing additional guidance to applicants, they should be consistent with and complementary to the overarching programme guidelines. Any inconsistency between programme documents can cause confusion for applicants and result in greater complexity in the administration of the programme. This was reflected in some potential applicants expressing to AGD confusion in respect to the objective of the programme, and questioning whether their time would be well spent on developing applications for LST funding.
2.18 The department’s April 2015 funding recommendation advice to the Attorney-General had outlined that, upon completion of their projects, funding recipients will be expected to register for the CVE Directory. Consistent with this expectation, the following statement was included in grant agreements:
On completion of your activity it is expected that your organisation will apply to register for the Commonwealth’s Directory of Countering Violent Extremism (CVE) Intervention Services. The Directory will be used by governments as a multi-list panel to connect individuals in need with services and programmes that will help them to disengage from extremism.
2.19 Following signing of the agreements, AGD designed a reporting questionnaire to be completed by grant recipients. It was provided to recipients for populating and returning to AGD in September 2015. It included asking whether recipients intended to apply in the future to be a part of the CVE Directory. In September 2015, of the 41 organisations that had entered into grant agreements, 13 recipients indicated that they were willing to be a part of the directory, but two have advised they were not and the intentions of a further 26 recipients is not yet known.19
3. Application process and eligibility checking
Areas examined
The ANAO examined whether the application and eligibility checking processes promoted transparent and equitable access to the available funding.
Conclusion
The Attorney-General’s Department (AGD) took a number of effective steps to make the programme accessible and equitable.
But the approach to administering programme eligibility was less than adequate. Firstly, those requirements were not clearly set out in the programme guidelines, and the department did not assess whether applications had met each of the eligibility requirements. Rather, each application that was assessed proceeded to the merit assessment stage, even in circumstances where it was evident the application was ineligible.
In addition, while 16 of the 17 late applications that were accepted for assessment were provided to assessors, one was not.
Areas for improvement
The ANAO has made a single recommendation, at the end of the report. The recommendation is that the department take effective steps to address continuing deficiencies in its approach to assessing the eligibility and merit of grant applications, and the resulting provision of advice to decision-makers about those applications that should be funded and those that should be rejected.
Was the application process accessible?
The department took a number of steps to make the programme accessible. This included promoting the funding opportunity in a number of ways in recognition that potential applicants would not be aware of the programme and the impending call for applications. But there were also some aspects of the department’s approach to administering the programme that were not consistent with accessibility and equity principles.
3.1 In late December 2014, AGD requested assistance from several Commonwealth20 and state and territory government agencies in disseminating information about the Living Safe Together (LST) grants and the Directory of Countering Violent Extremism (CVE) Intervention Services to their respective networks of community organisations. The department also contacted 1921 community organisations directly on 24 December 2014 to inform them of the upcoming opportunity for funding through the programme.
3.2 The LST programme funding round opened on 7 January 2015. It was scheduled to close at 5pm on 9 February 2015.
3.3 On 8 January 2015, AGD contacted the state and territory agencies again and also directly contacted 328 community organisations, advising that the LST round had opened.
3.4 On 6 February 2015, AGD wrote to these organisations again, informing them that the closing date had been extended by three weeks to 5pm on 2 March 2015. A new one-page factsheet explaining the relationship between the LST grants and the Directory of CVE Intervention Services (CVE Directory) was attached to this advice, and was also published on the AGD website.
Application requirements
3.5 Proportionality is a key consideration under the Commonwealth Grants Rules and Guidelines in the design of grants programmes. In this respect, it is expected that the amount of documentation and substantiating evidence requested from grants applicants be commensurate with the size and scale of the granting activity. Of note was that the extent of the required supporting documentation (in addition to a completed application form) was the inclusion of up to three letters of support for the proposed project.
3.6 By design, the letters of support were to be used within AGD’s assessments of merit criterion three (community need) to substantiate applicants’ claims. There was no required supporting documentation in respect to the other three criteria. Other information requested by the application form for the purposes of verifying applicants’ claims and conducting security checks was:
- the details of two referees;
- an indication of whether income sources were confirmed or not (by ticking a box on the application form);
- the organisation’s Australian Business Number (ABN);
- the organisation’s bank account details; and
- some personal information belonging to the authorised contact officers of the applicant organisation.
Community information sessions
3.7 AGD had received feedback that many organisations did not understand the work that was being implemented by AGD, nor how they could participate. In addition to developing and circulating the one-page information sheet, by 18 February 2015 AGD decided that it was also necessary to conduct three community information sessions to allow applicants to hear more about the programme and ask questions in relation to both it and the CVE Directory.
3.8 AGD invited 89 organisations and four Australian Government22 agencies to attend one of three information sessions that were conducted the week prior to the application closing date. Two sessions were conducted in Sydney on 23 February 2015, and one session in Melbourne on 26 February 2015.
3.9 An approach whereby a subset of potential applicants are invited to attend one of only a few community information sessions may create a real or perceived competitive advantage for those applicants that were invited to attend.23 Therefore it is important for administering departments to ensure that, if possible24, details of information sessions are made publicly available, preferably within the programme documentation.
Enquiries from potential applicants
3.10 Potential applicants were given access to departmental officers via email if they required assistance during the application process. Queries from potential applicants predominantly related to confusion as to:
- whether they or their projects were eligible or suitable under the LST programme; and
- the closing date for applications.
3.11 In this latter respect, from their publication on 7 January 2015, the programme guidelines had stated that the closing date was 16 February 2015, whereas the application form stated that it was 9 February 2015. Despite receiving several emails in relation to the issue, the inconsistency was not addressed until the documents were amended at the end of January 2015 to reflect the extended closing date of 2 March 2015.
3.12 Extensions to the closing date for submitting application forms and/or letters of support were requested from time to time. Initially, AGD advised that extensions would not be granted and that ‘applications and associated documentation received after the deadline will not be considered.’ However, at the community information sessions and on the closing date (2 March 2015), advice was given to prospective applicants that they could submit a ‘holding’ application form by the due date, and then submit a more complete version as soon as possible. There was a risk that such an approach may have disadvantaged some potential applicants.
3.13 Following the closing date, a decision was taken to accept 17 late applications.
Acceptance of late applications
3.14 The closing date for LST applications was provided in the programme guidelines. While the guidelines indicated that there was some scope for the submission of late applications25, AGD responded throughout January and February 2015 to requests for extensions by advising that late applications would not be considered. However, closer to and following the 2 March 2015 closing date AGD relaxed its position on accepting late applications.
3.15 On 3 March 2015, AGD decided to accept all 96 applications that it had received.26 Of these, 17 applications had been submitted after the deadline. In deciding to accept those applications, AGD did not consider whether the delays were ‘due to exceptional or unanticipated circumstances’.27 AGD accepted all 17 on the basis that they were submitted within an hour of the deadline or on the next day. Another shortcoming was that AGD unintentionally did not assess one of the late applications it had decided to accept, meaning this applicant was treated unfairly in comparison to other late applications.
3.16 Two additional late applications required AGD’s consideration, as follows:
- on 18 March 2015, the Department of Education requested that AGD accept a late application. AGD advised the Department of Education that while the project appeared to be worthwhile, it did not meet the programme guidelines. AGD suggested that the Department of Education contact the Department of Social Services to find support for the project; and
- on 13 April 2015, AGD decided to accept and assess an application that had been provided on the application form for the CVE Directory and submitted to AGD via the CVE Directory email address. This project was subsequently funded under the LST programme.
One of the accepted late applications was not assessed
3.17 In its 27 April 2015 briefing to the Attorney-General, AGD advised that 96 applications had been received. This advice was incorrect as it included only 16 of the 17 late applications the department had decided to accept. One of the late applications that the department had decided to accept had not been provided to the assessment team and, as a result, was not assessed.
Were the programme eligibility requirements clearly set out?
The eligibility criteria were predominantly set out under the ‘eligibility’ section of the guidelines but eligibility requirements were also included in other parts of the guidelines. Some other eligibility requirements were not reflected in the guidelines but in other, related, documentation. These approaches do not assist applicants, or make for an effective approach to the eligibility checking process.
3.18 As outlined in Table 2.1, AGD had contemplated and assessed the risk of grants being awarded to ineligible applicants. As part of this assessment, the department noted that the following were likely to be causes of this risk:
Programme guidelines lack clarity around eligibility criteria; application form does not seek the information required to determine eligibility; [and] programme staff do not correctly assess applicant eligibility.
3.19 To mitigate this risk, AGD recorded that it intended to ensure that the ‘programme guidelines explicitly state eligibility criteria’ and the ‘assessment tool check[s] eligibility before assessment’. Following the application of these mitigation strategies, AGD assessed the residual risk as ‘very low’.
3.20 The eligibility criteria were predominantly set out under the ‘eligibility’ section of the guidelines. But further eligibility requirements were included under various other headings within the guidelines or other supporting documents (examples are provided by Table 3.1).
Source document |
Section or page |
Requirement |
LST guidelines |
Eligibility |
The following are not eligible to apply:
|
Funding is not available for the following purposes:
|
||
Organisations must clearly demonstrate that their project will enhance their organisational capacity to contribute to encouraging individuals away from violent extremism. |
||
These grants are not for delivery of services or for broad based community-focused social cohesion initiatives. |
||
Funding |
Projects are required to be completed within one year of the funding agreement being executed. |
|
Application process |
Applications must be submitted by 5.00pm Australian Eastern Daylight Time on Monday 2 March 2015. |
|
Applicants must fully complete and submit the application form, including a budget. |
||
Selection process |
Applicants are required to nominate and provide the contact details of two referees. |
|
Conflict of interest |
Applicants should list any potential or existing conflicts of interest that may exclude them from being considered for public funding. |
|
Application form |
Page 1 |
Applications must be submitted by email to lstgrants@ag.gov.au. |
Page 6 |
You must attach the following supporting documentation to your application:
|
|
FAQs |
Applying for funding |
All applications must be submitted on the official application form available on the Living Safe Together Grants Programme page. |
About the Living Safe Together Grants Programme |
Applicants may not seek funding for large capital costs such as purchasing or leasing buildings or for purchasing motor vehicles. In addition, operational or administrative costs that are not directly related to the development of the proposal will not be considered. |
Source: ANAO analysis of AGD records.
Was the assessment of eligibility requirements effective?
The approach taken to assessing the range of eligibility requirements that had been set out in programme documentation was poorly planned and ineffective. Only a small number of those requirements were explicitly addressed in the assessment template with the result that ineligible applications proceeded to the merit assessment stage and, in a number of instances, were recommended and approved for funding.
3.21 The department’s Grants Administration Guide states that ‘once applications have been received; departmental staff must assess the compliance of each application with the … eligibility criteria stated in the grant guidelines.’ The assessment template that supports the guide includes a range of typical eligibility considerations. Similarly, the programme guidelines clearly stated that ‘ineligible applications will not be fully assessed’.
3.22 In December 2014, AGD commenced work on a ‘registration screening’ template. This template was intended to address eligibility.28 The template was not completed, nor was an alternative approach developed for consistently checking eligibility.29
3.23 Instead, a consolidated application assessment template was developed. This template was focussed on the applicant responses to the merit criteria, and analysis of organisational risk. It included very limited consideration of eligibility requirements set out in the programme guidelines. Specifically, the eligibility criteria that were addressed by the template and considered during the threshold assessment are identified in Table 3.2.
Section name |
Section instructions to assessors |
ABN |
From Question 1 on application form—Record Australian Business Number (ABN) and confirm it is correct—www.abr.business.gov.au. |
Eligibility |
Pass/Fail—ineligible if: an individual; Federal, State/Territory Agency; for profit or commercial organisation; political organisations; or no ABN. |
Source: AGD records.
3.24 The department’s assessment identified two applications (out of the three that should have been identified) as either ineligible or potentially ineligible on the basis of results from the ABN checks.30 One of these proceeded to be assessed against the first merit criterion, while the other was assessed against all merit criteria.31 The third application, which was assessed as eligible by AGD, provided an ABN with insufficient digits and ticked ‘individual’ as its legal entity type. It went on to be assessed against all merit criteria, but was not recommended for funding. As a result, there were no applications that were excluded from an assessment against criterion one on the basis of ineligibility.
3.25 The eligibility requirements not addressed by the assessment template related to:
- submitting a fully complete application form signed by a person with legal authority to represent the organisation;
- submitting an application for funding on the official LST application form and to the correct email address; and
- excluding provision of LST funding for:
- the delivery of services or for broad based community-focused social cohesion initiatives;
- academic research;
- political party campaign activities;
- projects involving capital works or the purchase of capital equipment valued at more than five per cent of the total amount of funding sought;
- projects involving high staffing costs where there is no contribution from the applicant or other source; and
- general operational or administrative costs of the applicant, or partner organisations.
3.26 On a few occasions assessors made observations or expressed concerns in respect to some of the above criteria in the assessment sheets, but it was more common for assessments to not address the full range of eligibility requirements. As a result, applications that should have been assessed as ineligible progressed to full merit assessment and funding consideration. Table 3.3 illustrates this situation.
Reason for ineligibility |
Number of applications (out of 96) |
Number assessed against all merit criteria (out of 83) |
Number recommended for LST funding (out of 42) |
|||
No letter of support |
15 |
16% |
14 |
17% |
3 |
7% |
No referee details |
5 |
5% |
5 |
6% |
2 |
5% |
‘High’ᵃ staffing costs and no applicant contribution |
8 |
8% |
7 |
8% |
4 |
10% |
Project duration greater than 12 months |
13 |
14% |
11 |
13% |
4 |
10% |
Social cohesion or ‘proposed services aimed at groups or communities more generally’ᵇ |
25 |
26% |
19 |
23% |
4ᶜ |
10% |
Note a: For the purpose of this analysis and in the absence of a definition of ‘high staffing costs’ in the programme guidelines, the ANAO identified all applications that contained budgeted staff-related costs equal to or exceeding 50 per cent of the total project cost.
Note b: As per the LST Frequently Asked Questions, which outlined that these services ‘will not be considered for funding’.
Note c: All four of these applications were approved as part of the second tranche of LST funding. Prior to deciding to recommend them as part of a second tranche, three were provided by AGD to the Department of Social Services as projects that would be suitable for funding under social cohesion programmes. The assessment for the remaining application noted that it could be ‘considered in the social cohesion policy space’, but it had failed to satisfactorily meet any of the merit criteria.
Source: ANAO analysis of AGD records.
3.27 Recognising that some applications failed to meet multiple eligibility requirements, the ANAO’s analysis was that 38 applications should have been assessed as ineligible32, but instead proceeded to be assessed against the merit criteria (12 of which were subsequently recommended for funding).
4. Merit assessment
Areas examined
The ANAO examined whether the assessment process identified the best applications in terms of the published criteria.
Conclusion
There were four merit criteria. The first, which was the key policy criterion, related to how grant funding would be used to build or enhance the applicant’s capacity to deliver services aligned to the policy objective.
The merit criteria were well designed and appropriate for the programme. The criteria weightings, combined with the requirement for applications to satisfactorily address and meet the key policy criterion in order to progress in the assessment process—that is, to go on to be assessed against the other three merit criteria—provided a strong underpinning to the conduct of the merit assessment process.
Applications were assessed through a sufficiently rigorous process. This included adopting a numerical rating scale, providing some guidance to assessors in the form of a scoring guide, and peer review/quality assurance processes (to promote a consistent high standard of assessment). Overall, the assessment process provided a sound basis for the department to identify those applications that should be recommended for funding, and those that should be rejected. It would have been strengthened by the department clearly setting out minimum scores for applications to achieve against each merit criterion in order to progress in the assessment process as a possible candidate for funding (as was previously recommended by the ANAO).
The results of the assessment work revealed that a significant proportion of the applications did not meet one or more of the published criteria. This included a high proportion (61 per cent) of applications not meeting the key policy criterion.
Area for improvement
The ANAO has made a single recommendation, at the end of the report. The recommendation is that the department take effective steps to address continuing deficiencies in its approach to assessing the eligibility and merit of grant applications, and the resulting provision of advice to decision-makers about those applications that should be funded and those that should be rejected.
Were the merit criteria appropriate?
The merit criteria were well designed and appropriate to the programme.
4.1 A grant programme’s merit criteria are expected to indicate the characteristics of individual proposals that will successfully contribute to achieving a programme’s objective. It is therefore reasonable that some criteria—usually those most aligned with the underlying policy intent of the programme—will be more important than others. The importance of each criterion is typically indicated by the weighting allocated to it. The four merit criteria established for the Living Safe Together (LST) programme (see Table 4.1) were appropriate, as were the associated weightings when combined with the clearly expressed requirement that only those applications that satisfactorily addressed and met the first criterion would be progressed in the assessment process. This gateway approach, given the importance of the first criterion (the key policy criterion) was an appropriate design feature, including by offering programme implementation efficiencies.
Programme objective |
|||
To support community-based, non-government and local government organisations to develop new and innovative services to help individuals move away from violent extremism (either directly, or through their families/friends). This could include services such as mentoring, counselling, education and employment services, and other diversionary programmes. The age of individuals most at risk of radicalisation is likely to range from early teens to mid-thirties. |
|||
Criteria |
|||
Criterion 1—Proposal |
Criterion 2—Governance Structure and Experience |
Criterion 3—Community Need |
Criterion 4—Potential Risks |
How does your organisation propose to use the funding to build or enhance its capability to deliver services aligned to the programme objective? Your proposal needs to include a budget detailing proposed expenses. |
What is your demonstrated experience in successfully developing and delivering activities that are the same or similar to your proposed project? Outline the structure of your business and what organisations you have collaborated or worked in partnership with in the past. In enhancing your organisational capacity would this involve collaborating with other organisations? |
What is the community need for the proposed service interventions you propose to develop within the geographical area you intend to deliver services in or on a broader regional or national level? [sic] What support do you have from communities for your proposal? Please provide up to three current letters of support with your application. |
What are the potential risks associated with your proposed project and how do you plan to mitigate against these risks? Describe your mitigation strategy. |
40% |
20% |
20% |
20% |
Source: LST programme guidelines (ANAO emphasis).
Was the assessment work conducted to a sufficiently high standard?
Applications were assessed through a sufficiently rigorous process.
4.2 The ANAO’s recent audit of the Safer Streets programme33 concluded that the merit assessment process for that programme had been handled particularly poorly. Of note was that it was common for the department to complete its assessment of applications without fully addressing each criterion, and without having obtained sufficient information from the applicant. Instead of pursuing the information that applicants had not provided or assessing the application as not satisfactorily meeting the relevant criterion, the department made generous assumptions about the quality of many of the proposals that had been submitted for assessment.
4.3 For the LST programme, the published guidelines and related documentation outlined an assessment process that included the following elements:
- an eligibility assessment;
- a threshold assessment—that is, assessing that proposals met the programme objective (criterion one);
- a competitive assessment against the remaining criteria (criteria two to four) for those proposals that satisfactorily addressed and met the first criterion;
- consideration of results from referee and security checks; and
- consideration of organisation, project and security risks.34
4.4 A team of six external contractors were engaged by the Attorney-General’s Department (AGD) for a three week period to assess applications. The assessment team received an induction session35 on the day of commencement (3 March 2015) and was provided with assessor induction kits, which included:
- the programme guidelines;
- an assessment template;
- a deed of confidentiality form;
- a conflict of interest declaration form; and
- a one-page information sheet explaining the relationship between the Directory of Countering Violent Extremism Intervention services (CVE Directory) and LST grants.
4.5 As part of the initial induction session, the contracted assessors received a policy briefing by two CVE Branch members. On 20 March 2015 (some two and a half weeks after the start of the assessments), the CVE Branch Head addressed the assessors after clarification was requested of the policy objective.
4.6 A numerical scoring guide (illustrated by Table 4.2) was developed, and verbal updates were provided to the contracted assessors. The department’s approach went some way towards addressing concerns raised by ANAO in 2014 during the audit of the Safer Streets programme (which were later reflected in the third recommendation made by the ANAO in its Audit Report No.41 2014–15). Specifically:
- it was consistent with the first part of that recommendation, which was that the department articulate benchmarks or standards to inform the judgment of assessors when considering the extent to which an application can reasonably be considered to have met the published merit criteria; but
- minimum scores were not set that an application must achieve in order to progress in the assessment. This meant that the second part of the recommendation was not implemented. This was particularly significant in respect to the first merit criterion (see further starting at paragraph 4.11).
4.7 AGD’s approach led to some inconsistencies in the interpretation of criteria by individual assessors. In turn, this meant that some applications had to be reassessed. To provide for a consistent, high standard of assessment work without the inefficiency of reassessing applications, it has become common for administering entities to develop an internal procedures manual and/or documented assessment methodology. This approach merits consideration by AGD.
Rating |
Score |
Descriptor |
Additional Information |
High |
10 |
Exceptional |
Exceptionally strong with essentially no weaknesses |
9 |
Outstanding |
Extremely strong with negligible weaknesses |
|
8 |
Excellent |
Very strong with only some minor weaknesses |
|
Medium |
7 |
Very Good |
Strong but with numerous minor weakness |
6 |
Good |
Strong but with at least one moderate weakness |
|
5 |
Satisfactory |
Some strengths but also some moderate weaknesses |
|
Low |
4 |
Fair |
Some strengths but at least one major weakness |
3 |
Marginal |
A few strengths and a few major weaknesses |
|
2 |
Poor |
Very few strengths and numerous major weaknesses |
|
1 |
Very Poor |
No strengths—unaligned with policy objective |
Source: AGD records.
4.8 The department used a staged assessment process. The initial assessment was completed by a contracted assessor. For quality assurance purposes, there was then a peer review of the assessment undertaken by another contracted assessor.36 For 75 applications (of the 83 that were assessed against all merit criteria), the peer review was followed by a moderation process, undertaken by an AGD officer. For 71 of those applications the moderator endorsed the outcome of the peer review process. The moderator decided that two applications that were not supported following the peer review should be reassessed, and that two applications that were supported following the peer review should not be recommended for funding.
4.9 Through this process, scores against each merit criterion were finalised so as to support decisions as to which applications the department would recommend for funding approval, and those that would not be supported.
4.10 AGD advised the ANAO during the audit that ‘qualitative’ assessment work was undertaken including by an SES panel in addition to the process described in paragraphs 4.8 and 4.9. But the department was unable to provide any evidence concerning the conduct of this work, although it was evident through the lack of alignment between the department’s funding recommendations and the recorded assessments against the published criteria that those assessments were not the only factors that were taken into account by the department in providing its recommendations. Most importantly, the LST programme guidelines did not outline that a further ‘qualitative assessment’ phase would be undertaken but, rather, required the department to comparatively rank eligible applications based on their scores against each criterion, and that the criteria assessments inform decisions about which applications represented value for money and supported the policy objective. The department did not do this.
How well did the applications address and meet the key policy criterion?
The approach taken to the first merit criterion was inconsistent with the programme guidelines. The guidelines stated that only applications that satisfactorily addressed and met the first criterion would proceed to a full assessment (and therefore were able to be recommended for funding approval). AGD allowed 46 of the 59 applications that did not satisfactorily meet that criterion to proceed to a full assessment. As a result, 4 of those 46 applicationsa that had been scored poorly against the key policy criterion were approved for funding in April 2015. It also meant that the department wasted resources conducting more merit assessments than would have been necessary had it adhered to the published programme guidelines.
Note a: A further six of those 46 applications were recommended to the Minister for Justice for funding as part of the second tranche of approvals in May 2015.
4.11 The first merit criterion was crucially important to the programme achieving its policy objectives. Reflecting this, the programme guidelines outlined that:
- it was weighted as twice as important as each of the other three criteria (a weighting of 40 per cent compared with 20 per cent); and
- it was also a threshold criterion, such that only those proposals that ‘satisfactorily address and meet this criterion will be fully assessed against the remaining criteria’.
4.12 A key element in determining whether a proposal ‘meets’ an assessment criterion is the establishment of a benchmark, or minimum score. AGD did not establish a minimum score for the first merit criterion. AGD progressed a total of 83 applications to full merit assessment stage, but it was not evident from the assessment scoring that each of these had satisfactorily addressed and met the criterion.
4.13 The scoring guide (see Table 4.2) indicated that achieving half of the maximum score possible against a criterion would be considered ‘satisfactory’. But 59 applications (or 61 per cent) failed to achieve scores of 50 per cent or more against criterion one. Of those 59 applications, 46 progressed to be fully assessed against the remaining merit criteria. Four of these were recommended to and approved by the Attorney-General for funding.37
4.14 In December 2015, AGD advised the ANAO that in order to progress to full merit assessment, the approach it adopted was that applications needed to achieve minimum scores of 12 out of 30 against criterion one38 (a ‘fair’ score in terms of the scoring methodology, meaning the application had at least one major weakness in terms of the criterion). However, the ANAO’s analysis was that this is not the approach that was implemented:
- there were 15 applications that scored between 12 and 14 out of 30 against criterion one, but two of these applications did not progress to be assessed against the remaining criteria;39 and
- 33 of the 83 applications that proceeded to a full merit assessment had achieved scores of 11 or below for criterion one.
4.15 Adopting the approach that AGD advised it had taken should have resulted in 52 applications proceeding to full merit assessment. This situation is reflected in Figure 4.1.
4.16 In September 2015 AGD updated its assessment template and, in March 2016, advised the ANAO that in the future scoring against selection criteria will require a minimum score to be achieved.
Note a: AGD dated the reassessment for this application as having occurred on 6 July 2015, and the peer review was dated 2 November 2015.
Source: ANAO analysis of AGD records.
What did the assessment results reveal about the overall quality of the field of applications?
The department’s assessment of applications identified that the majority of applications lacked sufficient merit in terms of the published programme guidelines.
4.17 The distribution of scores awarded by AGD across the merit criteria is illustrated by Figure 4.2. It shows that most applications were assessed to be not of particularly high quality in terms of the published criteria. Of note was that only 25 of the 37 applications that achieved a score of at least 50 per cent against criterion one achieved scores of greater than 50 per cent across each of the other three criteria. Those 25 applications had sought $1.2 million in grant funding.
Source: ANAO analysis of AGD records.
4.18 The majority of applications were assessed to have not satisfactorily met one or more of the published criteria (see Table 4.3). This included 75 per cent of assessed applications not meeting at least three of the criteria. This comprised:
- twenty-eight (of the 83 that proceeded to full merit assessment) that were assessed as not satisfactorily meeting any of the four criteria, of which AGD recommended that two be awarded funding40; and
- forty-four that were assessed as not satisfactorily meeting three of the four criteria, with AGD recommending that one of these be awarded funding.
Criterion |
Number of applications recommended for funding |
Number of applications not recommended for funding |
|||
|
Met the criterion |
Did not meet the criterion |
Met the criterion |
Did not meet the criterion |
Not assessed against criterion |
1 |
30 |
4 |
7 |
55 |
0 |
2 |
31 |
3 |
18 |
31 |
13 |
3 |
27 |
7 |
10 |
39 |
13 |
4 |
26 |
8 |
16 |
33 |
13 |
Source: ANAO analysis of AGD records.
Were there any significant departures from the published assessment process?
There were a number of departures from the published programme guidelines. This involved steps outlined in the guidelines not being undertaken, as well as an additional, unpublished, assessment process employed.
4.19 As outlined, applications that had not met the programme eligibility requirements were not excluded from consideration. Nor were many of the applications that had not satisfactorily addressed and met the first merit criterion.
4.20 Another aspect of the published assessment process where departures were identified related to the conduct of referee and security checks. The guidelines stated that these checks would be conducted on all organisations recommended for funding, to ‘confirm that organisations have not been implicated in illegal actions such as providing any kind of support, including financial, to terrorist organisations, advocating the use of violence for political means, and other unlawful activities.’ But:
- the department did not follow up referees who were unable to answer the initial phone call. The department also decided not to contact some referees. ANAO analysis of the documentation available identified that only 23 of the 34 applicants expected to be recommended for funding (as at 21 April 2015)41 had at least one of their referees contacted by the department, with no referees ultimately contacted for the other 11 applicants; and
- twenty-one applicants did not provide letters of support, including four that were awarded funding. This situation is at odds with the statement in the guidelines that applicants were to provide ‘up to three current letters of support’, as well as February 2016 advice to the ANAO from the department that ‘the primary reference checks were the written letters of support that most applicants submitted’.
4.21 There were also key departures in relation to the published merit assessment process. In particular, despite statements to the contrary in the published guidelines:
- forty-six applications that had been scored as not satisfactorily addressing and meeting the first merit criterion were nevertheless assessed against the remaining criteria (see Figure 4.1);
- applications were not ranked comparatively against each other (see paragraph 5.1); and
- eight initially unsuccessful applications were reconsidered for funding by relying on a clause that permitted the department to ‘commission proposals outside the funding round’, but in only one instance (see paragraph 5.30) did the department seek a revised application for reassessment purposes (rather than reassessing the existing application).
5. Funding recommendations and decisions
Areas examined
The ANAO examined whether applications were ranked in priority order reflecting the results of the assessment process. The ANAO also examined whether relevant Ministers were appropriately briefed on the assessment results. In addition, the funding decisions taken were analysed in terms of transparency and consistency with the published programme guidelines.
Conclusion
There was a lack of alignment between the results of the merit assessment scoring process and the department’s funding recommendations. In particular:
- applications were progressed in the assessment process despite being identified as not having satisfactorily met the key policy criterion. The programme guidelines had stated that these types of applications would not be further assessed, but they were;
- applications were not ranked, despite the guidelines stating that those that had met the first criterion would be allocated a score against each criterion and then ranked comparatively against each other; and
- a significant proportion of the recommended applications had been scored by the department as not satisfactorily meeting one or more of the published criteria.
Funding was approved in two tranches. Neither briefing adequately outlined the assessed merit of the applications that were recommended for funding. In addition:
- the first briefing recommending that 34 projects be awarded $1.6 million in grant funding incorrectly stated that the projects recommended for funding had satisfactorily addressed all of the selection criteria aligned to the policy objective; and
- the second briefing recommending that the Minister for Justice approve $365 122 in grant funding to eight projects provided no advice as to how the applications had performed against the published criteria. The Commonwealth Grant Rules and Guidelines require that this advice be provided.
There was also a lack of transparency around how the eight applications approved in the second tranche were identified and reconsidered for funding (an opportunity not afforded to all unsuccessful applicants, including some that had scored more highly during the original assessment). Seven of the eight were recommended on the basis of their original applications that had been previously ruled out because they were assessed as either: ineligible; not satisfactorily addressing the policy objectives; or not representing value for money.
Overall, of the 42 recommended and approved applications, only 21 should have been successful (involving funding of $1 million) had the department only recommended eligible applications that it had scored as satisfactorily meeting each of the published merit criteria.
Areas for improvement
The ANAO has made a single recommendation, at the end of this chapter. The recommendation is that the department take effective steps to address continuing deficiencies in its approach to assessing the eligibility and merit of grant applications, and the resulting provision of advice to decision-makers about those applications that should be funded and those that should be rejected.
Did the assessment process rank in priority order the applications that had been assessed as meritorious against the published criteria?
Applications were not ranked in priority order. Whilst a numerical rating scale was used during the assessment process, the scores resulting from the assessment process were not used as the basis for identifying the applications that were recommended for funding.
5.1 For 83 applications, the Attorney-General’s Department (AGD) awarded a score against each of the four published criteria. But the scores were not used to rank applications. This was despite the guidelines stating that those that had met the first criterion would be allocated a score against each criterion and then ranked comparatively against each other.
Was the Attorney-General appropriately briefed when he approved funding for the first tranche of 34 projects?
The department provided a written funding recommendation to the Attorney-General in April 2015 in a briefing that contained most of the information that is required toa be provided to Ministers when approving grants. However, the extent of the accuracy and comprehensiveness of the application-specific information provided meant that AGD fell short of fully meeting these requirements.
Basic project information was provided for the 34 applications that AGD recommended for funding, as well as advice that all 34 had satisfactorily addressed all the selection criteria. Based on applications’ final assessment scores, this advice was incorrect for 12 of these applications.
Also provided was a list of the organisations whose applications were not recommended. Notably absent from this list was advice as to whether or to what extent each individual application had been assessed and scored as meeting the merit criteria.
Note a: Department of Finance, Commonwealth Grants Rules and Guidelines, July 2014, p. 11.
5.2 On 27 April 2015, AGD provided the Attorney-General with its funding recommendation. AGD advised the Attorney-General to:
- note that the pool of funding could be increased from $1 million to approximately $1.6 million; and
- approve 34 applications for funding of up to $1.6 million.
5.3 Other documentation attached to the briefing was an outline of the application and assessment process, basic information on each project, draft letters to the successful applicants and a copy of the programme guidelines.
Details on the merit of individual applications
5.4 The briefing stated that all the recommended applications had satisfactorily met all the assessment criteria. But no details were provided in the briefing on the outcomes of individual applications’ assessments. The application-specific information that was provided was set out within two separate lists. One related to the 34 applications that AGD recommended for funding, and the other contained details for the 62 that were not recommended.
5.5 The extent of the details provided for each of the recommended applications was the:
- name of applicant organisation;
- state or territory in which the project would be delivered;
- activity description; and
- maximum funding amount recommended.
5.6 Similar details were provided for the 62 proposals that were not recommended. The exception was that the activity description paragraph was substituted for a sentence describing the ‘activity theme’. AGD advised that 40 of these had not been recommended for funding due to one or more of the following reasons:
- ineligibility;
- did not satisfactorily address the policy objectives and selection criteria; or
- did not represent value for money and were not competitive against competing applications.
5.7 The remaining 22 applications that were not recommended were arranged into one of the following categories:
- ‘projects with merit for Social Cohesion type programmes’ (seven applications); or
- ‘encourage organisations to apply for AGD Counter Narrative Programme’ (15 applications).
5.8 Notably absent from the advice were the applications’ assessment results or an indication of the assessed degree of merit represented by each in the context of the Living Safe Together (LST) programme guidelines (as the applications were not ranked). The omission of this information meant the Attorney-General was not informed of their relative merits, nor able to distinguish between the competing applications. Therefore, should he have decided (for example) to restrict the total amount of funding to the $1 million the programme guidelines had stated was available for awarding, he would have had difficulty doing so.
Characteristics of the projects recommended for funding approval
5.9 AGD’s briefing stated that:
The applications were assessed against the eligibility and selection criteria detailed in the LST Programme Guidelines. … The projects recommended for funding have satisfactorily addressed all the selection criteria aligned to the policy objective and therefore offer value for money.
5.10 This advice was inaccurate. Specifically:
- as outlined in Chapter 3, applications had not been assessed against each of the eligibility requirements set out in the programme guidelines; and
- not all of the projects recommended for funding had been assessed and scored as satisfactorily meeting all the selection criteria.
5.11 Awarding funding to applications assessed as not satisfactorily meeting the published selection criteria is an approach that is unlikely to provide a value for money outcome. Recommending that funding be approved for such applications is also inconsistent with a competitive, merit-based selection process. This is particularly the case when other applications that have been assessed and scored as meeting the criteria are not recommended for funding.
5.12 AGD recommended that funding be approved for 22 of the 25 applications assessed to have satisfactorily met each of the four published criteria. The advice to the Attorney-General did not identify, or address, why the remaining three applications that had been assessed to satisfactorily meet each of the published criteria were not recommended for funding. In relation to those three applications, the ANAO’s analysis was that:
- one was an intermediary project rather than for the development of services, which was submitted by an individual (and should have been, but was not, assessed as ineligible). The only assessment moderation comment attached to the assessment was that it was ‘Not Supported—after discussion with Branch Head and members of CVE team’; and
- the other two were placed onto the not recommended for funding ‘projects with merit as social cohesion initiatives’ list. Both were approved for funding as part of the second tranche although neither had resubmitted their application, and both had identical project budgets between their application and the grant agreement (the only change was some adjustment to the wording of the project objectives).42
5.13 There were also 12 applications that were recommended for funding notwithstanding that they had been assessed to not satisfactorily meet at least one of the published criteria.
5.14 The lack of alignment between the assessment scoring (as documented at the time the assessment of the 96 applications was conducted and when recommendations were provided to the Attorney-General), and the department’s funding recommendations are illustrated by Figure 5.1.
Note a: Application ineligible.
Note b: Application met all criteria.
Note c: Application met criterion one, but failed to meet one or more remaining criteria.
Note d: Application failed criterion one; should not have proceeded to further assessment.
Note e: Scores for six of the additional eight applications were later increased after 30 June 2015.
Source: ANAO analysis of AGD records.
Was an appropriate process used to identify which unsuccessful applications should be considered for funding in the second tranche?
The process through which AGD identified that eight of the 62 unsuccessful applications would be reconsidered for funding was not sufficiently transparent to be able to demonstrate an equitable approach had been employed. In addition, reassessments against the merit criteria were not documented until well after the eight applications had been approved for funding by the Minister for Justice and grant funding paid.
5.15 Following the Attorney-General’s approval of 34 applications under the LST programme, the Minister for Justice assumed ministerial responsibility for the CVE Programme. This occurred on 28 May 2015 when he gained an additional role as the Minister Assisting the Prime Minister on Counter-Terrorism.
5.16 The LST programme guidelines included a provision allowing for the commission of proposals if the programme was undersubscribed, or if additional funding became available. The guidelines did not require that an open call for applications be made, but outlined that the same criteria would be applied to any commissioned proposals as were to be applied to applications that had been received in response to the open call for applications that was made in January 2015, as follows:
Commissioned proposals will need to meet the programme objective and will be assessed against the eligibility and the selection criteria.
5.17 Relying on that provision of the guidelines, AGD advised the Minister for Justice on 28 May 2015 that it had reprioritised further funding43 and was therefore seeking further funding approvals under the LST programme. Specifically, approval was requested to directly approach44 and fund eight applicants that were considered to be significant stakeholders, but had not been successful in the first funding approval.
5.18 In terms of how those eight applications had been identified, AGD advised the ANAO in December 2015 that it revisited all 62 applications that were not recommended as part of the first funding approval, and from this process:
eight applications that had previously been assessed as not meeting the programme objective were identified as being either a significant stakeholder or were now considered to meet the programme objective and if necessary, would be requested to resubmit their applications.
5.19 Consistent with AGD’s advice, the Minister for Justice approved funding for the eight applications on 9 June 2015. Immediately thereafter, AGD informed all eight organisations that they had been successful in securing LST grants.
5.20 Departmental documentation indicates that the eight applications had been identified by 21 May 2015. Notably, this documentation also indicated that four of the eight applications required some form of amendment in order to meet the LST guidelines before they could be funded.
5.21 AGD advised the ANAO in May 2016 that: ‘(t)his information was included in the “Activity” section of each grant agreement’. But the ANAO’s analysis of the applications, assessments and grant agreements for these applications was that the project descriptions within the application forms and the activity descriptions within the grant agreements were largely the same.
5.22 In March 2016, AGD advised the ANAO on the ‘iterative process’ it said it applied in identifying these applications, but there was a paucity of records to identify the basis on which it was decided why those applications would be progressed45, and the other 54 unsuccessful applications would not.
5.23 During the earlier process that identified the first 34 applications that were approved for funding, the following assessments were made in respect to the eight applications funded under the second tranche:
- six were scored as not satisfactorily addressing criterion one (with scores ranging between four and 12 out of 30), with:
- only one of these not proceeding to full merit assessment against all four criteria; and
- of the five that were fully assessed, four were scored as failing to meet any of the criteria, while one met one criterion; and
- both applications that were scored as satisfactorily meeting criterion one went on to be assessed as meeting the requirements of all of the remaining criteria.46
5.24 Leaving aside these two latter applications, there were 19 applications that had been scored (during the original assessment) more highly than the remaining six that were recommended and approved as part of the second tranche.
Reassessment of eight identified applications
5.25 Departmental records included documented reassessments of each of the eight applications. AGD dated those reassessments as having been conducted on 25 or 26 May 2015 for seven applications and 17 June 2015 for the remaining application. But the ANAO’s analysis of the available evidence was that the reassessment records for the eight applications were not prepared until early July 2015. Notably, this was after:
- AGD had sought and received additional funding approvals for the eight applications from the Minister for Justice;
- grant agreements had been entered into with all47 successful LST recipients; and
- funding had been paid in full to these recipients.
5.26 Figure 5.2 illustrates the timing of these activities.
Note a: Seven of these eight were peer reviewed within the same period. The peer review comments for the remaining application were not recorded and filed until 2 November 2015.
Source: ANAO analysis of AGD records.
5.27 In March 2016, AGD advised the ANAO that:
As a result of competing priorities within the grants management team (in the last month of the financial year), the formalised assessments and peer review comments were recorded in early July 2015. AGD acknowledges that these should have been back-captured at the time of the assessment process in May 2015 and that one set of peer review comments was inadvertently overlooked and not back-captured until early November 2015.
5.28 The ANAO’s analysis of the available records (including those from the department’s email system) did not lend support to this perspective. In addition, AGD was not able to provide the ANAO with any evidence of the reassessments being conducted prior to the recommendations, approval and payments for the respective recipients. The ANAO also raised with AGD the related issue of the briefing recommendation making no reference to whether the eight applications had been assessed as meeting the criteria, but the department advised the ANAO that it had ‘inadvertently’ not provided such advice.
Results from the reassessment
5.29 Seven of the reassessments were based on the original applications that had been submitted in March 2015, using the same assessment template and scoring guide from the first assessment process. Of note is that one of these seven applications had not originally progressed to a full merit assessment, and was therefore not peer reviewed in the first assessment process. During the reassessment process it was assessed against all four merit criteria, but not peer reviewed until 2 November 2015.
5.30 Another was reassessed on the basis of a resubmitted application that was received (in response to AGD’s invitation and instructions48) in mid-June 2015. The overall results of these reassessments are illustrated by Figure 5.3.
5.31 For two applications, the recorded scores did not change as a result of the reassessment process. New scores were allocated to the remaining six.49 Of these:
- five applications went from not satisfactorily addressing the key policy criterion to either meeting or exceeding it; and
- four applications’ original total scores increased by more than 50 per cent.50
5.32 In March 2016, AGD advised the ANAO that:
the reassessment process established the links between these projects (or elements thereof) and the objectives and outcomes of the LST grants guidelines. As we’ve previously discussed, AGD’s thinking and understanding of CVE was evolving rapidly and some grant applications that were first considered not to meet the LST grant guidelines were subsequently found to, based on further understanding and experience.
5.33 Of particular note is that—despite being resubmitted—one application remained assessed as still not satisfactorily addressing any of the merit criteria. Additionally, a further:
- two applications remained assessed as not meeting two of the merit criteria; and
- one application remained assessed as not meeting one of the merit criteria.
Note a: Resubmitted application.
Note b: Dotted outlines represent the applications that had been identified by AGD on 21 May 2015 as requiring amendments in order to meet the LST programme guidelines (paragraph 5.20 refers).
Source: ANAO analysis of AGD records.
Was the Minister provided with sufficient information and an order of merit to enable effective discrimination of competing applications?
The written funding recommendation provided to the Minister for Justice in relation to the award of funding to a further eight applications was deficient in that it provided no advice on how the eight applications has been selected, nor their assessed performance against the published criteria.
5.34 On 28 May 2015, AGD provided the Minister for Justice with its recommendations for a second tranche of LST funding. AGD advised the Minister to:
- note that AGD had redirected resources to address critical priorities within the CVE Programme; and
- approve eight more LST applications for funding of up to $365 122, subject to security checks of the organisations.
5.35 On the basis of this briefing and some further background information provided by AGD51, the Minister for Justice approved funding of $365 122 for an additional eight organisations under the LST programme on 9 June 2015.
5.36 The briefing provided to the Minister included the same information in respect to the initial assessments and recommendations as that given to the Attorney-General (a copy of the April 2015 briefing formed an attachment to the May 2015 briefing).
5.37 But, unlike the April 2015 briefing to the Attorney-General, AGD did not advise the Minister for Justice as to how the eight applications had been assessed to have performed against the published criteria.52 Accordingly, the briefing did not meet the minimum standards required by the Commonwealth Grant Rules and Guidelines. In March 2016, AGD advised the ANAO that not providing the advice required by the Commonwealth Grant Rules and Guidelines was ‘inadvertent’.
5.38 The briefing also did not outline how the eight applications had been selected from within the 62 unsuccessful applications from the first tranche.
Recommendation No.1
5.39 The ANAO recommends that the Attorney-General’s Department take effective steps to provide greater accountability for, and effectively address, continuing deficiencies in its approach to:
- assessing the eligibility and merit of grant applications; and
- advising decision-makers about those applications that should be funded and those that should be rejected.
Attorney-General’s Department response: Agreed.
5.40 AGD agrees with the ANAO’s recommendation that it take effective steps to provide greater accountability for, and effectively address, continuing deficiencies in its approach to assessing the eligibility and merit of grant applications and advising decision-makers about those applications that should be funded and those that should be rejected.
Appendices
Appendix 1: Attorney-General’s Department response
Footnotes
1 The change in approver reflected a change of ministerial responsibilities.
2 ANAO Audit Report No.41 2014–15, The Award of Funding under the Safer Streets Programme, was tabled on 4 June 2015, and was examined in JCPAA Report 452, tabled on 7 December 2015. On 20 March 2015, during the assessment of LST applications, AGD’s Grant Policy Unit advised the CVE Branch Head of the Safer Streets audit report (which had not yet been tabled) and that the ANAO report of the Safer Streets programme was ‘critical of many of the processes and/or procedures that were followed’ and identified the senior executives within AGD that could provide the CVE Branch with ‘further insights’.
3 AGD advised the ANAO in December 2015 that the CVE Directory was ‘being re-scoped as a broader mapping exercise…’
4 There were ultimately 41 grant agreements that were entered into. This was due to one applicant declining the LST funding offer. AGD received written advice to this effect from the applicant on 26 June 2015.
5 Forming part of the Australian Government’s $630 million Counter-Terrorism package.
6 The change in approver reflected a change of ministerial responsibilities.
7 ANAO Audit Report No.41 2014–15, The Award of Funding under the Safer Streets Programme.
8 The Directory of CVE Intervention Services is a Commonwealth Government procurement panel, listed on the AusTender website.
9 AGD advised the ANAO in December 2015 that the CVE Directory was ‘being re-scoped as a broader mapping exercise…’ AGD provided an ad hoc grant to an external organisation to lead this work, with the initial results of the mapping exercise due to be delivered by 30 June 2016. However, the originally scoped CVE Directory remains open for new applications on AGD’s website (as at May 2016), with the respective Multi-use List on the AusTender website also being reported as ‘current’ (and closing in January 2018).
10 AGD developed a suite of tailored documentation for the CVE Directory, including an application form and guidelines for applicants.
11 This timeframe is from the commencement of the development of the programme guidelines through to and including the provision of funding recommendations to the Attorney-General (which were initially advised would be provided in March 2015).
12 Including a Frequently Asked Questions document, fact sheets, an application form and assessment templates.
13 These were the: Department of Finance; Department of the Prime Minister and Cabinet; and Australian Federal Police.
14 This was agreed on the basis of a total funding envelope of $500 000 for the LST grants. This amount was subsequently doubled to $1 million following approval by the Attorney-General and prior to the publication of the guidelines. The amount actually awarded was $1 925 661.
15 The Commonwealth Grants Rules and Guidelines require a risk assessment of the granting activities and associated guidelines be completed in consultation with the Department of Finance and the Department of the Prime Minister and Cabinet.
16 This was despite receiving advice that at least $300 000 of the funding that had been redirected for the grants programme was able to be rolled over to the 2015–16 financial year.
17 As is required under the Commonwealth Grants Rules and Guidelines. Further consultation to test the readability and appropriateness of the guidelines was initiated by AGD with the Australian Federal Police and a ‘key community stakeholder’.
18 Namely the programme objective, eligibility and selection criteria sections.
19 There was one applicant that had not submitted a completed questionnaire on the basis that AGD had provided them an exemption from reporting in the first quarter.
20 Including: the Department of Immigration and Border Protection; Department of Social Services; and the Australian Federal Police.
21 These contacts were referred to as a ‘sub-set of AGD’s community contacts’, on the basis that they were key stakeholders.
22 These were the Australian Federal Police; Department of Immigration and Border Protection; Department of Social Services; and Department of Human Services.
23 For example, the available evidence indicates that attendees invited to the information sessions were informed that they were able to submit a ‘holding application’ (discussed at paragraph 3.12), advice that was not otherwise published by AGD.
24 In some instances, such as with the LST grants programme, there may be valid reasons for not publicly announcing these events. In these cases, it is a reasonable expectation that invitations to the information sessions be distributed as widely as the initial announcement of the LST grants programme (which was 328 organisations, as opposed to 89).
25 Namely where the delay was ‘due to exceptional or unanticipated circumstances’.
26 The 97th application was not accepted by the CVE team until 13 April 2015, as outlined in paragraph 3.16.
27 As it stated it would do within the LST programme guidelines.
28 The checklist included consideration of the organisations’ legal status, outstanding acquittals, alignment with criterion one, whether the funding amount requested was within the funding threshold, and the proposed duration of the project.
29 This was in contrast to the assessment process employed for the related CVE Directory, which included ‘initial screening’ questions to consider some eligibility criteria.
30 Of these two applicants, one was assessed as ineligible on the basis of being an ‘individual/sole trader’ and the other had listed an ABN that was registered in another name.
31 This application was subsequently supported by AGD for funding, but removed from the assessment process at the security check stage and was not recommended to the Attorney-General for funding.
32 Applications were identified by the ANAO as ineligible on the basis that at least one of the following was true: applicant was an individual; referee details omitted; no letters of support; project duration exceeded 12 months; budget contains ineligible costs; and substantial details omitted from the application form.
33 ANAO Audit Report No.41 2014–15, The Award of Funding under the Safer Streets Programme, p. 19.
34 The security risk assessment was based on advice provided by Australian Government security and law enforcement agencies. This advice was only obtained in respect to applications that AGD proposed to recommend to the Attorney-General for funding.
35 AGD advised the ANAO that the induction session involved:
[a] policy overview by two CVE members; assessors taken through key documents by project manager, i.e. LST programme guidelines; [conflict of interest] and deed of confidentially; assessment template and scoring sheet; linkage between LST Directory and LST Grants Programme; and three completed applications to begin assessment.
36 For one application—which was subsequently awarded funding—there was no evidence of a peer review.
37 A further six were recommended for approval as part of the second tranche of funding. All six were approved by the Minister for Justice, but one subsequently withdrew their application. This is reflected in Figure 4.1.
38 Criterion one consisted of three elements, or sub-criteria, worth up to 10 points each.
39 One of these two applications was reconsidered in April 2015 and subsequently recommended to and approved by the Minister for Justice for funding.
40 A further six of these were recommended for funding as part of a second tranche of approvals under the LST Programme in May 2015. These approvals are analysed in Chapter 5.
41 Two of these applicants were ultimately not recommended for funding.
42 See further at paragraph 5.23.
43 AGD had already reprioritised funding on two prior occasions for the LST programme. This included $500 000 to establish the programme in November 2014 and an additional $550 000 in December 2014 to increase the funding pool to $1 million prior to publishing the programme guidelines.
44 AGD also noted within its advice that the Attorney-General had been made aware in April 2015 that AGD planned to take this action.
45 Two of the eight applicants had complained about the original outcome, and for another one representations had been made by a state government department (on behalf of the only organisation that was later provided with the opportunity to resubmit its application). AGD advised the ANAO that ‘many organisations were disappointed about not receiving LST grant funding, and expressed their disappointment to AGD’ with some, but not all, of those being included in the eight that were reconsidered for funding. AGD further advised the ANAO that no applications were reconsidered for funding because they had complained.
46 As outlined at paragraph 5.12, the ANAO’s analysis had raised questions about why these two applications had not been, on the basis of their assessed scores, included in the original tranche of applications recommended to the Attorney-General for funding.
47 There were ultimately 41 grant agreements that were entered into. This was due to one of the eight applicants from the second tranche declining the LST funding offer. AGD received written advice to this effect from the applicant on 26 June 2015.
48 AGD invited this organisation to resubmit an amended application after advising that it had been successful in securing a grant as part of the second tranche of funding. AGD provided the organisation with a copy of its previous application and a number of suggested improvements to be incorporated into its next submission. AGD copied into this correspondence the state government department that had made representations on the applicant’s behalf, as well as offering to provide it with assistance in redeveloping its application.
49 This includes the application that was resubmitted and reassessed in June 2015 following the Minister’s approval.
50 For one of these applications, the substantial increase in its total score was due to it being assessed against only one of the four merit criteria in its original assessment. Once reassessed, the application was scored against all four criteria.
51 Prior to approving AGD’s recommendation, the Minister requested that AGD provide additional information in respect to the eight applications. AGD provided this information on 2 June 2015; however it did not include advice as to how the applications had performed against the merit criteria. Rather, it was further background information on the applicant organisations and their projects.
52 As noted at paragraph 5.33, the documented reassessments outlined that four of the eight applications had not met one or more of the criteria.