Introduction

1. Literacy and numeracy are foundations on which further learning is built. Achieving appropriate literacy and numeracy skills affects an individual’s success in school and throughout life. The Department of Education, Employment and Workplace Relations (DEEWR) defines literacy and numeracy as follows:

Literacy is the ability to read, write, speak and listen to languages in a way that allows people to communicate with each other and make sense of the world. Numeracy helps [people] use mathematics effectively to meet the general demands of day-to-day life at home, at work and in society generally.¹

2. While Australian students are achieving good results in literacy and numeracy overall, a significant number are still failing to achieve minimum standards, particularly those from low socio-economic status (low-SES) communities and Indigenous students.² In 2008, 19.6 per cent of Australian students were at or below the National Minimum Standard (NMS) in reading, and 18.7 per cent were at or below the NMS in numeracy.³ International data also shows that although Australian students ranked highly in literacy and numeracy skills compared to the rest of the world, Australian testing results have declined in reading (2000–2009) and mathematics (2003–2009).⁴

National Partnership Agreement on Literacy and Numeracy

3. The National Partnership Agreement on Literacy and Numeracy (Literacy and Numeracy National Partnership or LNNP) aims to apply the collective resources and efforts of the Australian Government, state and territory governments (states) and education sectors, to put in place the infrastructure and practices that will deliver sustained improvement in literacy and numeracy outcomes for all students, especially those who are falling behind. The LNNP also aims to accelerate progress towards the literacy and numeracy target set by the Council of Australian Governments (COAG) to halve the gap between Indigenous and non-Indigenous students’ achievement in reading, writing and numeracy within a decade.⁵

4. Commencing in 2009, the LNNP was one of the first National Partnerships operating under the new Intergovernmental Agreement on Federal Financial Relations (IGA FFR). It was also one of the first National Partnerships to include reward payments to states. The IGA FFR sets out broad principles for the operation of National Partnerships, including: a focus on achieving outcomes in a cooperative spirit between governments; the Australian Government making payments for progress toward outcomes and outputs (rather than payments for inputs), thereby reducing prescriptions on service delivery by the states; and improved accountability of governments through simpler, standardised and more transparent public performance reporting.

5. The LNNP commits $540 million of Australian Government funding from 2008–09 to 2011–12. This funding is comprised of:

• $150 million⁶ in payments to states to support reform activities that aim to improve literacy and numeracy results (‘facilitation payments’) over the first two years of the LNNP (2009 and 2010);
• up to $350 million in payments to states to reward reform based on the achievement of agreed literacy and numeracy performance targets (‘reward payments’) over the last two years of the LNNP (2011 and 2012); and
• $40 million for research initiatives targeted at improving teaching capacity in literacy and numeracy.⁷

6. The LNNP also required that states match the Australian Government facilitation investment. The combined investment was to be focused on agreed reform activities to maximise impact, and develop a comprehensive understanding of literacy and numeracy initiatives that work in a variety of school settings and student cohorts.⁸

7. The broader objectives of the LNNP include: increasing collaboration between the government and non-government education sectors in achieving literacy and numeracy reform; and identifying and implementing evidence-based interventions which achieve accelerated and sustained improvements in literacy and numeracy outcomes for students.

8. To give effect to the LNNP, DEEWR coordinated and negotiated the development of bilateral agreements and implementation plans by respective state agencies, for consideration and agreement by the Australian Government and state education ministers. The agreements and plans outline the schools selected for participation in the LNNP by state agencies and Block Grant Authorities⁹ (collectively referred to as education authorities), the agreed reform strategies for these schools, and the required reporting arrangements. Approximately 1050 government and non-government schools, or approximately 10 per cent of all Australian schools, have received assistance under the LNNP.

9. DEEWR and the respective state agencies negotiated reform targets as part of the development of bilateral agreements and implementation plans, as the basis for reward payments. The reform targets were to represent ambitious, accelerated improvements in literacy and numeracy in the participating schools, based on National Assessment Program – Literacy and Numeracy (NAPLAN)¹⁰ data and optional local measures of performance.¹¹

10. Under the LNNP, the Australian Government facilitation payments were to be made to states in accordance with implementation milestones identified in bilateral agreements, with the first facilitation payment to be made on signing of the bilateral agreement. Reward payments were to be made to states following achievement of the performance (reform) targets for 2010 and 2011 identified in bilateral agreements and implementation plans, as assessed by the COAG Reform Council.¹²

11. DEEWR, on behalf of the Australian Government, had primary administrative responsibility for the LNNP with its role encompassing: planning and managing administrative arrangements for the LNNP; progressing Australian Government outputs, including a shared database of effective literacy and numeracy strategies known as the ‘Evidence Base’; and monitoring and reporting on the delivery of LNNP outcomes.

Audit objectives, criteria and scope

12. The audit objective was to assess the effectiveness of DEEWR’s administration of the National Partnership Agreement on Literacy and Numeracy (LNNP). The three high-level audit criteria used to form a conclusion examined the extent to which DEEWR:

• established sound administrative and payment arrangements consistent with government policy, including through its negotiation of bilateral agreements, implementation plans and reform targets;
• properly managed administrative and payment arrangements; and
• effectively monitored and reported on delivery and outcomes.

13. The audit report also includes analysis of changes in NAPLAN test results for participating schools, and presents case studies of literacy and numeracy initiatives. More broadly, the report examines national trends in literacy and numeracy performance between 2008 and 2011, since the commencement of the LNNP.

14. The Australian National Audit Office (ANAO) consulted education authorities in four states and other relevant stakeholders, including the Australian Council for Educational Research (ACER)¹³, the Australian Curriculum, Assessment and Reporting Authority (ACARA), the COAG Reform Council, the Treasury and several schools that received LNNP funding, to obtain their feedback on the administration and progress of the LNNP. Feedback from the stakeholders is included throughout the report, where appropriate. The audit covers the LNNP’s operation from the time that the LNNP was signed in 2009 to March 2012.

Overall conclusion

15. Literacy and numeracy are the foundations on which success in further learning and life are built. Australia generally continues to perform well in international literacy and numeracy testing. However, the nation’s ranking has declined over the past decade and students from low-SES and Indigenous backgrounds remain behind the rest of the population. In this context, the Australian Government announced in 2008 a $540 million commitment to the National Partnership Agreement on Literacy and Numeracy (LNNP)—an agreement between governments ‘to put in place the infrastructure and practices that will deliver sustained improvement in literacy and numeracy outcomes for all students, especially those who are falling behind’.¹⁴

16. The LNNP was one of first National Partnerships to include reward payments to states for achievement of reform targets. It envisaged ambitious, accelerated improvements in students’ literacy and numeracy outcomes in participating schools during the partnership timeframe, as the basis for making reward payments. The LNNP also included initiatives to disseminate effective practices which would support system-wide improvements in educational attainment. Within the framework established by the LNNP, DEEWR had responsibility on behalf of the Australian Government for negotiating specific administration arrangements with state agencies, monitoring their progress and sharing effective literacy and numeracy strategies, to help achieve the intended outcomes of the LNNP. As National Partnerships were still in their formative stages, DEEWR did not have access to formal guidance when developing the LNNP framework.

17. Australian Government expenditure to date under the LNNP is $322 million in facilitation, reward¹⁵ and research initiatives funding, with $212 million in reward payments¹⁶ remaining accessible to states. Through the LNNP, education authorities and schools have implemented a range of initiatives in the agreed reform areas of school leadership, quality teaching, and the effective use of student performance data, aimed at improving the literacy and numeracy outcomes for participating students. Common examples of initiatives include the engagement of literacy and numeracy coaches, and delivering professional development for teachers. Education authorities and schools consulted during the audit reported a range of positive impacts of the LNNP on schools, teachers and students, including through increased collaboration between government and non-government education sectors. However, ANAO analysis of NAPLAN data from 2008 to 2011 indicates that the LNNP is yet to make a statistically significant improvement, in any state, on the average NAPLAN results of schools that received LNNP funding, when compared to schools that did not receive funding.¹⁷ In this respect, for some states there were less than three months and 15 months respectively, between the commencement of LNNP initiatives and NAPLAN testing for 2010 and 2011. More broadly, national literacy and numeracy achievement has mostly been stable since the LNNP was introduced, including for low-performing and Indigenous students. In this light, it may take several years until a reliable assessment of the impact of the LNNP on literacy and numeracy outcomes can be made.

18. Overall, the effectiveness of DEEWR’s administration of the LNNP has been mixed. This assessment is made in the context of National Partnerships being a new form of program delivery and the LNNP being one of the first National Partnerships to include reward payments to states. DEEWR worked collaboratively with state agencies to establish administrative and payment arrangements for the LNNP within a reasonable timeframe, and has mostly managed the administrative and payment arrangements in accordance with the requirements established by the LNNP. However, the department did not apply a structured approach to negotiating key implementation arrangements, specifically the number of participant schools, performance indicators, and the reform targets for 2010 and 2011 which were the basis for making reward payments.¹⁸ Consequently, there was significant variability at a state level in the coverage of the LNNP and performance indicators used, and reward targets were not necessarily demanding.¹⁹ In this respect, DEEWR could have more actively pursued the outcomes sought by governments in developing the LNNP framework.

19. LNNP initiatives were undertaken in approximately 10 per cent of Australian schools as chosen by education authorities, and cover approximately 10 per cent of Australian students in Years 3, 5, 7 and 9. While LNNP funding was notionally allocated based on each state’s proportion of low-performing students, the estimated proportion of students targeted varied from only 3.7 per cent of students in New South Wales to almost one-quarter (24.3 per cent) of students in Western Australia. Inconsistencies in the coverage of the LNNP and the level of targeted improvement potentially disadvantaged those states that, in the spirit of the LNNP, had aimed for more challenging targets. In addition, given the proportion of students involved in the LNNP, expecting significant changes across the broader student population was also ambitious and difficult to achieve. While the coverage of the LNNP remained similar, as part of ongoing administrative improvements, DEEWR revised reform targets for 2011 in response to recommendations in the COAG Reform Council’s March 2011 performance report on the LNNP, through a better designed and more transparent process.

20. In addition, DEEWR did not progress the ‘Evidence Base’ of effective literacy and numeracy strategies as promptly as envisaged by the LNNP.²⁰ Implementation of the Evidence Base within a shorter timeframe would have provided education authorities and schools with additional guidance for planning and implementing effective LNNP-funded literacy and numeracy initiatives during the National Partnership. It would also have supported broader take-up of successful literacy and numeracy strategies.

21. DEEWR‘s approach to ongoing monitoring of the LNNP involves making an assessment of states’ annual and progress reports, and provision of advice to the Minister on LNNP performance. Evaluation of the LNNP, which is in its initial phases, will be crucial to assessing the longer-term impact of the National Partnership and in optimising learning about effective literacy and numeracy strategies. To evaluate impacts, DEEWR will need to analyse changes in NAPLAN results for participating schools and their different literacy and numeracy initiatives following the conclusion of the LNNP in 2012.

22. The ANAO has made two recommendations to strengthen program and payment design for future National Partnerships, and to better assess the impact of the LNNP on literacy and numeracy outcomes for participating schools following the conclusion of the LNNP. The implementation experience of the LNNP also gives rise to a number of broader lessons for responsible agencies in establishing National Partnership payment arrangements. These include that the design of National Partnerships should allow sufficient time for initiatives funded using facilitation payments to significantly influence performance results, prior to measuring achievement against reform targets and making reward payments. The design of implementation and payment arrangements should also provide the best opportunity for achievement of intended outcomes, such as by linking accessible reward funding with the coverage of planned initiatives and targeted levels of improvement. Such an approach would be consistent with the intent of governments that National Partnerships apply ‘good payment design’ that gives states an ‘incentive to invest appropriately to achieve agreed performance benchmarks or milestones, and trigger associated Australian Government payment’.²¹ Lessons for the administration of future National Partnerships, arising from the audit, are at Appendix 2.

Key findings

Developing the LNNP Framework

23. Sound implementation planning is an important element in the successful delivery of government policies.²² Following agreement to the LNNP by the Australian Government and states, DEEWR had responsibility (on behalf of the then Minister for Education) for developing appropriate administrative arrangements for its implementation. In this respect, DEEWR established sound relationship management mechanisms through staff assigned as contact points for each state, and formal LNNP multilateral governance arrangements which operate to enable relationships to be managed at a national level.

24. DEEWR was also responsible for negotiating bilateral agreements and implementation plans with respective state agencies, to support implementation, ongoing oversight and public accountability of LNNP-funded activities. In the absence of formal guidance, DEEWR sought clarification as appropriate from central agencies²³ within the Australian Government, as part of negotiating and finalising the bilateral agreements with each state between January 2009 and February 2010. All agreements were signed within 13 months of the signing of the LNNP. The timeframe for development of the bilateral agreements and implementation plans was reasonable in the context of, a four-year agreement, and the level of information and planning contained in the agreements. However, the LNNP requirements were not universally addressed in these documents.²⁴ Only two out of eight states included their specific reform targets in their publicly available bilateral agreements and/or implementation plans. The absence of reform targets in publicly available documents does not provide a point of comparison for interested stakeholders to assess performance by each state against expectations.²⁵ This was even more important given that the achievement of agreed reform targets was designed to be the trigger for the Australian Government to make reward payments. For five states, reform targets were settled separate to and after finalisation of bilateral agreements and implementation plans. As a result, prior to making the first round of reward payments in June 2011, a formal exchange of letters between education ministers had to occur to agree reform targets in order to make reward payments.

25. The timeframe to demonstrate improvements in literacy and numeracy as the basis for reward payments was a decision made as part of the LNNP, and was agreed to by all parties, including the Australian Government, in January 2009. However, after the finalisation of the bilateral agreements and implementation plans there was limited time (for some states less than three months) between securing agreement to proposed literacy and numeracy strategies, implementing the strategies, and students being tested to assess the effectiveness of the strategies. Given the short timeframe between the implementation of LNNP strategies and NAPLAN testing, any improvements measured for the first round of reward payments were unlikely to be significantly influenced by LNNP activities. Moreover, the LNNP NAPLAN targets did not necessarily represent ambitious targets for all states, particularly given the significant variability in the proportion of targeted students, the absence of a framework to set initial targets and a rigorous approach to assess ambition.

Ongoing Administration of the LNNP

26. Under the LNNP, DEEWR, on behalf of the Australian Government, has responsibility for advising its Minister and the Treasury on making facilitation and reward payments. LNNP facilitation payments were to be made on the basis of achieving milestones. Reward payments were to be made to the states based on their extent of achievement of reform targets, as assessed by the COAG Reform Council. The first tranche of facilitation payments occurred prior to the signing of the bilateral agreements. DEEWR advised that payments were made at this time so the implementation of the LNNP was not further delayed. Additionally, there were no explicit records made of the approval of spending proposals as required by FMA Regulations 9 and 12 for LNNP facilitation payments. Nonetheless, payments made under the LNNP were soundly based.

27. In the LNNP, co-investment from states is a requirement for receipt of facilitation and reward payments. The LNNP also required that bilateral agreements set out monitoring and reporting arrangements for state co-investments. However, states’ compliance with co-investment obligations was not monitored by DEEWR, and therefore was not covered in DEEWR’s payment advice and certifications. Further, acquittal data on actual LNNP co-investments, reported by states to the Treasury, but not to DEEWR, showed that co-investment obligations were not met by some states.²⁶ Where co-investment forms part of the requirements of a National Partnership, but is not being made in accordance with the agreement, there is an increased risk that the intended outcomes of the National Partnership will not be achieved. The reported co-investment shortfalls highlight the importance of the responsible department monitoring co-investments where such obligations are included in National Partnerships.

28. Under the LNNP, DEEWR was responsible for delivering two outputs which were partly designed to provide a legacy for the LNNP beyond its conclusion in 2012. Primary among these is the Framework for Effective Practice or ‘Evidence Base’ for the LNNP, which DEEWR expects to launch in the June quarter 2012, three years later than envisaged by the LNNP. The Evidence Base will be accessed through the Teach, Learn, Share website and should provide an important mechanism to strengthen the promulgation of information on effective practice in literacy and numeracy. DEEWR also allocated LNNP research initiatives funding to a range of projects, strategies and awards, although it could have adopted assessment criteria for funding proposals to help ensure funded initiatives were complementary and clearly aligned to the overarching objectives of the LNNP.

29. As part of its ongoing administration of the LNNP, DEEWR had responsibility for renegotiating reform targets for 2011. This was in response to the COAG Reform Council’s National Partnership Agreement on Literacy and Numeracy: Performance Report for 2010, which highlighted a number of concerns with the initial target setting and made recommendations to improve the 2011 targets. Stakeholders consulted by the ANAO who were involved in the initial negotiation of reform targets also raised concerns about variation in the level of ambition of reform targets due to the bilateral approach taken in the negotiation. DEEWR encouraged and worked with the states to revise the 2011 targets and improve the target-setting process. As a result, the process for establishing performance indicators and negotiating reform targets was better designed and more transparent than the initial target setting.

30. The renegotiation of 2011 reform targets was lengthy, with negotiations being finalised while some states had access to the actual 2011 NAPLAN results.²⁷ In the context of a four-year National Partnership and the complex nature of negotiations, the length of negotiations highlights the importance of setting appropriate performance indicators for targets at the outset of National Partnerships, including reaching broad agreement on the measurement approaches to be used. This was complicated in the case of the LNNP because for key NAPLAN performance data there was only one year of baseline data (and no trends) when reform targets were originally negotiated.

Monitoring, Reporting and Outcomes

31. Performance monitoring and reporting are an important element of accountability for government service delivery, and help inform service delivery improvements. DEEWR monitored the states’ compliance with bilateral agreements and implementation plans by reviewing states’ annual and progress reports. However, as mentioned previously, there was a gap in the monitoring of co-investment data.

32. DEEWR was also not in a position to be able to verify the accuracy of the performance results against reform targets, provided by states, as part of the performance assessment process. For future National Partnerships that use reporting from a national dataset as the basis of reward payments, administering agencies would benefit from working with states to coordinate the preparation of performance results and to consider related assurance processes. This would increase the likelihood that a consistent approach is taken to the calculation of performance results while reducing the risk of inaccurate reward payments. Additionally, such an approach offers the potential for efficiencies to the states in the preparation of their results.

33. The LNNP is one of three Smarter Schools National Partnerships for which DEEWR has commenced a national evaluation. The first phase of the evaluation was an analysis of reform activity and state evaluation efforts undertaken for each of the National Partnerships. The evaluation noted the considerable work being undertaken in schools to improve classroom practice in literacy and numeracy, and create learning environments within which students will have greater opportunity for success. However, given the complexities in measuring the effectiveness of reform activities, it may take several years until a reliable assessment of the LNNP approach can be made. DEEWR had not yet determined the scope of subsequent phases of the evaluation. To properly position the evaluation to assess the impact of the LNNP and different literacy and numeracy strategies, it will be important that subsequent phases analyse the literacy and numeracy outcomes for participating schools at an appropriate time following the conclusion of the LNNP. This would also assist create and sustain the legacy of the LNNP through dissemination of better practices.

34. For schools participating in the LNNP, education authorities and schools have reported positive impacts of the LNNP on school leadership, teacher practice and student engagement. However, ANAO analysis of NAPLAN data from 2008 to 2011 indicates that the LNNP is yet to make a statistically significant improvement, in any state, on the average NAPLAN results of schools that received LNNP funding, when compared to schools that did not receive funding. Among other things, the LNNP aims to accelerate progress towards the ambitious literacy and numeracy target set by the Council of Australian Governments to halve the gap between Indigenous and non-Indigenous students’ achievement in reading, writing and numeracy within a decade. In 2008, there was a significant gap between the proportion of Indigenous and non-Indigenous students at or above the NMS for reading and numeracy in Years 3, 5, 7 and 9, as measured by NAPLAN. In 2011, there continued to be a significant gap. These findings underline the importance of ongoing analysis of NAPLAN data for LNNP participating schools and groups targeted for assistance.

Summary of agency responses

35. DEEWR provided the following summary response to the audit report:

The Department of Education, Employment and Workplace Relations is committed to the sound administration of National Partnerships. The Auditor-General's report acknowledges the significant achievement of the Smarter Schools National Partnership for Literacy and Numeracy (LNNP), given that it was amongst the earliest partnerships and one of the first reward-based national partnerships to be implemented under the new COAG arrangements.

The Department accepts the recommendations in the report, and will work with relevant agencies to clarify the roles and responsibilities of all Commonwealth agencies involved in the administration of national partnerships.

The report highlights the significant challenges in enacting Government policy whilst adequately reconciling Commonwealth accountability and avoiding prescription in service delivery.

The LNNP contributes to literacy and numeracy outcomes of the National Education Agreement, which are themselves broad outcomes with a number of causative elements. The LNNP's contribution to the outcomes of this Agreement is specifically aimed at supporting students falling behind, especially Indigenous students.

In relation to this targeted cohort, findings to date show that some school-level improvements have been made in increasing the proportion of students achieving above national minimum standards. Specifically, from 2008 to 2011, in year 3 reading and year 5 numeracy, 70 per cent and 80 per cent respectively, of LNNP schools improved the proportion of students above national minimum standards.

To fully measure the impact of the investment in the national partnership, a broad analysis of all four outcomes of the LNNP would be required, including recognition of the different starting points in state literacy and numeracy achievement, their reform goals and the impact of different strategies and local measures adopted within each state. The Department will continue to monitor and evaluate the impact of national partnerships.

36. DEEWR agreed with the two recommendations in this report.

Footnotes

[1]   Department of Education, Employment and Workplace Relations (DEEWR), Your child’s future – Literacy and Numeracy in Australia’s schools [Internet], available at: <http://www.dest.gov.au/sectors/school_education/publications_resources/summaries_brochures/your_childs_future.htm> [accessed February 2012].

[2]   DEEWR, Importance of Literacy and Numeracy [Internet], available at: <http://www.deewr.gov.au/Schooling/LiteracyandNumeracyPilotProjects/Pages/LandNPilots.aspx> [accessed February 2012].

[3]   ANAO analysis of 2008 National Assessment Program – Literacy and Numeracy (NAPLAN) data.

[4]   Council of Australian Governments (COAG) Reform Council, Education 2010: Comparing performance across Australia, 2011, p. 27. The comparison is based on Performance for International Student Assessment (PISA) test results. PISA is an Organisation for Economic Co-operation and Development international study that aims to evaluate education systems worldwide by testing the skills and knowledge of 15 year-old students in member and non-member countries.

[5]   Council of Australian Governments, National Partnership Agreement on Literacy and Numeracy, January 2009, paragraph 4.

[6]   Following the agreement of the LNNP, $11 million was re-allocated from research initiatives to fund the participation of an additional 110 schools across Australia. This took the total funding for facilitation payments to $161 million.

[7]   This includes $13 million for the Australian Curriculum Assessment and Reporting Authority (ACARA), which is the independent authority responsible for the development of a national curriculum, a national assessment program and a national data collection and reporting program that supports 21st century learning for all Australian students.

[8]   Council of Australian Governments, National Partnership Agreement on Literacy and Numeracy, January 2009, paragraph 52.

[9]   Block Grant Authorities are bodies that represent non-government schools in the states and territories for funding purposes. There are 14 Block Grant Authorities, one for each of the two territories representing both the Catholic and independent sectors, and two in each state (one for Catholic schools and another for independent schools).

[10]   In 2008 NAPLAN commenced in Australian schools. Every year, all students in Years 3, 5, 7 and 9 are assessed on the same days using national tests in Reading, Writing, Language Conventions (Spelling, Grammar and Punctuation) and Numeracy.

[11]   The LNNP required that reform targets were to be ambitious and aim for accelerated improvement for schools, schools communities and students involved in the LNNP, including specific targets for Indigenous students, and also reflect the different starting points in each state.

[12]   Council of Australian Governments, National Partnership Agreement on Literacy and Numeracy, January 2009, paragraphs 65 and 66. The COAG Reform Council has the role of assessing and publicly reporting the achievement of agreed performance benchmarks before Australian Government reward payments are made to states under the LNNP.

[13]   The Australian Council for Education Research (ACER) was commissioned by DEEWR to provide technical advice on proposed measures of improvement and reform targets for each state under the partnership.

[14]   Council of Australian Governments, National Partnership Agreement on Literacy and Numeracy, January 2009, paragraph 4.

[15]   The Australian Government made reported reward payments totaling $138.5 million to states in June 2011 (out of $175 million in available reward funding for the period), based on the COAG Reform Council’s assessment of the extent to which states achieved reform targets for 2010.

[16]   This funding is subject to states’ achievement against 2011 reform targets for literacy and numeracy achievement in participating schools, as assessed by the COAG Reform Council.

[17]   In its response to the draft audit report, DEEWR noted some school level improvements in reading and numeracy in LNNP schools. Similarly, in its 2011 performance report on the LNNP, the COAG Reform Council noted that over the four years of the LNNP, students in participating schools generally improved their NAPLAN results in reading and numeracy. The ANAO analysis was different in that it compared the performance of LNNP schools with non-LNNP schools. The analysis examined the impact of a range of variables on changes in NAPLAN results, including the average base-year (2008) NAPLAN score; the school’s sector and geo-location; and whether the school year cohort was targeted by the LNNP. After allowing for the impact of baseline NAPLAN scores, the LNNP did not result in a statistically significant improvement in average NAPLAN scores between 2008 and 2011.

[18]   In its 2010 performance report on the LNNP, the COAG Reform Council noted the level of variation at a state level in: the proportion of participating schools and students; the criteria for selecting participating schools; the domains, year levels, size of student cohort, student characteristics and sectors selected for measurement; targets and the methodologies for establishing baselines; starting points of the performance of participating schools; the size of expected change over time; and the number of targets. Source: The COAG Reform Council, National Partnership Agreement on Literacy and Numeracy: Performance Report for 2010, 25 March 2011, p. xv.

[19]   The LNNP provided the states with flexibility to negotiate literacy and numeracy strategies and reform targets suitable to their particular circumstances. However, it also required that: agreed reform targets would be ambitious and aim for accelerated improvement for the schools/school communities and students involved; and that there would be consistency where possible in the measurement of improvement. Source: Council of Australian Governments, National Partnership Agreement on Literacy and Numeracy, January 2009, paragraphs 34, 37 and 38. In its 2010 performance report on the LNNP the COAG Reform Council noted the strong references to ambition in both the IGA FFR and the LNNP, and the apparent variation in the level of ambition of state agreed reform targets. For example, three states had one or more targets that were lower than their baselines; and some reform targets were agreed based on maintaining existing achievement or small improvements. As targets involved different calculations and trends to determine improvement, it was difficult for the COAG Reform Council to determine whether targets were actual accelerated improvements. Source: The COAG Reform Council, National Partnership Agreement on Literacy and Numeracy: Performance Report for 2010, 25 March 2011, pp. 89–90.

[20]   DEEWR did, however, provide a forum for sharing information about effective literacy and numeracy strategies through the November 2011 National Literacy and Numeracy Partnership Forum.

[21]   The Treasury, Federal Finances Circular No.2011/02, Developing National Partnerships under the Federal Financial Relations Framework, 9 December 2011, p. 29.

[22]   Department of the Prime Minister and Cabinet, Implementation Plan Guidelines [Internet], available at: <http://www.dpmc.gov.au/implementation/implementation_guide.cfm> [accessed February 2012].

[23]   Central agencies are: the Department of the Prime Minister and Cabinet, the Department of Finance and Deregulation and the Treasury.

[24]   Requirements not met pertained to some LNNP governance, funding, monitoring and reporting arrangements.

[25]   The complete set of 2010 reform targets was first published by the COAG Reform Council in its March 2011 performance report on the LNNP.

[26]   After all facilitation payments and the first round of reward payments were made, Queensland, South Australia, Tasmania, and the Australian Capital Territory advised the Treasury that they would be revising reported LNNP co-investments upwards in their 2011–12 acquittal report.

[27]   DEEWR sought to address this risk by requiring states to submit revised 2011 reform targets before they had access to their 2011 NAPLAN results, and by only allowing states to revise these targets upwards in ongoing negotiations once actual NAPLAN results were known. DEEWR also advised that ACER had assessed all states’ revised 2011 reform targets as sufficiently ambitious prior to their accessing 2011 NAPLAN data.