The National Health and Medical Research Council

1. The National Health and Medical Research Council (NHMRC) is a statutory agency within the Health and Ageing portfolio, with a total annual budget of around $1 billion. The agency, which has existed in various forms since 1936, is widely regarded as one of Australia's peak bodies in the area of evidence-based health advice, and is a significant provider of grants to support health and medical research in Australia.

2. Over the years, NHMRC grants have contributed to progress in many areas of health and medical science, from advancing knowledge and treatment of cancer to preventing cardiovascular disease and improving the health of Aboriginal and Torres Strait Islander Australians.1 NHMRC investment in health and medical research, on behalf of the Australian Government, is estimated at 16 per cent of the total national investment by both public and private sectors.2 In 2008, the NHMRC administered 3843 new and continuing grants, accounting for $595 million in expenditure.

3. The grants are a vital source of income for many health and medical researchers. Individual researchers can apply to the NHMRC via their universities or research organisations for grants to cover research projects or multi-component research programs, salaries and infrastructure support. The grant process is highly competitive, with less than 30 per cent of applications receiving funding each year.

4. Grant applications are assessed on the basis of scientific merit through a process of peer review and expert panels—the objective being to select the highest calibre research for funding. This selection process relies heavily on the participation of accomplished researchers, who themselves may be contenders or recipients of NHMRC grants. The integrity of the selection process is therefore fundamentally important, as it underpins the advice that the NHMRC provides to the Minister for Health and Ageing for approval of the grants with the highest potential to deliver beneficial outcomes for Australia.

Changes to the NHMRC since 2006

5. In July 2006, the NHMRC became a statutory agency with responsibilities specified under the 2006 amended National Health and Medical Research Act 1992 (NHMRC Act). The NHMRC Act defines the NHMRC as the Chief Executive Officer (CEO), the Council and its committees and the staff of the NHMRC. The NHMRC is also a prescribed agency under the Financial Management and Accountability Act 1997 and the Public Service Act 1999.

6. Since 2006, the NHMRC has experienced a period of transition, facing several challenges as it separates its administrative functions from the Department of Health and Ageing (DoHA) and adjusts its governance and administrative arrangements to support its legislative responsibilities and core business—particularly grant administration. The agency has also had a substantial change agenda, particularly in developing new IT systems to improve its data capacity and grant management functions.

Increased funding for NHMRC grant programs 2000–08

7. Funding for NHMRC grants is administered through a special account, the Medical Research Endowment Account (MREA), established under section 49 of the NHMRC Act. From 2000 to 2008, a series of government initiatives to bolster Australia's research capacity resulted in more than a three-fold rise in the NHMRC's grant budget and a corresponding two-fold increase in active (new and continuing) grants. Over this period, the NHMRC awarded more than eight thousand grants, an investment in research exceeding $3.2 billion.3

The NHMRC grant process

8. Each year, the NHMRC invites researchers in eligible Australian universities and research organisations to apply for funding through its range of scholarships and research programs. Grant programs generally fall into three groups based on the intended use (or type) of the grant: Research Support; Infrastructure Support; and People Support. In 2007–08 funding for Research Support was $440 million, with the largest scheme, Project Grants, accounting for $283 million of this amount (Table 1).

Table 1: NHMRC grants programs and funding 2006 to 2008*

Source: Data sourced from <http://www.nhmrc.gov.au/grants/dataset/_files/grantdata.xls> [accessed 16 February 2009]. Note: * 2006 and 2007 are actual expenditure incurred in the funded year; 2008 is expected commitments. A more comprehensive table is provided in Chapter 2 of the report.

9. NHMRC grant programs are based on a competitive selection process. Grant applications are reviewed and ranked by a process of peer review, using external assessors and expert Grant Review Panels (GRPs), with a view to selecting research of the highest calibre for funding.

10. The key steps in the grant process are shown in Figure 1.

Figure 1: Key steps in the NHMRC's grants process

Source: Compiled by the ANAO from NHMRC information on the 2008 Project Grant round. (AIs = Administering Institutions; GRPs=Grant Review Panels; CEO = Chief Executive Officer).

11. The NHMRC also calls each year for academics to participate as assessors and members of the GRPs. To comply with the NHMRC's policies and guidelines, and prior to accepting grants for review, these individuals are required to declare any conflicts of interest that could affect their impartiality in assessing and selecting grants.

12. In 2008, the NHMRC received over 2586 applications for Project Grants—the largest NHMRC grant scheme. For this scheme, over 449 assessors, 42 GRPs and 499 GRP members were involved in the grant selection process. Based on the selection process, advice is provided to the Minister for Health and Ageing, who has responsibility for the final approval of grants for funding. Success rates vary between the different schemes.

Deed of Agreement with Administering Institutions

13. Administering Institutions (mainly universities) play an important role in the NHMRC's grant process, by acting as a conduit for grant enquiries, submission of applications and post-award management of grants.

14. Under NHMRC policy, only approved Administering Institutions may receive NHMRC grants. Each Administering Institution must sign a Deed of Agreement (the Deed) with the NHMRC, which establishes the parameters and expectations for the management of grant funds, accountability and reporting requirements. Under this arrangement, each Administering Institution has responsibility for the effective management of the NHMRC research projects and associated grant funds provided by the Commonwealth. An important role for the NHMRC is in managing the relationship with the Administering Institutions to achieve effective and accountable administration of grants.

Previous audit coverage

15. A previous ANAO audit, Audit Report No. 29 2003–04, Governance of the National Health and Medical Research Council, examined the governance of the NHMRC and made six recommendations.4 Subsequent to that audit report, the accountability and governance arrangements of the NHMRC were amended (post Uhrig Review) to reflect a whole of government shift to improved governance and accountability.5 The NHMRC's revised governance arrangements are examined in the current audit, in the context of grant administration.

The audit objective

16. The audit objective was to form an opinion on the effectiveness of the NHMRC's grant administration. To meet this objective the NHMRC was assessed against four criteria:

• the NHMRC's governance arrangements provide appropriate accountability that it is meeting its objectives and obligations to Government (Chapter 2);
• there are strategic and systematic processes for developing and implementing grant programs (Chapter 3);
• the NHMRC manages grants post-award effectively, and complies with legislative requirements and program directives (Chapters 4 and 5); and
• the NHMRC monitors and evaluates its business to demonstrate that outcomes are being met (Chapter 6).

Overall conclusion

17. The National Health and Medical Research Council (NHMRC) has a key role in providing grants to support health and medical research in Australia. NHMRC grants are an important source of income for many health and medical researchers, and constitute a substantial Government investment in research and innovation in Australia. Over the period 2000 to 2008, Government initiatives to strengthen Australia's research capability resulted in more than a three-fold increase in NHMRC grant funding, with a corresponding two-fold rise in the number of grants. The NHMRC's investment in research during this time exceeded $3 billion.

18. Against this background, since 2006 the NHMRC has been adjusting to its new responsibilities and expectations as a statutory agency. Consistent with the revised National Health and Medical Research Council Act 1992 (NHMRC Act), high level governance arrangements are in place: a Chief Executive Officer (CEO); established governance structures which include the Council and its committees; and defined responsibilities for each of these governing entities. These arrangements constitute a sound basis for the agency's governance and a platform from which to address challenges and expectations arising from broader Government initiatives to enhance investment in Australia's health research sector.

19. However, the NHMRC is an agency in transition, with a substantial change agenda. Particularly evident is the gradual transfer of key administrative functions from the Department of Health and Ageing (DoHA), culminating in the NHMRC's growing administrative independence. In recognising weaknesses in its own management of grants, the NHMRC has also reviewed its grant processes and compliance framework, and commenced a $3 million project to develop a new grant management system.

20. Notwithstanding the agency's progress to date, there are several shortcomings in the NHMRC's administration of this substantial grant program, which combined impact significantly on the effectiveness of NHMRC's grant administration. In particular, there is a lack of consistency in applying guidelines and procedures for specific aspects of the NHMRC's selection process, including conflict of interest provisions, which leaves the agency exposed in terms of the transparency and defensibility of grant selection. In addition, poor compliance in managing grants post-award diminishes the agency's ability to provide sufficient assurance that grant funds are used for their intended purpose. Furthermore, the grant management systems do not adequately support the agency's administration of grants or allow sufficient collection of information to report against program outcomes.

21. A sustained effort will be required by the NHMRC to improve the overall effectiveness of its grant administration. To this end, benefits would be gained by focusing on: enhancing management of key aspects of the grant selection process, including peer review; improving assurance of the appropriate management and use of grant funds; and implementing an appropriate grant management system. These matters are expounded below.

Enhancing management of key aspects of the grant selection process

22. Selection of grants for funding involves a process of peer review, with appraisal of applications by external assessors and a Grant Review Panel (GRP) comprised of relevant experts. This process carries inherent risks for the NHMRC, as it relies on the commitment of experts from within the research community, who, at times, are members of the NHMRC Council and its committees, assessors and members of GRPs, or are themselves recipients of NHMRC grants. As NHMRC grants are highly competitive, the selection of the highest calibre grants is largely reliant on the NHMRC's ability to maintain a fair and defensible peer review process.

23. The NHMRC provides guidelines and procedures to assist reviewers in conducting peer review and grant selection, and expects them to adhere to conflict of interest provisions. However, the NHMRC was not consistent in its application of key elements of the grant selection process, including grant eligibility requirements, recording of grant scores and key actions of the GRPs, and implementation of conflict of interest provisions.

24. Closer monitoring and scrutiny of the selection process is required to provide the NHMRC with the confidence that its policies and guidelines are being consistently and appropriately implemented. Clear recording of the GRP's key actions and recommendations, and the reasons underpinning these, will promote a more defensible grant selection process and better position the NHMRC in responding to unsuccessful applicants or contested grant decisions. Overall, these improvements will allow the NHMRC to achieve greater transparency and probity in its grant selection process.

Improving assurance of the appropriate management and use of grant funds

25. To provide confidence that Commonwealth funds will be used appropriately and for the purpose they are intended, grants are awarded only to approved Administering Institutions, and administered under a Deed of Agreement (the Deed) that sets out the terms and conditions for the management of grants.

26. Owing to several shortcomings in the certification of Administering Institutions, and the monitoring and management of grants, the NHMRC is not well placed to provide adequate assurance about the use of grant funds. There is a general lack of compliance monitoring around reconciliation and reporting of grants, with NHMRC's main grant management systems having no monitoring capability. This has diminished the NHMRC's ability to account for grant funds, reducing its efficiency in managing grants post-award. Compounding this, the NHMRC had not implemented its own policy for approval of Administering Institutions or a compliance framework for post-award management of grants.

27. It will also be necessary for the NHMRC to implement a workable risk-based certification process for Administering Institutions and a systematic and sustainable approach to monitoring compliance with the Deeds, reconciliation of grants and recovery of debts.

Implementing an appropriate grant management system

28. A suitable automated grant management system can assist in monitoring the progress and outcomes of grants. This is particularly the case for the NHMRC given its considerable investment in research and the large volume of applications processed each year.

29. The NHMRC's information systems do not adequately support the NHMRC's core business—grant management. Its primary grant management system contains substantial data anomalies. Furthermore, the system does not accommodate the monitoring of grants' financial and progress reporting requirements, or capture qualitative information from submitted grant reports. This diminishes the NHMRC's capacity to gather and evaluate valuable information for reporting against program outcomes.

30. The NHMRC was advancing development of a new grant management system, and a data repository designed to improve the NHMRC's data capacity. To obtain the most benefit from its new systems will require the NHMRC to focus on system interfaces, adopting a more rigorous but sustainable program of data maintenance and improving staff training in grant management. It is important that the grant system incorporates adequate controls to allow better management of eligibility issues and non-compliance against the Deed.

31. The ANAO has made five recommendations to assist the NHMRC in strengthening its administration of grants.

Key findings by chapter

Chapter 2 – Governance arrangements

32. In line with the NHMRC Act, the NHMRC has a CEO and established governance structures which include the Council and its committees. The roles and responsibilities of key groups and individuals are appropriately defined in key corporate documents. These arrangements constitute a sound basis for governance. However, aspects of the NHMRC's underpinning administrative framework, systems and procedures for administering grants were not as solid. Particular areas for improvement were the approval of Administering Institutions and the establishment of more effective compliance activities.

33. Universities and research organisations constitute an important part of the grant administration arrangements. NHMRC policy defines that grants are paid only to approved Administering Institutions. The obligations and conditions for the management of awarded grants are established through a Deed of Agreement (the Deed) between each approved Administering Institution and the NHMRC. In this context the Administering Institutions act as third party providers of services (research) to the Commonwealth. These arrangements are intended to provide assurance that Commonwealth funds are used appropriately.

34. While the NHMRC funds grants in over 90 Administering Institutions, it cannot demonstrate that the institutions are approved in accordance with NHMRC's own policy. Different versions of Deeds are in operation across the various institutions and some institutions had not signed the most current version. Schedules to the Deeds listing grant details were often absent or unclear, and file records were incomplete, making it difficult to ascertain the grants covered by a particular Deed.

35. Administering Institutions are expected to comply with the Deeds, particularly the financial and quality reporting requirements. The NHMRC has a responsibility to pursue instances of non-compliance. However, compliance with the Deed was inconsistent, particularly reporting of individual grants, and monitoring of non-compliance issues by the NHMRC was irregular and often cumbersome due to a lack of automation. The NHMRC gave little indication of whether poor compliance with the Deed was taken into consideration in future grant applications.

36. The NHMRC commenced developing a new compliance framework in October 2008. This included revision of the Administering Institution approval policy and the Deeds, and a review of the approval status of all Administering Institutions planned for late 2009. To improve assurance and accountability to government, it will be necessary for the NHMRC to complete this work, and establish a workable certification process for Administering Institutions. Given the poor level of compliance, a more structured and consistent approach to implementing and monitoring compliance with the Deeds will be necessary.

Chapter 3 – Assessment and selection of grants

37. NHMRC research grants are highly competitive, so it is important that the selection process is conducted without favour or prejudice using processes that are transparent and defensible. To this end, the NHMRC produces comprehensive guidelines and procedures to assist applicants with the submission of their applications and to guide external assessors and GRP members in their assessment of grants.

38. Advice to applicants also includes eligibility requirements, selection criteria, and other essential requirements on which assessment of the application is based. Checking the eligibility of grant applications and applicants is initially performed by the NHMRC program area, with additional review of eligibility by the GRPs during the selection process. The NHMRC's initial checking process lacked consistency and rigour, with reasons for waiving eligibility requirements not always apparent. To assure the equity of the application process, closer monitoring and consistent application of initial eligibility checks by the NHMRC would be beneficial.

39. Assessors and GRP members are appointed by the NHMRC and allocated applications to assess largely on the basis of area of expertise. The NHMRC's use of external grant assessors is inconsistent, with some applications receiving the intended two external assessments and others less than this. To enhance the integrity and consistency of the assessment process, policies are needed to endorse the value and consistent use of external assessors. The NHMRC would also benefit from reinforcing its GRP–grant selection process. In particular, greater clarity is required in the scoring and ranking of applications against the selection criteria, and the recording of key actions and reasons underpinning GRP recommendations.

40. The NHMRC's conflict of interest guidelines and policies assist the CEO and GRP Chair in managing potential conflict of interest within the grant selection process. However, there are weaknesses and risks, including perceived conflicts of interest, in the NHMRC's current practice. In particular, a substantial number of GRP panel member applications are assessed by their own panel. Conflict of Interest declarations are limited in the range of conflicts declared. In light of this, to provide better protection to the NHMRC's selection processes and reputation, the NHMRC should more closely monitor its allocation of grants to reviewers. It should also strengthen conflict of interest guidelines and policies to include a more comprehensive listing of potential conflicts of interest and a register of private interests, together with protocols for its operation.

41. Strengthening of the NHMRC's conflict of interest provisions for GRPs and assessors would also assist in enhancing the probity and contestability of the grant process. There is the potential for conflict of interest in the NHMRC's peer review process in some specialist areas of medical science where the number of available experts is small. Guidance and vigilance are therefore required to minimise the impacts of perceived, potential or actual conflicts of interest and retain confidence in the selection process and outcomes. Reinforcing these areas of administration would greatly assist the NHMRC in enhancing the probity and contestability of grant selection and, thereby, improve confidence in the NHMRC grant process.

Chapter 4 – Post-award grant management

42. The Deeds of Agreements between the NHMRC and Administering Institutions establish the administrative arrangements and reporting requirements for post-award management of grants. Under the Deed, the NHMRC and the Administering Institutions have obligations to manage the grants according to sound financial practices, ensuring that grants are used as intended and monitoring compliance with the terms and conditions for reporting. Most grants are paid quarterly. While payments are usually accurate, Administering Institutions do not regularly receive a payment schedule and payments are frequently delayed due to system processing issues. This can impact on Administering Institutions in terms of their planning, financial management and cash flow.

43. Individual grant reporting requirements include a yearly financial report and final acquittal, a yearly progress report and final project report detailing the achievements of each research project against project objectives and milestones. However, the NHMRC has a history of poor compliance in its collection and monitoring of grant reports and acquittals. The 2003–04 ANAO performance audit of the NHMRC reported a deficiency in accountability of government funds owing to a backlog of 11 000 award acquittal statements dating from between 1991 and 1997. In October 2005, the NHMRC again experienced a backlog, with an estimated 1275 statements outstanding. By March 2009, the NHMRC had reduced this to 502.

44. Over the same period, the NHMRC's monitoring of debt recovery had not been particularly systematic and there was no register to track the status of debt invoices. Overall, these weaknesses considerably reduce the NHMRC's level of assurance that the grants it funds are being used for their intended purpose, or that they are progressing against their specified objectives and outcomes.

45. The NHMRC's has begun to address these problems, including: introduction of a new financial system planned for 2009; progress in developing a new Research Grant Management System (RGMS); and moving the functions and responsibilities for debt recovery and generation of invoices from DoHA to the NHMRC. Providing the NHMRC can establish its new systems and processes for post-award grant management, it should be in a better position to improve key aspects of compliance and performance against the Deed of Agreement and the FMA Act.

Chapter 5 – Systems supporting grant management

46. Automated grant management system can be a valuable asset to agencies administering grants. The NHMRC's management of grants is made difficult and inefficient due to limitations in its current main grant management information system (RMIS), reliance on a large number of unconnected additional data bases, and deficiencies in system integration and data quality.

47. The RMIS does not provide end-to-end grant management and lacks many desirable system controls for detecting inconsistencies or non-compliance with the NHMRC's legislative and policy requirements. This inhibits the NHMRC's ability to monitor grants' compliance with eligibility and reporting requirements. RMIS also has many shortcomings as a grant management system which impact on sound financial management practices. In particular, the system has no capacity for debt recovery or to remove incorrect totals from grant schedules. There are also problems caused by duplicate and multiple entries in the RMIS. These can inflate RMIS statistics, increase the risk of duplicate grant payments, and cause uncertainty when updating grant records or processing variations.

48. The NHMRC is aware of the existing deficiencies, and recognises the need for a more robust and encompassing system, and during the audit was progressing the development of the new system—RGMS. This is being phased in, with final roll out to all grant schemes by 2010. The system aims to enable end-to-end management of grants, incorporating application and selection through to acquittal, and providing meaningful and consistent reporting at all required levels.

49. Notwithstanding progress to date in developing the new system, there are particular vulnerabilities and areas of high risk which the NHMRC will need to strengthen to achieve effective and efficient grant management. To prevent inadvertent or deliberate non-compliance of grantees with legislative and policy requirements, will require the inclusion of adequate systems controls into the RGMS. A suitable interface between RGMS and the NHMRC's financial system is also necessary to facilitate accurate information exchange and reconciliation of the two systems. Phasing in of RGMS should also be accompanied by early implementation of a regular program for the verification and cleansing of data, adequate staff training, and ceasing the use of ancillary data bases.

Chapter 6 – Monitoring performance

50. The NHMRC has an obligation to meet whole of government reporting as well as those specifically required under the NHMRC Act. For 2008–09, the NHMRC met the government's Outcomes and Programs Framework requirements. In its Portfolio Budget Statements, the NHMRC identified an Outcome which suitably reflected the agency's broader role in supporting health and medical research and providing health advice. This was supported by an appropriate output for its research function, and key strategic directions reflecting government health priorities. In the 2009–10 Portfolio Budget Statements, the NHMRC has enhanced Outcomes and Programs Framework in line with the government's Operation Sunlight Outcome Statement Review, specifying an Outcome, Program, Key Strategic Directions, Key Performance Indicators (KPIs), strategies and major activities.

51. As required by the NHMRC Act, the NHMRC has included strategic directions in its Strategic Plan, and these align with those in the Output Programs framework. There is also a balance of performance measurement, utilising both qualitative and quantitative measures against identified performance indicators.

52. To measure performance against programs and outcomes it is important for agencies to have viable systems for the collection and analysis of suitable qualitative and quantitative data. The NHMRC's data capacity is limited by its IT systems. Despite this, the agency produces a range of useful statistics on the grant process, which it makes available through its website.

53. Final reports are important in providing information on individual grants, broader program Outcomes, and health and medical policy and practice. This is essential information for measuring the medium to long term impacts of the NHMRC's research programs. The NHMRC's future ability to meet the full ambit of reporting requirements will, to some degree, depend on its success in strengthening data collection and analysis through planned IT initiatives (RGMS and Datamart), and on improving compliance and quality of final grant reports.

NHMRC's response

54. The NHMRC provided the following summary comment and the responses to each of the recommendations in the body of the report comprised its formal response:

NHMRC welcomes the audit and agrees with its recommendations. Since establishment as an independent statutory agency on 1 July 2007, NHMRC has worked hard to ensure the efficiency and effectiveness of its administration and in particular its grant administration.

The ANAO's audit report affirms NHMRC's program of continuous improvement in grant administration, which includes the implementation of a new Research Grants Management System (RGMS), being piloted prior to full implementation from December 2009. This new technology will address many of the weaknesses identified by ANAO.

NHMRC will undertake further improvement of its peer review administration in the 2010 grant application round, in particular strengthening decision support, documentation and the management of conflicts of interest in Grant Review Panels. NHMRC is currently reviewing its policies and practices in relation to administering institutions, particularly independent medical research institutes.

The audit provides a valuable framework for NHMRC in striving to achieve best practice in grant administration.

Footnotes

1 NHMRC, Annual Report, 2007–08.

2 Department of Health and Ageing, Portfolio Budget Statements 2009–10, p. 673.

3 <http://www.nhmrc.gov.au/grants/dataset/ files/grantdata.xls> [accessed 16 February 2009].

4 ANAO Audit Report No.29 2003–04, Governance of the National Health and Medical Research Council.

 5 J. Uhrig, Review of the corporate governance of statutory authorities and office holders, June 2003.