A major obstacle to success in obtaining grant awards is failing to understand the peer-review process. Many applicants’ knowledge of peer review consists of: (a) knowing that it is a process by which applications are evaluated; (b) believing that reviewers decide which grant applications will be funded; and (c) trusting and/or hoping that a goodea and good luck are all that is needed to get funded. These beliefs are largely incorrect and result in applicants making poor decisions in preparing their applications. In other words, this misguided approach to the grant application and peer review process is much like playing the lottery – success is all a matter of chance.
In reality, grant success requires much more than a goodea and good luck. It requires not only understanding the peer-review process but also controlling, insofar as possible, the numerous elements that are the applicant’s responsibility. The fact is that applicants control a great deal of the process and eventual outcome by, among other things, the decisions they make in preparing their applications, and how they present theireas to the reviewers and funding agencies that make the critical go/no-go decisions. The peer-review process in U.S. federal government agencies is governed primarily by a series of laws, rules, regulations, and policies. However, peer-review practices also are based, at least to some extent, on the cultural and behavioral practices of the peer-review group performing the evaluations.
By law (the Federal Advisory Committee Act of 1972), peer reviewers advise agencies on the scientific and technical merits of the applications submitted, but do not make the funding decisions themselves. Those decisions are made by the funding agency. Here it should be noted that several essential and substantive aspects of managing the peer-review process are inherently governmental functions, including: (a) monitoring the entire review process to ensure it is not only both fair and thorough but also conforms to all laws, regulations, and policies applicable to the specific grant request being reviewed; (b)entifying and recruiting the most appropriate and knowledgeable persons available to serve as reviewers; (c) assigning those reviewers to focus on particular applications; and (d) summarizing the discussion at various review meetings and incorporating the reviewer recommendations in follow-on documentation of the review.
Fair, Equitable, Free of Bias & Loaded With Content
Understanding the peer-review process begins with the recognition that every funding agency, whether public or private, has a specific mission in mind (several missions, in fact, much of the time) – and improving the capability of carrying out that mission is the primary basis for awarding grant funds. Another peer-review goal is to ensure that the evaluation of grant applications is carried out in a fair and equitable manner and, a third goal, to ensure that the process is free of bias – thus helping the granting agency choose the best application capable of furthering the agency’s mission. However, the “best application” may or may not be the best application as judged by peer review of the scientific and technical merits of competing proposals. The decision of what to fund, and/or not to fund, is and must be discretionary – and is in any case the responsibility not of the reviewers but of the funding agency staff. In order to obtain the funding requested, therefore, applicants must satisfy not only the needs of reviewers but also the needs of the funding agency.
The latter goal – determining the needs of the funding agency – begins with the agency’s mission, which is more than simply a broad statement of goals and objectives. The mission includes the specific operational responsibilities (goals, objectives, instructions) that are articulated by an agency in a Funding Opportunity Announcement (FOA). FOAs are publicly available documents by which, among other things, agencies make known their intentions to award discretionary grants or cooperative agreements on particular topics or areas of mission need. The FOAs can take the form of Program Announcements, Requests for Applications, Notices of Funding Availability, Solicitations, etc., depending on the agency involved and the type of program under consideration. (FOAs for the National Institutes of Health [NIH] and other agencies, for example, can be found at Grants.gov and in the NIH Guide for Grants and Contracts.) To facilitate the process, government agencies are now standardizing their collective approach to peer review through the use of common application forms, the data requirements postulated, instructions for the preparation of documents, and similar review criteria.
Some applicants view FOAs, erroneously, as little more than descriptions of the program, the amount and duration of funds available, and a set of how-to instructions – related to page limits, for example, or the font size required – that must be followed for structuring an application. That view is totally misleading. The fact is that most if not quite all FOAs contain important information on the specific mission of the funding agency and the criteria that the reviewers will use to evaluate a grant application. Moreover, the instructions included in the FOAs usually provide information – in addition to the criteria needs of the reviewers – on the specific mission needs that grant applicants must directly address.
Rule #1: Please the Reader, Not the Writer!
The most common mistake that applicants make in preparing their grant applications is that they write the application to please themselves, thereby satisfying their own needs but not those of the funding agency. Writing an application to impress the applicants themselves (and/or their superiors, which is more likely) makes sense, though – but only if the applicants are going to fund the grant themselves. However, if they are not going to fund it themselves, then they will fail – because they prepared the application for the wrong audience. Success requires not only strict discipline but also a continuing focus on both the peer reviewers who evaluate the applications and the funding agencies that make the final decision on what projects are approved, and for what level of funding.
In short, funding applicants must always keep in mind that reviewers, in the public and private sectors alike, serve primarily as advisors to the funding agency, and are not the final decision makers. Agencies will and should of course consider the recommendations of review advisors, but may choose either to accept or not to accept those recommendations. The decision to fund or not to fund a particular application is based on several criteria, including: (a) the merit of the application as defined by peer review; (b) the “fit” of the proposed project or research to the mission of the funding agency; (c) the overall portfolio balance (the number of similar applications proposed for funding); and (d) most important of all, the availability of funds (the total of which may vary considerably from one year to the next).
Funding agencies carry out most peer reviews by recruiting knowledgeable professionals from outside their own agencies who possess the competence, experience, and depth of expertise required to evaluate not only the various subject matters contained in the applications but also the agency missions described in the FOAs. Reviewers evaluate applications to a large extent by using the review criteria published in the FOA so that the significance, importance, approach, and feasibility of each and all of the research projects proposed can be competently judged. Although the review criteria published by funding agencies are in the public domain, many applicants ignore their existence and do not write applications that are clearly and directly responsive to the review criteria postulated. When applicants ignore not only the mission of the funding agency but also the review criteria spelled out for them in public documents, they are writing the application primarily – as mentioned above – to please themselves and will almost surely fail to obtain the funding sought. Not because of bad luck, though, but because of poor decisions on the part of the applicants.
When focusing on the reviewers, therefore, it is particularly important to recognize that reviewers read applications because they are required to, not because they want to. They are, in fact, usually not compensated for their time and service as reviewers (although they may be provided a small stipend for their travel time to the meeting location and the expenses incurred en route). Most reviewers participate in the evaluation process as a matter of professional responsibility, and want the review process to be as effective and efficient as possible – thus reducing the time required for evaluating each and every application. Applicants who make the job of reviewers more difficult and/or take longer than absolutely necessary make reviewers less likely to be favorably disposed toward them. Failure under this scenario would once again not be due to bad luck but, rather, to the misguided and ill-advised decisions made by the applicants themselves.
Five Questions: What, Why, How, and Two More Whats
To help simplify the process for reviewers, and increase their own odds of receiving grant approval, applicants should answer what are considered to be the five fundamental questions asked by most reviewers: (a) What does the applicant propose to do? (b) Why should this be done? (c) How will it be done? (d) What will be the probable outcome? (e) What will be the probable impact of what is being proposed? All of which can be boiled down to a separate all-important question: Why should this project/proposal be approved for funding? Failing to provide clear and direct answers to all of these questions, no matter how they are worded, will result in a lot of additional (and unnecessary) hard work for the reviewers as they try to figure out the answers (often incorrectly) for themselves. The most likely consequence is that the priority score (i.e., the average of reviewers’ scores from 10 to 90, with 10 being the best) will suffer and the application will receive a score that will keep it from being recommended for funding consideration by the granting agency.
Expressed another way: Understanding the peer-review process means, above all, understanding that: reviewers are never wrong; reviewers are never right; reviewers simply provide an honest but expert evaluation of the material documentation provided in the application. To earn a different – i.e., more favorable – recommendation and evaluation outcome from reviewers, the application materials must be presented in a different, more logical, and more coherent manner – one that reviewers understand and can agree with in good conscience. If applicants want reviewers to know what they intend to say, they must present the information included in the application clearly enough to be understood without further explanation. Reviewers cannot read minds.
Some applicants believe – again, erroneously – that peer review is a hurdle or impediment that must be overcome on the way to funding. This belief will result in misguided decisions when preparing applications and will lead the applicant to make fundamental errors in the preparation process that will almost inevitably lead to failure rather than success. Peer review is, in short, actually an opportunity to persuade reviewers to serve as the applicant’s advocates with the funding agency. Understanding peer review is above all, therefore, “good grantsmanship” – knowing and understanding what to do, how to do it, when to do it, what to do when the process does not go exactly as planned, being willing to do what is both needed and expected, therefore, and doing what is needed for final success.
For additional information about:
The Federal Advisory Committee Act, see http://www.archives.gov/federal-register/laws/fed-advisory-committee
Finding grants (at Grants.gov) see http://www.grants.gov/applicants/find_grant_opportunities.jsp
The NIH Guide for Grants and Contracts, see http://grants.nih.gov/grants/guide/index.html
Presentations by Dr. Coelho on Scientific Peer Review and Grant Writing for Success, see http://ora.stanford.edu/ora/ratd/nih_04.asp
More information about Dr. Coelho, see http://www1.faseb.org/careers/CDSW/pages/page2d.htm
Anthony M. Coelho, Jr.
Anthony M. Coelho, Jr., Ph.D. is President of Grant Success Associates, which he founded after retiring as Senior Administrator at the National Institutes of Health (NIH). He is also Senior Associate with Health Research Associates, a consulting firm in Rockville, MD. During his 15 years at NIH, he served as NIH Review Policy Officer, Chief of the Clinical Studies and Training Review at the National Heart Lung and Blood Institute, and Acting Director of the Office of Federal Advisory Committee Policy (in 2005). Prior to joining NIH, he held various positions at the Southwest Foundation for Biomedical Research and University of Texas Health Sciences Center, both in San Antonio, Texas.