Life After the Deadline
Writing an NIH-quality proposal takes time and great attention to detail.
It also takes a thick skin. The chances that your first-time proposal will
NOT be funded are in the neighborhood of 80% to 85%. The majority
of NIH research proposals that are funded go through two or three review
cycles before the investigator ever sees any money. Be prepared to
rewrite and resubmit your proposal -- chances are good that you will have
to.
What happens to your application after submission to NIH depends in
part on whose idea it was to do the project. Investigator-initiated applications
(your idea, also called "unsolicited") are submitted to the Center for
Scientific Review (CSR) at NIH, a central review office where the application
will be screened and assigned to the appropriate review group.
Some responses to specific announcements (their idea, in the form of
RFA's or RFP's) are routed to the Institute or Agency
which issued the call for applications. Check your specific announcement
to see if the application is to be sent the Institute or to CSR. If your
application in response to an RFA or PA will be routed through CSR, be
sure to use the RFA label included in the PHS 398 application package,
and attach a copy of the announcement as well, to make sure that office
knows for whom it is intended. It is not unheard-of for such applications
to be assigned for review to an institute other than the one that issued
the PA or RFA.
In either case, your proposal will be assigned for review to a group
of your peers, volunteers with demonstrated expertise in the subject area.
In most cases this group will meet for a marathon review session, often
lasting several days, during which all the group's assigned proposals will
be presented, discussed in depth, and rated by members. A record will be
kept of the substance of these discussions, and NIH staff members will
prepare a summary of the group's critique of each proposal.
How do you find out who will review your application? When your application
has been assigned, CSR or the Institute will send you a notice of its placement.
Membership lists of CSR study sections can be found in the CSR
Study Section Roster Index. The membership of other review groups set
up by Institutes can be obtained directly from the project officer for
the opportunity in question.
Back to Top
Modular Grant Applications
As of the June 1, 1999 deadline, research applications to NIH requesting
less than $250,000 in direct costs for any year will use a revised, streamlined
format known as the Modular Research Grant Application. About 90%
of the most common extramural grants awarded by NIH are expected to fall
into this category.
Under the modular grant application procedures applicants will request
total direct costs in $25,000 increments up to $250,000 in any year
of a project. (Applicants requesting more than $250,000 in direct costs
in any year will continue to follow existing application and award
procedures.) Applicants will provide limited budget information in
a narrative format and will not have to submit other research support
information until just prior to award.
With the modular grant procedures, reviewers will evaluate proposed
project budgets on the basis of a general, expert estimate of the total
effort and resources required to conduct the proposed research. Reviewers
will recommend changes in a proposed project's budget in $25,000 modules,
and the NIH Institute staff will continue to make final award decisions.
For the applicant, this new policy means that a great deal of detail
that used to be submitted before the deadline can now wait until you know
the project will be funded.
More information on using the new modular grant procedure can be found
on the NIH
Modular Grant Page.
Back to Top
Triage: A New Way to Streamline Grant Review
In the past few years, NIH has been experimenting with ways to streamline
the grant review process so that reviewers and staff are not so heavily
burdened, and so that the turnaround time for receiving review results
may be shortened. One of these methods, triaging applications to eliminate
half from consideration at the start of the process, has been adopted by
the Center for Scientific Research as well as by a number of individual
Institutes.
Under this system, reviewers will designate approximately half of the
applications as "noncompetitive" for support. A designation of "noncompetitive"
requires unanimous agreement of the study section. "Noncompetitive" means
that the application is judged to be in the lower half, qualitatively,
of research project grant applications normally reviewed by that study
section.
Applications determined to be noncompetitive will not receive full discussion
at the study section meeting, will not receive a priority score, and will
not routinely be taken to second level of review by the national advisory
councils/boards.
The summary statement for an application determined to be noncompetitive
will consist of the customary administrative information and the reviewers'
critiques, verbatim. The summary statement for an application that
receives full discussion and a score will include, in addition to the reviewers'
critiques and the administrative information, a "Resume and Summary of
Discussion," which synopsizes the study section's discussion of the application.
Subsequent to the study section meeting, all applicants will receive the
customary snap-out mailer to advise them of the outcome of the initial
review.
Back to Top
What Happens to Your Research Application After It
Gets to NIH
[Adapted from an article in the January, 1998 Newsletter of the Cancer
Control Research Branch, National Cancer Institute]
On a major grant application receipt day, delivery trucks unload thousands
of packages containing grant applications at the loading docks of the Rockledge
2 Building, the home of the NIH Center for Scientific Review (CSR). Each
package is opened; the application is date-stamped and logged into the
NIH database for tracking.
Over a dozen Referral Officers review the contents of some 10,000 applications
each grant cycle and, using written guidelines, decide both which study
section would be most appropriate for assessment of scientific merit and
which Institute(s) of the NIH would be most suitable to fund the application,
should it be considered sufficiently meritorious. A unique application
number is assigned to each application.
The Referral Office does consider written requests from applicants for
both study section and Institute assignments (just include a cover letter
with the application). The assignment process is a collegial one, with
interaction, when necessary, on a case-by-case basis among Referral Officers,
study section Scientific Review Administrators (SRAs), Institute program
representatives, and applicants.
Within 10 days of application assignment, a computer generated letter
is mailed to each applicant and sponsored research office, listing the
study section and potential funding Institute. Upon receipt of this notice,
applicants can question the study section assignments by contacting either
the Referral Office (301-435-0715) or the study section SRA. There are
official guidelines defining the content and boundaries of the science
reviewed in each study section, but inevitably there is overlap. A particular
application may be reviewable by a number of different study sections.
The entire assignment process may take up to six weeks. If applicants have
not received notification at that time, they should contact the Referral
Office.
Back to Top
As applications are assigned to a study section, a continuing process
extending over six to seven weeks, the SRA begins to read through them,
analyzing content, checking for completion, and deciding which study section
members would be best suited to review each application, or act as discussants.
Some six weeks before the study section meeting, packages are mailed
to members which include all of the applications to be reviewed at the
meeting (with the exception of those applications for which a particular
member is in conflict.) Typically, two or three members are assigned to
provide written reviews of each application, and one or two additional
members to serve as discussants.
NOTE: A chartered CSR study section is composed generally of 18
to 20 individuals, nominated by the SRA from among the active and productive
researchers in the biomedical community, to serve for multi-year terms.
The goal is to have the group's combined knowledge span the diversity of
subject matter assigned to the study section for review. However, this
is difficult to accomplish, and the study section's membership is frequently
supplemented by temporary members and written outside opinions for any
particular meeting. In some instances, Special Emphasis Panels (SEPs) are
formed on an ad-hoc basis to review applications requiring special review
expertise, or due to special circumstances (such as when a conflict of
interest occurs).
Because of the multi-month period between submission and review of an
application, applicants often wish to submit supplementary materials. However,
each study section has policies for acceptance of such additional material
(e.g. length; time of submission). SRAs should be contacted prior to submission,
both as an alert for the SRA, and to ascertain acceptable content and format.
One week before the convening of a study section, the SRA solicits,
from all members, a list of applications believed not to rank in the top
half for scientific merit. The individual lists are coalesced, and a final
list is established at the outset of the study section meeting. Those applications
in the lower half are considered to be "streamlined". They are not scored
or discussed at the meeting, but summary statements are provided, and the
applicants may subsequently revise and resubmit the application. "Streamlining"
is not equivalent to disapproval, but rather represents a decision by the
study section that the application would not rank in the top half of applications
under review during that particular meeting.
With some minor variations, all regular CSR study section meetings follow
the same format. The meetings usually last two to three days. Members convene
around a conference table to maximize interaction. The chairperson (a member
of the study section) and the SRA sit together and are responsible for
jointly conducting the meeting. Observers (program representatives or other
staff from the various NIH Institutes) may attend, but must sit in chairs
set back from the conference table and do not participate in the discussions.
Back to Top
The order in which the applications are reviewed can vary. Sometimes,
applications may be discussed in alphabetical (or reverse alphabetical)
order of applicant, sometimes by numerical order (oldest or newest grant
first), sometimes grouped for discussion by Institute (AA, AG, AI, CA,
etc.) or by funding mechanism (e.g. R01, R29, R21, etc.).
After the assigned reviewers and discussants provide their evaluations,
any outside opinions are read. After general discussion, members mark their
priority scores privately for each application on scoring sheets provided
by the SRA. These sheets are collected by the SRA or an administrative
assistant at the conclusion of the meeting.
Within a few days after the meeting, all priority score information
has been entered into the application database. Computer generated priority
scores and percentiles are then automatically mailed to applicants.
Feedback to applicants is important. However, it still requires six
to eight weeks to generate an average of 80 summary statements. Once those
summary statements are produced and transmitted to the appropriate NIH
Institute for funding consideration, the SRA's control over the review
of those applications ends, and his attention turns to the next grant application
cycle. At this junction, it is the Institute program officials who become
the applicant's link to the NIH with regard to the interpretation of the
reviews and the disposition of the application.
There is a flow to the review process, repeated cycle after cycle (See
NIH
Deadlines and Review and Award Cycles ). For example, applications
submitted for the October/November receipt dates will be assigned to CSR
study sections by early December, and sent out to members of the study
section for scientific review in late
December/January.
Study sections meet between mid-February and mid-March, and summary
statements are prepared by late April/May. Institute Advisory Councils,
the second step in NIH peer review, meet in May/June to consider the study
sections' recommendations, and successful applicants can begin to receive
funding several months later.
Back to Top
Priorities and Pink Sheets
After the proposal has been submitted, you will wait about 5 months, in
most cases, before finding out what the reviewers thought of it. If it
is judged to be competitive (in the better half of the submitted applications,
as explained above) the review committee will assign to your proposal a
priority score, theoretically ranging from 100 to 500, based on certain
review
criteria. You will also receive the dreaded "pink sheets" (so
called because once they were printed on pink paper), which summarize the
review committee's critique of your proposal. Final funding decisions
are not sent with this packet; they usually take another 1-3 months.
The lower the priority score the better. These scores are based
on combined ratings between 1 and 5 (1 being the best, or first in line
for funding) given by each member of the review panel. If your score
is perhaps 160 or below, you have a chance of receiving funding. This is
not a hard-and-fast rule, however. Often the "funding line" -- the score
which demarcates the funded from the unfunded applications -- can be much
lower than 160 (perhaps as low as 120!) depending on the number and quality
of the applications in competition.
If your score comes in lower than about 225, you might be funded, but
be prepared to rewrite and resubmit. A score in
this range is a virtual invitation to resubmit the proposal. Follow the
critique in the pink sheet carefully, and rewrite the proposal to answer
every objection.
If your score is in the range around 225 to 300, read the reviewers'
comments carefully and think hard about what it would take to answer them.
If you can see feasible ways to restructure the proposal to meet the committee's
objections, you might want to go one more round. It will likely require
major restructuring of the application, however, to make it competitive.
If the score is above 300, your chances of getting any money, now or
ever, for this proposal are probably near zero. It would be wise to start
from scratch with a new idea.
Pink sheets can be brutal. Reviewers' comments are occasionally stupid.
Most, however, are carefully considered, highly informed critiques. If
you think of your pink sheets as an instruction manual customized especially
for you to get funds for your project, you can use them to your advantage.
A second stage of review is held to make the actual funding decisions.
These decisions are made by advisory councils or boards for the agency
making the award. NIH staff members play a strong role in making
these decisions. Final funding decisions are influenced by the number of
competitive grants balanced by the amount of money allocated, by the mix
of grants or investigators already funded by that agency, occasionally
by political considerations and possibly by solar flares. It is possible
to receive a very good priority score from your peer review group, yet
not be funded. It is possible for a proposal with a worse priority
score than yours to be funded, while yours is not.
You can see from this overview that it may take several years from the
time you first come up with your great idea until you actually begin the
project. As with natural selection, the prizes go to those who survive
and continue to adapt.
Back to Top
Appealing Your Peers' Review
What if you submitted a grant application to NIH, and it received a review
you consider totally off the mark? Do you have any recourse?
NIH provides an applicant who feels that some aspect of the handling or
peer review of his/her grant application has been inappropriate, biased,
or wrong with two opportunities to have these concerns addressed.
The first opportunity, called a "rebuttal", is a response to the summary
statement of the results of the initial review of your application's scientific
and/or technical merit. The applicant may submit a letter rebutting
the review to the program administrator who is responsible for the application.
The rebuttal letter will usually be made available to the National Advisory
Council/Board of the relevant NIH Institute/Center/Division (ICD) for consideration.
If the Council finds that the your objections have merit, it may recommend
that the application be deferred and rereviewed.
Should the Council concur with the initial review, then you have a second
opportunity to have your concerns heard, by submitting a formal appeal
of the Council's decision. The PI and the applicant institution, represented
by the institutional official authorized to sign applications, must jointly
sign an appeal and send it to the NIH Peer Review Appeals Officer. The
official representative's signature indicates that the applicant institution
endorses both form and substance of the appeal. The appeal letter must
explain fully the reasons for the disagreement, and append supporting documentation.
If you are considering an appeal, keep in mind that the most favorable
possible outcome for you in an appeal case can only be a decision that
the application in question be rereviewed,
since appeals cases examine only whether there were any flaws in the peer
review process. The other possible outcome is that the review of
the application was not substantially flawed and any minor flaws in the
review did not affect the recommendation regarding the application.
In that case, the review would stand and the application would not be rereviewed.
An appeal case may take at least four months (or one review cycle) to complete.
Given the possible outcomes and the timing of the appeal process, you
may wish to consider whether deficiencies in the original review were substantive
enough to pursue an appeal and, if not, to revise and resubmit the application
instead.
Funding decisions are made separately from the peer review process and
take into consideration several factors in addition to the review group's
recommendation. These decisions may not be appealed.
Back to Top
NIH "Three Strikes" Policy on Revised Applications
Applicants to the National Institutes of Health have
long been allowed to resubmit amended versions of applications that have
been declined on the first review. Since the NIH provides detailed feedback
on the shortcomings of non-funded proposals this policy has promoted fine-tuning
of projects and has led to greatly improved chances of funding on the second
or third submission. However, the policy has also made it possible for
some investigators to continue rehashing poor proposals many times over
when it would be preferable to start over from scratch.
Beginning with the October 1996 receipt date, the National
Institutes of Health (NIH) will no longer consider any more than two amendments
to an application and, regardless of the number of amendments, will not
accept a revised (amended) application submitted later than two years beyond
the date of the receipt of the initial, unamended application.
The reason for limiting amendements stems from a desire
to decrease the review burden at NIH by limiting less-competitive applications.
Recent NIH data indicate that amended applications make up more than one
third of all research project grant applications, and that applications
which are funded after amendments do not fare well in subsequent competitions
for renewal. The NIH believes that after three unsuccessful attempts
at funding, applicants should take a fresh start at their research plans.
Therefore, the NIH has adopted a policy that limits the number of amendments
to two. A two-amendment limit will still allow principal investigators
sufficient time to generate preliminary data if it is required by the reviewers,
and to consider new findings in the area of research.
Investigators submitting amended applications to NIH are
reminded that the deadlines for these applications are November 1, March
1 and July 1 of each year.
Back to Top
See Also:
NIH
Information on Peer Review
| 
