CSR Overview

Scope of Review Operations at CSR

applications icon
CSR reviews 77% of NIH proposals.

study sections icon
In over 200 chartered or recurring Study Sections

budget icon
CSR does this with less than 0.4% of the total NIH budget

reviewers icon
CSR engages 18,500 distinct reviewers

meetings icon
CSR conducts 1,600 annual review meetings

review officers icon
CSR employs 242 scientific review officers

CSR’s Mission: To see that NIH grant applications receive fair, independent, expert, and timely reviews – free from inappropriate influences – so NIH can fund the most promising research.

CSR Also Participates in the Review of These Initiatives and Inter-agency Collaborations

National Institute of Health
NIH All-of-Us Program Reviews, plus Other Transaction Authority Reviews

National Institute of Health
Brain Research through Advancing Innovative Neurotechnologies (BRAIN)

National Institute of Health
NIGMS Maximizing Investigators’ Research Awards (MIRA)

National Institute of Health
Native American Research Centers for Health (NARCH)

Global Alliance for Chronic Disease

National Institute of Health
Division of Program Coordination, Planning, and Strategic Initiatives (DPCPSI)/Common Fund

National Institute of Health
Many institute PARS and multi-institute RFAs

Food & Drug Administration
Tobacco Center of Regulatory Science for Research Relevant to the Family Smoking Prevention and Tobacco Control Act

National Institute of Health
Cancer Moonshot

National Institute of Health
Alzheimer's Disease (AD)

National Institute of Health
Office of Research on Women’s Health (ORWH) at NIH - Specialized Centers of Research Excellence (SCORE) on Sex Differences

National Institute of Health
Fogarty International Center

National Institute of Health
Office of the Director (OD)/Office of Research Infrastructure Programs (ORIP)

All USA-China Reviews

Ongoing Evaluations

Early Career Reviewer (ECR) Program

The Early Career Reviewer (ECR) Program was established in late 2011. The program exposes junior scientists to the peer review experience to make them more competitive as applicants and to enrich the pool of NIH reviewers.

  • 3,111 individuals have received ECR training and have served on study sections.
  • 19% of ECRs who have served on study sections are from under-represented racial/ethnic groups.
  • 47% of ECRs are women.
  • 143 former ECRs are now members of standing study sections.
early career reviewers icon

Analysis to be performed by CSR:

  1. Demographic analysis: gender, race, ethnicity, geographic distribution,and institution type for both the ECRs in our database and the ECRs recruited to review
  2. Analysis of success rates for R01s submitted by ECRs compared to early career researchers, matched for relevant demographic and career characteristics
  3. Survey experience to identify areas of program improvement

Anonymization Study Progress to Date

CSR is working with an outside contractor to examine various potential sources of bias.

Potential biases being examined include race, gender, career stage, and institution assessment.

1,200 previously reviewed applications, both full and redacted, are being used for the assessment.

The experimental approach, results, and data analysis will be peer-reviewed and shared sometime in 2019.

Acknowledgment: Lee Mann & Richard Nakamura

Redaction of Application Information


Reviewer Recruitment




Critiques Received


Data Gathering and Analysis


meeting group photo

New Framework for CSR Study Section Evaluations

Framework for Study Section Evaluations:

  1. Review of 12-20 scientific clusters, in a similar scientific area
  2. Assemble blue-ribbon working group of scientifically broad, senior scientists, with interests that cross over multiple study sections
  3. Provide information on current scientific guidelines for all study sections, titles, abstracts, specific aims, workload trends
  4. Provide enough time and guidance for meaningful evaluation and recommendation by the working group
  5. Ask 1 question designed to focus the discussion on science, not process

How well does the scientific scope of the study sections align with the current state of the science?

Recommend modifications if:

  • A study section is currently over or under subscribed
  • The scope of the science reviewed by a study section is not adequately representative of the scope of the field and emerging fields

Multiple Possible Actions for Restructuring Study Sections:

  • Change in scientific guidelines
  • Merge study sections
  • Create new study sections
  • Eliminate study sections
  • Move an area of science from one study section to another/others
  • Add emerging areas of science

Reviews, Restructuring and Alignment

Examples of Better Scientific Alignment, Accommodation for Emerging Fields

Redistribute basic biomaterials/biomimetics/fluidics in neural systems from Biomaterials and Biointerfaces (BMBI) to Bioengineering of Neuroscience, Vision and Low Vision Technologies (BNVT) – CSR Advisory Council approved and CSR implemented in 2016.

Move emerging area of microphysiological systems from BMBI to form new study section called Cellular & Molecular Technologies (CMT) – CSR Advisory Council approved in 2016 and CSR implemented in 2017.

Expand scope of Biomedical Computing and Health Informatics (BCHI) to include extraction of data from electronic health records, clinical decision support systems, telemedicine, large image databases, image informatics to guide decision-making – CSR Advisory Council approved and CSR implemented in 2016.

Areas Evaluated to Date

Bioengineering (Pilot, 2015)

Basic Cancer Biology (2016)

Imaging Technologies (2017)

HIV/AIDS (2017)

Vision Sciences (2018)

New Framework for Study Section Evaluation

December 2017 – June 2018: Trans-NIH Working Group on Peer Review developed a robust study section evaluation process.

The Working Group, co-led by CSR and representatives from NIH institutes, considered data/metrics, multiple inputs and measures to design a process that could be practically implemented by CSR.

Largely builds on CSR’s existing successful model of study section evaluations - external panel to evaluate science by examining workload trends, guidelines, random sample abstracts/specific aims, adds in publication/bibliometric data.

Adds a process evaluation component - internal NIH panel to look at process issues, e.g., distribution across institutes, scoring patterns, reviewer and program officer surveys, discussions, rosters, New Investigator application/award rates, etc.

Evaluation Process

  • Combines strength of expert opinion and objective metrics
  • Stakeholder engagement – CSR Advisory Council, and NIH institutes
  • Continuous and systematic approach - Evaluate ~20% of the study sections each year (i.e. a study section is evaluated every 5 years).
  • Addresses both NIH Strategic Plan element to "Optimize approaches to inform funding decisions" and CSR’s mandate to continuously examine the function of study sections

Evaluation Progress

  • Clusters of study sections in two scientific areas were evaluated to pilot the process
  • External Scientific Evaluation Group Meeting:
    December 2018
  • Internal Process Evaluation Group Meeting:
    January 2019
  • Report to the CSR Advisory Council:
    March 2019
  • Iterative Approach – Continuous refinement/modification of process based on experience and feedback

Evaluation Timeline

Completed Cluster Formation

Completed Prioritization Of Clusters

Months 1-4 External Scientific Evaluation Panel

Month 5 Internal Process Evaluation Panel

Months 6-8 EAWG and CSR Advisory Council

Months 9-12 Implementation by CSR

Integrity of the Peer Review Process

peer review group photo

Critically important for all of us

We must maintain the public trust in the NIH’s stewardship of taxpayer dollars to support U.S. biomedical science research.

Confidentiality is critical for candor in discussion and evaluation, and thus impacts the very basis of the peer review process.

Ensuring integrity requires the support of the entire research community – investigators, reviewers, study section chairs, NIH staff, institutional officials.

NIH is taking this issue very seriously. There do not appear to be widespread problems, but increased reporting and action is a cultural change.

What is the NIH Doing? Reporting & Action

We follow up on every allegation.

Cases are referred to Office of Management Assessment (OMA) – independent of CSR - an investigative unit conducts fact-finding, investigations, and issues a report of findings.

Actions have included

  • withdrawal of application
  • removal of reviewers from peer review committees
  • notification of the investigator/reviewer’s institution, which has led to personnel actions
  • pursuing government-wide suspension and disbarment
  • referral to other agencies for criminal violations

Making Reporting Easier

To make reporting easier, all scientific review officers include reporting information in their signature line in email messages:

Integrity matters. Say something! For concerns or questions about possible violations of peer review integrity, please contact your Scientific Review Officer, or the CSR Review Integrity Officer at csrrio@mail.nih.gov, or the NIH Review Policy Officer at reviewpolicyofficer@mail.nih.gov. See the NIH Guide Notice on integrity in review.

Scientific Review Officers regularly discuss integrity and reporting avenues. This is done through pre-meeting training of all reviewers and in opening statements at review meetings.