A draft product is sent to reviewers after all authors have indicated they are satisfied with its form and content. Reviewers receive the complete product, the Statement of Task, and these instructions and are asked to provide written comments on any aspects of the draft, but to pay particular attention to the review criteria listed below. The authors are expected to consider all review comments and submit a revised document, which is evaluated by one to two external experts who serve as impartial arbiters of the review. A product may not be released to sponsors or the public, nor may its findings be disclosed, until the review process has been satisfactorily completed and all authors have approved the revised draft. Furthermore, once this process is complete, no changes (other than minor editorial corrections) may be made.

To encourage reviewers to express their views freely, review comments are privileged documents given to the product authors with identifiers removed. After submitting comments, reviewers are asked to delete the draft manuscript and refrain from disclosing their comments or the contents of the draft. The names and affiliations of participants in the review process are made public when the product is released, but their comments remain confidential. Even after release, reviewers should not divulge their comments or any changes made to the draft manuscript to safeguard the integrity of the institutional review process.

The rationale for any findings, conclusions, and recommendations should be fully explained in the product. This rationale might include references to the literature, analysis of data, a description of the pros and cons of the alternatives (including the status quo), and the reasons for preferring a particular option. Reviewers are asked to consider whether, in their judgment, the evidence and arguments presented are sound and the product is fully responsive to the study charge, not whether they concur with the findings. The best National Academies’ products generate unique insights, provide clear and specific advice, and illustrate options that reduce uncertainty and facilitate decision making.

Reviewers may find the following guidelines to be useful, though they are encouraged to provide feedback on any improvements that could be made to the product. The manuscript should:

• Respond to all aspects of the Statement of Task but not go beyond it or the committee’s expertise.

• Have evidence, analysis, and argument to support its conclusions and recommendations.

• Acknowledge when there are uncertainties or incompleteness in the evidence.

• Be specific about any recommendations; that is, they should be directed at specific actors and fall within those actors’ authorities or capabilities. (Note that not all products have recommendations).

• Acknowledge if any conclusions or recommendations are based primarily on value judgments or the collective opinion of the committee, and if so, give adequate reasons for reaching those judgments.

• Maintain a tone of impartiality, consider alternative viewpoints, avoid advocacy, and treat sensitive issues with care.

• Be clear and easy to understand and communicate its key messages effectively, especially in the Summary.

In providing comments, reviewers are encouraged to distinguish issues they consider to be of general/major concern from other, less significant points. Minor editorial comments are not necessary, as products are professionally edited prior to publication.

The products of convening activities, called “Proceedings of a Workshop” or “Proceedings of a Workshop—in Brief,” reflect what transpired at the workshop and do not present the consensus view of the planning committee or workshop participants. The following questions are provided as a guide to reviewers.

• Given the Statement of Task and agenda for the workshop, how well does the introductory material explain the purpose and context for the proceedings? Does the proceedings include the items suggested by its introduction and, if necessary, does it explain what it does not cover?

• How well and how accurately does the content of the proceedings reflect the workshop presentations, discussions, and papers?

• Are all views expressed in the proceedings appropriately attributed to one or more individual participants at the workshop? Please identify any statements that might be misconstrued as reflecting consensus judgments of the participants.

• How balanced and fair is the presentation of material? Please identify passages, if any, in which sensitive policy issues have not been treated with proper care.

• Is the presentation suitable for readers who did not attend the workshop? How clear and comprehensive is the content of the proceedings to someone who was not present at the workshop?

• How well does the title reflect the content of the proceedings?