Appendix D

Approach to Literature Review

LITERATURE SEARCH STRATEGY

To identify primary literature and review the most relevant scientific publications available, staff initially conducted general searches on topics relevant to subject areas identified in the committee’s statement of task. Using the results of this primary search, staff then developed key search terms based on relevance to the study objectives. Searches were limited to English-language publications. After the initial search, staff designed a comprehensive search strategy in consultation with librarians at the George E. Brown Jr. Library of the National Academies. Search terms incorporated relevant MeSH (Medical Subject Headings) terms, as well as terms from the EMBASE thesaurus. Data-bases searched included Academic Search Premier, Agricola, Congressional Research Service, EconLit, ERIC, MEDLINE, NTIS, PsycINFO, and Web of Science (Science Citation Index and Social Science Citation Index). Database searches also included publications from the National Bureau of Economic Research and the U.S. Department of Agriculture’s Economic Research Service. Table D-1 provides an example of how the searches were conducted; only a subset of terms from the overall search are shown because including the entire search was impractical.

Staff limited the searches to publications dated 2000 and later. The initial search retrieved more than 1,100 citations, which were then sorted into predefined topics identified by the committee. The topical search terms included

•   access to food,

•   cost of food,

TABLE D-1 Example of Searches Using Key Words to Identify Relevant Literature

•   economic insecurity,

•   food choice and cultural preferences,

•   food policy,

•   health and nutrition effects, and

•   program access.

ORGANIZATION OF THE LITERATURE

Relevant references obtained from the initial search were screened and categorized according to the research taxonomy shown in Box D-1. The committee organized the publications obtained from its search into three focus areas:

1.   content area,

2.   design of research study, and

3.   source of paper/publication.

The topics within each focus area of the taxonomy were then expanded. Key citations were selected by committee members and annotated by staff. Reference lists of key citations were provided in tabulated format to facilitate the committee’s review and selection of critical publications for inclusion in this report.

EVALUATION OF THE LITERATURE

To evaluate the results of the literature search, the committee first used the organizational scheme described above to consider (1) the validity and (2) the generalizability of the studies and publications identified in its research taxonomy. Although the committee appreciates the range of evidence types and the strengths and weaknesses of various methodological approaches, it determined that the most useful approach was to examine research questions relevant to its statement of task with observational studies conducted in a real-world context. From this perspective, the quantitative approach of the randomized controlled trial was deemed limited because of the use of an artificial setting and the need to control variables that may have application to understanding and interpreting consumer behavior. Relevant experimental studies were not identified in the search and thus were not included in the evidence review.

The types of research studies that provided the most valid evidence for examining the behavior of participants in food assistance programs were field experiments and survey-based studies. The committee identified a range of observational evidence, including population-based surveys and impact studies based on secondary data analysis, that reflected the complexity of variables relevant to the outcomes of interest, as well as the generalizability of the evidence to pertinent research questions. The committee considered the merits of these studies on the basis of the methodological approach used and the overall quality of the research. Additional support for the findings from observational studies was obtained from evidence-based reviews, meta-analyses, and qualitative studies.

Finally, in addition to the body of peer-reviewed evidence from journal publications, the committee considered peer-reviewed evidence from government reports, in particular, relevant reports from the U.S. Department of Agriculture’s (USDA’s) Economic Research Service, as well as nonreviewed publications from stakeholder and nongovernmental organizations. Although government reports undergo a rigorous peer review, the process differs from that for peer-reviewed journal articles in that the review is not blinded. Specifically, the Office of Management and Budget Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies state: “In general, an agency conducting a peer review of a highly influential scientific assessment must ensure that the peer review process is transparent by making available to the public the written charge to the peer reviewers, the peer reviewers’ names, the peer reviewers’ report(s), and the agency’s response to the peer reviewers’ report(s).”¹ While

____________________

¹Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, Federal Register, Volume 67, Issue 24, p. 5465 (February 5, 2002).

the committee did not give these types of evidence equal weight with peer-reviewed journal publications, it took them into account as part of the totality of evidence because of the additional insight they provided into the behavioral aspects of participation in food assistance programs.

This page is blank