Henry Ledyard Goddard University Professor and Chair
Department of Biostatistics
Program in Public Health
and Co-Chair, Committee on Identifying the Needs of the Forensic Science Community
National Research Council
The National Academies
Committee on Commerce, Science, and Transportation
United States Senate
December 7, 2011
Chairman Rockefeller, Ranking Member Hutchison and Members of the Committee, thank you for the opportunity to submit written comments.
The NAS Committee on Identifying the Needs of the Forensic Science Community examined both the science and the practice in the forensic disciplines across the country. The Committee’s report, titled Strengthening Forensic Science in the United States, covered a broad range of challenges for forensic science, from disparities in resources, facilities and training across the country’s jurisdictions; to lack of mandatory standardization, certification, and accreditation; to the uneven development of the broad range of forensic disciplines; to political realities and evidence admissibility issues.
In my comments below I will concentrate on the state of the forensic science, which is characterized by the report as one of variable development across the forensic disciplines and low or non-existent research activity and infrastructure in many disciplines. Much research is needed not only to evaluate the reliability and accuracy of current forensic methods but also to innovate and develop them further. In order to achieve these goals on a national scale, an organized and well supported forensic science research enterprise is a key requirement.
Forensic science is an inherently multidisciplinary endeavor, utilizing methods and techniques from other scientific areas, such as molecular biology and analytic chemistry, as well as methods developed within the forensic communities, such as the analysis of patterns from fingerprints and handwriting. The forensic science disciplines conduct analyses and are asked to provide information for a variety of purposes in the criminal justice process. Broadly speaking, the questions they address can be divided in two categories:
a.) Can a piece of evidence be associated with a particular class of sources?
For example, can a hair specimen collected at the crime scene be reliably said to come from an individual of a particular ethnic group? Is a paint mark left at a crime scene consistent with the paint used in type of car defined by model and production year? Does a powder cargo contain cocaine?
b.) Can a piece of evidence be associated with an individual source?
For example, can a particular DNA sample be reliably said to belong to individual X?
The first category of questions leads to classification conclusions. The second leads to individualization conclusions. Although the goal of criminal investigations and trials is typically to assess the innocence or guilt of specific individuals, answers to both categories of questions are valuable. For example, classifying a piece of evidence may lead to decisions to exclude individuals from further consideration in the particular investigation. Moreover, the accuracy and overall performance of a forensic method should be judged only against the question it is called to address. Thus, analyses that can lead to classification should be evaluated on the basis of how correctly they classify and not on the basis of whether they can match a piece of evidence to a specific individual. This point may seem straightforward but lies at the root of many common misconceptions about the proper role of specific forensic analyses.
As with all scientific methods, it is important to assess the probability of various errors that can be made in the course of a forensic analysis. In particular, we need to study the frequency of how often the analysis can identify the source of the information correctly and how often errors will be made. Borrowing terminology from diagnostic medicine, we need to know the sensitivity of an analysis (probability that the analysis will identify a trait when it is actually present) and the specificity (probability that the analysis will declare the trait is absent when it is actually absent). The complements of these two quantities represent the rates of two common types of errors. Other measures of performance such as the positive and the negative predictive value can also be useful to analysts. A more detailed discussion is presented in Chapter 4 of the report.
A broad array of forensic disciplines is called upon to provide evidence in support of one or the other, or sometimes both categories of conclusions (classification and individualization). In Chapter 5 of the report the Committee presents a précis of each of the main disciplines, intended to summarize the state of their scientific underpinning and development, the way in which evidence is reported and used in investigations and court proceedings, and an assessment of current research and educational activity and needs for further development.
A key finding of the Committee was the wide variability across forensic science disciplines with regard to the techniques and methodologies used, the reliability of results, the types and frequencies of errors that occur, the soundness of the research base, the general acceptability of the discipline, and the availability of published peer reviewed research. Some of the forensic disciplines are rooted in traditional science. For example, DNA was developed in molecular biology and substance identification uses techniques from analytic chemistry. Such methods are generally on solid ground because the validity of those methods has been established scientifically through past and ongoing research, there is good understanding of uncertainties in their conclusions, and there is continuing development of their methodology. If they are executed according to the principles of science, they can be very reliable.
A number of other disciplines have been developed within forensic science, often with little input from the broader world of science. The goal of these analyses is to link a pattern from a crime scene—which may be a latent fingerprint impression, markings on a spent bullet, patterns from a fire, blood-spatter patterns, and so on—with analogous patterns from a weapon, tool, finger, etc., associated with a suspect.
In terms of the reliability and accuracy in making individualization conclusions, it is fair to say that, with the exception of nuclear DNA analysis, there is a lot we do not know about other forensic disciplines. Considerably more research and development is needed to provide a rigorous evaluation of the capacity of a method to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source. Such conclusions may be possible, but at present we simply do not have enough basic understanding to know. In terms of the reliability and accuracy in making classification conclusions, a number of forensic analysis methods show promise. However, even for classification analyses, there is only a modest amount of available research and systematic evaluation.
An unfortunate corollary of the low level of research and evaluation in many of the forensic disciplines is a tendency to consider and present the results of analyses as free from error. Such a disposition would be unthinkable in the context of scientific research and practice. It is therefore imperative to foster, encourage, and ultimately require the adoption and continued development of scientific methods and practices across the forensic disciplines. A body of research is required to assess the accuracy and reliability of analyses in many of the forensic disciplines and to address the impact of sources of variability and potential bias. These disciplines need to develop rigorous protocols to guide subjective interpretations and pursue equally rigorous research and evaluation programs.
The development of scientific research and a scientific culture in the forensic disciplines is not needed to evaluate currently used methods and practices. It is indeed a precondition for the evolution of these disciplines and for the development of new methods that address the evolving needs of the legal system.
In parallel to an analysis of the science of the forensic disciplines, the Committee undertook an examination of the practice in such disciplines across the country. As described in the report, there are great disparities among existing forensic science operations in federal, state, and local law enforcement jurisdictions and agencies. This is true with respect to funding, access to analytical instrumentation, the availability of skilled and well-trained personnel, and certification, accreditation, and oversight. As a result, it is not easy to generalize about current practices within the forensic sciences community. It is clear, however, that any approach to overhauling the existing forensic science system needs to address and help minimize the community’s current fragmentation and inconsistent practices.
The fragmentation problem is compounded because operational principles and procedures for many forensic disciplines are not standardized or embraced, either between or within jurisdictions. There is no uniformity in the certification of forensic practitioners or in the accreditation of crime laboratories. Indeed, many jurisdictions do not require forensic practitioners to be certified, and many forensic science disciplines have no mandatory certification programs. Moreover, the accreditation of crime laboratories is not required in most jurisdictions. Often, there are no standard protocols governing forensic practice in a given discipline. And, even when protocols are in place, they may be vague and not enforced in any meaningful way. In short, the quality of forensic practice in most disciplines varies greatly because of the absence of adequate training and continuing education, rigorous mandatory certification and accreditation programs, adherence to robust performance standards, and effective oversight. These shortcomings obviously pose a continuing and serious threat to the quality and credibility of forensic science practice.
I will close with a review of the Committee’s recommendations. The Committee’s major recommendation is that Congress should establish and appropriate funds for an independent federal entity, the National Institute of Forensic Sciences, or NIFS. Such a federal body will 1) bolster our ability to more accurately identify true perpetrators and exclude those who are falsely accused; 2) improve our ability to effectively respond to, attribute, and prosecute threats to homeland security; and 3) reduce the likelihood of convictions resting on inaccurate data.
In addition to this major recommendation, the Committee offers several additional specific recommendations regarding the separation of forensic science from law enforcement, addressing training and educational needs, improving certification and accreditation requirements, reforming the medicolegal death investigation system, creating interoperable fingerprint databases, and enhancing the role and quality of the forensic sciences in homeland security.
- Recommendation #2 highlights the need for standardized terminology and reporting of the results of forensic analyses.
The National Institute of Forensic Science (NIFS), after reviewing established standards such as ISO 17025, and in consultation with its advisory board, should establish standard terminology to be used in reporting on and testifying about the results of forensic science investigations. Similarly, it should establish model laboratory reports for different forensic science disciplines and specify the minimum information that should be included. As part of the accreditation and certification processes, laboratories and forensic scientists should be required to utilize model laboratory reports when summarizing the results of their analyses.
- Recommendation #3 addresses research needs in the forensic sciences.
Research is needed to address issues of accuracy, reliability, and validity in the forensic science disciplines. The National Institute of Forensic Science (NIFS) should competitively fund peer-reviewed research in the following areas:
(a) Studies establishing the scientific bases demonstrating the validity of forensic methods.
(b) The development and establishment of quantifiable measures of the reliability and accuracy of forensic analyses. Studies of the reliability and accuracy of forensic techniques should reflect actual practice on realistic case scenarios, averaged across a representative sample of forensic scientists and laboratories. Studies also should establish the limits of reliability and accuracy that analytic methods can be expected to achieve as the conditions of forensic evidence vary. The research by which measures of reliability and accuracy are determined should be peer reviewed and published in respected scientific journals.
(c) The development of quantifiable measures of uncertainty in the conclusions of forensic analyses.
(d) Automated techniques capable of enhancing forensic technologies.
- Recommendation #4 urges independence of forensic laboratories from law enforcement and prosecutorial offices.
To improve the scientific bases of forensic science examinations and to maximize independence from or autonomy within the law enforcement community, Congress should authorize and appropriate incentive funds to the National Institute of Forensic Science (NIFS) for allocation to state and local jurisdictions for the purpose of removing all public forensic laboratories and facilities from the administrative control of law enforcement agencies or prosecutors’ offices.
- Recommendation #5 emphasizes the need for assessing and minimizing bias and human error.
The National Institute of Forensic Science (NIFS) should encourage research programs on human observer bias and sources of human error in forensic examinations. Such programs might include studies to determine the effects of contextual bias in forensic practice (e.g., studies to determine whether and to what extent the results of forensic analyses are influenced by knowledge regarding the background of the suspect and the investigator’s theory of the case). In addition, research on sources of human error should be closely linked with research conducted to quantify and characterize the amount of error. Based on the results of these studies, and in consultation with its advisory board, NIFS should develop standard operating procedures (that will lay the foundation for model protocols) to minimize, to the greatest extent reasonably possible, potential bias and sources of human error in forensic practice. These standard operating procedures should apply to all forensic analyses that may be used in litigation.
- Recommendation #6 addresses the need for uniform standards and adoption of best practices in forensic laboratories across the country.
To facilitate the work of the National Institute of Forensic Science (NIFS), Congress should authorize and appropriate funds to NIFS to work with the National Institute of Standards and Technology (NIST), in conjunction with government laboratories, universities, and private laboratories, and in consultation with Scientific Working Groups, to develop tools for advancing measurement, validation, reliability, information sharing, and proficiency testing in forensic science and to establish protocols for forensic examinations, methods, and practices. Standards should reflect best practices and serve as accreditation tools for laboratories and as guides for the education, training, and certification of professionals. Upon completion of its work, NIST and its partners should report findings and recommendations to NIFS for further dissemination and implementation.
- Recommendation #7 stresses the need for mandatory accreditation and certification.
Laboratory accreditation and individual certification of forensic science professionals should be mandatory, and all forensic science professionals should have access to a certification process. In determining appropriate standards for accreditation and certification, the National Institute of Forensic Science (NIFS) should take into account established and recognized international standards, such as those published by the International Organization for Standardization (ISO). No person (public or private) should be allowed to practice in a forensic science discipline or testify as a forensic science professional without certification. Certification requirements should include, at a minimum, written examinations, supervised practice, proficiency testing, continuing education, recertification procedures, adherence to a code of ethics, and effective disciplinary procedures. All laboratories and facilities (public or private) should be accredited, and all forensic science professionals should be certified, when eligible, within a time period established by NIFS.
- Recommendation #8 calls for uniform quality control and quality assurance programs.
Forensic laboratories should establish routine quality assurance and quality control procedures to ensure the accuracy of forensic analyses and the work of forensic practitioners. Quality control procedures should be designed to identify mistakes, fraud, and bias; confirm the continued validity and reliability of standard operating procedures and protocols; ensure that best practices are being followed; and correct procedures and protocols that are found to need improvement.
- Recommendation #9 calls for a national code of ethics for forensic scientists.
The National Institute of Forensic Science (NIFS), in consultation with its advisory board, should establish a national code of ethics for all forensic science disciplines and encourage individual societies to incorporate this national code as part of their professional code of ethics. Additionally, NIFS should explore mechanisms of enforcement for those forensic scientists who commit serious ethical violations. Such a code could be enforced through a certification process for forensic scientists.
- Recommendation #10 calls for major emphasis on graduate education in the forensic sciences.
To attract students in the physical and life sciences to pursue graduate studies in multidisciplinary fields critical to forensic science practice, Congress should authorize and appropriate funds to the National Institute of Forensic Science (NIFS) to work with appropriate organizations and educational institutions to improve and develop graduate education programs designed to cut across organizational, programmatic, and disciplinary boundaries. To make these programs appealing to potential students, they must include attractive scholarship and fellowship offerings. Emphasis should be placed on developing and improving research methods and methodologies applicable to forensic science practice and on funding research programs to attract research universities and students in fields relevant to forensic science. NIFS should also support law school administrators and judicial education organizations in establishing continuing legal education programs for law students, practitioners, and judges.
- Recommendation #11 calls for the establishment of medical examiner offices across the country and the eventual elimination of existing coroner offices.
To improve medicolegal death investigation:
(a) Congress should authorize and appropriate incentive funds to the National Institute of Forensic Science (NIFS) for allocation to states and jurisdictions to establish medical examiner systems, with the goal of replacing and eventually eliminating existing coroner systems. Funds are needed to build regional medical examiner offices, secure necessary equipment, improve administration, and ensure the education, training, and staffing of medical examiner offices. Funding could also be used to help current medical examiner systems modernize their facilities to meet current Centers for Disease Control and Prevention recommended autopsy safety requirements.
(b) Congress should appropriate resources to the National Institutes of Health (NIH) and NIFS, jointly, to support research, education, and training in forensic pathology. NIH, with NIFS participation, or NIFS in collaboration with content experts, should establish a study section to establish goals, to review and evaluate proposals in these areas, and to allocate funding for collaborative research to be conducted by medical examiner offices and medical universities. In addition, funding, in the form of medical student loan forgiveness and/or fellowship support, should be made available to pathology residents who choose forensic pathology as their specialty.
(c) NIFS, in collaboration with NIH, the National Association of Medical Examiners, the American Board of Medicolegal Death Investigators, and other appropriate professional organizations, should establish a Scientific Working Group (SWG) for forensic pathology and medicolegal death investigation. The SWG should develop and promote standards for best practices, administration, staffing, education, training, and continuing education for competent death scene investigation and postmortem examinations. Best practices should include the utilization of new technologies such as laboratory testing for the molecular basis of diseases and the implementation of specialized imaging techniques.
(d) All medical examiner offices should be accredited pursuant to NIFS-endorsed standards within a timeframe to be established by NIFS.
(e) All federal funding should be restricted to accredited offices that meet NIF-endorsed standards or that demonstrate significant and measurable progress in achieving accreditation within prescribed deadlines.
(f) All medicolegal autopsies should be performed or supervised by a board certified forensic pathologist. This requirement should take effect within a timeframe to be established by NIFS, following consultation with governing state institutions.
- Recommendation #12 stresses the need to achieve interoperability of fingerprint data systems across the country.
Congress should authorize and appropriate funds for the National Institute of Forensic Science (NIFS) to launch a new broad-based effort to achieve nationwide fingerprint data interoperability. To that end, NIFS should convene a task force comprising relevant experts from the National Institute of Standards and Technology and the major law enforcement agencies (including representatives from the local, state, federal, and, perhaps, international levels) and industry, as appropriate, to develop:
(a) standards for representing and communicating image and minutiae data among Automated Fingerprint Identification Systems. Common data standards would facilitate the sharing of fingerprint data among law enforcement agencies at the local, state, federal, and even international levels, which could result in more solved crimes, fewer wrongful identifications, and greater efficiency with respect to fingerprint searches; and
(b) baseline standards—to be used with computer algorithms—to map, record, and recognize features in fingerprint images, and a research agenda for the continued improvement, refinement, and characterization of the accuracy of these algorithms (including quantification of error rates).
- Finally, Recommendation #13 calls for preparedness of forensic scientists and laboratories to address homeland security needs.
Congress should provide funding to the National Institute of Forensic Science (NIFS) to prepare, in conjunction with the Centers for Disease Control and Prevention and the Federal Bureau of Investigation, forensic scientists and crime scene investigators for their potential roles in managing and analyzing evidence from events that affect homeland security, so that maximum evidentiary value is preserved from these unusual circumstances and the safety of these personnel is guarded. This preparation also should include planning and preparedness (to include exercises) for the interoperability of local forensic personnel with federal counterterrorism organizations.
In the two years since the release of the report I have seen a lot of interest in its content and recommendations. However I have not seen major progress in implementing any of them. Specifically with respect to the first and most central recommendation, I understand that the current fiscal environment makes the establishment and funding of a new federal agency challenging. Short of this, I think there is much Congress could still do and I urge you not to allow the current fiscal environment to be a reason to undertake any forensic science reform. As one example, I believe that the National Institute of Standards and Technology (NIST) could serve as an incubator for NIFS as long as Congress acts in several years to make NIFS a fully independent agency. I note that this is the position of the American Statistical Association (ASA), of which I am a fellow: http://www.amstat.org/outreach/pdfs/RockefellerForensicScience.pdf. I also note the ASA Board of Directors approved a statement endorsing the Strengthening Forensic Science report: http://www.amstat.org/outreach/pdfs/Forensic_Science_Endorsement.pdf.
In closing, I would summarize the Committee’s work by saying that the Committee studied the science and practice of the forensic disciplines in the country and decided that a major buildup of the scientific enterprise and a massive overhaul of the forensic system are needed in order to meet the needs of the country, current and future.
An archived webcast of the hearing can be found on the Senate Commerce, Science, and Transportation Committee's Web site.