An Educational Module

Elizabeth Fisher
Professor of Environmental Law
Faculty of Law and Corpus Christi College, Oxford University

Pasky Pascual
U.S. Environmental Protection Agency 
 
Wendy Wagner
Joe A. Worsham Centennial Professor
 University of Texas at Austin School of Law

This module (and associated instructors’ guide) is an introduction to models and to their role in regulatory decision-making. It has been explicitly designed for non-scientists from a variety of different backgrounds including law, journalism, public policy, and business. The module covers three main themes: 

1. Core lessons on using models for policy making including what models are and what they are used for;

2. Key questions and ideas on the art and science of modeling including how models are used to establish causation and how models differ from other scientific techniques;

3. Key ideas on using models in policy-making including how models are used by agencies and the legal and policy issues they raise.

The module draws on scientific literature, regulatory documentation, newspaper articles, legal material, and video clips. The models we discuss relate to many different things – climate change, air quality, and even baseball results.  The material’s diversity reflects both the importance of models and that they often operate against a background of controversy.

The module has been created to be used in a variety of different learning environments ranging from instructor led teaching formats to environments where there is heavy use of student participation including break out groups. Throughout the module, students are guided by a roadmap, and there are checkpoints which can either be used for class discussion or independent student learning. The first part (Sections 1-4) of the module introduces students to the basic concepts of models.  The second part (Sections 5-6) then locates this understanding within applied legal and policy settings and includes a more detailed case study of the EPA’s use of models to set ambient air quality standards (Section 6).  If a course is not particularly policy-oriented, the final Section – Section 6: Case Study on Air Quality Standards – could be omitted, leaving more time for student engagement with the basic concepts and material.  

Module (PDF)
Module (DOCX)

An Educational Module

David H. Kaye
Associate Dean for Research and Distinguished Professor 
The Pennsylvania State University (Penn State Law)

DNA profiling has been profoundly disruptive of business as usual in forensic science and criminal justice. It spawned an innocence movement dedicated to correcting and preventing wrongful convictions, and it dramatically changed the investigation and prosecution of sexual assault cases. It sparked intense controversy among statisticians and population geneticists and ultimately stimulated efforts toward overtly probabilistic forensic identification for more complicated pattern and impression evidence.

This educational module on DNA evidence presents trial testimony, exhibits, and opinions in a case in which federal courts at every level discerned “inaccuracies” in the testimony of a leading expert about probabilities associated with the DNA evidence. By embedding these legal materials in background explanations, critical questions, and short problems, the module supports self-study and class discussions that together can elucidate key principles in scientific reasoning and quantitative analysis and that can help students avoid a classic and common misunderstanding about conditional probabilities. More specifically, the module includes information and exercises relating to the following scientific and statistical principles and skills:

Sampling Theory

Defining a relevant population
Spotting sources of bias and uncertainty
Interpreting confidence intervals

Modeling Profile Probabilities

Testing assumptions 
Sensitivity analysis

Probability Theory

The concept of mathematical probability
Probabilities of events such as a DNA match to a randomly selected individual
Independent and dependent events
Conditional probability and the transposed conditional
Likelihood ratios for pairs of hypotheses as a measure of the probative value of DNA matches and other scientific test results
Bayes’ rule for deducing a posterior probability from a likelihood ratio and a prior probability

The module was constructed with law students in mind, but the materials and questions are not intended for the traditional legal exercise of extracting the holdings of the courts and assessing the legal arguments for and against them. Rather, the goal is to give students in all fields the analytical tools needed to evaluate the presentation of the scientific evidence in the case. The topics of sampling, weight of evidence, and updating probabilities in response to new scientific evidence are of obvious importance in the domains of medicine, public policy, and business as well as law.

The most efficient way to cover the materials in class is with a minimum of lecturing and a maximum of class discussion and exercises such as a simulated cross-examination of the expert witness in the case. An instructor's manual offers detailed suggestions. In law schools, the module could be employed in advanced courses or seminars that cover expert evidence, scientific evidence, law and science, or law and genetics. The case materials also could be the basis for a component of a trial advocacy course involving expert witnesses.  

Module (PDF)
Module (DOCX)

An Educational Module

Paul Brest
Former Dean and Professor Emeritus (active)
Stanford Law School

The materials in this module are mainly prescriptive, focusing on how to make good decisions. But they also include examples of the systematic ways that people make poor decisions— usually a consequence of decision makers’ overconfidence in their judgment or in errors in perceiving risks. The prescriptive materials, drawn from statistics and economics, introduce some basic techniques of “decision science.”

The module has been developed for use in a short stand-alone seminar or as an adjunct to a course or seminar that studies the policy implications of various natural science, social science, and health disciplines.

The module:

1) surveys the meanings of utility;
2) examines decision making in conditions where risk is not an issue;
3) examines decision making in conditions where risk is an issue because the likelihood of success is not assured; and
4) examines errors and biases that can affect the decision-making processes.   

NOTE: The current version of the module is a January 2017 revision of the module originally published in June 2016.  The current versions of the Instructor's Guide and Ancillary Materials are April 2017 revisions of the materials originally published in June 2016.

Module (PDF)
Module (DOCX)
Instructor's Guide (PDF)
Instructor's Guide (DOCX)
Ancillary Materials (PDF)
Ancillary Materials (PPTX)

An Educational Module

Kevin W. Sharer
Senior Lecturer
Harvard Business School
Harvard University

This module is intended for use by business school students. It examines the decision-making process at a pharmaceutical company as its chief operating officer decides whether to invest in the development and licensing of a promising treatment for Parkinson’s disease.  The case elucidates under conditions of uncertainty the relationship between science/engineering and business opportunities so as to illustrate how an understanding of the science and engineering can reduce risk and enhance opportunity in business.

The module is structured around a drug development case that provides students with opportunities to:

1) analyze a rich and realistic description of the complex scientific and medical results associated with a promising therapeutic molecule and to consider whether the molecule warrants further investment;
2) examine the failure of clinical trials and consider why new trials utilizing a different delivery mechanism might succeed;
3) consider how a bid is structured and examine the trade-offs involved when making bid element choices; and
4) construct a best-case scenario for the further drug development

 The module will enable business students to:

• Analyze scientific and medical results collected over time in order to recommend a future course of action 

• List reasons why past drug trials failed and why a new trial might succeed

• Compare and contrast different ways in which a company might structure a bid, identifying the strengths and weaknesses of each

• Prepare a best-case drug development scenario that includes information about investments, operational decisions, pricing, regulations, and marketing plans

• Describe Parkinson’s disease, including who it affects and how it affects them

• Describe GDNF (glial cell-derived neurotrophic factor) and its relationship to Parkinson’s disease 

• Draw a time line for events in the history of GDNF

• Name the steps of the drug development process

• Summarize the findings of nonhuman studies of GDNF 

• Summarize the findings of a randomized, double-blind trial

Module (PDF)
Module (DOCX)

An Educational Module Using Three Case Studies and Three Scientific Concepts

John D. Graham
Dean, School of Public and Environmental Affairs
Indiana University

John A. Rupp
Adjunct Instructor, School of Public and Environmental Affairs, 
and Senior Research Scientist, Indiana Geological Survey
Indiana University 

Adam V. Maltese
Associate Professor of Science Education, School of Education, 
and Adjunct Faculty in Department of Geological Sciences
Indiana University

This module has been developed around the topic of the extraction of natural gas from shale. This practice, commonly referred to as fracking, involves several practices that have complex and uncertain consequences. In the module, three scientific concepts are explored in order to elucidate how science is applied toward addressing real-world problems. The scientific concepts are: 1) correlation is not the same as causation, 2) hazard is not the same as risk and 3) risk assessment includes the evaluation of probability of occurrence and severity of negative consequences.
 

Each of these concepts is explored with a case study that uses a problem-based learning approach to address the challenges surrounding the concept. The cases each include background materials from various sources, a student’s guide, and complementary instructor’s guide. Together, these materials are designed to stimulate thinking and guide inquiry using a scientific approach. Use of the materials and techniques provide non-science practitioners with an appreciation of the value of science in problem solving.

Additional details and descriptions of the cases are available in the introduction to the three cases. 

Module (PDF)
Module (DOCX)

An Educational Module

Nathan A. Schachtman
Lecturer in Law
Columbia Law School

This module is intended for the advanced undergraduate student or graduate student in a professional or graduate study of law, public policy, pharmacy, hospital pharmacy administration, drug development, and probably other areas as well. It does not pre-suppose any prior knowledge, and the materials can be sampled at different levels of sophistication. 

Using the currently topical issue of whether SSRI (Selective Serotonin Reuptake Inhibitor) anti-depressants cause birth defects, the module aims to provide students with a general understanding of the following:

• human risk factor causation as determined by epidemiologic methods, as seen through the case study of sertraline and cardiac birth defects;

• the limits of non-human toxicological evidence to the assessment of causality in humans;

• the importance of pharmacovigilance for all medications, but especially for the determination of harms to embryos and fetuses;

• special difficulties in identifying causes of human birth defects;

• the importance of ruling out chance, bias, and confounding, and other relevant considerations, in determining that an observed association is causal; and

• the role of the systematic review and meta-analysis in synthesizing judgments of causality from complex datasets.

There are other subsidiary goals embedded within these higher level goals, such as learning to read and evaluate the data from an epidemiologic study, recognizing the varying degree of rigor and validity of studies, and understanding the difficulties in synthesizing a judgment of causation from multiple studies with discordant results.  
 

Teaching the materials in this module will require five classes of 90 to 120 minutes. For students coming from pharmacy or public health programs, less time may be needed to present the fundamental aspects of epidemiology.  For law students, more time might be required. The assigned readings will consume at least as much time out of the classroom, as in the classroom.  Classes will involve directed discussions as well as lecture material.

Module (PDF)
Module (DOCX)

An Educational Module

Arturo Casadevall
Professor and Chair
W. Harry Feinstone Department of Molecular Microbiology and Immunology
Johns Hopkins University Bloomberg School of Public Health

This module engages students in learning about association and causation in the context of vaccines, their side effects, and legal issues that could arise as a result of side effects associated with vaccinations.

The module employs five case studies. In the first two case studies, a child receives a vaccination, and students must determine whether an event (vaccination) causes a side effect in the child.  In the third case study, a child who has not been vaccinated transmits a disease to another child whose vaccination likely “has not taken.”  The fourth case study involves a vaccine-related death and considers whether more should have been done to screen the patient.  The final case presents a hypothetical situation wherein an expanded use of a vaccine may be the causal factor in an increased prevalence of a certain disease. 

The module discusses side effects associated with vaccines and conditions which have been associated with the administration of vaccines, but which are not supported by scientific evidence.  The module illustrates the fact that association is not causation and explores how causation is established in the scientific realm. 

Discussion points are provided for the teacher and the student.  Some of the discussion points raise questions that do not have clear answers and are reflective of the type of decisions that patients and physicians must make when considering vaccines.  Thematically, the five cases are linked in that they seek to demonstrate causative links among a temporal series of events.

Module (PDF)
Module (DOCX)

An Educational Module

Simon A. Cole
Professor, Department of Criminology, Law, and Society
Director, Newkirk Center for Science and Society
University of California, Irvine 

Alyse Berthental
Ph.D. Candidate, Department of Criminology, Law, and Society
University of California, Irvine 

Jaclyn Seelagy
Scholar, PULSE (Program on Understanding Law, Science, and Evidence)
 University of California, Los Angeles School of Law
 

This module (and associated instructors’ guide) uses latent print (fingerprint) identification as a case study of the broader category of forensic pattern recognition evidence (which includes, e.g., firearms and toolmark comparison, questioned document analysis, shoeprint and tire track comparison). The case of forensic pattern recognition evidence is meant, in turn, to be a case study of the broader issue of the interaction between science and law and policy. The case of fingerprinting identification poses challenging  questions for legal professionals and policy-makers. Although fingerprint identification is one of the most widely used and widely trusted forms of forensic evidence, scientists and scientific institutions have questioned the scientific basis for it, as well as for forensic pattern recognition more generally. As Professor Michael Saks commented in 1998, “Fingerprint evidence may present courts applying Daubert[the legal case governing the admissibility of scientific evidence in federal courts] with their most extreme dilemma.”

The module has been explicitly designed for non-scientists from a variety of different backgrounds including law and public policy. The module draws on scientific and legal literature, policy documents, and court exhibits and opinions. The module is designed around an active learning approach. Each of the five proposed classes asks students to solve a problem, and deliver a product, such as a judicial opinion or a policy recommendation. These exercises and the provided background material are designed to invite students to confront the challenging epistemological, legal, and policy issues raised by fingerprint evidence. In so doing, students will be asked to understand and apply a variety of scientific competencies, such as validation testing, probabilistic reasoning, the quantification of uncertainty, the “prosecutor’s fallacy,” likelihood ratios, Bayes' theorem, decision theory, the nature of scientific consensus, and the role of scientific learned societies. Student learning will then be measured by assessment questions.

The module consists of five classes. The module is designed so that one or more classes may be omitted in order to meet the needs of the specific learning environment. The first class introduces students to the concept of validation testing by asking them to articulate the empirical questions relevant to fingerprint analysis and the empirical evidence that would be necessary to support that claim. The second class introduces the problem of legal admissibility of expert evidence and the epistemological bases behind different approaches to that problem and invites students to apply that problem to fingerprint evidence. The third and fourth classes introduce the problem of reporting forensic results and two approaches to that problem (hypothesis testing and likelihood ratio/decision theory). The fifth class applies all of this accumulated knowledge to legal and policy reform.

Module (PDF)
Module (DOCX)

An Educational Module

Steve C. Gold
Professor of Law
Rutgers School of Law-Newark

Michael D. Green
Williams Professor of Law
Wake Forest University School of Law 

Joseph Sanders
A.A. White Professor of Law
University of Houston Law Center

This module examines three scientific areas that provide evidence bearing on causation in the “toxic tort” or environmental disease context: epidemiology, toxicology, and genetics. These scientific disciplines are used in civil lawsuits and in regulatory proceedings in which causation or risk is an issue. The module is appropriate for non-scientist law students as well as others interested in learning the science of toxic tort causation, including practicing attorneys, judges, and public policy and public health students in the United States and elsewhere. It is designed to provide the following core competencies: 

1. An understanding of the concept of factual causation and how evidence bearing on that matter is brought to bear in judicial proceedings and the regulatory context.

2. How the results of an observational epidemiologic study bear on factual causation and the methods and limitations of studies in providing proof of causation.

3. Outcome measures of relationships found in epidemiologic studies, the role of random chance, bias and confounding in skewing those relationships, and the process of determining whether a correlation is causal or spurious. 

4. The difference between epidemiology and toxicology and the latter’s focus on whether and how a suspected toxic substance affects genes, cells, tissues, organs, and organisms in causing disease. Students should acquire an understanding of the mechanisms of how disease is caused and the various areas and types of toxicologic research. Students should obtain an understanding of the basics of extrapolating from various types of toxicology research to questions of causation in humans that are at the heart of legal inquiries.

5. An understanding of the legal issue of specific causation—whether the agent caused a given plaintiff’s disease. Students should obtain an understanding of how courts use, and sometimes misuse, epidemiologic outcomes as a way to answer specific causation questions. In addition, as genetic information increasingly is used to refine the specific causation question, student should gain an introductory understanding of how toxicogenomics and the use of biomarkers help address specific causation.  They should come to understand that currently the results of this area of science help to refine the probability of causation but rarely offer a clear answer to specific causation questions.

The module is designed for use in a classroom context with an instructor assisting students to absorb and understand the material in the module. A variety of slides designed to assist in this effort are included in the module along with study questions for class discussion. Extensive teaching notes guide the instructor and offer advice about how to present the material in class. The module contains descriptive material, figures and tables, abstracts and scientific studies, and legal opinions. A comprehensive hypothetical is provided for student assessment.

Module (PDF)
Module (DOCX)
Instructor's Guide (PDF)
Instructor's Guide (DOCX)
Ancillary Materials (PDF)
Ancillary Materials (PPTX)