6

Moving Forward: Priorities for Research and Funding

The final session of the workshop featured a roundtable conversation about research gaps and funding needs for emerging forensic DNA technologies; the session was moderated by Michael Coble, University of North Texas Center for Human Identification. Coble asked panelists questions and invited workshop participants to share questions and comments.

HIGH-PRIORITY ACTIONABLE ISSUES

John Butler, National Institute of Standards and Technology, began the session by identifying several issues that had come up repeatedly at the workshop as key areas for further research and work; he noted that many of these were also identified in a Government Accountability Office (2021) report about forensic technology (see Box 6-1). These high-priority areas include

• increased training for law enforcement analysts and investigators to improve the consistent and objective use of forensic algorithms and understanding results;
• development and implementation of standards and policies;
• increased transparency related to the testing and performance of forensic algorithms to improve public trust;
• research on how to best communicate uncertainty in results; and
• independent verification and validation of software used in forensic DNA testing.

Disparate Impacts of Technology

Coble noted that the executive order that prompted this workshop emphasized the need to eliminate racial disparities in the criminal justice system and to strengthen bonds between law enforcement and communities. He asked panelists to comment on the potential disparate impact of emerging technologies, in particular the impact on communities that already experience fractured trust with law enforcement. Daphne Martschenko, Stanford University, responded that the community most impacted by an emerging technology often depends on the technology itself. For example, new treatments for rare genetic disorders primarily impact affluent White people. Other technologies, such as those used in law enforcement, tend to primarily impact communities that are historically underserved or exploited. One overarching pattern, however, is that the people who “shoulder the burden” of uncertain technologies are most often those who are already in vulnerable positions. Returning to the example of treatments for rare genetic disorders, the patients who take on the risks of uncertain new treatments are those who are in the vulnerable position of having a serious medical condition that has no approved treatment. Law enforcement technologies disproportionately impact Black and Brown communities that are vulnerable due to societal structures and racism. Given the different impacts of emerging technologies on different communities, said Martschenko, it is critical that assessments of risks and benefits include the voices of these communities.

Speakers at the workshop repeatedly described trade-offs between different risks and benefits—for example, independent verification of software versus the protection of trade secrets. These assessments are part of the implementation of any new technology, Martschenko said, and it is essential to consider who is defining the risks and benefits, what value they place on these risks and benefits, and how other stakeholders might view them differently. Matthews agreed and said that verification for law enforcement software usually involves weighing the risks to the developers against the risks to those “being decided about”—that is, the people involved or potentially involved in the criminal justice system. Matthews said that the criminal justice system should prioritize fairness to those being decided about. Our current system does not usually prioritize that population, she said; one reason is because errors in the system can be difficult to detect or acknowledge. If a plane crashes in part because of a software bug, there is an acknowledgment that an error has occurred. But if an individual is arrested and convicted in part because of a bug, the report of an error may be dismissed as simply the complaint of a guilty person. In this context, there is limited incentive for the iterative improvement of software, said Matthews.

Reliability

A priority area for research, said Coble, is ensuring the reliability of forensic DNA technologies. He asked panelists how reliability should be defined and how stakeholders could collaborate to support research on reliability. Butler said that reliability can be defined as trustworthiness, and it can be measured by the degree to which results are consistently accurate. Many people consider direct matching with a single-source DNA sample to be definitive and low risk, he said, but as technologies move out farther away from this traditional model, it becomes less clear what data the methods are based on and how reliable they are. Butler pointed to the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine (2009) publication On Being a Scientist: A Guide to Responsible Conduct in Research, and said that a similar primer could be created for forensic DNA technologies.

Regarding facilitating collaboration between researchers, Martschenko said that this workshop has provided a valuable setting for cross-discipline conversations. Translating these conversations into action, however, is more difficult, and may require funding and academic incentives to encourage researchers to explore this area.

Funding as Leverage

Following on the topic of funding, Erin Murphy, New York University, said that federal funding has played an indispensable role in building the national network of databases and shaping what data are collected and what research is conducted. Funding can be a powerful lever for shifting focus through research funding and grants for casework, he said. Around $22 million was directed at clearing backlogs in 2022; these grants could have included conditions that reflected the best practices discussed at the workshop. For example, a grant for cold-case forensic investigative genetic genealogy work could include conditions that prohibit the generation of a single composite photo or that require the use of laboratories with accessible and transparent data.

Existing Frameworks

Coble asked panelists to identify fields, frameworks, or methodologies that could be useful in ensuring effective, ethical, and equitable use of these emerging technologies. Jeanna Matthews, Clarkson University, emphasized that software is increasingly being used in all disciplines, and requiring independent verification and validation is critical. She explained that the Institute of Electrical and Electronics Engineers’ Standard 1012 is widely

accepted throughout the U.S. government and should be used for any forensic software as well (see Box 3-4 in Chapter 3). Murphy said that with the use of these new technologies, forensic science should consider the ethical frameworks in fields such as health care.

Drawing from her professional experience, Martschenko explained that bioethicists have spent time considering the ethical and psychological impacts of revealing genetic information, and they could be useful in helping to determine where law enforcement should focus its efforts. Murphy explained that, in a related study that looked at 11,000 backlogged rape kits, researchers looked to other ethical frameworks to consider how reopening a decades-old case might impact the victim (Campbell et al., 2015). Martschenko then pointed to a program of the National Human Genome Research Institute called the ELSI Extramural Research Program. ELSI stands for ethical, legal, and social implications; the program is a mechanism for funding research on these implications in human genetics and genomics. This is one example, she said, of a framework outside the law enforcement arena that could be very useful in the ethical implementation of new technologies. Butler added that one way to bring other perspectives into the conversation is through exposure to other disciplines. For example, a forensic researcher might attend a molecular biology conference or a bioethics conference and then serve as an “ambassador” by bringing these perspectives back to their colleagues.

Federal Research Priorities

Coble asked the panel to imagine that they had the power to shape federal research priorities and asked them how they would use this power to design a request for proposal or to attach a condition to grant funding. Panelists had a number of suggestions:

• Require grant recipients to collect and report data from the research that was funded (e.g., how frequently does the use of forensic investigative genetic genealogy lead to a confirmed short tandem repeat match?). (Murphy)
• Fund a multistate survey that tracks violent crime and asks what tools were used to solve cases. (Murphy)
• Fund a multisite survey that compares crime rates and clearance rates in jurisdictions with different DNA policies. (Murphy)
• Fund qualitative and quantitative work on the impact of advanced forensic DNA technologies on Black and Native communities. (Murphy)

• Fund ELSI research on the impact of technologies, the risks and benefits to different groups, and strategies for minimizing harms and promoting benefits. (Martschenko)
• Fund independent verification and validation of software programs (Matthews) and require the resulting data to be publicly available. (Butler)
• Require public procurement funds to be spent on vendors that facilitate independent testing (e.g., agree to not use trade secret claims to impede electronic discovery obligations). (Matthews)
• Fund a grant to design and build training materials to help people understand the processes, limitations, and risks of new technologies. (Butler)
• Require research grantees to include a public education component around the research topic. (Murphy)
• When federal funding provides funding for the development of a tool, the tool should be open to scrutiny by the public and not protected by trade secret claims. (Coble)

Cross-Discipline Collaboration

Multiple discipline experts are involved in the development and deployment of forensic DNA technologies, said Murphy. It is important to respect their discipline expertise, acknowledge the boundaries of this expertise, and consider where and when additional expertise may be required. For example, she said, computer programmers working on autonomous vehicles could make decisions themselves about what lives to prioritize in what situation (e.g., the trolley problem, see Duignan, 2024), but the decision would likely be better informed if ethicists were involved. Martschenko agreed and told workshop participants about the Stanford University Institute for Human-Centered Artificial Intelligence, where ethicists partner with artificial intelligence researchers to work iteratively on identifying and mitigating potential societal harms.¹ There are also opportunities for experts in one area to be trained in other areas, she said. For example, graduate students in the genetics department at Stanford are seeking additional training and education on the ethical, social, and legal implications of genetic research. Field-specific training is important, she said, as is collaboration with colleagues who have expertise in other areas.

Murphy added that experts in law need to also be involved in these collaborations in order to identify and address potential legal consequences of genetic research. She gave the example of research seeking to identify a “violence gene” as a way to predict violent behavior. Defense attorneys, in their obligation to do everything in their power to prevent capital punish-

___________________

¹ See https://hai.stanford.edu/about

ment, have presented this research in court to show that their client was not entirely in control of his actions. The use of the gene in this way, said Murphy, opens the door to prosecutors arguing that people with the “violence gene” should be incarcerated to protect public safety. Conversations about these complicated issues need to be informed by experts across multiple disciplines, said Murphy, and a cross-disciplinary approach is critical for making rules about the use of new technologies.

Ray Wickenheiser, New York State Police Crime Lab System, said that vendors should be included in these conversations because understanding the concerns, needs, and requirements of different stakeholders will inform their redesign and improvement process. Murphy responded that there is little incentive for a highly lucrative vendor market to be engaged in these conversations, and that vendors are more likely to be engaged in improvement efforts if there are concrete consequences for providing software that is not validated or for operating without transparency. Consequences are more powerful for changing behavior than hearing critiques, she said.

Regulation and Oversight

Due to a lack of regulation and governance around forensic DNA technologies, said a workshop audience member, “regulation of a lot of these technologies really only happens in a criminal trial.” However, he said, relying on this type of “regulation” is problematic for two reasons. First, he pointed to existing bias toward prosecutors in admissibility hearings; a study of Daubert hearings in Wisconsin found that the prosecution was successful in 114 out of 114 cases. Second, he noted that if advanced forensic DNA technologies are used only for investigatory purposes, the prosecution does not have an obligation to turn over the information as exculpatory evidence. Murphy acknowledged these as serious concerns, noting that the most direct lever to address the issue is the federal government. She explained that the federal government is a huge source of funding and serves as a policy model for states. Murphy said that she hopes that a federal policy would follow a similar approach to Maryland Criminal Procedure Code Section 17-102 (see Box 2-4 in Chapter 2), with strong declarations of standards for use. In addition, if federal grants or assistance are given to conduct forensic DNA testing, funding could be conditioned on disclosure during trial. However, she said, most criminal cases—greater than 90%—do not end up in court because the defendant pleads out. While the assumption is that forensic DNA technologies will be regulated by the criminal system, “it is very much not going to be regulated by the system.” A few lawyers on high-profile cases may challenge the use of advanced forensic DNA technologies, but these challenges become difficult after the technologies have been in use in the justice system for several years. In the absence of

real regulation, Murphy encouraged a strategy of conditioning research and casework funding on mandates for disclosure and transparency.

Throughout the workshop, individual speakers identified resources for those interested in learning more about forensic DNA technologies. These are listed in Box 6-1.

CLOSING REFLECTIONS

At the close of the workshop, planning committee members and other workshop participants and audience members offered their thoughts on key themes and remaining issues.

Education and Training

One of the key points conveyed at the workshop, said Heather McKiernan, RTI International, was the importance of education and training of all players in the system, from investigators to medical examiners to attorneys. However, “education alone will not solve all of our problems”; regulation and oversight are necessary. Coble also highlighted the importance of education and training for all stakeholders affected by advanced forensic DNA technologies, noting that this will require an investment of resources. Alicia Carriquiry, Iowa State University, said that judges and attorneys are a priority for education. Most legal professionals are ill equipped to understand and convey the details of advanced forensic DNA technologies, she said, let alone conduct a robust cross-examination to interrogate their reliability and limitations.

Matthews explained that in her experience, it is a rare judge who is willing to take the time to understand the science enough to make an informed decision about admissibility. She said that when judges are willing to really look at the science, she has seen the prosecution pull scientific evidence so that the judge cannot make a ruling on it. Asking judges or defense attorneys to be the stopgap against potentially unreliable technologies is “not a great use of our societal resources,” she said. Norah Rudin, Forensic DNA Consulting, echoed Matthews’s observation that sometimes evidence is pulled, or a case is settled rather that going to trial, to avoid shedding light on problematic issues. She said that she and her colleagues made an effort a few years ago to educate judges, and they were unsuccessful. A major reason for their lack of success, she said, is that judges tend to only want to be educated by other judges. This is a major roadblock that needs to be addressed.

Culture of Science

Forensic tools “arise from a culture of law enforcement and not a culture of science,” said Natalie Ram, University of Maryland King Carey School of Law. This was stated in the landmark National Research Council (2009) report Strengthening Forensic Science in the United States: A Path Forward, and remains a persistent pattern today with these new technologies. We need to strengthen the culture of science in the forensic space by

encouraging adoption of scientific values such as transparency, standardization, validation, and independence, she said. Wickenheiser said that as a forensic scientist, he considers his duty to science. He explained that this means bringing the best technology to criminal cases, aiming to get it right, acknowledging when things go wrong, and taking steps to continuously improve.

Craig O’Connor, New York City Office of Chief Medical Examiner, said that scientists, laboratories, and other stakeholders are aware of the concerns that have been presented at this workshop, and that they are changing processes to address some of the concerns. For example, labs continue to evaluate new standards, guidelines, and recommendations; forensic science commissions have been put in place; labs are undergoing accreditation; and some states have published road maps on best practices for DNA technologies. Lab scientists are accustomed to being transparent, said O’Connor; they document each step in lab notebooks and publish data in peer-reviewed journals. O’Connor encouraged defense attorneys and their experts to reach out to labs to talk about the technologies used and the results obtained. “The last thing we want to hear is that some plea deal was taken with no communication done with the laboratory itself,” he said, emphasizing the need for communication and education between laboratories, courts, and members of the legal community.

Regulation and Oversight

McKiernan said that there is a need for policies, guidance, and codes of practice with federal or state oversight; self-regulation is insufficient. There is a need for research on the tools themselves as well as the societal impacts of the tools, and this research needs to be done in collaboration with other stakeholders to understand the downstream implications. Sarah Chu, Perlmutter Center for Legal Justice at Cardozo Law, agreed that guidance has limited utility; governance and regulation are necessary. Regulations often tackle one technology at a time, but “legislation will always lag tech development” if we do not use governance to address classes of technology, she said. This governance system must be the result of a collaboration among multiple stakeholders, including the impacted communities, and not at the “whim of law enforcement agencies.” We need to be mindful of “noble cause corruption and thin ethics,” otherwise appeals to public safety can shut down conversations about who benefits, who is harmed, and what justice means in the application of technology, stated Chu.

Carriquiry offered a summary of best practices for effective regulation and oversight of probabilistic genotyping (PG), based on the discussions among panelists from the PG session. First, PG systems should be subject to peer review by independent researchers who have no stake in the outcome.

Second, vendors of PG systems should not be allowed to use contract law to block independent review. Third, trade secret claims should not be allowed to impede electronic discovery obligations to disclose relevant evidence in criminal cases. Implementation of these best practices, she said, could be achieved through procurement guidelines and grant requirements.

Formal Systems of Repair

Chu called for the establishment of formal systems of repair when mistakes are made or errors occur. She suggested that this system should include the duty to correct and notify, conducting root and cultural cause analyses to reduce risk of reoccurrence, and establishing committees of diverse constituents to notify impacted people in a publicly accountable process. Creating these systems is the responsibility of the government as the sponsor of criminal cases, said Chu. O’Connor agreed that formal systems of repair are critical and added that this system should bring mistakes to light in a constructive rather than punitive manner. Dealing with mistakes through an open process will prevent laboratories from hiding mistakes for fear of punishment and will allow problematic aspects of the technologies to be addressed while strengthening the reliability of other aspects. O’Connor said that “we shouldn’t let anecdotal success stories […] drive the use of these technologies if they are unreliable.” At the same time, he said, “we shouldn’t let anecdotal bad actors lead us to throw the baby out with the bathwater” and not use technologies that have been proven reliable.

Chu said that cultural frameworks in aviation and health care are relevant here; these frameworks protect or incentivize people to come forward with problems they are aware of. Implementing this type of framework will require collaboration among multiple entities in the system to create a culture of accountability without fear of punishment. Martschenko drew an analogy between this proposed framework and how clinical trials deal with adverse events. In clinical trial research, there are clear criteria for what constitutes an adverse event, and the trial might be paused if a participant experiences a serious adverse event. In the context of forensic DNA technologies, she continued, there could be clear criteria established for what type of error or unexpected result requires reporting, and use of the tool might pause while the error is investigated.

Next Steps

Given the challenges and roadblocks discussed at the workshop, Chu identified several areas in which action could be taken. The federal government needs to support the development of a framework for oversight of

advanced forensic DNA technologies, she said. In addition to validation of tools prior to implementation, the framework should regulate how data are collected, used, retained, and expunged. Frameworks in the areas of artificial intelligence, genomics, and predictive policing could serve as models for this new framework. In the absence of a federal entity to regulate investigative and forensic methods and technologies, Chu said there is a need for interagency processes to bring more perspectives to the table. For example, the U.S. Department of Justice could convene diverse constituents of the forensic evidence ecosystem, including impacted communities, and make these convenings accessible and meaningful to stakeholders. This could require going out into communities and speaking through credible messengers to communicate with hard-to-reach communities, she said. Federal funding—through procurement, research funding, and grants—can be used as a lever to encourage more ethical implementation of advanced forensic DNA technologies. Chu pointed to the Federal Bureau of Investigation’s quality assurance framework and accreditation requirements for access to the Combined DNA Index System, and she suggested that a similar approach could be used for access to genetic databases.

In short, said Chu, the field of forensic DNA technologies needs validity, reliability, and accountability. Implementing such a system will require research, collaboration, transparency, and collective conversations, and all of these require resources. Throughout this work, she said, “we need to infuse every step of discussions with values and what we owe to each other in the context of the American criminal legal system, where we know that overpoliced and vulnerable communities will bear a concentrated harm from misuse of methods and technologies that may not be visible to non-marginalized communities.”

Butler closed by quoting Marie Curie, who said, “I was taught that the way of progress was neither swift nor easy.” This workshop identified a number of priority areas in which action needs to be taken, he said, and this will require a commitment to keep moving forward. The work will not be “swift or easy”; any project of importance takes time and effort.

This page intentionally left blank.