Proceedings of a Workshop—in Brief

Law Enforcement Use of Predictive Policing Approaches

Proceedings of a Workshop—in Brief

On June 24 and 25, 2024, the National Academies of Sciences, Engineering, and Medicine held a two-day public workshop exploring law enforcement’s use of person-based and place-based predictive policing strategies. Predictive policing strategies are approaches that use data to attempt to predict either individuals who are likely to commit crime or places where crime is likely to be committed, to enable crime prevention. The workshop was held in response to Executive Order 14074,¹ which discusses enhancing public trust and safety through accountable policing and criminal justice practices, and Executive Order 14110,² which addresses the use of artificial intelligence (AI) in law enforcement. David Weisburd (George Mason University and Chair of the workshop planning committee) began by noting that these executive orders reflected strong public concerns surrounding the idea of predictive policing, as well as critiques of specific implementations—in particular for these strategies’ disparate impact on communities of color. While planning the workshop, Weisburd said that the planning committee confronted several challenging issues. First, there is a lack of precise and clear definitions of what exactly constitutes predictive policing. Second, the term “predictive policing” is often avoided, even when approaches appear to meet conventional definitions. Predictive technologies include “automated,” “dynamic,” or “data-driven,” approaches. However, predictive policing is generally seen as involving predictive algorithms that identify individuals and locations that are more likely to be associated with crime in the future. Whatever the definition, law enforcement agencies routinely use tools that collect and analyze data to anticipate crime and facilitate police response. Weisburd highlighted that the method and extent to which police should rely on algorithmic approaches remain as real-world challenges for law enforcement officials.

This workshop, said Weisburd, comes at a time when original applications of predictive policing have come and gone, while algorithmic and big data technologies advance and continue to be applied in law enforcement contexts. “We may be on the precipice of a new era of predictive policing,” he said, “with the time and wisdom to consider what that could and should look like.”

Weisburd introduced Nancy La Vigne (Director of the National Institute of Justice [NIJ]), who provided a brief overview of NIJ’s perspective on predictive policing. La Vigne noted NIJ’s history of investing in aspects of

__________________

predictive policing technologies and acknowledged the unintended outcomes of some of these investments. The intent of NIJ’s investments in predictive algorithms was to identify, or predict, crime concentrations with the goal of discerning what other factors or correlates may represent the root causes of the hot spot so that they could be addressed. However, in practice, predictive algorithms have fueled hot spots policing that too often results in the over-policing of communities and residents, imposing biases that have detrimental impacts on people of color. To mitigate this trend, La Vigne emphasized the importance of focusing on what police do about predicted hot spots. The field needs better guidance on that front, or else the police may engage in more enforcement activities at the expense of both community trust and public safety. She cautioned that even if predictive algorithms were no longer used, police would continue to use data to guide resource allocation decisions, underscoring the importance of prescribing what types of actions police take in response to predictions. “Predictive policing is not just about the prediction but about the type of policing that happens after the prediction,” said La Vigne. Finally, she emphasized that this conversation is ultimately about public safety and encouraged stakeholders to consider how high-crime communities can best be served.

These opening remarks set the tone for the workshop and emphasized the importance of both addressing the critiques of predictive policing and the implications for community trust, while also exploring its potential benefits for law enforcement and public safety. The workshop began with a roundtable featuring individuals involved in community responses to predictive policing, providing context and reflecting the perspectives that, in part, motivated the call for this workshop. The next two sessions explored place-based and person-based predictive policing, describing the history and current use, evidence of effectiveness, as well as the legal, ethical, and social issues to consider. The workshop concluded with a roundtable conversation discussing the future of predictive policing approaches and key takeaways from the workshop. The following summary is organized to maximize clarity for a broad audience and does not directly follow the event’s agenda order.

PLACE-BASED PREDICTIVE POLICING

This workshop session focused on history, current use, and evidence for place-based predictive policing, and began with an overview from committee member Andrew Ferguson (American University Washington College of Law). Place-based predictive policing, he explained, is grounded in environmental criminology. The theory underpinning environmental criminology posits that crime is not evenly distributed across society but instead follows identifiable patterns influenced by environmental factors, leading to crime “hotspots.” At its inception, the application of environmental criminology was promoted as an objective, data-driven, efficient method of policing—particularly in terms of addressing racial bias. Over the following decade, police departments across the U.S. experimented with these technologies.

Ferguson described three early place-based predictive policing tools, which used crime data and other variables to predict future criminal risk in particular places. One early tool inspired by seismology relied on research that showed that certain property crimes are “contagious” and will lead to “aftershocks” of similar offenses. The algorithm used data from incident calls, calls for service, crime type, time, and place, to generate maps of areas with elevated risk. Another early tool was a “patrol management system” that forecasted areas of risk and suggested ways that police could respond. This model used data on past crime as well as information on seasonality, day of the week, holidays, and sporting events. Another early place-based predictive policing tool examined the risk environment to determine why crimes were occurring in certain areas. Ferguson explained that the final tool’s model viewed cities as areas with multiple, interconnected risks. The model proposed solutions to reduce these risks, including both policing strategies and modifications to urban design. All three of these early models were targeted primarily at deterring crime through increased police presence or structural changes to the environment (e.g., lighting, cameras).

Law Enforcement Perspectives on Place-Based Predictive Policing Approaches

Chief Jim Bueermann (ret.; Future Policing Institute) offered a law enforcement perspective on place-based predictive policing. He explained that these approaches have the potential to make communities safer by stop-

ping crime before it occurs, making policing more efficient, controlling costs, and increasing trust and confidence in policing. Conversely, there are concerns among community members that these approaches can exacerbate bias in policing; facilitate wrongful convictions, arrests, and detentions; negatively impact community perceptions of police; and remove the human element from policing. While use of the term “predictive policing” may be phasing out, said Bueermann, these issues will remain salient, particularly as data and AI are increasingly used by law enforcement. He hypothesized that a variety of AI and predictive tools may soon be packaged as part of a broader suite of technology marketed to law enforcement. This means, Bueermann explained, that whether or not police actively choose to use prediction tools, these tools may be available and “baked in” to the software programs (e.g., case management systems) law enforcement members use daily.

Evidence on Place-Based Predictive Policing

Predictive policing has two components: predictive analysis and “good policing,” said Jerry Ratcliffe (Temple University). He defined “good policing” as policing that is effective, proportionate, and procedurally just. Good policing prevents crime better than unfocused, reactive policing (e.g., the practice of assigning officers randomly across a city or sending officers to a location only after a crime has been committed). When Ratcliffe became a police officer 40 years ago, he said that police used a rudimentary form of predictive policing which involved the sergeant sending extra officers to areas that they thought might be crime-prone. While newer algorithms for predicting crime may not be perfect, he argued that they are an improvement upon this non-transparent, non-data-driven approach.

There is substantial evidence that “good policing” prevents crime, continued Ratcliffe, and predictive policing can be part of “good policing.” He shared evidence from his own published research documenting crime patterns and measuring the impact of law enforcement interventions. A study in Philadelphia, PA, found that two weeks after a shooting event, the likelihood of a repeat shooting within 400 feet was 33 percent higher.³ Another Philadelphia-based study involved randomly assigning different police responses to four city districts, including alerting officers to crime predictions, putting a marked police car in the target area, and putting an unmarked car in the target area. The interventions produced no difference in violent crime rates, but researchers found a 31 percent reduction in property crime in areas with marked cars.⁴

Youngsub Lee (University College London) provided an overview of his systematic review of place-based predictive policing interventions.⁵ As part of a larger project aimed at evaluating trust and legitimacy in technology-enabled policing, Lee and colleagues reviewed 161 studies and classified them based on the strength of the evidence of the intervention’s effectiveness. Six studies utilized randomization alongside real-world experiments and were classified as having the strongest evidence; these studies demonstrated “no”, “limited”, or “moderate” effectiveness of predictive policing approaches (Table 1). The remaining 155 studies were retrospective studies that focused primarily on prediction accuracy rather than real-world effectiveness. Importantly, said Lee, the six stronger studies found that matching a prediction to an appropriate intervention is a key element of effective predictive policing, suggesting that prediction models need to be paired with tailored interventions in the field.

This project, Lee said, suggests that while there is some evidence of the effectiveness of place-based predictive policing, there remain concerns about its implementation. More research is needed, he said. Specifically, Lee called for prospective, randomized controlled trials to test the impact of clearly specified interventions on target crimes. Only by conducting rigorous research on the effectiveness of predictive policing can decision-makers weigh the risks and benefits, he argued.

PERSON-BASED PREDICTIVE POLICING

Turning to the history, current use, and evidence around person-based predictive policing, planning committee member Sarah Brayne (The University of Texas at Austin) provided an introductory overview. Person-based predictive policing, she explained, involves using data to

__________________

TABLE 1 Systematic Review of Place-Based Predictive Policing Interventions

SOURCE: Lee, Y., Bradford, B., & Posch, K. (2024). The effectiveness of big data-driven predictive policing: Systematic Review. Justice Evaluation Journal, 1–34. https://doi.org/10.1080/24751979.2024.2371781

predict which individuals are more likely to be involved in future criminal activity, either as a victim or a perpetrator. It is based on the evidence that a small proportion of people are responsible for a large proportion of violent crime. Brayne noted that while police have always focused resources on high-risk individuals, contemporary predictive policing formalizes and quantifies these predictions. She briefly described three early uses of this approach: Los Angeles’ Operation LASER, Chicago’s Strategic Subject List (SSL), and Pasco County’s (Florida) “prolific offenders” list.

Operation LASER was a program designed to reduce crime by focusing on those who committed a larger proportion of offenses. Data from various sources were used to generate risk scores, using a system that allocated points for offenses (e.g., arrest with a handgun, gang affiliation).

Pictures and details about the highest risk offenders were distributed to officers; what the officers did when they established contact with a high-risk offender was at the discretion of the officer and varied widely, said Brayne. The SSL in Chicago was a similar effort designed to identify individuals who were most likely to commit gun violence or be a victim of it. The algorithm used in the SLL was based on research demonstrating that being a part of a social network in which gun violence occurred made an individual more likely to encounter gun violence themselves. Using data on affiliation, criminal records, and police contact, law enforcement officials employed the SSL to create a list of individuals at high risk of being a victim or perpetrator of gun violence. Similar to LASER, officers had discretion about what they did with the information. The third example that Brayne shared

was an initiative in Pasco County that generated a list of “prolific offenders.” This program was presented as one that could reduce bias in policing by using objective data, she said. Operation LASER ended after an audit by the Los Angeles Police Office of the Inspector General found that there was not strong evidence that LASER reduced crime rates; further there were civil rights concerns around inconsistent enforcement, opacity, and lack of accountability.⁶ The use of the SSL ended after a similar investigation conducted by the Chicago Police Office of the Inspector General.⁷

Law Enforcement Perspectives of Person-Based Predictive Policing Approaches

With this history in mind, Scott Mourtgos (Salt Lake City Police Department and University of South Carolina) discussed the current use of person-based predictive policing. He emphasized that person-based predictive policing is relatively rare compared to place-based predictive policing. Mourtgos outlined four primary strategies employed by police departments: focused deterrence, prolific offender lists, long-term predictions, and threat scores.

Focused deterrence involves identifying prolific offenders and offering them a choice between assistance through social services and community interventions or increased attention from law enforcement. Mourtgos noted that focused deterrence tends to be more popular than other predictive approaches and may also be more effective. However, he emphasized that the police are only a small part of a focused deterrence approach, which depends on support from other agencies and partners.

Prolific offender lists are used to increase police attention on individuals identified as high-risk through home visits, surveillance, and increased arrests and penalties, as seen in Pasco County. Mourtgos said that the idea is to deter or incapacitate offenders so that they do not commit further crimes. This approach is controversial; critics argue that it punishes individuals for actions they have not yet committed, while proponents believe it is a more strategic and focused use of resources, he said.

Long-term predictions are used to forecast the trajectory of an individual over time, rather than the short-term likelihood of involvement in a crime. This strategy uses data on criminal activity, social networks, educational information, and other factors to predict which individuals are at risk of becoming prolific offenders, said Mourtgos.

Threat scores can help officers take extra precautions when engaging with higher-risk individuals; however, Mourtgos pointed out that these extra precautions could lead to worse outcomes. For example, an officer’s knowledge of a threat score might escalate a situation unnecessarily, potentially leading to the use of force.

Evaluations of Person-Based Predictive Policing

John Hollywood (RAND) provided a detailed account of findings from evaluation of the SSL in Chicago. He began by introducing the audience to three published evaluations of predictive policing approaches, first a general technology assessment of predictive policing published in 2012,⁸ and two specific evaluations of the Chicago predictive policing experiment or SSL.⁹

In the first year following creation of the SSL, Hollywood reported, only three of the 405 Chicago homicide victims were people who had appeared on the SSL. Research found that individuals on the SSL were not more or less likely to become victims of a homicide or shooting than the comparison group, and that the pilot effort did not appear to have been successful in reducing gun violence.¹⁰ Evaluations of the Chicago place-based predictive policing programs revealed several operational problems, he continued. First, the predictive models were operationally unsuitable. There are a wide range of reasons people may be at increased risk for involvement in gun violence.

__________________

Simple risk levels did not provide enough information to guide tailored interventions, he explained. Second, Hollywood noted, gun violence risk is time sensitive, and may require rapid intervention, but predictive models were not updated to reflect real-time information. A second iteration of the SSL that involved more regular updates was never integrated into policing operations and was never validated as an intervention model. Third, he suggested that the police department did not integrate the predictive models into operations or sufficiently guide officers in how to use the resulting information. A key takeaway, Hollywood noted, was that the Chicago programs were misunderstood and contributed to unnecessary public fear. There were widespread concerns about the programs being a punitive “Minority Report” system, he explained, and the SSL was understood as the “bad guys list” despite also being a list of potential victims of crime.

Drawing from these evaluations, Hollywood speculated to why person-based predictive policing is relatively rare. First, punishing individuals for actions they have not committed presents a number of ethical, legal, reputational, and institutional risks, as well as generates significant public fear. Second, even if predictive models are statistically sound, their predictions are often too weak and generalized to effectively guide actionable measures.

Evaluations of Focused Deterrence

Thomas Abt (Center for the Study and Practice of Violence Reduction, The University of Maryland) highlighted focused deterrence as a predictive approach to crime reduction that has shown strong evidence of effectiveness. Focused deterrence involves analyzing data to assess the risk of future offending; identifying high-risk individuals are then offering services and opportunities through community partners. He described a systematic review¹¹ that examined 24 quasi-experimental tests of focused deterrence and 19 of the 24 showed “noteworthy” reductions in crime; post-review evaluations indicate continued effectiveness. Abt cautioned that referring to all data-driven policing as predictive policing could undermine the use of evidence-based programs like focused deterrence.

KEY CONSIDERATIONS FOR PREDICTIVE POLICING TECHNOLOGIES

The following sections delve into considerations for the implementation of both person-based and place-based predictive policing technologies. While these approaches differ in their specific focus, many of the considerations explored are applicable to both approaches. Where considerations apply to a single approach, this is noted.

Key Considerations for Place-Based Predictive Policing

Community Engagement

Community engagement is not optional but is rather a fundamental prerequisite for effective, ethical, and democratic policing, said Charlotte Gill (George Mason University). Balancing community safety with community trust requires a commitment to the co-production of public safety, where both the police and the community are equal partners in maintaining safety. Gill explained that partnering with the community is not just “doing the right thing” but is also the cornerstone of effective crime prevention. Community engagement ensures the representation of diverse voices, leads to effective and localized solutions, mitigates the alienating effects of law enforcement, and builds capacity for addressing root causes of crime.

Gill acknowledged that there are significant challenges to community engagement; she highlighted challenges in three areas: (a) authenticity and representation, (b) leadership and expertise, and (c) data and outcomes. Authentic engagement requires genuine collaboration, Gill emphasized, noting that her review of “community-oriented” policing programs found that the majority only passively or indirectly involved the community. Building a relationship with the community requires considerable time, effort, and attention—especially in communities with strained relations with the police. Leadership challenges can arise as police (who are traditionally seen as the experts in crime control) must cede some control to the community. Data challenges include ensuring that collected data do not perpetuate existing harms, and that they reflect the concerns of the community, not just those of law enforcement. Gill offered several consider-

__________________

ations specific to community engagement in predictive policing programs:

• Engage diverse stakeholders early and maintain engagement throughout the process;
• Incorporate community voices into decisions on technology deployment, data collection and use, and balancing privacy, civil rights, and public safety concerns; and
• Ensure that data and locations reflect community priorities.

Impact of Historic Racial Segregation

Rashida Richardson (Northeastern University) described the role that segregation—defined as the systemic spatial separation and social exclusion of groups—plays in place-based policing predictions. One way that segregation impacts predictive policing, she said, is through the use of data which are not explicitly about race but correlate strongly with racial demographics. For example, minority neighborhoods that have been marginalized and under-resourced are more likely to be over-policed, which can lead to greater police contacts and more crimes being recorded by police in these areas. Using these higher crime rates as inputs in a predictive algorithm can lead to further disproportionate targeting of these communities; thus exacerbating racial disparities in policing. Richardson called for more rigorous scrutiny of the data used in these systems and for the development of algorithms that can account for and mitigate these biases.

Legal, Ethical, and Regulatory Considerations

The reliance on data in policing is not new, said Elizabeth Joh (University of California, Davis). Policing has always involved collecting and analyzing information—such as citizen tips, police records, and surveillance—to identify suspicious individuals and prioritize areas for policing. The regulatory framework for policing, she said, assumes this type of human decision making. For example, the Fourth Amendment requires that an officer conducting a stop-and-frisk have a “reasonable suspicion” that the person is involved in criminal activity. Whether there is “reasonable suspicion” depends on the totality of the circumstances; this standard is intentionally flexible and without quantification or calculation, said Joh. Predictive policing represents an acceleration in technology that raises questions about how the traditional standards apply. Joh noted that because the “reasonable suspicion” standard is flexible, she does not believe that Fourth Amendment constraints, as currently applied, will fully address the civil liberties concerns raised by predictive policing. Given previous jurisprudence, she argued, it is likely that predictive policing analyses will be considered as part of this broad category. Another legal issue, Joh added, is the “handoff problem,” which refers to how officers use the information generated by technology. For example, an officer might focus on a particular area due to multiple predictions for that area, relying not on traditional policing information or a specific prediction as the technology is intended to be used.

As technology use grows, Joh said, there is a need to adopt a regulatory framework that addresses these concerns, rather than assuming that Fourth Amendment frameworks are adequate. She drew a parallel to the field of medicine, where regulators conduct thorough cost-benefit analyses of drugs before approval. Policing technologies, which also have the potential to harm individuals and communities, generally do not undergo such rigorous analysis before being deployed. Joh said that this lack of thorough evaluation raises ethical and practical concerns that should be addressed to ensure the responsible use of technology in policing.

Renée Cummings (The University of Virginia) agreed that, while AI and other tools for predicting crime are new, the practices and concerns are similar to those of past policing approaches, and as such individuals in underserved and under-resourced communities may feel re-victimized, re-traumatized, and re-marginalized as a result of their use. Using historical data in these algorithms not only maintains the status quo, but can amplify discrimination, codify bias, perpetuate systemic racism, and repeat past prejudices, she said. Cummings said that it is essential to ensure that these technologies serve the public good without perpetuating harm. She suggested that oversight of data science could be accomplished through an agency similar to the Food and Drug Administration (FDA); such an organization could ensure that there is appropriate oversight, robust security, rigorous accuracy, traceability, and detailed documentation. This is the

time, Cummings said, for a “radical re-imagining of predictive analytics” to ensure that they include, empower, and liberate communities while advancing public benefit and justice.

Given questions raised around accuracy, fairness, ethical, and legal issues in relation to person-based predictive policing, Jumana Musa (National Association of Criminal Defense Lawyers [NACDL]), focused her presentation on questions around the value of allocating limited state resources toward predictive policing technology. However, she said, because these technologies continue to be used, the NACDL has issued recommendations to mitigate harms associated with their use. The NACDL has called on the Department of Justice to (a) carefully consider whether federal law enforcement agencies should use these technologies; (b) condition federal funding on strict and expansive validation and disclosure requirements; and (c) increase requirements on companies providing the technology to open their systems to external validation and review by the criminal legal system.

Evaluating Effectiveness and Fairness: Insights from Computer Science

Elissa Redmiles, (Georgetown University) discussed insights from computer science as they apply to predictive policing. She noted that trust in technology can be broken into two considerations—whether the technology works and whether the technology is fair. The answer to whether a predictive policing tool “works,” Redmiles explained, is complicated for several reasons. First, outcome measurement is challenging because the prediction is for an outcome that cannot be fully validated. The algorithm purports to predict the risk of committing a crime, but outcome data are only available for crimes known to law enforcement. Second, predictive law enforcement tools may not be significantly more effective than humans at making predictions, said Redmiles, citing a finding that lay people’s predictions about recidivism were about as accurate as an algorithm.¹² Finally, predictive accuracy is difficult to assess for predictive policing approaches because of the distance between their prediction and their goal—accurately predicting future crimes is not the same thing as reducing crime.

In addition, said Redmiles, evaluations of whether person-based predictive tools “work” need to also consider unintended consequences, such as feedback loops.

To address issues of accuracy, Redmiles gave three suggestions for procurement and grantmaking organizations: (a) analyze whether the outcome used to measure success matches the prediction; (b) justify predicting negative behavior versus positive intervention efficacy; and (c) require organizations to articulate how they will act on predictions, and to use expert and community feedback to inform anticipation of potential unintended consequences. In addition to considerations around accuracy, she highlighted concerns around fairness. Redmiles suggested that selecting fairness metrics with reference to the specific context of a technology’s use and with broad public input and consideration of both inputs and outputs in evaluating fairness is needed.

Redmiles highlighted the AI risk management framework from the National Institute of Standards and Technology as a valuable guide for future iterations of predictive policing technology.¹³ She noted that leading computation researchers recently published an evaluation of a range of uses of predictive optimization and urged against the use of any techniques that use machine learning to predict future outcomes and make decisions about individuals based on those decisions, regardless of the domain in which they are applied.¹⁴ Redmiles concluded by describing best practices for technology system procurement—requiring external audits (regardless of a technology producer’s expertise or asserted independence), requiring ongoing open access for unsolicited independent evaluation, and publicly reporting incidents, evaluation outcomes, benchmarks, and design decisions.

Considerations Specific to Person-Based Predictive Policing: Inputs, Interventions, and Impacts

Andrew Ferguson (American University Washington College of Law) offered additional analysis specific to person-based predictive focusing on three key aspects:

__________________

(a) inputs, (b) interventions, and (c) impacts. If inputs into a system are not carefully considered and validated, he said, they can reify structural inequities, racial bias, and power imbalances. In Chicago, an Inspector General report concluded that SSL inputs were unreliable, citing poor quality and infrequent updates. Further, said Ferguson, the SSL used unreliable predictors as inputs; for example, scores were increased if a person was arrested for a crime even if they were ultimately not convicted.¹⁵ The SSL predicted an individual’s risk of offending or victimization within 18 months, but officers received no clear guidance on how to act on this information, he noted.

Shifting to LA, Ferguson again highlighted inputs. LASER’s chronic offenders list, he noted, was based on a point system that had little to no grounding in social science. Five points were assigned for certain offenses or affiliations and one point for each police contact. In addition to scientific weakness, this person-based predictive policing approach produced a feedback loop: individuals with high scores were sought out by police, thereby adding a point for each “quality police contact” and raising their score. An Inspector General report found several problems with the program: some people on the list were placed by informal referral and had no points at all, points were inconsistently assigned, and a sizable number of “chronic offenders” had no arrests for violent or gun-related crimes.¹⁶ Considering interventions, Ferguson noted that as in Chicago, LA police officers were not given clear instructions on appropriate interventions. Ferguson emphasized the real-life impacts these person-based predictive policing approaches can have on individuals and their potential for eroding community trust.

Ferguson offered several suggestions for law enforcement use of person-based predictive policing approaches: technology should only be adopted if proven effective through evaluation, inputs should not be police-generated or should be non-criminal and should not perpetuate existing inequities, interventions must be timely and actionable, and programs should undergo continuous audits to prevent biased or unconstitutional actions.

COMMUNITY PERSPECTIVES

Andrea Headley (Georgetown University) moderated a roundtable discussion that explored community perspectives on predictive policing (both person- and place-based) and ways in which communities have responded to the use of these technologies; community representatives were Freddy Martinez (Lucy Parsons Labs), Shakeer Rahman (Stop LAPD Spying Coalition), and Jeramine Scott (Electronic Privacy Information Center).

Scott provided an overview of some of the main community concerns related to predictive policing. He highlighted a fundamental issue: questioning whether technology can ever be deployed without bias and without infringing on civil rights, civil liberties, and privacy. Another concern is the focus on technological solutions at the expense of addressing the root causes of crime, he said. Scott emphasized that “technology does not solve crime and violence,” and predictive policing may serve as a superficial fix for inadequate social policies. Communities also worry that predictive policing could lead to unnecessary interactions with law enforcement or exacerbate tensions between police and the community. When algorithms designate certain individuals or locations as dangerous, this can influence police behavior and interactions with the community. Scott stressed the importance of carefully evaluating societal costs and measurable benefits of law enforcement use of predictive policing approaches.

Speakers shared their experiences working with communities to respond to law enforcement’s use of predictive technologies. Rahman said that predictive policing programs were promoted as a replacement for the subjective and potentially discriminatory practices of police, but instead they can perpetuate discriminatory practices.

Martinez discussed his work at Lucy Parsons Labs in Chicago, where he and his colleagues worked to respond to police use of the SSL. Martinez expressed concerns about the adequacy of expertise involved in the development of the approach, and the level of engagement with community perspectives. Martinez and Rahman emphasized

__________________

that non-professional community members are capable of raising informed and nuanced critiques of predictive policing.

Drawing on the Executive Order’s emphasis on public trust in law enforcement, Headley asked panelists to comment on what it would look like to take public trust seriously when considering the use of these technologies. Rahman responded that in his opinion, you have to start from a framework that understands the history of policing and trust in racially minoritized communities, and that technology alone cannot change this. Scott agreed that there is no silver bullet to improve public trust, but he said that there are things that can be done to make policing more accountable and transparent: looking at new technologies and approaches with a critical eye, welcoming public input at all stages of development, and requiring proof that new approaches are non-discriminatory before they are implemented.

Rahman also observed that addressing the root causes of crime is needed. Instead of investing in housing or education to make poor communities stronger and safer, decision makers tend to invest in policing and then make modest attempts to reform or refine policing practice, he observed. Martinez agreed that resources are needed to address the root causes of crime, and that adequate investment may require re-allocating resources currently tied to policing. Scott added that long-term planning and political will are needed to address the root causes of crime (poverty, wealth disparities, education, housing, and mental health care), whereas technological solutions can often be easier and quicker to implement.

THE FUTURE OF PREDICTIVE POLICING

In the final panel discussion of the workshop, speakers discussed the future of predictive policing. Weisburd asked probing questions about whether predictive policing is worth pursuing, given the ethical, legal, and community concerns, and if so, what the next generation of predictive policing would look like.

Many fields—from medicine to education—have made great progress by using data to drive practice, said Jens Ludwig, University of Chicago. While there are valid concerns about specific implementations of predictive policing approaches, he said that it would be a lost opportunity to completely dismiss the idea of data-driven policing. When considering whether data-driven policing is a useful tool, it is important to ask, “Compared to what?” said Joh. Because policing has traditionally involved a great deal of discretion and human judgment, there is theoretically a lot to be gained by using data to make decisions. However, she said, predictive policing tools are designed to serve as an aid for human decision making, rather than a substitute. Cummings noted that there are real risks to deploying emerging predictive technologies, but that the risks vary depending on the actions taken in response to the prediction. For example, if a place-based prediction then results in more lighting or a cleaned-up park, the risks of deployment can be low, but if a prediction results in harassment or unconstitutional searches of people, the risks can be high.

Community trust is a critical issue in policing today, said Bueermann. Ideally, policing is community-led and chooses priorities based on the community’s concerns. Synthesizing perspectives across diverse communities and making practical decisions is a challenge for police leaders, he said. Ferguson urged decisionmakers to listen to communities. Headley said that moving forward with these tools without forethought or community input can have costs for community trust, not just to policing but trust in government more broadly. She added that communities are generally aware of their own problems and needs, but that the lived experiences of community members are often undervalued as evidence.

Abt suggested that the term predictive policing is vague and outdated, has caused confusion, and is not widely used by practitioners. Ferguson agreed and noted that data-driven policing is not going away—as new forms of technologies are implemented in policing it is critical that thought be given to inputs, interventions, and impacts before these programs move forward. Bueermann said that predictive policing will become part of a broader ecosystem of police technology; Lukens agreed and said it is imperative to acknowledge and address harmful consequences now. Abt said, to avoid past concerns with implementation of predictive policing, future predictive policing approaches need to avoid using large and unreliable datasets without strong justifications and controls, be transparent to people impacted by it, implement regulations related to the development and purchase of technologies, and be evaluated before deployment.

Echoing other comments heard throughout the workshop, Ludwig said that a major issue is the lack of evaluation of predictive technologies and their implementation; he noted that there are measures of success beyond simply reductions in crime. Weisburd added that currently, policing programs often get implemented based on single studies or conversations and agreed with the need for a rigorous evaluation of programs prior to widespread implementation. Predictive technologies also have the potential to impact individuals and communities, noted Joh, yet there is no evaluation process to determine the costs and benefits of deployment. Ludwig and Headley echoed the suggestion for an FDA-type agency to evaluate the impact of these technologies. To conduct rigorous evaluations, he said it would be necessary to have better outcomes data, including data on community sentiment about policing—and noted that the federal government could play a key role in gathering high quality data on community sentiment. Bueermann agreed with the importance of community sentiment data, which could ensure that police chiefs hear a diverse range of views.

In the future, the federal government could use its grantmaking power to incentive research focused on accuracy and potential racial justice harms, said Ferguson. Several speakers identified other roles that the federal government could play, including influencing practice and policy through grants, convenings, disseminating information about evidence-based practices, technical assistance, and research and evaluation (Abt); requiring federal grantees to implement plans for evaluation, auditing, and controls (Ferguson); and setting national standards for the use of predictive approaches in policing (Weisburd).

Public-private partnerships are critical to innovation, said Cummings. However, said Joh, the incentives and interests of private companies may not necessarily aligned with law enforcement or the public, especially related to transparency. Other challenges related to working with private companies, she continued, include understanding and negotiating contracts and the availability of alternatives. The federal government could assist in this area with model data use agreements, Bueermann said. He suggested that in other fields, such as fire departments, there are internal officers tasked with monitoring internal practices. Bueerman then suggested that a similar role would be beneficial in policing—someone who could ask whether a practice is effective, just, or will cause harm. These questions are not part of the socialization or education of policing in the United States, he said, and the “culture of policing will eat all the best science every day for lunch.”

REFLECTIONS AND KEY TAKEWAWAYS

In the closing session, workshop planning committee members offered their final reflections and takeaways in a number of key areas of concern that had been identified throughout the workshop.

Community Engagement and Trust

Weisburd called attention to the value of building and maintaining community trust and suggested it ought to be a primary consideration in the deployment of predictive policing technologies. Building trust involves engaging with communities, being transparent about what data are employed, how technologies are used, and ensuring that interventions are fair and just. Headley echoed this sentiment, emphasizing the importance of considering community concerns and the potential for predictive policing to exacerbate harm. Kim Neal (City of Alexandria, Virginia) said that predictive policing could miss the human element in policing and further erode community trust, particularly in communities of color. In order to build and maintain community trust, Ferguson argued that place-based predictive policing be implemented only with democratic approval, clear ex ante policies,¹⁷ and meaningful audit mechanisms for community input. Lukens agreed that working with community partners is essential and argued that the police are an integral part of the community.

Data Quality and Transparency

Predictive policing is challenging in part due to issues with the available data, said Ferguson, noting problems with methodology, transparency, accountability, security, and administration. The type of data that are used in predictive tools matters, he said: both effectiveness and fairness can suffer when data on police-discovered

__________________

crimes, quality-of-life crimes, or public disorder crimes are used as inputs, and when feedback loops occur due to the data used. The data used in prediction programs are not neutral or objective, said Headley, but instead reflect existing inequities. Brayne said that one benefit of data-driven policing is that it gathers a significant amount of data about policing itself. These data can be used to measure the costs and impacts of policing, however, this requires a mechanism to allow access by independent researchers or oversight agencies.

Intervention and Impacts

Even with an accurate prediction, it is critical to pay attention to interventions and impact, said Ferguson. The identification of areas or people who are at risk is not inherently problematic, he said, but how that information is used is critical. There is a need for a more holistic response to identified risks, and investments in alternative approaches to crime prevention, such as community-based interventions and social services, can help address the root causes of crime. When prediction-based interventions are conducted by law enforcement, said Ferguson, intervention patterns need to be audited for effectiveness, transparency, and reifying discrimination.

Oversight and Regulation

Lukens raised concerns that commercial proprietary interests could pose challenges for public accountability, and said there could be greater oversight and regulation of technology development, including ensuring that contracts are transparent and provide provisions for accountability for ethical use of technology. Weisburd echoed Abt’s calls for increased federal investment in research and evaluation to develop standards for the use of predictive policing technologies and called for federal agencies to play an active role in ensuring that these technologies are used fairly, ethically, and effectively.

Evidence of Effectiveness

Ferguson argued that place-based predictive policing programs should not be adopted until the reliability of the prediction and the effectiveness of the intervention can be empirically tested in the real world. Headley agreed saying there is little evidence that these programs have enhanced public safety and urged decision-makers to think both about the potential harms of the tools as well as the strength of the evidence supporting their use. Other evidence from research finds that violence and crime can be reduced through efforts like providing mental health and substance abuse care and affordable housing. Several members of the planning committee noted that predictive policing technologies need rigorous evaluation before implementation. Weisburd echoed the statements of multiple earlier speakers in calling for a regulatory body or framework for evaluating and approving public policy initiatives, similar to the role of the FDA, to ensure that these technologies are thoroughly tested for effectiveness and fairness before being implemented on a large scale.

Closing

Individual speakers emphasized that the future of predictive policing will depend on how the ethical, legal, and practical issues that were described in the workshop are addressed. There are challenges and opportunities related to engaging with communities, ensuring the quality and transparency of data, pairing predictions with appropriate interventions, and measuring the effectiveness of predictive policing programs. The insights from the workshop provide valuable guidance into how interested parties can address these challenges and ensure that predictive policing technologies are used accurately and responsibly.

DISCLAIMER This Proceedings of a Workshop—in Brief was prepared by Erin Hammers Forstag as a factual summary of what occurred at the workshop. The statements made are those of the rapporteur or individual workshop participants and do not necessarily represent the views of all workshop participants; the committee; or the National Academies of Sciences, Engineering, and Medicine.

REVIEWERS To ensure that it meets institutional standards for quality and objectivity, this Proceedings of a Workshop—in Brief was reviewed by Charlotte Gill, George Mason University. We also thank staff member Ruth Cooper for reading and providing helpful comments on this manuscript. Kirsten Sampson Snyder, National Academies of Sciences, Engineering, and Medicine, served as the review coordinator.

COMMITTEE David Weisburd, George Mason University; Sarah Brayne, Stanford University; Andrew Ferguson, American University; Andrea Headley, George Washington University; Philip Lukens, Former Police Chief, Alliance, Nebraska; Kim Neal, City of Alexandria, Virginia

SPONSORS This workshop was supported by contracts between the National Academy of Sciences and the National Institute of Justice (AWD- 15PNIJ-23-GG-04263-NIJB). Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.

For additional information regarding the workshop, visit: https://www.nationalacademies.org/our-work/law-enforcement-use-of-person-based-predictive-policing-approaches-a-workshop

SUGGESTED CITATION National Academies of Sciences, Engineering, and Medicine. 2024. Law Enforcement Use of Predictive Policing Approaches: Proceedings of a Workshop—in Brief. Washington, DC: National Academies Press. https://doi.org/10.17226/28037.

Division of Behavioral and Social Sciences and Education Copyright 2024 by the National Academy of Sciences. All rights reserved.