State and local governments around the U.S. are harnessing AI for a range of applications — such as translating public meetings into multiple languages in real time to allow broader participation, or using chatbots to deliver services to the public, for instance.

While AI systems can offer benefits to agencies and the people they serve, the technology can also be harmful if misapplied. In one high-profile example, around 40,000 Michigan residents were wrongly accused of unemployment insurance fraud based on a state AI system with a faulty algorithm and inadequate human oversight.

“We have to think about a lot of AI systems as potentially useful and quite often unreliable, and treat them as such,” said Suresh Venkatasubramanian of Brown University, co-author of a recent National Academies rapid expert consultation on AI use by state and local governments.

He urged state and local leaders to avoid extreme hype about AI, both its promise and dangers, and instead to use a careful, experimental approach. “We have to embrace an ethos of experimentation and sandboxing, where we can understand how they work in our specific contexts.” 

Venkatasubramanian spoke at a National Academies webinar that discussed the new publication and other AI-related resources for state and local governments. He was joined by fellow co-author Nathan McNeese of Clemson University, and Leila Doty, a privacy and AI analyst for the city of San José, California, along with Kate Stoll of the American Association for the Advancement of Science, who moderated the session.

Focus on purpose and people

In considering whether to implement AI, McNeese advised state and city agencies to start by asking, “What’s the problem?” or “What’s the aspect of the organization that we want to enhance?”

“You do not want to introduce AI if there is not a specific need,” said McNeese. “You don’t want to implement AI because everyone else is.”

The point was seconded by Venkatasubramanian. “If you have a problem that needs to be solved, figure out what people need to solve it,” he said. “Maybe AI can be a part of it, maybe not. Don’t start by asking, ‘How can we bring AI to this?’ That way leads to problems.”

When AI is used, the publication urges a human-centered approach to designing it — one that takes people’s needs, wants, and motivations into account, explained McNeese.

Those who have domain expertise — employees who provide services of value to the public — should be involved in determining where AI tools might and might not be useful, said Venkatasubramanian. “It is really, really important to empower the people who have the expertise to understand the domain,” he stressed.

The goal should be to create complementary human-AI teams that allow humans to focus on what they’re good at by off-loading certain aspects of their jobs to AI, McNeese said. “We’re not trying to use AI to replace humans.”

Strategies for implementing AI

McNeese provided a brief overview of the publication’s strategies for states and cities as they plan for and implement AI. “Integrating AI into any organization is daunting,” he said. “There’s a lot of things that you have to consider and take into account early on in the process.”

After identifying a problem that AI might help solve, the agency needs to analyze whether implementing AI is feasible and how it will impact existing workflows, he said. The agency then needs to determine what type of AI to use and whether to develop it internally or use an external vendor.

McNeese urged organizations to consider risks related to the type of AI they plan to use, noting that the National Institute of Standards and Technology has an AI risk management framework that offers guidance.

Organizations should pilot AI within a subsection rather than implementing it everywhere at once, said McNeese. They should also develop evaluation criteria to assess the degree to which AI is helping to solve the problem to which it is applied. And leaders should ask their employees for feedback.

“You want people to have a voice,” McNeese said. “You want people to give you insights, and then … you want to adopt, refine, or abandon the utilization of AI based on that evaluation and that feedback.”

Venkatasubramanian noted that as agencies implement AI, they can seek help through partnerships with local experts, civil society organizations, and universities. “People at universities are very interested in working with state and local governments, because these demonstrate actual use cases in real settings” that shed light on how innovation in AI should proceed, he said.

A ‘community of practice’ for public-sector AI

Another source of support available to government agencies as they implement AI is the GovAI Coalition, which was established in November 2023 by the city of San José. Leila Doty explained its history and work.

The GovAI Coalition was founded after San José started to roll out an AI policy and governance framework in early 2023 and had trouble getting transparency from some AI vendors about their systems — the data used to train them, for example, and their accuracy and other performance metrics. Conversations with peers at government agencies across the country revealed that they were also running into this problem, as well as navigating the broader issue of how to do responsible governance around AI, said Doty.

“We felt that it made a lot of sense to come together, work with our peers, and create a standard within the field for how local and state agencies can do AI governance in a responsible way,” she said. The agencies used their collective power to demand more accountability from AI vendors and to create a large suite of AI governance resources, including templates for policies and vendor contracts.

“The scope of the work in the coalition has really expanded, and now it’s truly become a much broader community of practice around AI in the public sector,” Doty said, adding that over 850 agencies around the country — local, county, state, and a couple of federal — are now members.

She welcomed the National Academies’ new publication, noting that its strategies align with the coalition’s efforts. “I think [they] will be really helpful to state and local governments who are looking to adopt AI.”

Watch the webinar or read the rapid expert consultation .