1
Introduction
The Intelligence Community Studies Board (ICSB) of the National Academies of Sciences, Engineering, and Medicine convened a workshop on December 11–12, 2018, in Berkeley, California, to discuss robust machine learning algorithms and systems for the detection and mitigation of adversarial attacks and anomalies. With funding from the Office of the Director of National Intelligence, the ICSB established a Planning Committee on Ensuring the Quality of Machine-Generated Analytic Products from Multi-Source Data: A Workshop (biographical sketches provided in Appendix A) to develop the workshop agenda (see Appendix B). The workshop statement of task is shown in Appendix C.
Workshop speakers and participants discussed research challenges related to the following topics:
- Critical analysis of the current state of machine learning and artificial intelligence (AI) algorithms and systems that are used to generate analytic products from disparate structured and unstructured data types and to detect anomalies;
- Statistical methods that can be used to evaluate confidence hierarchies, model uncertainty, and error propagation, and manage risk as a function of time and complexity;
- Approaches for ensuring that machine-generated products compare favorably with those of trained human analysts; and
- Techniques for responding to adversarial manipulation of input data to influence analytical products by exploiting weaknesses in machine learning and AI algorithms and vulnerabilities in their implementation.
During the presentations and discussion sessions, attendees were asked to address the following questions, with particular emphasis on their role for the Intelligence Community:
- What are the key technical objectives and performance measures needed for success?
- What are the current and “next level” key performance metrics?
- What is the “level after next” of expected research and development performance?
- What is the research knowledge base?
- How can the government best prepare the scientific workforce to enhance discovery in this area?
- What are the requisite enabling technologies?
This proceedings is a factual summary of what occurred at the workshop. The planning committee’s role was limited to organizing and convening the workshop. The views contained in this proceedings are those of the individual workshop participants and do not necessarily represent the views of the participants as a whole, the planning committee, or the National Academies of Sciences, Engineering, and Medicine.
