Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force

Webinar

Report Release: Test and Evaluation Challenges for AI-Enabled Systems for the Department of the Air Force

September 7, 2023

12:00 PM - 1:00 PM (ET)

Past

About

This report release webinar will discuss the new National Academies Report Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force.

Link to join the Webinar: https://nas-sec.zoomgov.com/j/1601879929

Additional Information

Is it a Closed Session event?

Not Closed

Contact

Ryan Murphy

(202) 334-3276
rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Data-Gathering Meeting #7

March 13, 2023

9:00 AM - 4:00 PM (ET)

Past

Agenda

Download the Agenda

About

This is a data-gathering meeting for the study on Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Data-Gathering. The agenda will be posted once finalized.

Additional Information

Is it a Closed Session event?

Not Closed

Contact

Ryan Murphy

(202) 334-3276
rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Workshop

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force - Workshop

June 27 - 29, 2022

Past

Agenda

Download the Agenda

About

A three-day data-gathering workshop to gather data and engage experts in order to guide a study on testing and evaluation of artificial intelligence-enabled systems under operational conditions in the Department of the Air Force.

Additional Information

Is it a Closed Session event?

Not Closed

Contact

Ryan Murphy

rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Kickoff Meeting #1, Part 1

April 22, 2022

3:00 PM - 5:00 PM (ET)

Past

Agenda

Download the Agenda

About

A kickoff meeting for the AFSB study on Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force. We will hold the required project start-up briefings and discussions, hear from the project sponsor, and begin to plan the rest of the study.

Additional Information

Is it a Closed Session event?

Partially Closed

The following committee members were present at the closed sessions of the event

May Casterline
Tom Longstaff
Rama Chellappa
Jack Shanahan
Bob Bond
Rebecca Willett
Robin Murphy
David Rosenblum
Tamara Kolda
Craig Baker
Melvin Greer
Trevor Darrell
Humberto Silva III
Nandi O Leslie

The following topics were discussed in the closed sessions:

The required conflict and bias discussion for the project was held led by Scott Weidman, DEPS Deputy Executive Director

Date of posting of Closed Session Summary:

April 22, 2022

Contact

Ryan Murphy

rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Meeting Closed

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Writing Meeting

February 21, 2023

11:00 AM - 4:00 PM (ET)

Past

About

The committee will gather to discuss the chapter drafts for the final report.

Additional Information

Is it a Closed Session event?

Institution Closed

The following committee members were present at the closed sessions of the event

Dr. Thomas Longstaff
Dr. May Casterline
Dr. Rama Chellappa
Dr. Robin Murphy
Mr. Robert Bond
Mr. John Shanahan

The following topics were discussed in the closed sessions:

Committee members discussed the organization of different draft sections of the project's report.

The following materials (written documents) were made available to the committee in the closed sessions:

None

Date of posting of Closed Session Summary:

February 22, 2023

Contact

Ryan Murphy

(202) 334-3276
rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Meeting Closed

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Writing Meeting

January 24, 2023

11:00 AM - 5:00 PM (ET)

Past

About

This meeting will be used by the committee to write and discuss different sections of the project's report draft.

Additional Information

Is it a Closed Session event?

Institution Closed

The following committee members were present at the closed sessions of the event

Dr. May Casterline
Dr. Thomas Longstaff
Dr. Tammy Kolda
Dr. Rama Chellappa
Dr. Rebecca Willett
Mr. Robert Bond
Mr. John Shanahan

The following topics were discussed in the closed sessions:

Discussed writing assignments and chapter drafts of the final report.

The following materials (written documents) were made available to the committee in the closed sessions:

None

Date of posting of Closed Session Summary:

January 25, 2023

Contact

Ryan Murphy

(202) 334-3276
rmurphy@nas.edu

Location

Keck Center

500 5th St NW

Washington DC 20001, USA

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Meeting

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Meeting #6

December 6 - 7, 2022

Past

Agenda

Download the Agenda

About

This will be the sixth meeting of the committee. This meeting will be a data gathering session.

Additional Information

Is it a Closed Session event?

Not Closed

Contact

Ryan Murphy

(202) 334-3276
rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Meeting

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Meeting #5

November 30, 2022

1:00 PM - 5:00 PM (ET)

Past

Agenda

Download the Agenda

About

This will be the fifth meeting of the committee. This meeting will be a data gathering session.

Additional Information

Is it a Closed Session event?

Partially Closed

The following committee members were present at the closed sessions of the event

May Casterline
Thomas Longstaff
David S Rosenblum
Rama Chellappa
Rebecca Willett
Robert Bond

The following topics were discussed in the closed sessions:

The committee discussed the latest progress, writing sections, and planning for the final report

The following materials (written documents) were made available to the committee in the closed sessions:

N/A

Date of posting of Closed Session Summary:

November 30, 2022

Contact

Ryan Murphy

(202) 334-3276
rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Meeting

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Meeting #4

September 28 - 29, 2022

Past

Agenda

Download the Agenda

About

This will be the fourth meeting of the committee. This meeting will be a data gathering session.

Additional Information

Is it a Closed Session event?

Partially Closed

The following committee members were present at the closed sessions of the event

Dr. May Casterline
Dr. Thomas A. Longstaff
Dr. Rama Chellappa
Lt Gen. (Ret.) John N. Shanahan
Mr. Robert A. Bond
Dr. David S. Rosenblum

The following topics were discussed in the closed sessions:

The committee discussed and edited an outline for the final report.

The following materials (written documents) were made available to the committee in the closed sessions:

None

Date of posting of Closed Session Summary:

October 06, 2022

Contact

George Coyle

(202) 334-2567
gcoyle@nas.edu

Location

Keck Center

500 5th St NW

Washington DC 20001, USA

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Meeting

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Meeting #3

August 23, 2022

11:00 AM - 4:30 PM (ET)

Past

Agenda

Download the Agenda

About

This is the third committee meeting. This will be a data gathering meeting.

Additional Information

Is it a Closed Session event?

Not Closed

Contact

Ryan Murphy

rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Closed

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force Kickoff Meeting #1, Part 2

April 25, 2022

11:00 AM - 1:00 PM (ET)

Past

About

A kickoff meeting for the AFSB study on Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force. We will continue our discussions from April 22 and begin to plan the rest of the study.

Additional Information

Is it a Closed Session event?

Institution Closed

The following committee members were present at the closed sessions of the event

May Casterline
Thomas Longstaff
Rama Chellappa
Robert Bond
Jack Shanahan
Rebecca Willett
Robin Murphy
David Rosenblum
Tamara Kolda

The following topics were discussed in the closed sessions:

The committee continued their discussion from the first kickoff meeting the previous Friday

Date of posting of Closed Session Summary:

April 25, 2022

Contact

Ryan Murphy

rmurphy@nas.edu

Disclaimer

It is essential to the National Academies mission of providing evidence-based advice that participants in any of our meetings or events avoid political or partisan statements or commentary and maintain a culture of mutual respect. The statements and presentations during our meetings or events are solely those of the individual participants and do not necessarily represent the views of other participants or the National Academy of Sciences, which is a non-partisan, tax exempt organization that includes under its Charter the National Academy of Engineering and National Academy of Medicine, and that operates the National Research Council.

Log In

Sign in to access your saved publications, downloads, and email preferences.

Former MyNAP users: You'll need to reset your password on your first login to MyAcademies. Click "Forgot password" below to receive a reset link via email. Having trouble? Visit our FAQ page to contact support.

Members of the National Academy of Sciences, National Academy of Engineering, or National Academy of Medicine should log in through their respective Academy portals.

Email (required)
Password (required)

Don't have an account?

Register

Email (required)
New password (required)
Retype password (required) Password must be at least 8 characters
I accept the terms of use
terms of use

Thank You

Thank you for creating a MyAcademies account!

Enjoy free access to thousands of National Academies' publications, a 10% discount off every purchase, and build your personal library.

Forgot Password

Enter the email address for your MyAcademies (formerly MyNAP) account to receive password reset instructions.

Reset Requested

We sent password reset instructions to your email . Follow the link in that email to create a new password. Didn't receive it? Check your spam folder or contact us for assistance.

Reset Password

New password (required)
Retype password (required) Password must be at least 8 characters

Reset Password

Your password has been reset.

Log In

Verify email

Verify Your Email Address

We sent a verification link to your email. Please check your inbox (and spam folder) and follow the link to verify your email address. If you did not receive the email, you can request a new verification link below

Subscribe

First Name
Last Name
Email (required)
I agree to receive emails from the National Academies (required)

We respect your privacy and will never share your information.

My Academies

Testing, Evaluating, and Assessing Artificial Intelligence-Enabled Systems under Operational Conditions for the Department of the Air Force

Dropdown items

Report Release: Test and Evaluation Challenges for AI-Enabled Systems for the Department of the Air Force