13

Key Themes from the Workshop

The final session, moderated by Thomas R. Kurfess, planning committee chair, provided a summary of the most important themes that had emerged over the 3 days of the workshop that were offered by three planning committee members: Teresa Clement, Raytheon; Melissa Orme, The Boeing Company; and Alyson G. Wilson, North Carolina State University.

Clement began by making two overarching observations concerning the workshop. One of the best things about the workshop, she said, was the way that different communities—engineers, statisticians, and artificial intelligence/machine learning (AI/ML) experts and practitioners—came together to speak about their most relevant capabilities across disciplines with the goal of achieving “a data-driven maturation of all the processes within the AM value stream and life cycle.” She followed that with a look to the future, saying that the workshop attendees should hold each other accountable for being true stewards of statistically accurate and validated representations of the additive manufacturing (AM) value stream.

After that, Wilson and Orme joined to talk about the key challenges and opportunities that had been identified in the workshop. They organized these under four themes: (1) approaches that enhance dimensional accuracy in AM and post-additive processes and dimensional stability in usage, including potential innovations in process, control, and materials design; (2) recent advances and future directions in statistics, data analytics, AI, and ML that have the potential to aid automated procedures for machine calibration and processing parameters along the toolpath

correction in AM processes; (3) the issues associated with the rapid advance of AM material qualification and part certification; and (4) gaps and opportunities.

ENHANCING DIMENSIONAL ACCURACY AND STABILITY

Speaking about enhancing dimensional accuracy and stability, the panelists began by discussing the challenges. Orme said that there is an urgent need to rapidly discover, scale up, and implement new material solutions, with an emphasis on extreme environmental materials. It will be important to qualify materials faster, she said, and the vision is to use process-informed digital twin coupling experiments with modeling. Clement said that improving the AM of parts will require process-informed material design as well as the development and integration of novel in situ measurement sensors and methods. Another challenge, Orme said, is the fact that processes are defect-prone; this must be taken into account during the design phase.

There are multiple data-related challenges. To begin with, Clement said, data alone are not sufficient; data need to be turned into information, which enables intelligence using a connected digital thread. Another challenge, Wilson added, is understanding which data are important and identifying what matters. The standardization of metadata must also be addressed, Orme said; adoption is not within reach without standardization. More generally, Wilson added, working with data is a challenge, as they need to be integrated/connected, stored, curated, shared, protected, and made usable for analysis.

The panelists then moved to the opportunities related to enhancing dimensional accuracy and stability in usage. One such opportunity, Wilson said, is AI’s ability to handle complex, high-dimensional data. However, she added, there are open challenges in modeling across multiple data types. Orme pointed to the creation of the digital twin as a major opportunity, as it allows optimization in geometry, process, and material. Incorporating physics into the modeling of AM is another opportunity that should be pursued, she said, as it is key to many potential advances. She also encouraged increased collaboration, particularly in the creation of benchmark datasets.

Clement suggested that it will be important to learn to distinguish between defects and microstructure evolution through all phases of part production (advanced manufacturing and post-process) as well as in usage. Furthermore, she said, it will be valuable to integrate real-time controls for process–parameter optimization and defect mitigation.

Orme said that reducing variation in feedstock will help improve the quality and consistency of manufactured parts. She also suggested

that post-processing modeling be used, for example, in studying heat treatment versus microstructure, but it will still be necessary to figure out which microstructures are good and which defects are permissible. Last, Orme said there will be great benefit in designing for the entire value stream, including designing for post-processing and designing for inspectability.

STATISTICS, DATA ANALYTICS, ARTIFICIAL INTELLIGENCE, AND MACHINE LEARNING

In the area of statistics, data analytics, AI, and ML, the panelists identified a large number of challenges and opportunities. Beginning with the challenges, Wilson first pointed to the challenge of using full-factorial experiments naïvely. They typically are not feasible where there are large numbers of variables. Instead, there is a body of work on using ML for such things as designing experiments, figuring out where to take additional data, and determining which factors are driving variation.

A related issue, Orme continued, is understanding the physics of the relevant processes. This is necessary in order to understand which variables are important, which defects are acceptable, and which data to gather. It is important to use physics-based/physics-informed models to understand process. Knowledge is key.

Wilson spoke about the importance of getting people to understand that measurements are not ground truth and that there is uncertainty in the measurements that are being used to train the models—and thus they need to adjust their approaches accordingly. Uncertainty in all of its forms must be accounted for, she said. There is uncertainty in the material, in the physics, in the process, in the measurements, and so on, and it is important to keep improving the ways in which that uncertainty is mitigated.

Clement emphasized the importance of motivating people to capture machine and process data reliably and regularly, which is a big challenge even with robust standards and data schemas. One approach to dealing with this challenge is to automate data capture in pre-AM, AM, and post-AM processes, she said. It would also be valuable to automate procedures for machine calibration, she added. While this is not uniquely an AM need, the significant machine variability and the opportunities for the capture of metadata and data on equipment status/health make it critical to focus on automated machine calibration in order to stabilize the data sources flowing into the AI/ML analytical models. Last, Clement said it would be valuable to developing automated procedures for optimizing process parameters along the toolpath. This issue is unique to AM, and developing the procedures would make it possible to predict material

characteristics in a part family or when localized for specific key features or characteristics within the part.

Next, the panelists identified a number of opportunities related to statistics, data analytics, AI, and ML. Wilson said that new methods are needed for continuous verification, validation, and uncertainty quantification (VVUQ) in order to be able to adapt to changes in models, data, and decisions. One issue that must be addressed is how the models should be applied and linked in sequence. Wilson also said that improved methods are needed for quantifying uncertainty in the newer AI models.

Orme said that adaptive process control is needed so that parameters can be adjusted in response to changing conditions. Furthermore, she added, work needs to be done to understand how the models change and how they should be updated. Also, Orme said, it will be valuable to automate as much as possible, so researchers need to determine what can be automated in software, such as statistical process design, the control of defects, scan path design, and microstructure modeling. Similarly, Wilson added, it would be valuable to maximize transfer learning. But how much transfer learning can be done? How can calibrations be made applicable to different parts, different geometries, different machines, and different materials?

Clement stated that it would be valuable to move toward a reliable model of developing and validating digital twins, but what will that entail? Existing VVUQ techniques provide the foundation for data analysis, surrogate modeling, the design of experiments, and model calibration, which are critical elements of digital twins. On that topic, Wilson added that digital twins augment VVUQ with an emphasis on real-time continuous updating of models (predictions), data analysis, and decisions (control actions).

In addition to these challenges and opportunities, Orme noted that workshop speakers had also described a great deal of interesting new science as well as many valuable tools and approaches that will help move the field forward. As examples, she listed five tools, approaches, and techniques described by the speakers:

• A tool that studies melt pool geometry and translates that to porosity. This could be used to accelerate and inform qualification. A disadvantage, however, is that it currently takes 7 years to qualify new materials.
• Vision-based metrology. The objective here is to get better data.
• Computer vision, which enables the automatic measurement of parts made with AM and which scales for different production volumes.
• Advances in printing aortic valves and using active learning to reduce learning cycle time.
• Manipulation of microstructure with scanning strategy.

ADDITIVE MANUFACTURING MATERIAL QUALIFICATION AND PART CERTIFICATION

As AM moves forward and becomes an increasingly vital piece of the manufacturing sector, with parts created through AM placed in technologies of all sorts, the issues of qualifying the materials used in AM and certifying the parts produced by AM will become increasingly important. The panel identified a number of challenges related to AM material qualification and part certification. Orme said that it will be important to understand the underlying physics, while Wilson pointed to the need to understand what data to collect and what variables are important. Another challenge, Clement said, will be addressing the inconsistent and incomplete tagging of metadata, data definitions, and process parameters throughout the AM community that prevent digital reuse and integration of datasets across facilities and machines.

Last, Orme pointed to two other challenges: integrating the entire value stream, including post-processing, into digital threads and AI models, and developing AI models that can derive features that are physically interpretable.

Orme also addressed an issue that had come up the previous day concerning the use of AI for small lots versus large lots. The question had been raised whether ML was different for small-lot manufacturing versus large-lot manufacturing, and the answer given was that, yes, it is different because ML would not be used for small-lot manufacturing. Orme disagreed and argued that ML can be used for such things as materials characterization or predictive quality control where the information and insights provided by ML can be just as valuable for a handful of parts as for thousands. The materials system characterized by ML “doesn’t know if you’re making 3 parts or 3,000 parts,” she said.

GAPS AND OPPORTUNITIES

Last, the panel members identified a number of gaps and opportunities in the general area of AM. To begin with, Orme said, the current qualification and certification is conducted with static process parameters because of guidance from the Federal Aviation Administration (FAA), but static process parameters do not allow for real-time corrections or adaptive parameters owing to geometry changes. The future, she said, will likely depend on dynamic process parameters, which will provide higher control over part quality. A second issue, Orme said, is that it will be necessary to work with regulators to shift the current mindset on performing qualification and certification with adaptive process parameters. Last, she added, sensor signatures that provide information on AM process

characteristics such as melt pool attributes can be proxies for qualification testing in the future.

Wilson said that there is a need to understand what data as well as how many data are needed to train a model to the accuracy and precision needed for a particular application. These are difficult problems that will require engineers and data scientists to work together to make progress.

Clement said that it will be useful to have joint regulator workshops—with FAA, the Air Force, and so on—to accelerate the process of part acceptance by standardizing in situ process monitoring for process verification. She also suggested that a collaborative project be developed to verify and increase the number of datasets within the Digital Innovation Center of Excellence to include data from actual industrial fabricated AM parts. Last, she said, it would be useful to carry out a collaborative project to understand post-processing impacts on the process outcomes estimations, uncertainty, robustness, and repeatability.