12

Barriers to the Rapid Advance of Additive Manufacturing Material Qualification and Part Certification

The workshop’s final panel was moderated by planning committee member Adrian S. Onas, Webb Institute of Naval Architecture.

BARRIERS TO RAPID QUALIFICATION AND CERTIFICATION OF ADDITIVELY MANUFACTURED PARTS BY THE U.S. AIR FORCE

Mark D. Benedict, Air Force Research Laboratory, began by noting that qualification and certification are important to the Air Force because they are crucial to ensuring the safety of its planes. In determining the airworthiness of a particular part, the Air Force takes into account five basic considerations: stability, producibility, characterization, predictability, and supportability. And when it comes to additively manufactured materials, he said, the Air Force has great concerns relative to qualification and certification.

For those systems, such as a thruster engine for a spacecraft, whose service life may be only 10 hours, the Air Force has had greater success with additively manufactured materials, because testing is sufficient to predict how it will perform for its service life. But for something like a B-52 airframe, which must last for decades, testing will not suffice, and it becomes necessary to rely on models for qualification and certification, which can be a long process. For instance, it took 7 years for the Air Force to reach the point where it was willing to test-fly fused deposition materials on a crewed flight.

In short, he said, the basic problem is that additive manufacturing (AM) materials can be qualified, but the process is inefficient and negates many of AM’s key benefits. So qualification really is the barrier to the Air Force adopting AM parts. Currently, the Air Force is focused on qualifying a small subset of additive processes, and much remains to be done for AM parts to play a larger role.

AUTOMATED METROLOGY AND PART QUALITY PREDICTION

Davis J. McGregor, University of Maryland, spoke about a way to automate the job of detecting defective parts produced by AM. McGregor is the director of the MIRAGE Lab (MIRAGE stands for Manufacturing Intelligence Research and Advanced Geometry Evaluation). The laboratory does work in two core areas. The first is advanced metrology, in which computer vision and software automation are used to create scalable methods for feature recognition and extraction. The second is part quality prediction, which identifies sources of variability in parts, links them to process physics, and applies artificial intelligence/machine learning (AI/ML) strategies to predict the geometry of an additively manufactured part.

The goal is to answer the key AM question: How do manufacturers know if the parts are good? The solution requires a scalable metrology that is capable of measuring complex parts combined with analytical methods that can use those measurements to provide manufacturing process insights into the quality of those parts. Indeed, McGregor described experimental work that showed it is possible to predict the geometry of AM parts with high accuracy and low cost.

Although this is still a work in progress, it is already clear that this approach can provide valuable information quickly and efficiently, not just on which AM parts may deviate too much from design criteria, but also on which potential sources of variability in the manufacturing process are the most important and on what effects these sources of variability have on the finished product.

USING MACHINE LEARNING FOR ADDITIVE MANUFACTURING

Aaron Stebner, Georgia Institute of Technology, started off by noting the growing importance of ML, which has been identified as a foundational element that will help meet the current challenges in AM and, more generally, materials research.

But ML is useful only if it is done correctly, so Stebner offered a list of nine elements that the editors of the journal Additive Manufacturing expect from papers submitted that use ML.

1. Delineate the use of multiple statistical models.
2. Provide a clear definition of the process model formulation.
3. Provide a clear definition of the dataset as it is used for statistical modeling.
4. Provide evidence that the dataset supports statistical modeling.
5. Provide evidence that ML is necessary.
6. Document how the basic statistical properties of the dataset motivate the choice and selection of the statistical or ML model.
7. Document and discuss the parameter estimation (training) of the model.
8. Completely assess the performance of the statistical model’s predictions.
9. Document the scientific or engineering impacts gained from using the model.

In addition, Stebner said, the paper should document the edge cases for the model—that is, where the model is not expected to work because the case is outside the data the model was trained on.

He concluded by naming two related challenges for the future: automating everything upstream of ML and learning how to do ML modeling across multiple data types.

DISCUSSION

The topic of the qualification of AM parts was raised—specifically, how difficult it might be and whether it might negate many of the benefits of AM. Benedict offered an example of how a mission-critical AM part was certified for use on a fighter aircraft. They first demonstrated that they understood how to make good-quality coupons and density cubes; next, they printed a number of the parts to make sure there were no geometrical variances; and last, they tested many of them to failure. It was not an optimal case, he said, in the sense that it took a relatively long time to move the part into production, but the AM part did get adopted because it was shown to be higher quality than the part it was replacing. It is these situations—where AM is used to replace a lesser-quality material or a highly variable manufacturing practice—where it is most likely that there will be a positive business case for additive parts in the near future.

McGregor said that his feature recognition metrology system can be applied in three dimensions and, specifically, to computed tomography (CT) scans. There are difficulties in determining exactly where the surface of a part is, particularly with metals, but in general it is possible to use the method to recognize features in three dimensions.

Benedict spoke about the challenges of inspecting AM parts throughout their lifetimes to make sure that they are still acceptable. Any additive part that will fly must be subjected to volumetric inspection, he said, which can be an onerous requirement. CT is the go-to process now, and, as McGregor explained, its interpretation is difficult. So inspection and tracking of AM parts over their lifetimes are open areas of research.

Stebner spoke on the issue of requiring scientists who publish articles on AI models to provide the full training and testing data used to make the models. There is no agreement on this in the field, he said, or even on his editorial board. Many researchers, particularly those in industry, are reluctant to provide it in the open literature databases that they have paid to collect. He suggested that companies can internally require that level of documentation so that it can be checked, if necessary, and that academics should at least talk in detail about the processes they used to train their models even if they do not provide all of the data. Personally, he said, he would require researchers to publish their datasets, but on the other hand, he sees the value of publishing valuable research that might not otherwise be published if the backing datasets were also required to be published.

Stebner also spoke about the potential of ML models that might be applied to AM. There are many ML tools, he said, and they tend to offer capabilities that go far beyond what is needed for AM. Of these new techniques, he noted that people grasp onto them, but he encourages the community to think critically about what is the right tool for the task at hand. He went on to say that most AM data tend to be of different types and at different length scales. AM data also tend to be tied to the physics of the problem being solved. These are challenges with which the computer science community will typically not be as familiar and for which the engineering community will need to take the lead. “We have very few data, but they’re very rich, and they’re connected across all these different length and time scales,” he said, “and I think that’s a frontier that hasn’t been cracked yet.”

Expanding on how one can modify models to be useful in analyzing AM issues, Stebner gave two examples of the sorts of approaches he has used successfully. In one case, he recasts the output of a finite element model as a functional form: “y equals some function of my inputs plus some parameters.” Sometimes, he continued, “that means fitting a surrogate to your finite element data that’s of the same format as the machine learning data model” and dealing with the combined uncertainties. In a second case, where he attempted to use ML to make sense of data on the processing and properties of specific types of alloys being developed for NASA and Boeing, the ML approach was not working at all in the sense that it was not able to predict properties of interest from the data they had assembled; however, when Stebner inserted a growth kinetics model

about how phase transformations decay exponentially in time, he was able to produce a model that could predict those properties.

In response to a question about whether ML could be used to solve some of the machine-to-machine reproducibility issues in AM, Benedict said that he thought it could be, particularly in the case of AI-assisted machine calibration. For example, calibrating the lasers used in LPBF is a tedious, time-consuming process that can take several days, but using an AI-assisted process should speed up the process and bring different machines into close alignment. Given that some build runs can stretch over several months, during which time the lasers can lose calibration owing to drift, the AI-assisted calibration could be particularly useful for calibrating machines in the middle of runs to compensate for such drift.