An Assessment of Selected Research Programs and Goals of the Engineering Laboratory at the National Institute of Standards and Technology: Fiscal Year 2024 (2025)
Chapter: 3 Advanced Manufacturing Goal: Measurement Science for Additive Manufacturing Program
3
Advanced Manufacturing Goal: Measurement Science for Additive Manufacturing Program
This assessment of the Advanced Manufacturing Goal in the Measurement Science for Additive Manufacturing (MSAM) program is based on the description of the program provided to the panel:
To develop and deploy measurement science that will enable rapid design-to-product transformation through advances in: material characterization; in-process sensing, monitoring, and model-based optimal control; performance qualification of materials, machines, processes and parts; and end-to-end digital implementation and integration of metal additive manufacturing processes, parts, and systems (EL).1
The panel noted the context of this program within the National Institute of Standards and Technology (NIST) for which the overall mission is providing the trusted data, best practices, test methods, and protocols to provide a scientific foundation for government, industry, and international standards. In summary, all the work conducted at NIST in support of this goal ought to be in service of establishing, developing, and refining measurement science to support industry and industrial standards.
Considering this context, the primary objective of the MSAM program, as stated above, is considered timely and appropriate. It is inclusive of most of the major themes of additive manufacturing (AM) research and development (R&D), with the possible exception of sustainability and supply chain issues, although those themes may be implied by the focus on end-to-end integration and implementation of AM. It is important to note that the entire program currently focuses on metal AM (and in practice, metal laser powder bed fusion), as explicitly mentioned in the primary objective, whereas the title of the program (MSAM) is focused on AM more broadly. The panel will return to this point when discussing challenges and opportunities, because metal powder bed fusion represents only a fraction of industrial practice, market share, and R&D.
To support its overall objective, the MSAM program was organized into seven projects from 2018 to 2023, as follows:
- AM Feedstocks, Machine, and Process Qualification
- AM Part Qualification
- AM Machine and Process Control Methods
- Metrology for Real-Time Monitoring of AM
- Metrology for Multi-Physics AM Model Validation
- Data Integration and Management for AM
- Data-Driven Decision Support for AM
___________________
1 The National Institute of Standards and Technology (NIST) document “Measurement Science for Additive Manufacturing” was obtained by the panel on May 6, 2024, and is available are available in the public access file for this study (Email: paro@nas.edu).
The MSAM program has established a set of projects that are well aligned with NIST’s mission. The goals of the projects are timely and appropriate. The group’s productivity is very good in terms of service to professional societies, and the work has been recognized with significant awards from the government and professional societies. The group continues to lead and contribute to a broad suite of standards that are relevant to its mission. The experimental metal AM testbed and related work in metrology and controls are advancing the state of the art and serving a very important role in industry-related validation and qualification. The research under the MSAM umbrella has the potential to significantly advance measurement science in the field of AM.
There is room for improvement in two specific areas: strategic planning and impact. In this chapter, several areas where resource allocation and outcomes would be improved by strategic planning are described. A sound strategic plan will lead to success in the second area, impact. Both strategic planning and impact relate to NIST’s mission to directly support U.S. industry and are identified as overarching themes, with key recommendations, in Chapter 7.
ASSESSMENT OF TECHNICAL PROGRAMS
In terms of addressing NIST’s mission of advancing measurement science to support industry, some projects clearly meet that mission and provide outstanding technology and thought leadership to support it, whereas others appear quite disconnected from it. The projects that the panel believes are on the right track are addressed under “Accomplishments,” and the ones the panel believes are not as well connected to the mission are addressed in “Opportunities and Challenges.”
Accomplishments
The projects on AM Machine and Process Control Methods, Metrology for Real-Time Monitoring of AM, and Metrology for Multi-Physics AM Model Validation are examples of projects that have a significant impact on measurement science for industrial applications. The panel was especially impressed with ongoing work to establish one-of-a-kind instrumentation and fabrication equipment to support process monitoring and control and to establish data sets that can be used widely for calibration and process improvement by industry professionals. In an era in which most commercial machines are closed (or at least partially closed) in terms of process control and monitoring, these open architecture machines and monitoring capabilities are critical for moving the industry forward.
The AM Part Qualification project and Data-Driven Decision Support for AM project are viewed as useful but less well aligned with industry needs than the Metrology, Monitoring, and Controls projects. NIST’s support of benchmarking efforts such as the AM Bench—a continuing series of highly controlled benchmark tests for AM with modeling challenge problems, the results of which are discussed at the corresponding conference series—is excellent, as is NIST’s support of test artifacts.2 NIST is encouraged to continue to develop collaborations with other organizations, agencies, and researchers to coordinate these benchmarking efforts more widely. In terms of data-driven decision support, the impact of NIST’s efforts on standards is clear. For example, the American Society of Mechanical Engineers (ASME) standard Y14.46 addresses product definition methods and standards for AM, and ASTM International and the International Organization for Standardization (ISO) have issued a stream of standards related to design, including process-specific design guidelines and other design-related guides and standards.
Given the project topics on which MSAM focuses, the staff need to participate in as many relevant standards as possible; for example, as discussed in the “Assessment of Scientific Expertise” section, EL staff are very active in standards organizations and support the creation of new standards. Still, a very important process qualification standard, ISO/ASTM 52930, is not listed on the briefing
___________________
2 See the NIST “Additive Manufacturing Benchmark Test Series (AM-Bench)” webpage at https://www.nist.gov/ambench, accessed October 21, 2024.
materials but appears to be highly relevant to NIST projects. Based on its understanding of the state of the AM industry, the panel’s opinion is that the metrology and control projects are most directly impacting industry.
Opportunities and Challenges
There are opportunities to define the goals and objectives of some of the MSAM projects more tightly such that they are both measurable and achievable by the limited resources available in the MSAM program. Also, there are opportunities to better define NIST’s unique contributions in each area, which, given the mission of NIST, ought to be focused on measurement science, and to collaborate with leading researchers in each area to leverage their work and to avoid the potential for duplication of effort in the future at NIST.
For the AM Feedstocks, Machine, and Process Qualification project and the Data Integration and Management for AM project, it was much more difficult for the panel to discern the potential impact on industry and broader stakeholders. The AM Feedstocks, Machine, and Process Qualification project appears to trail the state of the art in feedstock characterization and the effect of feedstock on powder spreading and other process characteristics. It is not clear that this research is advancing the state of the art in related measurement science. The impact of this project’s work could be increased by having projects directed to standard methods for powder reuse and recommended methods and standards for in situ monitoring.
The Data Integration and Management for AM project aspires to significant goals of supporting data acquisition and management, but it is not clear exactly how its work supports those goals. The ties between the Data Integration and Management for AM project’s work and the experimental work at NIST more broadly and elsewhere need to be strengthened, for example, by applying the data management work to one of the experimental testbeds available at NIST, perhaps as part of the process controls and metrology projects. Data management and fusion is an important topic with significant potential to have applications, but it is not clear that the work EL is currently conducting could be implemented in an industrial setting or guide industrial implementations of data fusion and integration in a tangible way. It is not clear how the schema and frameworks under development in the Data Integration and Management for AM project can be translated into implementations.
The MSAM program’s experimental testbeds and their development are important assets to support fundamental measurement science. Testbeds as a tool could also be useful to support data-related projects and to support tighter integration between data-related projects and hardware-related projects.
Recommendation 3-1: The Measurement Science for Additive Manufacturing Program should expand the testbed concept to include data-related projects and to support tighter integration between data-related projects and hardware-related projects.
Another challenge is maintaining situational awareness of the broader field of AM, including the polymer AM industry, directed energy deposition, hybrid AM, and emerging materials. For example, powder-based metal AM, which is the exclusive focus of the MSAM program, represents a minority of AM machines and industrial applications with a significant portion of the AM industry focused on polymer AM and hybrid AM in its various forms. The panel understands that NIST programs are resource-constrained and need to focus on high-priority areas, but it is important to acknowledge that the entire program focuses on only one segment of the AM industry. This results in missed topics, such as heat treatment and post-processing, and opportunities for synergies such as the use of real data from the Machines, Monitoring, and Control project within the Data Management project. The program could have an additional impact by enhancing it to cover the entire AM process chain, including heat treatment and post-processing. A holistic consideration of the digital and physical processes required to transform feedstock material into end-use components is necessary to enable the qualification and industrial use of AM.
NIST is working with industry in some capacities, but ties with industry and academia could be strengthened. For example, closer ties with industry could help inform NIST priorities in this area. The practical impact of some projects can be enhanced through the use of industry-relevant data sets, generated internally or via industrial partners. Also, in many areas of interest to NIST, there are international experts in academia, national laboratories, and other venues that NIST could team with to move standards forward quickly and avoid trying to reinvent all that expertise in-house. In the future, sunsetting work that has become less relevant to industry or that duplicates work done in other places such as industry and academia would help with the effective allocation of limited resources.
Also, much of the current focus appears to be directed toward U.S. defense needs and objectives that, while critically important, may not align with the needs of commercial industry or the broader stakeholder community.
EL has identified five projects launched in 2024 on which the MSAM program intends to focus in the future:
- Fundamental Measurements for Metals AM
- Metrology for AM Model Validation
- Advanced Machines, Monitoring, and Controls for AM
- Advanced Informatics and Artificial Intelligence (AI) for AM
- Data Management and Fusion for AM Industrialization
It is quite early in the development of these projects, but the goals and plans indicate that they are well aligned with the EL mission. Cautionary statements about the alignment of previous projects apply to these new projects, as well, including the need to maintain the practical relevance of the data-related projects and to leverage the prodigious intellectual capital in academia and other groups concerning AI, informatics, and other related topics.
The goals for the Metrology for AM Model Validation project and the Advanced Machines, Monitoring, and Controls for AM project are appropriate, realistic, and achievable. These teams have established significant momentum toward their goals.
The goals for the Fundamental Measurements for Metals AM project are appropriate but very broad. Similarly, the goals defined for the Advanced Informatics and Artificial Intelligence for AM project and the Data Management and Fusion for AM Industrialization project are important and industry-relevant but much too broad for the team to accomplish with the resources available to them. These three projects would benefit from goals that are more focused and measurable, integrated with experimental practice, and aligned with resources that will be realistically available. As noted above, there is a vast amount of work being conducted in the AM space. It is important for EL to clearly define its role in this space. Collaborations could be very useful in helping EL to leverage and achieve a maximum return on the investment of its resources.
Recommendation 3-2: The Measurement Science for Additive Manufacturing (AM) Program’s leadership should refine the goals of the Fundamental Measurements for Metals AM, Advanced Informatics and Artificial Intelligence for AM, and Data Management and Fusion for AM Industrialization projects to be more focused and measurable, integrated with experimental practice, and aligned with realistically available resources. The program’s leadership should clearly define its role in the wider AM space and explore collaborations to leverage and achieve a maximum return on the investment of its resources.
ASSESSMENT OF SCIENTIFIC EXPERTISE
Research within the MSAM program encompasses a diverse range of expertise, including mechanical engineering, physics, materials engineering, industrial or manufacturing engineering,
computer science or electrical engineering, biomedical engineering, and AM. Team members include federal employees, associates (e.g., guest researchers and postdoctoral fellows), and students. Out of approximately 34 employees, approximately 18 are federal employees, 11 are associates, and 5 are students. The majority of federal staff and associates hold doctoral degrees. Staffing levels appear stable and near-term attrition does not appear to be of significant concern.
Accomplishments
Several MSAM personnel are deeply engaged with professional organizations, journals, and conference events, and many have been recognized for awards for their outstanding accomplishments, service, and contributions to standardization. Members of the team have a proven track record of accomplishment in relevant areas demonstrating their ability to deliver on the proposed research goals. These provide evidence that the personnel are integrated into their professional communities and building bridges between NIST and other R&D entities.
Skillsets are well aligned to the stated goals of most projects. The panel assessed that the team has a well-rounded mix of expertise across different disciplines or technical areas. The program has sufficient qualified personnel to conduct the research, including researchers, technicians, support staff, and any necessary consultants. In most cases, roles and responsibilities are defined for each team member. Research teams are exceptionally well qualified, and well composed, and have made significant contributions in fiscal year (FY) 2018–2023 projects covering AM Machine and Process Control Methods, Metrology for Real-Time Monitoring of AM, and Metrology for Multi-Physics AM Model Validation. Projects beginning in FY 2024 follow similar objectives and staff composition to past projects. Research teams are well composed for Metrology for AM Model Validation and Advanced Machines, Monitoring, and Controls for AM.
MSAM’s publication track record—with more than 60 journal papers, 45 conference proceedings, 10 NIST publications, 6 book chapters, and 23 public data sets in the past 3 years—indicates that the work is recognized to have archival value. The MSAM program is heavily involved in standards development for AM, as well, with participation in more than 40 standards over the past few years. Staff members participate in ASTM, ASME, ISO, and American National Standards Institute (ANSI) panels. This level of standards involvement is important for translating NIST work into industrial impacts. Staff hold several awards from technical societies, conferences, and journals along with awards from the Department of Commerce, NIST, and other government agencies. Notable awards include the following:
- Several ANSI Washington Academy of Sciences Leadership Awards in Manufacturing Engineering
- ASTM International Additive Manufacturing Young Professional Award
- ASTM International Award of Excellence in Standardization
- Fellow of the ASME
- Society of Manufacturing Engineering Eli Whitney Productivity Award.
Opportunities and Challenges
There are opportunities to better align and expand team expertise to meet program objectives. While the AM Feedstocks, Machine, and Process Qualification team holds diverse expertise in areas including but not limited to microscopy, in situ metrology, and AM process developments, direct prior expertise related to powder metrology is lacking. This is reflected in the limited strength, impact, and rigor of presented and published works on AM feedstocks, machines, and process qualification. The resources dedicated to the effort—4.25 full-time equivalent staff, including 5 federal employees and 2 associates—appear high relative to relevant outputs. An emphasis on qualification, despite its inclusion in the project title, is also lacking in the team’s backgrounds, outputs, and methods used. The addition of
experts knowledgeable in powder rheology, the state of the art in powder testing and modeling techniques, and qualification methods for machines and processes that use powder feedstock would benefit the team and its work.
The Data Integration and Management for AM team would benefit from closer alignment with the needs of industry and the problems limiting measurement sciences. This team would also benefit from more direct integration with teams generating experimental data sets. Both this team and the Data-Driven Decision Support for AM team could be enhanced through additional team members or collaborations with experts, particularly industry practitioners in data science and AI. Upskilling current staff or the addition of experts in applied data science and AI is also warranted. As projects beginning in FY 2024 follow similar objectives and staff composition to past projects, opportunities and challenges associated with the AM Feedstocks, Machine, and Process Qualification, Data Integration and Management for AM, and Data-Driven Decision Support for AM projects also apply to the Fundamental Measurements for Metals AM, Advanced Informatics and Artificial Intelligence for AM, and Data-Driven Decision Support for AM projects.
Recommendation 3-3: The Measurement Science for Additive Manufacturing program should better align team member expertise with practical, industrial measurement science needs. This can be accomplished through upskilling in applied data science and artificial intelligence. Also, they should add experts with relevant skills, such as powder rheology, the state of the art in powder testing and modeling techniques, and qualification methods for machines and processes that use powder feedstock. Finally, more direct engagement with industry would greatly help.
BUDGET, FACILITIES, EQUIPMENT, AND HUMAN RESOURCES
Budget
From the information provided, it appears that the MSAM program budget is adequate for continuing the current level of productivity. It is realistic and sufficient to cover all the anticipated expenses of the research program. However, an increased budget would enable the team to more fully address their objectives and expand the reach of their work. This includes personnel costs, equipment, materials, travel, and any potential contingencies. The budgets for the projects in the MSAM program are shown in Figure 3-1.
Equipment and Facilities
The MSAM program has access to the necessary equipment and facilities required for its research, including specialized laboratory equipment, computational resources, software licenses, and dedicated research spaces. The work by the team during FY 2018–2023 was heavily focused on the metal laser powder-bed process using a commercial machine (an EOS M290/M270) and an in-house-developed system that is ground-breaking in terms of its ability to control the power input and to measure and predict in situ temperature, which are all important to the quality of the final produced part. The in-house system provides information that is very useful to better understand process variability and inconsistent powder-bed performance. However, the other equipment available to the team—such as the Optomec LENS MR7 and the ExOne Mlab—are under used. More effort needs to be directed to the use of other additive modalities. Efforts are needed to plan for the replacement of commercial equipment in a time frame that is consistent with what is happening in industry.
NOTES: The projects in yellow fall under the Intelligent Systems Division. The projects in blue fall under the Systems Integration Division. AM, additive manufacturing.
SOURCE: Courtesy of NIST Engineering Laboratory.
Inconel 625 has been the main alloy studied so far; there is an indication of a pending report on Inconel 718 and CoCr alloys, but these were not discussed during the panel meeting. Studies on other widely used alloy types in the industry, such as Ti-6Al-4V, 316L, AlSi10Mg, and F357, in addition to polymers, are needed to help the U.S. industry and the broader stakeholder community.
The panel was unable to fully assess the equipment needs for metrology and data integration modules (FY 2021–2023) based on the information provided during the meeting.
Defect detection is listed as a priority for work in FY 2024. The objective is to develop a new artifact for evaluating different algorithms for defect detection in AM parts using X-ray computed tomography. The panel suggests including other non-destructive testing methods such as ultrasound and thermography in this defect evaluation program (Duarte et al. 2021). These other non-destructive testing processes are more readily accessible to small- and medium-sized businesses.
Human Resources
As noted above, the MSAM team is a cross-disciplinary one with some individuals who are world-renowned and participate in or organize conferences to share the results of their research and participate in many standards panels. The inclusion of other AM processes, alloys, and non-destructive testing modalities may be a tough challenge given the budget and human resources available. The panel suggests working in collaboration with other government laboratories and agencies to access additional expertise without necessarily having to bring it all onto EL’s staff.
EFFECTIVENESS OF DISSEMINATION EFFORTS
The MSAM research team has established collaborations with industry, academia, and institutions with relevant expertise. Staff members participate in ASTM, ASME, ISO, and ANSI panels, workshops, and committees and have contributed to more than 40 standards. There are active
engagements with small businesses, large corporations, industry consortia, government agencies, and over a dozen universities. To ensure efforts have a significant, practical impact on the U.S. economy, further development of professional networks with industry, particularly beyond the defense industrial base, is necessary. Such engagement can provide access to additional resources (e.g., real data sets), knowledge (e.g., industry-specific qualification processes), and potential opportunities for broader impact.
As discussed above in “Assessment of Scientific Expertise,” the research team’s publication record in reputable journals, conferences, and NIST reports demonstrates their ability to disseminate research findings effectively. Figure 3-2 shows a plot of the publication rate of research products, including peer-reviewed literature publications, book chapters, conference papers, NIST reports, and data sets for each of the projects. The rate is defined as the number of research publications per full-time equivalent (FTE) per year. The number of FTEs includes staff with PhD, MS, and BS degrees. The five new projects for FY 2024 are too new to assess the number of publications that are under review or are in preparation. The seven projects that ended in FY 2023 all had publication rates greater than one per FTE per year. Four projects had publication rates greater than two per FTE per year. The panel’s experience from managing research staff is that publication rates that are less than two per FTE per year are less than might be expected from staff who are dedicated to research full time.
Engagement with industry is a weakness of the MSAM program. There is an overreliance on literature publications as opposed to the direct transfer of knowledge to industry partners. The reliance on technical publications to transfer knowledge to industry may not be the optimal choice. Not all industries follow the peer-reviewed literature closely. Close direct engagement with industry appears to be lacking in this program. It follows that the program has a partial strategy and metrics for disseminating its research findings. Therefore, the projects only marginally meet expectations for dissemination strategy and metrics. One option to improve the visibility of NIST research to industry might be to maintain a mailing list of relevant industrial staff and use that list to make notifications of new publications. Another option would be to increase the number of NIST reports, especially for areas that are industry-relevant but less likely to result in peer-reviewed papers.
The MSAM program has an incomplete plan for ensuring the long-term accessibility and impact of its research outputs. This includes archiving data, making findings openly accessible, providing access to computer codes, and developing strategies for ongoing knowledge translation. Peer-reviewed papers could be made openly accessible, consistent with intellectual property rights. The project would benefit from developing and executing a plan to archive its research products and ensure they are readily available and accessible to stakeholders.
Recommendation 3-4: The Measurement Science for Additive Manufacturing program should explore other ways to disseminate the results of its work directly to industry. These could include mailing lists to inform industry stakeholders of new publications in a timely manner, producing more National Institute of Standards and Technology reports in areas that are industry-relevant but less likely to result in peer-reviewed papers, and making as many peer-reviewed papers openly accessible as possible.
REFERENCE
Duarte, V.R., T.A. Rodrigues, M.A. Machado, J.P.M. Pragana, P. Pombinha, L. Coutinho, C.M.A. Silva, et al. 2021. “Benchmarking of Nondestructive Testing for Additive Manufacturing.” 3D Printing and Additive Manufacturing 8(4). https://www.liebertpub.com/doi/10.1089/3dp.2020.0204.
NOTE: AM, additive manufacturing.
SOURCE: Data courtesy of NIST Engineering Laboratory.
