8

Summary of Findings, Conclusions, and Recommendations

The following section summarizes the key messages and aggregates the findings, conclusions, and recommendations outlined in this report. This recap highlights that the report’s findings, conclusions, and recommendations address broader systemic, translational, and programmatic topics, in addition to more focused digital twin research needs, gaps, and opportunities.

DEFINITION OF A DIGITAL TWIN

This report proposes the following definition of a digital twin:

SYSTEMIC, TRANSLATIONAL, AND PROGRAMMATIC FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS

The report emphasizes that a digital twin goes beyond simulation to include tighter integration between models, data, and decisions. Of particular importance is the bidirectional interaction, which comprises automated and human-in-the-loop feedback flows of information between the physical system and its virtual representation.

Finding 2-1: A digital twin is more than just simulation and modeling.

Conclusion 2-1: The key elements that comprise a digital twin include (1) modeling and simulation to create a virtual representation of a physical counterpart, and (2) a bidirectional interaction between the virtual and the physical. This bidirectional interaction forms a feedback loop that comprises dynamic data-driven model updating (e.g., sensor fusion, inversion, data assimilation) and optimal decision-making (e.g., control, sensor steering).

The report emphasizes the key role of verification, validation, and uncertainty quantification (VVUQ) as essential tasks for the responsible development, implementation, monitoring, and sustainability of digital twins across all domains.

Conclusion 2-2: Digital twins require VVUQ to be a continual process that must adapt to changes in the physical counterpart, digital twin virtual models, data, and the prediction/decision task at hand. A gap exists between the class of problems that has been considered in traditional modeling and simulation settings and the VVUQ problems that will arise for digital twins.

The report highlights the role that VVUQ has played in fostering confidence and establishing boundaries for the use of predictive simulations in critical decision-making, noting that VVUQ will similarly play a central role in establishing trust and guidelines for use for digital twins across domains.

Conclusion 2-3: Despite the growing use of artificial intelligence, machine learning, and empirical modeling in engineering and scientific applications, there is a lack of standards in reporting VVUQ as well as a lack of consideration of confidence in modeling outputs.

Conclusion 2-4: Methods for ensuring continual VVUQ and monitoring of digital twins are required to establish trust. It is critical that VVUQ be deeply embedded in the design, creation, and deployment of digital twins. In future digital twin research developments, VVUQ should play a core role and tight integration should be emphasized. Particular areas of research need include continual verification, continual validation, VVUQ in extrapolatory conditions, and scalable algorithms for complex multiscale, multiphysics, and multi-code digital twin software efforts. There is a need to establish to what extent VVUQ approaches can be incorporated into automated online operations of digital twins and where new approaches to online VVUQ may be required.

Finding 2-2: The Department of Energy Predictive Science Academic Alliance Program has proven an exemplary model for promoting interdisciplinary research in computational science in U.S. research universities and has profoundly affected university cultures and curricula in computational science in the way that VVUQ is infused with scalable computing, programming paradigms on heterogeneous computer systems, and multiphysics and multi-code integration science.

Finding 2-3: Protecting privacy and determining data ownership and liability in complex, heterogeneous digital twin environments are unresolved challenges that pose critical barriers to the responsible development and scaling of digital twins.

Despite the existence of examples of digital twins providing practical impact and value, the sentiment expressed across multiple committee information-gathering sessions is that the publicity around digital twins and digital twin solutions currently outweighs the evidence base of success. Achieving the promise of digital twins requires an integrated and holistic research agenda that advances digital twin foundations.

Conclusion 2-5: Digital twins have been the subject of widespread interest and enthusiasm; it is challenging to separate what is true from what is merely aspirational, due to a lack of agreement across domains and sectors as well as misinformation. It is important to separate the aspirational from the actual to strengthen the credibility of the research in digital twins and to recognize that serious research questions remain in order to achieve the aspirational.

Conclusion 2-6: Realizing the potential of digital twins requires an integrated research agenda that advances each one of the key digital twin elements and, importantly, a holistic perspective of their interdependencies and interactions. This integrated research agenda includes foundational needs that span multiple domains as well as domain-specific needs.

Recommendation 1: Federal agencies should launch new crosscutting programs, such as those listed below, to advance mathematical, statistical, and computational foundations for digital twins. As these new digital twin–focused efforts are created and launched, federal agencies should identify opportunities for cross-agency interactions and facilitate cross-community collaborations where fruitful. An interagency working group may be helpful to ensure coordination.

• National Science Foundation (NSF). NSF should launch a new program focused on mathematical, statistical, and computational foundations for digital twins that cuts across multiple application domains of science and engineering.
  • The scale and scope of this program should be in line with other multidisciplinary NSF programs (e.g., the NSF Artificial Intelligence Institutes) to highlight the technical challenge being solved as well as the emphasis on theoretical foundations being grounded in practical use cases.
  • Ambitious new programs launched by NSF for digital twins should ensure that sufficient resources are allocated to the solicitation so that the technical advancements are evaluated using real-world use cases and testbeds.
  • NSF should encourage collaborations across industry and academia and develop mechanisms to ensure that small and medium-sized industrial and academic institutions can also compete and be successful leading such initiatives.
  • Ideally, this program should be administered and funded by multiple directorates at NSF, ensuring that from inception to sunset, real-world applications in multiple domains guide the theoretical components of the program.
• Department of Energy (DOE). DOE should draw on its unique computational facilities and large instruments coupled with the breadth of its mission as it considers new crosscutting programs in support of digital twin research and development. It is well positioned and experienced in large, interdisciplinary, multi-institutional mathematical, statistical, and computational programs. Moreover, it has demonstrated the ability to advance common foundational capabilities while also maintaining a focus on specific use-driven requirements (e.g., predictive high-fidelity models for high-consequence decision support). This collective ability should be reflected in a digital twin grand challenge research and development vision for DOE that goes beyond the current investments in large-scale simulation to advance and integrate the other digital twin elements, including the physical/virtual bidirectional interaction and high-consequence decision support. This vision, in turn, should guide DOE’s approach in establishing new crosscutting programs in mathematical, statistical, and computational foundations for digital twins.
• National Institutes of Health (NIH). NIH should invest in filling the gaps in digital twin technology in areas that are particularly critical to biomedical sciences and medical systems. These include bioethics, handling of measurement errors and temporal varia-

• tions in clinical measurements, capture of adequate metadata to enable effective data harmonization, complexities of clinical decision-making with digital twin interactions, safety of closed-loop systems, privacy, and many others. This could be done via new cross-institute programs and expansion of current programs such as the Interagency Modeling and Analysis Group.
• Department of Defense (DoD). DoD’s Office of the Under Secretary of Defense for Research and Engineering should advance the application of digital twins as an integral part of the digital engineering performed to support system design, performance analysis, developmental and operational testing, operator and force training, and operational maintenance prediction. DoD should also consider using mechanisms such as the Multidisciplinary University Research Initiative and Defense Acquisition University to support research efforts to develop and mature the tools and techniques for the application of digital twins as part of system digital engineering and model-based system engineering processes.
• Other federal agencies. Many federal agencies and organizations beyond those listed above can play important roles in the advancement of digital twin research. For example, the National Oceanic and Atmospheric Administration, the National Institute of Standards and Technology, and the National Aeronautics and Space Administration should be included in the discussion of digital twin research and development, drawing on their unique missions and extensive capabilities in the areas of data assimilation and real-time decision support.

VVUQ is a key element of digital twins that necessitates collaborative and interdisciplinary investment.

Recommendation 2: Federal agencies should ensure that verification, validation, and uncertainty quantification (VVUQ) is an integral part of new digital twin programs. In crafting programs to advance the digital twin VVUQ research agenda, federal agencies should pay attention to the importance of (1) overarching complex multiscale, multiphysics problems as catalysts to promote interdisciplinary cooperation; (2) the availability and effective use of data and computational resources; (3) collaborations between academia and mission-driven government laboratories and agencies; and (4) opportunities to include digital twin VVUQ in educational programs. Federal agencies should consider the Department of Energy Predictive Science Academic Alliance Program as a possible model to emulate.

Existing literature and documented practices focus on the creation and deployment of digital twins; little attention has been given to sustainability and maintenance or life-cycle management of digital twins. Yet sustaining a robust, flexible, dynamic, accessible, and secure digital twin is a key consideration for creators, funders, and the diverse community of stakeholders.

Conclusion 7-1: The notion of a digital twin has inherent value because it gives an identity to the virtual representation. This makes the virtual representation—the mathematical, statistical, and computational models of the system and its data—an asset that should receive investment and sustainment in ways that parallel investment and sustainment in the physical counterpart.

Recommendation 4: Federal agencies should each conduct an assessment for their major use cases of digital twin needs to maintain and sustain data, software, sensors, and virtual models. These assessments should drive the definition and establishment of new programs similar to the National Science Foundation’s Natural Hazards Engineering Research Infrastructure and Cyberinfrastructure for Sustained Scientific Innovation programs. These programs should target specific communities and provide support to sustain, maintain, and manage the life cycle of digital twins beyond their initial creation, recognizing that sustainability is critical to realizing the value of upstream investments in the virtual representations that underlie digital twins.

The report calls out a number of domain-specific digital twin challenges, while also noting that there are many research needs and opportunities that cut across domains and use cases. There are significant opportunities to achieve advances in digital twin foundations through translational and collaborative research efforts that bridge domains and sectors.

Finding 7-1: Cross-disciplinary advances in models, data assimilation workflows, model updates, use-specific workflows that integrate VVUQ, and decision frameworks have evolved within disciplinary communities. However, there has not been a concerted effort to examine formally which aspects of the associated software and workflows (e.g., hybrid modeling, surrogate modeling, VVUQ, data assimilation, inverse methods, control) might cross disciplines.

Conclusion 7-2: As the foundations of digital twins are established, it is the ideal time to examine the architecture, interfaces, bidirectional workflows of the virtual twin with the physical counterpart, and community practices in order to make evolutionary advances that benefit all disciplinary communities.

Recommendation 5: Agencies should collaboratively and in a coordinated fashion provide cross-disciplinary workshops and venues to foster identification of those aspects of digital twin research and development that would benefit from a common approach and which specific research topics are shared. Such activities should encompass responsible use of digital twins and should necessarily include international collaborators.

Finding 7-2: Both creation and exploration of the applications of digital twins are occurring simultaneously in government, academia, and industry. While many of the envisioned use cases are dissimilar, there is crossover in both use cases and technical need within and among the three sectors. Moreover, it is both likely and desirable that shared learning and selective use of common approaches will accrue benefits to all.

Recommendation 6: Federal agencies should identify targeted areas relevant to their individual or collective missions where collaboration with industry would advance research and translation. Initial examples might include the following:

• Department of Defense—asset management, incorporating the processes and practices employed in the commercial aviation industry for maintenance analysis.
• Department of Energy—energy infrastructure security and improved (efficient and effective) emergency preparedness.
• National Institutes of Health—in silico drug discovery, clinical trials, preventative health care and behavior modification programs, clinical team coordination, and pandemic emergency preparedness.
• National Science Foundation—Directorate for Technology, Innovation and Partnerships programs.

Conclusion 7-3: Open global data and model exchange has led to more rapid advancement of predictive capability within the Earth system sciences. These collaborative efforts benefit both research and operational communities (e.g., global and regional weather forecasting, anticipation and response to extreme weather events).

Conclusion 7-4: Fostering a culture of collaborative exchange of data and models that incorporate context through metadata and provenance in digital twin–relevant disciplines could accelerate progress in the development and application of digital twins.

Recommendation 7: In defining new digital twin research efforts, federal agencies should, in the context of their current and future mission priorities, (1) seed the establishment of forums to facilitate good practices for

effective collaborative exchange of data and models across disciplines and domains, while addressing the growing privacy and ethics demands of digital twins; (2) foster and/or require collaborative exchange of data and models; and (3) explicitly consider the role for collaboration and coordination with international bodies.

The report notes that the successful adoption and progress of digital twins hinge on the appropriate education and training of the workforce, recognizing the particular importance of interdisciplinary degrees and curricula.

Finding 7-3: Interdisciplinary degrees and curricula that span computational, data, mathematical, and domain sciences are foundational to creating a workforce to advance both development and use of digital twins. This need crosses fundamental and applied research in all sectors: academia, government, and industry.

Recommendation 8: Within the next year, federal agencies should organize workshops with participants from industry and academia to identify barriers, explore potential implementation pathways, and incentivize the creation of interdisciplinary degrees at the bachelor’s, master’s, and doctoral levels.

DIGITAL TWIN RESEARCH FINDINGS, CONCLUSIONS, AND RECOMMENDATION

The study identified foundational research needs and opportunities associated with each of the elements of a digital twin: the virtual representation, the physical counterpart, the physical-to-virtual flowpath, and the virtual-to-physical flowpath. These findings, conclusions, and recommendation cover many technical areas, including multiscale, hybrid, and surrogate modeling; system integration and coupling; data acquisition, integration, and interoperability; inverse problems; data assimilation; optimization under uncertainty; automated decision-making; human-in-the-loop decision-making; and human–digital twin interactions.

Conclusion 3-1: A digital twin should be defined at a level of fidelity and resolution that makes it fit for purpose. Important considerations are the required level of fidelity for prediction of the quantities of interest, the available computational resources, and the acceptable cost. This may lead to the digital twin including high-fidelity, simplified, or surrogate models, as well as a mixture thereof. Furthermore, a digital twin may include the ability to represent and query the virtual models at variable levels of resolution and fidelity depending on the particular task at hand and the available resources (e.g., time, computing, bandwidth, data).

Finding 3-1: Approaches to assess modeling fidelity are mathematically mature for some classes of models, such as partial differential equations that represent one discipline or one component of a complex system; however, theory and methods are less mature for assessing the fidelity of other classes of models (particularly empirical models) and coupled multiphysics, multi-component systems.

Finding 3-2: Different applications of digital twins drive different requirements for modeling fidelity, data, precision, accuracy, visualization, and time-to-solution, yet many of the potential uses of digital twins are currently intractable to realize with existing computational resources.

Finding 3-3: Often, there is a gap between the scales that can be simulated and actionable scales. It is necessary to identify the intersection of simulated and actionable scales in order to support optimizing decisions. The demarcation between resolved and unresolved scales is often determined by available computing resources, not by a priori scientific considerations.

Recommendation 3: In crafting research programs to advance the foundations and applications of digital twins, federal agencies should create mechanisms to provide digital twin researchers with computational resources, recognizing the large existing gap between simulated and actionable scales and the differing levels of maturity of high-performance computing across communities.

Finding 3-4: Advancing mathematical theory and algorithms in both data-driven and multiscale physics-based modeling to reduce computational needs for digital twins is an important complement to increased computing resources.

Finding 3-5: Hybrid modeling approaches that combine data-driven and mechanistic modeling approaches are a productive path forward for meeting the modeling needs of digital twins, but their effectiveness and practical use are limited by key gaps in theory and methods.

Finding 3-6: Integration of component/subsystem digital twins is a pacing item for the digital twin representation of a complex system, especially if different fidelity models are used in the digital twin representation of its components/subsystems.

Finding 3-7: State-of-the-art literature and practice show advances and successes in surrogate modeling for models that form one discipline or one component of a complex system, but theory and methods for surrogates of coupled multiphysics systems are less mature.

Finding 3-8: Digital twins will typically entail high-dimensional parameter spaces. This poses a significant challenge to state-of-the-art surrogate modeling methods.

Finding 3-9: One of the challenges of creating surrogate models for high-dimensional parameter spaces is the cost of generating sufficient training data. Many papers in the literature fail to properly acknowledge and report the excessively high costs (in terms of data, hardware, time, and energy consumption) of training.

Conclusion 3-2: In order for surrogate modeling methods to be viable and scalable for the complex modeling situations arising in digital twins, the cost of surrogate model training, including the cost of generating the training data, must be analyzed and reported when new methods are proposed.

Finding 4-1: Documenting data quality and the metadata that reflect the data provenance is critical.

Finding 4-2: The absence of standardized quality assurance frameworks makes it difficult to compare and validate results across different organizations and systems. This is important for cybersecurity and information and decision sciences. Integrating data from various sources, including Internet of Things devices, sensors, and historical data, can be challenging due to differences in data format, quality, and structure.

Conclusion 4-1: The lack of adopted standards in data generation hinders the interoperability of data required for digital twins. Fundamental challenges include aggregating uncertainty across different data modalities and scales as well as addressing missing data. Strategies for data sharing and collaboration must address challenges such as data ownership and intellectual property issues while maintaining data security and privacy.

Conclusion 5-1: Data assimilation and model updating play central roles in the physical-to-virtual flow of a digital twin. Data assimilation techniques are needed for data streams from different sources and for numerical models with varying levels of uncertainty. A successful digital twin will require the continuous assessment of models. Traceability of model hierarchies and reproducibility of results are not fully considered in existing data assimilation approaches.

Conclusion 5-2: Data assimilation alone lacks the learning ability needed for a digital twin. The integration of data science with tools for digital twins (including inverse problems and data assimilation) will provide opportunities to extract new insights from data.

Finding 6-1: There is a need for digital twins to support complex trade-offs of risk, performance, cost, and computation time in decision-making.

Conclusion 6-1: There is value in digital twins that can optimally design and steer data collection, with the ultimate goal of supporting better decision-making.

Finding 6-2: In many cases, trusted high-fidelity models will not meet the computational requirements to support digital twin decision-making.

Finding 6-3: Theory and methods are being developed for reinforcement learning and for dynamically adaptive optimization and control algorithms. There is an opportunity to connect these advances more strongly to the development of digital twin methodologies.

Finding 6-4: Models and data play a synergistic role in digital twin decision-making. The abundance or scarcity of data, complexity of the decision space, need to quantify uncertainty, and need for interpretability are all drivers to be considered in advancing theory and methods for digital twin decision-making.

Conclusion 6-2: Communicating uncertainty to end users is important for digital twin decision support.

Finding 6-5: In addition to providing outputs that are interpretable, digital twins need to clearly communicate any updates and the corresponding changes to the VVUQ results to the user in order to engender trust.

Conclusion 6-3: While the capture of enough contextual detail in the metadata is critical for ensuring appropriate inference and interoperability, the inclusion of increasing details may pose emerging privacy and security risks. This aggregation of potentially sensitive and personalized data and models is particularly challenging for digital twins. A digital twin of a human or component of a human is inherently identifiable, and this poses questions around privacy and ownership as well as rights to access.

Conclusion 6-4: Models may yield discriminatory results from biases of the training data sets or introduced biases from those developing the models. The human–digital twin interaction may result in increased or decreased bias in the decisions that are made.