1

Introduction

OVERVIEW

The Office of the Under Secretary of Defense (Personnel & Readiness), referred to throughout the report as P&R, is responsible for the total force¹ management of all Department of Defense (DoD) components, including the recruitment, readiness, and retention of personnel. Its work and policies are supported by a number of organizations both within DoD, including the Defense Manpower Data Center (DMDC), and externally, including the federally funded research and development centers (FFRDCs) that work for DoD: the Institute for Defense Analyses (IDA), Center for Naval Analyses (CNA), and RAND. P&R must be able to answer questions for the Secretary of Defense such as how to recruit people with an aptitude for and interest in various specialties and along particular career tracks and how to assess on an ongoing basis servicemembers’ career satisfaction and their ability to meet new challenges. P&R must also address larger-scale questions, such as how the current realignment of forces to the Asia-Pacific area and other regions will affect recruitment, readiness, and retention.

New analytical methods for obtaining insight are of critical importance because of challenges such as maintaining a leaner but high-quality force, ensuring “reversibility” after the current personnel drawdown, managing escalating health-care costs, and identifying other cost issues early. Cur-

___________________

¹ An aggregation of military personnel, weapon systems, equipment, and necessary support, or combination thereof (DoD, Dictionary of Military Terms, 2015).

rently, personnel costs, including for training, health care, and compensation, absorb more than half of the budget for the DoD (CBO, 2012).²

While DoD makes use of large-scale data and mathematical analysis in intelligence, surveillance, reconnaissance, and elsewhere—exploiting techniques such as complex network analysis, machine learning, streaming social media analysis, and anomaly detection—these skills and capabilities have not been applied as well to the personnel and readiness enterprise. In the private sector, momentum has been building behind such efforts in recent years, and considerable work has been done for several decades. As noted in “They’re Watching You at Work,” from Atlantic Monthly, “The emerging practice of ‘people analytics’ is already transforming how employers hire, fire, and promote. . . . Predictive statistical analysis, harnessed to big data, appears poised to alter the way millions of people are hired and assessed” (Peck, 2013). Efforts have also been made in predictive (statistical) and prescriptive (optimization) analytics to address workforce (hiring and retention) management, skill and talent (readiness) management, and human capital resource allocation by the private sector (Hu et al., 2007; Cao et al., 2011; Hoffmann et al., 2012; Dietrich et al., 2014, Chapter 2). Even earlier related work can be found in, for example, White (1970), Bartholomew (1973), Vajda (1978), Gael (1988), and the references therein.

Secretary of Defense Ashton Carter has started several initiatives aimed at improving the Force of the Future, in part through the improved use of data analysis and updated technology (Garamone, 2015; Tilghman, 2015). The proposed opportunities for paid tuition, installation of new offices and occupations, and creation of new technologies all have the potential to improve recruitment, placement, and retention.

P&R has traditionally collected opinion data from surveys and focus groups, and those data underpin the analyses that have been performed. A large amount of administrative data are also available and hold great potential for further exploration. There has been a proliferation of available data and databases in the DoD enterprise functions, along with great advances in methods for gathering, storing, and accessing big data, and there have been advances in analytic techniques useful in working with large data sets. With these raw materials, data science can be applied to improve the efficiency and effectiveness of the DoD enterprise functions and to improve DoD’s planning in general.

There are great opportunities for new types of data collection and analysis. Some of the challenges to be overcome include the following:

• Modern tools and concepts of data science are evolving rapidly. Creating more effective analyses based on the vastly enlarged data

___________________

² This amount does not include benefits provided by the Department of Veterans Affairs.

space available today and by newer methods of modeling, analysis, and optimization will require the use of the latest capabilities.

• Data often reside in different forms, and in various places, and are too often weak and perhaps not readily shared.
• Integration of new data and analytical capabilities with traditional data and results from earlier analyses can be problematic.
• New ideas may be needed to make optimal use of distributed data, distributed computing, and distributed analysis, because it may no longer be practical or desirable for all of these assets to be colocated.
• New workforce skills are needed within DoD to exploit these opportunities.

DoD is not the only entity facing this challenge. However, its mission is essential to the welfare of the United States, and there are considerable opportunities to advance its analytic capabilities and unique opportunities for data collection and long-term tracking of large numbers of personnel.

STUDY ORIGIN AND APPROACH

This National Academies of Sciences, Engineering, and Medicine report, sponsored by DoD and the National Security Agency, addresses the statement of task for the Committee on Strengthening Data Science Methods for Department of Defense Personnel and Readiness Missions, presented in Box 1.1.

The study committee held 6 meetings of the full committee to collect information and deliberate. The open session presentations at these meetings are listed in Appendix C. The committee also conducted 14 site visits with the following organizations and groups: Office of the Under Secretary of Defense (Personnel & Readiness); Office of the Chief of Naval Operations, Manpower and Personnel; Office of the Assistant Secretary of the Army, Manpower and Reserve Affairs; Air Force Office of Manpower, Personnel and Services; RAND; CNA; IDA; Naval Postgraduate School; Defense

Manpower Data Center; Defense Readiness Reporting System; Google’s People Analytics group; Intel’s Talent Intelligence and Analytics group; Cornerstone OnDemand; and Workday. These visits allowed the committee to discuss data science challenges and lessons learned in more detail.

In the course of its meetings and site visits, the committee raised questions such as the following:

• What is the state of the art of data analytics for enterprise functions outside DoD? How might the newest methods be applied to recruiting, readiness, and retention? Which methods are appropriate for the special circumstances within DoD (such as not being able to recruit into service at advanced rank from outside)? What DoD responsibilities or problems are not covered by commercial technologies? How might the commercial technologies need to be adapted to the DoD environment?
• In which areas of importance to P&R (and the enterprise side of DoD more generally) are there likely to be benefits from new efforts in data analytics? What is the nature and size of those benefits? What might DoD be able to do that cannot be accomplished within the current analytical environment?
• How do the current DoD information technology and analytics infrastructure (both hardware and software) and practices compare with the state of the art? What are the benefits and costs of transitioning from the current infrastructure and data architectures? What types of human capital would be required to make the transition and then to operate in the new environment(s)?
• Which sources of external information (e.g., social media or demographic data) could complement the internal sources of information that P&R already collects and maintains? Which of these have potential for shortening problem detection and correction? Which of these could improve current estimation and projection products? What types of research and analysis are required to develop analytical capabilities that better leverage existing sources of information and enable appropriate use of new sources? How would current analytical approaches and skill sets need to change?
• How can P&R best leverage data analytics capabilities that already exist in other parts of DoD (e.g., in components dealing with R&D and with intelligence operations)?
• How can tracking, evaluation, and performance estimation capabilities be improved within an organization?
• What are the difficulties in modeling and analyzing unprecedented situations, such as shifts in deployment policy, including the location and duration of assignments?

• What would be a suitable overall strategy for transitioning P&R to support and exploit new data analytic capabilities? In general terms, what estimated level of effort and resources would be required?

While part of P&R’s mission relates to health care–related policies, the committee did not explicitly examine health care data and their unique considerations. Such a discussion would have strong links to (1) the challenges of developing and exploiting electronic health care records, which is a topic that extends well beyond DoD and cannot be usefully examined through any one study, and (2) the challenges of strengthening coordination between DoD and the Veterans Administration, which likewise cannot be usefully examined through any one study. Instead, the committee focused on how to use administrative, transactional, and unconventional data more effectively.

ORGANIZATION OF THE REPORT

While the primary audience for this report is individuals affiliated with P&R, the committee made an effort to have it be useful for any interested party looking for information on developing and employing data science methods. For those unfamiliar with P&R, Chapter 2 provides a brief overview, including its organizational structure, objectives, functions, and key responsibilities. Building on the information presented in Chapter 2, Chapter 3 discusses data and analysis capabilities available to P&R, both internal to DoD and external, utilizing resources such as the FFRDCs. That chapter also discusses how these data and analyses inform P&R decision making. For a more technical discussion of the mathematics and analyses utilized in current data science methods, Chapter 4 outlines the importance and challenge of data preparation and important descriptive, predictive, and prescriptive data science methods that may benefit from further exploration. A reader who is interested in the legal and privacy issues associated with implementation of data science methods can be directed to Chapter 5 for a discussion of privacy and confidentiality concerns relating to conducting analyses with personnel data. Chapter 6 discusses the commercial state of the art in human resources analytics, both in terms of available commercial products and practices used across industry. Chapter 7 provides the committee’s findings and recommendations, including opportunities and possible approaches for P&R to improve data quality and sharing through improved planning and coordination between data holders, streamlined Institutional Review Boards for outside researchers, and improved access with relevant privacy considerations; to enhance the use of data science methods by developing a longer-term approach to integrating advanced

predictive and prescriptive analytics and utilizing controlled experiments to test policy changes; and to enhance data science education within the P&R analytic workforce.

REFERENCES