9

Conclusions, Recommendations, and Research Agenda

The committee was tasked with reviewing the evidence on the interconnected factors that foster and/or hinder the successful implementation of promising Pre-K–12 science, technology, engineering, and mathematics (STEM) education innovations at local, regional, and national levels. As the committee reviewed the evidence around promising innovations, they explored the barriers to widespread and sustained implementation of such innovations and provided recommendations to address these barriers. Through the committee’s analysis of the research and evidence of promising Pre-K–12 STEM education innovations, the committee examined the landscape of the public Pre-K–12 education system including the various actors at different levels of the system (federal, state, local, and regional), including their roles and responsibilities. To understand the current state of STEM education within public schools, the committee examined the history of federal and national STEM education improvements.

In an effort to understand the barriers to “widespread and sustained implementation,” the committee interrogated the evidence on scale and the factors that foster and/or hinder the successful implementation of promising, evidence-based, Pre-K–12 STEM education innovations (to include programs, practices, models, and technologies). Through this interrogation of evidence, the committee recognized an inherent tension between the configurations of actors, decision makers, and financial resources involved in the development of an innovation and those involved in the implementation.

As the committee carried out their work, they recognized that although individual innovations are important and necessary contributions to the

broad Pre-K–12 STEM education system, they may not be sufficient in leading to the kind of system change that the committee believes is needed for all students to have differentiated and equitable access and opportunities to engage in STEM learning. And, systemic change is needed. If a potential aspirational end goal is to have a durable educational system that can continuously improve itself through cycles of reflection, innovation, implementation, and improvement, the committee makes the following consensus conclusions and recommendations based upon the available evidence to achieve this aspirational goal. The chapter concludes with a discussion of remaining research gaps.

CONCLUSIONS

What follows are the committee’s conclusions based on the review of the available evidence on the interrelated factors that enable or constrain the scalability of promising Pre-K–12 STEM education innovations, organized by themes. The committee first characterizes the complex Pre-K–12 education landscape—noting the roles and responsibilities of those actors in the implementation of STEM education innovations—as a starting point before describing the history of STEM education improvements. Then the committee goes on to articulate the conceptualization of scale as multidimensional, which includes spread, depth, sustainability, and ownership, and the implications for collecting evidence of the supporting conditions for promising Pre-K–12 STEM education innovations to scale. Lastly, the committee then describes conclusions related to the enabling and constraining factors for scaling and sustaining Pre-K–12 STEM education innovations.

Pre-K–12 Education Landscape and History of STEM Education Improvements

Numerous educational reform efforts at the federal and state levels have attempted to change the nature of STEM instruction and disrupt persistent inequitable patterns regarding students’ short- and long-term STEM outcomes. As described in Chapter 2, structural features of the U.S. education system present barriers as well as opportunities for STEM education innovations to take root and scale. Each of the multiple, nested levels of the education system contains regulations, ideas, and resources related to STEM education innovations. And various actors, positioned across multiple levels of the education system and facing different institutional and organizational conditions, deploy their agency while enacting STEM innovations. These actors not only engage in different responsibilities but also hold different levels of power and authority for motivating the implementation of various innovations.

Conclusion 1: Actors across the formal Pre-K–12 educational system not only have different spheres of influence but also have power and decision making at different levels (e.g., local, state, federal). This can result in misalignments across levels due to divergent constraints, potentially impacting the implementation, scaling, and sustaining of promising Pre-K–12 STEM education innovations.

The formal Pre-K context is different from the formal K–12 education system in important ways, and there is not clear alignment or coherence of policies, standards, and teaching practices from Pre-K into the K–12 system. Chapter 2 contains a description of the formal Pre-K system and the linkages to STEM education. Each state designs its own Pre-K system through authorizing legislation and funding, and determines eligibility, quality standards, and monitoring. Because of this, the governance is highly variable and fragmented. There are a number of ways in which states have attempted to enhance integration of STEM in their Pre-K policies; however, attention to STEM in Pre-K is sporadic. A few states and districts have developed a scalable and sustainable approach to integrating STEM fields through curriculum, professional preparation, and professional learning and development¹ of Pre-K teachers and administrators.

Conclusion 2: There is a lack of coherence in STEM education across early childhood, in part due to the disconnect that exists between preschool programs and the K–12 education system. Prior to elementary school, young children are served through a variety of programs (e.g., federal programs like Head Start, state-funded Pre-K, and various community agencies), which vary in their alignment to K–12.

Conclusion 3: Promising research and development efforts in early childhood have emerged in the past decades. However, more research is needed to understand how early STEM education innovations can be scaled and sustained to promote STEM learning in preschool and strengthen children’s later learning in K–12.

Chapters 2 and 3 describe the role of federal agencies in STEM education improvements. Historically, the U.S. federal government has played an indirect and influential role in education while states and local education agencies have direct control. Even today, the federal government exerts

___________________

¹ As noted earlier in the report, the terms “professional learning” and “professional development” are both in current usage, though various users may prefer one to the other and may intend to indicate different approaches or types of activities. Because various local, state, and national education agencies; funders; and other organizations may use one or the other, the committee’s recommendations include both terms.

limited control over schooling, with few direct throughlines to communities and classrooms. Yet researchers and practitioners point to multiple ways that federal legislation, guidance, and leadership affect local systems and activities. For example, early initiatives made significant advancements in leveraging science and mathematics curricular resources to improve STEM education at scale, but their potential to drive transformative and sustained improvements is limited without continuous, long-term funding, and easy-to-manage distribution channels.

Conclusion 4: The federal government and states have implemented different efforts with the goal of improving STEM education (e.g., systems change, standards, accountability). Although these efforts, particularly those with robust and coordinated plans, have shown promise and lead to improvements, few have been sustained once funding was reduced.

Conclusion 5: The decentralized system not only lacks the means to propagate large-scale improvement across the country but also presents a challenge for the ways in which the federal government can incentivize large-scale, sustained, and well-resourced improvement efforts.

The passage of the No Child Left Behind (NCLB) Act in 2001 marked a number of critical changes in the federal education policy landscape and significantly increased the role of states in holding schools responsible for the academic progress of all students. Chapter 3 describes how different federal pushes have ushered in a new era of high-stakes assessment in mathematics (and English language arts). This intense focus on math assessments led to a narrowing of the curriculum, with teachers often feeling pressured to “teach to the test.” Science and engineering learning became even more deprioritized, with instructional time and resources frequently diverted from these subjects to improving math and reading scores. This increased focus on student performance and achievement has fundamentally altered STEM education.

Conclusion 6: Across the Pre-K–12 spectrum, the distribution of emphasis placed on different STEM disciplines has restricted opportunities for some content areas and in some grades. This presents different innovation and implementation challenges for STEM education innovations within formal Pre-K–12 educational settings.

Notably, the education landscape includes actors beyond those most closely connected to formal spaces. Out-of-school time experiences take place in a vast range of environments and situations, including youth development programs, museums, libraries, zoos, botanical gardens, science

centers, and community centers. Chapter 3 describes the role that these spaces can play in supporting STEM education improvement efforts. For example, programs within out-of-school time settings often have flexibility to design learning experiences that reflect the interests and identities of the learners and communities they serve. Moreover, partnerships between K–12 and out-of-school time settings can enhance connections to support the teaching and learning of STEM through the development of resources. Additionally, Chapter 3 acknowledges many of the other actors in the system (e.g., families, communities, business/industry, professional societies to name a few) who can have influence on what happens in schools, including the development and implementation of Pre-K–12 STEM education innovations. This larger circle of actors can play an important role in advocating for change; however, they may need to be proactive and persistent as they seek to be recognized and understood. Decision makers and implementers within the formal education system who seek out partnerships with an expanded group of actors can gain a variety of supports for effectively adapting and sustaining innovations in their local contexts.

Conclusion 7: Out-of-school spaces (e.g., museums, science centers, libraries) not only provide opportunities for learners to engage in Pre-K–12 STEM education innovations but also are important spaces for their development. These settings can support experimentation and, the development of resources (e.g., professional learning, curriculum) that can translate into formal Pre-K–12 educational settings, as well as serve as backbone organizations in scaling efforts.

Conclusion 8: Actors and participants closely connected to formal spaces, including those traditionally underrepresented in the development and implementation of Pre-K–12 STEM education innovations, can play important roles in shaping what is (or is not) happening in schools.

Scaling and Sustaining Promising Pre-K–12 STEM Education Innovations

In the design of innovations, conceptualization of scale is diverse, and there is variability in the criteria to assess whether, and in what ways, an innovation has “scaled.” Chapter 4 recognizes that most discussions of scale focus solely on increasing numbers, or spread; however, scale is multidimensional and includes other dimensions such as considerations of the depth of implementation, sustainability, and shift in ownership. Assessments of scale often focus on surface-level indications of adoption (e.g., number of beneficiaries, presence of materials, time spent using the materials), with little attention to depth, sustainability, and ownership. Spread alone does

not support educators or researchers in knowing whether an innovation is resulting in the desired improvement and for whom, whether the innovation is sustained as enactors change, and why or why not.

Conclusion 9: The notion of scale is multidimensional (i.e., spread, depth, sustainability, and ownership); however, assessments of scale to date most often focus on the spread of innovations. Additional research is needed to better understand these other dimensions of scale.

Conceptualizing scale as multidimensional recognizes that some innovations are specifically tailored for a particular population, place, and/or problem of practice. Moreover, Chapter 4 illustrates that not all efforts to scale an innovation will explicitly attend to each of these dimensions, nor will they give equal weight or priority to the dimensions on which they focus.

Conclusion 10: Pre-K–12 STEM education innovations can be designed for different purposes. They can be designed to have deep impact with a more local focus, designed explicitly for large-scale impact with less attention to depth, or designed to be some combination of both. Targeted innovations can be just as impactful as those designed for broad reach.

The four dimensions of scale discussed in Chapter 4 are intertwined in complex ways. And the more ambitious an innovation is, the more challenging it may be to sustain if it necessitates a substantial change to “business as usual.” Research has shown that if the innovation involves superficial or minor changes to current practice, its adoption and assimilation by practitioners is more easily achieved. However, superficial change is inherently fragile. Innovations that entail substantial change to current practice represent a formidable challenge that demands time and significant investment in professional resources to accomplish. It requires a change of culture. But once achieved, it provides a foundation for sustainability.

Conclusion 11: In the identification and selection of Pre-K–12 STEM education innovations, a tension exists between innovations that are easy to spread despite limited evidence of impact and innovations that are harder to implement yet show robust evidence of impact.

Federal funding agencies have historically prioritized sequential studies of scaling innovations (i.e., pilot studies, efficacy studies, effectiveness studies, scale-up studies), whereby the innovation is implemented in tightly prescribed ways, in increasingly heterogeneous sites and/or populations, and in service of specified outcomes. However, federal funding is limited to investigate the sustainability of innovations (see Chapters 4 and 5).

Conclusion 12: Current funding structures encourage scalability after innovations are designed, tested, and proven efficacious; however, funding to ensure and study how innovations can sustain is limited, and therefore less is known about the mitigating factors for scaling and sustainability of innovations.

Chapter 6 begins with a discussion of how computer technologies have had profound impacts on education, especially as access to computers and the internet have increased in K–12 education settings. Tools and technologies rarely fully communicate meaning or information about how they can be best used, especially to support learning and cultural practices. Many other factors, such as pedagogy underlying curricula, participatory structures, and teachers play an essential role in helping students understand how and why these tools have been designed in certain ways and how they can be used to promote reflection, solve problems, and accomplish goals. Therefore, it is important to interrogate promising, evidence-based innovations for the features that support (and constrain) their scalability and sustainability.

Conclusion 13: Multiple forms of technology (e.g., simulations, models, visualizations, immersive environments, artificial intelligence technologies, dashboards for classroom orchestration) have shown evidence for supporting Pre-K–12 STEM learning and teaching. Although technology is being used to promote learning in Pre-K–12 STEM education environments, the impact of particular forms of technology on the scalability and sustainability of innovations needs to be better examined and understood.

Drawing from an examination of the literature as well as a synthesis of program information, Chapter 7 highlights the factors that support the conditions for scaling. For programs to successfully scale, it is important that they include a proven core program (that is clearly stated) with ample room for adaptation to different contexts and learners. This flexibility can support educator autonomy, which can aid in motivation and buy-in. The program also needs to align with the priorities set by policy, organizations, and individual practitioners. For example, showing how the program is aligned with standards can help with uptake by showing how implementing the program can be incorporated into existing practices and help achieve goals for learners. Providing support for practitioners through professional learning opportunities is important for capacity building and program adoption. However, there also needs to be support from the organization via an investment in resources and continued support. In some cases, this might mean focusing on a bigger shift in the organization or policies to establish a system where the essential principles of an innovation can be adopted and sustained. In this way, an organization’s capacity for adopting

any new program could be strengthened. Lastly, partnerships can be valuable for scaling in a variety of ways. Having external organizations partner with districts, schools, and teachers can provide in-depth insights into specific contexts to guide development and improvements. Moreover, external partners can also be valuable sources of resources, funding, and support.

Conclusion 14: The extant literature suggestions that some characteristics that allow a promising innovation to spread and sustain over time include:

• A strong, core program with ample room for adaptations for different contexts and learners
• Alignment of goals, policies, and practices
• Development of individual capacity and organizational support
• Development and support of partnerships and networks

Enabling and Constraining Factors

In understanding how Pre-K–12 STEM education innovations arise, take root, and spread it is important to recognize that there is an important distinction to be made between the configurations of actors, decision makers, and financial resources that are typically involved in the development of evidence-based innovations as compared to the configurations that come into play as innovations are implemented, sustained, and spread across settings and populations. Many funded projects are not sustained in a significant way beyond their original instantiation because of this (see Chapter 5). Although these innovations may generate important research evidence of impacts on STEM learning or other desired outcomes in their original context, they lack a functional dissemination model that reaches directly into classrooms or other educational settings. As noted above, partnerships play a key role in the scaling and sustainability of promising Pre-K–12 STEM education innovations.

Conclusion 15: In the design of innovations, there can be a disconnect between the landscapes of innovation and implementation.

Conclusion 16: Mature innovations often involve collaborative, iterative cycles of design across multiple sites and an extended period of time. Different aspects and phases of this work generally require the participation of actors with different experience and expertise.

Innovations that have shown promise for scaling and sustaining are often developed with input from practitioners about the needs of educators and students and with explicit attention to the varying contexts in which

the innovation could be implemented. Chapter 5 points to the research that shows that promising innovations with good evidence of efficacy often lack characteristics and features that would make them easier to be adopted and implemented more widely. It goes on to note that the original designers may lack incentives, knowledge, or expertise to build them out in more full-featured formats.

Conclusion 17: Promising innovations often have limited impact because their research and development are isolated from realistic contexts. The designs and methodologies prioritize artificially narrow or rigid interventions or conditions to obtain strong evidence, but then potential adopters or adapters lack information about how the innovation could be adapted for a particular context or goal(s).

As introduced in Chapter 5 and then taken up in Chapters 7 and 8, the complexity of educational environments results in embedded challenges that can negatively affect successful scaling and sustainability efforts. If a program is designed to align with specific policies or to meet specific goals, there can be a large threat to scaling and sustainability when policies or priorities change and are no longer in line with the direction of the program. Scaling and sustainability are aided by the buy-in and capacity building of individuals and organizations. If individuals championing the program at an organization or those trained to implement it leave, it can hinder continuity and efforts scale the program. During scaling efforts, it can be difficult to also update materials, as doing so may require additional development and testing. This can lead to materials that are out-of-date with the latest advances in STEM fields. Once a program has scaled broadly it is challenging to monitor and evaluate. When a program is being implemented and adapted in many different contexts, it is difficult to draw conclusions across sites and learners. Additionally, the cost of evaluating something at this scale in a deep way can be prohibitive.

Conclusion 18: Challenges to scaling promising Pre-K–12 STEM education innovations include changes to policies and priorities, staff turnover, keeping materials updated, cost, and monitoring implementation and outcomes at scale.

Policy structures and design features, resources, infrastructure, and leadership influence the STEM innovation implementation activities of various actors. Certain levers hold the potential to increase the agency of district and school leaders to more deeply implement STEM innovations, as described in Chapter 8. Policies and resources can reshape conditions to not only support individual actors, but also promote deeper, systemic change. Leaders can create conditions enabling teachers and system and

school leaders to interpret, learn from, and then respond to messages about STEM innovations in substantive ways. The coherence of STEM innovations with other educational improvement efforts influences the degree of organizational and individual change. By aligning with—or building upon—other improvement strategies and reducing the “too-muchness” of educational reform, coherence can catalyze the implementation of STEM innovations. An array of catalysts—nature of the innovation, resources, professional learning, leadership, system capacity, and coherence—enable the implementation of STEM innovations to scale in more equitable and sustainable ways across various contexts.

Conclusion 19: Critical factors that need attention during implementation include: the nature of the innovation, resources (e.g., funding, materials, and time), professional learning for educators, leadership, system capacity, and coherence. For innovations to scale and be sustained, it is critical to attend to these key factors during implementation so that these factors are supportive of, rather than hindrances to, educators’ enactment of innovations.

Chapter 8 also acknowledged the need for system change through the building of individual and organizational capacity. Providing high-quality professional learning supports for those charged with enacting the innovation is necessary but not sufficient for supporting sustainable implementation at some scale, especially if the innovation entails substantial change to the enactors’ current beliefs, knowledge, and/or practice. The sustainability of an innovation is easily threatened by turnover in the enactors. Thus, whether the innovation entails a minor or substantial change to “business as usual,” it is critical to build in structured opportunities to “onboard” new enactors. One strategy is to identify and grow a cadre of experienced educators who are provided with resources (e.g., time, compensation) to create ongoing opportunities for new members to learn about the innovation.

Conclusion 20: The preparation and development of preservice and in-service teachers and school leaders does not routinely include building internal capacities to continuously identify, evaluate, and implement new high-quality instructional resources and practices to adapt them for the needs of different students.

RECOMMENDATIONS

The committee was tasked with identifying barriers to widespread and sustained implementation of promising, evidence-based Pre-K–12 STEM education innovations and to make recommendations to the National

Science Foundation, the U.S. Department of Education, the National Science and Technology Council’s Committee on Science, Technology, Engineering, and Mathematics Education, state and local educational agencies, and other relevant stakeholders on measures to address such barriers. A prevailing issue emphasized by the committee is the need for coherent, systemic reform efforts that can build toward a nimble system that can innovate and respond to challenges as they emerge. This includes developing the capacity of all participants in the design, implementation, scaling, and sustaining of Pre-K–12 STEM education innovations whether at the level of individuals or a system that connects across multiple levels so that there is increased sharing of knowledge across levels of the system. For example, if an individual teacher is entering into a research-practice partnership, they may need support from the school, access to resources and time, and professional learning opportunities that allow them to fully engage in the research process. A school is more likely to be able to provide these types of experiences to the teacher if the school itself is embedded within a system that has the necessary resources for the work to be carried out. And federal agencies can be a source of funding to establish and build structures so that these infrastructure supports can be maintained.

Based upon the committee’s conclusions, the following recommendations are intended to be steps toward changing the current “business as usual” to a system that can effectively and efficiently plan, implement, evaluate, and adjust Pre-K–12 STEM education innovations.

Building Capacity and Monitoring Progress

The first four recommendations point to roles that federal agencies can play in building capacity within states to support implementation efforts. Chapter 8 highlights the need for individual and organizational capacity as supporting conditions for the scalability of promising Pre-K–12 STEM education innovations. To build capacity, there needs to be significant investment in the professional learning and development² of teachers, as they are the frontline in the implementation of STEM education innovations, as well as significant investment in building partnerships within the system to support organizational capacity. Building connected systems of teachers and administrators, curriculum specialists and developers, technology specialists, community learners and partners, and researchers and creating new roles and

___________________

² As noted earlier in the report, the terms “professional learning” and “professional development” are both in current usage, though various users may prefer one to the other and may intend to indicate different approaches or types of activities. Because various local, state, and national education agencies; funders; and other organizations may use one or the other, the committee’s recommendations include both terms.

spaces for them to work and learn together is foundational for supporting organizational capacity. Moreover, as acknowledged in Chapter 3, previous systemic efforts showed greater promise when given the opportunity to build strong plans, many of which require time to take shape and then be implemented.

Recommendation 1: The U.S. Department of Education should allocate funding for teacher professional learning and development in all STEM disciplines, to include science, technology, engineering, mathematics, computer science, and other emerging STEM-focused subjects (e.g., data science). As part of the funding allocation, states will need to provide a plan for the use of funding for professional learning and development that is based on established best practice (e.g., curriculum-embedded, sustained over time), metrics to achieve the goals (e.g., measures of quality teacher professional learning and development), and data that show evidence for achieving those goals. The funding should be renewable up to ten years, and if states have not shown improvement by year four, they must revise their plan.

Recommendation 2: The National Science Foundation should develop a new generation of Pre-K–12 STEM education systemic initiatives, with the goal of building infrastructure, capacity, and expertise to harvest promising evidence-based innovations, prepare them for wider implementation in new settings, and fund backbone organizations to organize the resources and support systems needed to carry out the implementations in schools. Funding should have a long enough time horizon (e.g., renewable up to ten years) for new structures and relationships to be iteratively refined and to take root in ways that could be sustained.

Recommendation 3: In alignment with the authorizing language of the CHIPS and Science Act of 2022 (P.L. 117–167, Sec. 10395), the National Science Foundation’s Directorate for Technology, Innovation, and Partnerships should partner with the Directorate for STEM Education (EDU) to fully leverage the expertise of the EDU Federal Advisory committee and ensure the inclusion of program officers with expertise in Pre-K–12 STEM education in the evaluation of proposals related to supporting multidisciplinary research centers for scaling promising Pre-K–12 STEM education innovations.

Recommendation 4: In an effort to facilitate coordination across federal agencies that implement, scale, and sustain Pre-K–12 STEM education innovations, the National Science and Technology Council’s Committee

on STEM Education (CoSTEM) should identify key metrics for scaling and sustaining innovations and identify an appropriate schedule for reporting them to the public, exploring where the data should be reported (e.g., science.gov, National Center for Education Statistics, National Center for Science and Engineering Statistics) to keep the data ever green. CoSTEM should use these data to inform future iterations of the strategic plan and coordinate consistent investments across the federal agencies.

Building a Research Infrastructure for Scalable and Sustainable Innovations

The committee also points to the need for additional research for better understanding the interrelated factors associated with sustaining Pre-K–12 STEM education innovations. The next three recommendations point to how to address those gaps in the research base, particularly for understanding issues of sustainability of innovations.

Recommendation 5: The U.S. Department of Education and the National Science Foundation should create a new funding category that allows for the study of sustainability of STEM education innovations, which could allow for a deeper understanding of the dimensions of scaling. This should include developing system-level measures of STEM education innovations that attend to the inter-related actors, structures, and interactions that shape how innovations take hold in multiple, diverse contexts for different learners with an eye toward equity.

Recommendation 6: The U.S. Department of Education and the National Science Foundation should encourage practice-initiated partnerships and planning grants that will connect teams of researchers, designers, and practitioners with expertise and experience in different aspects of innovation development and implementation, either within a project or across successive related projects. These projects should include considerations of implementation and adaptation for different learners across multiple, diverse contexts early in the development process.

Recommendation 7: The U.S. Department of Education and the National Science Foundation should continue to encourage research and development in early STEM education: specifically, efforts that tackle integrating professional learning and development opportunities with curricula that address the many domains of learning that educators are expected to promote in early childhood, planning grants to support practice-initiated partnerships, and focus on coherence and alignment across preschool and elementary.

State and Local Actors: Systemic Change and Continuous Improvement

The next five recommendations are directed toward state and local actors in the Pre-K–12 education system. The recommendations focus on including strategies for making processes for continuous improvement the norm and creating partnerships to connect and align all of the relevant roles and forms of expertise. In the building of networks and partnerships, it is necessary to take the time to figure out common goals/purposes and how to align them, and to frame the efforts to achieve the desired outcomes. Relatedly, within the formal education system, understanding who has decision-making power for a given initiative is important and may vary across locations (see Chapter 2).

Recommendation 8: School and district leaders should adopt and/or evaluate a networked continuous improvement framework, emphasizing iterative assessment and refinement of strategies to meet the evolving educational landscape. This involves a cycle of planning, implementing, evaluating, and adjusting, with engagement of pertinent individuals to ensure ongoing relevancy. Data-driven analysis and feedback mechanisms should allow for real-time monitoring and responsive adaptation. Embracing this approach fosters a culture of innovation, learning, dexterity, responsiveness, and resilience within the schools and across the district.

Recommendation 9: To understand the implementation and scaling of Pre-K–12 STEM education innovations, state and district partners should develop data systems that capture information about opportunities to learn including time for instruction, allocation of resources and funding, access to and enrollment in Pre-K–12 STEM education innovations, and qualifications of teachers and characteristics of teachers. These data should be disaggregated to examine trends by subgroups of students and by school characteristics.

Recommendation 10: To ensure continuous improvement in Pre-K–12 STEM education innovations, school and district leaders should engage with professional learning and development providers to offer curriculum-embedded, ongoing opportunities within and across years that includes specific emphasis on new teachers to ensure that their learning is commensurate with those who participated in opportunities in years prior.

Recommendation 11: Local school and district leaders should initiate and sustain partnership agreements across all levels of the STEM education learning ecosystem (e.g., teachers, teacher educators, education

researchers, designers of Pre-K–12 STEM education innovations, families, etc.) in order to combine STEM education expertise and local knowledge to attend to specific problems of practice and advance sustained development and implementation of promising, evidence-based Pre-K–12 STEM education innovations.

Regional Actors and Impact

The final set of recommendations acknowledge some of the broader range of actors that can have regional impact as they support the implementation, scaling, and sustaining of Pre-K–12 STEM education innovations.

Recommendation 12: Leaders of local and regional Pre-K–12 system should work to strengthen learning opportunities in STEM education among key actors in the STEM education learning ecosystem (e.g., teachers, school/district leaders, school board leaders, teacher educators, professional development providers, universities and colleges, museums, nonprofits, families, etc.) with an emphasis on building relational connections among communities and sharing knowledge.

Recommendation 13: In their support of Pre-K–12 STEM education innovations, the federal government, philanthropic organizations, and business and industry should provide support to projects that include designers of curricula partnering with education leaders, teachers, families/communities, and researchers to co-design resources that are evidence-based, meaningful, accessible, and able to be feasibly implemented to support STEM teaching and learning. Attention should be given to features that can lead to meaningful STEM learning, while also considering components needed to ensure resources can be sustained and adapted for use by others.

RESEARCH AGENDA

In addition to providing what is known about the scaling of Pre-K–12 STEM education innovations, the committee was asked to identify gaps in the research that can aid in a better understanding of the interrelated factors that support and hinder the widespread implementation of Pre-K–12 STEM education innovations. Through their analysis of the literature, the committee identified four crucial areas: (a) attending to all dimensions of scale, (b) focusing on scale early in the development process, (c) documenting evidence of impact across varied learners and contexts, and (d) examining systems-level impacts. Across these crucial areas and potential questions, it

is important to consider the methodologies employed, as both quantitative and qualitative methods are needed (sometimes simultaneously) to unpack the effects of the innovation at multiple levels. (For a deeper discussion of methodologies, see Chapter 6 of the 2022 National Academies of Sciences, Engineering, and Medicine report The Future of Education Research at IES: Advancing an Equity-Oriented Science).

Research Area 1: Attending to All Dimensions of Scale

The committee presents a multidimensional framework for scale that includes spread, depth, sustainability, and ownership. With changes in funding as described in earlier chapters, there has been increasing attention to understanding the scaling of innovations, with particular emphasis on the dimension of spread. Fewer studies have focused on the other dimensions of scale, including the relationship between the different dimensions. For example, how is depth related to scale? How is ownership related to sustainability? What are the connections between scalability and sustainability given that sustainability is included as a dimension of scale?

In addition to understanding each of the dimensions and their relationships, additional research needs to focus on ways of assessing scale and sustainability of Pre-K–12 STEM education innovations that attend to the depth of implementation and sustainability, and in relation to populations, geography (and other aspects of spread), and the impact on communities. In assessing these various dimensions, there needs to be attention to variability and how enactors and enablers respond to unwanted variation. This could allow for a deeper analysis of the conditions that allowed a particular innovation to be sustained. This might include teacher narratives combined with quantitative analyses of innovations they have developed/shared/continued to use that reflect the reality of innovations in the classroom (including limitations of the innovations that were present). Through this research, there would also need to be greater emphasis on the systematic tracking of the various factors and conditions—this could help with understanding how innovations can better learn and grow as the implementation context changes over time.

Research Area 2: Focusing on Scale Early in the Development Process

There has been a history of funding research that emphasizes a particular sequence: pilot studies, followed by efficacy studies, then effectiveness studies, and finally, scale-up studies. Not surprisingly, given this history and trajectory of funding, not all projects have considered the various dimensions of scale or the particular challenges in implementation identified in Chapters 5–7 early in the development phase. This includes having

an eye on support for professional learning of educators or the creation of resources that can be adopted in schools. That is, what are the plans for an innovation to scale (and sustain) beyond the local context? Having this information collected early in the process would aid in understanding challenges in implementation, and these constraints could be built into the scale-up design process. Potential questions could include:

• What are the decision-making processes, contexts, resources, and conditions that support or hinder district and school leaders to make decisions about scaling and sustaining Pre-K–12 STEM education innovations?
• When should innovations scale and when should they not?
• What should one pay attention to when thinking about how programs start partnerships and how does this play out across the different types of funded studies?
• How are innovations adapted as they are scaled and sustained, included enabling and constraining factors?

Research Area 3: Documenting Evidence of Impact Across Varied Learners and Contexts

As elevated throughout portions of this report, STEM outcomes are not the same for all kinds of cultural communities and place-based contexts. A number of the innovations discussed in Chapter 6 had been intentional in attending to understanding the impact of the innovation across varied learners and contexts. However, this research is limited and necessitates considering how the research is conducted. For example, it may be that there is a need for developing and/or leveraging new statistical methods for analysis and using larger sample sizes. Moreover, not all methods will be quantitative (see National Academies of Sciences, Engineering, and Medicine, 2022).

There is a need to investigate how researchers and educators have identified a variety of ways to measure, assess, and evaluate differentiated STEM learning outcomes for various groups of learners (e.g., students from rural communities, bilingual students, neurodivergent students, and tribal nations). This necessitates practical, affordable methodologies and multiple forms of assessment for gathering evidence of the purposes of STEM in differing contexts, what counts as success, and “what works for whom and under what circumstances” in a medium timeframe. Looking across contexts, it is important to identify the major drivers (e.g., contributors at the regional, state, and local levels) and the differences in their motivations and ability to scale and sustain Pre-K–12 STEM education innovations (e.g., differences between rural, suburban, and urban contexts as related to the enabling and constraining factors identified throughout the report).

Research Area 4: Examining Systems-Level Impacts

Expanding research-based knowledge about productive strategies to support the scaling and sustaining of Pre-K–12 STEM education innovations requires investment in research that documents not just the learning that occurs within individual innovations, but also the innovation’s system-level impacts and how the system is learning. For example, in what ways have policy shifts resulted in significant changes to the STEM learning ecology and at what level (district, state, regional, national)? What are some indicators that the system is learning? Beyond understanding how an individual innovation could have an impact on the system, there is also a need to understand robust community-level outcomes. Community-level goals, such as neighborhood resiliency, building community capacity to adapt to changing social and ecological systems, more access to healthy drinking water, increased biodiversity, and a sense of community belonging, often seem to be lacking in STEM learning outcomes. Community-level desires can be foundational measures of system-level educational impacts. These include questions such as, How is the program supporting community-level goals and outcomes? How is this program supporting broader social and ecological health and wellbeing?

Finally, in thinking about scale, the committee noticed a lack in outcomes that support STEM learning programs networking and learning with each other. Some Regional Education Labs are exemplars of how this might be done, and further work is needed into how we know that local system is in fact learning. What kinds of system-level measurements could help advance and enhance STEM learning that supports diverse visions of wellbeing for children and families, more just and vibrant community life, and healthier lands and waters where they live?

FINAL REFLECTION

The committee was asked to examine the vexing problem of why federal and other investments in research and development have led to a wealth of innovative ideas and resources with excellent potential to improve STEM teaching and learning, but these innovations are often unable to be scaled effectively and equitably in new settings and to be sustained over time. To produce this report, the committee examined both successes and shortcomings of attempts to spread and sustain promising innovations, drawing on historical analyses, contemporary evidence, and first-hand accounts of actors and participants throughout the landscape of Pre-K–12 STEM education.

The committee concluded that barriers that stand in the way of widespread dissemination and sustainability of promising innovations are generally not individual failures but rather systemic ones. Most developers would

like to see their innovations take root and spread, and most school leaders would like to access high-quality research-based innovations if they are responsive to school needs and can be tailored to local circumstances, but there is complicated terrain that must be navigated in between. The United States’ historically decentralized system, where research and development take place outside of the Pre-K–12 educational system while implementation happens locally within districts and schools, has resulted in a malfunctioning and fragmented system. However, decentralization and local control need not imply isolation, disconnection, or tunnel vision.

In this report, the committee presents a variety of examples and models demonstrating how systemic efforts to increase collaboration and partnerships and to build permanent mechanisms for actors with different capacities and responsibilities that coordinate coherently can result in robust, durable, and ongoing improvements in STEM education. Building systemic capacity will not happen quickly and will require considerable and sustained investments of funding and leadership at every level of the educational system. But there is an urgent need to engage in this work to fulfill the promise of public education to youth and their families and communities, and to reap the benefits of future workers and citizens who are empowered with knowledge, competencies, and motivation gained from a world-class STEM education.

REFERENCE

This page intentionally left blank.