7

Promising Pre-K–12 STEM Education Innovations

As highlighted throughout the report, Pre-K–12 science, technology, engineering, and mathematics (STEM) education initiatives can have a complex constellation of goals and related outcomes for beneficiaries (e.g., students, teachers). For example, they might be designed to advance the STEM workforce, to develop skills and competency in STEM disciplines, to foster a sense of belonging and identity as a STEM professional, and/or as a way to cultivate just, sustainable, and thriving human communities (National Academies of Sciences, Engineering, and Medicine [NASEM], 2024a). Innovations might be designed to meet multiple goals, some of which are explicitly stated, others that are more implicit. As an innovation is implemented, enactors (educators and administrators) may emphasize specific outcomes or introduce additional goals. Some of the goals may be measured, and others may not be. Regardless, throughout the curricula, programs, and experiences, the intended outcomes are intertwined. Recent research in STEM education has focused on understanding and measuring these broadened goals and outcomes of educational approaches, programs, curricula, and interventions (National Academy of Engineering [NAE] & National Research Council [NRC], 2014; NASEM, 2020, 2021a,b, 2022, 2023, 2024a; NRC, 2011, 2013).

A goal for some Pre-K–12 STEM education innovations is to scale beyond the contexts in which it was designed. Chapters 4 and 5 described some of the approaches to scaling and sustaining that innovations might take and the ways in which innovations can propagate. Throughout these discussions, the committee navigates the tensions between the landscape

of innovation and implementation as well as the tensions between features of the innovation itself that are necessary to support the scaling and sustainability, and broader system-level supports. These issues are necessarily intertwined and highly dependent.

Pre-K–12 STEM education innovations are highly variable in their implementation and reach, which presents challenges for understanding the landscape of access and opportunity. A number of recent reports have attempted to catalogue the patterns of opportunity and access to Pre-K–12 STEM education innovations (see NASEM, 2022, 2023, 2024a,b). Given the variability, the committee was unable to provide a detailed analysis of this opportunity landscape; however, as can be observed through the case summaries in the compendium that was completed by the Education Development Center (EDC; Appendix C), there are a number of innovations designed to increase opportunity and develop a sense of belonging and identity with STEM.

The goal for this chapter is to provide what is known about the features at the innovation level that support the scaling of promising, evidence-based Pre-K–12 STEM education innovations. This is accomplished by drawing on the findings recorded in the compendium that was completed by the EDC. First, the committee describes the development of the compendium. The committee then explores the conditions that support scaling promising, evidence-based, Pre-K–12 STEM education innovations. The chapter concludes with a brief introduction of the factors that were identified as hindrances to scaling innovations, issues that are taken up more fully in the subsequent chapter.

IDENTIFYING PROMISING PRE-K–12 STEM INNOVATIONS¹

The committee was tasked with producing a compendium of promising, evidence-based Pre-K–12 STEM education innovations. As noted in Chapter 1, the committee recognized the importance of this task and the enormity of the undertaking. Therefore, the committee commissioned the production of the compendium, which was developed by EDC. What follows is a brief outline of the methodology and a brief overview of promising innovations identified through this process.

___________________

¹ It should be noted that the compendium was produced while the committee was engaged in its analysis of the literature and reaching consensus on its evidence-based conceptualization of scaling. Therefore, the production of the compendium did not include questions that would allow for a more nuanced analysis and understanding of scaling using the dimensions of scale as described by the committee.

Methodology

EDC conducted a seven-step process with the goal of identifying promising Pre-K–12 STEM education innovations that have gone through a scaling-up process. This process included these steps:

1. Conduct a literature search to identify an initial set of factors that facilitate or hinder scaling of innovations that are broadly supported by the existing research literature.
2. Set parameters for inclusion in the compendium, which mainly follow from the charge to the committee.
3. Develop program nomination form to be used in an open call for online submissions. The form was guided by the factors identified in the literature review to ensure that the online submissions would provide information that would be likely to be relevant to the committee’s considerations of factors that facilitate or hinder scalability of innovations.
4. Work with a list of sources and field experts to generate the names of likely promising programs to make targeted invitations.
5. Invite identified programs to submit and also conduct an outreach campaign to advertise the call for submissions more broadly.
6. Review and analyze nominations.
7. Summarize programs, including their approach to scaling and sustainability and evidence of success.

The full methodology for the development of the compendium can be found in Appendix B, and the full compendium is provided in Appendix C.

Literature Search

Before identifying innovations, EDC first conducted a literature search to identify factors that appear to support innovations in their scaling efforts and those that may hinder them in service of understanding how innovations scale. EDC systematically searched Google Scholar and EBSCO using various combinations of the following keywords: STEM, science, mathematics, engineering, education, innovation, scaling, and scale-up. Results were filtered to identify literature published in the last ten years. In EDC’s review, it was determined that there is a growing body of research that has attempted to uncover the factors that enable or constrain the scaling of innovations (Dearing et al., 2015; Krainer et al., 2019; Looi & Teh, 2015; Lowrie, Leonard, & Fitzgerald, 2018; Maass et al., 2019; Sabelli & Harris, 2015; Young et al., 2016). From the literature, the following promising factors were identified:

• A “tight but loose” framework
• Alignment of goals, policies, and practices

• Capacity building and organizational support
• Study the innovation in a variety of settings
• Ease of adoption
• Partnerships and networks

These promising factors were intentionally built into the questions asked by EDC in its solicitation and in the rubric so that EDC could provide the committee with an understanding of how the innovations accounted for these factors during scaling and sustainability efforts.

Set Parameters for Inclusion

For an innovation to be considered, it needed to meet the following characteristics:

• Occur within the Pre-K–12 grade band
• Focus on one or more of the traditional STEM disciplines (i.e., science, technology, engineering, and mathematics), which could include computer and data science
• Be connected to formal classroom teaching and learning (e.g., curricula, professional learning)
• Have evidence of scaling (although this was not a strict requirement)
• Have conducted a formal evaluation process to measure their impact and outcomes (i.e., external evaluation or other peer review process)

Develop Nomination Form

A Google form was created for potential programs to self-nominate. The form requested general information (e.g., program name and point of contact); program information (e.g., Pre-K–12 grade band, subject domain[s], intended audience[s]); evidence of scale; and evidence of impact. Evidence of scale and evidence of impact were collected mostly as open-ended responses to allow programs maximum flexibility to explain how they view these parameters and apply them in their individual contexts.

Identify Programs

EDC developed a working list of sources and field experts to assist in identifying programs to directly contact regarding the self-nomination process. Based on this, EDC identified programs through the following:

• Professional connections internal and external to EDC
• Recommendations from Board on Science Education and the committee

• Recommendations from relevant organizations (e.g., National Science Foundation [NSF], Education Innovation and Research)
• Federal program databases for programs meeting the criteria that received an award in the past seven years (e.g., NSF and Department of Education [ED] award search databases, the Institute of Education Sciences What Works Clearinghouse database)
• Additional projects discovered throughout the process (e.g., through literature review)

In order to identify specific programs to reach out to, EDC first solicited suggestions from their personal contacts as well as through conversations with program officers from the ED, NSF, National Aeronautics and Space Administration, and National Oceanic and Atmospheric Administration. This resulted in 41 programs identified.

EDC then utilized publicly available databases to review federally funded projects from the past 15 years. This included programs funded by Education Innovation and Research scale-up grants from ED, as well as Innovative Technology Experiences for Students and Teachers (ITEST) and Discovery Research Pre-K–12 (DRK–12) grants from NSF. EDC included a search for the terms “scale” or “scale-up” in order to limit the results to those referencing scale. The list of programs was then reviewed to include only those programs that had scaled, were focused on a field of STEM, and were connected to formal classrooms. EDC also reviewed the What Works Clearinghouse to identify programs that met criteria. This resulted in the identification of an additional 81 programs.

Finally, EDC’s review of the literature identified an additional 14 programs that had not already been identified through other means. In total, EDC sent invitations to 136 programs inviting them to self-nominate (see Table B-1 in Appendix B).

In addition to reaching out to specific programs, EDC also sent information about this opportunity to organizations asking them to share the opportunity with programs that may be eligible for inclusion in the compendium. In total, EDC contacted individuals representing 64 organizations (see Table B-2 in Appendix B).

Invite Programs to Self-Nominate

Program contacts received an outreach email encouraging them to nominate their programs and share the opportunity with their networks. EDC also utilized social media for outreach with posts shared by EDC, Community for Advancing Discovery Research in Education, and STEM Learning and Research Center. Moreover, EDC sought word-of-mouth recommendations from knowledgeable colleagues and others in the field whenever possible.

Review Nominations

All programs were asked to provide a brief summary of evidence of impact to include evidence that (a) STEM learning goals were achieved (e.g., content-specific knowledge), (b) STEM skills were developed (e.g., critical thinking, problem solving, communication), (c) affective goals were achieved (e.g., generated interest in STEM), and/or (d) the innovation helped participants make connections to STEM careers. The various innovations were evaluated against a rubric to determine the strength of evidence for scale, impact, and additional factors related to the scalability of an innovation as noted above (see Appendix B for the Rubric). EDC examined innovations looking for: (a) evidence of fidelity during scaling,² (b) evidence of scaling to new audiences or contexts, (c) evidence that participant outcomes were achieved or skills developed, and (d) understanding of effectiveness for different contexts/learners. The innovations were also reviewed to determine to what extent there was evidence for the factors identified by the literature search that have contributed to successful scaling efforts.

Overview of Promising Innovations

A total of 65 nominations were received. Four submissions were excluded because they did not include an external evaluation or peer review, another four were excluded because they were not connected to formal classroom settings, and one was excluded because it did not focus on STEM. The remaining 56 programs were reviewed and an overview of their breakdown is summarized below (see Appendix C for individual descriptions of all of the programs).

Grade Bands and Subject Domains

At least half of the innovations were intended for elementary, middle, and/or high school with two-thirds focused on middle school. Fewer innovations focused on Pre-K (30%). More than half of the innovations (66%) indicated that they focused on two or more grade bands. Most innovations (70%) included a focus on science content, whereas about half (48%) included a focus on engineering or mathematics. More than half of the identified nominations (63%) indicated more than one domain, and 13 innovations (23%) selected all four of the main STEM disciplines.

___________________

² As previously noted, this work was done in parallel as the committee was reaching consensus on the more nuanced characterization of scaling; the committee recognizes that focusing on “fidelity of implementation” represents a limited view on scaling an innovation.

TABLE 7-1 Nominated Innovations by Domain and Grade Band

^* Innovations could select more than one response option.

SOURCE: EDC, 2024.

Looking at the distribution of domains across grade bands, as seen in Table 7-1, most Pre-K programs included a mathematics component (82%). Science was a component of the majority of middle and high school programs (79% and 78%, respectively). Elementary school programs had a more even distribution across the domains, but were most likely to include science, engineering, or mathematics.

Implementation Contexts

Innovations can arise from and be implemented in a variety of contexts (see Chapter 5 for more discussion). Although innovations needed to have a connection to Pre-K–12 settings, innovations were also frequently implemented in informal learning and afterschool settings. Twenty-eight (50%) of the innovations indicated that they take place in or are connected to formal Pre-K–12 settings only. Moreover, about half of the innovations (52%) have been in operation for more than 10 years, 23 percent have been operating for 2–5 years, and 20 percent for 6–10 years.

In terms of the intended audiences for the identified innovations, all but one program indicated that teachers/educators were the intended audience in addition to most (86%) also selecting students. At least half of the innovations also identified low-income communities, individuals from ethnic or racial groups traditionally underrepresented in STEM, and underresourced geographic areas as intended audiences (see Table 7-2). Given these intended audiences, many of the innovations (87%) incorporated supports for building teacher capacity (e.g., professional learning, differentiated teaching/learning, cultural/contextual responsiveness) to implement the innovation successfully with the anticipated participants.

INNOVATION-LEVEL FACTORS THAT SUPPORT SCALING

As described in the earlier methodology section, EDC conducted a review of the literature to identify conditions that hinder or support the

TABLE 7-2 Nominated Innovations by Audience

^* Programs could select more than one choice.

^** Other responses: All; School/District Administrators; designed to work for all students taking a science class; Tribal communities and high schools with a minimum of 60% enrollment of Native American, Alaska Native, Native Hawaiian and/or Pacific Islander students; teacher leaders, professional learning providers, administrators, state-level leaders.

SOURCE: EDC, 2024.

scaling of innovations (Dearing et al., 2015; Krainer et al., 2019; Looi & Teh, 2015; Lowrie, Leonard, & Fitzgerald, 2018; Maass et al., 2019; Sabelli & Harris, 2015; Young et al., 2016). As listed above, key factors identified as supportive conditions for scaling were:

• A “tight but loose” framework
• Alignment of goals, policies, and practices
• Capacity building and organizational support
• Study the innovation in a variety of settings
• Ease of adoption
• Partnerships and networks

These are the factors that are inherent (or need to be considered) in the design and implementation of an innovation and are related to the approaches articulated in Chapter 4. Many of these factors are also related to systems-level factors that can enable or constrain the scaling and sustaining of innovations—these system-level ideas will be taken up in Chapter 8. What follows is a discussion of each factor, using case studies from the compendium to support the ways in which the various factors supported the scaling (and sustainability) of the innovation. In describing each case study, the committee discusses the factors that contributed to the innovation’s success and how challenges were managed.

A “Tight but Loose” Framework

For innovations to successfully scale, it is important that they include a proven core with ample room for adaptation to different contexts and learners (Looi & Teh, 2015). Wylie and colleagues (2008) note that a “tight but loose” framework focuses on the tension between two opposing factors in school reform: the need for an innovation to stay consistent with its governing principles, while also offering ample opportunity to adapt to the countless STEM learning environments in which it makes sense to take up said innovation. The framework “combines an obsessive adherence to central design principles (the tight part) with accommodations to the needs, resources, constraints, and particularities that occur in any school or district (the loose part), but only where these do not conflict with the theory of action of the intervention” (p. 2). This is akin to the mutual adaptation discussion in Chapter 4.

The flexibility offered by this framework can also support educator autonomy, which can aid in motivation and buy-in. It is important that the core of the innovation be clearly stated and shown to be sufficient to meet desired outcomes (Dearing et al., 2015). As an example of an innovation using the “tight but loose” approach, the framework utilized by SCRIPT supports schools and districts in creating a plan for high-quality computer science implementation in a way that considers their individual contexts and goals (see Box 7-1). The SCRIPT framework not only allows but trains educators to appropriately adapt the innovation to support local relevance, leading to adoption across a variety of settings.

Additionally, adaptations by practitioners can have the added benefit of serving as a source of additional resources that could be used to expand upon the materials offered by the innovation (Sabelli & Harris, 2015). For example, the compendium describes FUSE,³ which is a program that is guided by core design principles while offering individual educators the freedom to design the program according to the needs of the school/organization and their community, including logistics like classroom schedules. Research-based recommendations are shared with all partners as they develop their plans to guide adaptation to fit their local needs. FUSE was first piloted in the Chicago area but has been expanded to additional urban, suburban, and rural partners.

Alignment of Goals, Policies, and Practices

For innovations to have the potential to scale, it is important for the innovation to be aligned with the priorities set by policy, organizations, and

___________________

³ FUSE is not an acronym. The name originated from youth who had participated in the program, which had been called ‘YouSTEM’ (see https://www.fusestudio.net/about).

individual practitioners (Looi & Teh, 2015; Lowrie, Leonard, & Fitzgerald, 2018). As noted earlier, innovations can be developed with multiple goals and purposes—from broadening participation and access, to developing skills and content knowledge, to advancing a sense of belonging and identity as a person who participates in STEM (among others). Each of these goals might be attended to in various ways in the construction and implementation of an innovation, and some of these may be easier to advance when they are aligned with other goals as well as existing policies and practices (this will be taken up more in Chapter 8).

Alignment with goals, policies, and practices helps to facilitate alignment across levels of the system and the different partners (Maass et al., 2019); alignment in this way can lead to lasting organizational and policy related changes (Sabelli & Harris, 2015). For example, showing how the program is aligned with standards can help with buy-in by showing how implementing the program can be incorporated with existing practices and help achieve goals for learners (Sabelli & Harris, 2015). The IRCEDE STEM for Our Youngest Learners (see Box 7-2) is an innovation that is aligned with both the Early Learning Outcomes Framework⁴ and the Next Generation Science Standards (NGSS Lead States, 2013). By providing aligned materials and

___________________

⁴ See https://iowaccrr.org/training/IELS/

professional learning, teachers gain important knowledge, skills, and abilities that support student learning across a variety of contexts and student populations. As teachers gain more confidence with STEM, they are able to adapt the program to their local contexts and learners, considering such factors such as grade, time schedules, cultures, and mandated curricula.

To ensure that all learners have access to STEM-related opportunities, there has been increasing focus on equity-related goals. Some studies have revealed that explicit connections to equity-oriented components within innovations yield better outcomes for learners who have been traditionally underrepresented in STEM (Cunningham et al., 2020; NAE & NRC, 2014; NASEM, 2021a,b). Box 7-3 presents the example of the Beauty and Joy of Computing (BJC) course—a high school Advanced Placement Computer

Science Principles course that has scaled to over 200 New York City (NYC) teachers and 600 non-NYC teachers. It is endorsed by the College Board, aligning to goals which can help administrators and teachers feel confident that this curriculum will help them meet their objectives. The objectives of BJC include providing students with a rigorous and engaging introduction to computer science and supporting participation of students from groups historically underrepresented in computer science. Findings from studies evaluating BJC have shown not only improved academic performance but also significant gains in confidence and identity for girls and Black and Latinx students.

Capacity Building and Organizational Support

In addition to ensuring that there is a core, adaptable framework and alignment with goals, policies, and practices, there is also a need for capacity building and organizational support. This can come from a variety of different forms, such as providing support for practitioners through professional learning (Maass et al., 2019; Young et al., 2016). Professional learning can support teachers as they navigate the changing landscape with growing expectations for what they need to know and do in their day-to-day practice of teaching (as described in Chapter 1; NASEM, 2020). Teachers are increasingly charged with ensuring that classrooms serve as equitable learning communities, fostering trusting and caring relationships among students and with teachers, and serving as a bridge between the school and families and communities. These increased expectations for learning, combined with the demand to create a responsive learning environment that supports the needs of all students, call for innovative approaches to instruction that may differ substantially from teachers’ own experiences as students or their preservice education (NASEM, 2020). At the same time, in the case of many STEM education innovations designed for students, teachers are considered key players in implementation. Thus, they are often supported through professional learning in an effort to facilitate effective implementation of these STEM education innovations. For example, STeLLA, a program included in the compendium, is a year-long science professional learning program for K–12 teachers, integrating video-based analysis of practice and a focus on student thinking (see Box 7-4). It provides approximately 90 hours of professional learning, including a summer institute and school year study groups. Research has provided evidence of impacts on teacher content knowledge, teacher pedagogical content knowledge, and teacher classroom practice that resulted in improved student achievement.

Beyond ensuring opportunities for professional learning, there is also a need for support from the organization via an investment in resources and continued support/buy-in (Krainer et al., 2019; Maass et al., 2019). In some cases, this might mean focusing on a bigger shift in the organization or

policies to establish a system where the essential principles of an innovation can be adopted and sustained (Looi & Teh, 2015; Sabelli & Harris, 2015). An example is Making Sense of Science, a train-the-trainer approach that can support local leaders in facilitating high-quality teacher professional learning that, coupled with support from administrators, can develop systems of support for STEM learning (see Box 7-5). In this way, an organization’s capacity for adopting any new program could be strengthened. These ideas are expanded upon in Chapter 8.

Study the Innovation in a Variety of Settings

As introduced in Chapters 4 and 5, research has suggested that practitioners can be key to providing developers with a better sense of what works

for whom and in what contexts (Looi & Teh, 2015; McLaughlin & Mitra, 2001; Sabelli & Harris, 2015; Sarama & Clements, 2013). Additionally, testing the program in a variety of different settings (e.g., via design-based research) with different learners and including the input of educators whenever possible can provide valuable insights to identify challenges and adaptations (Lowrie, Leonard, & Fitzgerald, 2018). NURTURES (see Box 7-6) is an innovation that began with implementation in urban districts before expanding to rural districts and then military-connected districts in seven states. Throughout the program, research has been conducted to understand how to implement the innovation in different settings and with different audiences.

Ease of Adoption

If the goal is to get many practitioners from a variety of contexts to try out a program, considering the feasibility (“trialability”), usability (“simplicity”), and cost can help increase the chances that the program will be implemented. Dearing and colleagues (2015) described case studies from three community college advanced technological education programs as they were supported in scaling innovations. Several lessons were learned through these case studies. First, it was determined that scaling required critical thinking, trial and error,

and reflection during implementation. That is, factors aiding in scale included the ease with which adopters can experiment with an innovation (i.e., trialability), how easy the innovation is to understand and use (i.e., simplicity), compatibility with diverse settings, and how much implementation of the innovation costs in terms of both time and money. As discussed in Chapter 4, an inherent tension exists between the ease of adoption and the depth of change that is needed to scale and sustain an innovation, and it is important to recognize that just because an innovation might by easy to adopt does not mean that it will result in the depth of change that is needed to be sustained.

Moreover, scaling benefits from encouraging adopters to adapt the innovation (within limits) and utilize these adaptations to improve the innovation. For example, Project Learning Tree (PLT) is an innovation that is supported by a network of state-level coordinators to engage with educators and communities at the local level. Because PLT programs are directed and implemented locally, educators received state-specific supplements to PLT’s educational materials that address the local environment, which aids in the ease of adoption of the materials. As another example, AlgebraByExample (Box 7-7) is a set of supplementary assignments that were explicitly designed to narrow the achievement gap for minority students.

However, in striving to develop an innovation that can be scaled across a variety of contexts, it is key to not fall into the “trap of perfection” (Dearing et al., 2015). As described in Chapter 4, there are a number of dimensions for scaling. Maintaining depth (what the program is supposed to do) while also allowing for simplicity and flexibility is desirable for scalability. Therefore, there can be a tradeoff between ease of adoption and impact. That which is easily adopted in terms of curricular innovation may be less likely to push on practice in the way today’s STEM standards require. The committee returns to this issue in Chapter 8.

Partnerships and Networks

Partnerships can be valuable for scaling in a variety of ways (Lowrie, Leonard, & Fitzgerald, 2018; Maass et al., 2019; Young et al., 2016). For example, the common online data analysis platform (CODAP) is free educational software for data analysis with wide uptake and acknowledgement among the statistics education community and beyond. This web-based data science tool is designed both as a platform for curriculum developers and as an application for students in grades 6–12. CODAP has utilized partnerships with other projects to incorporate and spread their work and is currently in broad use by approximately 60,000 users monthly. More than 130 learning science researchers have incorporated CODAP into their research and development efforts, resulting in dozens of curricula and lessons nationwide. Through the varied partnerships across multiple communities, CODAP has had the potential to scale across multiple contexts.

Moreover, partnerships can be leveraged to help in scaling culturally specific programming across the country. For example, Seeding Innovation, described in the compendium, is an innovation in which American Indian Science and Engineering Society (AISES) and the Kapor Center have partnered to provide a sequence of culturally revitalizing computer science curricula to partner schools across the country. The partner organization works collaboratively with school sites to create an engaging computer science curriculum, while also working with teachers and, when possible, community members to integrate cultural traditions, language, stories, art and more.⁵

Although external partners can be valuable sources of resources, funding, and support for those developing innovations, partnering with districts, schools, and teachers can provide in-depth insights into specific contexts to guide development and improvements. Building networks in this way can allow for the innovation to be carried out in a larger number of settings across a larger geographic area. For example, Box 7-8 highlights the

___________________

⁵ For more information, see Kapor Foundation & AISES (2023) report, State of diversity: The native tech ecosystem, https://www.kaporcenter.org/wp-content/uploads/2023/10/StateofDiversity.TheNativeTechEcosystem.pdf

Exploratorium California K–12 Science Leader Network, which grew from serving a single county office to spanning the state of California.

CHALLENGES TO SCALING PROMISING INNOVATIONS

The complexity of educational environments results in embedded challenges that can negatively affect successful scaling and sustainability efforts. Challenges identified from the literature review in combination with the compendium submissions include:

• Changing policies and priorities
• Staff turnover
• Keeping materials updated
• Monitoring implementation and outcomes at scale

If an innovation is designed to align with specific policies or to meet specific goals, there can be a large threat to scaling and sustainability when policies or priorities change and are not in line with the direction of the innovation. Scaling and sustainability are aided by buy-in from and capacity

building with individuals and organizations who are the enactors. If individuals championing the innovation at an organization or those trained to implement it leave, it can hinder continuity and efforts to scale the innovation. During scaling efforts, it can be difficult to also update materials as these would require additional development and testing. This can lead to materials that are out-of-date with the latest advances in STEM fields. Working with partners, especially teachers, who adapt and update materials can help mitigate this. Lastly, once a program has scaled broadly, it is challenging to monitor and evaluate. When an innovation is being implemented and adapted in many different contexts, it is difficult to draw conclusions across sites and learners. Additionally, the cost of evaluating something at this scale in a deep way can be prohibitive.

Although these issues were identified as having impacts on the individual innovations, they represent more systemic issues. Chapter 8 provides a more in-depth discussion of these challenges and offers suggestions for how the system can respond.

SUMMARY

Pre-K–12 STEM education innovations can be implemented in a variety of contexts to achieve a variety of different goals. As highlighted throughout the compendium commissioned for this report, although there can be notable challenges to scaling and sustaining innovations, there are several factors that support the conditions for scaling, to include a “tight but loose” framework; alignment of goals, policies, and practices; capacity building and organizational support; adaptability to a variety of settings; ease of adoption; and partnerships and networks. It is important to note that these are not the only factors that are needed for scaling, nor are they a requirement for successful scaling efforts. Looking across the various innovations, it is clear that there is not a “one-size-fits- all” approach to scaling, but rather, design of scaling innovations depends on the goals of the innovation, the resources available, and the contexts and learners they are trying to reach. Moreover, the process of evaluating and understanding implementation and outcomes can be challenging at scale, especially harder-to-study aspects such as fidelity of implementation, integrity of implementation (as described in Chapter 4), and implementation in a variety of contexts. Although many innovations in the compendium were able to collect this information and use it to scale the innovation, evidence cannot cover all settings and all participants. Finally, as discussed in Chapter 5, continuous funding, especially larger grants from federal sources, are important to funding these efforts, which can lead to a robust innovation that has scaled across a range of contexts.

Overall, in an effort to best understand the supportive conditions that allow for the scaling and sustaining of innovations, there is a need for

consideration as to how and in what ways evidence of scale and implementation is gathered, what types of evidence and understanding are important for the development and scale of innovations, and how funders can support these efforts. Chapter 8 will unpack these ideas more.

REFERENCES