A content analysis of alignment messages to the Next Generation Science Standards

Teachers are a critical component to standards-based reform systems, which require that reforms conceived at the national level pass through several layers of the educational system before impacting learning in the classroom. The Next Generation Science Standards (NGSS) are an example of this type of reform and pose significant challenges for alignment between levels given their three-dimensional nature alongside inclusion of ambitious and novel reform ideas. To examine translation of NGSS reforms across levels, we provide a content analysis of alignment messages conveyed to teachers through practitioner literature. Analysis indicates some coherence with national messaging around alignment to performance expectations and science and engineering practices. Additionally, alignment to broader reform ideas like engaging in science practices, integration, engineering, and focus on phenomena were represented to teachers. However, qualitative analysis of these representations indicate that reforms are often superficially portrayed, variably defined, or missing altogether. Findings indicate that teachers receive numerous messages regarding what it means to align to the NGSS and few elaborations on how to operationalize reforms. Our work suggests a need for intentional consideration of how to design representations for practitioners that consider teacher sensemaking around novel reforms. Additionally, we see a need for further development of coherence among the research community regarding alignment to the NGSS and agreement on definition of key reform ideas. Future work should consider how teachers use and understand these representations as they enact the NGSS in their local contexts.


Introduction
Standards-based reforms like the Next Generation Science Standards (NGSS) are developed with the intention to improve teaching and learning in science education (NGSS Lead States, 2013). However, the mechanism of standards-based reform (SBR) dictates that standards written at the national level must pass through several layers of the educational system before reaching teachers

Disciplinary and Interdisciplinary Science Education Research
A content analysis of alignment messages to the Next Generation Science Standards Jamie Tanas 1* and Gavin Fulmer 1 Page 2 of 18 Tanas et al. Disciplinary and Interdisciplinary Science Education Research (2023) 5:5 defined and calculated numerically (Webb, 2007), but has also been defined broadly by examining what components of standards and key reform ideas are operationalized during implementation (Massell et al., 1997;Smith & O'Day, 1990). The NGSS present a particular challenge for alignment given their three-dimensional nature (Fulmer et al., 2018). Additionally, there are other key reform ideas present within the NGSS documents that may be translated differently across these levels and channels including use of phenomena, integration of science and content, incorporation of design or engineering, and others (NGSS Lead States, 2013). As such, it is critical to examine how alignment is conceptualized and reform ideas are represented at each level in the SBR system in order to understand how standards impact teaching and learning.
As most proximal to student learning, teachers play a critical role in the success or failure of science education reforms. Reforms conceptualized at the national level through policy documents like the Next Generation Science Standards (NGSS) must ultimately be operationalized by teachers in the classroom to afford any real changes to teaching and learning (Bybee, 2014;Pruitt, 2014). We know that many teachers develop their own instructional materials (86% at the US secondary level) and, even when utilizing prepared curricula, teachers' understanding of reform ideas impacts their enactment in the classroom (Banilower et al., 2018;Tekkumru-Kisa et al., 2019). As sensemaking professionals, then, teachers' interpretations of the key conceptual shifts within the NGSS will determine the success of these ambitious reforms.
In order to examine how teachers are making sense of the standards, we must first determine what messages teachers are receiving regarding NGSS reforms. As NGSS reforms are translated for communication to teachers, what messages are teachers receiving regarding what it means to align to the NGSS regarding pedagogy and instructional materials? How do these messages compare to those at the national level, within the research literature, and as intended by writers of national reform documents? Practitioner journals speak directly to teachers and include messages from both peer in-service teachers and science education researchers. These journals are written for a teacher audience with a focus on practical application of research findings and reform ideas (Guidelines for Authors: The Science Teacher, n.d.). As such, articles within these journals provide one comprehensive national sample of the messages teachers receive regarding how to interpret the NGSS into lessons, assessments, and pedagogical practices for and by teachers. This paper will examine the nature and representation of these alignment messages within practitioner literature as a first step in furthering our understanding of teacher sensemaking and, ultimately, enactment of NGSS reforms within the SBR system. Taking a sensemaking lens, our systematic content analysis also includes a qualitative analysis of the language used to represent NGSS reforms with particular attention to coherence, potential misconceptions, and clarity of ideas. In this study we will examine the following research questions: 1. What messages regarding alignment to the NGSS are conveyed to teachers in practitioner literature? 2. How are these alignment messages represented and defined for teachers? What language is used to represent NGSS reforms?

Reform representation, sensemaking, and implementation
Studies of SBR from the common core standards, particularly mathematics, provide insight into the role of individual cognition and policy representation in affecting the success or failure of reform. One aspect of policy representation critical to sensemaking is the language used to communicate reforms. Spillane (2000) asserts the central role of language in reform implementation as it is, "the chief medium that policymakers have for representing their ideas about reforming practice" (p. 152).
Language is used to communicate the central ideas of reform. However, studies have shown that the language used to convey the nature of reforms is often interpreted and attended to differentially by individuals. In examining SBR in mathematics, Spillane (2000) found that individuals attended to various "reform signals" including hands-on/manipulation, problem solving, other subject integration, and real-world connections, among others. Additionally, the intended meaning of these signals was often not taken up by implementors, and instead used to reinforce previous ideas about teaching practices resulting in little substantive change in individual's ideas about instruction. Hill (2001) notes the same issues in examining teachers tasked with translating mathematics standards into curriculum. In this study, teachers misunderstood language, held local understandings of vocabulary, or perceived some reforms as similar when they were not, resulting in a misalignment between the curriculum and the standards. Notably, Hill (2001) relates this misalignment directly to the representation of policy, stating alignment could have been improved with "better inputs from state policy" like "fewer sub objectives", videos, or example lessons (p.313). This finding directly relates representation of policy to the sensemaking that occurs when individuals implement reforms, emphasizing the importance of analyzing both the form and the language used to convey reforms to stakeholders. Sensemaking by individuals involved in reform implementation has also been demonstrated to occur differentially across the various levels of implementation, suggesting study of the role of context and cognition should occur at each level for comparison and tracking of how reform messages may have changed. In their examination of common core standards implementation across several districts Coburn et al. (2016) note that lack of coherence across states and districts around reforms can be attributed to differential learning processes among teachers, school administrators and state leaders. They concluded that lack of alignment between curriculum materials, assessments, and professional development contributed to differential learning, or sensemaking, among these individuals, ultimately resulting in implementation that "looked different from one classroom to the next, even within the same school and district" (p. 245). Ball et al. (2011) refer to this as the "problem of meaning" in policy, describing how teachers and others involved in policy enactment actively shape interpretation by explaining and translating policy, selecting, and enforcing meanings. This pattern is seen across disciplines (Spillane & Zeuli, 1999) and levels of the educational system (Coburn, 2005). Researchers attribute differences in sensemaking to representations of the policy itself (Spillane, 2000) as well as contextual factors that influence sensemaking including communities of practice (Galluci, Coburn 2001), local policy (Spillane, 1999;Desimone, 2013), opportunities for professional development (Coburn, 2001;Klieger & Yakobovitch, 2012), and teachers' conceptions of accountability (Louis et al., 2005). These studies assert that study of reform without consideration of individual sensemaking will be missing a critical component to understanding the process.

NGSS implementation and teachers
To date, no study has examined the role of the representation of reforms and teacher cognition as it relates to NGSS implementation. However, several studies have dealt more broadly with teacher sensemaking related to the standards and the contextual factors that influence this process. Smith and Nadelson (2017) examined teachers' perceptions of alignment when implementing NGSS based instruction, with a particular focus on the science and engineering practices (SEPs). As a sensemaking model predicts, they found that teachers attended to certain SEPs more than others, and implementation of practices were related to individual beliefs regarding science teaching and learning. Similarly, others have found that teachers engage less in modeling, investigation, and argument (Hayes et al., 2016), and have difficulty incorporating engineering as called for in the NGSS (Richmond et al., 2016;Sherwood, 2020) gives a clear demonstration of the role of cognition in teachers' attempts to implement the NGSS through examination of pre and post drawings illustrating teachers' conceptions of changes in teaching practice after NGSS professional development. She notes that teachers have difficulty operationalizing the NGSS in practice due to ambiguity when integrating new and old ideas about teaching and may perceive reforms as similar enough to result in a "business as usual" approach (p. 592). While the Sherwood (2020) study focused on the nature of teacher knowledge, our theoretical frame draws attention to the role of representations of reform presented to teachers during professional development that resulted in difficulties differentiating new ideas from old. This lens has yet to be examined and discussed.
Others have noted that teachers do, in fact, make differing interpretations of NGSS reforms as they learn about the standards. Kawasaki and Sandoval (2020) found that teachers revising their lessons to align the with standards adopted differing instructional strategies with those lessons, often misaligned with the intent of the NGSS. Allen and Penuel (2015) note that teachers may enact their learning from NGSS professional development differently as they return to their home schools and experience a lack of coherence and ambiguity between those sites. Additionally, local learning communities can play a part in teachers' sensemaking around the standards as they negotiate meaning together and are influenced by local tools and language (Friedrichsen & Barnett, 2018). Even within a single district, Cherbow et al. (2020) found that each school in their study "faced significant vertical incoherence concerning the goals for adoption of reform science standards" and that this incoherence contributed to differing interpretations of NGSS reforms among teachers and administrators (p. 466). Similarly, in studying the sensemaking of district science coordinators around the NGSS, Haverly et al. (2022) found three distinct themes in their understanding of science reform which were impacted by local contextual factors like involvement in professional work groups and participating in professional networks. These studies demonstrate that study of stakeholder sensemaking at each level of implementation is critical to understanding how reforms are translated to the classroom.

NGSS and alignment
Previous iterations of science standards, like the National Science Education Standards (NRC, 1996) or Benchmarks for Science Literacy (AAAS, 1994), outline content standards in the form of lists or short statements. The NGSS, however, are written as performance expectations describing what students should be able to do at the end of instruction. Additionally, the NGSS emphasize threedimensional learning requiring students to "operate at the intersection of practice, content and connection" and require a significant shift in thinking about how the standards can be used to align curriculum, assessment, and teaching (NGSS Lead States, 2013, p. xvi the standards for use throughout the educational system and make defining alignment difficult. From a sensemaking perspective, this change must be considered at all levels of implementation, and particularly for teachers. Within the research literature there have been numerous methods described for determining alignment to the NGSS (Fulmer et al., 2018). Some suggest starting with a Performance Expectation (PE), or bundle of PE's, unpacking them into smaller units such as learning goals, and using those to build a teaching sequence, storyline, or assessment task (Pellegrino, 2014, Krajcik et al., 2014, Pruitt, 2014, Harris et al., 2017. Others have focused on a single component of the three-dimensional PE's and used that to determine alignment. An alternative method involves using the SEPs as a foundation for building a storyline or as a priority in identifying instruction aligned to the NGSS (Hayes et al., 2016;Reiser et al., 2017). The Crosscutting Concepts (CCC's) have also been proposed for use in developing NGSS aligned units for instruction (Fick et al., 2017). In a more holistic approach, other researchers have viewed alignment more broadly in terms of inclusion of engineering (Moore et al., 2015), integration of content and practices (Debarger et al., 2017) or by examining coherence across grade levels or learning progressions (Hermann-Abell & DeBoer, 2018). Tools used to determine alignment to the NGSS are often complex and qualitative in nature, like PEEC (Achieve, 2017) and EQUIP (Achieve, 2016). These lengthy rubrics reflect the difficulty and complexity in determining alignment.
As a research community we have regularly studied NGSS professional development materials and NGSS aligned curricula (Debarger et al., 2017;Allen & Penuel, 2015;Hayes et al., 2019;Duschl & Bybee, 2014;Tuttle et al., 2016), but rarely wider representations that teachers may access and learn from themselves like practitioner literature. To date there is no analysis of this type of NGSS reform translation. In this study we use a sensemaking framework to examine messages of NGSS reform as conveyed to teachers through practitioner literature, with particular focus on what it means to align both teaching and instructional materials to the standards.

Methods
To examine the representation of alignment messages directed at teachers regarding NGSS reforms, qualitative content analysis (QCA) was employed (Mayring, 2015). Content analysis is a research technique used to make valid inferences from texts, specifically useful for comparing similar phenomena as represented in different texts (Krippendorff, 2018). As such, this method is well suited for our research questions exploring the representation of NGSS alignment messages within practitioner literature. QCA was selected as this methodology allows for use of both quantitative and qualitative representation of textual material, providing an in-depth and systematic summary of the content. Additionally, QCA outlines a method for both deductive and inductive coding of textual material (Mayring, 2015). This was particularly important to our research questions as we explored emerging, previously unstudied, messages regarding NGSS alignment in practitioner literature.

Sampling
Relevance sampling was used to select the texts included in the analysis. Krippendorff (2009) defines this type of sampling as "selecting all textual units that contribute to answering given research questions" (p. 119). As the research questions focus on the representation of NGSS reforms for teachers in the classroom, we focused our analysis on texts written for a teacher audience and, within that sample, texts that address the use of NGSS for alignment of instructional materials. As such, texts selected for analysis included those that: (1) Were intended for a practitioner audience and (2) Contained material related to alignment of instruction or instructional materials to the NGSS.
Texts for analysis were identified through a systematic search of the literature. The two major educational databases EbscoHost and Education Source were used to search for literature with the following terms: NGSS and Alignment, NGSS and Lessons, NGSS and Assessment, and NGSS and Teachers. This search resulted in a total of 564 articles. Each of these articles were read, in entirety, to determine inclusion according to the criteria. Since the focus of this study is on messages directed at practitioners, literature directed at a non-practitioner audience, such as conference papers, empirical studies, and theoretical papers were excluded. The authors exercised their judgement to determine the intended audience, research or practitioner, for each study. Additionally, literature that lacked details related to alignment or use of the NGSS in the classroom were also excluded. For example, book reviews, news briefs, and articles with no mention of the NGSS within the text. Lastly, literature for analysis was limited to K-12 instruction, therefore articles outside of this range were also excluded. After excluding articles that did not meet inclusion criteria and removing duplicates, this process resulted in a total of 185 articles for inclusion in the analysis (See Table 1).

Unitizing and coding
Unlike traditional content analysis, which specifies predetermined lengths of text for analysis, Qualitative Content Analysis allows for a broader definition of units (Mayring, 2015). This method of unitizing is useful when prioritizing the context and meaning of the material, as in this study. Therefore, we identified complete ideas or messages as the unit of analysis (Seibert & Draper, 2008). A message was defined as a segment of text detailing how a teacher should use the NGSS, describing the focus of the NGSS, or specifying what component of the standards the teacher should attend to in terms of alignment of instructional materials. These segments ranged in length from a single sentence to several sentences. Every message fitting this definition was assigned a code. Additionally, text segments could be given multiple codes if they applied to the message. This process of simultaneous coding (Saldaña, 2016) allowed for analysis of co-occurring codes and maintenance of the complexity of messages as they would be read by teachers. NVIVO, a computer assisted qualitative data analysis software, was used to organize data during coding and analysis of all articles. Messages were coded using a combination of a priori, or hypothesis (Saldaña, 2016), codes and inductively developed codes. A priori codes for alignment messages were developed from our previous review of the research literature related to NGSS alignment (Fulmer et al., 2018, Table 2). When a message did not fit an apriori code, new codes were developed to characterize the text. Emergent codes were developed and tested iteratively throughout the coding process using a constant comparative method (Corbin & Strauss, 1990;Miles et al., 2020). For example, during coding we found that many articles referred to engaging in practices as a key reform idea within the NGSS. This code was added to capture this message within the practitioner literature. These inductive codes will be discussed further in the findings as they may indicate differences between conceptions of alignment between.
literature aimed at the research community versus what is conveyed in practitioner literature to teachers.
Second cycle coding (Miles et al., 2020) was performed after the first round of coding to collapse codes into a smaller number of analytic units and themes. For example, during the first round of coding there were two separate codes for coherence and learning progression. During second cycle coding, these two codes were determined to be similar enough to be combined into one category and given the code learning progression.

Analysis
Secondary analysis of coded messages was performed iteratively throughout the coding process through analytic memo writing (Glesne, 2016a, b). These memos noted emerging patterns noticed by the primary researcher, patterns to explore further, and further detail on noted unique cases within the analysis.
After second cycle coding, a code frequency table (Miles et al., 2020) was used to determine the prevalence of the various alignment messages within the literature. The most frequent codes were analyzed individually to further define patterns and meaning within those messages. Using NVIVO, all text assigned a particular code was, first, analyzed for themes by looking across each code for patterns. Analysis was grounded in the sensemaking framework for reform implementation (Spillane et al., 2002). As such, the researcher paid particular attention to the language used within the messages, definitions, elaborations on reform ideas, explicit similarities and differences and representational form of the messages (tables, figures, narrative writing etc.). Spillane et al. (2002) assert that these factors are particularly relevant to teacher sensemaking as individuals with the policy literature to make sense of NGSS reforms, and therefore guided the analysis of coded messages.

Trustworthiness and dependability
As a primarily qualitative study, the reliability and validity concerns of this study are of a qualitative nature and, therefore, we conceptualize these in terms of dependability (Merriam & Tisdell, 2015) and trustworthiness (Glesne, 2016a, b). First, we acknowledge the interpretative nature of this study, recognizing that the findings represent one interpretation of the body of NGSS alignment messages aimed at teachers. However, in reporting  our methods, codes, and findings with detail and rich description we contend that this interpretation provides a valuable and theoretically sound analysis of the content and, further, allow the reader to judge the transferability and limitations of the findings. Throughout the study a detailed and regular audit trail of procedures, decisions, and analytic memos were recorded using NVIVO to ensure reliable reporting. Like many qualitative studies, coding and analysis were carried out primarily by the first author, as such reflexivity and positionality are important considerations when interpreting the findings (Glesne, 2016a, b). The first author is a white, female, graduate student and former secondary science and mathematics teacher. She is a coauthor on the research team's first publication, a literature review of NGSS alignment methods (Fulmer et al., 2018). As such, she has a high level of familiarity with current literature on the topic and experience analyzing the literature through the relevant theoretical lens. Her experience as a secondary teacher also provided insight into the potential interpretation and use of practitioner literature by teachers.
Finally, peer examination (Merriam & Tisdell, 2015), involving regular discussion of findings and codes with the co-author regarding the congruency of findings and interpretations occurred regularly throughout both the coding and analysis processes. These examinations provided ongoing checks for bias, consistency, and grounding in theory. Throughout analysis we also engaged in negative case analysis (Miles et al., 2020) to test our findings and interpretations.

Materials coded
In all, 185 articles were coded and analyzed for alignment messages. These articles came from a variety of practitioner journals (see Fig. 1), but the majority (85%) came from four journals: The Science Teacher, The American Biology Teacher, Science Scope and Science and Children. The Science Teacher is published by the National Science Teachers Association (NSTA) and is written for "grade 9-12 teachers, university faculty responsible for teacher preparation, and state and district science supervisors and leaders" (NSTA, n.d.). The American Biology Teacher is published by the National Association of Biology Teachers (NABT) and is "designed to support the teaching of K-16 biology and life science" (NABT, n.d.). Science Scope is published by the NSTA and designed for the audience of middle level and junior high school science teachers. Lastly, Science and Children, also published by the NSTA, is written for the elementary-level science teachers. This indicates that the sample of articles analyzed includes articles intended for a range of subjects and grade levels, although may be weighted to the 9-12th grade practitioner audience. Since the sample was selected from all available practitioner literature, this may also indicate a greater number of practitioner articles regarding the NGSS for this audience. Further study of both frequency and if the messages differ between the elementary and secondary level may be valuable for the research community.
Articles for analysis were selected in May of 2020. The articles ranged in publication date from 2012 to 2013, consistent with the publication of the NGSS in 2013 (NGSS Lead States, 2013) (Fig. 2). The most articles (24%) were published in 2014, directly after the writing of the NGSS and has dropped significantly since, with less than half of that amount in 2017-2018.

Alignment messages
Alignment messages in the practitioner literature were characterized in two categories: mention of a specific component of the standards (PE, DCI, SEP, CCC) or broader conceptualizations of alignment to NGSS reform ideas (Integration, Learning Progression, Engineering etc.). For each of these, a quantitative measure of their prevalence as coded will be given alongside a qualitative discussion of how these alignment messages are represented and defined in this body of literature.

Alignment to NGSS dimensions
Articles were coded for each use of a specific dimension of the NGSS standards for alignment. These codes were derived from our previous review of the literature (Fulmer et al., 2018) and include using a single PE to align materials, bundling more than one PE, or using one of the three dimensions for alignment: SEP, DCI or CCC.
Definitions and examples for these the primary codes are included in Table 3. Additionally, while analyzing the literature we found some articles used combinations of these dimensions for aligning material by focusing on two of the dimensions (CCC and SEP, CCC and DCI). Other unique cases included combining more than one SEP or DCI and referring to all three dimensions (SEP, DCI and CCC) without mention of a specific PE. Lastly, one article used evidence statements, and another learning performances as referents for alignment and are included to highlight these unique approaches.
As in the research literature, we see the three-dimensional standards used in varying ways to demonstrate alignment to the NGSS. Not surprisingly, use of a performance expectation as a single PE or a bundle of PEs were most frequently used for alignment. This is consistent with a review of the research literature recommending use of PEs for alignment (Krajcik et al., 2014;NGSS Lead States, 2013). PE or bundle of PEs was coded together 133 times, or 48% of the alignment codes compared to the rest (See Fig. 3).
The third most frequent referent for alignment coded for was a single SEP. These articles referenced alignment to the NGSS in terms of inclusion or focus on a single science and engineering practice within the instructional material. We further coded these messages according to the SEP specified and found the SEPs were not equally referenced. Engaging in argument from evidence was aligned to the most, while asking questions and defining problems was aligned to the least of the eight (See Table 4). One of the articles providing examples of activities aligned by a single science practice, McNeill et al. (2015), also provides a framework for grouping these practices by investigating, sensemaking, and critiquing practices. Interestingly, our frequencies align with this grouping, indicating that instructional materials focusing on single science practices most often referenced the critiquing practices (argument and evaluating information), followed by the sensemaking practices (models and constructing explanations), and least often the investigating practices (computational thinking and asking questions). This may indicate a shift in instructional materials toward focus on those sensemaking and critiquing practices coinciding with the introduction of the NGSS. The primary learning outcome of the Bird Box Survey Project was to increase student knowledge, awareness, understanding, and application of the science inquiry process… and the scientific practice of planning and carrying out investigations" (Willis, 2014, p.

40) CCC
Cross Cutting Concept is used as the referent for alignment "As students undertake this review, they naturally begin to connect many crosscutting concepts identified by the NGSS. These include learning about spiders' forms and functions, their niches within the environment, and the overall complexity and patterns of the natural systems in which they live. " (Craven et al., 2019, p  The Botanical Phylo-Card Game addresses several components of Next Generation Science Standards such as Inheritance/Variation in Traits (3-LS3-1, HS-LS3-1, HS-LS4-2) and Natural Selection/Evolution (MS-LS4-2, HS-LS4-1) (Gibson & Cooper, 2017, p. 241)  Messages coded as alignment to multiple SEPs listed more than one specific practice or generally referred to alignment to the NGSS practices. For example, "the project aligns with all eight of the science and engineering practices (SEP) embodied in the Next Generation Science Standards" (Ortolano et al., 2017, p. 53) or "this investigation incorporates the Next Generation Science Standards high-leverage practices of scientific modeling and argumentation" (Williams et al., 2018). Single SEP or multiple SEPs as referents when combined were coded a total of 84 times, equal to the highest referent: single PE. This indicates that the science practices, either individually or collectively, are one of the most common referents presented to practitioners for demonstrating alignment of instructional materials to the NGSS.
Materials coded as using a DCI or multiple DCIs for alignment focused on the subject-matter core ideas often listing the disciplinary code (LS, PS, ES, ETS), core-idea (1,2,3 etc.) and sub-idea (A, B, C, etc.). For example, "these activities correlate with disciplinary core idea ETS1.B: Developing possible solutions" (Zissman, 2013, p. 72) or more broadly as in, "these goals align with various disciplinary core ideas from NGSS, including Structure and Properties of Matter, Definitions of Energy, Conservation of Energy and Energy Transfer, and Relationship Between Energy and Forces" (Stroupe & Kramer, 2014, p. 72).

Broad focus for NGSS alignment
In addition to calling for alignment to a specific standard or component of a standard, NGSS policy documents also call for broader changes to curriculum and pedagogy that may be used to judge alignment to the standards. Therefore, we also coded for instances where alignment to the NGSS was described or judged in terms of broader reform ideas associated with the standards. Within the practitioner literature reviewed, we found five reform ideas used most frequently as indicators of alignment to the NGSS. These include integration of content and practices, inclusion of engineering or design, use of phenomena, learning progressions, and engaging in science practices (See Table 5). Four of these were a priori codes from our review of research literature (Fulmer et al., 2018), while engaging in science practices or "Doing Science" was an emerging code developed during coding and analysis. For each of these codes, we will discuss the nature of the reform ideas as represented in practitioner literature with specific examples.
Two less frequently coded themes include student-centered (9) and inquiry (11). However, we find these notable A major modification to the lesson that is different from the common version is that students are encouraged to think and act like a scientist/engineer to build and test their own models rather than following existing step-by-step procedures. and worth discussing as they are both ideas more closely associated with previous science education reforms and may, from a sensemaking perspective, indicate ongoing negotiation of relationships between, or definitions of, new and old science reform concepts.

Engaging in science practices or "Doing Science"
After the second round of analysis, these two codes were combined and are reported together due to their similarity in content. 42 articles referenced engaging in science practices or "doing science" as a key conceptual shift of the NGSS. Of the two, the most frequently coded was that the NGSS require students to engage in science and engineering practices. Within these articles, the SEPs themselves were directly defined as what scientists do (Puttick & Drayton, 2017) and "an attempt to capture the essence of how the scientific community works to generate knowledge" (Duncan & Cavera, 2015, p. 70). More often, however, engaging in science practice was defined by contrast, or emphasizing changes from previous iterations of science education reform. Engaging in practices was contrasted with teaching about science (Huff, 2016), following a set of scientific processes (Curran et al., 2016), repeating steps predetermined by the teacher (Stroupe & Kramer, 2014), memorization of facts (Joyner & Marshall, 2016), learning facts or content (Tuttle et al., 2014;Passmore 2015), and stating what students should know (Fink, 2014). Analysis of these articles also reveal the rationales directed at teachers for engaging in science practices as a focus of the NGSS. The most frequent rationale dealt with the use of science and engineering practices for sensemaking, learning or developing knowledge (Curran et al., 2016;Potter et al., 2016;Quinlan, 2019;Stroupe & Kramer, 2014;Lawrence et al., 2016;West et al., 2015). Authors also stated that students should engage in science practices to develop understanding of what scientists do (West et al., 2015;Deffit et al., 2017;Harmon et al., 2019), gain an appreciation for science and engineering (Ewing, 2015), clarify relevance of science to everyday life and increase engagement (Stuart et al., 2017), and prepare for college career and citizenship (Bokor et al., 2015).
When describing what scientists do, the authors referred broadly to science practices, but also specified engaging in investigation (Yochum et al., 2013;Puttick & Drayton, 2017), testing models (Lawrence et al., 2016), generate hypotheses (Duncan & Cavera, 2015), answer questions and solve problems (Ewing, 2015), analyze data (Bouwma-Gearhart & Bouwma, 2015), and communicating ideas (Passmore, 2015). As use inclusion of the SEPs and use of the term practices is a novel aspect of the NGSS, how these terms are represented in the practitioner literature and differentiated from previous reforms will be critical to understand. From this analysis, it appears there is still some ambiguity around what it means to engage in science practices or "do science" according to the NGSS.

Integration
The second most common broad focus code for NGSS alignment was integration. Although integration was used frequently as an indicator of alignment to the standards, both what was to be integrated and how integration was defined varied in practitioner literature.
An elaborated explanation of integration was given in only two of the 185 articles analyzed. In all other instances integration was used without definition outside of stating what was to be integrated. A definition may be inferred by several synonyms related to integration within the literature. Integration was used synonymously with "blending" or "blend" (Krajcik, 2013(Krajcik, , 2014Puttick & Drayton, 2017), "couple" (Passmore, 2015), "teach alongside" (Gould et al., 2014), "woven together" or "interweave" (Lauren et al., 2016;Schatz & Fraknoi, 2017;Passmore et al., 2013), and "use collectively, " among others (Fumagalli, 2016). Although these synonyms give a sense that multiple components of the standards must be used together, how this is operationalized both pedagogically in the classroom, and within instructional materials is left largely unexplained in this set of practitioner literature.
What was to be integrated was also highly variable across the literature. Most frequently called for was integration of "three dimensions" or specific reference to integration of CCC, DCI and SEPs (too many to cite or add quantify). However, integration was also used to refer to just two of the three dimensions; DCI and SEP (Krajcik, 2013), CCCs and single SEP (Haines et al., 2017), and "content ideas and crosscutting concepts" (Gould et al., 2014). More broadly, others describe integration of "content and practice" (Passmore et al., 2013;Huff, 2016) or "knowledge and practice" (Talanquer, 2019). Additionally, others noted that the NGSS "call for the integration of science and engineering" (West et al., 2015;Turgeon, 2014), "real-world" integration (McConnell & Dickerson, 2014), and integration of multiple performance expectations (Concannon & Brown, 2017).
Only two articles elaborate on the how and why of integration. Houseal (2015) uses a Venn diagram to illustrate how all three dimensions integrate within a PE, emphasizing that integration occurs if "at least one activity within the entire lesson or summative assessment will map in the center (PE)" (Houseal, 2015, p. 61 paucity of explicit guidance on how and why to integrate NGSS dimensions contrasts with the relatively high number of articles that introduce integration. This may inadvertently signal to teachers that they should have an awareness of integration but that its practical implementation is somehow beyond them or reserved for established experts.

Engineering or design
Incorporation of engineering or design within science lessons was seen as a focus of the NGSS in 28 of the practitioner articles. These authors highlighted the inclusion of engineering as a novel aspect of NGSS as Willard et al. (2012) state, "one new aspect of NGSS is the inclusion of engineering as a core idea alongside life, earth, and physical science" (p. 37). In fact, three of these articles focused specifically on adapting or developing instructional materials with engineering in mind (Whitworth & Wheeler, 2017;Boesdorfer & Greenhalgh, 2014;Westfall, 2015) conceptualizes the inclusion of engineering as "not much different from what…most teachers have been doing in science classrooms for years" requiring "small but effective changes" to incorporate engineering (p. 34). In contrast, Whitworth and Wheeler (2017) focus on "designing a solution to a problem under constraints without stepby-step instructions" as an explicit definition of engineering within instruction (p. 26). Elsewhere in these coded articles, engineering was defined by the process of design or design cycle (Boesdorfer & Greenhalgh, 2014), as the SEP constructing explanations and designing solutions termed the "engineering practice" (Moyer & Everett, 2013, p. 80), or as solving problems (Brown et al., 2014;West et al., 2015) explicitly describe the components of engineering design as described in NGSS Appendix I in outlining how engineering should be incorporated in a classroom activity: defining the problem, designing solutions, and optimizing the design solution (p. 65). Defining what engineering looks like when incorporated in instructional materials is an ongoing challenge for the research community. Of the articles describing instructional materials aligned to the NGSS for this code, there were seven articles aligning to engineering standards (ETS) alone for alignment and seven incorporated engineering standards alongside other disciplinary content standards (LS, PS, or ES). This indicates that inclusion of engineering is being translated in various ways, including explicit use of engineering standards as well as broader conceptions of engineering processes or design. The other articles coded to this category did not explicitly align to a PE, but when describing how their materials aligned to the NGSS they highlighted the inclusion of engineering or design.

Phenomena focus
25 articles were coded as pointing to phenomena as a focus for alignment to the NGSS. In describing the use of phenomena as central to the NGSS, these articles state that students should make sense of phenomena (6) or explain phenomena (8). Use of phenomena was associated with use of a driving or open-ended question in five of the articles. Rationale for phenomena use and detail on defining and using a phenomenon was limited in the literature. Four articles used the term anchor phenomena, suggesting the phenomena should ground or lead a lesson or unit. While others more specifically stated to "lead with a new initial phenomenon" when adapting materials to align to the NGSS (Forsythe, 2018, p. 74). Only two articles directly define phenomena, both centered on teaching phenomena (Like et al., 2019) and phenomenabased teaching (Hancock & Lee, 2018;Like et al., 2019) defines phenomena as, "observable or natural events" (p. 152) and Hancock and Lee (2018) as "objects and events that can be observed and/or measured" (p. 44). Others described the type of phenomena that should be used including scientifically rich (Campbell et al., 2013), and familiar to students (Madden et al., 2014). Two articles gave rationale for their choice of phenomena. Turley et al. (2016) states that an anchoring phenomenon should be "chosen to spark questions" (p. 36). Hancock and Lee (2018) describe steps for adapting lessons for phenomena-based teaching and suggest choosing a phenomenon that relates directly to the PEs for the unit and one that is "complex" and "piques student interest" (p. 44). The frequency of this code indicates that phenomena is seen as a central reform idea of the NGSS, however the practitioner literature varies in describing the rationale for use of phenomena and in the language used to describe the role of phenomena in instruction.

Learning progression
15 articles noted learning progression as broader reform idea indicating alignment to the NGSS. For example, Johnson and Dodson (2016) state, "this unit, which aligns with the Next Generation Science Standards, is based on research into learning progressions, defined as descriptions of the successively more sophisticated ways of thinking…as children…investigate a topic over a broad span of time" (p. 54). In addition to explicit reference to learning progressions, others described learning progressions as purposefully building (Krajcik, 2013), "a continuum of exposure" (Bryce et al., 2016, p. 38), and a series of coherent activities (Edwards et al., 2020). Additionally, references to learning progressions were linked to both consideration of development (Mohl et al., 2016) and grade-appropriate content (Ewing, 2015;Fink, 2014). Only one article elaborated on how learning progressions Page 11 of 18 Tanas et al. Disciplinary and Interdisciplinary Science Education Research (2023) 5:5 are developed using research and "longitudinal studies" (Parker et al., 2015, p. 233).

Inquiry and student-centered
These two codes were associated with NGSS alignment in 11 and nine articles, respectively. We found these valuable to note because they are reform ideas that have been associated with former science standards documents like the National Science Education Standards and Benchmarks for reform (NRC, 1996;AAAS, 1993). From the perspective of a sensemaking framework for implementation of reform, it may be particularly important to explore how previous reform ideas are differentiated from new reforms or how ideas may be perceived as similar and not requiring change on the part of individuals (Spillane et al., 2002). The codes for inquiry largely composed of referring to NGSS aligned lessons as inquiry based. Some authors used more specific language like modeling-based inquiry (Bouwma-Gearhart & Bouwma, 2015). Student-centered was connected to descriptions of students using the SEPs and change in students' role from passive to active in their learning as a key of the NGSS. In one case these two co-occurred: "the NGSS put an emphasis on science and engineering practices while also focusing on student centered activities and the involvement of inquiry in our lessons" (Goode, 2019, p. 340). Again, the appearance of these terms along other NGSs terminology like the SEPs may be problematic for teachers who may not know how to differentiate between enactment of the SEPs as intended by the authors of the NGSS versus inquiry and student-centered teaching from previous reforms.

Elaboration and representations of alignment
As we coded for alignment referent, we also noted articles where the authors elaborated on processes or considerations for how to align materials to the standards, outside of simply listing a PE or standard dimension for alignment. This exploration was grounded in our sensemaking framework for policy implementation asserting that external representations are critical to the sensemaking process for individuals (Spillane et al., 2002). As such, design features of documents communicating policy, in this case practitioner literature, are pertinent to examine in relationship to how they may afford or constrain individual sensemaking (Greeno, 1998). Considering this, we noted articles that elaborated on the alignment process, showed explicit connection between instructional materials and the standards, or provided tools for alignment of instructional materials. In doing so we attended to both the content of the material as well as the form in which the information was presented. These elaborations detailing how to align materials to the standards took three general forms: tables, explicit steps or checklists, and rubrics. We highlight the characteristics of each type of representations with examples below. The most common form of representation communicating a detail on how instructional material was aligned to the NGSS was a table listing how each dimension of the NGSS (DCI, SEP, CCC) maps to the instructional material. The detail of how the material mapped to each of the dimensions varied in detail from a simple lesson number or title (Ortolano et al., 2017) to more descriptive "connections to classroom activity" (Lottero-Perdue et al., 2015). Additionally, authors described either student tasks (Cochrane, 2014), assessment questions (Furtak & Heredia, 2016), or subject matter content (Sultany & Bixby, 2016). As such, communicated alignment between the standard and the instructional material can vary in level of detail, grain size for alignment (whole lesson, versus individual question for example), and how the alignment is made (student action, scientific content, lesson activity, or assessment question). This is important for the research community to note and consider in the design of representations intended to scaffold these alignment connections. Future research may examine the affordances and constraints of each of these in terms of teacher learning, particularly considering situational factors like the purpose and audience of the material in determining how and what to represent when illustrating alignment of instructional materials. Additionally, it is valuable to note that nearly all of the tables provided space for connection to each dimension separately (DCI, SEP, CCC), which warrants further exploration of how this may constrain demonstration of the integration of dimensions as an important component of alignment. Further, the primary alignment code recorded in instructional materials in this body of practitioner literature was to a PE, or bundle of PEs, while elaboration on the connection to instructional materials via PE was rarely demonstrated. This brought us to consider, how do we represent alignment to a PE if not by separating the dimensions into DCI, SEP and CCC?
Other authors represented alignment to the NGSS with explicit steps or checklists. Veal and Sneed (2014) provide a checklist of questions to determine if a lesson meets the NGSS, narrowing alignment down to six yes or no questions to consider. Similarly, but narrowed to the engineering focus of the NGSS, Whitworth and Wheeler (2017) provide a self-check table to determine "is it engineering or not?" (p. 26). Hancock and Lee (2018) focus on the phenomena driven component of NGSS reform, detailing steps for "purposefully repurposing" existing activities to become phenomena-based (p.43) and Forsythe (2018) provide steps to modify 5E inquiry lessons to be more practice focused. Uniquely, Houseal (2015) uses a visual representation of alignment through a Venn diagram and instructs teachers to map (somewhat of a visual checklist) their lesson to the NGSS, examining if it incorporates each dimension and an integration of all dimensions. In contrast to checking if previous material aligns to the standards, others provide explicit steps for developing materials aligned to the standards. German (2017a, b) provides a step-by-step process for constructing an NGSS aligned assessment, and Puttick and Drayton (2017) do the same for developing an NGSS-aligned curriculum from learning performances. Representing alignment with steps or checklists provides teachers with explicit process, or how-to, information that is not present when illustrating alignment of a final product as in a table form. Rubrics provided another representational form for illustrating alignment to the NGSS. These provided finer grain detail on levels of student actions that demonstrate adherence to the standards. For example, instead of simply indicating what activity students will do to engage in an SEP, McNeill et al. (2015) provide a rubric for each of the SEPs detailing student performance from level 1 (not present) to level 4 (exemplary). The rubric is supplemented in the article with detailed narratives and classroom examples to show instructors both activities to engage students with an SEP and descriptions of student actions that demonstrate competence in the practice. Complementary, is Cherbow et al. 's (2019) Science Practices Lesson Adaptation Resources, a rubric adapted to activity for teachers where vignettes of instruction can be ranked by according to student engagement with the SEPs as they learn what classroom instruction using the SEPs should look like. Similarly, Mohl et al. (2016) provide a rubric for two CCCs: energy and matter and system and system models. The authors assert that the rubric should be used for backward planning of instruction and improved NGSS alignment to the CCCs. Unlike previous representations, rubrics demonstrate quality of alignment of instruction and instructional materials to the NGSS. Additionally, from a sensemaking perspective these tools provide explicit means for teachers to attend to the characteristics of reform enactment as intended by the authors of the standards and differentiate between potential misconceptions or misinterpretation of reforms.

Discussion
This content analysis gives a comprehensive overview of the current alignment messages to the NGSS as conveyed to teachers through practitioner literature. We assert that these messages play a critical role in the implementation of NGSS in classrooms as teachers construct their own understanding of reforms from these representations of policy (Spillane et al., 2002). Scholars who study reform implementation from a sensemaking perspective claim that attention to the representation of reform policies as they are conveyed across levels of the educational system is imperative to understanding the overall success or failure of reforms; particularly, the way in which these representations may constrain or afford the sensemaking of implementing individuals. In analyzing a representative body of NGSS practitioner literature we found that teachers receive multiple representations of alignment messages. Alignment to the dimensions of the standards is conveyed primarily by recommending bundling PEs or aligning to a single PE. More broadly, alignment is also conceptualized through reform concepts like "doing science" or engaging in science practices, integration or three-dimensionality, incorporation of engineering, or use of phenomena, among others.
Using Spillane and colleague's (2002) framework, we will discuss these findings in terms of three essential characteristics of policy representation that afford sensemaking for reforms, like the NGSS, that require significant, fundamental, change in the way practitioners think about teaching and learning. Spillane et al. (2002) suggest that reforms are represented (a) with clarity and coherence, (b) communicate deep underlying principles in a way that prevents adoption of superficial aspects of reform rather than the deeper ideas intended by the authors of policy, and (c) provide a balance between general and specific, or abstract and concrete, representations. We will discuss our findings considering these principles with examples from our analysis while making connections to current science education research.

Clarity and coherence
Clarity and coherence have long been recognized as important aspects of successful educational reform policy (Cohen & Spillane, 1992;Porter, 1994;Spillane et al., 2002) note that, "when policy is inconsistent or ambiguous it increases the discretion of implementing agents… over whether and how to put policy proposals into practice" (p. 414). Particularly in standards-based reform, where reform at a national level is intended to affect many other levels (state, district, school) and channels in the system (curriculum, assessment, teacher development), consistency is critical. From a sensemaking perspective, Spillane et al. (2002) focus on how these aspects affect individual sensemaking and may contribute to misunderstanding or misinterpretation of reforms by stakeholders as they implement them. In our study we found both consistency and coherence across the messages in the literature in terms of alignment messages, and other areas where alignment was defined either ambiguously or quite differently across articles.

NGSS dimensions
Our findings show some clarity and coherence around the message that the PEs should primarily be used for Page 13 of 18 Tanas et al. Disciplinary and Interdisciplinary Science Education Research (2023) 5:5 alignment of instructional materials, either in a bundle or one at a time. This was the most frequent method used to represent alignment (48% of alignment codes). This aligns with the research literature describing steps for bundling PEs to develop instructional units (Krajcik et al., 2014) and the NGSS itself, which frame the PEs as the "clear and specific targets for curriculum, instruction, and assessment. " (NGSS Lead States, 2013, p. xxii). Additionally, there is some coherence, or agreement, on the big ideas of NGSS reform in our analysis: engaging in science practices, integration, engineering and use of phenomena. Again, we see these themes appear elsewhere in the science education research literature, like the alignment framework by Lowell et al. (2021) which centers four features of the NGSS: phenomena based, threedimensional, student epistemic agency, and coherent; and EQUIP (Achieve, 2016) which focuses on explaining phenomena/designing solutions, three dimensionality, and integration.

Definitions of reforms
However, our findings also point to some inconsistency and ambiguity when it comes to both prioritizing what components of reform should be considered for alignment, and in defining what reform terminology means in terms of NGSS alignment and implementation. For example, in our analysis of the use of integration as a measure of alignment to the standards, we found considerable disagreement on both what was integrated, and how integration should occur. Integration was used to refer to two or three dimensions, but also several PEs, real-world integration, and science and engineering integration. Although the term integration was used frequently, how to integrate was left largely unexplored in this body of literature-and the few that did elaborate on integration presented differing processes (Cian et al., 2019;Houseal, 2015). Similarly, engineering and phenomena were used to purport alignment to the NGSS but were defined variably. How to incorporate engineering, and what that may look like was answered differently throughout this literature. Likewise, use of phenomena was frequently used to indicate NGSS alignment, but how to select a phenomenon and the purpose or function of the phenomenon as it relates to instruction were described differently throughout. Although these differences may seem trivial, a sensemaking perspective on reform implementation would argue that these ambiguities in definitions and processes may result in differing interpretations by teachers as they work to implement the standards in their classrooms. Additionally, if we recognize the key role of teachers in implementing these reforms, we must provide clear messaging around reform ideas to support teacher understanding.
Not surprisingly, there is disagreement in the research community regarding the primary definitions of NGSS reform terminology as well. For example, there has been ongoing debate as to how the term practice as used in the NGSS differs from previous conceptions of inquirybased science or hands-on science (Ford, 2015;Furtak & Penuel, 2019;Osborne, 2019), and continued discussion of how the CCCs should be used in three-dimensional learning (Fick & Arias, 2019;Nordine & Lee, 2021), among numerous other examples. Further, the discussion to identify the core components of NGSS reform is also still ongoing. Cherbow et al. (2020) outline four key shift in the NGSS: Phenomena-based, Three-dimensional, Student epistemic agency, and coherent. Others have highlighted key ideas like: Recognize learning progressions, Include engineering design, Address the nature of science, and Coordination with language arts and mathematics (Bybee, 2014). The research in this content analysis points to the need to continue these discussions within the research community to engage in defining and clarifying these complex reforms amongst us in order to present these ideas with clarity and coherence to teachers who, despite engaging with NGSS literature or development, may still find that the goals of NGSS "remain elusive" (Sherwood, 2020, p. 578).

Representations of reform and teacher sensemaking
Second, Spillane et al. (2002) emphasize that design of policy representation must consider the tendency of individuals to assimilate superficial aspects of reform, or those that may involve minimal changes to an individual's current thinking and teaching practice. We see this reflected in implementation studies (Haug, 1999;Hill, 2001;Spillane & Zeuli, 1999), and these findings align with cognitive research that shows individuals are more likely to both notice and assimilate ideas that fit with their current mental models or ideas (Smith et al., 1994). In terms of policy design the authors recommend that representations explicitly support teachers "looking beneath the surface" through juxtaposing ideas, thicker descriptions of behavior changes, and consideration of prior knowledge (Spillane et al., 2002, p. 417). With this lens, we examined our findings to see how current representations of NGSS alignment messages fit these recommendations.
One way these principles are visible in this body of NGSS policy representation is through explicit contrast of NGSS reforms with previous iterations of science education reform, or potential misinterpretation of reform terminology. For example, in defining what it means to engage in science practices, some authors emphasized what this type of engagement does not look like: repeating steps predetermined by the teacher, memorization of facts, or following scientific procedures (Curran et (2019) have taken a similar approach which they describe as "building a bridge to current reforms" (p. 173) with science practices, phenomena, and engineering by discussing their current framing in light of previous conceptions of reform. A sensemaking perspective would encourage translation of this approach in communicating with stakeholders at all levels of the educational system.

Balancing the general and the specific
Related, is Spillane and colleagues (2002) last recommendation that policy finds a balance between the general and specific when conveying reforms. An example of a general representation in this body of literature would be the use of an NGSS dimension (PE, SEP, DCI, CCC) for communicating alignment of a lesson, with no specific indication of how or why that dimension is related to the instructional material. For example, within the practitioner literature it is typical to include a table that lists the NGSS PE or dimension that the material is aligned to (see Kujawski 2014, p. 44). However, these tables do not link the NGSS dimension to any specific activity or component of the instructional materials. Some alignment tables indicate specific links to student activities (see Bubnick et al., 2016, p. 75), providing greater specificity and allowing the reader to connect the NGSS standard with how it is operationalized within the material. Further study of how these representations may afford or constrain teacher sensemaking around alignment to the NGSS is warranted. Additionally, the lack of specificity around key reform ideas like integration also leave teachers with ambiguity regarding how to operationalize reforms. While it is valuable for reform policies to be general enough to apply to multiple audiences, lack of specificity around key ideas can lead to misunderstanding or misinterpretation. Again, these same concepts are seen illustrated in current science education research as we work to develop rubrics, tables, and tools that illustrate alignment to the NGSS (Cherbow et al., 2020;Tekkumru-Kisa et al., 2015;Achieve 2014). Examining these tools in the context of sensemaking as they are understood and implemented by practitioners is a valuable research goal and requires thinking around appropriate design (representation), grain size (unit, lesson, task) and focus of alignment (teacher action, student action, instructional material).

Limitations
We identify two potential limitations of this study: the scope of the literature and the timeframe of the literature. This study examines one group of messages conveyed to practitioners regarding what it means to align to the Next Generation Science Standards. As such, the data and analysis give researchers insight into what messages are consistent with NGSS documents and related research literature, and what messages that may have been lost or changed in translation when directed at teachers. This literature focuses on one period of time and additional review of articles could reveal changes to these findings, particularly during Covid or in light of the concomitant increase in the use of virtual instruction. Additionally, further insight into this process of translation can be gained through study of other literature and artifacts developed with intent to translate the NGSS to teachers. This includes professional development materials at the national and state levels, messages within developed curricula, state assessments, and others. We hypothesize that, like our study, messages translated from original NGSS documents and research literature will change in emphasis and interpretation, but there may be differences by context and educational level. Integrating the findings from this study with further research can provide insight the pathways of NGSS reform implementation as they affect science education in the classroom. Integrating these findings with further study of how teachers' make sense of and operationalize ideas regarding NGSS reforms are logical next steps for research.

Conclusion
This content analysis provides a substantial summary of the current messages of alignment to the NGSS that practitioners may be interacting with. As such, it provides a window into how NGSS reforms have been represented to teachers who must grapple with enactment of ambitious and complex ideas like three-dimensionality, integration, and use of phenomena in instruction. These reforms require teachers to make substantial changes in the ways they think about designing instructional materials and teaching practice and, from a sensemaking perspective, will require significant learning. This work has implications for the research community engaged in operationalizing the Next Generation Science Standards through curriculum, tools, and development opportunities for teachers. The findings indicate that teachers receive multiple messages regarding what it means to align instructional materials to the NGSS in terms of both standards components (PE, DCI, SEP, CCC) and broader reform ideas (integration, engaging in science practices etc.). Additionally, NGSS reforms are ill-defined in practitioner literature with few concrete examples for teachers to engage with. When elaborated examples are provided, they lack consistency across the literature (for example in defining integration). As such, theoretical frameworks and previous research on SBR implementation indicate that NGSS reforms are likely to be differentially implemented as teachers make sense of and operationalize reforms. Further, without clear alignment messaging, teachers may have insufficient information to differentiate new reforms from previous conceptions of science teaching and learning, perhaps attending to superficial aspects of reform and missing deeper conceptual shifts intended by policy authors.
As such, we recommend continued discussion toward consensus among the research community regarding the core components of NGSS reforms. Additionally, each of these reforms require clear messaging and examples around definitions and how to practically operationalize reforms in both instructional materials like lessons and assessment. Consideration of how new ideas may differ from those teachers have learned about or enacted previously (including inquiry, student-centered, and others) should be addressed and contrasted for teachers. Development of materials and experiences to engage teachers in deep sensemaking around the ideas, with particular focus on differentiating new ideas from previous reforms is required.
This analysis provides examples and considerations for improvements in the way we represent these reforms to teachers with consideration to how they may afford or constrain sensemaking. Future research should consider improvements in the way we represent these complex reforms to teachers around the principles of clarity and coherence, understanding deeper versus superficial aspects, and striking a useful balance between the general and specific. Lastly, as this study provides an analysis of the representations of reform, further work should examine teachers' interpretation of these materials as they interact with them, including how teachers understand the language of the NGSS, how they conceive alignment to the standards, and what materials they develop themselves (lesson plans, assessments etc.). Given the central role of teachers in enactment of the standards, it is imperative that the research community not only focus on the development of instructional materials aligned to the standards, but additionally consider the teacher as developer of materials. In this model, teachers understanding, learning, and use of the standards must be studied and developed to facilitate successful reform implementation.