Alignment analysis of teaching–learning-assessment within the classroom: how teachers implement project-based learning under the curriculum standards

Abstract

This multi-case study examined the strengths and weaknesses of aligning teaching–learning-assessment of classroom project-based learning to curriculum standards and offered suggestions for teacher training and instructional improvement. The study constructed an alignment analysis framework for analyzing the cognitive dimension of classroom project-based learning and analyzed the situation of two junior high schools in Zhejiang Province using deductive and inductive content analysis. According to the results, the cognitive demands of classroom instruction activities and classroom assessments were much higher than those of teaching objectives and curriculum standards. Simultaneously, classroom instruction paid insufficient attention to engineering topics, and all instructional implementation elements exhibited content and cognitive deficiencies. The study suggests that teachers' dearth of engineering knowledge and the characteristics of project-based learning in the classroom are the primary reasons for the lack of alignment among three instructional implementation elements with curriculum standards. Similarly, it was discovered that classroom project-based learning has the characteristics of co-development of physical knowledge and engineering content and that future research can focus on developing more effective forms of classroom content organization and time distribution.

Introduction

Science education reform emphasizes alignment (Jin et al., 2019). After U.S. national normative examinations failed to assess students' knowledge and skills, Bloom focused on aligning academic assessment tasks with behavioral goals in the 1960s. Alignment was originally used to investigate the assessment and instructional goals. And later, it was included in educational reforms by the American Association for the Advancement of Science (1993) and the NGSS Lead States (2013). Webb (1999) defined alignment as the degree of conceptual, procedural, and methodological congruence among the elements of a curriculum system. It explains that alignment analysis can be conducted on all curriculum system elements. State Standards and State Assessment Systems: A Guide to Alignment (Columbia, USA) defines alignment as two or more items aligning. Alignment also means combining pieces into a suitable whole (Marca et al., 2000). All of these states clarified that two or more elements can be involved in congruent matching. Achieving alignment in the classroom is a critical aspect of standards-based curriculum reform. (Yang, 2020). The purpose of the curriculum is to enhance learning outcomes (Wiggins & McTighe, 2005), and alignment of teachers' teaching objectives with the instructional activities students engage in assists students in achieving their learning objectives (Cui & Lei, 2015; Duis et al., 2013). The alignment of teaching objectives with classroom assessments helps reveal important aspects of student’s learning process, understand students' learning, and promote the achievement of teaching objectives (Cui & Lei, 2015; Duncan & Hmelo-Silver, 2009). Aligning instructional activities with classroom assessments exposes teachers' instruction and helps them give students feedback that helps students meet instructional goals (Cui & Lei, 2015; Hall, 2002). When teachers' instructional goals match student activities and classroom assessments, teaching and learning are maximized and student learning is promoted (Biggs, 1996; English & Larson, 1996; Krajcik et al., 2007; Liu, 2009). In addition to guiding instructional implementation, curriculum standards also define the scope of academic-level examinations (PRC, 2022). Thus, aligning curriculum standards, instructional objectives, classroom teaching activities, and classroom assessments helps implement curriculum standards, improve teaching, and improve student achievement (Krajcik et al., 2007; Porter, 1997).

However, studies in some countries show that there are still significant discrepancies between the current stage of curriculum standards and actual curriculum implementation (Turan-Özpolat & Bay, 2017). Similar problems exist in the implementation of the curriculum in China. First, for a long time, the objectives of exams in China have been in a state of separation from the curriculum standards. As a result, for students to perform well in exams, teachers have gradually developed a teaching philosophy of teaching to the test with trivia as the goal (Lu & Jiang, 2022; Yang, 2017). Second, teachers' teaching philosophy focuses on what should be taught rather than what students should learn, resulting in students' participation in classroom instructional activities that are mainly memory and cognitive-type activities used for information transfer and lack of participation in various higher-order cognitive activities required by the curriculum standards (Lu & Jiang, 2022). Finally, in-classroom implementation, teachers focus only on the process and strategies of teaching, not on the content and methods of assessment, resulting in the absence of classroom assessments in the classroom, and even though some teachers focus on student assessment, there is a discrepancy between classroom assessments and classroom instructional activities in terms of the content and depth of knowledge required of students (Lu & Jiang, 2022; Wu & Gao, 2022). The inconsistency between classroom assessments and classroom instructional activities can further lead to teachers' inability to understand students' actual learning in a timely and therefore unable to adjust their teaching content to promote students' true understanding of teaching objectives and implement core literacy (Cui, 2013). In April 2022, the Ministry of Education of the People's Republic of China revised the compulsory education curriculum and released new curriculum standards that use core competencies as curriculum goals, "content requirements" as a means to achieve learning and knowledge goals, and "academic quality" as a standard for student achievement performance. Anderson, an educational researcher, suggests that to achieve curriculum goals, it is important to focus on learning, teaching, assessment, and adjustment. (Anderson et al., 2001). Thus, it is inevitable to make significant changes in teaching methods, learning approaches, and assessment techniques to attain the newly required core competencies. At the same time, it is important to ensure that those three elements are aligned with the new curriculum standards.

To understand the coherence of the elements of the curriculum system, various well-established paradigms for coherence analysis have been developed so far (Näsström & Henriksson, 2017). Examples include the SEC model, the Webb model, the Achieve model, and the revised Bloom Taxonomy of Educational Objectives. However, most of the existing research has focused on standardized tests, instructional materials, and alignment with curriculum standards (Cui & Lei, 2015; Yang, 2020). Although there are some instructional-related studies such as PISA, and TIMSS, they include questionnaires related to the instructional process to understand the alignment between curriculum standards, instructional process, and assessment. Most of these studies involve the relationship between instruction, curriculum standards, and summative assessments during a school period (Yang, 2020). Meanwhile, the release of the new version of the compulsory education curriculum program has pushed challenges to physics teaching. It is necessary to understand the implementation of teaching objectives, classroom instructional activities, and classroom assessments in junior high school physics classrooms under the core competency requirements, to identify inconsistencies in the teaching process, and to make targeted suggestions for teaching improvement.

Project-based learning

The twenty-first century is a time of opportunity and challenge, with economic, social, cultural, digital, demographic, environmental, and epidemiological forces shaping the lives of young people (OECD, 2019). In the face of challenges, traditional knowledge of memory no longer meets the demands of the new century on young people. Developing their scientific literacy has become a requirement for talent development in the new era. Project-based learning helps foster students' curiosity, improve their understanding of core scientific ideas, and empower them with problem-solving skills to become scientifically literate and responsible citizens (Zhao & Wang, 2022). As a result, this model of teaching and learning has received support from many governments, researchers, and teachers (Markula & Aksela, 2022; Novak & Krajcik, 2019). The Chinese Science Curriculum Standards for Compulsory Education (2022 Edition) states that teachers should be "guided to actively explore context-based, problem-oriented, deep-thinking, and highly participatory teaching models," which has led teachers in many parts of China, under the guidance of university experts and local researchers, to explore project-based learning in their classroom teaching (Luo et al., 2021).

The new version of the compulsory science curriculum standards sets core competencies as curriculum goals and divides the content into three parts: subject themes, engineering themes, and experimental investigations. The core competencies include scientific concepts, scientific thinking, inquiry practice, and attitude and responsibility. The scientific concept means that students need to form a general understanding of things based on understanding the core concepts of the subject and engineering technology concepts; scientific thinking refers to the way of understanding the essential properties, intrinsic laws, and interrelationships of objective things from a scientific perspective; inquiry practice refers to acquire scientific inquiry ability, technology, and engineering practice ability, and independent learning ability formed in the process of understanding and exploring nature, acquiring scientific knowledge, solving scientific problems, and technology and engineering practice, and attitude and responsibility is the scientific attitude and social responsibility gradually formed based on awareness of the nature and laws of science and understanding the relationship among science, technology, society, and the environment (PRC, 2022). Research has shown that these four areas are whole and have a catalytic effect on each other, together influencing students' understanding of concepts, their perception of phenomena, and their thoughts and approaches to problem-solving (Michaels et al., 2008; Yang, 2022). Scientific concepts and scientific thinking provide direction and operational guidance for inquiry practices, and inquiry practices and scientific thinking provide an environment for understanding and forming scientific concepts as well as attitude and responsibility (Krajcik & Czerniak, 2018). Project-based learning provides just the right conditions for the integration of the four dimensions of core competencies.

Project-based learning allows students to study and solve real-world problems in a project. Students learn and use scientific thinking through the process of asking questions, processing and analyzing data, interpreting and evaluating results, and communicating with others to improve project outcomes. This not only helps students to understand the meaning of concepts and their interconnections and form scientific concepts, but also helps them to discover the value of core concepts and their relationship to technology, engineering, and society, and to develop attitude and responsibility (Guo et al., 2020; Thomas, 2000). Empirical studies also further illustrate that project-based learning can be more effective than traditional science instruction methods in promoting the development of students' multidimensional competencies (Ayaz & Söylemez, 2015; Barak & Raz, 2000; Hasni et al., 2016). Research has shown that project-based learning can help students gain a deeper understanding of core concepts in the subject and improve student performance (Santyasa et al., 2020; Harris et al., 2015), promote the development of multiple cognitive skills such as problem-solving (Hasni et al., 2016), critical thinking, and creativity (Sasson et al., 2018), enhance emotions and attitudes, such as improving students' motivation (Holmes & Hwang, 2016), interest in learning (Bencze & Bowen, 2007), and attitude toward learning (Kanter & Konstantopoulos, 2010), and promote the development of social skills, such as improving students' interpersonal skills (Lee & Reigeluth, 2015). Project-based learning is effective for implementing the core competencies of the new version of the curriculum standards.

The depth of curriculum reform has led to the emergence of various teaching models, and besides project-based learning, problem-based learning has received a lot of attention. Most of the core features of problem-based learning and project-based learning are the same, both require solving contextualized problems to acquire subject knowledge and skills, both emphasize collaboration, the mediating role of the teacher, and active student participation and so on (Hasni et al., 2016; Steele, 2023). However, project-based learning emphasizes the generation of products by students (Hasni et al., 2016), students need to create engineering works and communicate with other members of the learning group about the findings. Engineering works are the product of combining science and engineering, and students can reflect on the relationship among science, technology, and society, understand the nature of science, and promote the formation of scientific attitudes and responsibilities in the process of creating products and communicating them to others. Compared to problem-based learning, project-based learning creates a learning environment for the development of the nature of science and scientific attitudes and responsibilities (Krajcik & Czerniak, 2018), which is in line with core competencies (PRC, 2022). This shows that project-based learning is the most functional and comprehensive teaching model nowadays to implement the new version of the curriculum standards.

However, research has shown that project-based learning can increase teachers' instructional difficulties. First, the problem context of project-based learning can lead to discrepancies between the choice of content of instructional activities and the "content requirements" of the curriculum standards, as well as between the depth of instructional activities and the "academic quality" of the curriculum standards (Krajcik et al., 2007; Xue, 2022). Secondly, the actual scaffolding provided by the teacher in discussions with students can also result in discrepancies between the instructional activities and the instructional objectives (Alozie et al., 2010; Hong et al., 2010; Xue, 2022). Finally, although teachers design formative assessments, they are often used at the end of the project, reducing the function of assessments in promoting learning (Xue, 2022). Therefore, it is necessary to understand the differences and connections between curriculum standards, teaching objectives, classroom instructional activities, and classroom assessments in the implementation of project-based learning by Chinese teachers at the time of the implement of new curriculum standards, to summarize the shortcomings and characteristics of teachers' project-based learning implementation, and to provide empirical evidence and suggestions for improvement in teacher training and teaching (Hasni et al., 2016; Krajcik et al., 2007).

The Beijing Normal University's "Core Competencies Oriented Project-based Learning Regional Reform Project" uses a three-stage interaction among subject experts, regional teaching researchers, and school teachers to guide the implementation of project-based learning in junior high schools in many provinces, cities, and regions of China. Schools participating in this project were selected for this study.

Physics subject competence

Subject competence is a hot topic of attention in the field of international and domestic basic education in recent years. Subject competence has been defined in large-scale assessments such as TIMSS and PISA, and documents such as the Next Generation Science Standards. However, most of them take mathematics, language and reading, and science as the major subject competence areas, and classify subject competence in terms of content attributes and process attributes such as core knowledge, competence activities, and cognitive level. At the same time, there is more focus on developing criteria to measure subject competency performance (Guo & Ma, 2012; NGSS Lead States, 2013; Wang, 2016; Xu, 2013). However, the existing research suffers from the separation of subject competence and knowledge experience, the lack of coherence between the competencies of each subject area, and the disconnect between the theory and performance assessment of subject competence and the teaching practice of subject competence (Wang, 2016). For this reason, the disciplinary team of Beijing Normal University (physics, chemistry, and biology) constructed a model of the composition of disciplinary competencies with common cognitive dimensions (learning and understanding, application and practice, and migration and innovation) based on theories of competencies such as generalized experience theory of ability and based on a systematic psychological analysis of the core cognitive and problem-solving activities of the subjects (Wang, 2016). The specific indicators of each dimension and their meaning were developed independently by each disciplinary team.

The Physics Subject Competence was developed by Professor Guo's team in the Department of Physics at Beijing Normal University. After synthesizing various national and international studies on scientific thinking competencies (Duan & Wu, 1988; Klahr, 2002; Lawson, 1985; Li, 2002; Yan et al., 1991; Zimmerman, 2007) and combing various national and international educational programmatic documents (NGSS Leading States, 2013; OECD, 2013; PRC, 2003), they develop a framework of competency indicators pointing to core competence. This framework is characterized by first integrating theoretical and performance assessments of subject competence, using behavioral performance to describe students' cognitive profiles. Research has shown that students' intrinsic psychological traits can be reasonably inferred using their external performance in solving physics problems (Guo et al., 2017). Second, the use of specific physics knowledge and physics problem situations to describe physics subject competence meaning reflects the uniqueness of physics subject competence (Guo et al., 2017). Finally, the physics subject competence is used to describe the core competencies of students. Core competencies are psychological traits that students internalize through learning physics, and this trait is expressed in problem-solving behaviors. Physics subject competence integrates various national and international core competencies measured about physics subjects, and therefore, it provides a practical pathway to describe the behavioral performances of students' core competencies in China (Guo et al., 2017). Synthesizing the above analysis, to achieve a localized study of curriculum alignment, this study selected physics subject competence as the level of the cognitive dimension.

Physics subject competence is constructed from three main activities (cognitive dimensions) at the basic education stage: learning and understanding, application and practice, and migration and innovation, and nine specific cognitive levels are used to describe students' cognitive competence indicators. Among them, "learning and understanding" point to the internalization and absorption of students' physics knowledge, "application and practice" point to the routine use of physics knowledge, and "migration and innovation" point to the transfer and use of physics knowledge and methods as well as innovation and creativity (Guo et al., 2016). The specific indicators and meanings are described as shown in Table 1.

Table 1 Physics subject competence (Guo et al., 2017)

However, physics subject competence meaning describes the cognitive processes of students rather than the instructional processes of teachers, which limits the function of physics subject competence in guiding teachers in practice (Wang, 2016), especially for interdisciplinary project-based learning instructional practices with engineering characteristics (Krajcik & Shin, 2014). To examine classroom teaching of project-based learning in deeper detail and to expand the value of physics subject competence in practice. This study further developed the characterization of cognitive indicators of physics subject competence regarding teaching behavioral performance based on the teaching objectives of the case schools and the characteristics of project-based learning instructional activities.

Aims of the study

This study developed an analytical framework for understanding the implementation of physics subject competence in project-based learning classroom instruction and expanded the research methods and elements of alignment in classroom research, which to understand the characteristics and disadvantages of alignment of the three elements of Teaching–Learning-Assessment with the curriculum standards at this stage of project-based learning classroom instruction.

The detailed research questions are as follows:

• RQ-1:What are the characteristics of teachers’ implementation of classroom teaching–learning-assessment in alignment with curriculum standards under project-based learning?

• RQ-2: How do teachers implement classroom teaching–learning-assessment in alignment with curriculum standards in practice?

Methods

This study used a multi-case study approach (Yin, 2014) to analyze the instructional situation in schools with Project-based learning. Multi-case studies allow for comparing similarities and differences between cases to understand the similarities and differences in teaching and learning across schools(Yin, 2014). The data sources used in the case studies were project-based learning instructional designs, instructional videos, classroom PowerPoints, and students' task sheets. The instructional videos and instructional designs were the primary materials analyzed. At the same time, the classroom PowerPoints and students' task sheets were used to supplement and validate the information that was not adequately provided in the instructional videos and instructional designs. These study materials came from two Zhejiang Province schools participating in the project-based learning Instructional Improvement Project at Beijing Normal University.

Core competencies oriented project-based learning regional reform project

The Core Competencies Oriented Project-based Learning Regional Reform Project, initiated by Beijing Normal University, provided three pieces of training for participating schools to enhance their ability to implement project-based learning in the classroom. At the beginning of the semester, the participating schools first identified the grade levels, knowledge topics, and project content. The first version of the project-based learning unit instructional design was developed through in-school discussions. Subject experts from Beijing Normal University reviewed the submitted designs, made modifications, and provided on-site training on the designs. Subsequently, teachers from participating schools revised the design, developed a list of tasks and teaching materials for student learning, conducted instructional activities, and recorded videos of a class participating in the complete project-based learning process. Finally, through personal reflection and expert teaching training, teachers developed pedagogical reflections and questions, which they discussed with experts during the third training to promote the team's overall understanding of project-based learning and to improve project-based learning design and implementation skills. In general, schools provide instructional videos along with corresponding instructional designs, student learning task sheets, and instructional PowerPoints. Also, to understand the usefulness and effectiveness of project-based learning for students and teachers, the subject improvement project team collects information on student pre and post-test learning in the improvement classes and interviews with teachers and students about project-based learning. Because the purpose of this study was to understand the alignment of teachers’ designed instructional objectives, classroom instruction activities, and classroom assessments with the curriculum standards, student-specific information was not used for the study. At the same time, since the information from teacher interviews focused on understanding the cognitive and non-cognitive effects of expert guidance on teachers' professional development, teacher interviews were also not used as research materials. Finally, instructional videos, instructional designs, student learning task sheets, and instructional PowerPoints were selected as this study’s materials.

The characteristics of project-based learning as a good way to implement core competencies at this stage have been understood differently by different research teams (Condliffe et al., 2017; Markula & Aksela, 2022). However, in general, project-based learning shares some common characteristics: (1) driving questions, (2) learning goals, (3) scientific practices, (4) collaboration, (5) use of technological tools, and (6) creation of artifacts. (Krajcik & Shin, 2014; Markula & Aksela, 2022). The project-based learning improvement team at Beijing Normal University developed a classroom project-based learning instructional model based on the recognition of these common characteristics for classroom instruction in various courses (physics, chemistry, and biology). Teachers implement each project in the classroom through three types of lessons, the introductory lesson, the process lesson, and the presentation lesson. In the introductory lesson, the teacher arouses students' interest in learning by creating a context close to their lives, which leads to driving questions and learning objectives by the teacher or by the teacher and students to help students understand the value and context of the project. In the process class, the teacher designs a series of subtasks that allow students to carry out hands-on science activities in collaboration with others to solve problems. The subtasks that students engage in are disassembled inquiry problems that involve the development of many physics subject competence indicators. Also, in the problem-solving process, students and teachers can use a variety of information technology tools to facilitate problem-solving. In the presentation class, groups of students present their artifacts and explain the principles, process, and reflections of the artifacts through PowerPoint (Zhao & Wang, 2022). It is important to note that this study uses a school-based physics curriculum.

Participants

The project-based learning materials cover March–July 2022. The materials were submitted after one semester of project-based learning and expert guidance. Thus, teachers in the schools concerned understand the process of project-based learning and have some experience in its implementation, and they can follow the same model for instructional design and classroom activities, which can fully reflect the actual situation of teachers' implementation and allow for easy analysis and comparison.

Eight Wenzhou City, Zhejiang Province, schools participated in project-based learning. Two schools offered entire project-based learning materials (instructional videos, task sheets, instructional designs, and classroom PowerPoints). The case study chose these two schools. The new compulsory science curriculum standards in Zhejiang Province were employed for this alignment analysis.

The new version of the science curriculum standards requires students to "understand interdisciplinary concepts and apply them in authentic contexts" (PRC, 2022), whereas the scientific practices and production artifacts that define project-based learning foreshadow its interdisciplinary nature (Krajcik & Czerniak, 2018). Thus, this study proposes that project-based learning implementation can only be reflected by interdisciplinary concepts. The case study assignment, "Making school logo projection light," focused on "light phenomena" in physics and "engineering." Thus, these two interdisciplinary concepts were case study content analysis themes.

The students involved were all seventh graders, one class from each school (Table 2). It is essential to note that although the two schools conducted the same project, the subject content and number of hours of the project differed due to differences in project design. Specifically, the school N project lacked the content of straight line propagation of light, the law of light reflection, plane mirror imaging, and light refraction.

Table 2 Schools’ information

Alignment

Since Webb's first model of curriculum alignment analysis, various alignment analysis tools have emerged as independent analytical frameworks (Rothman, 2003). The Surveys of enacted curriculum (SEC), a tool developed by Porter (2002) to examine the implementation of the U.S. curriculum, was designed to understand the level of alignment between what teachers teach, the practical activities students experience, and the assessments. According to Polikoff's findings, the SEC analysis method's content and cognitive dimensions can be modified (Polikoff et al., 2019). To better localize the study, the physics subject competence was used in the cognitive levels (Guo et al., 2017), which aligns with the requirements of physics subject core competencies in China, with nine cognitive levels. In the content areas, the analysis content is mainly classroom teaching videos, the content topics involved are relatively few, and they all follow the requirements of the Compulsory Science Curriculum Standards. So the content areas are designed according to the curriculum standards "content requirements," generating nine content topics for analysis. It is important to note that since project-based learning integrates knowledge learning into producing engineering products (Krajcik & Shin, 2014), the content includes engineering and light phenomena topics. The two dimensions form the vertical and horizontal axes of the matrix, respectively, to form the framework for alignment analysis (Table 3). The calculation was performed using the SEC alignment indices, $P=1-\frac{\sum \left|X-Y\right|}{2}$, where X denotes the proportion of standard cells in one matrix and Y denotes the proportion of standard cells in the other matrix.

Table 3 Alignment analysis framework

In addition, due to the complexity of the factors influencing classroom instruction, besides the curriculum standards, other factors, such as the student's instruction environments, must be considered. Therefore, the definition of the degree of alignment indices by John Smithson et al., director of the SEC program, was used, with 0.5 as the optimal value to reflect the alignment between the curriculum standards and the elements of classroom instruction (Zhen, 2017).

Content analysis

The purpose of deductive content analysis is to investigate existing models or theories (Hsieh & Shannon, 2005). Two dimensions under the SEC analysis paradigm were used as the basis for the content analysis. After deductive analysis, the cognitive dimension was ultimately used in the categories of teaching behavior performance shown in Table 4. It was shown that students' performance in physics subject activities can be used to represent students' physics subject competence (Guo et al., 2016), while in instructional implementation, students' activity performance is derived from teachers' instructional instructions and activities, which means teachers' performance of instructional behaviors. From this, it can be seen that students' learning behavior performance can be inferred from teacher teaching behavior performance, which points to the corresponding physics subject competence indicators. All the teaching behavioral performances used in this study were derived from the analysis of the teaching objectives in the instructional design and classroom videos of the case schools. For the teaching objectives, the study derived the corresponding student activities and physics subject competence based on the teacher's instruction, which constituted a coding list. For the classroom videos, the classroom videos were firstly decoded according to the teacher's instruction and students' corresponding learning activities, and then the corresponding physics subject competence was determined by analyzing the corresponding students' behavioral performance to form a coding table. Finally, the teaching objectives and classroom videos of teachers with the same physics subject competence were summarized in terms of teaching instructions and activities characterizing the teaching behavior performance. In this study, curriculum standards, teaching objectives, classroom activities, and classroom assessments were coded and placed in separate SEC coding sheets for the cognitive and content dimensions, and the overall coding procedure was conducted independently by two graduate students. According to Porter's study, the analysts for the analysis of the comparison should be 2–6 (Porter, 2002). The Cohens Kappa value for both coders was 0.781 (p = 0.000). Discrepancies were discussed and negotiated to reach a consensus.

Table 4 Coding framework of cognition dimension for content analysis

The study analyzed the "content requirements" and " achievement requirements" of the new curriculum standards. The "content requirements" are the third-level themes of the learning content, which provide the learning content and the learning level. The " achievement requirements" are the specificity of the achievement quality in the first-level themes, which reflect the achievement of the students after completing the first-level themes (Li et at., 2022), namely, the level of thinking, competence, and emotional and attitudinal performance expectations of students in terms of knowledge. Content requirements point to the process of learning knowledge, while achievement requirements point to the performance of applying knowledge. Each selected item was split according to the form of an action verb + a noun or phrase, coded separately and counted as 1, and placed in the cell corresponding to the alignment analysis framework. This method was also used for the analysis of teaching objectives. In addition, the non-cognitive objectives, such as "care about the typical cases of light technology changing production and life, and pay attention to light pollution" (Ministry of Education of the People's Republic of China, 2022), were deleted as emotional-attitudinal objectives. According to the new curriculum standards definition, the engineering theme's content is interdisciplinary and practical. Its purpose is to develop students' ability to apply knowledge interdisciplinarily and their comprehensive ability to analyze and solve problems (Ministry of Education of the People's Republic of China, 2022). Therefore, the cognitive levels of the curriculum standards for engineering topics are put into the dimension of "migration and innovation".

Due to the continuity of the instructional videos, it was impossible to divide the coded items directly. The study referenced Porter's (2002) investigation of classroom instruction by calculating the proportion of instructional time allocated to each content area and cognitive level. Each instructional video's teacher's instructions and students' learning activities were segmented according to the Flanders Interaction Analysis Categories (FIAC) using the 30 s as the fundamental unit. Teacher and student behaviors and language were integrated to form the final instructional activity codes and incorporated into the corresponding alignment analysis framework based on the principles of a single content area and cognitive level. It is crucial to notice that certain instructional activities incorporate multiple content areas or cognitive levels. For instance, the activity "The teacher instructs students to observe experimental phenomena and has them draw various light propagation phenomena in the experiment" incorporates three fundamental ideas: straight-line propagation of light, the law of light reflection, and plane mirror imaging. The total amount of time is divided evenly between each cell. For classroom assessments, because effective assessment requires assessment criteria, this study only considers classroom activities with assessment criteria in the instructional design as classroom assessment activities and only considers assessment indicators that can be found in the content and cognitive dimensions of the SEC analysis framework for the analysis. And in the same way as for multi-content or cognitive instructional activities, classroom assessments use the total time averaged into the corresponding cells. Notably, to compare data between the four elements of curriculum standards, teaching objectives, instructional activities, and classroom assessments, the SEC data obtained under each element were normalized to obtain the percentage of each cell in that element, which means the degree of importance.

Analysis of classroom teaching behaviors

The Flanders interaction analysis category (FIAC) is a widely used method for analyzing interactions in the classroom (Zhang, 2014). Researchers identify, categorize, and record classroom speech acts by recording a code that best describes teacher and student behavior every 3 s to enable observation and study of the classroom (Zhang, 2014). However, FIAC's coding interval is too short and difficult to implement. Since the primary purpose of this study was to split classroom activities through the analysis of instructional videos, the 30 s was chosen as the minimum essential time interval for the content analysis of teacher and student behaviors and language.

Results

Comparison of alignment

Based on the calculation method of Porter's alignment indices described in the previous sections, the P-values between the curriculum standards, teaching objectives, teaching instruction activities, and classroom assessments were calculated for M and N Schools, respectively (Table 5). Only the alignment indices P for teaching instruction activities and classroom assessments in M and N Schools were larger than 0.5, suggesting a high level of alignment. In contrast, the alignment indices of the remaining analysis topics were all less than 0.5, indicating a lack of alignment.

Table 5 Statistic of alignment indices

Figure 1 shows relationship topographs of curriculum standards, teaching objectives, classroom instruction activities, and teaching assessments for N and M Schools. The red color represents School M and the blue color represents School N. The different thicknesses of the lines represent different ranges of alignment index values, and the thicker the line is, the higher the alignment index P is. In both schools, there is a lack of alignment with the curriculum standards, especially between the curriculum standards and classroom assessments. As for the elements of instructional implementation, the alignment between classroom assessments and instructional activities was achieved in both schools, but the alignment between teaching objectives and classroom assessments and between teaching objectives and instructional activities was lacking, especially the alignment between teaching objectives and classroom assessments.

Fig. 1

Relationship diagram of curriculum standards, teaching objectives, classroom instruction activities, and teaching assessments for M and N Schools

Topographs analysis

Figures 2 and 3 show the corresponding topographs of curriculum standards, teaching objectives, teaching instruction activities, and classroom assessments for M and N Schools. The horizontal axis of the topographs indicates the core concepts of the content areas, and the vertical axis indicates the nine cognitive levels. The colors at the intersections of the horizontal and vertical axes indicate the different ratio distributions. The different ratio levels are indicated by dark blue (0.00–0.03), orange (0.03–0.06), gray (0.06–0.09), yellow (0.09–0.12), light blue (0.12–0.15), and green (0.15–0.18), respectively.

Fig. 2

Topographs of curriculum standards, teaching objectives, classroom instruction activities, and teaching assessments for M School

Fig. 3

Topographs of curriculum standards, teaching objectives, classroom instruction activities, and teaching assessments for N School

According to the topographs of the teaching objectives and curriculum standards in M School, both emphasize the I6-A1 intersection. And although both emphasize the engineering topics on the C cognitive dimension, the teaching objectives only emphasize E3-C1 and E3-C2 intersections, which is inconsistent with the curriculum standards' emphasis on the E3-C3 intersection and ignores the E1 and E2 contents. The teaching objectives at N School differ marginally from those at M School in that it emphasizes the I6-A2 intersection. In both schools, the differences between the teaching objectives and curriculum standards are primarily due to the reality that the teaching objectives do not accurately reflect engineering themes.

According to the topographs of instructional activities and curriculum standards in M School, the instructional activities emphasize the I6-A2 intersection and do not emphasize the engineering thematic content. The situation in N School is slightly different from M School, mainly in its emphasis on the E2-C1 and E3-C3 intersections, which meet some of the requirements of the curriculum standards. In general, the differences between the instructional activities of the two schools and the curriculum standards mainly lie in the fact that the cognitive requirements of the physics curriculum content are higher than the curriculum standards and the lack of attention to the engineering topics in the instruction.

According to the topographs of classroom assessments and curriculum standards in M School, the classroom assessments emphasize I6-A2 and I6-C3 intersections and the engineering theme emphasizes E1-C2 and E3-C3 intersections, which are both requirements of the curriculum standards. The situation in N School is slightly different from M School, mainly because it emphasizes only I6-C3 and E3-C3 intersections. Overall, the classroom assessments in both schools have higher cognitive requirements for physical content than the curriculum standards, and the engineering themes both emphasize the E3-C3 intersection, which falls within the requirements of the curriculum standards.

According to the topographs of teaching objectives and instructional activities in M School, the instructional activities emphasize the I6-A2 intersection and the teaching objectives emphasize the I6-A1 intersection, while N School differs slightly from M School in that its instructional activities emphasize I5-A2, I5-A3, and I5-C3 intersections and the teaching objectives emphasize I5-A1, I5-A2, and I5-B1 intersections. Overall, the cognitive demands of the instructional activities in both schools are slightly higher than the teaching objectives they set.

According to the topographs of teaching objectives and classroom assessments in M School, the classroom assessments emphasize I6-A2, I6-C3, and E3-C3 intersections, which are above the requirements of the teaching objectives. The situation in N School is similar to that of M School. Overall, the cognitive requirements of the classroom assessments in both schools are higher than the teaching objectives they set.

According to the topographs of instructional activities and classroom assessments in the two schools, there is a high degree of consistency in the focus of instructional activities and classroom assessments, but classroom assessments focus more on engineering topics and their corresponding C cognitive dimension.

Histogram analysis

Figures 4 and 5 display the histograms of curriculum standards, teaching objectives, instructional activities, and classroom assessments for M and N schools in the content dimension and cognitive dimension, with four colors representing each of the four elements. The height of the histogram is proportional to the horizontal axis's weight.

Fig. 4

Histogram of the content dimension of curriculum standards, teaching objectives, classroom instruction activities, and classroom assessments for M and N Schools

Fig. 5

Histogram of the cognition dimension of curriculum standards, teaching objectives, classroom instruction activities, and classroom assessments for M and N Schools

In the content dimension, both schools emphasize I5, I6, and E3, which include both physical and engineering knowledge. Compared to the curriculum standards, N School emphasizes I5 and I6 content significantly more. In addition, both schools lack E1 and E2 related teaching objectives, classroom assessments, and instructional activities.

In the cognitive dimension, the two schools share a lack of or insufficient emphasis on teaching objectives in the C3 indicator and the same issue in instructional activities and classroom assessments in the C1 indicator, as compared to the curriculum standards.

In particular, teaching objectives at M School are centered on A2 indicators, while teaching activities and classroom evaluations are centered on A2, B1, and C2, respectively. The cognitive demands of teaching objectives, instructional activities, and classroom assessments increase sequentially, and N School demonstrates the same trend.

Figure 4 reveals an intriguing phenomenon: according to the analysis, the project from N School contains less physical knowledge than M School. Figure 5 demonstrates, however, that both schools place the most emphasis on teaching objectives, instructional activities, and classroom assessments for the A2 indicator, followed by the C2 and C3 indicators. The curriculum standards do not affect this pattern, regardless of the weight they assigned to these cognitive indicators. Thus, it is evident that the information emphasizes on the cognitive and content dimensions of the elements of project-based learning instructional implementation is distinct and stable.

Implementation of alignment in the projects

Since the content dimension of the two schools meet the requirements of the curriculum standards, the cognitive dimension primarily reflects the alignment situation of the instructional implementation elements. The cognitive dimensional emphasis is ultimately obtained through normalization by integrating the total data on the cognitive dimension of each element of instructional implementation in the two case schools, as depicted in Table 6.

Table 6 Teaching behavioral performance ratio of case Schools of curriculum standards, teaching objectives, classroom instruction activities, and classroom assessments

The teaching objectives for A2 and A1 emphasize teaching behavioral performances to help students to understand physical concepts through observation, deduction, and summarization, as well as to help students to remember and describe experimental phenomena. The instructional activities and assessments focus on A2 as well as to help students to use multiple knowledge to create complex products, which is emphasized in C3.

In addition, the B1 indicators required by the curriculum standards mainly include four experiment-related teaching behaviors, which are basically reflected in all elements of teaching implementation, but the actual attention is relatively low. For the C cognitive dimension required by the curriculum standards, there are primarily 12 engineering-related teaching behaviors, among which there is a severe lack of teaching objectives.

Discussion

The primary purpose of this study is to understand the alignment between curriculum standards, teaching objectives, classroom instruction activities, and classroom assessments, as well as the characteristics and disadvantages of implementation for Project-based Learning. These aims will be discussed concerning the two questions of the study.

Comparison of alignment implemented by the teachers

The study reveals that the various instructional implementation elements of project-based learning in the two case schools cover the physical concepts and their cognitive dimension required by the curriculum standards and emphasized the development of higher-order cognitive indicators, but the level of attention to engineering topics is insufficient, and the cognitive and content dimensions are absent from both the teaching objectives and classroom assessments. The Compulsory Science Curriculum Standards (2022 Edition) were published in March 2022, and the project-based learning case that this study used was developed in May of the same year. As a result, teachers have limited time to learn about the engineering topics added to the new curriculum standards, which results in a lack of emphasis on teachers' teaching. Simultaneous, the implementing teachers, in this case, have only implemented one semester of project-based learning before carrying out the project, and the teachers are unable to fully integrate engineering into classroom instruction, confirming Mentzer et al.'s (2017) conclusion that teachers need three years to truly master the project-based learning approach to instruction.

In light of the preceding analysis, it is essential to improve teachers' knowledge of engineering-themed curriculum standards and the implementation of engineering-themed project-based learning. In teacher training, for instance, teachers can be permitted to participate in the project-based learning engineering production process to help them comprehend the relationship between engineering and physical knowledge as well as the steps and processes of engineering practice in practice (Hasni et al., 2016).

In the case Schools, the cognitive demands of the instructional activities exceed the teaching objectives and curriculum standards. By analyzing the findings of previous studies, this study concludes that, in addition to the teacher's design abilities, this is due to two other factors. On one hand, it is associated with the characteristics of project-based learning (Krajcik et al., 2007; Xue, 2022). Project-based learning develops students' various higher-order thinking skills by creating instructional activities with higher-order cognitive requirements (Hasni et al., 2016; Sasson et al., 2018), thus indicating that instructional activities under project-based learning emphasize higher-order cognition more. On the other hand, it relates to students' genuine cognitive situation (Alozie et al., 2010; Hong et al., 2010; Xue, 2022). In this study, classroom videos are analyzed to determine the instructional activities, with each teacher's problem scaffolding considered a separate activity. Therefore, the results of the analysis reflect the actual teacher-student interaction, which means the cognitive understanding process of the students (Doyle, 1983).

The alignment between instructional activities and classroom assessments is greater than that of other elements. This is directly related to the characteristic of classroom project-based learning that combines assessment activities into core instructional activities. The specific analysis reveals that the cognitive demands of classroom assessments are slightly higher than those of instructional activities, indicating that the C cognitive dimension is emphasized. This is closely related to the project-based learning characteristics of presenting and communicating engineering products (Krajcik & Shin, 2014; Markula & Akela, 2022). However, the cognitive demands of classroom assessments that exceed the instructional activities and teaching objectives can also lead to difficulties in project-based learning for students of intermediate and below ability, ultimately diminishing their desire to learn science (Kennedy et al., 2007; Sewagegn, 2020). Therefore, on the one hand, teachers should provide some scaffolding to help these students participate in project-based learning effectively, such as by adding extracurricular club activities to assist students with product creation or by summarizing the problems students encounter when creating products and providing uniform classroom responses. On the other hand, teachers can use backward design to improve the effectiveness of instruction by designing classroom assessments first according to the teaching objectives and then designing instructional activities to increase the alignment among teaching objectives, instructional activities, and classroom assessments (Wiggins & McTighe, 2005; Hargreaves, 2005).

The analysis of histograms reveals that project-based learning of different knowledge capacities has similar cognitive dimensional characteristics, which reflects the unique characteristics of classroom project-based learning, namely the ability to balance the cognitive requirements of disciplinary instruction and the cognitive requirements of engineering topics of project-based learning. From the perspective of pedagogical practice, it has been demonstrated that project-based learning can be developed in the classroom (Tal et al., 2006) and that it can ensure that the core instructional requirements of both disciplinary instruction and project-based learning are developed within similar periods. While previous study finds that project-based learning should be developed in the classroom (Thomas, 2000), scheduling and organizational challenges in the classroom (Aksela & Haatainen, 2019; Viro et al., 2020) lead teachers to conduct project-based learning aimed at developing students' soft skills outside of the regular courses. This study presents the temporal organization characteristics of project-based learning on engineering topics in the classroom (Markula & Aksela, 2022), demonstrating not only the feasibility of project-based learning in the classroom to develop students' core content and cognition from the perspective of data but also presents the temporal organization characteristics of project-based learning on engineering topics in the classroom. In addition, the histogram analysis reveals that the elements of project-based learning practices that carry less physics knowledge do not reduce the proportion of time spent on the cognitive dimension of learning understanding, a feature that assists students in comprehending core physics concepts, but rather lengthen classroom instruction. Similarly, if a project contains a greater number of core physics concepts, it can integrate knowledge and enhance classroom utilization, but it can also increase the cognitive load for students and decrease their learning efficiency (Sweller et al., 2019). When determining the content carrying capacity of a project, it is necessary to consider not only the logical organization of the content but also the cognitive complexity and difficulty of the content for the students.

The preceding analysis shows that characteristics of project-based learning in the classroom are related to the alignment of instructional implementation situations. Given that the characteristics of classroom project-based learning are primarily reflected through teachers' activities, a specific analysis of the relationship between the performance characteristics of teachers' teaching behaviors and cognitive indicators of classroom project-based learning can be used not only to understand the current situation of teachers' implementation of project-based learning but also to provide recommendations for consistent instructional improvement in teaching. Consequently, this study investigates the behavioral teaching performance of the case institutions at various cognitive levels for the second problem.

Teachers’ implementation of alignment

The teaching objectives of project-based learning for teachers appear to necessitate an emphasis on the content design of engineering topics. Nowadays, principal teaching objectives include memorization and generalization of physical concepts and laws. Understanding the value of engineering, designing engineering solutions, manufacturing products (using single knowledge), evaluating products, and enhancing engineering solutions are the main aspects of engineering topics. Attention should be devoted to the design of the integration of engineering content and physical knowledge when developing educational objectives. Making engineering products is one of the characteristics of project-based learning (Markula & Aksela, 2022), and ensuring the design of teaching objectives for engineering content promotes alignment among the instructional implementation elements of project-based learning.

According to the previous analysis, classroom project-based learning exhibits both learning physical knowledge and project-based learning traits. Moreover, the specific analysis of the teaching behavioral performance reveals that it emphasizes the process of students' generalization of physical concepts and laws and the application of multiple knowledge to produce complex products, indicating that classroom project-based learning has two core characteristics: engineering product creation and the generalization of physical knowledge.

Experimentation, a crucial physics research method, is also a crucial requirement of the curriculum standards. Although the case schools include experimental content in three elements of curriculum implementation, experiments are not emphasized in three elements. According to the curriculum standards, physics experiments are viewed as an important means of developing students' core competencies (PRC, 2022), and the emphasis on experiments is reflected primarily in the ability to design simple experimental procedures, conduct experimental operations, and record data. Therefore, project-based learning should emphasize providing physics experiment learning opportunities and incorporating more physics experiment activities in project-based learning.

Limitations

Content analysis can only focus on what is visible in the materials (Cohen et al., 2007). The study only includes what is happening in the two schools' classrooms. The materials could not represent the teacher's instruction and interaction with students outside of the classroom. However, classroom instruction is the primary vehicle for project-based learning, so it can be assumed that the teachers' primary instructional activities are completed in class, and the materials provided emphasize important instructional content.

Because submitting the Project-based Learning materials is voluntary, the sample of schools and teachers included in the study is likely to represent only schools that actively participated in and implemented Project-based Learning. Therefore, the results are not necessarily representative of the general implementation of Project-based Learning, which limits the generalizability of the results.

In addition, the materials used in the study only include teachers' Project-based Learning instructional designs, instructional videos, classroom Power points, and students' task sheets. The data could only be analyzed at an objective level, and future studies could add teacher interview sessions to obtain a more in-depth analysis of the results.

Conclusion

This study examines the implementation of project-based learning aligned with curriculum standards for teaching objectives, instructional activities, and classroom assessments. Due to the short time that project-based learning has been promoted in regular classes in China and the new version of curriculum standards that have been proposed, no systematic analyses of the alignment of project-based learning in Chinese classrooms have been conducted. This study devised a framework for analyzing the cognitive dimension of classroom project-based learning to analyze cognitive indicators of teaching behavioral performance to analyze classroom alignment. It correlates the theory of physics subject competence with project-based learning instructional practices, which is relatively uncommon in existing subject competence research (Wang, 2016), and other researchers or teachers can use this framework to analyze classroom project-based learning alignment on the cognitive dimension.

This study provides empirical evidence to enhance project-based learning instruction and teacher training. This is because it describes the similarities and distinctions between the implementation of project-based learning and the new version of the curriculum standards, as well as the characteristics of classroom project-based learning implementation. The authors concluded, based on the findings and discussion, that three areas should be emphasized. The first is to improve teachers' comprehension of the new version of the curriculum standards, particularly the content about interdisciplinary practices (engineering). This will allow them to acquire a deeper understanding of the engineering implementation process, the value of each step, and the relationship between engineering and physical knowledge to provide a more integrated solution for project-based learning in the classroom. Second, we investigate the traits and advantages of classroom project-based learning, the structure of classroom project-based learning required by the new version of curriculum standards, and the content and time distribution of physics knowledge, engineering content, and classroom experiments. According to research (Markula & Aksela, 2022), project-based learning is still a challenging instructional method that requires further development. Third, project-based learning test questions should be developed under the requirements of the new version of the curriculum standards to standardize classroom instruction and clarify the key content and cognitive level of project-based learning, particularly in the field of engineering.

In addition, this study provides new insights into research on curriculum alignment. From the perspective of flexible implementation, alignment of teaching objectives, instructional activities, and classroom assessments with curriculum standards does not necessarily imply strict alignment. Misalignment is not necessarily negative if it involves and emphasizes higher-order cognitive skills (Liu et al., 2009). Rather, it facilitates the development of core competencies among students. According to curriculum level theory, teaching objectives are part of the perceived curriculum, which is developed by teachers based on their understanding of curriculum standards, whereas instructional activities and classroom assessments are part of the operational curriculum, which is based on the actual teaching and learning process and student performance (Goodlad et al., 1979). In project-based learning, knowledge is constructed through teacher and student participation, interaction, and discussion of artifacts (Sawyer, 2006). This implies that the actual teaching process of the teacher adapts and generates new instructional activities based on the student's situation, resulting in higher cognitive demands of instructional activities and classroom assessments than teaching objectives when students are more capable. It is evident that the implementation of a curriculum under project-based learning is oriented towards enactment(Jackson & American Association for the Advancement of Science, 1992). This study contends that teaching objectives can be viewed as the minimum standards that should be met in the classroom and that the enactment orientation of curriculum implementation can be used to conduct research on curriculum alignment in an era of core competencies.